A Framework for Software Diversification with ISA Heterogeneity


Xiaoguang Wang, SengMing Yeoh, and Robert Lyerly, Virginia Tech; Pierre Olivier, The University of Manchester; Sang-Hoon Kim, Ajou University; Binoy Ravindran, Virginia Tech


Software diversification is one of the most effective ways to defeat memory corruption based attacks. Traditional software diversification such as code randomization techniques diversifies program memory layout and makes it difficult for attackers to pinpoint the precise location of a target vulnerability. Some recent work in the architecture community uses diverse ISA configurations to defeat code injection or code reuse attacks, showing that dynamically switching the ISA on which a program executes is a promising direction for future security systems. However, most of these work either remain in a simulation stage or require extra efforts to write the program.

In this paper, we propose HeterSec, a framework to secure applications utilizing a heterogeneous ISA setup composed of real-world machines. HeterSec runs on top of commodity x86_64 and ARM64 machines and gives the process the illusion that it runs on a multi-ISA chip multiprocessor (CMP) machine. With HeterSec, a process can dynamically select its underlying ISA environment. Therefore, a protected process would be capable of hiding the instruction set on which it executed or detecting abnormal program behavior by comparing execution results step-by-step from multiple ISA-diversified instances. To demonstrate the effectiveness of such a software framework, we implemented HeterSec on Linux and showcased its deployability by running it on a pair of x86_64 and ARM64 servers, connected over InfiniBand. We then conducted two case studies with HeterSec. In the first case, we implemented a multi-ISA moving target defense (MTD) system, which introduces uncertainty at the instruction set level. In the second case, we implemented a multi-ISA-based multi-version execution (MVX) system. The evaluation results show that HeterSec brings security benefits through ISA diversification with a reasonable performance overhead.

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

@inproceedings {259693,
author = {Xiaoguang Wang and SengMing Yeoh and Robert Lyerly and Pierre Olivier and Sang-Hoon Kim and Binoy Ravindran},
title = {A Framework for Software Diversification with {ISA} Heterogeneity},
booktitle = {23rd International Symposium on Research in Attacks, Intrusions and Defenses (RAID 2020)},
year = {2020},
isbn = {978-1-939133-18-2},
address = {San Sebastian},
pages = {427--442},
url = {https://www.usenix.org/conference/raid2020/presentation/wang-xiaoguang},
publisher = {USENIX Association},
month = oct