Final published version, 3.09 MB, PDF document
Available under license: CC BY: Creative Commons Attribution 4.0 International License
Final published version
Licence: CC BY: Creative Commons Attribution 4.0 International License
Research output: Contribution to Journal/Magazine › Journal article › peer-review
Research output: Contribution to Journal/Magazine › Journal article › peer-review
}
TY - JOUR
T1 - VMGuards
T2 - A Novel Virtual Machine Based Code Protection System with VM Security as the First Class Design Concern
AU - Tang, Zhanyong
AU - Li, Meng
AU - Cao, Shuai
AU - Chen, Meiling
AU - Ye, Guixin
AU - Gong, Xiaoqing
AU - Fang, Dingyi
AU - Wang, Zheng
PY - 2018/5/12
Y1 - 2018/5/12
N2 - Process-level virtual machine (PVM) based code obfuscation is a viable means for protectingsoftware against runtime code tampering and unauthorized code reverse engineering. PVM-basedapproaches rely on a VM to determine how instructions of the protected code region are scheduledand executed. Therefore, it is crucial to protect the VM against runtime code tampering that altersthe instructions and behavior of the VM. This paper presents VMGuards, a novel PVM-based codeprotection system that puts the security of VM as the first class design concern. Our approachadvances prior work by promoting security of the VM as the first class design constraint. We achievethis by introducing two new instruction sets to protect the internal implementations of criticalcode segments and the host runtime environment where the VM runs in. Our new instructionsets not only have an identical code structure as standard virtual instructions, but also provideadditional information to allow the VM to check whether the critical internal implementation or theruntime environment is affected. We evaluate our approach by using a set of real-life applications.Experimental results show that our approach provides stronger and more fine-grained protectionwhen compared to the state-of-the-art with little extra overhead.
AB - Process-level virtual machine (PVM) based code obfuscation is a viable means for protectingsoftware against runtime code tampering and unauthorized code reverse engineering. PVM-basedapproaches rely on a VM to determine how instructions of the protected code region are scheduledand executed. Therefore, it is crucial to protect the VM against runtime code tampering that altersthe instructions and behavior of the VM. This paper presents VMGuards, a novel PVM-based codeprotection system that puts the security of VM as the first class design concern. Our approachadvances prior work by promoting security of the VM as the first class design constraint. We achievethis by introducing two new instruction sets to protect the internal implementations of criticalcode segments and the host runtime environment where the VM runs in. Our new instructionsets not only have an identical code structure as standard virtual instructions, but also provideadditional information to allow the VM to check whether the critical internal implementation or theruntime environment is affected. We evaluate our approach by using a set of real-life applications.Experimental results show that our approach provides stronger and more fine-grained protectionwhen compared to the state-of-the-art with little extra overhead.
KW - software tamper-proofing
KW - process-level virtual machine protection
KW - code obfuscation
KW - software protection
U2 - 10.3390/app8050771
DO - 10.3390/app8050771
M3 - Journal article
VL - 8
JO - Applied Sciences
JF - Applied Sciences
SN - 2076-3417
IS - 5
M1 - 771
ER -