Geodesic
Application of x86 extensions for code protection
Prikladnaya Diskretnaya Matematika. Supplement, no. 14 (2021), pp. 138-140

Voir la notice de l'article provenant de la source Math-Net.Ru

A new approach is proposed to protect the program code against reverse engineering tools, such as decompilers and symbolic execution tools. The approach is based on the usage of uncommon x86 processor instructions that could be implemented incorrectly in the aforementioned tools. Existing approaches to this problem are also considered, and the relative performance advantage of the proposed approach is noted. A method for numeric constants obfuscation, following this approach, is developed with the usage of AES-NI extension for the x86 architecture and its AESENC instruction in particular. This method is implemented for Clang compiler with the help of LLVM Intermediate Representation and tested against reverse engineering tools, such as IDA and Ghidra decompilers and angr symbolic execution tool.
Mots-clés : code protection
Keywords: reverse engineering, decompiler, symbolic execution, x86 processor architecture. 
@article{PDMA_2021_14_a30,
     author = {R. K. Lebedev and I. A. Koryakin},
     title = {Application of x86 extensions for code protection},
     journal = {Prikladnaya Diskretnaya Matematika. Supplement},
     pages = {138--140},
     publisher = {mathdoc},
     number = {14},
     year = {2021},
     language = {ru},
     url = {http://geodesic.mathdoc.fr/item/PDMA_2021_14_a30/}
}
TY  - JOUR
AU  - R. K. Lebedev
AU  - I. A. Koryakin
TI  - Application of x86 extensions for code protection
JO  - Prikladnaya Diskretnaya Matematika. Supplement
PY  - 2021
SP  - 138
EP  - 140
IS  - 14
PB  - mathdoc
UR  - http://geodesic.mathdoc.fr/item/PDMA_2021_14_a30/
LA  - ru
ID  - PDMA_2021_14_a30
ER  - 
%0 Journal Article
%A R. K. Lebedev
%A I. A. Koryakin
%T Application of x86 extensions for code protection
%J Prikladnaya Diskretnaya Matematika. Supplement
%D 2021
%P 138-140
%N 14
%I mathdoc
%U http://geodesic.mathdoc.fr/item/PDMA_2021_14_a30/
%G ru
%F PDMA_2021_14_a30
R. K. Lebedev; I. A. Koryakin. Application of x86 extensions for code protection. Prikladnaya Diskretnaya Matematika. Supplement, no. 14 (2021), pp. 138-140. http://geodesic.mathdoc.fr/item/PDMA_2021_14_a30/