The paper proposes a method of forming a signature of a virus program based on API call tracing. The developed signature is compact and makes it possible to assign the virus program not only to one of the classes of viruses, but also allows determining its modification. An approach is proposed that allows detection of a virus program represented by a developed signature. The developed method for carrying out experimental researches is realized in distributed multilevel detection system of malicious software in local area computer networks.
To implement the virus detection process, a system has been developed that consists of three main components: the signature of the virus program, the signature database and the method of detection, which involves comparing the signatures of the virus with the database of signatures.
Signature of the program's behavior based on API call tracing can be represented as a set of two components: the call frequency and the nature of the interaction of critical API calls. Analysis of the first component allows determining the distribution of critical API calls by groups of harmful activity and displays the quantitative component of the signature. The second component of the signature implies the mapping into the vector space the nature of the interaction of critical API functions that characterize virus program and describes the relationship between critical API functions. Analysis of the second component of the signature provides an opportunity to distinguish virus programs from useful applications not only in the presence of critical API calls, but also in their interaction with each other.
A number of experimental studies have been carried out to determine the accuracy of the proposed method for detecting virus programs using the developed signature. Viral family programs Delf, Bifrose and MyDoom were used as test data. The best detection rate was received for MyDoom virus programs (accuracy was 93%) with the use of the J48 binary classification algorithm.