Anomaly detection is commonly used for detecting malicious behavior, such as attacks in computer networks or fraud in financial operations. In these settings, criminals have clear incentives to maximize the effectiveness of their behavior, while minimizing the probability of detection.
The existing anomaly detection techniques cannot utilize this knowledge and assume that all anomalies are equally harmful. We study the problem of detecting malicious behavior in the framework of game theory. The defender trains a classifier based on a dataset of benign behavior, attacker’s preferences over the attacks, and a desired false positive rate. The attacker is assumed to be rational who aims to maximize his attack reward while not being too anomalous. We propose a new classifier training procedure that closely approximates the optimal solutions against these attackers and evaluate the algorithm on synthetic and real dataset.
Karel Durkota is a researcher at the Czech Technical University in Prague in the Artificial Intelligence Center Group. His research interests include the application of AI techniques for complex decisions in network security problems.