Combining Defences Against Data-Poisoning Based Backdoor Attacks on Neural Networks

Machine learning-based systems are increasingly used in critical applications such as medical diagnosis, automotive vehicles, or biometric authentication. Because of their importance, they can become the target of various attacks. In a data poisoning attack, the attacker carefully manipulates some input data, e.g. by superimposing a pattern, e.g. to insert a backdoor (a wrong association of the specific pattern to a desired target) into the model during the training phase. This can later be exploited to control the model behaviour during prediction, and attack its integrity, e.g. by identifying someone as the wrong user or not correctly identifying a traffic sign, thus causing road incidents. Poisoning of the training data is difficult to detect, as often, only small amounts of the data need to be manipulated to achieve a successful attack. The backdoors inserted into the model are hard to detect as well, as its unexpected behaviour manifests only when the specific backdoor trigger, which is only known to the attacker, is presented. Nonetheless, several defence mechanisms were proposed, and in the right setting, they can yield usable results; however, they still show shortcomings and insufficient effectiveness in several cases. In this work, we thus try to answer the extent to which combinations of these defences can improve their individual effectiveness. To this end, we first build successful attacks for two datasets and investigate factors influencing the attack success. Our evaluation shows a substantial impact of the type of neural network models and datasets on the effectiveness of the defence. We also show that the choice of the backdoor trigger has a big impact on the attack and its success. Finally, our evaluation shows that a combination of defences can improve existing defences in several cases.