Backdoor attacks, which induce a trained model to behave as intended by an adversary for specific inputs, have recently emerged as a serious security threat in deep learning-based classification models. In particular, because a backdoor attack is executed solely by incorporating a small quantity of malicious data into a dataset, it poses a significant threat to authentication models, such as facial cognition systems. Depending on whether the label of the poisoned samples has been changed, backdoor attacks on deep learning-based face recognition methods are categorized into one of the two architectures: 1) corrupted label attack; and 2) clean label attack. Clean label attack methods have been actively studied because they can be performed without access to training datasets or training processes. However, the performance of previous clean label attack methods is limited in their application to deep learning-based face recognition methods because they only consider digital triggers with instance-specific characteristics. In this study, we propose a novel clean label backdoor attack, that solves the limitations of the scalability of previous clean label attack methods for deep learning-based face recognition models. To generate poisoned samples that are instance agnostic while including physical triggers, the proposed method applies three core techniques: 1) accessory injection; 2) optimization-based feature transfer; and 3) $N$ :1 mapping for generalization. From the experimental results under various conditions, we demonstrate that the proposed attack method is effective for deep learning-based face recognition models in terms of the attack success rate on unseen samples. We also show that the proposed method not only outperforms the recent clean label attack methods, but also maintains a comparable level of classification accuracy when applied to benign data.
