Capturing the form of feature interactions in black-box models

Detecting feature interactions is an important post-hoc method to explain black-box models. The literature on feature interactions mainly focus on detecting their existence and calculating their strength. Little attention has been given to the form how the features interact. In this paper, we propose a novel method to capture the form of feature interactions. First, the feature interaction sets in black-box models are detected by the high dimensional model representation-based method. Second, the pairwise separability of the detected feature interactions is determined by a novel model which is verified theoretically. Third, the set separability of the feature interactions is inferred based on pairwise separability. Fourth, the interaction form of each feature in product separable sets is explored. The proposed method not only provides detailed information about the internal structure of black-box models but also improves the performance of linear models by incorporating the appropriate feature interactions. The experimental results show that the accuracy of recognizing product separability in synthetic models is 100%. Experiments on three regression and three classification tasks demonstrate that the proposed method can capture the product separable form of feature interactions effectively and improve the prediction accuracy greatly.