Multi-modal Fusion Based Automatic Pain Assessment

Does he/she look pain? It is a difficult question for computer to reply. Accurate pain assessment is important for proper treatment. It's such a demanding work for human, so an automatic pain assessment system is needed. In this paper, a multi-modal based pain detection method was proposed. A series of features were learned from videos and biomedical signals. Dynamic facial expression analysis and dynamic biomedical signal analysis were conducted for multi-dimensional representation, such as appearance, geometric and head pose, which were presented in temporal manner. Both decision-level and feature-level fusion schemes were utilized to fuse multi-modal and multi-dimensional dynamic features for automatic pain assessment. The proposed scheme achieved the accuracy of 85% and 86% for feature-level fusion and decision-level fusion respectively, which outperformed the results of single-modality. Moreover, the biomedical signal played a key role in the fusion modal.