Speech and gesture analysis: a new approach

Speech and gesture are interlinked components and are critical for felicitous communication. Both these components have certain characteristics, and when convoluted together, they open up greater insights about psychological aspects of the human brain. We use the real-time courtroom truth and lie dataset (2015) comprising 153 videos of truth and lie labeled videos of a courtroom. We have used a convoluted framework and combined both linguistic and gesture approaches to generate a more robust tool for deception analysis and attain a classification accuracy between 65 and 75%. The results show this model will find a great application in interrogative environments where human interactions cannot be completely trusted for deriving results. Besides this, we also wish to introduce a novel approach with our framework to expand our research into the field of medical diagnosis, by using Autism detection as an application to our underlying framework. Through this approach, we wish to address the problem of early autism detection, which is difficult to ascertain through regular clinical practice. The results would enable early diagnosis and early treatment of the concerned problem.