Hidden Markov models for reading words from the human brain

Recent work has shown that it is possible to reconstruct perceived stimuli from human brain activity. At the same time, studies have indicated that perception and imagery share the same neural substrate. This could bring cognitive brain computer interfaces (BCIs) that are driven by direct readout of mental images within reach. A desirable feature of such BCIs is that subjects gain the ability to construct arbitrary messages. In this study, we explore whether words can be generated from neural activity patterns that reflect the perception of individual characters. To this end, we developed a graphical model where low-level properties of individual characters are represented via Gaussian mixture models and high-level properties reflecting character co-occurrences are represented via a hidden Markov model. With this work we provide the initial outline of a model that could allow the development of cognitive BCIs driven by direct decoding of internally generated messages.