Hidden Conditional Ordinal Random Fields for Sequence Classification

Conditional Random Fields and Hidden Conditional Random Fields are a staple of many sequence tagging and classification frameworks. An underlying assumption in those models is that the state sequences (tags), observed or latent, take their values from a set of nominal categories. These nominal categories typically indicate tag classes (e.g., part-of-speech tags) or clusters of similar measurements. However, in some sequence modeling settings it is more reasonable to assume that the tags indicate ordinal categories or ranks. Dynamic envelopes of sequences such as emotions or movements often exhibit intensities growing from neutral, through raising, to peak values. In this work we propose a new model family, Hidden Conditional Ordinal Random Fields (H-CORFs), that explicitly models sequence dynamics as the dynamics of ordinal categories. We formulate those models as generalizations of ordinal regressions to structured (here sequence) settings. We show how classification of entire sequences can be formulated as an instance of learning and inference in H-CORFs. In modeling the ordinal-scale latent variables, we incorporate recent binning-based strategy used for static ranking approaches, which leads to a log-nonlinear model that can be optimized by efficient quasi-Newton or stochastic gradient type searches. We demonstrate improved prediction performance achieved by the proposed models in real video classification problems.