CNN based spatial classification features for clustering offline handwritten mathematical expressions

To help human markers mark a large number of answers of handwritten mathematical expressions (HMEs), clustering them makes marking more efficient and reliable. Clustering HMEs, however, faces the problem of extracting both localization and classification representation of mathematical symbols for an HME image and defining the distance between two HME images. First, we propose a method based on Convolutional Neural Networks (CNN) to extract the representations for an HME. Symbols in various scales are located and classified by a combination of features from a multi-scale CNN. We use weakly supervised training combined with symbols attention to enhance localization and classification predictions. Second, we propose a multi-level spatial distance between two representations for clustering HMEs. Experiments on CROHME 2016 and CROHME 2019 dataset show the promising results of 0.99 and 0.96 in purity, respectively. (C) 2020 Elsevier B.V. All rights reserved.