End-to-End Detection and Recognition of Arithmetic Expressions

The detection and recognition of handwritten arithmetic expressions (AEs) play an important role in document retrieval [ 21] and analysis. They are very difficult because of the structural complexity and the variability of appearance. In this paper, we propose a novel framework to detect and recognize AEs in an End-to-End manner. Firstly, an AE detector based on EfficientNet-B1 [17] is designed to locate all AE instances efficiently. Upon AE location, the RoI Rotate module [11] is adopted to transform visual features for AE proposals. The transformed features are then fed into an attention mechanism based recognizer for AE recognition. The whole network for detection and recognition is trained End-to-End on document images annotated AE locations and transcripts. Since the datasets in this field are rare, we also construct a dataset named HAED, which contains 1069 images (855 for training, and 214 for testing). Extensive experiments on two datasets (HAED and TFD-ICDAR 2019) show that the proposed method has achieved competitive performance on both datasets.