A Multi-level Progressive Rectification Mechanism for Irregular Scene Text Recognition

Rectifying irregular texts into regular ones is a promising approach for improving scene text recognition systems. However, most existing methods only perform rectification at the image level once. This may be insufficient for complicated deformations. To this end, we propose a multi-level progressive rectification mechanism, which consists of global and local rectification modules at the image level and a refinement rectification module at the feature level. First, the global rectification module roughly rectifies the entire text. Then, the local rectification module focuses on local deformation to achieve a more fine-grained rectification. Finally, the refinement rectification module rectifies the feature maps to achieve supplementary rectification. In this way, the text distortion and interference from the background are gradually alleviated, thus benefiting subsequent recognition. The entire rectification stage is trained in an end-to-end weakly supervised manner, requiring only images and their corresponding text labels. Extensive experiments demonstrate that the proposed rectification mechanism is capable of rectifying irregular scene texts flexibly and accurately. The proposed method achieves state-of-the-art performance for three testing datasets including IIIT5K, IC13 and SVTP.