Japanese historical character recognition by focusing on character parts

Japanese historical documents provide valuable information. Character recognition is a critical technology for the digitalization of historical documents. Sample imbalance is a significant obstacle in recognizing Japanese historical characters, kuzushiji. Thousands of kuzushiji only have less than a few samples. Thus, recognition performance deteriorates greatly in kuzushiji with a few samples. In this study, we propose a framework for transferring knowledge of character parts from font to kuzushiji. The pretraining learns character parts from synthesized font images. However, fine-tuning to kuzushiji is more complex. We propose calculating a mean squared error loss between feature vectors of kuzushiji and font images, resulting in consistent feature vectors in kuzushiji and font. Consequently, we can perform zero-shot recognition for kuzushiji using the font images of zero-sampled kuzushiji. The experimental results show that the proposed method recognized zero-sampled kuzushiji at approximately 48% accuracy. Consequently, we significantly expand the number of recognizable kuzushiji.