License plate recognition using 3D rotated character recognition and deep learning

Currently, license plate recognition is being used in a variety of situations, such as parking lot body management and tracking of wanted vehicles. These applications require license plate recognition at any camera angle. Therefore, we propose a method to perform license plate recognition on input images taken from various camera angles. First, the license plate area is detected from the input image using object detection method YOLOv5 [5]. Next, MSER is used to detect character candidates. After applying 3D rotated character recognition to them, the final license plate recognition results are output by graph matching. This method is highly versatile because it does not depend on the layout of cense plates, which vary from country to country and region to region. Two datasets are used: a set of images with Japanese license plate numbers collected by ourselves, and a set of publicly available images with Taiwanese license plates by Hsu et al.. Two recognition rates are evaluated: license plate character recognition rate and license plate recognition rate in which all characters in one plate were recognized. The license plate character recognition rates are 90.7% for Japanese license plates and 80.5% for Taiwanese license plates. The license plate recognition rates are 80.5% and 84.0%, respectively.