Real-time license plate detection and recognition in unconstrained scenarios

To solve the problem that current mainstream license plate recognition methods have unsatisfactory results under unconstrained scenarios, a real-time license plate detection and recognition algorithm based on extended YOLOv5 and PP-OCR v2 is proposed. First, the output of the YOLOv5's detection head is extended to predict four vertex coordinates of the candidate license plate, and the accurate license plate region can be located with these four vertices. Furthermore, the vertex coordinates can be used to correct the license plate image with the inverse perspective transformation. After that, the license plate image fed into the following recognition network can present a frontal view, which can effectively reduce recognition errors. Then, the lightweight text recognition network derived from the PP-OCR v2 with the recurrent layer removed is utilized to recognize the corrected license plate image. The experiment results on the public dataset CCPD show that, the proposed algorithm achieves 99.34% detection accuracy and 97.61% recognition accuracy, and the processing speed of license plate detection reaches 84FPS and license plate recognition detection reaches 667FPS under the mid-end GPU, and the processing speed of the whole license plate recognition system is 87FPS, which meets the practical real-time requirements.