CODEX
7 Steps to Build Automatic Number Plate Recognition in Python
Image Processing and Object Detection is one of the areas of Data Science and has a wide variety of applications in the industries in the current world. Many industries looking for a Data Scientist with these skills.
This article explains how to develop a number plate object detection model from scratch. We developed even developed an API using Flask. However, in this article, we explained how to train a customize object detection model from scratch.
For building the license plate recognition we need data. For that, we need to collect the vehicle images where the number plate appears on it. Here is the sample data that I used for building this project. You can also download it from my google drive.
Project Architecture:
Now, let's look into the project architecture of the number plate recognition and OCR we follow to build.
In the above architecture, there are six modules. Labeling, Training, Save Model, OCR and Pipeline, and RESTful API. But this article limits to only the first three modules. The process is as follows. First, we will collect the image. Then we have to label images for object detection of License Plate or Number Plate using Image Annotation Tool which is open-source software developed in python GUI. Then after labeling the image we will work on data preprocessing, build and train a deep learning object detection model (Inception Resnet V2) in TensorFlow 2. Once we have done with the Object Detection model training process, then using this model we will crop the image which contains the license plate which is also called the region of interest (ROI), and pass the ROI to Optical Character Recognition API Tesseract in Python (PyTesseract). In this module, we will extract text from images. Now, we will put it all together and build a Pipeline Deep Learning model. In the final module, we will learn to create a web app project using FLASK Python. With that, we are finally ready with our App.
Step -1: Labeling
For building the license plate recognition we need data. For that, we need to collect the vehicle images where the number plate appears on it. Here is the sample data that I used for building this project. You can also download it from my google drive. For label images, I used the LabelImg image annotation tool. Download the labelImg from GitHub and follow the instruction to install the package. After that open, the GUI as give the instruction and click on CreateRectBox and draw the rectangle box as shown below and save the output in XML.
pip install pyqt=5
pip install lxml
pyrcc5 -o libs/resources.py resources.qrc
python labelImg.py
python labelImg.py [IMAGE_PATH] [PRE-DEFINED CLASS FILE]
This is a manual process and you need to do it for all the images. Be careful while doing labeling because the labeling process has a direct impact on the accuracy of the model. Click here for a video tutorial
Step-2: Parsing Information from XML
Once you are done with the labeling process now we need to do some data preprocessing.
Since the output of the label is XML in order to use this for the training process we need data in array format. For that, we will take the useful information from the label which are the diagonal points of the rectangle box or bounding box which are xmin, ymin, xmax, ymax respectively as shown in figure 3. This is available in XML. So we need to extract the information and save it in any convenient format, here I will convert bounding information into CSV and later on, I will convert that into an array using Pandas. Now let’s see how to parse the information using Python.
Parsing data from XML and Converting it into CSV
Here I am using xml.etree python library to parse the data from XML and also import pandas and glob. Using glob let first get all the XML files that are produced during labeling.
In the above code, we individually take each file and parse into xml.etree and find the object -> bndbox which is in line 2 to 7. Then we extract xmin,xmax,ymin,ymax and saved those values in the dictionary which is in lines 8 to 17. After then we convert it into a pandas data frame and save that into CSV file as shown below.
With the above code, we successfully extract the diagonal position of each image and convert the data from an unstructured to a structured format.
Now also extract the respective image filename of the XML.
Step-4: Verify the Data
As till now we did the manual process it is important to verify the information is we got is valid or not. For that just verify the bounding box is appearing properly for a given image. Here I consider the image N1.jpeg and the corresponding diagonal position can found in df.
Step -5: Data Processing:
This is a very important step, in this process we will take each and every image and convert it into an array using OpenCV and resize the image into 224 x 224 which is the standard compatible size of the pre-trained transfer learning model.
After that, we will normalize the image just by dividing with maximum number as we know that the maximum number for an 8-bit image is
That the reason we will divide our image 255.0. The way of diving an array with the maximum value is called Normalization (Min-Max Scaler).
We also need to normalize our labels too. Because for the deep learning model the output range should be between 0 to 1. For normalizing labels, we need to divide the diagonal points with the width and height of the image. And finally values in a python list. In the next step, we will convert the list into an array using Numpy.
Now split the data into training and testing set using sklearn.
Step -6: Build and Train Transfer Learning
Now we are ready to train a deep learning model for object detection. Here we will use the InceptionResNetV2 model with pre-trained weights and train this to our data. Let’s start this by importing necessary libraries from TensorFlow 2.3.0
The model we are building is an object detection model and the number of output expected from the is model 4 which are the diagonal points. Hence we will add an embedding neural network layer to the transfer learning model which is shown in lines 5 to 9.
Now compile the model and train the model
That’s it with that we trained the model. This process usually takes 3 to 4 hours depends upon the speed of the computer. Here I showing you the loss of the model in TensorBoard.
It seems that the model converged after 100 epochs.
Note: For better reduction in loss we need to train model with lots of data at least 10000 images
Step-7: Make Bounding Box Prediction
This is the final step in object detection. In this step, we will put it all together and get the prediction for a given image.
That’s it !!!. This is how we can develop license plate detection from Scratch.
This article explains just 50% of the project architecture. The next process involves extract text from the number plate and developing RestfulAPI in Flask. Here is the output of the project
Click here for 3 hr Video Project Explanation and building Number Plate Web App from Scratch.
