ANPR: A Novel Technique in Crime Reduction

1.0. Introduction

For example, according to Peter and Osaat (2021), kidnapping for ransom in Nigeria is now a thriving crime and competes well with other crimes now, it is now the third after Boko Haram and pastoral terrorism, it is so successful that it is one of the most lucrative crimes. If were to look critically at the foundation of these trending crimes, we’d see that ordinary citizens and community leaders are always involved in successful kidnapping operations and the basis of their operations on further research could be tied to intelligence reports on the target from the locals and deep down, there is an escape map that always comprises mobility that would get kidnapping victims off the scene. An understanding that vehicles and motorcycles, and community involvement in successful hijacking operations, can inform us of a technological approach that could track and identify vehicle license plates when compared to the vehicle of interest already on file in the police database could give us a providing another opportunity to fight them effectively head-on, applications like this could help the police quickly intercept vehicles that may be of interest or assist in the police investigation of a crime; this, among other things, ANPR would increase safety and allow for more seamless traffic control and vehicle monitoring.

The application of license plate recognition can be an invaluable technology for automated parking, electronic gateway payment, road surveillance, and traffic violations (Fang et al., 2004). Different countries have different license plate formats and this is perhaps at the root of the various challenges the early ANPR developers coped with; Challenges such as license plate type, color, and font (Connie et al, 2018). ANPR may also have difficulty recognizing license plates under arbitrary environmental conditions such as weather (Hurtik, Vaigel, 2017). The ANPR system can be roughly divided into three stages: image acquisition, image processing, and character recognition; The image processing phase is critical in ANPR because it includes contour, localization, and segmentation. Figure 1 below shows the flow chart of an ANPR system.

This paper reviews related work on ANPR well and this research would contribute to the existing literature by presenting optical character recognition-based vehicle number plate recognition using Pytesseract and testing its practical implementation on Nigerian vehicle number plates for which results would be noted and field of possible future work would also be identified. It is important to note that many early researchers only approached the problem with mere image processing and segmentation. Take a good look at some of these approaches and identify the gaps that our work is trying to fill. Our main goal is to design and implement a state-of-the-art Automatic Number Plate Recognition (ANPR) system that supports a high-resolution camera and works in real time. In addition, a separate ANPR algorithm is developed and optimized by exploiting the technical properties of digital image processing algorithms. The study of the algorithm and its optimization focused in particular on real-time image and video processing for License Plate (LP) or Number Plate Localization (NPL), LP Character Segmentation (NPS), and Optical Character Recognition (OCR). the three key phases of the ANPR process. Automatic number plate recognition is an integral part of intelligent transport systems. Its applications include identifying vehicles by their number plates for law enforcement purposes, access control, and toll collection (Wen et al., 2011).

The remainder of this paper is organized as follows. Section 2 presents a review of the method that has been used to extract and recognize license plates number, the strength and weaknesses of the methods are presented. Section 3 presents our materials and methods. We present results and discussion in section 4 where we discussed the detailed processes and steps involved in the design, building, and implementation of ANPR. The conclusion and recommendation are discussed and are presented in section 5.

2.0. Literature Review

There have been several approaches to license plate recognition and extraction over the years; Many of these approaches have focused on using machine learning and image pre-processing algorithms for license plate extraction and recognition. However, the need to use a more accurate technique in computer vision and optical character recognition cannot be overstated, perhaps due to the ANPR's growing popularity in traffic control and security. In this section, very briefly describe the previous methods used in the design and implementation of Automatic Number Plate Recognition and identify the limitations that our current work would satisfy.

2.1. Number Plate Recognition System

In research on Automatic Number Recognition performed by Umar (2017) in the Faculty of Computer Science, Islamia University of Bahawalpur, the author designed and implemented ANPR working in high definition (HD) and in real-time. In addition, a separate ANPR algorithm is developed and optimized by exploiting the technical properties of digital image processing algorithms. The study of the algorithm and its optimization focused in particular on real-time image and video processing for License Plate (LP) or Number Plate Localization (NPL), LP Character Segmentation (NPS), and Optical Character Recognition (OCR). the three key phases of the ANPR process. ANPR is often part of an intelligent transport system. Its applications include identifying vehicles by their number plates for police, access control, and toll collection. The author trained the system using the collected license plate data, then validated and repeated the process until the machine learning algorithms learned all the patterns needed to identify license plates using new data. The system takes an image of the vehicle as input, but the image should be captured from a distance of 2 to 3 feet, then processed through an extractor and segmented as output. According to the author, segmentation starts with extraction, it separates the characters in the image and then saves them as a data matrix. To recognize a license plate, this matrix of license plates was then fed into an artificial neural network. The research was carried out in Matlab.

In a similar work performed by Muhammad T.Q and Muhammad Asif in 2009, the authors design and implement a license plate recognition system combining software and hardware model. The software consists of a set of algorithms written in Matlab and mainly consists of three phases: acquire the image, extract the plate from the image, and recognize the number from the extracted plate. To achieve the first stage of image capture, a USB camera captures an image in RGB format as a result of the action of an IR sensor that detects approaching vehicles. The second stage extracts the license plate from the image of the vehicle. This is achieved by using a yellow search algorithm to locate the region of interest (ROI)

In another similar study conducted by Emina et. al (2012) on Automatic License Plate Recognition, the authors stated that the development of an ANPR might be a better tool to curb the incessant crimes being committed, the author specifically identified intelligence and intelligence gathering, support for local law enforcement agencies as many two of the many Areas where an ANPR could be deployed; Having identified vehicles as a key factor behind rising crimes in Nigeria, recognizing a license plate number registered and captured in a database could help quickly dispatch police to intercept any system-detected vehicles involved in related violent crimes. To implement this, the authors used optical character recognition with openCV to extract license plates. The authors used a total of 20 vehicles with Nigerian number plates, vehicles obtained from the Internet, and a cell phone camera to develop and test the system. All python script was written in pycharm IDE. The method used by the authors first involves an image processing of the vehicles, all the vehicles are color images in RGB format, therefore all the images have been converted to grayscale, then the images have been localized and segmented, then the text in the license plate has been extracted with contours

Our deductions following a clear review of previous work done in the field are as follows;

• Most related work on ANPR approach the problem of license plates recognition as an image preprocessing problem thus, most authors divide it into three stages the following
  • o Image Preprocessing, converting the image to grayscale, noise removal, etc.
  • o Image segmentation
  • o Extraction of license plates number
• All related work on ANPR contributes to how license plates recognition could be extracted even though most follow the same principles without much recourse to the peculiarity of the local license plates i.e country designs and insignia
• Certain factors that could affect the smooth extraction of license plates were either ignored or not in the scope of the studies
• Pytessaract is a very powerful optical recognition algorithm specifically for a problem like this either not used or succulently discussed

Consider in this research paper the use of pytessearate for license plate extraction considering the specific details and factors of license plate design in Nigeria while many related works considered the generality regarding the application of ANPR. Affection for its applications in the Nigerian security system.

3.0. Methods

In this section, the evaluation of the research methods used focused solely on computer vision and optical character recognition and other work in this field. Some of the methods, particularly those that were critical to the completion of this work, would be evaluated, including criticism of the theory used where relevant to the research work.

3.1. Nigeria ANPR

Automatic number plate recognition is a computer vision problem based on optical character recognition. However, the earlier method uses image segmentation and contouring, a method solely focused on Python OpenCV, but recent advances in technology, particularly computer vision, have made optical character recognition the next ideal for extracting license plates from a vehicle to identify. As stated clearly and for our study, we wanted to use OpenCV in addition to Pytesseract, an optical character recognition algorithm to extract and recognize Nigerian license plates. Others include edge detection and contour finding, the steps that could be grouped to form an image process.

3.1.1. Nigeria License Plates Number

An ANPR with Nigerian number plates based on its configuration makes it much easier to track criminal masterminds in real-time. Nigerian number plates have a white background and the number can be in either red, blue, or green, representing commercial, personal, and government entities respectively in the top left. This is where the Nigerian flag is located while the state registers a number and has its slogan displayed in the top center of the plate. The unique way Nigerian license plates have been configured makes them easy to use with ANPR as security applications. The plate is a combination of format ABC-123DE; The first three letters indicate the municipality in which the license plate is registered, followed by a unique three-digit number for the vehicle and two letters. With growing cases of kidnappings across the country and the unique way the license plates are configured, ANPR tracks vehicles of interest in the police database and could achieve real-time interception.

3.1.1. Optical Character Recognition, OCR

Optical character recognition or optical character reader (OCR) is the electronic or mechanical conversion of images of typed, handwritten, or printed text into machine-encoded text, be it from a scanned document, a photograph of a document, a still (Karthik, et al., 2020). Widely used as a form of data entry from printed paper records, whether passport documents, invoices, bank statements, computerized receipts, business cards, mail, hard copies of static data, or any suitable documentation, it is a common way to digitize printed text, allowing it to be electronically edited, searched, more compact stored, displayed online and used in machine processes such as cognitive computing, machine translation, (extracted) text-to-speech, key data. Early versions had to be trained with images of each character and worked on a font at a time. Advanced systems capable of producing a high degree of recognition accuracy for most typefaces are now common and support a variety of digital image file format inputs. Some systems can reproduce formatted output that closely resembles the original page, including images, columns, and other non-text components and text mining. OCR is a research area in the fields of pattern recognition, artificial intelligence, and computer vision (Karthik, et. al, 2020)

3.1.2. Types of OCR

There are quite many types of optical character recognition; it could be said that where optical character recognition is applied and the function it performs informed its many varieties. Some of its types are as follows;

• Word Recognition: WR recognizes handwritten text. The algorithm works in such a way that a structured or unstructured corpus of words is recognized rather than picking each of the characters that form the words
• Character Recognition: CR unlike WR captures each character contained in a handwritten text. Each character is identified one after another.
• Optical Word Recognition: It targets typewritten text while Optical Character recognition, another type identifies each character contained in most of the case typewritten text.

The practical implementation of the study especially in extracting license plate numbers from any vehicle in Nigeria involves pytessaract, one of the optical character recognition algorithms.

3.2. Pytessaract

Python-tesseract is an optical character recognition algorithm tool for Python that can read and recognize text embedded in images and license plates. Other OCR algorithms include easyOCR which can be used to recognize and extract text in documents and also for pattern matching. Tesseract is an open-source text recognition (OCR) engine available under the Apache 2.0 license. It can be used directly or (for programmers) with an API to extract printed text from images. It supports a variety of languages. Tesseract doesn't have a built-in GUI but there are several on the 3rd party side. Tesseract is compatible with many programming languages ​​and frameworks through wrappers that can be found here. It can be used with the existing layout analysis to detect text in a large document, or it can be used in conjunction with an external text detector to detect text from an image of a single line of text. Pytessaract was applied to extract text from a license plate.

Table 1. Important libraries for automatic number plate recognition.

Libraries	Functions	Installation
OpenCV	Processing image, contours, and edge detection	pip install opencv-python
pytesseract	License plate extraction	pip install pytesseract==0.3.7
pandas	Storing extracted license plates in CSV format	pip install pandas
imutils	Image resizing	pip install imutils

Table 1. Important libraries for automatic number plate recognition.

Libraries	Functions	Installation
OpenCV	Processing image, contours, and edge detection	pip install opencv-python
pytesseract	License plate extraction	pip install pytesseract==0.3.7
pandas	Storing extracted license plates in CSV format	pip install pandas
imutils	Image resizing	pip install imutils

3.4. Data Preprocessing

Recognizing the license plate is the first thing that takes place before extracting the license plate information. As clearly stated, extracting license plate information involves the pytesseract OCR algorithm, but before the extraction could take place, the recognition of license plates was done with a series of steps that included image segmentation, edge detection, canny and contours; the process or steps are combined to be image pre-processing. In this section, discuss the steps to preprocess the captured vehicles (images). Use OpenCV as a tool for image preprocessing.

3.4.1. OpenCV

OpenCV (Open-Source Computer Vision Library) is an open-source computer vision and machine learning software library. OpenCV was developed to provide a common infrastructure for computer vision applications and to accelerate the use of machine vision in commercial products. As a BSD-licensed product, OpenCV makes it easy for companies to use and modify the code. The library has more than 2500 optimized algorithms, including a comprehensive set of classic and cutting-edge computer vision and machine learning algorithms. This study used OpenCV algorithms to process images of the captured vehicle; The processing steps include contour finding, edge detection, image manipulation, image resizing, etc.

3.4.2. Grayscaling Vehicle Images

OpenCV does not support the native RGB channels on which images are saved; Grayscale becomes important in others for us to manipulate the image, find contours and recognize license plates. Grayscale is the process of converting an image from other color spaces, e.g. RGB, CMYK, HSV, etc. up to greyscale. It varies between all-black and all-white (geeksforgeeks, 2023). The importance of grayscale images of a vehicle is that we can work with a single-channel image as supported by OpenCV.

Figure 3. Original colored car image(author, 2023).

Figure 3. Original colored car image(author, 2023).

Figure 4. Gray image (author, 2023).

Figure 4. Gray image (author, 2023).

Figure 5. Bilateral filtered image(author, 2023).

Figure 5. Bilateral filtered image(author, 2023).

3.4.3. Canny Edge Detection

Edge is the border between two regions with different colors or different regions. Edge detection is the most common approach to detecting meaningful discontinuities in intensity values ​​(Lourdes, 2021). Smart edge detection is an edge detection operator that uses a multi-level algorithm to detect a variety of edges in images (Wikipedia, undated). The canny edge algorithm is included below

• Noise Reduction
• Gradient Calculation
• Non-Maximum Suppression
• Double Threshold
• Edge Tracking

Figure 6. Canny-edged detection(author,2023).

Figure 6. Canny-edged detection(author,2023).

3.4.4. Contours

The contour can be referred to as a curve connecting all continuous points along a boundary with the same chroma; It's an algorithm with a cunning edge that helps find the position of a license plate in it. The OpenCV function findcontour() helped to find the license plate area on the image.

3.4.5. Measure of Performance

The performance of the ANPR is based on how accurately the system can recognize license plates and correctly extract useful information embedded in them.

4.0. Results and Discussion

We have pre-processed the vehicle and this can perhaps be considered the most burning task in our research work, the automatic number plate recognition systems. The images of the vehicle were first captured by the webcam camera connected to the ANPR system. The input vehicle image was changed from RGB color code to grey format which is the acceptable OpenCV color code, the grayscale was during pre-processing, and then a bilateral filter happened to the vehicle image which smoothened the now already greyscaled vehicle image. The edges in the image were then detected with the help of the Canny Edge Detection algorithm. The sly edges are recognized. Contours with 4 edges are sorted. The contour with the highest rank is considered the license plate area and clipped; it was after this that the character on the license plate was extracted. Pytesseract is an OCR that performs character recognition on this area of ​​the image.

As clearly stated in the previous paragraph, the pre-processing of the images is considered an important milestone for the implementation of ANPR; Once a vehicle is detected, the system reads the image using OpenCV imread (c2.imread()) and the image is resized to the expected shape. This step is captured with the following code

image = cv2.imread(folder_in_path + filename)

image = imutils.resize(image, width=500)

gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

The OpenCV cv2.cvtcolor() as seen from the above code snippet converts the input image to grayscale (see fig. 3), and bilateral filtering (see fig. 4) is applied to smoothen the image and reduces the noise on the image.

edged = cv2.Canny(gray, 170, 200)

cv2.imshow("4 - Canny Edges", edged)

The edges of the image are detected with cv2 canny edge(see fig. 5) and it’s from this edge detection we find all of the contours present in the image the top 30 contours are sorted, and the contour selections are at random and only the top ones are selected; with a for statement, we set the perimeter of each of the contours and we select only the ones with four(4) corners. Pytesseract is then used to extract information from the recognized plate number.

Figure 7. License plate number detected (author, 2023).

Figure 7. License plate number detected (author, 2023).

Figure 8. Undetected license (author, 2023).

Figure 8. Undetected license (author, 2023).

The rest of the code is found in Appendix 1

License plates were extracted only after all necessary things were checked and marked, the license plate on images taken by an ANPR after going through its processing steps was assumed and identified as the position of a license plate, and a rectangular box was placed on top of the identified Specify license plate. After identification, the information about it was extracted with pytesseract and the extracted license was saved in a local database in csv format. This implies that security agencies could keep a reference of the license number each time a vehicle is spotted and if it was a police vehicle of interest the police could easily dispatch a team to intercept and arrest the vehicle if necessary.

Figure 9. ANPR use case Diagram (author, 2023).

Figure 9. ANPR use case Diagram (author, 2023).

Table 2. Results.

Total number of vehicles	Total accurate read	Success %
20	18	95

Table 2. Results.

Total number of vehicles	Total accurate read	Success %
20	18	95

5.0. Conclusion

In this research paper, the applications of automatic number plate recognition and the processes involved in its implementation are discussed. ANPR, especially in developing countries like Nigeria where various security challenges such as kidnappings are rampant, could be used to track criminal masterminds of such acts in real-time and assist police in adequately providing security and timely arresting of those involved in crimes are involved. Their application can be used both in gateway payment and in traffic violations. The Nigerian license plate extraction algorithm developed in this work turned out to be very effective and efficient. It had 100% accuracy in license plate location but 90% reading accuracy. All images tested were images of a stationary vehicle. It has not been tested for moving vehicles. These two valves are for images under different lighting conditions.