An Improved Approach for Detecting Car in Video using Neural Network Model

: The study represents a novel approach taken towards car detection, feature extraction and classification in a video. Though many methods have been proposed to deal with individual features of a vehicle, like edge, license plate, corners, no system has been implemented to combine features. Combination of four unique features, namely, color, shape, number plate and logo gives the application a stronghold on various applications like surveillance recording to detect accident percentage(for every make of a company), authentication of a car in the Parliament(for high security), learning system(readily available knowledge for automobile tyro enthusiasts) with increased accuracy of matching. Video surveillance is a security solution for government buildings, facilities and operations. Installing this system can enhance existing security systems or help start a comprehensive security solution that can keep the building, employees and records safe. The system uses a Haar cascaded classifier to detect a car in a video and implements an efficient algorithm to extract the color of it along with the confidence rating. An gadabouts trained classifier is used to detect the logo (Suzuki/Toyota/Hyunadai) of the car whose accuracy is enhanced by implementing SURF matching. A combination of blobs and contour tracing is applied for shape detection and model classification while number plate detection is performed in a smart and efficient algorithm which uses morphological operations and contour tracing. Finally, a trained, single perceptron neural network model is integrated with the system for identifying the make of the car. A thorough work on the system has proved it to be efficient and accurate, under different illumination conditions, when tested with a huge dataset which has been collected over a period of six months.


NTRODUCTION
In real time traffic there are different objects of interest. The objects have characteristic and have a high probabilistic behavior in interaction with other objects in real time. The object characteristics can be analyzed as features of interest and can be modeled in real time video.
The features extracted can be analyzed using several approaches. The primary objective of feature extraction aims at achieving dimensionality reduction. The features analyzed vary from domain to domain.
There are lots of classification mechanisms in research. A proper classification methodology can be applied for classifying the objects belonging to different classes.

Previous research:
The algorithm which is used for feature extraction is K nearest neighbor local margin maximization. The comparison is done using with Linear Discriminant Analysis (LDA), Principal Component Analysis (PCA) and Maximum Margin Criterion (MMC). Many feature extraction algorithms have been proposed in the last decades. Principle Component Analysis (PCA), as a well-known unsupervised feature extraction method, aims to extract a subspace in which the reconstruction error can be minimized, or equivalently the variance of the total papered data points can be maximized (Pan et al., 2009). This method performs better in high dimensionality. It reduces error rate and complexity by selecting only limited K nearest neighbors for each point. The approach will not work with high dimensional data.
Improve harris corner detection and descriptor of SIFT method are applied to extract features from images. First, the method of Harris corner detection is used for finding the interest points from the two images to be matched, in which an extreme tactics is adopted for exactly determining the interest points (Zhang, 2004;Zhang et al., 2009). This is followed by adopting a sub-block checking method for eliminating the cluster and reduce the 15 number of interest points. Sobel operator shows good results in edge extraction and inhabits the noisy better than the other two methods.
General optical flow algorithm detects noisy and irrelevant features. The extended version of this algorithm extracting low level features. To overcome the problem due to complex scenes it introduces a frame jump technique along with thresholding of unwanted features. The demerits of this method is integrated with other feature tracking algorithm will increase the possibility of classifying variety of human actions (Choi, 2010).
The polygonal line algorithm applied to extract nonlinear feature extraction, in which the nonlinearities among the multivariable data can be described by a set of local linear models. The polygonal line algorithm can produce robust and accurate nonlinear curve estimation for different multivariate data types and it is helpful in reducing the computation complexity for existing principal curve approaches when the sample size is large (Zhang, 2004). This approach decrease noise effect in estimation accuracy.

Proposed approach:
The proposed approach combines the approach of combining surf features with Neural network model is shown in Fig. 1. The extracted features include color, shape, logo and number plate region (Shi and Tomasi, 1994;Sebe et al., 2003). The real time detection analysis of features forms the basis in the proposed model (Stojmenovic, 2005;Jang and Turk, 2011;Jazayeri et al., 2011).

Object detection using haar like features and SURF features:
The surf features is combined with haar like features for detecting objects (AT, 2011).
Step 1: Object detection using haar wavelet and confining roi.
The object (car) detection is done by Haar wavelet method. Haar wavelet technique uses a haarcascade car XML file which detects all rigid bodies. The detected car region is set as our ROI.
Step 2: Getting rgb and hsv value for color.
The central pixels (area = 5×5) of ROI is considered for identifying the color.
The RGB value of the pixel is converted to a HSV (Hue-Saturation-Value) as HSV is better at handling lighting differences and it gives an easy to use color value.
The approximate color is found based on HSV components.
User defined function used-getcolortype (int H, int S, int V).
Step 4: Finding confidence percentage of color.
Confidence percentage=initial confidence * (tally max count * 100/pixels) Tally max count=number of pixels found using each color type.
Apply Gaussian technique for smoothing.
Step 2: Morphological operations: Tophat Top hat detects the License plate which will be lighter than its neighbors.
It exaggerates those portions which are lighter than the surroundings.
Assumption-The license plate mostly contains light portions.
The result is to be thresholded so that the light regions are more prominent for contour extraction. Thresholding has been done for all pixels above 128-255.
Step 4: Contour extraction and applying restrictions to roi.
The area between. 600 and 5000 and ratio between. 3.0 and 8.0 is the threshold set for contour extraction.

Logo detection and SURF matching:
The logo recognition has been completed for a three class of cars using adaboost classifier (Burkhard et al., 2011).
Step 1: Creating descriptive and vec files.
Creating the descriptive file of positive and negative samples (collected manually) and a vec file is created. A vec file is created using the create samples utility provided with opencv package (NB, 2010).    Step 2: Train the adaboost classifier to obtain logo xml files.
The vec file obtained is converted to an XML file for processing.
Step 3: Logo detection using xml files.
XML files (which comprises of coordinate values.) of Suzuki, Hyundai and Toyota obtained as a result of classification is used to draw the ROI around the respective logos.
Content of XML-image name, the number of objects to be detected in the image, x-y coordinates of the location of the object in the image.
Step 4: SURF matching of detected logo with logo templates.
SURF detects ipoints (interest points) in the test data and the template and performs matching between these two. It records the number of matches found if any.
The logo detection results are better than the existing approaches (Sulehria and Zhang, 2007;Psyllos et al., 2010a).

Shape extraction:
Step 1: Detection of the largest blob in the templates Step 2: Draw contour around the largest blob detected Step 3: Repeat Step 1 and 2 for a test image.
Step 4: Perform contour matching with test and template data.
The following templates as shown in Table 3 are chosen for shape detection and classification.

MATERIALS AND METHODS
The neural network has been designed for training and testing for recognizing the car in the video (Smach et al., 2005).
Step 1: Parameters calculated Detection time per frame, average detection time (for all features), confidence percentage (color) per frame, Average confidence percentage (color), error percentage (shape), No. of SURF matches with each logo template, percentage of frames with number plate detected Step 2: Obtained the feature vector < Color, number plate status, logo, shape template, % shape error> Step 3: Collected training samples for NN 3 images of 3 models (Ex: Swift, Dezire, SX4 for Suzuki) for each logo were chosen for training. Their corresponding feature vectors were obtained.
Step 4: Train a single layer perceptron neural network model The initial weights is 0. The inputs are 6 <feature vector > including a constant bias of 1. The classes used were -1, 0 and 1 to classify cars as Suzuki, Hyundai or Toyota. The error for each training sample to change the weights were found using the classes and the dot product of the weights and the feature vector including bias.

RESULTS
The proposed approach has been tested on datasets that have been shot using a PowerShot SX110 IS Canon camera with 9.0 Mega Pixels and 10X Optical Zoom lens (10X IS). The image analysis characteristics is also considered for feature analysis Image Analysis Labs, 2010. The implementation has been completed in opencv with dotnet framework Structural Analysis and Shape Descriptors, 2010(AB, 2010. The results for the different brand of cars is presented belowin Fig. 2-4.
The results obtained have been tested on videos containing yellow number plate on yellow black ground and similarly white number plate on white background and for front and back view of the cars as in Fig. 5 and 6.
The number plate detection has been tested for the top view and back view of cars and detection has been good as in other cases (Psyllos et al., 2010b) as shown in Fig. 7 and 8. The comparisons are tabulated in Table 4 and 5.
Training of the single perceptron neural network: A sample of 27 images (3 images of 3 models for each logo) was collected for training and their corresponding feature vectors were obtained.

Input: Training data
Processing: The initial weights is 0. The inputs are 6 <feature vector > including a constant bias of 1. The classes used were -1, 0 and 1 to classify cars as Suzuki, Hyundai or Toyota. The error for each training sample to change the weights was found using the classes and the dot product of the weights and the feature vector including bias. Table 6 shows the different classes for classifier.

DISCUSSION
After training with 27 images of different models, the final weights were obtained as -1.5, -0.5, -1.5, 4.875, 0 and 0.067. The test feature vectors of videos were given as input and they were classified into their appropriate classes with the help of the final weights obtained after training. Figure 10 shows the results after recognition stage.
In the generated results upto 0.5% of Root mean square error is allowed for classification.

CONCLUSION
For a live streaming video, the color (approximate) of the car can be displayed along with its confidence percentage, provided the video is not too zoomed in (few 100 cms near the car). If the car is really close to the camera then the color detection code does not work properly as the Haar cascade file fails to identify the car. The program also handles multiple objects and it can detect upto 4 cars along with their colors and confidence percentages. Color values are restricted to eleven.
The proposed system is designed to detect the number plate of an Indian car. For detecting vehicles of other nationalities, blob filtering values have to be altered. Scope can be extended to detect license plates of other vehicles too by changing the restrictions defined during contour extraction. The algorithm works well for number plates with lighter shade backgrounds other than white also. Logo detection has been done for Hyundai, Suzuki and Toyota cars. SURF is combined with Haar to overcome the drawbacks of Haar and produce results with better classification accuracy (Pavania et al., 2010). This module handles videos which have a wellzoomed version of a moving car better than a zoomedout version.
The proposed work a novel method to extract the shape of the object by combining blobs and contours. It also helps to decide if it is the extracted shape is that of a car or not, by comparing it with template of car images. The feature vector is built and verified with the Single Perceptron Neural Network Classifier that has been built.
The application developed can be used for surveillance cameras. It is useful for detecting essential features of a car with which a lost or stolen vehicle can be identified. The feature vector generated is useful in recognizing and distinguishing cars from one other. The feature vector could take fuzzy values that can also be tested in future (Lin and Wang, 2007).