© 2007 Science Publications Lizard Learning Algorithm for Invariant Pattern Recognition

Researches are keen to know astonishing and intricate details of the nature. Each creature has its own admiring abilities and performs their routine task in more efficient manner. The bug navigation system has drawn keen attention among research community to know how they are able to perform their routine task in utmost skillful manner. The lizard is capable of identifying slowly varying features and able to trap the insects with more admiring skill set. The Lizard Learning Algorithm (LLA) was proposed for tracking invariant features which uses modified slow feature analysis. The article covers mathematical treatment for the slow feature analysis, proposed modification, higher order neural network training and ORL database for experimentation purpose. The results are most pleasing compared to conventional classifiers for the invariant features.


INTRODUCTION
The bug-navigation system reveals most effective techniques carried out by the small living creatures to meet their bread and butter. It was shocking to note that the small paddle bug is able to choose exact straight-line path to reach their nest from unknown/new-location. The ant column is triggered the development of more efficient tracking algorithms. Further the small spider is capable of choosing an optimal shortest path in a widely spread environment. On the same line in this paper it is proposed a lizard learning algorithm capable of tracking invariant features. The study reveals that the effortless techniques used by lizards while capturing the insects are so effective. They capture the slowly varying features of the insect while tracking and detecting its exact location. These techniques are suitable for implementing the invariant pattern recognition.
Slow feature analysis (SFA) [1,2] is a new unsupervised algorithm to learn nonlinear functions that extract slowly varying signals from time series [1] . SFA was originally conceived as a way to learn salient features of time series in a way invariant to frequent transformations [2] . Such a representation would of course be ideal to perform classification in pattern recognition problems. Most such problems, however, do not have a temporal structure and it is thus necessary to reformulate the algorithm. The basic idea is to construct a large set of small time series with only two elements chosen from patterns that belong to the same class. In order to be slowly varying, the functions learned by SFA will need to respond similarly to both elements of the time series and therefore ignore the transformation between the individual patterns. As a consequence, patterns corresponding to the same class will cluster in the feature space formed by the output signals of the slowest functions, making it suitable to perform classification with simple techniques such as Gaussian classifiers.
The higher order neural network require one pattern should be presented only once during the learning stage. This ability makes them faster compared to single order multi-layered networks. However their computational complexity exponentially grows with respect to the size of the image. In this study some modifications are suggested so that the network can be trained for OCR database.
The SFA algorithm: We can now formulate the Slow Feature Analysis (SFA) algorithm [2] Mathematical constraints: Given a multidimensional time series x(t) = (x 1 (t),... ,X N (T)) T , t ε [t 0 ,t 1 ], find a set of real-valued functions g 1 (x),..., g M (X) lying in a function space F such that for the output signals y j (t) Under the constraints , ( ) 0 ∀ < = i j t i j y y (decorrelation and order), (4) with (.) t and y indicating time averaging and the time derivative of y, respectively. Equation (1) introduces a measure of the temporal variation of a signal (the -value of a signal) equal to the mean of the squared derivative of the signal. This quantity is large for quickly-varying signals and zero for constant signals. The zero-mean constraint (2) is present for convenience only, so that (3) and (4) take a simple form. Constraint (3) means that each signal should carry some information and avoids the trivial solution gj(x) = 0. Alternatively, one could drop this constraint and divide the right side of (1) by the variance (yj)t-Constraint (4) forces different signals to be uncorrelated and thus to code for different aspects of the input. It also induces an order, the first output signal being the slowest one, the second being the second slowest, etc. .

Linear expansion:
For the linear case the function g j (x) becomes Where x is the input vector and w is the weight vector. On solving the above function with the constraints The eigenvectors are stored in the ascending order of eigenvalues to provide slowly varying signals with smaller indices [3] . = AW BWA (9) : ( ) = T t A zz (10) :  [4] define the nonlinear input-output functions g 1 (x),..., g K (X) ε F: which satisfy Constraints (2)-(4) and minimize (1).

SFA for pattern recognition:
The pattern recognition problem can be summarized as follow. Given C distinct classes c 1 ,..., c C and for each class c m a set of P m patterns p 1 (m ) .., p (m) Pm we are requested to learn the mapping c(.) between a pattern p j (m) and its class c(p j (m ) = c m . We define P := Σ c m=1 P m to be the total number of patterns. In general in a pattern recognition problem the input data does not have a temporal structure and it is We reformulate Constraints (2)-(4) by substituting the average over time with the average over all patterns, such that the learned functions are going to have zero mean, unit variance and be de-correlated when applied to the whole training data. This reduces to an optimization problem. More information about SFA is available [5] . Several authors [4][5][6] proposed variant solutions for slow feature analysis.
HONN architecture: The HONN (Higher Order Neural Network) [7][8][9] out performs single order multi layered networks. The HONN requires application of learning pattern only once. Hence its response is faster. They are able to exploit the inter-relations among the input data while providing invariant pattern recognition. The third order network is shown below.
The output of a third-order network can be described by the following equation: where i is the output index, w is the weight associated with a particular triangle, y is the actual output, x is a binary input and a, b and c are the indices of the inputs. A schematic description of this network is shown in Fig. 1.
In the training phase, a perceptron-like rule is used: where t is the expected training output, y is the actual output, g is the learning rate and x is a binary input. The exponential increase of the triangles with the input image is the major problem in higher order neural networks. By restricting to the contour cells (active pixels) and similarities among the triangles are used to reduce the number of triangles and weight classes during the process of training. The following tabular form gives the complexity problem of HONN with image size [10,11] . Number of weight classes is equal to the number of triangles which is equal to where IN is the number of input nodes. The number of possible triangles for different input sizes is given in Table 1.
Example application: We illustrate our method by its application to a person identification using eyes and faces for invariant recognition. We consider the CBCL and ORL face database, which contains of a 20 people approximately 200 images for the person. From the ORL database 10 eyes inclusive of left eye and right eye are considered during the process of training (Fig. 2). They are able to recognize the person in-spite of variations due to the expressions. The features obtained from SFA are trained using both conventional back-propagation algorithm and higher order neural network. The higher order neural network with eyes features given better recognition rate.

CONCLUSION
Our experimental results show the higher order neural network training gives 15% higher performance conventional backpropagation (Fig. 3). We have tried with double backpropagation training the results are not uniformly encouraging. The work can be extended by adding self similarity probabilities while capturing slow features. Further this work can be extended for seen/expression analysis.