Change Detection Using Neural Network with Improvement Factor in Satellite Images

: Problem statement: The aim of this study is to investigate the applicability of using the neural network techniques in change detection of remotely sensed data. Approach: In addition, the tuning parameters of the network, such as encoding the output classes, adding the momentum term and learning rate, are investigated in order to achieve best network performance. Results: Neural network-based change detection system in this study is implemented using back propagation-training algorithm. This trained network is designed to be able to detect efficiently any variation between two images and provide adequate information about the type of changes. In an effort to meet these requirements, neural network scheme with improvement factor, leaning rate and momentum term is proposed to monitor environmental changes in Toshka area, Egypt. Two sets of satellite images with different dates are used, the first set contains of two sample satellite images, the second set of images acquired on 1984, 2000 and 2003. Conclusion/Recommendations: Comparing the output of the proposed model with the mostly used change detection techniques; ratio and classification, results show a great potential as the proposed scheme was able to identify not only the changed and non-changed area but also it was capable to identify the nature of these changes.


INTRODUCTION
Change detection is the process of identifying differences in the state of an object or phenomenon by observing it at different times [11] . It involves the ability to quantify temporal effects using multi-temporal data sets [15] . Some of the major applications of change detection, in images obtained from earth orbiting satellites, are land-use change analysis, monitoring of shifting cultivation, assessment of deforestation, damage assessment, disaster monitoring and the acquisition of military geographic information. Besides, many other applications can use change detection such as security systems and quality control.
A number of problems can be posed as change detection problems. This study presents an overview and assessment of the conducted digital change detection techniques using remotely sensed data, giving examples of their use in both environmental and military application. It cites also a number of comparative studies. Development of an efficient algorithm for detecting changes in satellite images suitable for all land-use is required. In an effort to meet this requirement, this study was directed toward design, develop and implement an algorithm for spatial change information extraction system based on artificial neural networks.
Many papers, classified from different viewpoint, Among the various ANN models developed and implemented for image classification, the multilayer perceptron is the most widely used network [3,7] have summarized the current methods of using Neural network.
This study proposes a change detection technique of remotely sensed data using neural networks. (Abstract, introduction, results and discussion, conclusion, and references) Neural network model: There are many different types of neural networks; from relatively simple to very complex. Among them feed-forward, multi-layer neural networks which are one of the most popular designs for neural networks. Network learning algorithms play an important role for the neural network in change detection schemes. There is no unique learning algorithm for the design of neural networks. Rather, we have a kit of tool represented by a variety of learning algorithms, each of which offers advantages of its own. Basically, learning algorithms differ from each other in the way in which the adjustment to a synoptic weight of a neuron is formulated using Back-Propagation learning Network (BPN) [2,4] .
The back-propagation learning algorithm is used to train the weights in the network and updating the weights of a multilayered network which undergo supervised training [5] . has developed a technique for adapting the weights and then in 1986, Rumelhart et al. [14] had improved it into neural network [6] . This improvement of weight adaptation rule is known as back-propagation.
Back-propagation algorithm [18] used supervised learning approach due to the fact that in BPN, the target output vectors are defined earlier in the system. The learning process begins with the presentation of an input pattern to the BPN. In which, the net total input is found using the standard summation of products as defined in Eq. 1 below: Where: (W ij ) = The weight associated with the connection between neuron j and I (o i ) = The output of neuron i in output layer, (µ) Learning rate (δj) = Output error from last cycle (α ) = Momentum (n) = Number of cycles The delta rule algorithm [16] will adjust the weights leading to the output units. It is affected by the variation of the η learning rate and α momentum. Each factor can be changed independently of the others to obtain the best results. In some experiments we changed two factors simulations to investigate their effects on the results.
In the Eq. 1, the term α is introduced to add in a proportion of the previous weight change, this is due to reduce the likelihood of the weight changes to oscillate. Therefore, the weight change for pattern n+1 is dependent on the weight change for pattern n. Those steps discussed above were repeated for each pattern defined in the systems. The patterns were continually presented to the network, epoch after epoch. This step is conducted until during one epoch, all outputs for each pattern are within tolerance [16] .
Variable learning rate: The back-propagation error surface usually consists of a large amount of flat regions as well as extremely steep regions (Hush and Horne, 1993), so there is difficulty how to choose an appropriate value of learning rate. As such, the backpropagation algorithm with a variable learning is desirable [1] .
Momentum: This is a modification based on the observation that convergence might be improved if we could smooth out the oscillations in the trajectory. Among the used techniques a low-pass filter shows a promising result [9,10] .

Improvement factor:
The response of the neural network to noise is an important factor in determining its suitability to a given application. A data set used to train a neural network may have random speckles in it. In the process of training, a matrix is applied to the neural network to see how well the network has learned the training data. The performance is compared with and without the adaptation of the Improvement Factor (IF) to investigate its importance [13] .

Proposed scheme:
The proposed scheme is designed to accept any number of digital images (two images in our case) and produce changed image where changed and non-changed areas are assigned to different codes. Different types of changes and non-changes are included as prescribed in the learning phase. The system can be divided by functionality into the following three modules as illustrated in Fig. 1.    Table 1 shows the arrangement construction of entering process assuming the two inputs represented by satellite images and the desired output are set arbitrary to binary codes.
Neural network engine: The network will adapt the matrix of random weights in order to assign the input pattern to the expected output in forward feed-back direction [18] as shown in Fig. 2 and then the set of computed weights are used in forward path to classify the rest of images. These are done in two phases: • Learning phase

• Execute phase
Learning phase: In this stage, the network weights are randomly generated and the training process will try to adjust the weights so that the actual output comes out closer to the expected output [8] according to the following procedures: One of the patterns to be learned will be put on the input units. Values of the output of hidden layers units and output layer units are calculated.
The errors on the output layer units are calculated.
The delta rule algorithm [17] will adjust the weights leading to the output units. The errors on the hidden units are calculated. The weights leading into the hidden layers will be adjusted.
Steps from 1-6 are repeated for all the input patterns.
Steps from 1-7 are repeated until the actual output is closer to desired output.
These procedures of learning are summarized in Fig. 2.
Execution phase: The two images are executed by the network. One pixel from the first image and the corresponding one from the other image at a time and a set of precalculated weights matrix, in training phase, is used to calculate the output values in forward propagation [12,14] .
Change extractor: In this part the values of output neurons for each input vector are compared to each other and the order of neuron with the highest value with respect to the other neurons (winner neuron) represents the output class for the given input vector.

RESULTS
One of the most important aspects of any learning paradigm in neural network is how it scales according to problem size and complexity. To carry out this task with known training error and pre-specified maximum number of training updates, the convergence of the back-propagation algorithm is investigated, for both sample sets of satellite data, with respect to: • Criticality of the initial weight set  A C++ program is implemented to scale the neural network parameters to obtain the best possible results for the data sets. The output of the program is the scaled neural network though adjusting the following parameters: • Variable learning rate: Control the speed of the learning process • Momentum: Increase the learning stability and reduce error • Improvement factor: Increase the quality of the images by smoothing the range of pixel values For each parameter, a careful designing phase was carried out in order to assess the best performances provided by neural network. The trials are summarized as follows: • For the weights parameter, we carried out different trials with different randomized weight values. For the network structure, over 100 different topologies from one to three hidden layers are tested • For both the learning rate and momentum parameters, we carried out different trials over the whole range from 0-0.9 • For the improvement factor parameter, different values were added to the training set The Accuracy (AC) of any class is defined as the ratio between the positively true (pixels classified to be in a class and are truly in that class) and all pixels that are used as ground truth of this class.  (Table 3). Figure 5 shows the difference of the system performance by using IF.

Experiment 2:
The three satellite images (data set 2) are normalized to the range from zero to one as shown in Fig. 6 and 8. There are nine and eleven different classes representing the different types of changed and non-changed areas for the two modules respectively.
No changes from water to water, changed from water to vegetation, changed from water to sabkha, no changes from vegetation to vegetation, changed from vegetation to desert, changed from sabkha to water, changed from sabkha to vegetation, no changes sabkha to sabkha, changed from desert to vegetation, changed from desert to sabkha and no changes from desert to desert (Table 5). Figure 9 shows the difference of the system performance by using IF.

DISCUSSION
From the above it is clear that with the use of this design led to the separation between the changes and put it in a separate layer and specialists to discuss this area in the National Authority for Remote Sensing and Space Sciences and the Ministry of Agriculture acknowledged the credibility of the output of this design .and after comparing the output field trip in 2003, assessed the output of the design by 78%.and after the study recommended that the recommendations of the three: • The hunting process in a very healthy low Toshka and lakes where the fish breed in this region • Agriculture in this region is necessary that the spaces away from the lakes enough • Not be built next to the areas that are leakage of water from which the lakes

CONCLUSION
Using the neural network system for digital change detection shows a great potential compared with other techniques (ratio, classification). Especially, if the two images are not completely radiometricaly balanced and there is no previous information about the value of the threshold, since the other techniques can provide spurious effect on change and non-change classes. Classification of the output can be easily added to the system just by adding additional nodes to the output layer to specify different types of changes. Therefore we can conclude that: • Adding momentum term in learning phase will reduce the time needed and reduced oscillation of output network • Using learning rate optimization will reduce the time and effort needed to select an appropriate value of learning rate • Using improvement factor optimization will reduce superiors data in training phase and improve output results of neural network There is no universally optimal change detection technique. The choice is dependent upon the application. Even the question of, which is best for any particular application, remains generally open.