Modified Self-Organizing Feature Map Neural Network with Semi-supervision for Change Detection in Remotely Sensed Images (Image Analysis)

Abstract

Problem of change detection of remotely sensed images using insufficient labeled patterns is the main topic of present work. Here, semi-supervised learning is integrated with an unsupervised context-sensitive change detection technique based on modified self-organizing feature map (MSOFM) network. In this method, training of the MSOFM is performed iteratively using unlabeled patterns along with a few labeled patterns. A method has been suggested to select unlabeled patterns for training. To check the effectiveness of the proposed methodology, experiments are carried out on two multitemporal remotely sensed images. Results are found to be encouraging.

Keywords: Semi-supervised learning, change detection, fuzzy set, self-organizing feature map.

Introduction

Change detection, a process of finding out changes in a land cover over time by analyzing remotely sensed images of a geographical area captured at different times [9], can be viewed as an image segmentation problem, where two groups of pixels (changed and unchanged) are formed. Between the two traditional approaches (supervised and unsupervised), the applicability of supervised method [1] is less for insufficient ground truth information. Whereas, ground truth is not necessary for unsupervised methods [1,5]. Due to the depletion of labeled patterns, unsupervised techniques seem to be compulsory here. The three consecutive steps used for unsupervised technique are image preprocessing, image comparison and image analysis [9]. In preprocessing step, images are made compatible by operations like radiometric and geometric corrections, co-registration etc. Then, image comparison is done pixel by pixel to generate a difference image (DI). A situation may occur where the categorical information of a few labeled patterns can be collected easily by experts. If the number of these labeled patterns is less, then this information may not be sufficient for developing supervised methods. In such a scenario, knowledge of labeled patterns, though very poor, will be unutilized if unsupervised approach is taken. Under this circumstance, semi-supervised approach [2] could be opted instead of unsupervised or supervised one.

In the present work, a neural network based semi-supervised change detection method is proposed. Here, the idea of semi-supervised learning [2] is incorporated into the existing unsupervised change detection technique based on the MSOFM network [6]. This work is inspired from a work on change detection using semi-supervised multilayer perceptron (MLP) [8]. The MSOFM network [4,6] has two layers: input and output. For each feature of the input pattern, there is a neuron in the input layer. The output layer is two dimensional and there is a representative neuron for each pixel position of DI. In the present work, change vector analysis (CVA) technique [1,9] is used for generating DI. The input pattern for a particular pixel position is generated using gray value of the said pixel as well as those of its neighborhood pixels for considering (spatial) contextual information from neighbors. In the present work 2nd order neighborhood is used. A mapping algorithm [4,6] is used to normalize the feature values of the input pattern in [0,1]. Each neuron in the output layer is connected to all the neurons in the input layer. The connection weights of the representative neuron of the labeled pixel position are initialized with the normalized feature values of the corresponding labeled pattern; whereas the weight vector for others is initialized randomly between [0,1]. Iterative learning of the MSOFM network is performed by the labeled patterns along with the selected unlabeled patterns until a given convergence criterion is satisfied.

Kohonen’s Self-Organizing Feature Map

In the proposed algorithm, we have used the same MSOFM network as used in [4,6]. The MSOFM network is motivated from the Kohonen’s self-organizing feature maps (SOFM) [7] and has similar network structure as that of the SOFM. SOFM is learned iteratively and generates topological map of input patterns gradually. The input pattern is represented bydenotes the synaptic weight vector ofoutput neuron (where j = 1, 2, …,z and z be the number of neurons in the output layer). The winner neuron j in the output layer is selected as the best-matched neuron for the given input pattern where the similarity measureis maximum. Let, h(itr) be the topological neighborhood function at iteration itr that shrinks after each iteration. The weight updating of winning neuron is done as,

where,denotes the learning rate in iteration itr and decreases with increase of itr.

The Proposed Algorithm

In the present technique, labeled patterns from both the classes are picked up randomly from the ground truth. Let, the y-dimensional input pattern of pixel position of DI is denoted byThe

y-dimensional weight vector betweenneuron in the output layer and all the neurons of the input layer is represented by

During training, input patternsare passed to the network one by one. Each time, the dot product d(m, n) betweenis calculated as,

At the beginning of the training phase, the connection weights of the MSOFM network are updated in the following manner using the labeled patterns only. If the class label of thepixel position is known, then the weight vector for all neighboring neurons (defined byoutput neuron is updated using equationinput pattern is unlabeled. For the labeled patterns, the weight vector is not updated during the entire learning process. Learning using labeled patterns is continued iteratively until convergence. To check convergence,is calculated at each iteration using,

where, t is a predefined threshold value [6]. The weight updating is preformed untilwhere 6 is a small positive quantity. The value of t varies within [0,1]. The components of the weight vectorare normalized in a way so that the dot product d(m, n) lies between [0,1]. After each epoch, the learning rateand the size of the topological neighborhooddecrease.

After each training step, the soft class labels of unlabeled patterns are calculated by presenting them to the network. Let us consider that there exists two fuzzy sets: one for the changed class and the other for unchanged one. For each unlabeled pattern,is computed by equation (2); if,, then

pattern is more likely to belong to changed class than unchanged one; else in unchanged class. Let,be the membership value of

unlabeled pattern, whereare the membership values in unchanged class and changed class, respectively. The values can be calculated as,

After that, the target value of the unlabeled pattern is updated by ^-nearest neighbor technique as it was used in [8]. To search for ^-nearest neighbors, instead of considering all the patterns, we consider only those which lie within a window around that unlabeled pattern. Let, M be the set of ^-nearest neighbors of (i,j)th unlabeled pattern. Now, target value unlabeled pattern is estimated as,

Then, the unlabeled patterns which are more likely to belong to the changed class, are selected for training the MSOFM network again. The process of training the MSOFM using labeled patterns and the selected unlabeled patterns continued until convergence. The training of network and re-estimation of soft class labels of unlabeled patterns using equations (4) and (5) are continued iteratively (for DI of size p x q) by computing the sum of square error, £(N), after each Nth training step as,

Results and Analysis

To evaluate the effectiveness of the proposed method, experiments are carried out on two multitemporal remotely sensed images corresponding to the geographical areas of Mexico and Sardinia island. Similar findings are obtained for both the data sets. For illustration, results of Mexico data sets are presented here. The performance of the proposed technique is compared with those of the existing unsupervised technique using MSOFM [6] and a supervised method using MLP. The performance indices are the number of missed alarms (changed class pixels identified as unchanged ones-MA), the number of false alarms (unchanged class pixels classified as changed ones-FA), the number of overall errors (OE) and Kappa measure (KM) [3]. The average (Avg.) and standard deviation (written in brackets in the table) values (over 10 simulations) of overall errors and Kappa measure are considered for comparison. The best results (denoted by ‘Min’ in table) and the worst results (denoted by ‘Max’ in table) for MA, FA and OE, considering all the simulations, are also given in the table.

The data set of Mexico is made of two multispectral images over an area of Mexico [6,8] taken on April 18, 2000 and May 20, 2002 (Fig. 1). The corresponding difference image and the reference map are shown in Figure 1. The change detection maps corresponding to minimum overall error (over 10 simulations) obtained using optimal threshold with the unsupervised technique and the proposed semi-supervised technique (using 0.1% patterns from the changed class and 0.1% from the unchanged class) are depicted in Figure 2.

Table 1 presents the comparative results obtained using the unsupervised method, the supervised method and the proposed semi-supervised method. From the table, it is noticed that the proposed method (considering all the cases of percentage of training patterns) outperforms the corresponding unsupervised version. It is also found that the standard deviations obtained for both overall error and Kappa measure using MLP based supervised technique are very high as compared to the corresponding unsupervised and semi-supervised approaches. This may be due to the unavailability of sufficient number of training samples which reflect the real life scenario. By comparing the results of the supervised and the proposed semi-supervised methods, it is seen that the maximum overall error (denoted by ‘Max’) obtained by the semi-supervised approach is less than that of the corresponding supervised method. In short, the results show that the proposed semi-supervised version has an edge over the unsupervised as well as the supervised techniques when a few labeled patterns are available.

Fig. 1. Images of Mexico area. (a) Band 4 image acquired in April 2000, (b) band 4 image acquired in May 2002, (c) corresponding difference image, and (d) a reference map of the changed area.

Fig. 2. Change detection maps obtained for Mexico data set: (a) using the unsupervised method based on optimal threshold, and (b) using the proposed semi-supervised technique based on optimal threshold (with 0.1% training pattern) 

Table 1. Results and comparison

		Training
Techniques	Threshold	patterns	Max/	MA	FA	OE	Avg .OE	Avg.KM
used		used	Min
Unsupervised	0.216		Min	1366	1618	2984	2991.70	0.935532
(based on MSOFM)	(optimal)	-	Max	1366	1634	3000	(4.360046)	(0.000089)
Supervised		0.1%	Min	1345	1429	2774	3086.40	0.927558
(based	-		Max	2268	1139	3407	(186.277857)	(0.004676)
on MLP)		0.5%	Min	1269	1406	2675	2834.3	0.915087
			Max	875	2192	3067	(95.736148)	(0.001731)
Proposed	0.216	0.1%	Min	1512	1218	2730	2741.40	0.940312
semi-supervised	(optimal)		Max	1516	1239	2755	(6.666333)	(0.000141)
(based		0.5%	Min	1470	1235	2705	2723.70	0.940475
on MSOFM)			Max	1502	1232	2734	(8.331266)	(0.000193)

Conclusion

In this paper, an attempt has been made to improve the performance of change detection of remotely sensed images under the scarcity of sufficient number of labeled patterns by exploring the self-organizing capacity of Kohonen’s network integrated with semi-supervision. From the results, it has been found that the proposed algorithm is better suited for change detection where a small amount of labeled patterns is available.