Guido Pasquariello, Annarita D’Addabbo
Institute for Electromagnetic Sensing of the Environment, National Research Council of Italy
(CNR-IREA), Bari (Italy)
Research theme aims
When the spatial resolution of remote-sensing images becomes very high, the assumption of pixel independence does not hold, thus it is necessary to use and model the spatial context information. Following this idea, some Change Detection algorithms have been recently proposed for Very High Resolution (VHR) images, which exploit the spatial context of a pixel [1-3]. On the other hand, Deep Learning (DL) based framework is suitable to extract high-level features that are effective in capturing information about objects in an image.
Most popular DL architectures are based on a Convolutional Neural Networks (CNN), a multilayer architecture with a huge number of parameters to be learned in the training phase, that is very time consuming and it needs a lot of labelled examples. In order to overcome this problem, a solution is to adopt a transfer learning approach, based on the use of a pre-trained CNN (i.e. a CNN trained on a very large “standard” image data base).
The proposed approach to the Change Detection problem for two VHR images I1, I2, is summarized in the following scheme:
Figure 1: Scheme of changes detection (CD) proposed
Different CNN models, such as AlexNet, VGG Networks and ResNet , have been considered. All the experimental tests have been performed by using AlexNet, a CNN created in 2012 by Krizhevsky et al. (2012) . The input to AlexNet is an RGB image of size 227×227 pixel. In particular, we are interested in the Convolutional Layers (in AlexNet there are 5 convolutional layers), which are used to compute deep features for each VHR image.
Featured map analysis and change detection
In this experiment we used only the first convolution layer conv1 (96 kernels 11x11x3 with stride 4). The Features maps Fi, i=1,2, were obtained by convolution of original image Ii with conv1:
Where Fi is a 96-bands image. In order to appreciate the change at the position (x,y), due to the feature K, we introduced: the difference DFk(x,y):
and the Normalized Difference measured with respect to the feature K:
are the sample mean and standard deviation for the variables DFk(x,y), computed considering all the positions (x,y) in the images. In a scene where the majority of locations does not change, the probability density function of NDFk can be assumed as a normalized Gaussian distribution with 0 mean and standard deviation 1. In this case, at a given interval of confidence, we can assume that at position (x,y), with respect to the feature K, there is a Change if the absolute of NDFk is greater than a threshold “th”. Given a threshold value “th”, we introduced the overall Change Index CI(x,y,th) referred to a position (x,y), as the sum over all the “nk” features (in our case nk=96):
Data, Results and Discussion
The dataset used for the experiment consisted of a couple of VHR aerial RGB images from LeicaADS80, orthorectified and coregistered at 50 cm resolution, acquired respectively in July 2015 (I1) and July 2017 (I2) over the Fiumicino area.
Test area 1: urban area
Applying the methods described above we obtained the CI map for the urban area (Fig.2). In this map CI values are in the range [0,4] and higher values di CI correspond to higher probability of Change. The percentage of pixel with CI=0. (NOT Probable Change) is more than 96.2 %; pixels with CI=1 (LOW Probability of Change) are 3.1%. The remaining pixels are candidate to represent Probable Change (CI=2 0.5%; CI=3 0.2 %; CI=4 0.1%). The validation of the CI-map for Probable Change pixels (CI ≥ 2) was based on visual inspection, on a qualitative base.
Figure 2: CI-map: Blue: CI=1; Green: CI=2; Red: CI=3, White: CI=4.
Test area 2: coastal area
For a more quantitative analysis, we chose to apply the proposed approach to a larger area, at the interface between land and sea (coastal area). In Fig. 3A and 3B, the true color image at time T1 (July 2015) and time T2 (July 2017) are shown, respectively. In figure 3C, the corresponding change map (CI) is shown. In the change map, CI values are in the range [0,4], and higher values of CI correspond to higher probability of Change. The percentage of pixel with CI=0 (NO Change) is more than 96.7 %; pixels with CI=1 (LOW Probability of Change) are 2.3 %. The pixels with CI ≥ 1 are candidates to represent Probable Change. On the CI map, we selected 22 verification points (VP), corresponding to location of connected pixels of Probable Change (CI ≥ 1). The validation was based on the visual interpretation of windows centred on a corresponding VP. 20 samples seem to correspond to real changes on the ground, while 2 seem to correspond to differences created by lighting conditions.
Figure 3: A: coastal test area at time T1 (July 2015); B: coastal test area at time T2 (July 2017); C: Change map (from 2015 to 2017). CI-map: Blue: CI=1; Green: CI=2; Red: CI=3, White: CI=4. ⊕: position of verification points.
Further studies will be conducted by considering other DL pre-trained architectures and different methodologies to compare deep features.
Publications and Presentations
- A.D’Addabbo, G. Pasquariello, A.Amodio, “Urban change detection from VHR images via deep-features exploitation”, ICICT 2021, London, 25-26 February 2021.
- Thonfeld, F., H. Feilhauer, M. Braun, G. Menz, (2016). Robust change vector analysis (RCVA) for multi-sensor very high-resolution optical satellite data. Int. J. Appl. Earth Observ. Geoinf., 50, 131-140.
- Bovolo, F. (2009). A multilevel parcel-based approach to change detection in very high resolution multi-temporal images. IEEE Geosci. Remote Sens. Lett., 6(1), 33-37.
- Falco, N., Dalla Mura, M., Bovolo, F., Benediktsson, J.A., Bruzzone, L. (2012). Change detection in VHR images based on morphological attribute profiles. IEEE Geoscience and Remote Sensing Letters, 10(3), 636-640.
- Zhu X.X., D. Tuia, L. Mou, G. Song Xia, L. Zhang, F. Xu, F. Fraundorfer, (2017). Deep Learning in Remote Sensing, IEEE Geosci and Remote Sensing Magazine, 8-36.
- Krizhevsky, A., I. Sutskever, G. Hinton, (2012). Imagenet classification with deep convolutional neural networks. in Proc. Advances in Neural Information Processing Systems (NIPS). 1097-1115