2013), etc., provide important multi-scale geometric information about image objects to improve the change representation. Recently, deep learning-based CD approaches have shown great potential in extracting more high-level deep features, which represents a popular direction in CD research (Mou et al. 2019; Saha et al. 2019).
Change index construction: the change index represents the temporal variations extracted from multitemporal image features. It can be constructed based on different operators and algorithms, such as univariate image differencing (Bruzzone and Prieto 2000a), change vector analysis (CVA) (Bovolo and Bruzzone 2007b), ratioing (Bazi et al. 2005), distance or similarity measures (Du et al. 2012), etc. Transformation approaches such as iterative reweighted multivariate alteration detection (IR-MAD) (Nielsen 2007), principal components analysis (PCA) and its kernel version (Nielsen and Canty 2008; Celik 2009), independent component analysis (ICA) (Liu et al. 2012), are also designed to transform the change information from the original data space into a projected feature space. However, a careful selection of specific components representing user-interested changes is required. This is often very difficult in an unsupervised CD case without prior knowledge about the considered study area and dataset, which may limit the automation degree of the CD application. For a summary of the related methods for constructing different types of change index, readers can refer to the paper by Bovolo and Bruzzone (2015).
Change detection algorithm design: unlike the supervised and semi-supervised CD methods that rely on the available reference samples, unsupervised CD algorithms focus more on the automation and accuracy. Thus, basically, most of the unsupervised CD approaches are data-driven by analyzing the multitemporal data itself. Within this context, for binary CD, if we consider a given change index generated in the previous step, for example, the magnitude of differencing image, automatic thresholding such as empirical segmentation (Bruzzone and Prieto 2000b), Kittler–Illingworth (KI) (Bazi et al. 2005), Otsu (1979) and Bayesian-based expectation–maximization (EM) (Bruzzone and Prieto 2000a) are all simple but effective algorithms proposed in the literature. However, the successful use of such methods depends on the assumption of a certain data distribution such as Gaussian or Rayleigh–Rice mixture (Bovolo et al. 2012; Zanetti et al. 2015), where a wrong estimation may lead to many detection errors. On the contrary, clustering algorithms such as k-means, fuzzy c-means and Gustafson–Kessel clustering (GKC) have been used to address the binary CD problem (Celik 2009; Ghosh et al. 2011), which are distribution-free but require a specific setting to avoid unstable performance, such as the accuracy decrease due to random initialization.
For a multiclass CD case, the unsupervised task becomes more complex since several sub-problems should be solved simultaneously, including the binary change and no-change separation, the number of multiclass change estimation and the multiclass change discrimination (Liu et al. 2019c). In particular, among many solutions, we recall the classical multiple CD technique – change vector analysis (CVA) (Malila 1980). It was designed to analyze possible multiple changes in pairs of bitemporal image bands. A theoretical definition was given to the original CVA approach in the polar domain to provide a more clear mathematical explanation to CVA (Bovolo and Bruzzone 2007b). However, it still has a limitation, i.e. only a part of all possible changes can be detected since only two selected bands are considered in each implementation. If more spectral channels are considered, it becomes very difficult to simultaneously model and visualize multidimensional changes. To break this constraint, a compressed change vector analysis (C2VA) approach was proposed, which successfully extended the original CVA to a two-dimensional (2D) representation of the multi-band problem (Bovolo et al. 2012). Other works in the literature developed different variations of CVA. For example, a modified CVA was developed to determine the magnitude threshold and direction by combining single-date image classification results (Chen et al. 2003). An improved thresholding approach on change magnitude was designed to optimize the binary separation on each specific change class (Bovolo and Bruzzone 2011). A hierarchical version of C2VA with an adaptive and sequential projection of spectral change vectors (SCVs) at each level of the hierarchy was proposed to detect multiple changes in bitemporal hyperspectral images (Liu et al. 2015). In this chapter, we also explore the potential capability of C2VA and extend it from the spectral–spatial point of view.
Performance evaluation: similar to the supervised CD methods, unsupervised binary and multiclass CD approaches can usually be assessed according to the detection accuracy or error index, such as overall accuracy (OA), Kappa coefficient (Kappa), omission errors (OE) and commission errors (CE), receiver operating characteristic (ROC) curve and area under the curve (AUC) value. In this case, the accuracy evaluation usually relies on the manually interpreted change reference map. Note that such a reference map is only used for accuracy evaluation, which is not considered as training data as in the supervised case. In addition, the computational time cost is also another important indicator that reflects the automation and efficiency of unsupervised methods.
1.2.2. Open issues and challenges
The current development of unsupervised CD techniques for multispectral remote sensing images has had great success in many practical applications. However, there are still open issues and challenges that deserve to be further analyzed, which include but are not limited to the following:
1 1) a high-precision multitemporal pre-processing procedure, for example, co-registration techniques;
2 2) multitemporal data quality improvement due to bad imaging conditions, such as system noise, cloud contamination and seasonal spectral variations;
3 3) advanced techniques for correctly estimating the real number of multiclass changes in image scenarios;
4 4) spectral–spatial modeling of change targets to enhance the original pixel-wise spectral representation;
5 5) robust and efficient CD approach in an unsupervised fashion, especially for a large complex CD scene;
6 6) change feature representation by taking advantage of both machine learning and deep learning techniques.
1.2.3. Spectral–spatial unsupervised CD techniques
Despite the success of aforementioned CD methods, especially the CVA-based methods, they mainly focus on the spectral changes in each individual pixel or a local neighborhood (Bovolo 2009; Bovolo et al. 2012; Liu et al. 2015). The geometrical characteristics of change targets are not fully modeled and preserved. This may increase the ambiguity due to abnormal spectral variations in isolated pixels and errors (e.g. co-registration errors), leading to the presence of omission and commission errors, especially when dealing with VHR images. In this case, traditional pixel-based CD methods may lose their effectiveness since they were developed under the assumption that pixels are spatially independent. However, for multispectral images in complex urban scenarios, challenging issues may arise due to the limited spectral representation; thus, the same class of objects may have different spectra, or different objects may have the same or very similar spectra. This may significantly increase the detection difficulty, especially when considering the multiclass CD task.
To address the above problems in pixel-based CD (PBCD) techniques, spectral–spatial joint analysis and object-based CD (OBCD) methods are mainstream techniques proposed in the literature. For the former, morphological filters (i.e. self-dual reconstruction filters and alternating sequential filters) were combined with CVA for binary CD in VHR images (Mura et al. 2008). However, a sliding window (i.e. structuring element (SE)) for filtering should be fixed at a given level; thus, it is not robust for multilevel implementation. Morphological attribute profiles (APs) were applied to extract structure-related geometrical features within the scene from each date of panchromatic images (Falco et al. 2013). It includes a multilevel extraction of connected regions in the scene at different scales. Building change information based on the difference in the multitemporal morphological building index (MBI) at the feature and decision level was considered for detecting building changes in VHR images (Huang et al. 2014). A spectral–spatial band