Superpixel-based time-series reconstruction for optical images incorporating SAR data using autoencoder networks. Issue 8 (16th November 2020)

• Record Type: Journal Article
• Authors: Zhou, Ya'nan; Yang, Xianzeng; Feng, Li; Wu, Wei; Wu, Tianjun; Luo, Jiancheng; Zhou, Xiaocheng; Zhang, Xin
• Is Part Of: GIScience & remote sensing. Volume 57:Issue 8(2020)
• Page Start: 1005
• Page End: 1025
• Publication Date: 2020-11-16
• Subjects: Time-series reconstruction -- cloud and shadow -- Landsat image -- Sentinel-1 SAR -- deep learning -- superpixel; Geodesy -- Periodicals; Cartography -- Periodicals; Aerial photogrammetry -- Periodicals; Remote sensing -- Periodicals; 526.05
• DOI: 10.1080/15481603.2020.1841459
• Languages: English
• ISSNs: 1548-1603

Abstract

ABSTRACT: Time-series reconstruction for cloud/shadow-covered optical satellite images has great significance for enhancing the data availability and temporal change analysis. In this study, we proposed a superpixel-based prediction transformation-fusion (SPTF) time-series reconstruction method for cloud/shadow-covered optical images. Central to this approach is the incorporation between intrinsic tendency from multi-temporal optical images and sequential transformation information from synthetic aperture radar (SAR) data, through autoencoder networks (AE). First, a modified superpixel algorithm was applied on multi-temporal optical images with their manually delineated cloud/shadow masks to generate superpixels. Second, multi-temporal optical images and SAR data were overlaid onto superpixels to produce superpixel-wise time-series curves with missing values. Third, these superpixel-wise time series were clustered by an AE-LSTM (long short-term memory) unsupervised method into multiple clusters (searching similar superpixels). Four, for each superpixel-wise cluster, a prediction-transformation-based reconstruction model was established to restore missing values in optical time series. Finally, reconstructed data were merged with cloud-free regions to produce cloud-free time-series images. The proposed method was verified on two datasets of multi-temporal cloud/shadow-covered Landsat OLI images and Sentinel-1A SAR data. The reconstruction results, showing an improvement of greater than 20% in normalized mean square error compared to three state-of-the-art methods (including a spatially and temporally weighted regression method, a spectral–temporal patch-based method, and a patch-based contextualized AE method), demonstrated the effectiveness of the proposed method in time-series reconstruction for multi-temporal optical images.