(Linying Zhao & Shunping Ji, CNN, RNN, or ViT? An evaluation of different deep learning architectures for spatio-temporal representation of Sentinel time series, JSTARS, 2022)
The MTS12 dataset named as is composed of optical Sentinel-2 and SAR Sentinel-1 time series from January to December 2019 covering the whole Slovenia, an area of 20,271 square kilometers, for land cover classification. The whole dataset included 936 patches, each with the size of 500×500 pixels and the temporal dimension of 12 (one per month), consisting of eight land-cover classes, which are cultivated land, forest, grassland, shrubland, water, wetland, artificial surface, and bare land.
Fig. 1. Land-cover map of Slovenia.
The S2 images were obtained from the Slovenia 2019 Land Cover Classification Dataset [1], which are level L1C images, with a total of six surface reflectance bands (2, 3, 4, 8, 11, and 12), which have been resampled at 10 m. We manually filtered out the images with cloud contamination, resulting in 12 clean (one per month) S2 images. Then, according to the date of the selected S2 images, 12 corresponding S1 images were further chosen and downloaded.
All the S1 images were in Interferometric Wide (IW) Ground Range Detected (GRD) mode with dual polarization (VV+VH) at a 10-m resolution, which we downloaded from the Sentinel Hub [2]. These images were preprocessed with calibrating to the beta coefficient, removing thermal noise, and orthorectification by the Copernicus digital elevation model (DEM). All the acquired S1 images were converted to the logarithmic dB scale and normalized to values between 0 and 255. We also applied non-local means filtering to the S1 images to remove speckle noise, and then the Z-score normalization method to normalize the S1 and S2 data.
The corresponding ground truth for the land cover was obtained from the Slovenia 2019 Land Cover Classification Dataset, at a 10-m spatial resolution. The labels included 936 patches, each with the size of 500×500 pixels, consisting of eight land-cover classes, which are cultivated land, forest, grassland, shrubland, water, wetland, artificial surface, and bare land.
The dataset, i.e., the images and the geo-aligned labels, was split into three parts according to the spatial distribution. In total, 297 patches from the south of the country were used for the testing, 130 patches are randomly selected from the north of the country as the validation set, and the remaining 509 patches from the north of the country were used for the training.
You can open the data with the following command:
import scipy.io as scio
'''
The size of label is 500*500. Class definitions: 0 represents no data, 1 represents cultivated land, 2 represents forest, 3 represents grassland, 4 represents shrubland, 5 represents water, 6 represents wetland, 7 represents artificial surface, and 8 represents bare land.
The format of Sentinel_2 and Sentinel_1 is Temporal * Height * Width * Channel
(here Temporal = 12; height = width = 500; channel number is 6 for S2 and 2 for S1)
'''
label = scio.loadmat('download_path/eopatch_id_0_col_0_row_19_label.mat')['label']
Sentinel_2_data = scio.loadmat('download_path/eopatch_id_0_col_0_row_19.mat')['band']
Sentinel_1_data = scio.loadmat('download_path/eopatch_id_0_col_0_row_19.mat')['band']
'''
If one needs to pad the size of label up to 512*512, use:
import numpy as np
label_pad = np.pad(label,((6,6)),'constant')
Sentinel_2_pad = np.pad(Sentinel_2_data,((0,0),(0,0),(6,6),(6,6)),'constant')
Sentinel_1_pad = np.pad(Sentinel_1_data,((0,0),(0,0),(6,6),(6,6)),'constant')
'''
We use the same naming rule as in [1]，including id numbers, row and column numbers, as shown in Fig 2. According to Fig. 2, we can then merge these patches into a complete image using the GDAL library, if required.
Fig. 2. Naming rule of patches
[1] “Example dataset of EOPatches for Slovenia 2019,” 2019. [Online]. Available: http://eo-learn.sentinel-hub.com/
[2] “Sentinel-hub.” [Online]. Available: https://www.sentinel-hub.com/