Learning discriminative spatiotemporal features for precise crop classification from multi-temporal satellite images

Abstract

Precise crop classification from multi-temporal remote sensing images has important applications such as yield estimation and food transportation planning. However, the mainstream convolutional neural networks based on 2D convolution collapse the time series information. In this study, a 3D fully convolutional neural network (FCN) embedded with a global pooling module and channel attention modules is proposed to extract discriminative spatiotemporal presentations of different types of crops from multi-temporal high-resolution satellite images. Firstly, a novel 3D FCN structure is introduced to replace 2D FCNs as well as to improve current 3D convolutional neural networks (CNNs) by providing a mean to learn distinctive spatiotemporal representations of each crop type from the reshaped multi-temporal images. Secondly, to strengthen the learning significance of the spatiotemporal representations, our approach includes 3D channel attention modules, which regulate the between-channel consistency of the features from the encoder and the decoder, and a 3D global pooling module, which selects the most distinctive features at the top of the encoder. Experiments were conducted using two data sets with different types of crops and time spans. Our results show that our method outperformed in both accuracy and efficiency, several mainstream 2D FCNs as well as a recent 3D CNN designed for crop classification. The experimental data and source code are made openly available at http://study.rsgis.whu.edu.cn/pages/download/.