Final published version
Licence: CC BY: Creative Commons Attribution 4.0 International License
Research output: Contribution to Journal/Magazine › Journal article › peer-review
Research output: Contribution to Journal/Magazine › Journal article › peer-review
}
TY - JOUR
T1 - Spatio-temporal multi-level attention crop mapping method using time-series SAR imagery
AU - Han, Zhu
AU - Zhang, Ce
AU - Gao, Lianru
AU - Zeng, Zhiqiang
AU - Zhang, Bing
AU - Atkinson, Peter M.
PY - 2023/12/31
Y1 - 2023/12/31
N2 - Accurate crop mapping is of great significance for crop yield forecasting, agricultural productivity development and agricultural management. Thanks to its all-time and all-weather capability, integrating multi-temporal synthetic aperture radar (SAR) for crop mapping has become essential and challenging task in remote sensing. In recent years, deep learning (DL) has demonstrated excellent crop mapping accuracy to interpret crop dynamics. However, existing DL-based methods tend to be incapable of capturing spatial and temporal features at different scales simultaneously, and this often leads to severe mis-classification due to the complex and heterogeneous distribution of crops and diverse phenological patterns. In this paper, we propose a novel spatio-temporal multi-level attention method, named as STMA, for crop mapping using time-series SAR imagery in an end-to-end fashion to increase the capability of crop phenology retrieval. Specifically, the multi-level attention mechanism is designed to aggregate multi-scale spatio-temporal representations on crops via cascaded spatio-temporal self-attention (STSA) and multi-scale cross-attention (MCA) modalities. To ensure a fine extraction of multi-granularity features, a learnable spatial attention position encoding is proposed to adaptively generate the position priors to facilitate multi-level attention learning. Experimental results on Brandenburg Sentinel-1 dataset, public PASTIS-R dataset and South Africa dataset demonstrated that STMA can achieve state-of-the-art performance in crop mapping tasks, with the accuracy of 96.54% in the Brandenburg Sentinel-1 dataset, 86.77% in the PASTIS-R dataset and 83.37% in the South Africa dataset, validating its effectiveness and superiority. Further comparison of spatio-temporal generalization capability reflected its excellent performance in spatio-temporal modeling on different crops and scenarios. This research provides a viable and intelligent spatio-temporal framework for large-area crop mapping using time-series SAR imagery in complex agricultural systems. The Brandenburg Sentinel-1 dataset and the STMA code will be publicly available at https://github.com/hanzhu97702/ISPRS_STMA.
AB - Accurate crop mapping is of great significance for crop yield forecasting, agricultural productivity development and agricultural management. Thanks to its all-time and all-weather capability, integrating multi-temporal synthetic aperture radar (SAR) for crop mapping has become essential and challenging task in remote sensing. In recent years, deep learning (DL) has demonstrated excellent crop mapping accuracy to interpret crop dynamics. However, existing DL-based methods tend to be incapable of capturing spatial and temporal features at different scales simultaneously, and this often leads to severe mis-classification due to the complex and heterogeneous distribution of crops and diverse phenological patterns. In this paper, we propose a novel spatio-temporal multi-level attention method, named as STMA, for crop mapping using time-series SAR imagery in an end-to-end fashion to increase the capability of crop phenology retrieval. Specifically, the multi-level attention mechanism is designed to aggregate multi-scale spatio-temporal representations on crops via cascaded spatio-temporal self-attention (STSA) and multi-scale cross-attention (MCA) modalities. To ensure a fine extraction of multi-granularity features, a learnable spatial attention position encoding is proposed to adaptively generate the position priors to facilitate multi-level attention learning. Experimental results on Brandenburg Sentinel-1 dataset, public PASTIS-R dataset and South Africa dataset demonstrated that STMA can achieve state-of-the-art performance in crop mapping tasks, with the accuracy of 96.54% in the Brandenburg Sentinel-1 dataset, 86.77% in the PASTIS-R dataset and 83.37% in the South Africa dataset, validating its effectiveness and superiority. Further comparison of spatio-temporal generalization capability reflected its excellent performance in spatio-temporal modeling on different crops and scenarios. This research provides a viable and intelligent spatio-temporal framework for large-area crop mapping using time-series SAR imagery in complex agricultural systems. The Brandenburg Sentinel-1 dataset and the STMA code will be publicly available at https://github.com/hanzhu97702/ISPRS_STMA.
KW - Computers in Earth Sciences
KW - Computer Science Applications
KW - Engineering (miscellaneous)
KW - Atomic and Molecular Physics, and Optics
U2 - 10.1016/j.isprsjprs.2023.11.016
DO - 10.1016/j.isprsjprs.2023.11.016
M3 - Journal article
VL - 206
SP - 293
EP - 310
JO - ISPRS Journal of Photogrammetry and Remote Sensing
JF - ISPRS Journal of Photogrammetry and Remote Sensing
SN - 0924-2716
ER -