TY - GEN

T1 - An Interpretable Deep Semantic Segmentation Method for Earth Observation

AU - Zhang, Ziyang

AU - Angelov, Plamen

AU - Almeida Soares, Eduardo

AU - Longepe, Nicolas

AU - Mathieu, Pierre Philippe

PY - 2023/1/25

Y1 - 2023/1/25

N2 - Earth observation is fundamental for a range of human activities including flood response as it offers vital information to decision makers. Semantic segmentation plays a key role in mapping the raw hyper-spectral data coming from the satellites into a human understandable form assigning class labels to each pixel. Traditionally, water index based methods have been used for detecting water pixels. More recently, deep learning techniques such as U-Net started to gain attention offering significantly higher accuracy. However, the latter are hard to interpret by humans and use dozens of millions of abstract parameters that are not directly related to the physical nature of the problem being modelled. They are also labelled data and computational power hungry. At the same time, data transmission capability on small nanosatellites is limited in terms of power and bandwidth yet constellations of such small, nanosatellites are preferable, because they reduce the revisit time in disaster areas from days to hours. Therefore, being able to achieve as highly accurate models as deep learning (e.g. U-Net) or even more, to surpass them in terms of accuracy, but without the need to rely on huge amounts of labelled training data, computational power, abstract coefficients offers potentially game-changing capabilities for EO (Earth observation) and flood detection, in particular. In this paper, we introduce a prototype-based interpretable deep semantic segmentation (IDSS) method, which is highly accurate as well as interpretable. Its parameters are in orders of magnitude less than the number of parameters used by deep networks such as U-Net and are clearly interpretable by humans. The proposed here IDSS offers a transparent structure that allows users to inspect and audit the algorithm's decision. Results have demonstrated that IDSS could surpass other algorithms, including U-Net, in terms of IoU (Intersection over Union) total water and Recall total water. We used WorldFloods data set for our experiments and plan to use the semantic segmentation results combined with masks for permanent water to detect flood events.

AB - Earth observation is fundamental for a range of human activities including flood response as it offers vital information to decision makers. Semantic segmentation plays a key role in mapping the raw hyper-spectral data coming from the satellites into a human understandable form assigning class labels to each pixel. Traditionally, water index based methods have been used for detecting water pixels. More recently, deep learning techniques such as U-Net started to gain attention offering significantly higher accuracy. However, the latter are hard to interpret by humans and use dozens of millions of abstract parameters that are not directly related to the physical nature of the problem being modelled. They are also labelled data and computational power hungry. At the same time, data transmission capability on small nanosatellites is limited in terms of power and bandwidth yet constellations of such small, nanosatellites are preferable, because they reduce the revisit time in disaster areas from days to hours. Therefore, being able to achieve as highly accurate models as deep learning (e.g. U-Net) or even more, to surpass them in terms of accuracy, but without the need to rely on huge amounts of labelled training data, computational power, abstract coefficients offers potentially game-changing capabilities for EO (Earth observation) and flood detection, in particular. In this paper, we introduce a prototype-based interpretable deep semantic segmentation (IDSS) method, which is highly accurate as well as interpretable. Its parameters are in orders of magnitude less than the number of parameters used by deep networks such as U-Net and are clearly interpretable by humans. The proposed here IDSS offers a transparent structure that allows users to inspect and audit the algorithm's decision. Results have demonstrated that IDSS could surpass other algorithms, including U-Net, in terms of IoU (Intersection over Union) total water and Recall total water. We used WorldFloods data set for our experiments and plan to use the semantic segmentation results combined with masks for permanent water to detect flood events.

KW - Earth observation

KW - semantic segmentation

KW - flood detection

KW - interpretable deep learning

KW - prototype-based classifiers

KW - U-Net

KW - WorldFloods

U2 - 10.1109/IS57118.2022.10019621

DO - 10.1109/IS57118.2022.10019621

M3 - Conference contribution/Paper

SN - 9781665492768

T3 - 2022 IEEE 11th International Conference on Intelligent Systems (IS)

SP - 1

EP - 8

BT - 2022 IEEE 11th International Conference on Intelligent Systems (IS)

A2 - Atanassov, Krassimir T.

A2 - Doukovska, Lyubka

A2 - Kacprzyk, Janusz

A2 - Krawczak, Maciej

A2 - Owsinski, Jan W.

A2 - Sgurev, Vassil

A2 - Szmidt, Eulalia

A2 - Zadrozny, Slawomir

PB - IEEE

ER -