TY - JOUR

T1 - Multi-Feature Collaborative Fusion Network with Deep Supervision for SAR Ship Classification

AU - Zheng, Hao

AU - Hu, Zhigang

AU - Yang, Liu

AU - Xu, Aikun

AU - Zheng, Meiguang

AU - Zhang, Ce

AU - Li, Keqin

PY - 2023/12/31

Y1 - 2023/12/31

N2 - Multifeature synthetic aperture radar (SAR) ship classification aims to build models that can process, correlate, and fuse information from both handcrafted and deep features. Although handcrafted features provide rich expert knowledge, current fusion methods inadequately explore the relatively significant role of handcrafted features in conjunction with deep features, the imbalances in feature contributions, and the cooperative ways in which features learn. In this article, we propose a novel multifeature collaborative fusion network with deep supervision (MFCFNet) to effectively fuse handcrafted features and deep features for SAR ship classification tasks. Specifically, our framework mainly includes two types of feature extraction branches, a knowledge supervision and collaboration module (KSCM) and a feature fusion and contribution assignment module (FFCA). The former module improves the quality of the feature maps learned by each branch through auxiliary feature supervision and introduces a synergy loss to facilitate the interaction of information between deep features and handcrafted features. The latter module utilizes an attention mechanism to adaptively balance the importance among various features and assign the corresponding feature contributions to the total loss function based on the generated feature weights. We conducted extensive experimental and ablation studies on two public datasets, OpenSARShip-1.0 and FUSAR-Ship, and the results show that MFCFNet is effective and outperforms single deep feature and multifeature models based on previous internal FC layer and terminal FC layer fusion. Furthermore, our proposed MFCFNet exhibits better performance than the current state-of-the-art methods.

AB - Multifeature synthetic aperture radar (SAR) ship classification aims to build models that can process, correlate, and fuse information from both handcrafted and deep features. Although handcrafted features provide rich expert knowledge, current fusion methods inadequately explore the relatively significant role of handcrafted features in conjunction with deep features, the imbalances in feature contributions, and the cooperative ways in which features learn. In this article, we propose a novel multifeature collaborative fusion network with deep supervision (MFCFNet) to effectively fuse handcrafted features and deep features for SAR ship classification tasks. Specifically, our framework mainly includes two types of feature extraction branches, a knowledge supervision and collaboration module (KSCM) and a feature fusion and contribution assignment module (FFCA). The former module improves the quality of the feature maps learned by each branch through auxiliary feature supervision and introduces a synergy loss to facilitate the interaction of information between deep features and handcrafted features. The latter module utilizes an attention mechanism to adaptively balance the importance among various features and assign the corresponding feature contributions to the total loss function based on the generated feature weights. We conducted extensive experimental and ablation studies on two public datasets, OpenSARShip-1.0 and FUSAR-Ship, and the results show that MFCFNet is effective and outperforms single deep feature and multifeature models based on previous internal FC layer and terminal FC layer fusion. Furthermore, our proposed MFCFNet exhibits better performance than the current state-of-the-art methods.

KW - Multi-Feature fusion

KW - handcrafted feature

KW - deep supervision

KW - synthetic aperture radar (SAR)

KW - SAR ship classification

U2 - 10.1109/TGRS.2023.3297648

DO - 10.1109/TGRS.2023.3297648

M3 - Journal article

VL - 61

SP - 1

EP - 14

JO - IEEE Transactions on Geoscience and Remote Sensing

JF - IEEE Transactions on Geoscience and Remote Sensing

SN - 0196-2892

M1 - 5212614

ER -