TY - JOUR

T1 - Toward Class-Agnostic Tracking Using Feature Decorrelation in Point Clouds

AU - Tian, Shengjing

AU - Liu, Jun

AU - Liu, Xiuping

PY - 2024/12/31

Y1 - 2024/12/31

N2 - Single object tracking in point clouds has been attracting more and more attention owing to the presence of LiDAR sensors in 3D vision. However, existing methods based on deep neural networks mainly focus on training different models for different categories, which makes them unable to perform well in real-world applications when encountering classes unseen during the training phase. In this work, we investigate a more challenging task in LiDAR point clouds, namely class-agnostic tracking, where a general model is supposed to be learned to handle targets of both observed and unseen categories. In particular, we first investigate the class-agnostic performance of state-of-the-art trackers by exposing the unseen categories to them during testing. It is found that as the distribution shifts from observed to unseen classes, how to constrain the fused features between the template and the search region to maintain generalization is a key factor in class-agnostic tracking. Therefore, we propose a feature decorrelation method to address this problem, which eliminates the spurious correlations of the fused features through a set of learned weights, and further makes the search region consistent among foreground points and distinctive between foreground and background points. Experiments on KITTI and NuScenes demonstrate that the proposed method can achieve considerable improvements by benchmarking against the advanced trackers P2B and BAT, especially when tracking unseen objects.

AB - Single object tracking in point clouds has been attracting more and more attention owing to the presence of LiDAR sensors in 3D vision. However, existing methods based on deep neural networks mainly focus on training different models for different categories, which makes them unable to perform well in real-world applications when encountering classes unseen during the training phase. In this work, we investigate a more challenging task in LiDAR point clouds, namely class-agnostic tracking, where a general model is supposed to be learned to handle targets of both observed and unseen categories. In particular, we first investigate the class-agnostic performance of state-of-the-art trackers by exposing the unseen categories to them during testing. It is found that as the distribution shifts from observed to unseen classes, how to constrain the fused features between the template and the search region to maintain generalization is a key factor in class-agnostic tracking. Therefore, we propose a feature decorrelation method to address this problem, which eliminates the spurious correlations of the fused features through a set of learned weights, and further makes the search region consistent among foreground points and distinctive between foreground and background points. Experiments on KITTI and NuScenes demonstrate that the proposed method can achieve considerable improvements by benchmarking against the advanced trackers P2B and BAT, especially when tracking unseen objects.

U2 - 10.1109/TIP.2023.3348635

DO - 10.1109/TIP.2023.3348635

M3 - Journal article

VL - 33

SP - 682

EP - 695

JO - IEEE Transactions on Image Processing

JF - IEEE Transactions on Image Processing

SN - 1057-7149

ER -