Home > Research > Publications & Outputs > Frequency Decoupled Masked Auto-Encoder for Sel...

Electronic data

  • paper

    Accepted author manuscript, 846 KB, PDF document

    Available under license: CC BY: Creative Commons Attribution 4.0 International License

Links

Text available via DOI:

View graph of relations

Frequency Decoupled Masked Auto-Encoder for Self-Supervised Skeleton-Based Action Recognition

Research output: Contribution to Journal/MagazineJournal articlepeer-review

E-pub ahead of print
  • Ye Liu
  • Tianhao Shi
  • Mingliang Zhai
  • Jun Liu
Close
<mark>Journal publication date</mark>31/12/2025
<mark>Journal</mark>IEEE Signal Processing Letters
Number of pages5
Pages (from-to)546-550
Publication StatusE-pub ahead of print
Early online date3/01/25
<mark>Original language</mark>English

Abstract

In 3D skeleton-based action recognition, the limited availability of supervised data has driven interest in self-supervised learning methods. The reconstruction paradigm using masked auto-encoder (MAE) is an effective and mainstream self-supervised learning approach. However, recent studies indicate that MAE models tend to focus on features within a certain frequency range, which may result in the loss of important information. To address this issue, we propose a frequency decoupled MAE. Specifically, by incorporating a scale-specific frequency feature reconstruction module, we delve into leveraging frequency information as a direct and explicit target for reconstruction, which augments the MAE's capability to discern and accurately reproduce diverse frequency attributes within the data. Moreover, in order to address the issue of unstable gradient updates caused by more complex optimization objectives with frequency reconstruction, we introduce a dual-path network combined with an exponential moving average (EMA) parameter updating strategy to guide the model in stabilizing the training process. We have conducted extensive experiments which have demonstrated the effectiveness of the proposed method.