Final published version, 826 KB, PDF document
Available under license: Unspecified
Final published version
Research output: Contribution in Book/Report/Proceedings - With ISBN/ISSN › Conference contribution/Paper › peer-review
Research output: Contribution in Book/Report/Proceedings - With ISBN/ISSN › Conference contribution/Paper › peer-review
}
TY - GEN
T1 - Else-Net
T2 - Elastic Semantic Network for Continual Action Recognition from Skeleton Data
AU - Li, Tianjiao
AU - Ke, Qiuhong
AU - Rahmani, Hossein
AU - Ho, Rui En
AU - Ding, Henghui
AU - Liu, Jun
PY - 2022/2/28
Y1 - 2022/2/28
N2 - Most of the state-of-the-art action recognition methods focus on offline learning, where the samples of all types of actions need to be provided at once. Here, we address continual learning of action recognition, where various types of new actions are continuously learned over time.This task is quite challenging, owing to the catastrophic forgetting problem stemming from the discrepancies between the previously learned actions and current new actions to be learned. Therefore, we propose Else-Net, a novel ElasticSemantic Network with multiple learning blocks to learn diversified human actions over time. Specifically, our ElseNet is able to automatically search and update the most relevant learning blocks w.r.t. the current new action, or explore new blocks to store new knowledge, preserving the unmatched ones to retain the knowledge of previously learned actions and alleviates forgetting when learning new actions. Moreover, even though different human actions may vary to a large extent as a whole, their local body parts can still share many homogeneous features. Inspired by this, our proposed Else-Net mines the shared knowledgeof the decomposed human body parts from different actions, which benefits continual learning of actions. Experiments show that the proposed approach enables effective continual action recognition and achieves promising performance on two large-scale action recognition datasets
AB - Most of the state-of-the-art action recognition methods focus on offline learning, where the samples of all types of actions need to be provided at once. Here, we address continual learning of action recognition, where various types of new actions are continuously learned over time.This task is quite challenging, owing to the catastrophic forgetting problem stemming from the discrepancies between the previously learned actions and current new actions to be learned. Therefore, we propose Else-Net, a novel ElasticSemantic Network with multiple learning blocks to learn diversified human actions over time. Specifically, our ElseNet is able to automatically search and update the most relevant learning blocks w.r.t. the current new action, or explore new blocks to store new knowledge, preserving the unmatched ones to retain the knowledge of previously learned actions and alleviates forgetting when learning new actions. Moreover, even though different human actions may vary to a large extent as a whole, their local body parts can still share many homogeneous features. Inspired by this, our proposed Else-Net mines the shared knowledgeof the decomposed human body parts from different actions, which benefits continual learning of actions. Experiments show that the proposed approach enables effective continual action recognition and achieves promising performance on two large-scale action recognition datasets
UR - https://openaccess.thecvf.com/content/ICCV2021/papers/Li_Else-Net_Elastic_Semantic_Network_for_Continual_Action_Recognition_From_Skeleton_ICCV_2021_paper.pdf
U2 - 10.1109/ICCV48922.2021.01318
DO - 10.1109/ICCV48922.2021.01318
M3 - Conference contribution/Paper
SN - 9781665428132
SP - 13414
EP - 13423
BT - 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
PB - IEEE
ER -