Home > Research > Publications & Outputs > Structured Context Enhancement Network for Mous...

Electronic data

  • Structured Context Enhancement Network for Mouse Pose Estimation

    Rights statement: ©2021 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.

    Accepted author manuscript, 11.1 MB, PDF document

    Available under license: CC BY-NC: Creative Commons Attribution-NonCommercial 4.0 International License

Links

Text available via DOI:

View graph of relations

Structured Context Enhancement Network for Mouse Pose Estimation

Research output: Contribution to journalJournal articlepeer-review

E-pub ahead of print

Standard

Structured Context Enhancement Network for Mouse Pose Estimation. / Zhou, F.; Jiang, Z.; Liu, Z.; Chen, F.; Chen, L.; Tong, L.; Yang, Z.; Wang, H.; Fei, M.; Li, L.; Zhou, H.

In: IEEE Transactions on Circuits and Systems for Video Technology, 21.07.2021.

Research output: Contribution to journalJournal articlepeer-review

Harvard

Zhou, F, Jiang, Z, Liu, Z, Chen, F, Chen, L, Tong, L, Yang, Z, Wang, H, Fei, M, Li, L & Zhou, H 2021, 'Structured Context Enhancement Network for Mouse Pose Estimation', IEEE Transactions on Circuits and Systems for Video Technology. https://doi.org/10.1109/TCSVT.2021.3098497

APA

Zhou, F., Jiang, Z., Liu, Z., Chen, F., Chen, L., Tong, L., Yang, Z., Wang, H., Fei, M., Li, L., & Zhou, H. (2021). Structured Context Enhancement Network for Mouse Pose Estimation. IEEE Transactions on Circuits and Systems for Video Technology. https://doi.org/10.1109/TCSVT.2021.3098497

Vancouver

Zhou F, Jiang Z, Liu Z, Chen F, Chen L, Tong L et al. Structured Context Enhancement Network for Mouse Pose Estimation. IEEE Transactions on Circuits and Systems for Video Technology. 2021 Jul 21. https://doi.org/10.1109/TCSVT.2021.3098497

Author

Zhou, F. ; Jiang, Z. ; Liu, Z. ; Chen, F. ; Chen, L. ; Tong, L. ; Yang, Z. ; Wang, H. ; Fei, M. ; Li, L. ; Zhou, H. / Structured Context Enhancement Network for Mouse Pose Estimation. In: IEEE Transactions on Circuits and Systems for Video Technology. 2021.

Bibtex

@article{212cd816ece24bbb9666a9fc26cb815f,
title = "Structured Context Enhancement Network for Mouse Pose Estimation",
abstract = "Automated analysis of mouse behaviours is crucial for many applications in neuroscience. However, quantifying mouse behaviours from videos or images remains a challenging problem, where pose estimation plays an important role in describing mouse behaviours. Although deep learning based methods have made promising advances in human pose estimation, they cannot be directly applied to pose estimation of mice due to different physiological natures. Particularly, since mouse body is highly deformable, it is a challenge to accurately locate different keypoints on the mouse body. In this paper, we propose a novel Hourglass network based model, namely Graphical Model based Structured Context Enhancement Network (GMSCENet) where two effective modules, i.e., Structured Context Mixer (SCM) and Cascaded Multi-level Supervision (CMLS) are subsequently implemented. SCM can adaptively learn and enhance the proposed structured context information of each mouse part by a novel graphical model that takes into account the motion difference between body parts. Then, the CMLS module is designed to jointly train the proposed SCM and the Hourglass network by generating multi-level information, increasing the robustness of the whole network. Using the multi-level prediction information from SCM and CMLS, we develop an inference method to ensure the accuracy of the localisation results. Finally, we evaluate our proposed approach against several baselines on our Parkinson{\textquoteright}s Disease Mouse Behaviour (PDMB) and the standard DeepLabCut Mouse Pose datasets. The experimental results show that our method achieves better or competitive performance against the other state-of-the-art approaches. ",
keywords = "Context modeling, Diseases, graphical model, Graphical models, Mice, Mixers, Mouse Behaviour dataset, multi-level information, Parkinson’s Disease, Pose estimation, Standards, Graphic methods, Mammals, Automated analysis, Competitive performance, Context information, Human pose estimations, Learning-based methods, Network-based modeling, Prediction informations, State-of-the-art approach, Deep learning",
author = "F. Zhou and Z. Jiang and Z. Liu and F. Chen and L. Chen and L. Tong and Z. Yang and H. Wang and M. Fei and L. Li and H. Zhou",
note = "{\textcopyright}2021 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE. ",
year = "2021",
month = jul,
day = "21",
doi = "10.1109/TCSVT.2021.3098497",
language = "English",
journal = "IEEE Transactions on Circuits and Systems for Video Technology",
issn = "1051-8215",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

RIS

TY - JOUR

T1 - Structured Context Enhancement Network for Mouse Pose Estimation

AU - Zhou, F.

AU - Jiang, Z.

AU - Liu, Z.

AU - Chen, F.

AU - Chen, L.

AU - Tong, L.

AU - Yang, Z.

AU - Wang, H.

AU - Fei, M.

AU - Li, L.

AU - Zhou, H.

N1 - ©2021 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.

PY - 2021/7/21

Y1 - 2021/7/21

N2 - Automated analysis of mouse behaviours is crucial for many applications in neuroscience. However, quantifying mouse behaviours from videos or images remains a challenging problem, where pose estimation plays an important role in describing mouse behaviours. Although deep learning based methods have made promising advances in human pose estimation, they cannot be directly applied to pose estimation of mice due to different physiological natures. Particularly, since mouse body is highly deformable, it is a challenge to accurately locate different keypoints on the mouse body. In this paper, we propose a novel Hourglass network based model, namely Graphical Model based Structured Context Enhancement Network (GMSCENet) where two effective modules, i.e., Structured Context Mixer (SCM) and Cascaded Multi-level Supervision (CMLS) are subsequently implemented. SCM can adaptively learn and enhance the proposed structured context information of each mouse part by a novel graphical model that takes into account the motion difference between body parts. Then, the CMLS module is designed to jointly train the proposed SCM and the Hourglass network by generating multi-level information, increasing the robustness of the whole network. Using the multi-level prediction information from SCM and CMLS, we develop an inference method to ensure the accuracy of the localisation results. Finally, we evaluate our proposed approach against several baselines on our Parkinson’s Disease Mouse Behaviour (PDMB) and the standard DeepLabCut Mouse Pose datasets. The experimental results show that our method achieves better or competitive performance against the other state-of-the-art approaches.

AB - Automated analysis of mouse behaviours is crucial for many applications in neuroscience. However, quantifying mouse behaviours from videos or images remains a challenging problem, where pose estimation plays an important role in describing mouse behaviours. Although deep learning based methods have made promising advances in human pose estimation, they cannot be directly applied to pose estimation of mice due to different physiological natures. Particularly, since mouse body is highly deformable, it is a challenge to accurately locate different keypoints on the mouse body. In this paper, we propose a novel Hourglass network based model, namely Graphical Model based Structured Context Enhancement Network (GMSCENet) where two effective modules, i.e., Structured Context Mixer (SCM) and Cascaded Multi-level Supervision (CMLS) are subsequently implemented. SCM can adaptively learn and enhance the proposed structured context information of each mouse part by a novel graphical model that takes into account the motion difference between body parts. Then, the CMLS module is designed to jointly train the proposed SCM and the Hourglass network by generating multi-level information, increasing the robustness of the whole network. Using the multi-level prediction information from SCM and CMLS, we develop an inference method to ensure the accuracy of the localisation results. Finally, we evaluate our proposed approach against several baselines on our Parkinson’s Disease Mouse Behaviour (PDMB) and the standard DeepLabCut Mouse Pose datasets. The experimental results show that our method achieves better or competitive performance against the other state-of-the-art approaches.

KW - Context modeling

KW - Diseases

KW - graphical model

KW - Graphical models

KW - Mice

KW - Mixers

KW - Mouse Behaviour dataset

KW - multi-level information

KW - Parkinson’s Disease

KW - Pose estimation

KW - Standards

KW - Graphic methods

KW - Mammals

KW - Automated analysis

KW - Competitive performance

KW - Context information

KW - Human pose estimations

KW - Learning-based methods

KW - Network-based modeling

KW - Prediction informations

KW - State-of-the-art approach

KW - Deep learning

U2 - 10.1109/TCSVT.2021.3098497

DO - 10.1109/TCSVT.2021.3098497

M3 - Journal article

JO - IEEE Transactions on Circuits and Systems for Video Technology

JF - IEEE Transactions on Circuits and Systems for Video Technology

SN - 1051-8215

ER -