Home > Research > Publications & Outputs > Social Behavioral Phenotyping of Drosophila wit...

Electronic data

  • Draft_Revised

    Accepted author manuscript, 889 KB, PDF document

    Available under license: CC BY: Creative Commons Attribution 4.0 International License

Links

Text available via DOI:

View graph of relations

Social Behavioral Phenotyping of Drosophila with a 2D-3D Hybrid CNN Framework

Research output: Contribution to journalJournal articlepeer-review

Published

Standard

Social Behavioral Phenotyping of Drosophila with a 2D-3D Hybrid CNN Framework. / Jiang, Ziping; Chazot, Paul L; Celebi, M. Emre; Crookes, Danny; Jiang, Richard.

In: IEEE Access, Vol. 7, 15.05.2019, p. 67972 - 67982.

Research output: Contribution to journalJournal articlepeer-review

Harvard

Jiang, Z, Chazot, PL, Celebi, ME, Crookes, D & Jiang, R 2019, 'Social Behavioral Phenotyping of Drosophila with a 2D-3D Hybrid CNN Framework', IEEE Access, vol. 7, pp. 67972 - 67982. https://doi.org/10.1109/ACCESS.2019.2917000

APA

Jiang, Z., Chazot, P. L., Celebi, M. E., Crookes, D., & Jiang, R. (2019). Social Behavioral Phenotyping of Drosophila with a 2D-3D Hybrid CNN Framework. IEEE Access, 7, 67972 - 67982. https://doi.org/10.1109/ACCESS.2019.2917000

Vancouver

Jiang Z, Chazot PL, Celebi ME, Crookes D, Jiang R. Social Behavioral Phenotyping of Drosophila with a 2D-3D Hybrid CNN Framework. IEEE Access. 2019 May 15;7:67972 - 67982. https://doi.org/10.1109/ACCESS.2019.2917000

Author

Jiang, Ziping ; Chazot, Paul L ; Celebi, M. Emre ; Crookes, Danny ; Jiang, Richard. / Social Behavioral Phenotyping of Drosophila with a 2D-3D Hybrid CNN Framework. In: IEEE Access. 2019 ; Vol. 7. pp. 67972 - 67982.

Bibtex

@article{08fc982ebcb84e50b4a1910f1ab87bf2,
title = "Social Behavioral Phenotyping of Drosophila with a 2D-3D Hybrid CNN Framework",
abstract = "Behavioural phenotyping of drosphila is an important means in biological and medical research to identify genetic, pathologic or psychologic impact on animal behviour. Automated behavioural phenotyping from videos has been a desired capability that can waive long-time boring manual work in behavioral analysis. In this paper, we introduced deep learning into this challenging topic, and proposed a new 2D+3D hybrid CNN framework for drosphila{\textquoteright}s social behavioural phenotyping. In the proposed multitask learning framework, action detection and localization of drosphila jointly is carried out with action classification, and a given video is divided into clips with fixed length. Each clip is fed into the system and a 2-D CNN is applied to extract features at frame level. Features extracted from adjacent frames are then connected and fed into a 3-D CNN with a spatial region proposal layer for classification. In such a 2D+3D hybrid framework, drosophila detection at the frame level enables the action analysis at different durations instead of a fixed period. We tested our framework with different base layers and classification architectures and validated the proposed 3D CNN based social behavioral phenotyping framework under various models, detectors and classifiers.",
author = "Ziping Jiang and Chazot, {Paul L} and Celebi, {M. Emre} and Danny Crookes and Richard Jiang",
year = "2019",
month = may,
day = "15",
doi = "10.1109/ACCESS.2019.2917000",
language = "English",
volume = "7",
pages = "67972 -- 67982",
journal = "IEEE Access",
issn = "2169-3536",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

RIS

TY - JOUR

T1 - Social Behavioral Phenotyping of Drosophila with a 2D-3D Hybrid CNN Framework

AU - Jiang, Ziping

AU - Chazot, Paul L

AU - Celebi, M. Emre

AU - Crookes, Danny

AU - Jiang, Richard

PY - 2019/5/15

Y1 - 2019/5/15

N2 - Behavioural phenotyping of drosphila is an important means in biological and medical research to identify genetic, pathologic or psychologic impact on animal behviour. Automated behavioural phenotyping from videos has been a desired capability that can waive long-time boring manual work in behavioral analysis. In this paper, we introduced deep learning into this challenging topic, and proposed a new 2D+3D hybrid CNN framework for drosphila’s social behavioural phenotyping. In the proposed multitask learning framework, action detection and localization of drosphila jointly is carried out with action classification, and a given video is divided into clips with fixed length. Each clip is fed into the system and a 2-D CNN is applied to extract features at frame level. Features extracted from adjacent frames are then connected and fed into a 3-D CNN with a spatial region proposal layer for classification. In such a 2D+3D hybrid framework, drosophila detection at the frame level enables the action analysis at different durations instead of a fixed period. We tested our framework with different base layers and classification architectures and validated the proposed 3D CNN based social behavioral phenotyping framework under various models, detectors and classifiers.

AB - Behavioural phenotyping of drosphila is an important means in biological and medical research to identify genetic, pathologic or psychologic impact on animal behviour. Automated behavioural phenotyping from videos has been a desired capability that can waive long-time boring manual work in behavioral analysis. In this paper, we introduced deep learning into this challenging topic, and proposed a new 2D+3D hybrid CNN framework for drosphila’s social behavioural phenotyping. In the proposed multitask learning framework, action detection and localization of drosphila jointly is carried out with action classification, and a given video is divided into clips with fixed length. Each clip is fed into the system and a 2-D CNN is applied to extract features at frame level. Features extracted from adjacent frames are then connected and fed into a 3-D CNN with a spatial region proposal layer for classification. In such a 2D+3D hybrid framework, drosophila detection at the frame level enables the action analysis at different durations instead of a fixed period. We tested our framework with different base layers and classification architectures and validated the proposed 3D CNN based social behavioral phenotyping framework under various models, detectors and classifiers.

U2 - 10.1109/ACCESS.2019.2917000

DO - 10.1109/ACCESS.2019.2917000

M3 - Journal article

VL - 7

SP - 67972

EP - 67982

JO - IEEE Access

JF - IEEE Access

SN - 2169-3536

ER -