TY - JOUR

T1 - Emotion recognition from scrambled facial images via many graph embedding

AU - Jiang, Richard

AU - Ho, Anthony T. S.

AU - Cheheb, Ismahane

AU - Al-Maadeed, Noor

AU - Al-Maadeed, Somaya

AU - Bouridane, Ahmed

PY - 2017/7/1

Y1 - 2017/7/1

N2 - Facial expression verification has been extensively exploited due to its wide application in affective computing, robotic vision, man-machine interaction and medical diagnosis. With the recent development of Internet-of-Things (IoT), there is a need of mobile-targeted facial expression verification, where face scrambling has been proposed for privacy protection during image/video distribution over public network. Consequently, facial expression verification needs to be carried out in a scrambled domain, bringing out new challenges in facial expression recognition. An immediate impact from face scrambling is that conventional semantic facial components become not identifiable, and 3D face models cannot be clearly fitted to a scrambled image. Hence, the classical facial action coding system cannot be applied to facial expression recognition in the scrambled domain. To cope with chaotic signals from face scrambling, this paper proposes an new approach – Many Graph Embedding (MGE) to discover discriminative patterns from the subspaces of chaotic patterns, where the facial expression recognition is carried out as a fuzzy combination from many graph embedding. In our experiments, the proposed MGE was evaluated on three scrambled facial expression datasets: JAFFE, MUG and CK++. The benchmark results demonstrated that the proposed method is able to improve the recognition accuracy, making our method a promising candidate for the scrambled facial expression recognition in the emerging privacy-protected IoT applications.

AB - Facial expression verification has been extensively exploited due to its wide application in affective computing, robotic vision, man-machine interaction and medical diagnosis. With the recent development of Internet-of-Things (IoT), there is a need of mobile-targeted facial expression verification, where face scrambling has been proposed for privacy protection during image/video distribution over public network. Consequently, facial expression verification needs to be carried out in a scrambled domain, bringing out new challenges in facial expression recognition. An immediate impact from face scrambling is that conventional semantic facial components become not identifiable, and 3D face models cannot be clearly fitted to a scrambled image. Hence, the classical facial action coding system cannot be applied to facial expression recognition in the scrambled domain. To cope with chaotic signals from face scrambling, this paper proposes an new approach – Many Graph Embedding (MGE) to discover discriminative patterns from the subspaces of chaotic patterns, where the facial expression recognition is carried out as a fuzzy combination from many graph embedding. In our experiments, the proposed MGE was evaluated on three scrambled facial expression datasets: JAFFE, MUG and CK++. The benchmark results demonstrated that the proposed method is able to improve the recognition accuracy, making our method a promising candidate for the scrambled facial expression recognition in the emerging privacy-protected IoT applications.

KW - Facial expression

KW - Emotion recognition

KW - User privacy

KW - Chaotic patterns

KW - Many graph embedding

U2 - 10.1016/j.patcog.2017.02.003

DO - 10.1016/j.patcog.2017.02.003

M3 - Journal article

VL - 67

SP - 245

EP - 251

JO - Pattern Recognition

JF - Pattern Recognition

SN - 0031-3203

ER -