Rights statement: ©2017 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.
Accepted author manuscript, 26.9 MB, PDF document
Available under license: CC BY-NC: Creative Commons Attribution-NonCommercial 4.0 International License
Final published version
Research output: Contribution to Journal/Magazine › Journal article › peer-review
Research output: Contribution to Journal/Magazine › Journal article › peer-review
}
TY - JOUR
T1 - Unconstrained Face Recognition Using A Set-to-Set Distance Measure on Deep Learned Features
AU - Zhao, Jiaojiao
AU - Han, Jungong
AU - Shao, Ling
N1 - ©2017 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.
PY - 2018/10
Y1 - 2018/10
N2 - Recently considerable efforts have been dedicated to unconstrained face recognition, which requires to identify faces "in the wild" for a set of images and/or video frames captured without human intervention. Unlike traditional face recognition that compares one-to-one medium (either a single image or a video frame) only, we consider a problem of matching sets with heterogeneous contents of both images and videos. In this paper, we propose a novel Set-to-Set (S2S) distance measure to calculate the similarity between two sets with the aim to improve the accuracy of face recognition in real-world situations such as extreme poses or severe illumination conditions. Our S2S distance adopts the kNN-average pooling for the similarity scores computed on all the media in two sets, making the identification far less susceptible to the poor representations (outliers) than traditional feature-average pooling and score-average pooling. Furthermore, we show that various metrics can be embedded into our S2S distance framework, including both predefined and learned ones. This allows to choose the appropriate metric depending on the recognition task in order to achieve the best results. To evaluate the proposed S2S distance, we conduct extensive experiments on the challenging set-based IJB-A face dataset, which demonstrate that our algorithm achieves the stateof- the-art results and is clearly superior to the baselines including several deep learning based face recognition algorithms.
AB - Recently considerable efforts have been dedicated to unconstrained face recognition, which requires to identify faces "in the wild" for a set of images and/or video frames captured without human intervention. Unlike traditional face recognition that compares one-to-one medium (either a single image or a video frame) only, we consider a problem of matching sets with heterogeneous contents of both images and videos. In this paper, we propose a novel Set-to-Set (S2S) distance measure to calculate the similarity between two sets with the aim to improve the accuracy of face recognition in real-world situations such as extreme poses or severe illumination conditions. Our S2S distance adopts the kNN-average pooling for the similarity scores computed on all the media in two sets, making the identification far less susceptible to the poor representations (outliers) than traditional feature-average pooling and score-average pooling. Furthermore, we show that various metrics can be embedded into our S2S distance framework, including both predefined and learned ones. This allows to choose the appropriate metric depending on the recognition task in order to achieve the best results. To evaluate the proposed S2S distance, we conduct extensive experiments on the challenging set-based IJB-A face dataset, which demonstrate that our algorithm achieves the stateof- the-art results and is clearly superior to the baselines including several deep learning based face recognition algorithms.
KW - Face recognition
KW - IJB-A
KW - kNN-average pooling
KW - S2S distance
KW - Deep learning
KW - Face recognition algorithms
KW - Human intervention
KW - Illumination conditions
KW - Recognition accuracy
KW - Similarity scores
U2 - 10.1109/TCSVT.2017.2710120
DO - 10.1109/TCSVT.2017.2710120
M3 - Journal article
VL - 28
SP - 2679
EP - 2689
JO - IEEE Transactions on Circuits and Systems for Video Technology
JF - IEEE Transactions on Circuits and Systems for Video Technology
SN - 1051-8215
IS - 10
ER -