Facial symmetry analysis can provide an important role in the diagnosis and rehabilitation of medical conditions like facial paralysis issues such as bell’s palsy. Recent advances in computer vision techniques specifically the use of deep convolutional neural networks and multi-task learning provide a gateway to fast and state-of-the-art accurate methods for object detection tasks. In this paper, we present a novel unified multi-task CNN framework for simultaneous object proposal, face detection and face symmetry analysis. We highlight the potential possibilities for the use of such a framework within the medical domain through the experimental results on two test data sets. The results are promising showing high level of accuracy for both the task of face detection and symmetry analysis while also highlighting the efficient computational overhead for our proposed method which can process an image in 0.04 s.