Facial Expression Analysis Using Deep Learning for Student classroom Engagement Recognition

What is it about?

In this paper, facial expression-based student engagement analysis in a classroom environment is proposed. Face detection has been achieved by YOLO (You only look once) detector to find multiple faces in the classroom with maximum speed and accuracy. Consequently, by adopting the Ensemble of Robust Constrained Local Models (ERCLM) method, landmark points are localized in detected faces even in occlusion, and therefore, feature matching is performed. Besides, the matched landmark points are aligned by an affine transformation. Finally, having different expressions, the aligned faces are fed as input to Faster R-CNN (Faster Regions with Convolutional Neural Network Features). It recognizes behavioral activities such as Attentiveness (Zero-in (ZI)), Non-Attentiveness (NA), Day Dreaming (DD), Napping (N), Playing with Personal Stuff in Private (PPSP), and Talking to the Students Behind(TSB). The proposed approach is demonstrated using the TCE Classroom dataset, and web datasets. The proposed framework outperforms the state-of-the-art algorithms.

Featured Image