Classroom engagement analysis and feedback using multimodal emotion recognition

Student engagement is critical to educational success, influencing learning outcomes and overall academic performance. Traditional methods like surveys and teacher observations can be time-consuming, subjective, and prone to bias. Various AI algorithms have been explored to address these limitations.
From an instructor's perspective, understanding student engagement levels is crucial for improving teaching styles and methods. This research aims to bridge that gap by providing a practical tool to assess engagement, enabling informed adjustments to instructional strategies, particularly in large classrooms where focusing on each student is difficult. It is especially valuable for beginner educators who find monitoring engagement challenging.
The initial research investigates applying deep learning techniques to classify student emotions in classrooms using the publicly available DAiSEE dataset, comprising 9068 videos labelled with academic affective states - boredom, confusion, engagement, and frustration. This multi-label dataset was converted into a single-label format, and videos were split into 5 and 2-second segments, creating new datasets. A deep-learning model is developed, which integrates time-distributed convolutional layers for spatial feature extraction and an LSTM layer for capturing temporal dynamics.
The results demonstrated the model's effectiveness in accurately detecting all emotions except confusion. Distinguishing this remains challenging due to its similarity to other emotions, like Frustration. The results also showed that collecting behaviours like body movements and gestures, not only facial expressions, is important to finding student engagement.
Future research will lay the groundwork for developing an AI-driven system to assist educators in monitoring and improving student engagement, ultimately fostering better educational outcomes.

Key learning outcomes for participants:
This presentation will give the participants a brief overview of how AI can be used from the instructor’s perspective to improve student engagement in a classroom.
• Limitations of Traditional Methods: They will recognise the limitations of traditional methods like surveys and teacher observations in assessing student engagement, including time consumption, subjectivity, and bias.
• Application of AI Algorithms: Participants will gain insights into how various AI algorithms can address these limitations by providing more objective and efficient assessments of student engagement.
• Importance of Comprehensive Behaviour Analysis: Participants will understand the necessity of collecting comprehensive behavioural data, including body movements and gestures, in addition to facial expressions, for accurate engagement detection.
• Challenges and Future Directions: Participants will be informed about the challenges, such as privacy concerns and the future directions for research, including developing an AI-driven system to assist educators in monitoring and improving student engagement.