Identifying Student Profiles Within Online Judge Systems Using Explainable Artificial Intelligence

Abstract

Online Judge (OJ) systems are widely used in programming courses to provide fast and objective evaluation of students’ code. However, these systems usually deliver only a binary outcome—pass or fail—which offers limited educational value. To address this limitation, we propose a learning-based approach that leverages the behavioural data captured by OJ systems to generate richer and more informative feedback. Our method employs Multi-Instance Learning and traditional Machine Learning techniques to model student behaviour, while Explainable Artificial Intelligence (XAI) ensures that predictions and feedback remain interpretable and actionable. The approach was validated on a case study involving 2,500 submissions from 90 students in a Computer Science programming course. Results show that the model can accurately predict student outcomes based solely on behavioural patterns and identify at-risk groups. This contributes valuable insights for both learners and instructors, enhancing guidance, early intervention, and teaching strategies beyond binary evaluation.