A Comparative Analysis of Machine Learning Models for Predicting Student Performance: Evaluating the Impact of Stacking and Traditional Methods

Abstract

This study investigates the application of machine learning models to predict student performance using socio-economic, demographic, and academic factors. Various models were developed and evaluated, including Linear Regression, Random Forest, Gradient Boosting, XGBoost, LightGBM, Support Vector Regressor, and a Stacking Regressor. The models were assessed using key evaluation metrics: Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), R-squared (????²), Mean Squared Log Error (MSLE), and Mean Absolute Percentage Error (MAPE). The Support Vector Regressor demonstrated the best overall performance, with an MAE of 4.3091, RMSE of 5.4110, and an ????² of 0.8685, surpassing even the more complex ensemble models. Similarly, Linear Regression achieved strong results, with an MAE of 4.3154 and ????² of 0.8685. In contrast, the Stacking Regressor, while effective, did not significantly outperform its base models, achieving an MAE of 4.5340 and ????² of 0.8563, highlighting that greater model complexity does not necessarily lead to better predictive power. The analysis also revealed that MAPE was highly sensitive to outliers in the dataset, indicating the need for robust data preprocessing to handle extreme values. These results suggest that, in educational data mining, simpler models can often match or exceed the performance of more complex methods. Future research should investigate advanced ensembling strategies and feature engineering techniques to further enhance the accuracy and reliability of student performance predictions.