Performance Comparison of SelectKBest and Permutation Importance in Feature Selection for Diabetes Prediction

Nita Cahyani; Rahmat Irsyada

doi:10.47709/brilliance.v5i1.6507

Authors

Nita Cahyani Universitas Padjadjaran, Indonesia
Rahmat Irsyada Politeknik Negeri Subang, Indonesia

DOI:

https://doi.org/10.47709/brilliance.v5i1.6507

Keywords:

Classsification, Diabetes, SelectKBest, Permutation Importance, Machine Learning

Abstract

This study evaluates the effectiveness of two feature selection methods, namely the statistics-based SelectKBest and the model-based Permutation Importance, in improving the performance of classification algorithms for diabetes prediction. A dataset consisting of 17 clinical and demographic features was used to train 11 machine learning algorithms with two subsets of selected features. Performance evaluation used accuracy, precision, recall, F1-Score, ROC AUC, and training time. Based on the results, the SelectKBest method was able to improve the performance of Random Forest with an accuracy of 82.7%, a precision of 0.8, a recall of 0.5, and an F1-Score of 0.615. Meanwhile, the Permutation Importance method showed more consistent performance, with six models including Random Forest, K-Nearest Neighbors, and Quadratic Discriminant Analysis (QDA) achieving an accuracy of up to 86.2%. QDA stood out with the highest ROC AUC of 0.887, indicating better class detection capabilities. These findings underscore the superiority of Permutation Importance in selecting relevant and varied features, including demographic factors, thereby improving overall prediction accuracy. In practice, Random Forest with SelectKBest is recommended for applications requiring fast and interpretable models, while QDA and Gradient Boosting with Permutation Importance are recommended for those requiring high accuracy and sensitivity. This study strengthens the foundation for developing more accurate and applicable diabetes prediction models across various contexts.

References

Aliyu, H. A., Muritala, I. O., Bello-Salau, H., Mohammed, S., Onumanyi, A. J., & Ajayi, O.-O. (2024). Optimizing machine learning algorithms for diabetes data: A metaheuristic approach to balancing and tuning classifiers parameters. Franklin Open, 8, 100153. doi: https://doi.org/10.1016/j.fraope.2024.100153

Allgaier, J., Mulansky, L., Draelos, R. L., & Pryss, R. (2023). How does the model make predictions? A systematic literature review on the explainability power of machine learning in healthcare. Artificial Intelligence in Medicine, 143, 102616. doi: https://doi.org/10.1016/j.artmed.2023.102616

Baniecki, H., Sobieski, B., Szatkowski, P., Bombinski, P., & Biecek, P. (2025). Interpretable machine learning for time-to-event prediction in medicine and healthcare. Artificial Intelligence in Medicine, 159, 103026. doi: https://doi.org/10.1016/j.artmed.2024.103026

Cahyani, N., Irsyada, R., & Kartini, A. Y. (2025). Implementasi Machine Learning Model sebagai Sistem Prediksi Penyakit Breast Cancer. Digital Transformation Technology, 4(2), 1112–1120. doi: 10.47709/digitech.v4i2.5209

Cahyani, N., Irsyada, R., & Mahmuda, R. (2025). Penerapan Algoritma Neural Network untuk Klasifikasi Diabetes Mellitus: Perbandingan Backpropagation dan Resillient Backpropagation. Digital Transformation Technology, 4(2), 1067–1074. doi: 10.47709/digitech.v4i2.5208

Cahyani, N., & Kartini, A. Y. (2022). DLNN dan BPNN-GA Pada Prediksi Penyakit Diabetes di Bojonegoro. Journal of Mathematics Education and Science, 6(1), 1–9. doi: 10.32665/james.v6i1.416

Cahyani, N., Putri, W. A., & Irsyada, R. (2025). Improving Multiclass Rainfall Prediction with Multilayer Perceptron and SMOTE: Addressing Class Imbalance Challenges. Brilliance: Research of Artificial Intelligence, 4(2), 901–908. doi: 10.47709/brilliance.v4i2.5203

Cantú-Brito, C., Mimenza-Alvarado, A., & Sánchez-Hernández, J. J. (2010). Diabetes mellitus and aging as a risk factor for cerebral vascular disease: epidemiology, pathophysiology and prevention. Revista de Investigacion Clinica, 62(4), 333–342.

Cao, Z., Wu, X., Wu, B., Zhang, Z., & Sun, J. (2025). Combining UDT with XGBoost to identify the geographical origin of black beans by near-infrared spectroscopy. Current Research in Food Science, 11, 101131. doi: https://doi.org/10.1016/j.crfs.2025.101131

Chang, V., Ganatra, M. A., Hall, K., Golightly, L., & Xu, Q. A. (2022). An assessment of machine learning models and algorithms for early prediction and diagnosis of diabetes using health indicators. Healthcare Analytics, 2, 100118. doi: https://doi.org/10.1016/j.health.2022.100118

Cottin, A., Zulian, M., Pécuchet, N., Guilloux, A., & Katsahian, S. (2024). MS-CPFI: A model-agnostic Counterfactual Perturbation Feature Importance algorithm for interpreting black-box Multi-State models. Artificial Intelligence in Medicine, 147, 102741. doi: https://doi.org/10.1016/j.artmed.2023.102741

Daza, A., Ponce Sánchez, C. F., Apaza-Perez, G., Pinto, J., & Zavaleta Ramos, K. (2024). Stacking ensemble approach to diagnosing the disease of diabetes. Informatics in Medicine Unlocked, 44, 101427. doi: https://doi.org/10.1016/j.imu.2023.101427

Delpino, F. M., Costa, Â. K., Farias, S. R., Chiavegatto Filho, A. D. P., Arcêncio, R. A., & Nunes, B. P. (2022). Machine learning for predicting chronic diseases: a systematic review. Public Health, 205, 14–25. doi: https://doi.org/10.1016/j.puhe.2022.01.007

Dharmarathne, G., Jayasinghe, T. N., Bogahawaththa, M., Meddage, D. P. P., & Rathnayake, U. (2024). A novel machine learning approach for diagnosing diabetes with a self-explainable interface. Healthcare Analytics, 5, 100301. doi: https://doi.org/10.1016/j.health.2024.100301

Gavin, J. R., Rodbard, H. W., Battelino, T., Brosius, F., Ceriello, A., Cosentino, F., Giorgino, F., Green, J., Ji, L., Kellerer, M., Koob, S., Kosiborod, M., Lalic, N., Marx, N., Prashant Nedungadi, T., Parkin, C. G., Topsever, P., Rydén, L., Huey-Herng Sheu, W., … Schnell, O. (2024). Disparities in prevalence and treatment of diabetes, cardiovascular and chronic kidney diseases – Recommendations from the taskforce of the guideline workshop. Diabetes Research and Clinical Practice, 211, 111666. doi: https://doi.org/10.1016/j.diabres.2024.111666

Ghasemieh, A., Lloyed, A., Bahrami, P., Vajar, P., & Kashef, R. (2023). A novel machine learning model with Stacking Ensemble Learner for predicting emergency readmission of heart-disease patients. Decision Analytics Journal, 7, 100242. doi: https://doi.org/10.1016/j.dajour.2023.100242

Jiang, L., Xia, Z., Zhu, R., Gong, H., Wang, J., Li, J., & Wang, L. (2023). Diabetes risk prediction model based on community follow-up data using machine learning. Preventive Medicine Reports, 35, 102358. doi: https://doi.org/10.1016/j.pmedr.2023.102358

Katiyar, N., Thakur, H. K., & Ghatak, A. (2024). Recent advancements using machine learning & deep learning approaches for diabetes detection: a systematic review. E-Prime - Advances in Electrical Engineering, Electronics and Energy, 9, 100661. doi: https://doi.org/10.1016/j.prime.2024.100661

Khaire, U. M., & Dhanalakshmi, R. (2022). Stability of feature selection algorithm: A review. Journal of King Saud University - Computer and Information Sciences, 34(4), 1060–1073. doi: https://doi.org/10.1016/j.jksuci.2019.06.012

Kumari, S., Kumar, D., & Mittal, M. (2021). An ensemble approach for classification and prediction of diabetes mellitus using soft voting classifier. International Journal of Cognitive Computing in Engineering, 2, 40–46. doi: https://doi.org/10.1016/j.ijcce.2021.01.001

Mehta, R. I., Capuano, A. W., Biswas, R., Bennett, D. A., & Arvanitakis, Z. (2025). Permutations of cerebrovascular pathologies in older adults with and without diabetes. Cerebral Circulation - Cognition and Behavior, 8(March), 100381. doi: 10.1016/j.cccb.2025.100381

Mienye, I. D., Obaido, G., Jere, N., Mienye, E., Aruleba, K., Emmanuel, I. D., & Ogbuokiri, B. (2024). A survey of explainable artificial intelligence in healthcare: Concepts, applications, and challenges. Informatics in Medicine Unlocked, 51, 101587. doi: https://doi.org/10.1016/j.imu.2024.101587

Molnar, C., König, G., Bischl, B., & Casalicchio, G. (2024). Model-agnostic feature importance and effects with dependent features: a conditional subgroup approach. Data Mining and Knowledge Discovery, 38(5), 2903–2941. doi: 10.1007/s10618-022-00901-9

S, G., Venkata Siva Reddy, R., & Ahmed, M. R. (2024). Exploring the effectiveness of machine learning algorithms for early detection of Type-2 Diabetes Mellitus. Measurement: Sensors, 31, 100983. doi: https://doi.org/10.1016/j.measen.2023.100983

Wang, C., Xu, X., Luo, S., Luo, M., Li, S., & Si, J. (2025). Interpretable machine learning insights into the association between PFAS exposure and diabetes mellitus. Ecotoxicology and Environmental Safety, 302, 118569. doi: https://doi.org/10.1016/j.ecoenv.2025.118569

Wang, L., Wang, Y., & Chang, Q. (2016). Feature selection methods for big data bioinformatics: A survey from the search perspective. Methods, 111, 21–31. doi: https://doi.org/10.1016/j.ymeth.2016.08.014

Wong, P.-Y., Zeng, Y.-T., Su, H.-J., Lung, S.-C. C., Chen, Y.-C., Chen, P.-C., Hsiao, T.-C., Adamkiewicz, G., & Wu, C.-D. (2025). Effects of feature selection methods in estimating SO2 concentration variations using machine learning and stacking ensemble approach. Environmental Technology & Innovation, 37, 103996. doi: https://doi.org/10.1016/j.eti.2024.103996

Zhou, P., Liang, J., Yan, Y., Zhao, S., & Wu, X. (2024). Explainable feature selection and ensemble classification via feature polarity. Information Sciences, 676, 120818. doi: https://doi.org/10.1016/j.ins.2024.120818