DOI:
https://doi.org/10.64539/msts.v2i2.2026.505Keywords:
Predictive Maintenance, Machine Learning, Random Forest, Industry 4.0, SMOTE, Class imbalanceAbstract
The transition toward Industry 4.0 has established predictive maintenance (PdM) as a critical strategy for optimizing operational efficiency and reducing unexpected downtime through real-time sensor data analytics. However, the practical implementation of PdM is frequently hindered by the extreme class imbalance inherent in industrial datasets, where equipment failure events are significantly rarer than normal operating hours. This study presents a comprehensive comparative evaluation of five machine learning algorithms—Random Forest (RF), Decision Tree (DT), Support Vector Machine (SVM), K-Nearest Neighbors (KNN), and Logistic Regression (LR)—utilizing the AI4I 2020 Predictive Maintenance Dataset. By implementing a rigorous preprocessing pipeline that employs Z-score normalization for feature scaling and the Synthetic Minority Over-sampling Technique (SMOTE) to mitigate majority-class bias within the training partition. The models were evaluated on a hold-out test set of 2,000 instances using metrics including accuracy, precision, recall, and the F1-score. Results indicate a pronounced “Accuracy Paradox” in linear and distance-based models; while KNN and Logistic Regression achieved deceptively high accuracies exceeding 96%, they failed to reliably detect actual failure signatures. In contrast, Random Forest emerged as the superior architecture, achieving an F1-score of 97.10% and a recall of 98.53%, correctly identifying 67 out of 68 failure instances. This research concludes that F1-score and Recall are more vital indicators of industrial reliability than simple accuracy. The findings provide a standardized framework for selecting ensemble-based classifiers to support scalable, data-driven maintenance strategies in modern smart manufacturing environments.
References
[1] R. Y. Zhong, X. Xu, E. Klotz, S. T. Newman, “Intelligent Manufacturing in the Context of Industry 4.0: A Review” Engineering, vol. 3, no. 5, pp. 616-630, 2017. https://doi.org/10.1016/J.ENG.2017.05.015.
[2] T. P. Carvalho, F. A. A. M. N. Soares, R. Vita, R. P. Francisco, J. P. Basto, and S. G. Alcalá, “A systematic literature review of machine learning methods applied to predictive maintenance,” Comput. Ind. Eng., vol. 137, Art. no. 106024, 2019. https://doi.org/10.1016/j.cie.2019.106024.
[3] M. Molęda, B. Małysiak-Mrozek, W. Ding, V. Sunderam, and D. Mrozek, “From Corrective to Predictive Maintenance—A Review of Maintenance Approaches for the Power Industry,” Sensors, vol. 23, no. 13, Art. no. 5970, 2023. https://doi.org/10.3390/s23135970.
[4] C. Tsallis, P. Papageorgas, D. Piromalis, and R. A. Munteanu, “Application-wise review of machine learning-based predictive maintenance: Trends, challenges, and future directions,” Appl. Sci., vol. 15, no. 9, Art. no. 4898, 2025. https://doi.org/10.3390/app15094898.
[5] S. Kaparthi and D. Bumblauskas, “ Designing predictive maintenance systems using decision tree-based machine learning techniques,” Int. J. Qual. Reliab. Manag., vol. 37, no. 4, pp. 659–686, 2020. https://doi.org/10.1108/IJQRM-04-2019-0131.
[6] P. Stodola and J. Stodola, “Model of predictive maintenance of machines and equipment,” Appl. Sci., vol. 10, no. 1, Art. no. 213, 2020. https://doi.org/10.3390/app10010213.
[7] A. de Giorgio, G. Cola, and L. Wang, “Systematic review of class imbalance problems in manufacturing,” Journal of Manufacturing Systems, vol. 71, pp. 620–644, 2023. https://doi.org/10.1016/j.jmsy.2023.10.014.
[8] S. Matzka, “Explainable Artificial Intelligence for Predictive Maintenance Applications,” in Proc. 3rd Int. Conf. Artificial Intelligence for Industries (AI4I), 2020. https://doi.org/10.1109/AI4I49448.2020.00023.
[9] T. Zonta et al., “ Predictive maintenance in the Industry 4.0: A systematic literature review,” Comput. Ind. Eng., vol. 150, 2020. https://doi.org/10.1016/j.cie.2020.106889.
[10] M. Paolanti et al., “ Machine Learning approach for Predictive Maintenance in Industry 4.0,” in 14th IEEE/ASME International Conference on Mechatronic and Embedded Systems and Applications (MESA), 2018. https://doi.org/10.1109/MESA.2018.8449150.
[11] C. Cortes and V. Vapnik, “Support-vector networks,” Mach. Learn., vol. 20, no. 3, pp. 273–297, 1995. https://doi.org/10.1007/BF00994018.
[12] S. Ayvaz and K. Alpay, “Predictive maintenance system for production lines in manufacturing: A machine learning approach using IoT data in real-time,” Expert Systems with Applications., vol. 173, 2021. https://doi.org/10.1016/j.eswa.2021.114598.
[13] P. K. Mannepalli, “A comparative study of machine learning algorithms for predictive maintenance in manufacturing systems,” Global J. Eng. Technol. Res., vol. 1, no. 1, p. 8, 2025. https://gjetr.ep-journals.org/index.php/gjetr/article/view/2.
[14] F. Chen, G. Zhou, G. Wang, F. Li, C. Chen, and J. Zhang, “ Deep Learning Based Prediction of Automotive Maintenance Projects,” Intell. Transp. Eng., vol. 72, pp. 186–193, 2025. https://doi.org/10.3233/ATDE250419.
[15] D. M. W. Powers, “ Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation,” arXiv preprint arXiv:2010.16061, 2020. https://doi.org/10.48550/arXiv.2010.16061.
[16] J. Korstanje, Advanced Forecasting with Python: With State-of-the-Art-Models Including LSTMs, Facebook’s Prophet, and Amazon’s DeepAR, 2021. https://doi.org/10.1007/978-1-4842-7159-9.
[17] Z. M. Çınar et al., “Machine learning in predictive maintenance towards sustainable smart manufacturing in Industry 4.0,” Sustainability, vol. 12, no. 19, Art. no. 8211, 2020. https://doi.org/10.3390/su12198211.
[18] D. Chicco and G. Jurman, “The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation,” BMC Genomics, vol. 21, Art. no. 6, 2020. https://doi.org/10.1186/s12864-019-6413-7.
[19] M. A. Bouke, and A. Abdullah, “An empirical study of pattern leakage impact during data preprocessing on machine learning-based intrusion detection models reliability”, Expert Systems with Applications, vol. 320, 2023. https://doi.org/10.1016/j.eswa.2023.120715.
[20] R. Agarwal, D. Kalel, H. Mohan, and R. Singh, “Sensor fault detection using machine learning technique for automobile drive applications,” in Proc. Nat. Power Electronics Conf. (NPEC), 2021. https://doi.org/10.1109/NPEC52100.2021.9672546.
[21] L. Wang, Z. Zhu, and X. Zhou, “ Dynamic predictive maintenance strategy for system remaining useful life prediction via deep learning ensemble method,” Reliab. Eng. Syst. Saf., vol. 245, Art. no. 110012, 2024. https://doi.org/10.1016/j.ress.2024.110012.
[22] H. He and E. A. Garcia, “Learning from imbalanced data,” IEEE Trans. Knowl. Data Eng., vol. 21, no. 9, pp. 1263–1284, 2009. https://doi.org/10.1109/TKDE.2008.239.
[23] R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, 2nd ed. Cambridge, MA, USA: MIT Press, 2018. https://mitpress.mit.edu/9780262039246/reinforcement-learning/.
[24] R. Kohavi and G. H. John, “Wrappers for feature subset selection,” Artif. Intell., vol. 97, nos. 1–2, pp. 273–324, 1997. https://doi.org/10.1016/S0004-3702(97)00043-X.
[25] E. Alpaydin, Introduction to Machine Learning, 4th ed. Cambridge, MA, USA: MIT Press, 2020. https://books.google.co.id/books?id=uZnSDwAAQBAJ.
[26] T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning, 2nd ed. New York, NY, USA: Springer, 2009. https://doi.org/10.1007/978-0-387-84858-7.
[27] N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, “SMOTE: Synthetic minority over-sampling technique,” J. Artif. Intell. Res., vol. 16, pp. 321–357, 2002. https://doi.org/10.1613/jair.953.
[28] A. Fernández, S. García, F. Herrera, and N. V. Chawla, “SMOTE for Learning from Imbalanced Data: Progress and Challenges, Marking the 15-year Anniversary,” J. Artif. Intell. Res., vol. 61, pp. 863–905, 2018. https://doi.org/10.1613/jair.1.11192.
