Feature Selection and Machine Learning Based Software Defect Prediction: A Performance Evaluation

Abstract

Software Defect Prediction (SDP) is a vital technique for enhancing software quality and reducing development and maintenance costs by identifying fault-prone components at early stages of the software life cycle. Despite significant progress in machine learning–based approaches, limited attention has been given to understanding which software features most strongly influence prediction performance and how different algorithms compare under standardized evaluation. This study investigates feature identification and evaluates the performance of several machine learning classifiers for SDP using the JM1 dataset from the NASA PROMISE repository, which contains 10,885 software modules described by McCabe and Halstead complexity metrics. Data preprocessing was performed using correlation-based feature selection and K-means clustering to reduce redundancy, mitigate multicollinearity, and improve data representation. Four classifiers Support Vector Machine, Naïve Bayes, Random Forest, and AdaBoost together with a stacking ensemble model were implemented and evaluated using accuracy, precision, recall, and F1-score, both with and without Particle Swarm Optimization. Experimental results indicate that the Random Forest classifier consistently outperformed the other models, achieving up to 99.91% accuracy and superior precision, recall, and F1-score, while Naïve Bayes produced the lowest performance. The ensemble approach, particularly the combination of Random Forest and AdaBoost, further improved robustness. Feature analysis revealed that line count of code, cyclomatic complexity, essential complexity, total operators and operands, program volume, and program length are the most influential predictors of software defects. These findings demonstrate that careful feature selection combined with robust ensemble-based learning models can significantly improve the reliability and effectiveness of software defect prediction systems.