A Systematic Literature Review on Malware Detection and Classification Models: Addressing Class Imbalance, Concept Drift, and Model Interpretability

Abubakar Bello Bodinga, Ahmed Baita Garko, Nurudeen Mahmud Ibrahim, Danlami Gabi

Abstract


Malware remains one of the most persistent threats to computer security, and its evolving nature poses challenges for detection and classification systems. This study systematically reviews malware detection and classification techniques, focusing on class imbalance, concept drift, and model interpretability. A systematic search of major scientific databases will be conducted following PRISMA guidelines. Studies will be screened, evaluated, and synthesized based on predefined inclusion/exclusion criteria. The review will provide insights into the effectiveness of existing approaches in handling imbalance, concept drift, and interpretability, as well as the role of deep learning models such as Graph Convolutional Network (GCN), Recurrent Neural Network (RNN), and Generative Adversarial Network (GAN) in malware detection. The findings aim to inform the design and evaluation of improved malware detection and classification models.


Full Text:

PDF

References


Ahmim, A., Maglaras, L., Ferrag, M. A., Derdour, M., & Janicke, H. (2019). A novel hierarchical intrusion detection system based on decision tree and rules-based models. In 2019 15th International Conference on Distributed Computing in Sensor Systems (DCOSS) (pp. 228–233). https://doi.org/10.1109/DCOSS.2019.00059

Aljurayyil, S., Al-Haj, A., & Farhat, W. (2022). Explainable deep learning for malware detection using SHAP. In Proceedings of the ACM Workshop on AI and Security (pp. 1–10). https://doi.org/10.1145/3564292.3564294

Alsuwat, E., Solaiman, S., & Alsuwat, H. (2023). Concept drift analysis and malware attack detection system using secure adaptive windowing. Computers, Materials & Continua, 75(2). https://doi.org/10.32604/cmc.2023.035126

Azeem, M., Khan, D., Iftikhar, S., Bawazeer, S., & Alzahrani, M. (2024). Analyzing and comparing the effectiveness of malware detection: A study of machine learning approaches. Heliyon, 10(1). https://doi.org/10.1016/j.heliyon.2023.e23574

Bach, M., Werner, A., Żywiec, J., & Pluskiewicz, W. (2017). The study of under- and over-sampling methods’ utility in analysis of highly imbalanced data on osteoporosis. Information Sciences, 384, 174–190. https://doi.org/10.1016/j.ins.2016.09.038

Ben Abdel Ouahab, I., Elaachak, L., & Bouhorma, M. (2023). Improve malware classifiers performance using cost-sensitive learning for imbalanced dataset. IAES International Journal of Artificial Intelligence, 12(4), 1836–1844. https://doi.org/10.11591/ijai.v12.i4.pp1836-1844

Bensaoud, A., Kalita, J., & Bensaoud, M. (2024). A survey of malware detection using deep learning. Machine Learning with Applications, 16, 100546. https://doi.org/10.1016/j.mlwa.2024.100546

Biecek, P., & Burzykowski, T. (2021). Local interpretable model-agnostic explanations (LIME). In Explanatory Model Analysis: Explore, Explain and Examine Predictive Models (Vol. 1, pp. 107–124).

Bilot, T., El Madhoun, N., Al Agha, K., & Zouaoui, A. (2024). A survey on malware detection with graph representation learning. ACM Computing Surveys, 56(11). https://doi.org/10.1145/3664649

Bisoyi, S. S., Panda, B., Patra, B., & Mishra, P. (2025). An ensemble technique for imbalanced multiclass malware classification by leveraging API call semantics. Discover Computing, 28(1), 100. https://doi.org/10.1007/s10791-025-09615-0

Bruzzese, R. (2024). Building visual malware dataset using VirusShare data and comparing machine learning baseline model to CoAtNet for malware classification. In Proceedings of the 2024 16th International Conference on Machine Learning and Computing (pp. 185–193). https://doi.org/10.1145/3651671.3651735

Buczak, A. L., & Guven, E. (2016). A survey of data mining and machine learning methods for cyber security intrusion detection. IEEE Communications Surveys & Tutorials, 18(2), 1153–1176. https://doi.org/10.1109/COMST.2015.2494502

Catak, F. O., Ahmed, J., Sahinbas, K., & Khand, Z. H. (2021). Data augmentation based malware detection using convolutional neural networks. PeerJ Computer Science, 7, e346. https://doi.org/10.7717/peerj-cs.346

Catak, F. O., & Yazi, A. F. (2021). A benchmark API call dataset for Windows PE. ResearchGate.

Catak, F. O., & Yazi, M. (2019). A benchmark API call dataset for Windows PE malware. arXiv. https://doi.org/10.48550/arXiv.1905.01999

Chen, J., Yuan, C., Li, J., Tian, D., Ma, R., & Jia, X. (2023). ELAMD: An ensemble learning framework for adversarial malware defense. Journal of Information Security and Applications, 75, 103508. https://doi.org/10.1016/j.jisa.2023.103508

Chollet, F. (2017). Deep learning with Python. Manning Publications.

Dunmore, A., Jang-Jaccard, J., Sabrina, F., & Kwak, J. (2023). Generative adversarial networks for malware detection: A survey. arXiv. https://arxiv.org/abs/2302.08558

F. Alshmarni, A., & A. Alliheedi, M. (2024). Enhancing malware detection by integrating machine learning with Cuckoo Sandbox. Journal of Information Security and Cybercrimes Research, 7(1), 85–92. https://doi.org/10.26735/wzng1384

Ferdous, J., Islam, R., Mahboubi, A., & Islam, M. Z. (2025). A survey on ML techniques for multi-platform malware detection: Securing PC, mobile devices, IoT, and cloud environments. Sensors, 25(4). https://doi.org/10.3390/s25041153

Fernandes, M., & Silva, J. (2021). Hybrid SMOTE and ensemble learning for malware family imbalance. Journal of Information Security. https://doi.org/10.1007/s12065-021-00678-1

Gilpin, L. H., Bau, D., Yuan, B. Z., Bajwa, A., Specter, M., & Kagal, L. (2018). Explaining explanations: An overview of interpretability of machine learning. In 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA) (pp. 80–89). https://arxiv.org/abs/1806.00069

Gulli, A., & Pal, S. (2017). Deep learning with Keras. Packt Publishing.

Gupta, D., & Rani, R. (2020). Improving malware detection using big data and ensemble learning. Computers & Electrical Engineering, 86, 106729. https://doi.org/10.1016/j.compeleceng.2020.106729

Gupta, I., Kumari, S., Jha, P., & Ghosh, M. (2024). Leveraging LSTM and GAN for modern malware detection. arXiv. https://doi.org/10.48550/arXiv.2405.04373

Han, Y., Wei, Z., & Huang, G. (2024). An imbalance data quality monitoring based on SMOTE-XGBOOST supported by edge computing. Scientific Reports, 14, 10151. https://doi.org/10.1038/s41598-024-60600-x.


Refbacks

  • There are currently no refbacks.