Investigating an approach to identify and classify mutations based on the characteristics of mutations with machine learning algorithms
Zeinab Asghari
1
(Department of Computer Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran)
Bahman Arasteh
2
(Department of Computer Engineering, Tabriz Branch, Islamic Azad University, Tabriz, Iran)
Abbas Koochari
3
(Department of Computer Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran)
Keywords: software testing, mutation testing, equivalent mutants, test adequacy, mutation score.,
Abstract :
Software mutation testing is one of the effective methods to evaluate code quality and detect hidden errors. However, this method faces challenges such as producing equivalent mutants and the process being time-consuming. The test cases designed for software testing must have the necessary sufficiency. For this purpose, the criterion of mutation test score is used. One of the main issues related to software mutation testing is the generation of equivalent mutants. Equivalent mutants are mutants that do not change the behavior of the program and have the same output as the original program. Identifying these mutants can make the mutation testing process time-consuming and costly. It is also possible that these mutants are mistakenly classified as hard to kill mutants. While hard to kill mutants can be rejected by changing the test items and strengthening them. In this paper, we present an efficient method for classifying program mutants to identify and separate equivalent mutations from hard to kill mutants. Using machine learning algorithms, we intend to optimize this process and improve the efficiency and effectiveness of the method. This is done by extracting different features from the mutants and using them to train machine learning models.
1. Richard A. DeMillo, Richard J. Lipton, and Fred G. Sayward. "Hints on test data selection: Help for the practicing programmer." IEEE Computer, 11(4): 34-41, 1978.
2. Jeff Offutt and Ammei Lee. "An empirical evaluation of weak mutation." IEEE Transactions on Software Engineering, 24(8): 649-660, 1998.
3. Mark Harman. "The current state and future of search-based software engineering." In Proceedings of the 28th International Conference on Software Engineering (ICSE'06), pages 342-351. ACM, 2006.
4. Simon Poulding and James A. Jones. "An empirical study of the robustness of MacOS applications using random testing." In Proceedings of the 10th International Workshop on Dynamic Analysis (WODA'12), pages 35-40. IEEE, 2012.
5. López, J., Kushik, N., & Yevtushenko, N. (2018). Source Code Optimization using Equivalent Mutants. Inf. Softw. Technol., 103, 138-141. https://doi.org/10.1016/j.infsof.2018.06.013.
6. Ghiduk, A., Girgis, M., & Shehata, M. (2019). Employing Dynamic Symbolic Execution for Equivalent Mutant Detection. IEEE Access, 7, 163767-163777. https://doi.org/10.1109/ACCESS.2019.2952246.
7. Zeinab Asghari, Bahman Arasteh, and Abbas Koochari. 2024. Effective Software Mutation-Test Using Program Instructions Classification. J. Electron. Test. 39, 5–6 (Dec 2023), 631–657. https://doi.org/10.1007/s10836-023-06089-0
8. Ghiduk, A., Girgis, M., & Shehata, M. (2019). Employing Dynamic Symbolic Execution for Equivalent Mutant Detection. IEEE Access, 7, 163767-163777. https://doi.org/10.1109/ACCESS.2019.2952246.
9. Jammalamadaka, K., & Parveen, N. (2021). Equivalent mutant identification using hybrid wavelet convolutional rain optimization. Software: Practice and Experience, 52, 576 - 593. https://doi.org/10.1002/spe.3038.
10. Souza, B., & Gheyi, R. (2020). A Lightweight Technique to Identify Equivalent Mutants. . https://doi.org/10.5753/CBSOFT_ESTENDIDO.2020.14630.
11. Tenorio, M., Lopes, R., Fechine, J., Marinho, T., & Costa, E. (2019). Subsumption in Mutation Testing: An Automated Model Based on Genetic Algorithm. 16th International Conference on Information Technology-New Generations (ITNG 2019). https://doi.org/10.1007/978-3-030-14070-0_24.
12. Fernandes, L., Ribeiro, M., Gheyi, R., Delamaro, M., Guimarães, M., & Santos, A. (2022). Put Your Hands In The Air! Reducing Manual Effort in Mutation Testing. Proceedings of the XXXVI Brazilian Symposium on Software Engineering. https://doi.org/10.1145/3555228.3555233.
13. Abuljadayel, A., & Wedyan, F. (2018). An Approach for the Generation of Higher Order Mutants Using Genetic Algorithms. International Journal of Intelligent Systems and Applications, 10, 34-45. https://doi.org/10.5815/IJISA.2018.01.05.
14. Basile, D., Beek, M., Cordy, M., & Legay, A. (2020). Tackling the equivalent mutant problem in real-time systems: the 12 commandments of model-based mutation testing. Proceedings of the 24th ACM Conference on Systems and Software Product Line: Volume A - Volume A. https://doi.org/10.1145/3382025.3414966.
15. Naeem, M., Lin, T., Naeem, H., & Liu, H. (2020). A machine learning approach for classification of equivalent mutants. Journal of Software: Evolution and Process, 32. https://doi.org/10.1002/smr.2238.
16. Ayad, A., Marsit, I., Loh, J., Omri, M., & Mili, A. (2019). Estimating the Number of Equivalent Mutants. 2019 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), 112-121. https://doi.org/10.1109/ICSTW.2019.00039.
17. Ayad, A., & Mili, A. (2020). Automated Estimation of the Rate of Equivalent Mutants. 2020 International Conference on Computational Science and Computational Intelligence (CSCI), 1794-1799. https://doi.org/10.1109/CSCI51800.2020.00331.
18. Kazerouni, A., Davis, J., Basak, A., Shaffer, C., Servant, F., & Edwards, S. (2021). Fast and accurate incremental feedback for students' software tests using selective mutation analysis. J. Syst. Softw., 175, 110905. https://doi.org/10.1016/j.jss.2021.110905.
19. Devroey, X., Perrouin, G., Papadakis, M., Legay, A., Schobbens, P., & Heymans, P. (2018). Model-based mutant equivalence detection using automata language equivalence and simulations. J. Syst. Softw., 141, 1-15. https://doi.org/10.1016/j.jss.2018.03.010.
20. Marsit, I., Ayad, A., Kim, D., Latif, M., Loh, J., Omri, M., & Mili, A. (2021). The ratio of equivalent mutants: A key to analyzing mutation equivalence. J. Syst. Softw., 181, 111039. https://doi.org/10.1016/J.JSS.2021.111039.
21. Khan, R., & Amjad, M. (2019). Mutation-based genetic algorithm for efficiency optimisation of unit testing. Int. J. Adv. Intell. Paradigms, 12, 254-265. https://doi.org/10.1504/IJAIP.2019.10019862.
22. Petrović, G., Ivankovic, M., Fraser, G., & Just, R. (2021). Practical Mutation Testing at Scale: A view from Google. IEEE Transactions on Software Engineering, 48, 3900-3912. https://doi.org/10.1109/TSE.2021.3107634.
23. Naeem, M., Lin, T., Naeem, H., Ullah, F., & Saeed, S. (2019). Scalable Mutation Testing Using Predictive Analysis of Deep Learning Model. IEEE Access, 7, 158264-158283. https://doi.org/10.1109/ACCESS.2019.2950171.