Test case prioritization using transfer learning in continuous integration environments

Abstract

Continuous integration (CI) systems use automated tests to verify software builds and optimize the testing process through test case prioritization (TCP). Recent research studies on TCP in CI environments have employed machine learning (ML) techniques to address the dynamic nature of CI. However, the performance of ML for TCP may decrease because of the low volume of data or less failure rate, whereas using existing data with similar patterns from other domains can be valuable. Therefore, this thesis investigates the potential of transfer learning (TL) algorithms to improve test case failure prediction and prioritization in CI environments. We conduct a comparative analysis of traditional TL algorithms to improve test failure prediction of large-scale industrial projects. Our experimental results show that parameter-based TL is most effective, and its usefulness is further emphasized by the scarcity of publicly available large-scale CI datasets due to data privacy regulations. We also present a new technique called TCP TB that prioritizes test cases using the prediction probability of test failures generated by a parameter-based transfer learning algorithm called TransBoost. We compare TCP TB’s performance with different ML approaches and CI-RTP/S on 24 study subjects and demonstrate that TCP TB outperforms them, improving TCP performance in 82.61% of the cases.