Test Prioritization based on Change Sensitivity: an Industrial Case Study

Abstract

In the context of service-based systems, applications access software services, either home-built or third-party, to orchestrate their functionality. Since such services evolve independently from the applications, the latter need to be tested to make sure that they work properly with the updated or new services. In a previous work we have proposed a test prioritization approach that ranks test cases based on their sensitivity to external service changes. The idea is to give priority to the tests that detect the highest number of artificial changes (mutations), because they have a higher chance of detecting real changes in external services. In this paper, we apply change-sensitivity based test prioritization to an industrial system from IBM within the FITTEST European project. Results indicate that the ranked test cases achieve automatically comparable performance as manual prioritization made by an experienced team.