Random Testing with Austere Budgeting in T3: Benchmarking at SBST2019 Testing Tool Contest

Abstract

Random testing has the benefit of being simple to implement. However, if left unmanaged, random testing can easily exhaust the allocated computation time without being effective. In a previous work we extended an automated random testing tool called T3 with a budget management algorithm. Given a time budget and a target class with multiple methods to test, the algorithm tries to spread the budget over the methods by dividing it into multiple budget slices. While this guarantees a certain measure of fairness, the algorithm was programmed towards using up the budget. This does deliver extra coverage, but beyond a certain point this becomes quite costly. In an attempt to improve this we consider an austere budgeting policy as an alternative. While this work is still in progress, we decided to participate in the SBST Unit Testing Tool Competition 2019 to obtain insight on the performance of the policy. This paper discussed its results.