Budget-aware random testing with T3: benchmarking at the SBST2016 testing tool contest

Abstract

Random testing has the advantage that it is usually fast. An interesting use case is to use it for bulk smoke testing, e.g. to smoke test a whole project. However, on a large project, even with random testing it may still take hours to complete. To optimize this, we have adapted an automated random testing tool called T3 so that it becomes aware of the time budget we set for a given target class. Test suites are now generated incrementally, and their refinements are adaptively scheduled towards maximizing the coverage, given the remaining time. This paper presents an evaluation of the performance of this adaptation, using the benchmark provided by the SBST 2016 Java Unit Testing Tool Contest.