How Effective Is Automated Trace Link Recovery in Model-Driven Development?

Abstract

[Context and Motivation] Requirements Traceability (RT) aims to follow and describe the lifecycle of a requirement. RT is employed either because it is mandated, or because the product team perceives benefits. [Problem] RT practices such as the establishment and maintenance of trace links are generally carried out manually, thereby being prone to mistakes, vulnerable to changes, time-consuming, and difficult to maintain. Automated tracing tools have been proposed; yet, their adoption is low, often because of the limited evidence of their effectiveness. We focus on vertical traceability that links artifacts having different levels of abstraction. [Results] We design an automated tool for recovering traces between JIRA issues (user stories and bugs) and revisions in a model-driven development (MDD) context. Based on existing literature that uses process and text-based data, we created 123 features to train a machine learning classifier. This classifier was validated via three MDD industry datasets. For a trace recommendation scenario, we obtained an average F 2 -score of 69% with the best tested configuration. For an automated trace maintenance scenario, we obtained an F 0.5 -score of 76%. [Contribution] Our findings provide insights on the effectiveness of state-of-the-art trace link recovery techniques in an MDD context by using real-world data from a large company in the field of low-code development.