Navigating through Work Items in Issue Tracking Systems via Natural Language Queries

Abstract

Issue Tracking Systems (ITSs) serve multiple purposes in software development: they represent requirements and bugs, facilitate communication between team members, and support project management by assigning tasks. As software systems and projects increase in complexity, the number of work items and their relationships grows considerably, forming a complex network that is difficult and time-consuming for team members to navigate using traditional ITSs. Motivated by the interest of two companies in exploring the use of Artificial Intelligence in software development, we introduce the GraphRAG Dialogue Insights (GDI) framework. GDI uses knowledge graphs, Retrieval-Augmented Generation (RAG), and Large Language Models (LLMs) to enable users to query work items in natural language (NL). In addition to a NL response, GDI returns supporting information to help users understand how the LLMs queried the knowledge graph. We empirically validate GDI through two proof-of-concept implementations, each addressing a use case: i) onboarding and ii) trace link recovery. Initial user-centered validations indicate that practitioners find the system intuitive, useful, and accurate in generating responses. Moreover, they express willingness to adopt the system and identify additional potential use cases, including impact analysis, troubleshooting, and issue resolution. However, further refinement is required to deploy the system in practice.