Generating Business Process Models with Open Source Large Language Models Using Instruction Tuning

Abstract

Domain models play a central role in software development. They support communication, foster collaboration, and serve as references throughout the process. They also allow for automated analysis and can be used for code generation in model-driven development. Despite these benefits, software practitioners often neglect building domain models because they demand significant time, deep subject-matter expertise, and advanced modeling skills. To address these challenges and automatically generate domain models from text, we leverage generative large language models and apply instruction tuning to further enhance their performance. We demonstrate the applicability of our approach by tuning an open-source large language model to generate dynamic domain models as business process models and comparing its performance with off-the-shelf models. The results show that the instruction-tuned model outperforms off-the-shelf models in not only the key text-similarity metrics, BLEU, ROUGE, and METEOR, but also in Graph Edit Distance that captures structural correctness. Our qualitative assessment confirms that industry experts find the outputs accurate and useful. These experts demonstrate a willingness to integrate the system into their workflow. Our study demonstrates the first application of instruction tuning to business process model generation and highlights the potential for domain-specific adaptation of large language models.