On Process Discovery Experimentation: Addressing the Need for Research Methodology in Process Discovery

Abstract

Process mining aims to derive insights into business processes from event logs recorded from information systems. Process discovery algorithms construct process models that describe the executed process. With the increasing availability of large-scale event logs, process discovery has shifted towards a data-oriented research discipline, aiming to design algorithms that are applicable and useful in practice. This shift has revealed a fundamental problem in process discovery research: Currently, contributions can only be considered in isolation. Researchers conduct experiments to show that they move the field forward, but due to a lack of reliability and validity, the individual contributions are hard to generalize. In this article, we argue that one reason for these problems is the lack of conventions or standards for experimental design in process discovery. Hence, we propose “process discovery engineering”: a research methodology for process discovery, consisting of a shared terminology and a checklist for conducting experiments. We demonstrate its applicability by means of an example experimental evaluation of process discovery algorithms and discuss the implications of the methodology on the field. This article is not meant to be prescriptive but to invite and encourage the community to contribute to this discussion to advance the field as a whole.