Bringing Rigor to the Qualitative Evaluation of Process Mining Findings: An Analysis and a Proposal

Abstract

Before the findings of a process mining project can be turned into actionable insights or recommendations, it is essential to make sure that the findings are actually valid. Therefore, the evaluation of the findings is a crucial part of a successful process mining project. Current process mining methodologies, however, fall short in providing actionable support to perform such an evaluation. This is especially true when domain experts are involved. To close this gap, we performed a literature study considering all process mining case studies published in the last two decades. In total, we identified 244 candidate papers of which we analyzed 80 in depth. Based on this literature study, we found a need for a more systematic approach for qualitative evaluations in process mining projects where domain experts are involved. Therefore, we build on these results to propose six validation strategies, which originate from qualitative research. We believe that this proposal for more rigor in the evaluation phase of process mining projects helps to move the discipline forward.