A data-driven decision model for selecting ML models in research software

Abstract

Context: The process of selecting machine learning models is complex for research software engineers, requiring careful consideration of factors like trainability and comprehensibility to ensure long-term usability and success. Objective: This study aims to develop and evaluate a data-driven decision model that supports research software engineers in systematically selecting suitable ML models for integration into research software. Method: A meta-model was created to guide model selection, drawing from systematic literature reviews, expert interviews, case studies, and design science. Each phase contributed valuable insights and refined the decision-making framework. Results: The study analyzed 43 models across 72 attributes, resulting in a taxonomy of ML paradigms, approaches, and domains. Key findings include trends in model selection, combinations, evaluation metrics, and datasets. The decision model was further refined through expert feedback and validated with 11 case studies. Contribution: This data-driven decision model supports research software engineers in selecting optimal ML models for integration into research software. Continued development is recommended to enhance its accuracy and applicability across varied research scenarios.