Decision analytic modeling was useful to assess the impact of a prediction model on health outcomes before a randomized trial

Abstract

OBJECTIVE: To demonstrate how decision analytic models (DAMs) can be used to quantify impact of using a (diagnostic or prognostic) prediction model in clinical practice and provide general guidance on how to perform such assessments. STUDY DESIGN AND SETTING: A DAM was developed to assess the impact of using the HEART score for predicting major adverse cardiac events (MACE). Impact on patient health outcomes and health care costs was assessed in scenarios by varying compliance with and informed deviation (ID) (using additional clinical knowledge) from HEART score management recommendations. Probabilistic sensitivity analysis was used to assess estimated impact robustness. RESULTS: Impact of using the HEART score on health outcomes and health care costs was influenced by an interplay of compliance with and ID from HEART score management recommendations. Compliance of 50% (with 0% ID) resulted in increased missed MACE and costs compared with usual care. Any compliance combined with at least 50% ID reduced both costs and missed MACE. Other scenarios yielded a reduction in missed MACE at higher costs. CONCLUSION: Decision analytic modeling is a useful approach to assess impact of using a prediction model in practice on health outcomes and health care costs. This approach is recommended before conducting an impact trial to improve its design and conduct.