Evaluation of techniques for automated classification and artery quantification of the circle of Willis on TOF-MRA images: The CROWN challenge

Abstract

Assessing risk factors for intracranial aneurysm (IA) development on images is crucial for early detection of high-risk cases. IAs often form at bifurcations within the circle of Willis (CoW), but manual assessment of these arteries is both time-consuming and susceptible to inconsistencies. Previous studies on imaging markers for IA development lack sufficient evidence for clinical implications, highlighting the need for automated methods to assess CoW morphology. No systematic approach currently exists to identify the best methodological strategies. To address this, we organized a scientific challenge to compare various techniques against a clinical reference standard. Participants were tasked with (1) automated classification of CoW anatomical variants and (2) automated prediction of CoW artery diameters and bifurcation angles. We provided 300 TOF-MRA scans for training and another 300 for testing, all manually annotated. Submissions were evaluated using balanced accuracy, mean absolute error, and Pearson correlation coefficient metrics. This paper provides a detailed analysis of the results from six participating teams. The findings show that various methods may be suitable for automated CoW assessment, but that these need further improvement to meet clinical standards. The challenge remains open for future submissions, offering a benchmark for new techniques.