Validation of artificial intelligence software for automatic calcium scoring in cardiac and chest computed tomography

Abstract

OBJECTIVE: Coronary artery calcium scoring (CACS), i.e. quantification of Agatston (AS) or volume score (VS), can be time consuming. The aim of this study was to compare automated, artificial intelligence (AI)-based CACS to manual scoring, in cardiac and chest CT for lung cancer screening. METHODS: We selected 684 participants (59 ± 4.8 years; 48.8 % men) who underwent cardiac and non-ECG-triggered chest CT, including 484 participants with AS > 0 on cardiac CT. AI-based results were compared to manual AS and VS, by assessing sensitivity and accuracy, intraclass correlation coefficient (ICC), Bland-Altman analysis and Cohen's kappa for classification in AS strata (0;1-99;100-299;≥300). RESULTS: AI showed high CAC detection rate: 98.1% in cardiac CT (accuracy 97.1%) and 92.4% in chest CT (accuracy 92.1%). AI showed excellent agreement with manual AS (ICC:0.997 and 0.992) and manual VS (ICC:0.997 and 0.991), in cardiac CT and chest CT, respectively. In Bland-Altman analysis, there was a mean difference of 2.3 (limits of agreement (LoA):-42.7, 47.4) for AS on cardiac CT; 1.9 (LoA:-36.4, 40.2) for VS on cardiac CT; -0.3 (LoA:-74.8, 74.2) for AS on chest CT; and -0.6 (LoA:-65.7, 64.5) for VS on chest CT. Cohen's kappa was 0.952 (95%CI:0.934-0.970) for cardiac CT and 0.901 (95%CI:0.875-0.926) for chest CT, with concordance in 95.9 and 91.4% of cases, respectively. CONCLUSION: AI-based CACS shows high detection rate and strong correlation compared to manual CACS, with excellent risk classification agreement. AI may reduce evaluation time and enable opportunistic screening for CAC on low-dose chest CT.