External validation of WGS-based antimicrobial susceptibility prediction tools, KOVER-AMR and ResFinder 4.1, for Escherichia coli clinical isolates

Abstract

Objective: To externally validate whole genome sequence-antimicrobial susceptibility testing phenotype prediction tools KOVER-AMR and ResFinder 4.1 for Escherichia coli clinical isolates from Dutch routine care. Methods: A random sample of 234 E. coli and 283 third generation cephalosporin-resistant E. coli isolates from urine and blood were collected (2014–2017). Culture-antimicrobial susceptibility testing was performed using VITEK 2 and BD Phoenix. Sequences were used as input for KOVER-AMR-SCM, KOVER-AMR-CART, and ResFinder 4.1. The concordance, major error rate (MER), and very major error rate (VMER) were calculated, with subsequent comparison to U.S. Food and Drug Administration (FDA) criteria (MER ≤3% and VMER with a 95% confidence interval ≤1.5–≤7.5%). Results: ResFinder 4.1 performed better than KOVER-AMR-models; however, neither tool achieved overall (V)MERs below FDA criteria. KOVER-AMR-SCM, KOVER-AMR-CART, and ResFinder 4.1, MER (cumulative all antimicrobials) were: 5.1% (4.4–5.9), 4.3% (3.6–5.0), and 5.1% (4.5–5.8), respectively. MERs ≤3% were achieved for 6 (SCM) and 5 (CART) of the 11 tested antimicrobials for KOVER-AMR-models and for 9/13 antimicrobials tested with ResFinder 4.1. KOVER-AMR-SCM, KOVER-AMR-CART, and ResFinder 4.1 cumulative VMERs were: 26% (24–28), 29% (27–31), and 11% (9.2–12). VMERs with a 95% CI ≤ 1.5–≤7.5 were only achieved for 4/13 tested antimicrobials with ResFinder 4.1. Discussion: In this study, whole genome sequence-antimicrobial susceptibility testing phenotype prediction tools KOVER-AMR and ResFinder 4.1 did not meet the FDA criteria needed for clinical diagnostic use in 517 E. coli clinical isolates from Dutch routine care. The tested tools should be further improved before they can be used for clinical decision making.