Automatic chord label personalization through deep learning of shared harmonic interval profiles

Abstract

Current automatic chord estimation systems are trained and tested using datasets that contain single reference annotations, i.e., for each corresponding musical segment (e.g., audio frame or section), the reference annotation contains a single chord label. Nevertheless, theoretical insights on harmonic ambiguity from harmony theory, experimental studies on annotator subjectivity in harmony annotations, and the availability of vast amounts of heterogeneous (subjective) harmony annotations in crowd-sourced repositories make the notion of a single-harmonic “ground truth” reference annotation a tenuous one. Recent studies suggest that subjectivity is intrinsic to harmonic reference annotations that should be embraced in automatic chord estimation rather than resolved. We introduce the first approach to automatic chord label personalization by modeling annotator subjectivity through harmonic interval-based chord representations. We integrate these representations from multiple annotators and deep learn them from audio. From a single trained model and the annotators’ chord-label vocabulary, we can accurately personalize chord labels for individual annotators. Furthermore, we show that chord personalization using multiple reference annotations outperforms using just a single reference annotation. Our results show that annotator subjectivity should inform future research on automatic chord estimation to improve the state of the art.