Assessing the performance of 38 machine learning models: The case of land consumption rates in Bavaria, Germany

Abstract

Machine learning (ML) is at the forefront of land-use change modeling. Due to numerous available ML approaches, the model choice is complex and usually based on ad hoc decisions, though informed through a few comparative studies that considered a limited number of models. This study contributes a comprehensive comparison of 38 ML models to examine land consumption rates (LCR) (i.e. the transition of landscapes to built-up areas). We modeled LCR for 2009–2015 in Bavaria, Germany, and predicted rates for 2015–2021 at a municipality level. To assess the performance of each approach, we measured the mean absolute error (MAE), the root-mean-square error (RMSE), and the coefficient of determination (R2) using crossvalidation. All algorithms consistently predicted that the land consumption rate for Bavaria will increase. eXtreme gradient boosting decision trees performed best with respect to the RMSE (0.500) and R2 (0.183), while the support vector machine with polynomial kernel has the lowest MAE (0.288). The generalized additive model and the random forest models also performed well. We recommend these ML approaches for future land consumption and land-use change studies. A poor performance was found for recursive partitioning by decision trees, self-organizing maps, and partitioning using deletion, substitution, and addition moves.