Negative log-likelihood and statistical hypothesis testing as the basis of model selection in IDEAs

Abstract

In this paper, we analyze the most prominent features m model selection criteria that have been used so far m iterated density estimation evolutionary algorithms (IDEAs, EDAs, PMBGAs) These algorithms build probabilistic models and estimate probability densities based upon a selection of available points We show that the negative log likelihood is a basis of the inference features when the Kullback-Leibler divergence is used We show how previously found to be problematic issues in the case of continuous random variables can be resolved by starting from the derived basics By doing so we have a probabilistic model search metric that can be justified through the use of statistical hypothesis tests This in turn reduces the need for additional complexity penaltie.