Mixed IDEAs

Abstract

Building and using probabilistic models to perform stochastic optimization in the case of continuous random variables, has so far been limited to the use of factorizations as the structure of probabilistic models Furthermore, the only probability density function (pdf) that has been successfully tested on a multiple of problems, is the normal pdf The normal pdf however strongly generalizes the data and cannot cope with non-linear interactions among the samples In this paper we show how clustering algorithms can be used to overcome this problem We also show how the normal mixture pdf can be used m the case of a general factorization instead of the normal pdf We formalize the notion of a probabilistic model and propose to use two practical instances for the model structure, which are the factorization and the mixture of factorizations We propose to use metrics to find good factorizations and thereby eliminate a complexity parameter K that was required in previous continuous approaches in the case of a general factorization We also show the background of the metrics through general model selection on the basis of likelihood maximization, which demonstrates their connection with previously used factorization selection algorithms We use the IDEA framework for iterated density estimation evolutionary algorithms to construct new continuous evolutionary optimization algorithms based on the described techniques Then performance is evaluated on a set of well known epistatic continuous optimization problem.