Model selection criteria : how to evaluate order restrictions

Abstract

Researchers often have ideas about the ordering of model parameters. They frequently have one or more theories about the ordering of the group means, in analysis of variance (ANOVA) models, or about the ordering of coefficients corresponding to the predictors, in regression models.A researcher might have the expectation that the parameters exhibit an increasing trend: θ1 ≤ … ≤ θk, where θj is, for example, the mean of group j or the regression coefficient corresponding to predictor j, for j = 1, …, k. These types of restrictions are called order restrictions or inequality constraints. Although researchers have directional expectations about the parameters, they usually evaluate it in an exploratory manner. That is, they examine all possible configurations of groups of parameters being equal or a subset of these often based on the ordering of the sample parameters. Hence, they inspect which groups of parameters are equal and which are not, while they do not investigate the ordering of the (groups of) parameters. Irrespective of the resulting configuration, it generally does not give insight into the hypotheses of interest, that is, the directional hypothesis. There exist, however, methods that can be used to evaluate order restrictions directly, the so-called confirmatory methods. Why are they not used then? Probably because little is known about these methods, most of them can only be applied to a limited set of models, and there is no software available to employ them. This dissertation provides insight in evaluating order restrictions with (confirmatory) model selection techniques: 1) It compares exploratory methods with their confirmatory counterparts in ANOVA models. Three types of methods are distinguished: hypothesis testing, model selection using information criteria, and Bayesian model selection; 2) It extends a confirmatory model selection technique, such that it can be applied to i) a more general form of (order) restrictions, ii) multivariate normal linear models, and iii) small samples; 3) It describes how information criteria should be calculated in the presence of missing data; 4) It proposes a method of combining statistical evidence for the hypotheses of interest from multiple studies regarding the same concept; and 5) It offers software for each of the discussed methods