Causal inference in multilevel designs
The general theory of causal effects (Steyer et al., 2009) is used to develop a theory of causal inference for multilevel designs - i.e., for designs in which the effects of treatments are evaluated on units nested within clusters - that extends and consolidates previous approaches. Two multilevel causality spaces for different classes of multilevel designs are used to define true-effect variables, average causal effects, conditional causal effects and prima-facie effects. Unbiasedness, as the weakest condition under which average and conditional causal effects are identified, and its sufficient conditions are outlined. Next, stability assumptions for causal inference in multilevel designs are discussed in relation to the general theory of causal effects and a taxonomy of multilevel designs is introduced. Building upon this theoretical framework, the generalized analysis of covariance (ANCOVA), that extends the conventional multilevel ANCOVA by identifying the average causal effect in the presence of interactions, is developed for non-randomized multilevel designs with treatment assignment at unit- and at the cluster-level. Two simulation studies tested several statistical implementations of the generalized ANCOVAs. The results showed that contextual effects have to be taken into account in the specification of adjustment models, that predictors have to be modeled as stochastic to obtain correct standard errors of the average causal effects and that the unreliability of the empirical cluster means has to be accounted for in designs with treatment assignment at the cluster-level. The statistical methods studied in the simulations were applied to two empirical examples from educational research to demonstrate the implementations in practice. Finally, the scope of the general theory of causal effects, the advantages and disadvantages of the generalized ANCOVA and alternative adjustment methods are discussed and an overview of further research needs is given.