ANOVA and Alternatives for Causal Inferences

Analysis of variance (ANOVA) is a procedure frequently used for analyzing experimental and quasi-experimental data in psychology. Nonetheless, there is confusion which subtype to prefer for unbalanced data. Much of this confusion can be prevented when an adequate hypothesis is formulated at first. In the present paper this is done by using a theory of causal effects. This is the starting point for the following simulation study done on unbalanced two-way designs. Simulated data sets differed in the presence of an (average) effect, the degree of interaction, total sample size, stochasticity of subsample sizes and if there was confounding between the two factors (i.e. experimental vs. quasi-experimental design). Different subtypes of ANOVA as well as other competing procedures from the research on causal effects were compared with regard to type-I-error rate and power. Results suggest that different types of ANOVA should be used with care, especially in quasi-experimental designs and when there is interaction. Procedures developed within the research on causal effects are feasible alternatives that may serve better to answer meaningful hypotheses.