A GOOD MATCH IS HARD TO FIND - COMMENT

R. E. Mayer, H. Tajika, and C. Stanley (1991) reported that U.S. students scored higher on problem-solving tasks than their Japanese peers when matched on computation skills. Contrary to Mayer et al., we believe that these results are artifacts that reveal little about the nature of Japanese and U.S. education. Drawing on work by P. E. Meehl (1970), we use Mayer et al.'s study to illustrate pitfalls of matching cases in an attempt to create equivalent groups of subjects. Besides well-known regression effects, matching (whether by selecting subjects or by statistical correction) is subject to three major criticisms: (a) It produces systematically unmatched samples with respect to other meaningful variables, (b) it results in unrepresentative samples from which it is hard to draw meaningful generalizations, and (c) causal interpretation becomes nearly impossible. These pitfalls must be considered when interpreting the results of ex post facto matched designs.