Aligning evaluation approaches with context: reductionism, system thinking, and pragmatic synthesis

The Campbellian validity typology has been used as a foundation for outcome evaluation and for developing evidence-based interventions for decades. As such, randomized control trials were preferred for outcome evaluation. However, some evaluators disagree with the validity typology's argument that randomized controlled trials as the best design for outcome evaluation, and with its strategy to maximize internal validity at the expense of external validity. The debates over these issues have been ongoing and intensive. This paper proposes a fresh idea for approaches to address this controversy, by bringing problem-solving philosophies into discussions. Three problem-solving philosophies have been used to develop intervention programs: reductionism, systems thinking, and pragmatic synthesis. This paper argues the validity typology is appropriate for evaluating reductionism-based programs only, but not systems thinking- or pragmatic synthesis-based programs. Mismatched applications of research designs in evaluation are sources of confusion and controversies. This insight indicates a pressing need for developing evaluation theories and approaches for systems thinking and pragmatic synthesis. Because the majority of community programs are pragmatic synthesis-based programs, this paper devotes to developing such solutions.