The Pooled Data Approach for the Estimation of Split-Ballot Multitrait-Multimethod Experiments

Saris, Satorra and Coenders (2004) proposed a new approach for estimating the quality of survey questions, which combines the advantages of the following two existing approaches: the multi-trait-multi-method (MTMM) approach and the split-ballot (SB) approach. In practice, this new approach led to frequent occurrences of non-convergence and improper solutions in the case of a two-group design (Revilla & Saris, 2013). This was very problematic because this approach had been used in many experiments to determine the reliability, validity and method effects of questions that are used in the European Social Survey, which is used in approximately 30 countries. In this article, we propose a new estimation procedure, which we call estimation using pooled data (EUPD), to overcome the problems with this experimental design. We show that EUPD can also be applied in a general setting of structural equation modeling when the sample size is too small for precise estimation but several similar data sets are available.