Towards greater validity in Schwartz's portrait values indicator using experimental research

In Schwartz's 21-item portrait values questionnaire (PVQ) each item consists of two statements which refer to a single underling value. In each of the 21 items the respondents are asked to give a single response to the two statements. Anecdotal evidence and cognitive response theory alert us to potential measurement error in this two-statements-single-response approach. Respondents might be influenced by different themes contained in the statements and their answers might not be comparable. This paper addresses the question: Do the responses to the 21 items in Schwartz's PVQ (where each item contains two statements) differ significantly when two separate responses are allowed per item (in the split version) compared to when a single response is allowed (in the combined version)? In order to answer this research question we adopted an experimental design in a two-wave panel study. In the first wave we used Schwartz's combined version of the PVQ. In the second wave we split the two statements and treated each statement as a separate item, thus requiring responses to each statement. Data was collected from Sociology classes at two universities: one in Austria (n = 52) and the other in South Africa (n = 61). We used statistical and non-statistical methods of analysis. The overall statistical assessment (z test) supports the split version although not all the various z test results unanimously concur. The non-statistical assessment does not support either the split version or the combined version. These mixed results necessitate further interrogation of the continued use of the combined version in the PVQ.