On the construct and criterion validity of problem-solving scenarios based on the prediction of supervisor assessment of job performance

Computer-based problem-solving scenarios have been well established in German-language psychology and are also relied upon in personnel selection. The present article focuses on the so far largely neglected question of how much validity such scenarios might claim. Problem solving scenarios cannot make a well-founded claim for content validity. However, in some studies systematic relations have been demonstrated between problem-solving achievement and criteria of job performance. This article deals with the question whether these scenarios measure a new construct, "problem-solving capacity" or whether the criterion-relevant part of the measurement taps variance that can also be accessed by traditional intelligence and knowledge tests. In the empirical section the findings of a study are presented where intelligence and knowledge tests as well as two computer-based problem-solving scenarios were used. At first, construct validity of control performance was analyzed based on data from 104 policemen. It turned out that a substantial part of problem-solving achievement could be attributed to intelligence and knowledge. In a second step, the predictability of supervisor assessment of job performance by intellectual capacity, knowledge and problem-solving capacity was compared for a subgroup of 73 persons from the sample. Taking problem-solving capacity into account did not make a significant incremental contribution to the prediction of job performance by intelligence and knowledge. The final discussion deals with the ways computer-based problem-solving scenarios might contribute to diagnostic purposes.