Selection for postgraduate training for general practice: The role of knowledge tests

Background, Postgraduate training for general practice is a legal requirement in most countries of the European Community, and includes posts in general practice as well as in hospitals. The effectiveness of the training has not been fully evaluated, and it is largely unknown whether the results are satisfactory or what the impact of the separate training components is - nor is it known which characteristics or prior achievements of the trainee influence the end-of-training performance. Aim. To determine the value of knowledge tests in the context of entry selection for postgraduate training in general practice. Methods. Three (equated) knowledge tests were administered during the two years' postgraduate training of 85 Dutch trainees. The first test was taken at entrance, the second eight months later, and the third shortly before the end of the entire training period. Complete data for 67 trainees were available for analysis. A multiple regression analysis was performed to estimate the predictive values of test I and test 2 scores, separately and in combination, for test 3 scores. Since the knowledge test may be used for selection purposes, the analysis was repeated using logistic regression with two pass/fail criteria: a 'minimum criterion' and an 'exellence criterion'. Results. Neither of the two analyses yielded a predictive value of test 1 that was high enough to warrant the use of knowledge tests in the context of entry selection. A 'below minimum' score on test 2 correlated 100% with a 'below minimum' score on test 3. However, the positive predictive value of an above minimum score on test 2 was only 86%. Conclusions. The knowledge tests used in this study are not suitable in the context of entry selection. However, trainees that score 'below minimum' after eight months of training may be regarded as 'at risk' in that they will probably score 'below minimum' at the end of training.