Prediction of general practice licensure/certification examination performance by a measure of 'questionable' activities in clinical practice: a retrospective cohort study

Purpose 'Low-value' clinical care and medical services are 'questionable' activities, being more likely to cause harm than good or with disproportionately low benefit relative to cost. This study examined the predictive ability of the QUestionable In Training Clinical Activities Index (QUIT-CAI) for general practice (GP) registrars' (trainees') performance in Australian GP Fellowship examinations (licensure/certification examinations for independent GP).Methods The study was nested in ReCEnT, an ongoing cohort study in which Australian GP registrars document their in-consultation clinical practice. Outcome factors in analyses were individual registrars' scores on the three Fellowship examinations ('AKT', 'KFP', and 'OSCE' examinations) and pass/fail rates during 2012-21. Analyses used univariable and multivariable regression (linear or logistic, as appropriate). The study factor in each analysis was 'QUIT-CAI score percentage'-the percentage of times a registrar performed a QUIT-CAI clinical activity when 'at risk' (i.e. when managing a problem where performing a QUIT-CAI activity was a plausible option).Results A total of 1265, 1145, and 553 registrars sat Applied Knowledge Test, Key Features Problem, and Objective Structured Clinical Exam examinations, respectively. On multivariable analysis, higher QUIT-CAI score percentages (more questionable activities) were significantly associated with poorer Applied Knowledge Test scores (P = .001), poorer Key Features Problem scores (P = .003), and poorer Objective Structured Clinical Exam scores (P = .005). QUIT-CAI score percentages predicted Royal Australian College of General Practitioner exam failure [odds ratio 1.06 (95% CI 1.00, 1.12) per 1% increase in QUIT-CAI, P = .043].Conclusion Performing questionable clinical activities predicted poorer performance in the summative Fellowship examinations, thereby validating these examinations as measures of actual clinical performance (by our measure of clinical performance, which is relevant for a licensure/certification examination). Key message What is already known on this topic- Frequent performance of 'Questionable' clinical activities-activities more likely to cause harm than good or whose benefit is disproportionately low compared with its cost-can be seen a marker of the quality of health care delivery and clinical competence. Summative medical licensure/certification examination performance should reflect clinically important attributes of the candidate and provide confidence of future patient safety. What this study adds- This study demonstrates that the Royal Australian College of General Practitioners' (RACGPs') summative examination performance reflects clinical performance as assessed by a measure of 'questionable' actions in clinical practice. Performance on all three individual examination components-Applied Knowledge Test (a multiple-choice examination), Key Features Problem (a short answer plus multiple responses selection examination), and Objective Structured Clinical Exam (a clinical skills station examination), plus pass/fail status-was predicted by 'questionable' clinical practice. How this study might affect research, practice or policy- The findings provide evidence for the validity of the RACGP Fellowship examination and similar licensure/certification examinations.