Development and validity evidence for the intraprofessional conflict exercise: An assessment tool to support collaboration

Background Effective collaboration is the foundation for delivering safe, high quality patient care. Health sciences curricula often include interprofessional collaboration training but may neglect conflicts that occur within a profession (intraprofessional). We describe the development of and validity evidence for an assessment of intraprofessional conflict management. Methods and findings We designed a 22-item assessment, the Intraprofessional Conflict Exercise, to evaluate skills in managing intraprofessional conflicts based on a literature review of conflict management. Using Messick's validity framework, we collected evidence for content, response process, and internal structure during a simulated intraprofessional conflict from 2018 to 2019. We performed descriptive statistics, inter-rater reliability, Cronbach's alpha, generalizability theory, and factor analysis to gather validity evidence. Two trained faculty examiners rated 82 trainees resulting in 164 observations. Inter-rater reliability was fair, weighted kappa of 0.33 (SE = 0.03). Cronbach's alpha was 0.87. The generalizability study showed differentiation among trainees (19.7% person variance) and was highly reliable, G-coefficient 0.88, Phi-coefficient 0.88. The decision study predicted that using one rater would have high reliability, G-coefficient 0.80. Exploratory factor analysis demonstrated three factors: communication skills, recognition of limits, and demonstration of respect for others. Based on qualitative observations, we found all items to be applicable, highly relevant, and helpful in identifying how trainees managed intraprofessional conflict. Conclusions The Intraprofessional Conflict Exercise provides a useful and reliable way to evaluate intraprofessional conflict management skills. It provides meaningful and actionable feedback to trainees and may help health educators in preparing trainees to manage intraprofessional conflict.