Comparative outcomes of face-to-face and virtual review meetings

Background: The present study proposes to evaluate the effects of face-to-face (FF) and virtual review (VR) sessions on peer reviewers' scores and consistency of peer review. Methods: Retrieved review sessions conducted between 2012 and 2014 yielded 119 and 51 discussed applications for the FF and VR groups, respectively. Changes between preliminary scores, post discussion scores and final matrix scores were analyzed. Consistency between the two meeting modalities was measured by percentage and increments of score changes. Results: Discussion changed the preliminary scores in 37% of applications reviewed in the FF group and 24% of applications reviewed in the VR group (no difference between groups). Applications that received a preliminary score in the 10 to 30 point-range were more positively than negatively impacted by discussion in both modalities. FF discussion led to a wider range of scoring changes (-10 points to 17 points) than VR (-7 points to 10 points), but discussion was not found to differentially improve or worsen scores between the two modalities. When comparing post-discussion and final matrix scores, 27 (23%) applications' scores changed in the FF meetings compared to 13 (25%) in the VR meetings (no difference between groups). Conclusions: FF and VR sessions result in (1) minimal differences in preliminary scores, (2) nonsignificant percentage changes in scoring, and (3) non-significant change in the percentage of magnitude of scoring. The two review methods appear to be similar in evaluating grant applications. Published by Elsevier Ltd on behalf of Surgical Associates Ltd.