An Objective Assessment of Performance during Robotic Partial Nephrectomy: Validation and Correlation of Automated Performance Metrics with Intraoperative Outcomes

Purpose: Automated performance metrics provide a novel approach to the assessment of surgical performance. Herein, we present a construct validation of automated performance metrics during robotic assisted partial nephrectomy. Materials and Methods: Automated performance metrics (instrument motion tracking/system events) and synchronized surgical videos from da Vinci (R) Si systems during robotic assisted partial nephrectomy were recorded using a system data recorder. Each case was segmented into 7 steps: colon mobilization, ureteral identification/dissection, hilar dissection, exposure of tumor within Gerota's fascia, intraoperative ultrasound/tumor scoring, tumor excision, and renorrhaphy. Automated performance metrics from each step were compared between expert (>= 150 cases) and trainee (<150 cases) surgeons by MannWhitney U test (continuous variables) and Pearson's chi-squared test (categorical variables). Clinical outcomes were collected prospectively and correlated to automated performance metrics and R.E.N.A.L. (radius, exophytic/endophytic, nearness of tumor to collecting system, anterior/posterior, location relative to polar line) nephrometry score by Spearman's correlation coefficients (r). Results: A total of 50 robotic assisted partial nephrectomy cases were included for analysis, performed by 7 expert and 10 trainee surgeons. Automated performance metric profiles significantly differed between experts and novices in the initial 5 steps (p<0.05). Specifically, experts exhibited faster dominant instrument movement and greater dominant instrument usage (bimanual dexterity) than trainees in select steps (p <= 0.045). Automated performance metrics during tumor excision and renorrhaphy were significantly correlated with R.E.N.A.L. score (r >= 0.364; p <= 0.041). These included metrics related to instrument efficiency, task duration, and dominant instrument use. Conclusions: Experts are more efficient and directed in their movement during robotic assisted partial nephrectomy. Automated performance metrics during key steps correlate with objective measures of tumor complexity and may serve as predictors of clinical outcomes. These data help establish a standardized metric for surgeon assessment and training during robotic assisted partial nephrectomy.