Novel data visualization method to measure alert efficiency in computerized physician order entry (CPOE) system

Objectives: To introduce a novel visualization technique for evaluating the efficacy of clinical decision support system (CDSS) alerts as perceived by physicians and to differentiate between various alert categories for optimization. Methods: We developed a visualization method, which segments into four distinct zones: Appropriate (+/+), Over-frequent yet Effective (-/+), Potentially Problematic (-/-), and Less Effective but Acceptably Frequent (+/-). Alerts from a 908-bed academic medical center in Northern Taiwan were collected over two years and classified using this technique, along with three perspectives: Safety, Completeness, and Response. Results: We collected the viewpoints of 72 clinical physicians on the system's top 20 most frequent alerts. The proposed visualization technique offers a user-centric, adaptable method for assessing CDSS alerts. Roughly five alerts were categorized as Potentially Problematic, whereas another five were deemed Appropriate. Intriguingly, certain alerts, while not beneficial for patient safety, were found to assist physicians in completing clinical workflows. Conclusions: This approach, emphasizing visual clarity and adaptability, diverges from traditional methods that lean heavily on expert opinions or statistics. It paves the way for diverse assessment perspectives, furnishing healthcare institutions with a valuable tool to improve CDSS alert systems, ensuring a harmonious balance between user efficiency and patient safety.