Development of an algorithm to assess appropriateness of overriding alerts for nonformulary medications in a computerized prescriber-order-entry system

Purpose. An algorithm for assessing the appropriateness of physician overrides of clinical decision support alerts triggered by nonformulary medication (NFM) requests is described. Methods. Data on a random sample of 5000 NFM alert overrides at Brigham and Women's Hospital over a four-year period (2009-12) were extracted from the hospital's computerized prescriber-order-entry (CPOE) system. Through an iterative process, a scheme for categorizing the reasons given by prescribers for alert overrides was developed. A pharmacist and a physician used the categorization scheme to classify and group alert override reasons, and the resultant data guided the development of an algorithm for assessing alert overrides. Results. In free-text comments written in response to NFM alerts, prescribers provided more than 1150 unique reasons to justify formulary deviation. The compiled reasons were analyzed and grouped into nine categories through the iterative process, with a high degree of interrater agreement (k = 0.989; 95% confidence interval, 0.985-0.992). An initially developed 30-item "NFM alert override appropriateness algorithm" was simplified to create an 8-question algorithm that was presented to an interdisciplinary team for evaluation, with subsequent refinements for enhanced clinical creditability. The final algorithm can be used by researchers and formulary managers to develop strategies for limiting NFM alert overrides and to avoid the labor-intensive task of creating appropriateness criteria for each NFM. Conclusion. A multistep process was used to develop a generalized algorithm for categorizing the appropriateness of reasons given for NFM alert overrides in a CPOE system.