Table 2.

Reach, Effectiveness, Adoption, Implementation, and Maintenance evaluation framework applied to clinical decision support.

RE-AIMa domain	Described	Potential CDSb outcome measures
Reach (individual level)	Proportion and representativeness of those impacted by the intervention (and reasons for these results)	Number of patients the CDS tool fired for divided by the number of patients the CDS tool should have fired for Characteristics of each group in numerator and denominator Investigation of reasons not fired
Effectiveness (individual level)	Impact of the intervention, including heterogeneity across subgroups and any negative outcomes (and reasons for these results)	Number of patients the CDS tool changed care for divided by the number of patients the CDS tool fired for Characteristics of each group in numerator and denominator Reasons care did or did not change Number and type of unintended or negative outcomes
Adoption (setting and staff at multiple levels)	Proportion and representativeness of those accepting or using the intervention At levels of health systems, departments, and individuals (and reasons for these results)	Number of clinicians who respondedc to the CDS tool (did not outright dismiss) divided by the number of clinicians the CDS tool fired for Number of patients who the CDS fired for that were not outright dismissed divided by the number of patients the CDS tool fired for Number of practices, setting or clinicians participating divided by the number invited Characteristics of each group in the numerators and denominators above Reasons for or not to participate or dismiss
Implementation (setting and staff at multiple levels)	Fidelity of the intervention and implementation strategy Adaptations Burden of delivery, including costs	Adaptation: number and type of changes to the CDS build or workflow integration after deployment Usability of the CDS tool (eg, SUSd) Interviews on experience and adaptations Cost of implementing (eg, time, resources)
Maintenance (individual level and setting and staff at multiple levels)	Long-term effects of the intervention and extent the intervention becomes a routine part of care	Long-term outcomes (eg, change in mortality) Sustained workflow integration and effectiveness Interviews on intended or actual sustainment or further modification

^aRE-AIM: Reach, Effectiveness, Adoption, Implementation, and Maintenance framework.

^bCDS: clinical decision support.

^cTechnically in PRISM or RE-AIM, adoption is defined as only initial agreement to participate in (or be trained in) a program. In this paper, it will be defined as above to be consistent with how this term is used in informatics and to reflect the fact that end users do not always have the choice to interface with a CDS tool.

^dSUS: System Usability Scale.