Table 9.

Implementation/Interpretive FE

HIV/AIDS QUERI

Implementation goal: as in many conditions, HIV care processes fall short of best practice recommendations. We sought to implement and evaluate real-time computerized clinical reminders (CR) and a collaborative intensive quality improvement program, based on the Institute for healthcare improvement breakthrough series (IQS).

FE activity:

i. By conducting ethnographic interviews and observations at the participating CR sites, we assessed existing organizational barriers and facilitators regarding use of reminders.

ii. The degree to which each site implemented the IQS improvement technique was measured at regular intervals using a Site Activation Scale (SAS) developed specifically for this project. Longitudinal tracking of SAS scores allowed calculation of time-to-improvement as well as qualitative comparisons of activation. We also assessed whether specific barriers or characteristics were associated with how well a site scored on activation.

Value:

i. We identified 6 barriers to the effective use of the CR intervention, including workload, lack of time to follow-up, inapplicability, limited training, interruption of patient-provider face time, and use of paper for physician orders at some sites. Seventeen prioritized short- and long-term recommendations were generated to improve the usefulness and usability of the 9 reminders.59

ii. By using the SAS scoring system, we were able to identify that higher performing IQS sites rapidly adopted and applied basic quality improvement concepts, like the PDSA (PLAN, Do, Study, Act). In contrast, we were able to see that lower performing IQS sites were slow to begin applying key concepts and had not completed an entire PDSA cycle well into an action period. Although the contrast between levels of activation was not always marked, the sites showing the most rapid adoption and generalization of quality improvement concepts tended to show significant improvements in care. IQS FE was also used to refine the intervention, to some extent, during the project, and to a major extent for the next “nationwide rollout” phase.

QUERI, quality enhancement research initiative; FE, formative evaluation; CR, clinical reminder; SAS, site activation scale.