Reflection (With Some Coaching) on a Quality Improvement Initiative: Finding Our Collective Voice Along the Journey

The Quality Improvement Success Story article by Roberts et al. (1) titled “Screening for Mental Health Comorbidities in a Pediatric Diabetes Clinic Setting” is a wonderful example of how the quality improvement (QI) lens can be focused on different kinds of outcomes. The clinic at Seattle Children’s Hospital elected to pursue improvement of its rate of mental health screening given the association of depression and diabetes distress with poor clinical outcomes such as A1C. Much is to be learned by their initiative and energies. I present below a top-10 list (in no particular order of importance) of my reflections on this project.

1. Cycle Length

The team used the plan-do-study-act (PDSA) methodology to create change. The PDSA process enables practices to create many small, iterative changes rather than focus on one large change, hope for the best, and remeasure at some point in the future. The multiple months between intervention, data collection, and change in the initiative by Roberts et al. was quite long. Ideally, small samples of data would be collected weekly to determine whether the intervention needed to be redesigned or deployed in a different manner. To gain that knowledge quickly, I encourage practices to have shorter cycle times between PDSA cycles.

2. Sampling

Much can be learned from simply looking at five randomly selected patients per week who were exposed to the intervention to see whether the intervention behaved as was intended. At the end of each week, team members can look at the results and ask themselves (the “A” in PDSA): “Should we Abandon the initiative, do it Again” (i.e., collect data for another week), Adapt it (i.e., tweak the design of the intervention and then study that change going forward), or Adopt it as a standard process because enough data have been collected to show that it is working reliably over time.

3. Assessing Overall Effectiveness

At the end of the study period, the clinic can look at the overall effectiveness of its intervention by performing either an electronic query of the total population (if the electronic health record [EHR] system has that level of functionality) or performing a chart audit of a sample of the population. For a large practice such as the one described by Roberts et al., looking at 30–50 patients at the end of the intervention/change effort is more than enough to determine performance within the entire population. This query or audit will reflect the impact of all of the PDSA cycles performed over time from the beginning to the end of the initiative timeframe. Crucially, to ensure that “drift” does not occur over time, another sample should be taken 6 months or 1 year after the initiative has ended to ensure that initial attained improvements have been retained.

4. Context and Culture

Practices need to pay close attention to the attitudes and beliefs of their staff before initiating a new intervention. Tools that are imported from the outside that have been shown to work in one location may fail miserably in another if the stakeholders in the change are not allowed to have a voice and to participate in customizing the tool. Equally important is for all to be able to discuss their attitudes and beliefs about any aspect of the initiative. If concerns are not raised initially, they may affect the effort negatively, and the positive results seen in other locations may not be realized. Attending to culture takes time and space, but the investment in having conversations to create shared meaning and a common vision will pay dividends in ensuring that barriers are removed and all staff are fully engaged.

5. Patient Co-Design

The most important stakeholders in QI are the patients and their families. Ensuring they are a part of the intervention and tool design is key to attaining optimal outcomes. This can be done effectively through multiple different channels such as focus groups, membership on the QI team, ideas boxes, or patient and family advisory councils. Engaging family members is important given the crucial support role they play in chronic illness management. It is not evident that Roberts et al. sought out the voice of patients (or families) in designing the change they wanted to implement, and they are encouraged to do so in the next phase of their work.

6. Patient-Reported Outcomes

The study by Roberts et al. used the nine-item Patient Health Questionnaire for assessment of depression and the Problem Areas in Diabetes Teen Version for assessment of diabetes distress. The team should be commended for developing processes to systematically incorporate administration of these questionnaires into practice workflow. Too often, these types of tools are relegated to and remain in clinical research studies but do not find their way to the figurative bedside. At the end of the day, developing efficient processes that allow for integration of these assessments into the office visit (either through paper-based, electronic, telephonic, or other capture mechanisms) will help to ensure that attention is paid to the patient as a whole person. The use of patient-reported outcome measures reduces the variability among clinical staff in the degree to which they listen to the patient’s “voice.” It also allows for trend monitoring to determine whether patients, from their own perspective, are improving over time.

7. Co-Management

As we transition to new models of care, practices that screen for depression, anxiety, substance abuse, and other mental health conditions will find them at high prevalence. To address these issues in the new world of value-based payment, it is crucial for practices to seek mechanisms to expand their definition of “team” through arrangements such as co-management of patients with various specialists. Co-management allows for a practice to maintain primary ownership of the patient’s care, while gaining expert guidance through case conferences, consultations, and the use of validated disease-specific scales that can track progression over time. At its heart, QI seeks to improve structures, processes, and culture to positively affect outcomes. Consider making significant structural changes such as introducing co-management to your practice should your QI efforts reveal the need.

8. Health Outcomes Survey

As previously mentioned, the health care environment is moving increasingly to value-based payment for clinicians. Patients' voices, now more than ever, will be incorporated into the compensation equation. An example (and on the other end of the age spectrum from the population in the initiative by Roberts et al.) is the Health Outcomes Survey, which is used as a component of quality assessment to determine payment for services to Medicare patients. The survey asks patients for their perceptions of their physical and emotional health and their ability to attend to activities of daily living. Office practice processes ideally would be designed to capture this voice at every visit so we can be guides for our patients in their effort to seek optimal health. In addition to disease-specific tools such as those deployed by Roberts et al., consider asking a simple, generic question such as, “What matters most to you?” This approach will move the discussion (and your management efforts) away from purely biological markers such as the A1C and its long-term, downstream implications and toward outcomes that are more proximal and meaningful for patients today.

9. Learning Health Systems

As technology continues to evolve at an increasingly rapid pace, new opportunities involving “big data” are arising for clinics and health systems. For practices such as the one at Seattle Children’s, which is embedding new scales and tools in its EHR, a representative from the health system analytics team ideally should be a part of the core QI team. The new EHR elements ideally would be captured by an enterprise-wide data warehouse to allow for real-time report generation for performance assessment. This analytic approach allows the clinic to take on some of the attributes of a true “learning health system” and to develop the capability for real-time understanding of performance to enable emergence of insights and continued learning.

10. Agency

The team at Seattle Children’s Hospital should be lauded for focusing their QI effort on patients’ mental health and voice. Attend to the teams’ own mental health and voice is equally important. As evidenced by the participation of author Roberts in the local QI Scholars program, there was intent to gain new skills that could assist the group. Through the collective energies of initiative participants, they began a journey that will lead to improvement in their own well-being as they increasingly understand that they have agency—the power to create change combined with the courage to make it happen. Gaining this understanding and building on early successes will promote a positive culture, protect against provider burnout, and yield positive results for providers and patients.

The QI initiative described by Roberts et al. was simple, yet its impact on patients and providers will run deep. Sharing their experience with others through publication allows for additional reflection and for new insights to emerge and new possibilities to unfold.