Pursuing Perfection: An Asthma Quality Improvement Initiative in School-Based Health Centers with Community Partners

SYNOPSIS

This article describes an innovative asthma quality improvement initiative conducted in school-based health centers (SBHCs) with collaboration among multiple community partners including the children's hospital, a federally qualified health center, the health department, the public school, and parents of children with asthma. The aim was to improve the health of children with asthma in these schools, as measured by minimal asthma-related activity restriction and reduction in asthma-related emergency department (ED) visits. Process measures tracked included the percent of children with (1) asthma severity classified, (2) persistent asthma with controller medication prescribed, and (3) written care plans.

Data supported a statistically significant decreasing trend for the percent of SBHC children reporting activity restriction due to asthma. In addition, trend analysis demonstrated a statistically significant difference in ED visits for asthma in the SBHC group compared with a non-SBHC group. Improvements were demonstrated for all process measures. Factors contributing to success included emphasis on community engagement, transparency in sharing ideas and results, benefits outweighing human resource costs of participating, readily available data to drive improvement, and the use of multiple intervention strategies.

This article describes the challenges and processes involved in the implementation, as well as the outcomes, of an asthma quality improvement (QI) initiative in school-based health centers (SBHCs). The project was initiated and funded as part of a larger QI initiative at the Cincinnati Children's Hospital Medical Center (CCHMC), Pursuing Perfection: Raising the Bar for Health-Care Performance, a program of the Robert Wood Johnson Foundation to “help hospitals and physician organizations dramatically improve patient outcomes by pursuing perfection in all their major care processes.”

Numerous studies have reported on asthma care or interventions in schools,^1–9 but only a few have reported interventions or outcomes for the SBHC setting.^10,11 Additional studies report specifically on the use of QI methodology as the intervention for improving asthma care. Most of these studies are conducted in primary care outpatient clinics and community-based practices, not in school settings.^12–15 One study did use QI methodology and incorporated community partners, including schools, in their collaborative work.¹⁶ However, the authors are unaware of any published studies detailing work in an SBHC setting using QI science and methodology as the framework for the intervention.

BACKGROUND

Asthma is the most common chronic medical illness in school-aged children as well as the population served by SBHCs. According to the Child Well-Being population health survey conducted by the Child Policy Research Center at CCHMC,¹⁷ the prevalence of childhood asthma in Greater Cincinnati in 2005 was 13.9%, but was 21.8% in the city of Cincinnati where these SBHCS are located. Because these SBHCs serve a predominantly African American population, another important reason for selecting asthma as the focus for the QI initiative was the strong literature support as well as personal experiences of the health-care team with health disparities in both morbidities and outcomes for low-income, minority children with asthma.^18–21

Improving asthma care was also of interest to the Cincinnati Health Department (CHD) School Health Program, which provides school nurses for Cincinnati Public Schools (CPS) in which the SBHCs are located. A study conducted by CHD school nursing supervisors in 2000 identified the need for creative strategies to address asthma needs in high-poverty schools in Cincinnati.²²

OBJECTIVE

The project aim was to improve the health of elementary school-aged children with asthma who attend schools with SBHCs utilizing the Chronic Care Model, as measured by minimal asthma-related activity restriction and reduction in the number of asthma-related emergency department (ED) visits. The Chronic Care Model identifies six elements of the health-care system that can be leveraged to improve the quality of care of patients with chronic illness: health service organization/leadership, self-management support, delivery system design, provider decision support, clinical information systems, and links to community resources.²³ The majority of the interventions developed and implemented as part of this initiative focused on changes or improvements in one of the components of the Chronic Care Model.

PURSUING PERFECTION ASTHMA IMPROVEMENT INITIATIVE

Setting

The Pursuing Perfection Asthma Improvement Initiative (hereafter called “the Initiative”) takes place in four CPS that have an SBHC and serve students in kindergarten through eighth grade. The schools range in size from 200 to 580 students with a total population of approximately 1,400 students. The urban schools have high rates of student mobility, with many students coming into or leaving the school over the course of the school year. Students in the four schools come from predominantly low-income African American families, with more than 85% of students in the free-reduced school lunch program, a marker for poverty.

The SBHCs offer a comprehensive model of health care, including preventive, primary, dental, and mental health-care services. The SBHCs are operated by a federally qualified health center organization, Neighborhood Health Care, Inc. (NHC), in affiliation with CCHMC's Department of General and Community Pediatrics. Through a contractual arrangement, CCHMC provides the medical director, the supervising physicians, and two pediatric nurse practitioners. NHC provides the SBHC project director, two nurse practitioners, four health technicians, and a medical assistant who is shared among sites. CHD provides school nursing services with four public health nurses.

Structure

Based on lessons learned from other CCHMC improvement projects, the first step was to develop a steering committee and a workgroup. The steering committee consisted of members of CCHMC's QI leadership team, leaders of the Initiative, and a senior-level administrator. The steering committee provided a mechanism for dialogue between the project and CCHMC leadership to ensure success by reducing barriers and providing institutional support.

The workgroup included the following members from CCHMC and NHC: a project director from the Initiative, a QI consultant, information systems personnel for registry development, the SBHC medical director, the SBHC project director, four nurse practitioners, and four health technicians. CHD workgroup members included the school health nursing supervisor and the four school nurses. CPS members included teachers, although their involvement was sporadic. Community members in the workgroup included parents of children with asthma.

Two other groups met on a regular basis. One was a project leadership team that consisted of the SBHC medical director and program director, the project director of the Initiative, the QI consultant, the CHD supervisor, and the data management support person. Another team was responsible for the development, implementation, and ongoing training and use of the Web-based asthma registry. The institution-wide CCHMC Initiative provided the opportunity for all members of the SBHC workgroup to meet with QI experts from the Institute of Health Improvement (IHI) on a quarterly basis. These experts provided an external perspective and functioned as a sounding board as the project team tested and implemented interventions to achieve the desired outcomes. The project was approved by the Institutional Review Board at CCHMC.

METHODS

Community engagement

The community engagement process was initiated through various group and individual meetings. Project team leaders met with each school principal, as well as with the leadership of each partnering organization, to inform them about the initiative and gain support. Before bringing parents into the larger workgroup, a grassroots advocate from the community led a discussion with interested parents about how to best engage in the process and empower them to participate.

Developing measures, key drivers, and interventions

The project leadership group in conjunction with the workgroup used the Model for Improvement to guide the development of outcome measures, key drivers, and interventions for the Initiative.²⁴ The Model of Improvement suggests using three questions to help frame improvement work: (1) What are we trying to accomplish? (2) How will we know that a change is an improvement? and (3) What changes can we make that will result in improvement.²⁴ The workgroup participated in a visioning exercise that asked them to “Describe what perfect asthma care means to you.” Nominal group and multi-voting techniques were used to group individual comments into themes and then rank them in a prioritization matrix. Using the themes, goals were written as promises to parents and/or the child with asthma, and measures were developed to align with those goals for objective measurement of improvement. The promises and their corresponding measures are listed in Figure 1.

Figure 1. Promises and measures for the SBHC Asthma Improvement Initiative.

SBHC = school-based health center

Process and outcome measures

The primary outcome measures for the project were (1) the percent of children reporting activity restriction due to asthma as “rarely” or “never” and (2) the quarterly and monthly number of ED visits for children seen in the SBHC per 1,000 children with asthma. The project also tracked ED visits for asthma by children who lived in zip codes with a school in CPS to compare with a similar population without the QI intervention.

Process measures included percent of children identified with asthma, percent of children with asthma severity classification assessed, percent of children with persistent asthma-prescribed controller medication, percent of children with a written care plan, percent of children with perfect care (i.e., children with severity classification, controller medication prescribed when appropriate, and a written care plan), percent of children with a self-management goal set or addressed at most recent visit, and percent of children with both rescue medication and consent to use rescue medication on file in the SBHC. Of note, the percent of children with persistent asthma with controller medications prescribed can only be accurately calculated within the population seen by the provider for which severity classification has been completed. In addition to these more clinical measures, the percent of asthmatic children known by their teachers to have asthma was collected. This was measured by asking teachers to use class lists and highlight those students they believed had asthma, then comparing the list with the data in the Web-based registry that was developed as part of this project.

The process and outcome measures were tracked in a monthly report that was routinely shared with the asthma improvement team. The report displayed individual graphs of each key measure, including a run chart with performance over time, annotated by key process changes that impacted performance or measurement. In addition, the run charts also contained the goal for each measure. Initially, data were reported with two denominators—as the percent of children seen in the SBHC for a visit and as the percent of children based on expected population prevalence of asthma. The denominator for the latter measure was calculated monthly by taking the total school enrollment multiplied by the population prevalence estimate (20.9%) from the Child Policy Research Center at CCHMC. Reporting data based on population prevalence was to ensure the team was working toward achieving these goals in a largely yet to be identified population.

Reporting data on children seen for a visit was conducted to motivate the work of the project staff and monitor adherence to new processes. Once standard identification processes were in place (by September 2005), the data were reported using children identified with asthma as the single denominator. While interventions started in October 2004, data could not be consistently or accurately reported until February 2005, when the registry had initial functionality.

With the exception of ED visits, all outcomes were tracked using data collected by SBHC staff and entered into the Web-based asthma registry by health technicians. ED data came from CCHMC's database and were linked through the CCHMC medical record number so that data would be available for clinical use. ED use by asthma students was queried monthly and quarterly by hospital analysts using medical record numbers and tracked separately on run charts, including ED visits that had a primary or secondary diagnosis of asthma (International Classification of Diseases, Ninth Revision [ICD-9] code of 493) at discharge.

Once outcome measures were determined, the team focused on understanding the key drivers or influential factors for improving these outcomes, as well as on the appropriate interventions that would impact those key drivers (Figure 2). Each intervention is related to a component of the Chronic Care Model. For example, development of the registry and use of registry reports to support clinical decision-making align with the clinical information systems and provider decision support components of the Chronic Care Model. Staff training and implementation of self-management skills with the target population align with the Chronic Care Model's self-management support component. Clearly defining roles and responsibilities of each SBHC staff member in the provision of asthma care is an improvement in delivery system design.

Figure 2. Key drivers and interventions for the SBHC Improvement Initiative.

SBHC = school-based health center

While not a process or outcome measure, additional data for the project were collected at the end of the second year through individual interviews with each member of the SBHC team by a CCHMC QI employee not directly involved with the work of the Initiative. Interviews consisted of a series of questions developed by the workgroup to assess perceived successes and challenges of the project. Responses were collated by the CCHMC employee and shared with the workgroup, keeping the respondents' identities anonymous.

Statistical analysis

Trend analysis was performed for each outcome measure. For ED visits, the trend lines were calculated by doing a linear regression of the month (or quarter) on the number of ED visits per 1,000 population. To test for the difference in the slope lines for asthma-related ED visits by children in the SBHC compared with children in CPS zip codes, a t-test was performed on the two slope parameters. For the activity restriction measure, the trend line was calculated by doing a linear regression of the month on the percent of children reporting activity restriction due to asthma as “never” or “rarely.” For the activity measure, data points prior to September 2005 were not included as they represented the time frame in which we were working to maximize identification of children with asthma. The estimated intercept and slope parameters were used to compute the points for the trend line. P-values were calculated for the null hypothesis that the slope parameter is equal to zero.

Timeline for the improvement work

In Year 1 (April 2004 to June 2005), initial efforts and Plan/Do/Study/Act (PDSA) cycles focused on defining the population of children with asthma seen in SBHCs, setting up and standardizing protocols for delivery of care, and developing a Web-based asthma registry to track improvement data and provide clinical decision support to practitioners. Specific PDSA cycles around identification included asking teachers to share information regarding children they either knew to have asthma or suspected might have asthma. Other PDSA cycles involved teachers conducting brief classroom surveys to identify children who had asthma but who had not completed school enrollment paperwork or had not visited a health center where asthma could be identified.

The paperwork process involved in caring for children with asthma in the school setting was streamlined. At the beginning of the project, separate forms were used by the CHD school nurses and the SBHC staff to collect information regarding children with asthma, as well as for emergency plans. The workgroup collaborated to ensure data required by both parties could be collected and agreed to use common forms for similar purposes. These asthma forms were incorporated into the mailings that go out as part of the school enrollment process to increase completion by parents and return to the SBHC.

In Year 2 (July 2005 to June 2006), the team interacted with teachers and school staff to increase their knowledge about asthma and keep them informed regarding which children in their classrooms had asthma. Education occurred at teacher staff meetings and one-on-one with physical education instructors. Self-management skills of SBHC nurses and nurse practitioners were increased by conducting training sessions and scheduling follow-up mini-sessions to further refine new skills. Self-management training included instruction on assessment of readiness to change and importance of change as well as confidence in making a change; developing and monitoring an action plan; setting and monitoring goals with students; and motivational interviewing techniques. Training was conducted by staff at CCHMC who were experienced in the application of self-management skills in the clinical setting. Use of the asthma registry and its reports allowed selection of subpopulations of children for self-management interventions (i.e., those with frequent ED visits or increased levels of reported activity restriction). Self-management PDSA cycles included asking practitioners to work with higher-risk children to set and monitor self-management goals. Lessons learned from individual children regarding applying self-management techniques were expanded to multiple children in each SBHC setting.

In Year 3 (July 2006 to June 2007), the focus was to maintain the achievements of the Initiative without the monetary support of the initial grant. The leadership team developed an action and sustainability plan that transitioned the work done by project staff to SBHC staff. Activities requiring transition were related to data and registry support and generation of the monthly measures report.

RESULTS

In October 2004, at the onset of the project, the team was aware of 112 children with asthma, or a population prevalence of 6% using current school enrollment as a denominator. The estimated population prevalence of asthma was 21% or an estimated 428 children. By March 2005, the figure rose to 17% and remained consistently between 15% and 17% throughout subsequent months. Once the team was confident in the identification process, the mean number of children with asthma in the registry was 234. Figure 3 demonstrates the actual number of children identified and seen during the Initiative once the Web-based registry was functional.

Figure 3. Number of children seen and identified with asthma.

Outcome measures

Trend analysis demonstrated an increasing trend of 1.8 ED visits for asthma per 1,000 population per quarter for CPS children from July 2003 through June 2007. This increasing trend was statistically significant at the p=0.05 level (Figure 4). There was a decreasing trend of 0.7 ED visits for asthma per 1,000 population per quarter for SBHC children from July 2003 through June 2007. However, the trend was not statistically significantly different from zero (p=0.4) (Figure 4). In comparing the two slopes, however, there was a statistically significant difference in ED visits for the SBHC group compared with the CPS group (p=0.04).

Figure 4. Comparison of ED visits per 1,000 population of children known to have asthma: SBHCs vs. CPS.

ED = emergency department

SBHC = school-based health center

CPS = Cincinnati Public Schools

Trend analysis demonstrated an insignificant decreasing trend of 0.09 ED visits per 1,000 population per month from July 2003 through June 2007 (p=0.4) (Figure 5). During Year 3, there was decreased variability in the number of ED visits per month in the population served by SBHCs.

Figure 5. Number of ED visits for children seen at SBHCs per month per 1,000 population.

ED = emergency department

SBHC = school-based health center

The percent of identified children reporting activity restriction due to asthma as “rarely” or “never” was 24% at the beginning of the project, 43% at the time identification processes were reliably implemented (September 2005), and 60% in June 2007 (Figure 6). Trend analysis revealed an increasing trend of less than 1% of children reporting activity restriction as “rarely” or “never” per month from September 2005 through June 2007 (p<0.001).

Figure 6. Percent of identified children reporting activity restriction due to asthma as “never” or “rarely”.

Process measures

In February 2005, only 14% of children identified had their asthma severity classified. The percent of children with severity classified rose to 97% of children identified with asthma by June 2007 (Figure 7). Sixty percent of children with persistent asthma had controller medications prescribed at the beginning of the project, which rose to 88% by June 2007 (Figure 8). In the beginning of the project, only 9% of identified children had a written care plan for asthma compared with 98% of children with asthma by June 2007 (Figure 9). A little less than 6% of identified children with asthma were receiving “perfect asthma care” at the project's onset, compared with 92% of children identified by June 2007 (Figure 10). The percent of children with a self-management goal set or addressed at the most recent asthma encounter with SBHC staff rose consistently from 0%, reaching 92% in June 2007. The percent of asthmatic children known by their teachers to have asthma was 55% during the 2005–2006 school year.

Figure 7. Percent of identified children with asthma at SBHCs with severity classification in chart.

SBHC = school-based health center

Figure 8. Percent of identified children with persistent asthma with appropriate controller medication prescribed.

Figure 9. Percent of identified children with asthma with written care plan in chart.

Figure 10. Percent of identified children receiving perfect care.

In March 2005, 26% of identified children with asthma had both rescue/bronchodilator medication on-site at school and a signed parental consent form to take the medication. By the end of that school year, 68% of children identified with asthma had both items. School district policy requires completion of the consent form annually, so the percent dropped to 0% at the beginning of the following academic years. However, through implemented improvement processes, the rate of rise was more rapid in subsequent years, and the percent of students with both rescue medication and signed consent forms in June 2007 was 76% (Figure 11).

Figure 11. Percent of identified children with albuterol inhaler and consent to use at school.

DISCUSSION

The Initiative demonstrates that QI work can be successfully implemented in an SBHC environment with multiple community partners and can translate into improved care delivery for children with asthma. Trend analysis did demonstrate a decrease in ED visits by both month and quarter, but the decreases were not statistically significant. However, when ED visits of children in SBHCs were compared with other CPS children, the t-test of the slope lines indicated a difference in the ED rates for these two populations. It is plausible that without the SBHC QI interventions, instead of a decrease in ED visits, an increase similar to what was observed in the comparison population would have been observed.

Although trend analysis revealed statistically significant improvement in the percent of children reporting minimal activity restriction due to their asthma, the improvements were modest and did not reach our goal. The workgroup struggled with the activity restriction measure and raised issues early on regarding the validity of child responses. PDSA cycles were completed to ensure standardization in the way the question was being asked and recorded, but concern remained in how accurately the extent of activity restriction was represented by the measure. Despite measurement issues, which could have underestimated the percent of children with minimal activity restriction, there are opportunities for continued efforts to improve activity restriction of these children with asthma.

The key clinical reason for success was that implemented processes, especially standardized clinical encounter forms, supported an increased number of children with persistent asthma receiving prescriptions for controller medications. Access to a medical provider in the SBHC was critical. In addition, self-management training of providers improved the skill set needed to help guide children with asthma and their families toward setting and achieving the goal of the children taking their medications accurately.

The project's success was related to several other key nonclinical elements. These conclusions were based on data collected through the individual interviews with SBHC staff at the end of Year 2, as well as discussions among the project leadership team. First, there was an intentional, focused effort to engage the community and maintain the involvement of partners, especially parents and teachers. The resulting collaboration was genuine, one in which the partner entities were able to equitably contribute to the process and have true accountability for outcomes. Community partners were invaluable in moderating the academic focus of members of the group from CCHMC. Sometimes the zeal for measurement, or the need to take a process to a degree of reliability, was met with the challenge of limited available resources within the community setting. Recognizing the limits of the community organizations and their resources was a key to success.

The second element of success was the commitment to transparency in executing the work. One of the biggest motivators for getting the workgroup engaged in improving identification of children with asthma was the transparency around the number of children known in the school buildings with asthma—6% of the population vs. the expected population prevalence of 21%. SBHC providers were further motivated by the realization that many of the children unknown to the health center but subsequently identified were children with significant asthma, not children with mild or intermittent disease.

The third factor that led to success was that the perceived benefits of doing the work ultimately outweighed the perceived “costs” of participation. All staff had positive experiences with individual students. Observing the health benefit to a student was a strong impetus for the eventual buy-in to the project's work. The workgroup also agreed that making progress through a series of small tests of change made it feel manageable and not overwhelming, given the complex context of the care environment.

The fourth factor of success was having data readily available for project staff to monitor their performance. The use of the monthly project report at team meetings allowed for healthy competition among sites and for the team to see the impact of the improved processes on the outcomes. Staff's ability to see either steady improvement or poor performance on a measure was a strong motivator for behavior change. Weir describes the ability to monitor provider performance as a vital component to a successful chronic disease management program.²⁵ Staff also suggested that the asthma registry provided much more information for the management of clinical decisions than working from a traditional list of students with chronic illnesses.

The fifth contributing factor to success was using multiple QI strategies in executing the SBHC improvement work. In “Closing the Quality Gap: A Critical Analysis of QI Strategies: Volume 5 on Asthma Care,”²⁶ the authors concluded that studies employing multiple strategies were more likely to demonstrate positive impact on clinical and/or utilization outcomes. The document describes nine types of strategies: provider reminders, facilitated relay of information, audit, feedback, provider education, patient education, promotion of self-management, organizational changes, and financial strategies. The project successfully incorporated all nine strategies. Wheeler et al. discuss key lessons for successful school-based asthma interventions. Three of these were strategies used in this project: establishing links to asthma care clinicians, targeting students who are most affected, and using a coordinated, multicomponent approach.²⁷

Finally, it is important to note that there were incentives in place that motivated the work, and equal participation by all members. For nurse practitioners, clinical productivity was expected by their employers. Therefore, workgroup meetings were held outside their regular work hours, and funding in Years 1 and 2 paid the nurse practitioners overtime to attend. School nurses already worked full-time, so they could not get additional compensation; therefore, funds were made available for all school nurses to attend a local educational conference that included content on asthma care. Parents were offered compensation for their time and travel. In Years 1 and 2, each school received a small stipend to purchase supplies, equipment, and materials that would support the care of children. Despite not having any money for these activities in Year 3, the level of engagement continued.

Achieving true collaboration among the various organizational structures was the project's biggest challenge and took determination, time, and trust. Due to alignment of the improvement work with performance evaluation, the employees of NHC and CCHMC bought into the initiative more quickly. For the CHD nurses, it took continued dialogue with the school nursing supervisor and regular meetings with the school nurses to get ultimate buy-in. The nursing supervisor was supportive of the project, but struggled with how to prioritize the additional work for staff in the context of multiple other nursing priorities, such as screening, immunization compliance, dental screening/referral, and case management of students with other chronic conditions. Introduction of protocols and procedures developed through the project put stress on the school nurses as they were asked to transition their roles and redefine their work relationship with other health-care professionals in the SBHC. One example was the work on keeping teachers informed of children with asthma. The staff was committed to keeping teachers informed, but could not commit to allocate time to measure it. A decision could have been made to have CCHMC project staff measure this, but it was agreed that implementing tasks that could not or would not be sustained by the team was not prudent. It was agreed that school nurses would provide a list of children with asthma to teachers twice a year, and notify teachers of any new students with asthma as they were identified.

Future sustainability was considered when making decisions about new processes. While many excellent ideas for improvement often surfaced in team meetings, if they required additional resources that were not likely to be available on an ongoing basis, project leaders selected alternative processes or plans for improvement.

Limitations

The Initiative was implemented as a QI project, not as a research study. With the exception of ED visit data collected on children with asthma who lived in zip codes within CPS, there was no control group or comparison group for the work that was completed. As stated previously, while interventions started in October 2004, data could not be consistently or accurately reported until February 2005, when the registry had initial functionality. This could have contributed to underestimation of change. It is also important to note that data can only be collected on many measures if the child with asthma is actually seen for a visit. Therefore, early on, the lower percentages on measures using children identified as a denominator were more a reflection that it took time to get identified children in for an initial visit to the SBHC than an indication of actual performance improvement.

In addition, the mobility of the student population, which is often characteristic of inner-city schools, impacted project interventions and outcomes. Many of the measures would fluctuate between academic years as well as early in an academic year, as children who withdrew from schools were removed from the registry, and new children with asthma were identified through the processes in place.

Lastly, the project was conducted in four SBHCs in Cincinnati as a pilot project to determine if QI strategies could result in improved care delivery and outcomes of children with asthma in schools with SBHCs. Therefore, the results may not be generalizable to other environments or SBHCs.

CONCLUSIONS

QI around chronic illness care can be successfully executed in SBHCs by community partners working in collaboration. The Initiative augmented the organizations' capacities and skill sets by sharing the tools of process improvement. The opportunity for change and improvement in health-care delivery and outcomes was the greatest when all partners were engaged.