Care Quality and Implementation of the Chronic Care Model: A Quantitative Study

Abstract

PURPOSE We wanted to test whether improvements in care quality were correlated with changes in the Chronic Care Model (CCM) in a large medical group that attempted to implement the CCM.

METHODS The leaders of 17 primary care clinics in this medical group completed the Assessing Chronic Illness Care (ACIC) survey measure of CCM implementation before and after care system changes were made. We used administrative data to measure care quality changes for yearly samples of patients with diabetes, coronary heart disease, or depression.

RESULTS The total ACIC score for the CCM increased by an overall average of 1.4 points (from 5.8 to 7.2 on a scale of 1 to 11, P = .02) and significant increases occurred for 3 of the 6 components of the CCM. During this time, patients experienced a significant increase in the proportion meeting a composite outcome measure for low-density lipoprotein (LDL) and glycated hemoglobin levels (from 15.7% to 25.5%, P = .001). Heart disease patients meeting a composite measure for LDL values increased from 46.8% to 57.8%, and the percentage of patients with 1 or more cardiac events dropped from 17.2% to 11.4% (P = .001 for each). Persistent use of new antidepressants did not change, but more of these patients had follow-up visits (P = .02). Only the diabetes measure was significantly correlated with 2 CCM elements—clinical information systems and decision support.

CONCLUSION Despite implementation of the CCM and improvements in quality measures for 3 chronic diseases, there were few significant correlations between these changes. Showing such a relationship may require larger changes, a larger number of clinics, changes in other CCM elements, or a more-sensitive measurement tool.

INTRODUCTION

The need to greatly improve the quality of care for patients with chronic medical conditions was highlighted by the 2001 report from the Institute of Medicine (IOM), Crossing the Quality Chasm and its subsequent report, Priority Areas for National Action.^1,2 Data on the size of the gap needing attention soon followed in the RAND and ACOVE studies of care quality in the United States.^3,4 The RAND study used 439 quality indicators for 30 acute and chronic conditions, as well as preventive services, among 6,712 adults in 12 metropolitan areas; it showed that, on average, only 55% of patients received any care required by the quality indicators. The ACOVE study was similar but focused on 207 quality indicators for 22 conditions among those aged 65 years and older; again, only 55% of patients received the care required by the indicators. Pressure to improve has been added by policy makers worrying about the growing number of elderly, most of whom have multiple chronic conditions.^5,6

If we are to improve chronic disease care, it is clear that we will need a transformed approach to care delivery that includes ongoing systematic attention to the special needs of patients with chronic conditions rather than seeing them as a variant of medicine’s traditional focus on acute and episodic care. The most widely accepted model for such a transformed approach is the Chronic Care Model (CCM) of Wagner et al.^7–9 This model posits 6 essential elements that recognize the importance of community linkages and support from the health care organization while focusing on delivery system redesign, effective clinical information systems, access to decision support, and good self-management support for patients. Taken collectively, these 6 elements are proposed to produce effective interactions between proactive prepared practice teams and informed activated patients that lead to better functional and clinical outcomes. This model has also been adapted by the World Health Organization as the framework for care of a broad range of chronic conditions.¹⁰ Although the evidence base for some of these elements is incomplete, it is clearly a comprehensive and promising way to conceptualize a path to better care for people with chronic conditions.^8,11

The problem is that we have no complete examples of an implemented CCM and no specifics about either the best care changes to make or the most-effective change process to use for implementing them. Also, we only recently are beginning to have techniques to measure the presence and functioning of the CCM elements, so there is little or no information about the relationship between the presence of CCM elements and indicators of care quality. One current measurement system depends upon a survey of health care professionals in the care system (the ACIC or Assessing Chronic Illness Care), another on a survey of patients (PACIC or Patient Assessment of Chronic Illness Care), and the third on ratings by an external team (a RAND instrument).^12–14 More information is needed about how well these instruments work.

If we are to have real guidance for care delivery organizations interested in improving chronic disease care, we will need specific examples of the care changes needed for each CCM domain, good measures of their implementation and use, and more information about the best change processes for implementing them. When a large multispecialty medical group in the Midwest decided to transform its approach to care for patients with chronic diseases by implementing the CCM elements beginning in 2002, this natural experiment provided an opportunity to study both the implementation of the CCM and changes in care quality for patients with diabetes, coronary heart disease, and depression.

Thus, this article and the one that follows are among the first attempts to assess comprehensively an effort to implement the CCM.¹⁵ Here we report on the use of the ACIC, both to evaluate the success of CCM implementation and to learn whether that implementation was associated with changes in various quantitative measures of care quality for patients with these 3 common chronic conditions. In the article that follows, we report on a parallel effort using qualitative methods to understand the issues associated with the change process used in the implementation. Together these articles represent a planned effort to use the muti-method approach to research that is becoming more widely accepted as important to providing translatable lessons from research to practice.^16–18

METHODS

Preintervention Context

The medical group studied was a 600-physician multispecialty group in the Minneapolis-St. Paul metropolitan region. At the time its leaders made the decision to implement the CCM in primary care, this group provided care for about 300,000 patients, two thirds of whom were members of one health plan; the others were covered (or not) by a variety of insurance mechanisms. The demographic characteristics of these patients reflected those of the Twin Cities metropolitan area, with about 14% minority patients, but there was considerable variation in the number of patients and their characteristics at each clinic.

The 18 core primary care clinics in the medical group also had varied staffing, ranging from 3 family physicians to 16 pediatricians, internists, and family physicians. Most clinics also had 1 or 2 nurse-practitioners, and some also had a variety of other subspecialists. The clinicians were assisted by licensed practical nurses or medical assistants, usually working in dyads with no formal larger care teams. Although there were 1 to 3 registered nurses in each clinic, they mostly worked on their own to respond to patient telephone calls, facilitate medication refills, and provide selective services for patients with complex conditions. Two clinics had piloted an electronic medical record for several years, but the rest relied on a paper chart supplemented by a rudimentary separate electronic record that was not accessible in the examination rooms. Each clinic was managed coequally by a physician and an administrator, who reported to a similar leadership dyad for all of primary care.

Intervention

The changes involved in the group’s care transformation efforts addressed all 6 elements of the CCM to a varying extent, although the main focus was on delivery system design. Although the change strategy was revised several times during this period, from 2001 to 2004, the principal changes involving chronic disease care included the 5 listed below.

1. Delivery system redesign

   • Redesigned care teams, called prepared practice teams, each composed of a clinician, rooming nurse, shared registered nurse, and shared receptionist
   • More delegation of care to nonphysicians on the team, including more use of standing orders
   • Enlargement of the care focus from that of an individual visit to include previsit, postvisit, and between-visit efforts to organize information and outreach to patients for services they needed
   • Continued emphasis on same-day appointment access to each patient’s clinician
2. Clinical information system

   • Implementation of a complete electronic medical record and an associated reorganization of work flows for reporting test results, posting clinician reminders, etc
3. Decision support

   • Construction of additional or more-advanced practice systems (such as reminders and patient registries) to implement evidence-based guidelines
4. Self-management support

   • More emphasis on systematic use of patient education brochures, as well as referral of diabetes patients to internal patient educators and classes
5. Health care organization

   • Active organizational leadership actions to encourage and support the changes and to improve the spread and standardization of best practices
   • Development of a clinic-based pay-for-quality performance plan for diabetes, with supporting performance measures at the clinician and clinic levels
6. Community linkages

   • Participation in several community activities designed to support and improve care quality, including active membership in a statewide collaborative quality improvement organization of medical groups and health plans, as well as participation in diabetes improvement activities sponsored by the Minnesota Department of Health.

Leaders of the medical group fluctuated in the change process and focus of change, which is more fully described in the associated article.¹⁵

Evaluation

Because 1 of the 18 clinics was a teaching facility, this study was limited to the 17 more typical primary care clinics. When the study began, the only available validated instrument for measuring CCM-related care delivery changes was the ACIC survey. Developed and validated by Wagner et al, the ACIC had previously been used to evaluate implementation of the CCM in several collaborative improvement efforts.^7,13,19

The ACIC requires that internal participants or observers rate a series of 28 component items grouped to correspond with the 6 CCM elements, with 3 to 6 items per element. Items are each scored on a scale from 1 to 11, with higher scores indicating greater implementation. We modified the published approach to rating the ACIC components through group consensus by instead asking the administrative site supervisor in each clinic to complete this rating, first in June 2002 before any implementation had occurred and again in November 2004 after substantial change efforts had taken place and the prepared practice team concept had been implemented. The scores for each of the 3 to 6 component items within a CCM element were averaged to obtain a scale score for each CCM element. Although each item represented a different aspect of a CCM element, each contributed equally to the score for that element. Finally, the 6 element scores were averaged to arrive at an overall score for each clinic. Chronbach’s α for the overall scores were very high, 0.95 in 2002 and 0.93 in 2004, with ranges by element of 0.86–0.93 in 2002 and 0.78–0.92 in 2004.

We chose to measure the quality of care for 3 separate conditions (diabetes, coronary heart disease, or depression) because we wanted to assess the ability of the CCM to improve care across conditions and because the medical group had been the object of focused improvement efforts. Using administrative data, we developed case definitions to reliably identify medical group patients with each of these conditions and to create cross-sectional populations of patients with each condition for 2002 and 2003–2004.²⁰ For each condition, the positive predictive value of its case definition proved to be at least 0.95. This method identified about 7,500 diabetes patients, 3,750 heart disease patients, and 3,000 patients with a new diagnosis of depression in each year for use in the analysis.

To capture as late as possible any data that might document an effect of the intervention, the population for 2003 that was still present in the first half of 2004 had its rates for each of these measures based on the year from July 2003 to June 2004 rather than the calendar year 2003. The proportions still present in 2004 were 92.5% for diabetes, 92.9% for heart disease, and 88.9% for depression.

Next, we created measures of care quality for each condition that would be available from health plan claims data and based on national guidelines. Having to rely on this data source meant that we could measure care only for the approximately 90% of the patients of these clinics who had such insurance. For diabetes and heart disease, the measures we selected were patient-centered composite measures of the proportions of patients with those conditions who have achieved both of 2 separate guideline-recommended goals.²¹ Such combined measures are harder to achieve than individual measures, so they have lower performance rates that allow more room for change.²² This composite measure method was highlighted recently in the IOM report on national health care measurement standards.²³ The depression measures are similar to standard HEDIS measures of depression treatment. These measures for each 12-month period were as follows:

1. Diabetes

   • Composite process: percentage of patients with at least 1 glycated hemoglobin (A_1c) tests and 1 low-density lipoprotein cholesterol (LDL) test
   • Composite outcome high: percentage of patients with A_1c <8% and LDL <130 U/L
   • Composite outcome low: percentage of patients with A_1c <7% and LDL <100 U/L
2. Coronary heart disease

   • Composite outcome: percentage of patients with an LDL test and LDL <100 U/L, or if >124 U/L, the measurement improved by 20% during the next 12 months
   • Cardiac event rate: percentage of patients with at least 1 cardiac event in the calendar year, including death from any cause, acute myocardial infarction (ICD-9 code 410.xx), coronary syndromes (411.x), or angina (413.x) if angina was the principal discharge diagnosis for inpatient care
3. Depression treatment

   • Acute phase: percentage of patients with a new diagnosis of depression who had an antidepressant prescribed and were still on any antidepressant 84 days later
   • Continuation phase: percentage of patients with a new diagnosis of depression who had an antidepressant prescribed and who were still taking any antidepressant 180 days later
   • No follow-up: percentage of patients with a new diagnosis of depression who had an antidepressant prescribed in primary care and no primary care follow-up visits with a depression code in the next 180 days

All analyses treated the clinic as the unit of analysis and weighted each of the 17 clinic-level observations by the number of patients nested within whose data contributed to that observation. Paired t tests assessed whether there was significant improvement or decline in ACIC elements or quality measures at the clinics between 2002 and 2004. Change in ACIC score was computed by subtracting the 2002 clinic scores from the 2004 clinic scores, so that a positive score indicated an improved rating; the same approach was used for each quality measure. Pearson correlations between difference scores at each clinic quantified the extent to which changes in ACIC scores were associated with changes in quality measures. We also constructed a regression model for each quality measure change score in which change in ACIC predicted change in the quality measure while controlling for the age and sex of patients in each clinic.

RESULTS

In 2002, 16 site supervisors returned completed ACIC survey instruments, and in 2004, all 17. Because of changes in personnel, however, in 2004, 7 of the 17 responding site supervisors were new since 2002 (41%). Table 1 ▶ shows the overall and elemental scores and changes during the intervention period. Although all scores increased, those for delivery system design and self-management support changed the least and were not significant at P <.05. The overall change of 1.42 represents a 24% improvement in CCM implementation, although there was considerable variation among the clinics, and 4 clinics actually had lower scores at the follow-up time period (Table 2 ▶).

Table 1.

ACIC Score Changes After Implementation

Dimension	2002	2004	Change	P Value	Range
No. of clinics	16	17	16
Overall	5.8	7.2	1.4	.02	−2.6 to +5.4
Delivery system design	5.7	6.8	1.2	.11	−3.0 to +5.1
Self-management support	6.1	7.0	1.1	.08	−4.0 to +3.8
Clinical information system	5.2	6.7	1.6	.01	−1.2 to +6.1
Decision support	6.0	7.3	1.4	.10	−3.5 to +7.5
Delivery organization	6.4	8.0	1.7	.01	−3.8 to +5.7
Community linkages	5.7	7.2	1.6	.03	−3.3 to +6.3

ACIC = Assessing Chronic Illness Care.

Table 2.

Overall ACIC Score Changes by Clinic

Clinic	2002	2004	Change
1	1.7	6.5	4.8
2	5.4	5.3	−0.1
3	5.9	7.6	1.7
4	5.0	8.4	3.4
5	5.0	6.3	1.3
6	7.2	7.3	0.1
7	9.5	9.8	0.3
8	8.4	5.8	−2.6
9	–	5.4	–
10	6.5	7.5	1.1
11	4.9	7.9	3.0
12	4.6	7.4	2.8
13	4.7	4.0	−0.7
14	9.0	7.4	−1.5
15	3.8	9.2	5.4
16	6.7	9.7	2.9
17	5.2	6.1	0.9
Mean (SD)	5.8 (2.0)	7.2 (1.6)	1.4 (2.2)
Range	1.7 to 9.5	4.0 to 9.8	−2.6 to 5.4

ACIC = Assessing Chronic Illness Care.

Table 3 ▶ documents the average clinic scores and changes for each of the quality measures between 2002 and 2003–2004. Each measure improved significantly except for the diabetes process and depression medication measures, whereas there had been no significant change in any of these measures in the preintervention year between 2001 and 2002 (data not shown).

Table 3.

Quality Measure Changes From 2002–2004 (n = 17)

Measures	2002	2004	Change	Clinic Range	P Value
Diabetes
Number	7,423	7,650		145–826
Composite process, %	63.5	63.1	−0.4	−9.1 to +7.4	.70
Composite high (8/130), %	43.9	52.4	+8.5	+4.2 to +16.3	.001
Composite low (7/100), %	15.7	25.5	+9.8	+4.3 to +17.7	.001
Heart Disease
Number	3,726	3,761		55–473
Composite control, %	46.8	57.8	+11.1	+2.9 to +19.2	.001
Cardiac event rate, %	17.2	11.4	−5.9	−13.5 to +1.9	.001
Depression
Number	3,154	2,788		37–400
Acute phase (>84 d), %	68.0	69.1	+1.1	−5.6 to +8.7	.39
Continuation phase (>180 d), %	51.2	51.9	+0.6	−8.4 to +9.1	.61
No primary care follow-up visit, %	14.5	10.9	−3.5	−13.1 to +11.2	.02

In Table 4 ▶, the correlations between change in ACIC score and these quality measures are shown. Only 4 are above 0.4 (all for diabetes), and only the relationship of the most-intensive diabetes control (composite low) to clinical information system and decision support are significant (P = .049 and .029 respectively). For heart disease and depression, 70% of these insignificant correlations are directionally negative. The results of the regression models were not meaningfully different from the results of the correlation analyses, so the correlation coefficients are presented for ease of interpretation.

Table 4.

Correlations Between Change in ACIC and Change in Quality Measures

Measure	Overall	Health Care Organization	Delivery System Design	Self-Management Support	Clinical Information System	Decision Support	Community Linkage
Diabetes
Composite process	+0.17	+0.20	+0.33	+0.21	+0.01	+0.03	+0.14
Composite high	+0.28	+0.17	+0.21	+0.16	+0.44	+0.32	+0.12
Composite low	+0.42	+0.37	+0.17	+0.34	+0.50*	+0.54*	+0.25
Heart disease
Composite control	−0.23	+0.08	−0.26	−0.18	−0.29	−0.15	−0.40
Cardiac events	−0.01	−0.07	+0.16	+0.07	−0.06	+0.01	−0.18
Depression
Acute phase	−0.26	−0.07	−0.31	−0.35	−0.30	−0.02	−0.38
Continuation phase	+0.05	+0.09	+0.02	+0.06	−0.14	+0.22	−0.08
No primary care follow-up visit	−0.10	−0.20	−0.02	−0.12	+0.05	−0.13	−0.10

ACIC = Assessing Chronic Illness Care.

* P <.05

DISCUSSION

This medical group appears to have improved both its implementation of most elements of the CCM and its quality of care for patients with diabetes or heart disease (but not much for depression) during this 2-year period. Our analysis, however, has been unable to find much evidence of a relationship between the measures of CCM implementation and quality improvements for patients with these conditions. Although the degree of measurable change in the CCM was not large, it seems reasonable that the elements of decision support and clinical information systems are the only ones to show any relationship to care improvement, as they had the greater quantitative increases and they are closely related. Whereas delivery system design was indeed the area getting the most attention in the medical group’s transformation effort, our following qualitative article suggests that only foundational work had been done by the time of our analysis in 2004.¹⁵

Unfortunately, the RAND approach to measuring the presence of the CCM elements was not available to us in time to use it for this analysis. Pearson et al, however, found significant positive relationships between their RAND measure of the depth of implementation and differences in self-reported ACIC scores before and after the implementation.¹⁴ The depth ratings were significantly correlated with ACIC improvements in the CCM overall as well as for 5 of the 6 elements (not self-management support). Although these correlations were positively correlated, the size of the correlations was relatively modest (r = 0.35 to 0.44).

There are important limitations to these findings. The internal validity of the relatively subjective ACIC variable may be questioned, as it was complicated by the turnover in site supervisors between 2002 and 2004. The sample size of 17 clinics could also negatively affect the power of the analysis, because with such a small sample, a very large correlation coefficient would be needed to be considered significant. It is also possible that the changes undertaken, at least in this early stage of their implementation, had not been sufficiently great or been in place for enough time to create the kind of care differences that could lead to their being the cause of the improvements noted.

We do recognize that the ACIC has the potential to be affected by a person’s role in the clinic, duration of work in that site, and understanding of the CCM elements. Previously published work has shown, however, that ACIC is capable of measuring change and that those change measurements are relatively well-correlated with independent faculty ratings of progress of specific quality improvement teams.¹³ ACIC has also been used by the later developers of other measurement tools to compare with their instruments.^12,14 Feifer et al have further shown in a study of only 9 clinics that there were strong correlations between CCM systems as measured by the ACIC and measures of care quality for diabetes and heart disease.²⁴

Although these findings must be considered preliminary and tentative, there are a number of interesting implications. One is that 2 of the CCM elements which failed to achieve significant improvements (delivery system design and self-management support) may be the most important elements in improving care. That problem, even though the organization’s change efforts had focused especially on the former CCM element, will be discussed further in the qualitative article that follows. In an analysis of an earlier relation between ACIC scores and changes in care quality for diabetes, we found that only delivery system design was significantly related to change in care quality, whereas self-management support nearly achieved signicance.²⁵ Feifer et al also found those 2 elements to be the ones positively correlated to a composite score for 7 chronic conditions.²⁴ Other studies have suggested that delivery system redesign using electronic medical records helps to improve testing rates, but is not sufficient to improve control of diabetes (although Table 4 ▶ suggests that may not be necessarily true).^26–28

Despite depression being one of the foci for this improvement effort, our main measures of quality of depression care did not change, perhaps because only a few of these clinics selected depression care as an improvement focus. Moreover, depression was a relatively new quality initiative compared with diabetes and heart disease, measurements were not widely available to guide improvement activity in this area, and the strategies to improve care were less well established. It is therefore not surprising that less improvement in depression care was found in this setting. It is also interesting that, regardless of condition, the quality measures showing improvement during this period were those measuring changes in outcome (such as levels of A_1c and LDL) rather than those reflecting changes in the process of care, such as testing rates and depression medication use. We might have expected the opposite as the result of efforts to change the care process for these patients. It is possible that, despite the attempts to improve and standardize processes in support of the physician-patient visit, the principal driver of quality improvement may have been pharmacologic intensification resulting from increased clinician knowledge and awareness of chronic disease care.²⁹

Finally, although the CCM seems correct in its emphasis on systems and on certain types of systems, and although there is some evidence for the effectiveness of most of its elements, this evidence is mixed and mostly limited to the 4 elements of delivery system redesign, clinical information systems, decision support, and self-management support.^7,9,11,30,31 It is possible this model is too broad and conceptual to serve as a practical guide to successful implementation. The following article provides an in-depth qualitative analysis of this medical group’s care changes and change processes; our analysis supports the possibility that the theoretical nature of the CCM was a large barrier to implementation.¹⁵ Clearly, we need more research that helps us to identify which interventions and care changes really matter the most and how they are best implemented. More-sensitive, reliable, and valid tools for measuring the presence and function of specific care delivery processes and systems would be helpful to researchers, as well as to those who are struggling in practice to improve care for their patients with chronic diseases.

Annals Journal Club selection—see inside back cover or http://www.annfammed.org/AJC/.