The Indiana Chronic Disease Management Program

Abstract

The Indiana Chronic Disease Management Program (ICDMP) is intended to improve the quality and cost-effectiveness of care for Medicaid members with congestive heart failure (chronic heart failure), diabetes, asthma, and other conditions. The ICDMP is being assembled by Indiana Medicaid primarily from state and local resources and has seven components: (1) identification of eligible participants to create regional registries, (2) risk stratification of eligible participants, (3) nurse care management for high-risk participants, (4) telephonic intervention for all participants, (5) an Internet-based information system, (6) quality improvement collaboratives for primary care practices, and (7) program evaluation. The evaluation involves a randomized controlled trial in two inner-city group practices, as well as a statewide observational design. This article describes the ICDMP, highlights challenges, and discusses approaches to its evaluation.

The spiraling costs and substandard quality of chronic illness care have elicited an array of efforts to redesign its delivery (Lozano et al. 2003; Rothman and Wagner 2003). These efforts include paying incentives to physicians who meet quality goals (Epstein, Lee, and Hamel 2004), reshaping individual practices by cultivating the chronic care model (Wagner et al. 2001), and applying that model in large-scale disease management programs (Bodenheimer, Wagner, and Grumbach 2002a; Finkelstein et al. 2002; Villagra 2004; Wheatley 2001). The chronic care model involves six areas of change: (1) community resources and policies, (2) health care organization, (3) self-management support, (4) delivery system design, (5) decision support, and (6) clinical information systems (Bodenheimer, Wagner, and Grumbach 2002a). Implementation of the model has been effective in some settings but has produced mixed or ineffective results in others (Bodenheimer, Wagner, and Grumbach 2002b; Coleman et al. 1999; Olivarius et al. 2001; Schonlau et al. 2005; Solberg et al. 2000).

Attracted by the promise of lower costs and better quality of care, large payers and purchasers are turning increasingly to chronic disease management programs (Gold and Kongstvedt 2003; Wilson 2003; Smith 2003). The focus is often on high-risk patients. However, the programs' overall effectiveness may be greater if they also attend to the wider, moderate- to low-risk population—those with less complicated illness (Cook et al. 1995; Rose 1992). State Medicaid agencies urgently need to improve care and control costs (Rosenbaum 2002). After an unusual period of sound finances in the late 1990s, latent forces reemerged (Iglehart 2003; Weil 2003): Medicaid agencies faced growing numbers of eligible members, rising costs of medical care, and pressure from state legislatures, themselves in budget crises, to rein in expenditures.

In the past decade, more than twenty state Medicaid agencies instituted disease management programs. Some states contracted with commercial disease management vendors, while other state programs used a combination of commercial and homegrown elements (Faulkner 2003; National Conference of State Legislatures 2003; Wheatley 2001; Gillespie and Rossiter 2003). Even though vendors promise a large return on investment (Groeller and Silva 2003), high-quality studies of the cost-effectiveness of disease management programs have shown mixed results. Therefore, much more study is needed if the vendors' claims are to be supported (Bodenheimer, Wagner, and Grumbach 2002b; Fireman, Bartlett, and Selby 2004; Office of Program Policy Analysis and Government Accountability 2004; Rothman and Wagner 2003; Wilson 2003). In addition, some vendor programs have been controversial. In one state, a pharmaceutical company took up a Medicaid disease management program and simultaneously obtained preferred status on the Medicaid formulary (Groeller and Silva 2003).

Several states have chosen a combination of commercial and local approaches. In 2001, the program Florida: A Healthy State was implemented jointly by a pharmaceutical company subsidiary and the Florida Agency for Health Care Administration. The largest Medicaid disease management initiative to date, it enrolled more than 150,000 Floridians with diabetes, congestive heart failure (chronic heart failure), asthma, or hypertension. Intensive care management was provided for a high-risk subpopulation (approximately 19,000 people) identified by analyzing claims data. Care managers were based in ten catchment areas centered on hospitals throughout the state, and high-risk participants outside these ten areas were contacted by an out-of-state telephone center run by a commercial disease management vendor (White et al. 2005). To date, evaluation of the program has produced inconclusive or even conflicting results (Florida Agency for Health Care Administration 2004; Office of Program Policy Analysis and Government Accountability 2004). Other states also are combining disease management vendor services with state-coordinated services. Mississippi contracted with a commercial vendor to establish an in-state, nurse-staffed telephone center. This program collaborated with local partners, including the University of Mississippi Medical Center, the Mississippi Primary Health Care Association and its Community Health Centers, and a vendor that provides field-based nurse care management (Crowder 2003). Mississippi's program also contained an innovative component, led by pharmacists, to provide drug therapy management for patients with diabetes or asthma (Young 2003). Other states, such as Texas (for its Medicaid members with diabetes), required their Medicaid managed care organizations to offer disease management services; West Virginia implemented a state-led (Bureau of Public Health and the Bureau of Medicaid Services) disease management program for its Medicaid members with diabetes or asthma (Association of State and Territorial Health Officials 2002).

In Indiana, the Medicaid director, Melanie Bella, and the state health commissioner, Dr. Gregory Wilson, first considered proposals from disease management vendors in response to a state-issued request but ultimately decided instead to assemble their own program, primarily from state and local resources. Vendors emphasized the money they could save, but their programs were nonetheless expensive, delivered several of the key components of chronic disease management services from out-of-state centers, and, if implemented, seemed unlikely to significantly enhance the capabilities of Indiana's existing systems of care. A state-assembled program, it was reasoned, would be challenging to build and implement but might offer the potential for sustained, large-scale changes within Medicaid and other systems in Indiana. Such a program could incorporate activities typical of disease management vendors, centered outside of the medical care system, but also could advance the chronic care model in physician practice (Casalino 2005). In testimony before the U.S. House of Representatives, Director Bella outlined the objectives of the Indiana Chronic Disease Management Program (ICDMP):

• “Provide higher quality care to Medicaid recipients that improves health status, enhances quality of life and teaches self-management skills.

• Provide support to primary care providers and integrate primary care with case management.

• Utilize and strengthen the public health infrastructure.

• Reduce the overall cost of providing health care to Medicaid patients suffering from chronic diseases.

• Achieve long-term results by changing the way primary care is delivered across the state, not just for Medicaid.” (Bella 2003)

The Indiana leadership also opted for a rigorous evaluation of the ICDMP. A partnership was developed with the Regenstrief Institute, Inc., a medical informatics and health services research institute affiliated with the Indiana University School of Medicine, to provide technical consultation and to design and conduct the evaluation. The authors of this report are the ICDMP consultants and evaluators at the Regenstrief Institute. The Institutional Review Board of Indiana University-Purdue University Indianapolis approved the Regenstrief Institute's activities in the ICDMP.

Following the Indiana state legislature's direction, the Office of Medicaid Policy and Planning (OMPP) launched the ICDMP on July 1, 2003, for eligible participants with diabetes or congestive heart failure (CHF). The ICDMP was implemented in stages. The central third of the state began ICDMP activities in July 2003. The northern and southern thirds of Indiana, as well as a statewide asthma disease management program, were added during 2004. The program's implementation was staggered regionally for logistical reasons, to facilitate hiring and training personnel, and for efficient outreach to physicians and participants. Diabetes, CHF, and asthma were selected because of their prevalence, morbidity and mortality, costs to the state, and the potential effectiveness of a disease management initiative for these conditions.

The state also staggered the program's implementation in two large urban group practices, to enable comparison between participants who became eligible for ICDMP services in 2003 with contemporaneous controls who would become eligible approximately eighteen months later. One of these two large urban group practices is home to the Regenstrief Medical Records System (RMRS), a state-of-the-art electronic medical record that includes diagnoses, laboratory results, progress notes, discharge summaries, vital signs, and a computerized physician order entry system (McDonald et al. 1999; Tierney et al. 2003). The RMRS would prove valuable, along with Medicaid administrative data, in the development and evaluation of the ICDMP.

This article focuses on the ICDMP for adults with diabetes and/or CHF. We first describe the main components of a large-scale disease management program and discuss some of Indiana's innovative approaches in assembling them. Then we look at several of the developmental hurdles faced by an “assemble-rather-than-buy” initiative. Finally, we outline the planned evaluation and highlight some of its inherent challenges.

Program Components and Assembly

In a state that buys a complete Medicaid disease management initiative from a vendor, responsibility for shaping the initiative probably rests largely with the vendor. In contrast, a state that assembles its own disease management initiative must spend considerable energy and time designing and planning: building consensus, reviewing what other payers have tried, analyzing in-state data, exploring policy options, creating materials and tool kits, awarding contracts for each of the program's components, and guiding and coordinating the contractors' activities. During the winter and spring of 2003, the Chronic Disease Advisory Council, led by the Indiana state health commissioner, drew up consensus guidelines for the primary care of diabetes and congestive heart failure. A two-day ICDMP planning meeting was convened in March 2003 by the Medicaid director and the health commissioner, with the MacColl Institute of Seattle as the facilitator. In addition to the state agencies and the MacColl Institute's national expert panel, the planners included the developers of the guidelines, the programmers of the information system, the leaders of organizations that would provide the nurse care management and telephonic care management services, and representatives from the Indiana University School of Medicine and the Regenstrief Institute. Work groups (each about eight to twelve people) were then formed to design the ICDMP's components. The work groups covered the identification and risk stratification of eligible participants, nurse care management, telephonic care management, the information system, and the design of educational materials for patients. The work groups were composed of Indiana Medicaid and Department of Health staff, the contractors that would provide each service, and management consultants. Each work group included at least one adviser from the Regenstrief Institute. These groups met throughout the spring of 2003, and the components that they developed are described next and are depicted in Figure 1.

figure 1.

Components of the ICDMP.

Identification of Participants and Creation of Regional Registries

Shortly before the beginning of the ICDMP, Indiana Medicaid introduced a managed care program (Medicaid Select) for its aged/blind/disabled Medicaid population. Medicaid Select is not risk-based managed care (RBMC); rather, it is primary care case management (PCCM), in which each member's primary medical provider (PMP) receives a $4 monthly fee to coordinate care and referrals. The member's health care otherwise is reimbursed on a fee-for-service basis. About 60,000 (8 percent) of the 800,000 people in Indiana Medicaid are in Medicaid Select. Since the mid-1990s, Indiana Medicaid also has had a combined PCCM/RBMC program, called Hoosier Healthwise, for its much larger (N= 525,000) income-eligible and SCHIP (State Children's Health Insurance Program) populations, primarily women and children. More than 85 percent of the ICDMP's participants with diabetes or CHF are members of the state's aged/blind/disabled (Medicaid Select) program, a population that tends to be less mobile and to have more continuous enrollment in Medicaid. More than 40 percent of the participants with diabetes or CHF have dual Medicaid/Medicare eligibility. The Indiana Medicaid leadership specified Medicaid Select or Hoosier Healthwise membership as a prerequisite for participation in the ICDMP. Accordingly, the ICDMP excludes people in nursing homes or other “waiver” eligibility categories—settings in which, it is hoped, integrated care is already provided. For the same reason, the ICDMP also excludes members in risk-based managed care.

The initial selection of participants with a particular illness requires readily available data. Automated queries of Medicaid claims were created to identify people with diabetes or CHF, based on relevant International Classification of Diagnosis (ICD-9) codes or disease-specific prescriptions in the previous year. The queries are repeated approximately every three to six months in order to find eligible participants not previously identified. These condition-specific lists of participants, sorted by the PMPs' practice location and county, constitute the regional registries of ICDMP eligibles. Participants are enrolled according to the selection criteria and are informed by mail of their ICDMP eligibility, although they may opt out. Participants also may become ineligible for various reasons, such as if they move into a nursing home, into a waiver category, or into risk-based managed care, or if they cannot be contacted after multiple attempts.

The first stage of the ICDMP began in July and August 2003 for thirty-two central Indiana counties (including Indianapolis) and enrolled approximately 4,500 adults who were identified as having diabetes and/or CHF (the vast majority had diabetes). Among these 4,500 people identified and assigned to ICDMP, about 11 percent opted out and another 15 percent otherwise became ineligible during the first two years of the program.

Risk Stratification

Risk stratification was performed in order to assign ICDMP-eligible participants to different program services. In the ICDMP context, “risk” refers to the likelihood of a participant's higher or lower utilization of health services in the next year, and the cost of these services to Indiana Medicaid. Cost-prediction models were created and validated for Indiana Medicaid members with diabetes and/or CHF (Li et al. 2005). Based in part on empirical cost distributions for its populations with chronic illnesses, Indiana decided to assign nurse care managers to the highest-risk 20 percent of eligible participants with CHF or diabetes, while telephone care coordinators would focus on the lowest-risk 80 percent of participants. (The mean prior-year costs for the high-risk 20 percent were at least five times higher than those for the low-risk 80 percent of participants.) After completion of the nurse care management intervention, the high-risk participants transitioned to telephonic care management. The nurse care management and telephone care management services are described later.

Stratification and service prioritization involve estimating a risk score for each participant and choosing a threshold for assignment to high- or low-intensity care management. The stratification algorithm was developed using two years of retrospective claims data: a hypothetical disease management cohort was created using data from year 1; candidate predictors from year 1 were modeled against costs in year 2 to create a parsimonious risk stratification algorithm for adults with CHF and/or diabetes. Of the many candidate predictors modeled, the final algorithm involved three predictors: total net Medicaid claims costs in the past year, Medicaid aid category (e.g., the beta-weight for the “disabled” category was higher than that for the “aged” category), and total number of unique medications filled in the past year (Li et al. 2005).

After comparing various algorithms, the Regenstrief Institute consultants recommended that for ICDMP's purposes, the most relevant metric was not R² but classification efficiency: minimizing mismatches in a 3 × 3 table of observed versus predicted costs. The 3 × 3 table categorized individuals into the highest 20 percent, the next 30 percent, and the lowest 50 percent of predicted and observed costs. Because the cost-effectiveness of a disease management initiative depends partly on how well it targets high-intensity intervention to those participants most in need, we tried to minimize the number of people with high predicted and low observed costs, or vice versa. In the future, the Indiana policymakers and the Regenstrief Institute team will consider enhanced stratification algorithms that incorporate self-reported data that the ICDMP is collecting by telephone on all program participants. Useful items might include self-rated health, expected health care utilization in the next year, or whether or not a participant named an individual doctor as his or her primary source of care.

Nurse Care Management for the Highest-Risk Participants

ICDMP-funded nurse care managers educate their patients, encourage self-management, facilitate their communication with physicians, and make referrals to community resources. Before visiting a participant, the nurse care managers typically begin by contacting the primary care physician and collecting data from the patient's medical record. The data elements are based on the Indiana guidelines, for example, for diabetes, the dates and results of the most recent A1C test, blood pressure, lipid profile, urine microalbumin test, fundoscopic eye exam, and lower extremity monofilament test, as well as the dates of the most recent office visit, dental visit, flu shot, and pneumococcal vaccine. The nurse care manager is then in a position to encourage the participants and the physicians to follow the guideline-recommended schedules for these activities.

At home visits, the nurse care managers establish a relationship, assess needs, and help participants set self-management goals. After home visits, the nurses usually work with the participants by phone, although in urban areas more than one home visit may be feasible. The Indiana Office of Medicaid Policy and Planning (OMPP) encourages the nurses to accompany participants to at least one doctor visit. The nurses do not manage medications under protocol or otherwise act in a direct clinical capacity. Rather, they provide educational support and activation, using self-management goal sheets and “stages of change” (Prochaska and DiClemente 1982) protocols developed or adapted for the ICDMP. The nurses encourage participants to maintain their supply of medicine and their adherence to physician-directed treatment plans. The nurses also encourage them to eat wisely and exercise, not smoke, and make and keep appointments with the PMP. The self-management goals for participants with diabetes include good foot care and self-monitoring of glucose levels, and, for participants with CHF, a low-sodium diet and daily weight measurement. Using the eight-item Patient Health Questionnaire (PHQ-8) (Kroenke, Spitzer, and Williams 2001), the nurses can screen participants for depression and then notify the PMP about those with scores indicating a high risk.

The nurses must be hired and trained for these care management roles, and effective coordination with the telephonic care management center (described in the next section) must be developed. The OMPP contracted with the Indiana Minority Health Coalition and the Indiana Primary Health Care Association to provide nurse care management services, with each county assigned to one organization. The nurses carry a tool kit—developed by the OMPP, the Indiana State Department of Health (ISDH), and collaborators—containing guidelines, self-management protocols, and educational and motivational materials. The nurse care management organizations give the state monthly reports on the nurses' caseloads and the participants' progress in self-management. In the first nine months of the program, nurse care managers made an initial telephone contact with approximately 40 percent of the 1,100 high-risk participants in central Indiana who had been identified in the claims data. A smaller percentage completed a home visit and then a full period of engagement with a nurse care manager. The nurse care managers visited the PMPs' offices to review the written medical records of most of the high-risk participants.

After approximately six months of interacting with a participant, the nurse completes a discharge summary, which is recorded in the information system. The telephone center then follows up with two brief scripted calls, two and six weeks later. The nurse care manager selects the topics for these calls, such as nutrition planning, diabetes medicines, self-monitoring of blood glucose, and making and keeping appointments. After these brief follow-up calls, the participants enter the queue of full-length outbound educational and motivational calls that the telephone center makes to other participants (described later).

Telephonic Care Management for All Participants

Some disease management programs are entirely telephonic; a stratified approach involving both nurse and telephonic care management affords a variety of design options. In the ICDMP, those participants not under nurse care management receive telephonic care management. Trained nonclinical personnel, the “care coordinators,” supervised by a registered nurse, call participants quarterly, if possible. The telephone call scripts are designed by the Regenstrief Institute, in collaboration with the telephone center, and are designed to stimulate self-care, encourage the provision of core medical care, and offer educational resources. The care coordinators use the ICDMP information system software (described in more detail later), which supports outbound telephone calls with a branching structure—tailored to the participant's responses—and storage of multiple-choice and free text responses for future analysis.

Scripts with condition-specific branches were created for four interactions between the telephone care coordinators and the participants, as well as for the follow-up calls made to those who recently completed a period of interaction with a nurse care manager. National and state evidence-based guidelines for the care of persons with diabetes and CHF were used to identify the content of the telephone “curriculum.” Motivational interviewing constructs (Emmons and Rollnick 2001), health care literacy concerns, and field testing shaped the scripts' diction. The first diabetes and CHF telephone call begins with solidarity-building and then turns to general questions about the participants' health and care. Flu shots are encouraged. The next calls make increased use of scripted, tailored dialogues. The second call focuses on medicine supply and adherence and encourages participants to keep a list of their medicines and to bring it to their PMP visits. At one point, the participant is asked to leave the telephone, to gather his or her medicines, and then to return; the care coordinator times this activity to gauge how much difficulty the participant had. Using scripted advice, the care coordinators discuss problems the participants might have had in obtaining their medicine or in taking it. This call also includes questions and messages about foot care, blood glucose monitoring, aspirin use, weight checks, exercise, smoking, and depression. The third and fourth calls offer a choice of topics, such as diet, physical activity, and preventing complications. Those topics not selected for one call are deferred to subsequent calls. The telephone center can mail ICDMP-designed low-literacy educational materials and a one-page, individualized summary of key points that were discussed, encouraging the participant to bring the summary to his or her next visit to the doctor.

The OMPP contracts for telephone care management services with its PCCM (primary care case management) administrator. This firm was already running a telephone center providing enrollment assistance to Medicaid members and PMPs. Several of the ICDMP telephone care coordinators were transferred from that call center and trained for their new role. The telephone center also responds to calls from participants, provides administrative support for the nurse care managers (e.g., helps them reach a participant), and makes referrals to community resources (e.g., for transportation).

Information System

An information system can contribute in various ways to chronic disease management. At a minimum, it serves as a central list of the selected and stratified participants. At its best, an information system provides decision support—making treatment guidelines available at the point of care (for clinicians or nurse care managers)—plus advanced registry functions such as clinical reminders and performance feedback for providers. For the ICDMP, the Indiana team adapted and augmented a Web-based system, CDMS (Chronic Disease Management System), which was developed by the Montana Medicare Quality Improvement Organization.

In the ICDMP, CDMS maintains a central list of participants who are assigned to nurse care management or telephonic care management. Indiana Medicaid's fiscal agent flags the ICDMP participants in its own system and sends nightly updates of eligibility information to CDMS. CDMS gives the nurse care managers their own lists and enables them to enter and retrieve data, care plans, and progress notes. The templates for the care plans and progress notes reflect the Indiana consensus guidelines and the ICDMP's self-management goal sheets for each chronic illness. The ICDMP created an elaborate CDMS telephone-center application to support the outbound, individualized telephone calls just described. It also maintains queues of calls scheduled quarterly, tracks each participant's call history, stores his or her responses to each item, and records the purpose of any inbound calls. CDMS continues to be modified in order to meet the program's needs: the linkage between the nurse care management and the telephone center applications is being strengthened to enable a nurse care manager to view a participant's telephone call responses, and a telephone care coordinator to view relevant data recorded by the nurse care manager.

The CDMS's clinical office application, a Web-based disease management registry that is used by several primary care practices in the western United States, was offered free of charge by the state of Indiana to any practice that wished to adopt it. The ICDMP offered the CDMS training, Web-based demonstrations, and a help desk for practices. Ideally, a system like CDMS in clinical offices would provide not only decision support and registry functions within the office but also a way to exchange data between the office and the nurse care managers or telephone center. However, this long-term vision of data integration has not yet been realized. Only one clinical office in Indiana has implemented CDMS completely. A few other practices, in their ongoing quality improvement activities, have made use of the program's eligibility lists stored in CDMS by the ICDMP administrators. The long-term hope is for a centralized information system to incorporate data from the offices' own billing systems, as well as from claims files from the Indiana Medicaid fiscal agent; the software and data would drive decision support and clinical reminders at multiple points of care.

Practice Change and Quality Improvement Collaboratives for Primary Care Practices

Quality improvement that supports planned and integrated care in office practice is an essential ingredient in the chronic care model (Bodenheimer, Wagner, and Grumbach 2002a). Promoting it for thousands of clinicians statewide is not easy; merely promoting guidelines is not enough (Cabana et al. 1999). One approach is to invite practices to participate in quality improvement collaboratives. The OMPP and ISDH conduct these collaboratives, based on the model for quality improvement in chronic care (Kilo 1999). The first series, focused on diabetes and CHF, began in June 2003 for approximately twenty practices in central Indiana. Five of the twenty were clinical sites within the Indiana University Medical Group. Many of the others were federally qualified health centers (FQHCs). These twenty practices provide primary care for approximately 15 percent of the aged/blind/disabled Medicaid population in central Indiana. Three new collaborative series were begun in 2004; they covered all regions of the state and focused on care for children with asthma as well as for adults with diabetes or CHF. Most of the participating practices were community health centers or FQHCs. The participating practices set quality improvement goals and reported their performance once a month. Ideas were also shared during monthly conference calls and through an email listserve. Indiana University subspecialists answered general educational questions from providers by email or during conference calls. The nurse care managers and telephone center leaders also attended the collaborative learning sessions.

The primary care collaboratives extend to only those practices that can be recruited to attend. The ICDMP also seeks to improve quality throughout Indiana. Provider tool kits—containing clinical guidelines and educational material for clinicians, and self-management and educational tools for patients—are mailed to all primary medical providers in the state who see Medicaid patients. Any of these practices (if there is an ICDMP-eligible participant on its patient panel) may be contacted by an ICDMP nurse care manager or telephone care coordinator or may encounter patients who have been engaged by the nurse care managers or telephone care coordinators.

Challenges in Program Development

Once the OMPP made the “assemble-rather-than-buy” decision, it needed to contract for new elements of a delivery system (telephone care coordinators and field-based nurse care managers) to enhance and strengthen the care of Indiana's Medicaid members with chronic illnesses. The OMPP was accustomed to acting as a payer, but creating and managing part of the delivery system itself were breaking new ground. The Regenstrief Institute, an academically affiliated research organization, needed to be both a support unit for program development and an evaluator. This kind of work is often at the mercy of the program's exigencies. In contrast to the typical, deliberative, “scientific laboratory” mission of an academic organization largely supported by external grants, this work tends to focus on making good use, quickly, of the best immediately available information.

Assembling the Registry: Knowing Who Is Eligible for the Program

There are many ways to define a registry using administrative data. In the ICDMP, the inclusion criteria for diabetes were an ICD-9 code for diabetes or a prescription claim for diabetes medicine or supplies in the past year; CHF was defined as an ICD-9 code for CHF or prescriptions for both digoxin and a loop diuretic during the past year, but different definitions may also be reasonable. The first telephone call serves as an additional eligibility check: people are simply asked whether they have diabetes or CHF. Using their responses as the “gold standard,” the administrative data query had a positive predictive value of approximately 95 percent for diabetes and 90 percent for CHF. In addition to the definitional challenge, creating a list of eligible participants may be technically complicated. With large, complex data systems, multiple iterations may be required to hone the list. Two different databases were available in the ICDMP for cross-checking or validation: (1) a second Indiana Medicaid administrative data warehouse and (2) the Regenstrief Medical Records System (RMRS). The former, jointly maintained by Indiana Medicaid and a vendor, offered a combination of data warehouse and analytic functions; it received its statewide data directly from Indiana Medicaid's fiscal agent and then provided additional processing (merged tables, aggregated variables, etc.). Midway through the ICDMP's first two years, the OMPP replaced its second data warehouse with a different system, from a new vendor. The RMRS data were available for a large group practice in Indianapolis, which included the majority of participants in the randomized controlled trial component of the evaluation. A related difficulty was the tie between eligibility for disease management and prerequisite eligibility for aged/blind/disabled or other primary care case management programs, which are not static. As a result of such complexity, disease management databases and their links with other Medicaid program data must be regularly kept up to date.

Risk Stratification: Identifying Who Is Most Likely to Benefit

Focusing intensive interventions on those with highest risk makes intuitive sense, but a program's effects on health and cost-effectiveness will be even greater if the intensive services are extended to those participants who can actually benefit from them. The best way to predict who is most likely to benefit has not yet been discovered. The ICDMP's approach thus far is to stratify based on expected costs in the next year. The telephone calls provide an additional opportunity to move participants from the low-risk to the high-risk (nurse care management) group. The telephone center's nurse supervisor may refer a participant to the nurse care management organizations if he or she seems confused or disorganized during telephone contacts or at the request of a primary care physician. During the first year of the program, nineteen participants were transferred to nurse care management in this way. As the ICDMP evolves, the risk-stratification methods might expand. We are exploring various additional approaches, which identify participants who have poor self-rated health, who have recently been to the hospital or emergency department, who are running out of critical medicines, and/or who have a new claim for a sentinel prescription.

Nurse Care Management: Guiding a Critical Function That Is Not Easily Observed

The success of a multifaceted disease management program may rest on whether its nurse care managers work effectively with high-risk participants and their physicians. While nurse care management is critical to the systems approach to chronic care, it is not easily observed, supported, and improved, because much of the nurses' activity takes place in the field (visiting patients and interacting with physicians in widely dispersed practice sites). Quality control is difficult to achieve, particularly when two different organizations are providing the services. In some ways, however, it is an advantage to have two different organizations, with differing styles and approaches, as the OMPP and ISDH continue to shape the delivery of nurse care management. One challenge is how much of the nurse care management should take place in the home instead of by telephone. Home visits can mean substantial travel time and costs, especially in rural areas.

Telephonic Care Management and Vulnerable Populations: Designing Effective Education/Motivation Tools

The population served by the ICDMP is quite ill, with one-third of the adults with CHF or diabetes self-rating their health as “poor.” Coexisting illnesses are common, and literacy is low. Many of the educational and motivational tools for this population, including the telephone call scripts, had to be developed anew. In addition, it can be difficult to reach the eligible population by telephone and to keep them in the program. After ten months of activity by the telephone center, two-thirds of the initial cohort of adults with CHF or diabetes had successfully completed the first telephone script. Many of the others could not be reached. But when participants were reached, most found the calls helpful. Ninety-seven percent of those with diabetes or CHF who completed the third call described it as helpful; they appreciated the specific content areas and/or just that someone cared enough to call to discuss their health. Clearly, maintaining access to this vulnerable population to deliver supplemental services is a challenge that has not been fully resolved.

Information Systems: Overcoming Technical and Other Barriers to Widespread Acceptance

Even under the best of circumstances, a new information system might not be welcomed by busy physician practices (Bodenheimer and Grumbach 2003). The ICDMP information system's slow connectivity and real-time operation discouraged its adoption. Group practices that already had electronic medical record systems were reluctant to “double-enter” data into a new system. In addition, building an automated interface between two electronic systems requires a lot of time and resources. In the ICDMP, software development for telephone and nurse care management was the principal focus of the programmers. An outreach and help-desk effort was made to encourage some primary care clinicians to adopt the CDMS office interface. But only one clinical office in Indiana has adopted CDMS completely, and a few other practices are using part of it. A very small staff of CDMS programmers in the ICDMP had to build an extensive new telephone-center application, adapt existing CDMS functions for use in nurse care management and in the ICDMP registries, and ensure appropriate data privacy, all while helping interested office practices with start-up and troubleshooting. Sometimes these priorities prevented the permanent storage of all data elements. The long-term vision—a centralized information system offering decision support, reminders, and provider feedback to clinical practices, as well as integration with nurse care management and telephonic data—has not yet been realized.

Collaboratives and Other Aspects of Quality Improvement in Clinical Practice: Achieving and Sustaining Results

A fundamental challenge involves improving quality in certain settings and then spreading these gains statewide. Participation in a quality improvement collaborative requires an investment, especially of time. Community health centers reacted more favorably than private practices to the collaborative invitations, and even with maximal turnout, the collaboratives would involve only a fraction of practices in the state. Despite the introduction of a new billing code for group patient visits, it has proved difficult for the collaboratives to stimulate such visits. Practices can resent mandates from large commercial insurers or state agencies. New interactions between nurse care managers and physicians can be dissonant, despite efforts by the Regenstrief and Indiana Medicaid ICDMP teams to improve the nurse care managers' “academic detailing” skills (Soumerai and Avorn 1990). (Academic detailing is another way to influence the practice behaviors of primary care clinicians. It is modeled on pharmaceutical face-to-face salesmanship [“detailing”]. What is marketed, however, is not a pharmaceutical product but evidence-based clinical practice, and, in the ICDMP context, the promotion of patient self-management and the chronic care model.) The OMPP and ISDH conducted extensive outreach and sent provider tool kits statewide to every primary care practice that sees Medicaid patients. The ICDMP probably has not affected the statewide rate at which physicians participate in Medicaid itself. There have been a number of reported successes in coordinated interaction among participants, physicians, and nurse care managers or telephone care coordinators, but spreading quality improvement and “planned care” throughout the state is an ongoing challenge.

There is one overarching challenge—to create sufficient capacity for implementing programs widely. The management tasks in assembling a new program like the ICDMP are complex. State Medicaid agencies have limited management capacity to create and run unprecedented new delivery programs (Thompson 1998). Budgets are limited; existing personnel are fully committed to existing programs; and time is short, given the legislative pressures for performance. The state government's relatively small staff must wrestle with huge volumes of complex data for many programs (not just the ICDMP). One of Indiana's strengths was the farsighted leadership of both the state legislature and the executive branch officials who developed the ICDMP. In many other states, disease management programs are being pressed to demonstrate cost savings within a fixed, short, time. In Indiana, both branches of government recognized the ICDMP's need to develop and grow before better health outcomes could be translated into cost savings. This supportive environment cultivated self-management, health promotion, and quality improvement as the primary goals, with (eventual) cost savings as a secondary consideration.

Program Evaluation and Its Challenges

Most disease management program evaluations involve observational designs. Their results can be contaminated by regression to the mean, temporal trends, or other less quantifiable biases. The ICDMP's evaluation was designed to avoid these biases, using both observational and randomized components: an observational comparison of the central, northern, and southern regions of Indiana; and a randomized controlled trial in two large urban group practices. The randomized trial, it is hoped, will help us identify and measure biases that could arise in the observational designs. The staggered implementation statewide permits an observational analysis of time intervals before and after each regional start date: a repeated measures design with boundary analysis. We will control as much as possible for secular and seasonal trends, geographic and practice variation, regression to the mean, and maturity of the disease management program.

The staggered implementation of the CHF and diabetes program in two urban group practices allowed a randomized comparison between participants who started in 2003 and those who started in 2005. Allocation was by clinic site: the largest clinic was randomly allocated to 2003 versus 2005; the others were then allocated one by one, by minimizing the between-group difference in number of participants. Participants with primary medical providers at the 2003-start sites became eligible for nurse care manager and telephone center services during the summer of 2003 (the “ICDMP” group); those with PMPs at the 2005 sites became eligible during mid-2005 (the “abeyance” group). Providers and staff from the 2003 sites were invited to the 2003/04 ICDMP quality improvement collaborative, and those at the 2005 sites could be invited to a 2005/06 collaborative, if one is held.

General Design Vulnerabilities: Power and Bias

A randomized trial in the midst of a rapidly implemented, larger-scale initiative faces all the usual challenges, plus some extras. The staggered implementation in two large group practices made possible the randomized controlled trial. But even in the state's largest group practice, the size of each arm (ICDMP and abeyance) was modest. The number of participants (and the statistical power) depended largely on the statewide, policy-driven inclusion and exclusion criteria and on the extent to which the nurse care managers and telephone care coordinators reached those who were eligible. There was no recruitment period to extend.

Other challenges include those associated with allocating individual Medicaid participants to each arm by clinical site (by cluster). Some participants moved between primary care sites during the evaluation period. In both the trial and the statewide time-series there can be bias from secular trends in costs and quality of care. With regional start dates scattered throughout the year, seasonal variation is an important consideration, but it will be controlled as much as possible by assessing pre-ICDMP variation across several years of longitudinal data.

The Nature of the Interventions: Moving Targets and Crowded Playing Fields

Disease management interventions must be adaptive and continuously improve: move or get “run over” (Finkelstein et al. 2002). An academically affiliated support team can suggest adaptations, such as a new systematic, timely intervention for participants prescribed a sentinel medicine, or extra training to help build the nurse care managers' academic detailing skills. A key challenge for the evaluation is that all ICDMP interventions and adaptations take place in the context of other, ongoing, practice improvement activities. In a large, urban group practice, these activities include randomized trials of other interventions, enhancements to information or electronic order-entry (Gopher) systems, and smaller-scale initiatives to improve clinical documentation. The ICDMP also takes its place amid other Medicaid benefits and eligibility policies, which themselves are not static. In these shifting, crowded playing fields, it might be impossible for the evaluation team to understand the full extent of activities outside the ICDMP, to isolate the effects of the ICDMP, or to “credit” the ICDMP for practice changes that it induces indirectly.

Time Frame and Horizon: What Durations Are Sufficient?

The best time frame (duration of the intervention) and time horizon (follow-up of the program's effectiveness) for an evaluation are not clear. How long does a new telephonic service unit require before the services it offers are mature? How long should a nurse care manager work with each participant before turning to others? How many years are required before the effects of chronic disease management (e.g., the prevention of diabetes complications) are realized? A particular difficulty for the preliminary evaluation is that a disease management program needs time to become fully functional. If it is evaluated before the program is fully under way, its beneficial effects may be substantially underestimated. It is also difficult to discern when that point has been achieved. The ICDMP's initial nurse care management caseload goals had not been reached by the end of the first year. Was the program not at full capacity, or had the goals been unrealistic? Beyond the evaluation of the ICDMP itself there are additional considerations; a state that assembles a program may generate healthy ripple effects among other insurers, employers, clinicians, and delivery systems, thereby increasing the societal return on its investment over time.

Discerning Start-Up versus Steady-State Costs: Judgment Calls in a “Game of Inches”

Identifying marginal costs can be difficult. The ICDMP had five distinct cost centers: (1) nurse care management, (2) telephonic care management, (3) the information system, (4) the development of materials used by service providers, and (5) the time spent by Indiana Medicaid staff to administer the program. Although the OMPP was able to provide the total resources consumed or amounts paid to the various contractors in each cost center, these figures had to be adjusted to deal with two separate issues. First, some of the resources devoted to the ICDMP in its first year were needed for the program's development and not its operation. Including such one-time start-up costs in the cost analysis would likely overstate the cost of delivering the program to participants in subsequent years. Second, the ICDMP may have operated below capacity during its first year because the participants had to be phased in by region and by targeted patient groups. Because some of the operating costs are fixed (i.e., do not change with patient volume), an excess capacity would result in average costs overstating the true costs of delivering the program to additional participants in subsequent years. An additional complication arose with the OMPP's statewide phase-in of mandatory risk-based managed care for children, pregnant women, and low-income families (completed in 2005). As that phase-in took effect, the asthma disease management for children in the ICDMP became the responsibility of the Medicaid managed care organizations.

For these reasons, we supplemented the cost data from OMPP with additional information provided by the contractors. We asked the fiscal officers or organizational leaders of each of these five cost centers to estimate the total amount of resources the ICDMP consumed between July 1, 2003, and May 31, 2005, to distinguish between one-time start-up costs (e.g., office equipment) and recurring operational costs. We also asked them to divide recurring operational costs: those that were affected by patient volume (variable costs like nurses' salaries and benefits) versus those that, while recurring annually, do not change with patient volume (fixed costs like insurance). Members of the evaluation team met with the fiscal officers to explain the differences between these cost types and to make sure that the different providers used consistent algorithms for allocating the costs. At this writing, the cost analyses of the ICDMP are in progress.

Data: Imperfections and Competing Priorities

Evaluations use data from several sources, each of which presents challenges. Our evaluation uses data from claims, an electronic medical records system, telephone center logs, and self-reported data from practices in the learning collaboratives. Claims data can be incomplete and inaccurate. State-of-the-art electronic medical records data also have limitations. Matching clinical and claims data can bring to light unexpected problems: fewer than 15 percent of A1C tests assayed at the point of care and fewer than 40 percent of all A1C tests could be matched to a specific Medicaid claim for A1C testing. Still, this ability to compare administrative and clinical data adds depth to the evaluation. The practices' self-reported data to the collaboratives were uneven: small- and large-group practices had different challenges in reporting, and some practices dropped out entirely. The collection of new data by nurse care managers and telephone care coordinators, as well as permanent storage of these data in the decision support system, competes with other priorities in the setting of scarce program resources and personnel.

Generalizability: Difficult for a Multifaceted Intervention with Complex Data

The results of this evaluation could be difficult to generalize. The novel nature of this tailored chronic disease management program, the idiosyncratic characteristics of the practice environment in Indiana, and the uniqueness of the Regenstrief Medical Records System may limit the generalizability of our findings to other settings. A comprehensive, computerized medical records system may be available in few places other than Indianapolis. The complex assembly required for success may not permit a straightforward transfer of this program to other states. Still, designers of other “assemble-not-buy” programs may find the various evaluation findings and lessons learned in Indiana to be useful.

The Precarious Stance of the Evaluator: Arbiter, Advocate, or Apparatchik?

Consider the role of the evaluator. The Indiana OMPP's decision to seek an independent evaluation is distinctive and commendable. The stance adopted by the university team in an effort to reduce role confusion is to provide the best available information for the OMPP managers—the university provides decision support; the state makes the decisions. But the line between decision support and policy advocacy is sometimes not easily drawn. The university-based teams (Regenstrief Institute, as well as the MacColl Institute) bring an a priori perspective that disease management initiatives are worthy endeavors. Working closely with the state on some specific aspects of the program's development (such as call scripts for telephonic care management) as well as on the program's evaluation, allegiances naturally strengthen. On the one hand, such allegiances are essential to a practical evaluation (Finkelstein et al. 2002). For states that choose a large-scale, “assemble” approach, one of the necessary ingredients for success might be effective collaboration between government and university teams. On the other hand, this role carries with it the danger of loss of objectivity—the independent evaluator as either advocate or “hired gun.” Both the state managers and the university-based team would like disease management to prove effective. Some processes that might help address this challenge include separating university personnel who work on the program's development from those who work on its evaluation (an approach that is not always feasible) or submitting the evaluation results for publication in a peer-reviewed journal (a process that both the state and the university support).

Conclusion

Rather than buy a chronic disease management program from a national program vendor, the state of Indiana decided to assemble and manage its own initiative. Disease management on a large scale presents challenges but also significant opportunities to improve self-management, quality of care, and health for populations with chronic illnesses. The challenges in program development and evaluation are not intractable. The innovative program in Indiana—in its successes and challenges—can offer valuable information to other states, whether they “build,” “buy,” or “assemble” a disease management program. The randomized controlled trial component and the combined use of Medicaid and electronic medical records data in Indianapolis will strengthen the evaluation of the ICDMP. We hope that by describing the ICDMP and explaining some of its principal challenges, we will provide others with a sense of the territory we have explored and the rocky fields we are attempting to till.