Return on Investment in Disease Management: A Review

Abstract

The results of 44 studies investigating financial impact and return on investment (ROI) from disease management (DM) programs for asthma, congestive heart failure (CHF), diabetes, depression, and multiple illnesses were examined. A positive ROI was found for programs directed at CHF and multiple disease conditions. Some evidence suggests that diabetes programs may save more than they cost, but additional studies are needed. Results are mixed for asthma management programs. Depression management programs cost more than they save in medical expenses, but may save money when considering productivity outcomes.

Introduction

Enthusiasm about DM programs is growing. This is evidenced by (1) the number of Medicare demonstrations underway testing alternative DM models, (2) legislative proposals that include provisions for widespread access to DM vendors, and (3) heightened interest by health plans and employers implementing these programs to improve patients' health and save health care dollars (Short, Mays, and Mittler, 2003; Lagorce, 2003; Foote, 2003).

Despite high expectations, the value of DM in controlling health care costs is still largely unknown. Recently, Foote (2003) offered a convincing argument that Medicare should strongly consider testing population-based DM programs in fee-for-service (FFS) Medicare. Foote's assertion, supported by a panel of experts assembled by the Health Insurance Reform Project, was that DM programs hold promise for improving the health of seniors, their quality of life, and their day-to-day functioning, while potentially saving Medicare money, by reducing unnecessary and expensive health care utilization. This line of thinking was also endorsed in testimony before the Senate Special Committee on Aging (Crippen, 2002).

As the DM industry continues to expand, with annual revenues increasing from $85 million in 1997 to more than $600 million in 2002 (Foote, 2003), it is important to examine the assumptions related to the financial impact of these programs on health care expenditures. As noted by Short and colleagues (2003): “In theory disease management and intensive case management programs offer health plans and employers opportunities to reduce health care costs and improve quality without resorting to restrictive utilization management or benefit reductions. In practice, DM programs must demonstrate cost savings if they are to help slow rapidly rising health costs.”

Evidence supporting the basic elements of DM has been accumulating for many years (Brown, 1990; DeBusk et al., 1994; Weingarten, et al., 2002; Bodenheimer, Wagner, and Grumbach, 2002). Reports of the actual experience with these programs are emerging in the private sector from employers and health plans. Evidence of significant improvements in quality of care and health outcomes as a result of DM can be found for several disease categories, including diabetes (Norris et al., 2002), heart failure (Roglieri et al., 1997; Rich et al., 1995), arthritis (Lorig et al., 2001), and depression (Wells et al., 2000). A literature review by the Institute of Medicine (2001) found substantial evidence that “programs providing counseling, education, information feedback, and other supports to patients with common chronic conditions are associated with improved outcomes.”

Understandably, most studies have focused on whether DM programs encourage application of evidence-based clinical guidelines in the treatment of acute and chronic disease, and whether adherence to guidelines improves patient health and functioning. However, a small subset of studies have also considered financial savings from DM and, in particular, whether such programs can achieve a positive ROI.

This article examines the limited, but growing research literature on medical cost savings, and ROI attributed to DM programs in five clinical areas: asthma, CHF, diabetes, depression, and multiple risk categories. These diseases were selected because there were several financial impact studies for each disease category. The DM programs studied may not be generalizable to other disorders, but these programs (with the exception of depression) are among the most frequently offered by leading DM vendors, as reported by Health Industries Research Companies (2003). Mental health problems are addressed by DM less often, but depression is a major comorbidity of asthma, diabetes, heart disease, and other disorders, and a highly prevalent disorder in its own right (Goetzel et al., 2003).

This review is focused primarily on benefits arising from savings in medical costs. We acknowledge that additional savings can be derived in other expense categories. These include reduced absence and disability; fewer on-the-job safety incidents and workers compensation claims; and reductions in on-the-job productivity losses (presenteeism). We limited our review to medical cost savings because this expense category is especially relevant to Medicare beneficiaries, most of whom are no longer employed, and the long-term viability of the Medicare Program is paramount in the mind of policymakers and Congress.

Although several financial impact studies are reviewed within each category, there are some notable limitations to this review that should be mentioned before our analysis is presented. First, DM is defined and practiced differently across studies, thus limiting direct comparisons. Some programs rely on face-to-face, clinician-based interventions, while others employ larger scale health plan- or employer-sponsored programs delivered by mail, Internet, and telephone to targeted patient groups. Some programs direct their activities at physicians by providing them with cues, reminders and prompts to deliver evidence-based medicine. Other programs bypass the physician and offer self-management programs directly to patients.

The studies examined cut across different age groups and were conducted in various settings. As such, information about the value of DM programs resembling those offered by managed care organizations and employers in the care of elderly Medicare enrollees is limited. In addition, the DM interventions uncovered in this review varied considerably in terms of their design, comprehensiveness, intensity, duration, and cost. DM evaluations often used small sample sizes that limited analyses of cost data.

In spite of these important limitations, we believe this review of the ROI literature would be helpful to policymakers considering the value of DM for the Medicare Program. This knowledge may help policymakers make better decisions about whether such programs, at face value, hold promise for employers, health plans, and Medicare, from a purely financial perspective.

As noted, this analysis of DM programs is an economic one. We acknowledge that the primary aim of these programs should be to improve health and functioning of patients—rather than to save money. Nonetheless, program funders often require a business case argument for new programs and benefits. For most Medicare and Medicaid Programs, innovations are expected to be at least cost-neutral, returning as many dollar benefits as they cost. Thus, when introducing new health management initiatives, it is often necessary to develop a cogent and defensible financial impact analysis, with an associated ROI projection.

Defining Disease Management

The Disease Management Association of America (DMAA) 2004 defines DM as a “multi-disciplinary, coordinated, continuum-based approach to healthcare delivery and communications for populations with, or at risk for, established medical conditions.” DMAA notes that effective DM programs should contain the following eight elements: (1) an identified population with specific health and disease conditions; (2) the application of evidence-based practice guidelines to treat those patients; (3) a process that encourages collaboration among physicians and other providers; (4) risk stratification, matching interventions with need; (5) patient self-management education (that may include primary prevention, behavior modification programs, and compliance/surveillance); (6) process and outcomes measurement, evaluation, and management; (7) routine reporting and feedback loops that include communication with the patient, physician, health plan, and ancillary providers; and (8) appropriate use of information technology (including use of specialized software, data registries, automated decision support tools, and callback systems) (Disease Management Association of America, 2003).

Methods

Data Sources

Relevant articles were compiled from three sources: (1) the National Library of Medicine's MEDLINE and HealthSTAR electronic databases; (2) reference lists from published reviews of high-quality peer-reviewed studies; and (3) unpublished but demonstrably high-quality studies identified by the authors and other content experts.

Studies were classified into three research design categories: (1) randomized clinical trials (RCTs); (2) controlled before and after (CBA) studies employing a quasi-experimental design in which data for the intervention group are compared to data from a matched control group, or where appropriate statistical methods are used to control for potential confounding variables when comparing treatment and comparison group subjects; and (3) descriptive before and after (pre-post) studies employing non-experimental designs that lack control subjects.

Procedures

Studies were categorized into the main research design groups. When reviewing and analyzing results, more weight was given to RCTs and CBA designed studies since these, by definition, are more rigorous and therefore, subject to fewer internal validity problems. Since the analysis was primarily focused on financial results, particular attention was given to studies where dollar savings were calculated, usually by comparing differences in gross costs per patient for treatment versus control subjects.

In the analysis, we distinguished between studies reporting cost savings and those that calculated ROI. Many studies reporting cost savings leave out an accounting of what was spent to run the program which, in turn, achieved cost savings. Thus, cost savings reported in this article are gross savings. However, when calculating ROI, we report the ratio of gross savings to program expenses. Our analysis used terminology familiar to finance professionals when they decide on the relative merits of various investments, typically reported in terms of net present value (NPV) or benefit-to-cost ratio ROI.

In the studies examined, cost and benefit information was most often derived from administrative claims data rather than extrapolation of self-reported or health care utilization records. We examined the differences in expenditures between intervention and control subjects at the conclusion of the study, subtracting out baseline cost differences. To calculate cost-benefit ratios, we sought studies that reported program expenses and gross savings. In some cases, costs and gross savings were recalculated from charts and tables found in the published studies. This was done to isolate direct from indirect expenditures or combine data across several patient groups. Thus, in certain situations, the calculated costs and benefits reported here may differ from those reported by study authors.

To facilitate the analysis, the number of subjects included in the study, duration, cost savings, and program expenditures were recorded for each study reviewed.

Results

Asthma Disease Management Programs

Twelve asthma studies were examined in this review (Table 1). Seven were RCTs, two were CBA studies, and three were pre-post evaluations. Two of the RCTs reported ROI data; these were studies by Kelly et al. (2000) and Greineder et al. (1999), which used relatively small samples in their intervention groups (38 and 29, respectively). Intervention program expenses reported by these authors averaged $293 per participant ($395 and $190, respectively) while savings averaged $1,068 per participant ($543 and $1,592, respectively). Thus, the ROIs for the two controlled studies were $1.38 in savings per dollar spent on the program, and $8.37 to per dollar spent, respectively. However, the Kelly et al. (2000) program expenses did not include projected drug costs which, if included, would have increased per participant costs significantly and yielded an ROI of $0.72.

Table 1. Disease Management ROI Analysis for Asthma Studies.

								ROI
Study Design	Sample Size		Evaluation Period (Years)	Intervention Program Cost		Intervention Program Savings		Total Benefits/Costs
	Intervention	Control		Total	Per Participant	Total	Per Participant
Experimental Design
RCT (A)	38	4	1.0	$15,000	$394.74	$20,634	$543.00	1.38
RCT (A)	29	28	1.0	5,520	190.34	46,182	1,592.48	8.37
Average	34	34	1.0	10,260	292.54	33,408	1,067.74	3.65
Experimental Design
RCT (B)	55	59	1.0	—	—	—	(FINM 649)	NA
RCT (B)	64	70	5.0	—	—	—	£46	NA
RCT (B)	77	80	1.0	—	—	—	(FINM 674)	NA
RCT (B)	32	33	< 1	22,822	713.20	1,526	47.70	0.07
RCT (B)	515	518	2.0	173,555	337.00	(125,995)	-244.65	(0.73)
Average	149	152	2.3	98,189	525.10	(62,234	-98.475	(0.19)
Quasi-Experimental Design
CBA	2,415	16,627	1.3	NA	NA	54,540	22.58	NA
CBA	526	494	1.0	—	—	574,392	1,092.00	NA
Average	1,471	8,561	1.1	—	—	314,466	557.29	NA
Pre-Post Design
Pre-Post	317	NA	0.5	96,051	303.00	359,425	1,133.83	3.7
Pre-Post	53	NA	1.0	11,115	209.72	87,315	1,647.45	7.8
Pre-Post	61	NA	1.0	—	—	295,563	—	NA
Average	144	—	0.8	53,583	256.36	223,370	1,390.64	5.42

NOTES: ROI is return on investment. RCT is randomized clinical trials. CBA is controlled, before and after study design. NA is not applicable.

SOURCE: Cornell University Institute for Policy Studies, 2004.

Reviewing results from the five other randomized trials, per participant costs averaged $525 for the two studies reporting program expenses. These two studies produced an average loss of $98, with one study showing savings of $48 while another showing a loss of $245. Three other studies reported their economic impacts in Finnish Marks currency. Their results showed no significant differences in direct medical costs between intervention and control groups. For the two studies in this grouping reporting costs and benefits, the ROI for one was $0.07, while the other showed a gross loss of $0.70 per dollar spent on the program.

The two CBA studies reported a very different net savings ($23 and $1,092), whereas the three pre-post studies reported average savings of $1,391 per participant. ROI values for the two pre-post studies with both cost and benefit data were calculated as $3.74, and $7.86.

Table 2 summarizes results across all 12 studies, regardless of the level of rigor employed. The table shows that an average of 449 subjects participated in asthma DM programs over a 1.3-year period. Per-participant costs averaged $269 and savings $729. An overall ROI of $2.72 was calculated for studies providing both cost and benefit data. Of the seven RCTs examined, six produced savings in medical costs, but only two had savings that were high enough to result in a positive ROI, and those two studies had very few cases.

Table 2. Summary of Disease Management ROI Analysis for Asthma.

	Number	Average Sample Size for Intervention	Average Evaluation Period (Years)	Average Per participant Cost and Savings		Average ROI
						Total Benefits/Costs
Study Design				Cost	Savings
RCT (A)	2	34	1.0	$292.54	$1,067.74	3.65
RCT (B)	5	149	2.3	525.10	(98.48)	(0.19)
CBA	2	1471	1.1	—	557.29	NA
Pre-Post	3	144	0.8	256.36	1,390.64	5.42
Total	12	449	1.3	268.50	729.30	2.72

NOTES: ROI is return on investment. RCT is randomized clinical trials. CBA is controlled, before and after study design. NA is not applicable.

SOURCE: Cornell University Institute for Policy Studies, 2004.

CHF Disease Management Programs

Twelve studies of CHF were examined: five RCTs—four reported savings in U.S. dollars and a fifth reported findings in Australian dollars; four CBA studies; and three pre-post evaluations (Table 3).

Table 3. Disease Management ROI Analysis for Congestive Heart Failure Studies.

								ROI
Study Design	Sample Size		Evaluation Period (Years)	Intervention Program Cost		Intervention Program Savings		Total Benefits/Costs
	Intervention	Control		Total	Per Participant	Total	Per Participant
Experimental Design
RCT (A)	140	142	0.3	$30,240.00	$216.00	$64,400.00	$460.00	2.13
RCT (A)	80	11	1.0	16,640.00	208.00	104,000.00	1,300.00	6.25
RCT (A)	44	44	1.0	23,320.00	530.00	330,660.00	7,515.00	14.18
RCT (A)	102	98	0.5	92,208.00	904.00	(252,348.00)	(2,474.00)	-2.74
Average	92	99	0.7	40,602.00	464.50	61,678.00	1,700.25	3.66
RCT (B)	49	48	1.5	19,310.00	1190.00	1269,500.00	15,500.00	28.90
Quasi-Experimental Design
CBA	283	173	0.1	—	—	77,825.00	275.00	—
CBA	120	120	0.5	39,600.00	330.00	24,600.00	205.00	0.62
CBA	457	803	1.0	779,642.00	1,706.00	841,500.00	1,841.36	1.08
CBA	396	19	1.0	—	—	959,864.40	2,423.90	NA
Average	314	323	0.6	409,621.00	1,018.00	608,654.80	1,490.09	1.46
Pre-Post Comparisons
Pre-Post	347	407	1.0	104,000.00	299.71	387,946.00	1,118.00	3.73
Pre-Post	117	NA	0.4	175,000.00	1,495.73	1,002,807.00	8,571.00	5.73
Pre-Post	214	NA	0.5	1,358,900.00	6,350.00	3,359.000.00	15,696.26	2.47
Average	226	407	0.6	545,966.67	2,715.15	1,583,251.00	8,461.75	3.12

¹Australian dollar estimate.

NOTES: ROI is return on investment. RCT is randomized clinical trials. CBA is controlled, before and after study design. NA is not applicable.

SOURCE: Cornell University Institute for Policy Studies, 2004.

The four RCTs conducted by Rich et al. (2003), Cline et al. (1998), Krumholz et al. (2002), and Kasper et al. (2002) reported intervention program costs ranging from $208 to $904. Kasper and colleagues reported program losses of $2,474 while the other researchers reported savings ranging from $460 to $7,515. Consequently, the ROIs ranged from a loss of $2.74 per dollar spent on the program, to a savings of $14.18 per dollar spent; the average ROI was $3.66.

A fifth clinical trial conducted by Stewart et al. (1999) involved a very small sample (49 intervention subjects). The intervention cost was $190 Australian and consisted of a single home visit. Program savings were calculated as $5,500 Australian. Thus, the ROI generated (a savings of $28.90 per dollar spent on the program) appears unrealistic, given the nature of the intervention and the small sample size.

Of the four CBA studies, Riegel et al. (2000) and vanVonno et al. (2003), reported program expenses ($330 and $1,706, respectively). Savings reported across all four studies averaged $1,490. When considering the two studies with cost and benefit data, one reported an ROI of $0.62 (a savings of $0.62 per dollar spent on the program), while the second ROI was barely break even at $1.08. For the three before and after studies, per-participant costs averaged $2,715 (driven largely by the very expensive Fonarow et al. [1997] study) while savings averaged $8,462 per participant. The average ROI for theses studies was a savings of $3.12 per dollar spent on these programs.

Table 4 summarizes the results across all twelve studies focused on CHF. As shown, an average of 170 subjects participated in CHF DM program studies over a slightly less than 1-year period. Per-participant costs averaged $1,399 and savings averaged $3,884. The average ROI across studies was $2.78. Of the five RCTs examined, all but one produced a positive ROI.

Table 4. Summary of Disease Management ROI Analysis for Asthma.

	Number	Average Sample Size for Intervention	Average Evaluation Period (Years)	Average Per Participant Cost and Savings		Average ROI
						Total Benefits/Costs
Study Design				Cost	Savings
RCT (A)	4	92	0.7	$464.50	$1,700.25	3.66
RCT (B)	1	49	1.5	1190.00	15,500.00	128.90
CBA	4	314	0.6	1,018.00	1,490.09	1.46
Pre-Post	3	226	0.6	2,715.15	8,461.75	3.12
Total	12	170	0.9	1,399.22	3,884.03	2.78

¹Australian dollar estimate.

NOTES: ROI is return on investment. RCT is randomized clinical trials. CBA is controlled, before and after study design. NA is not applicable.

SOURCE: Cornell University Institute for Policy Studies, 2004.

Diabetes Disease Management Programs

Eight studies reported on diabetes DM programs: four RCTs, one CBA, two controlled (quasi-experimental) before-after studies study, and two pre-post evaluations (Table 5).

Table 5. Disease Management ROI Analysis for Congestive Heart Failure Studies.

								ROI
Study Design	Sample Size		Evaluation Period (Years)	Intervention Program Cost		Intervention Program Savings		Total Benefits/Costs
	Intervention	Control		Total	Per Participant	Total	Per Participant
Experimental Design
RCT	89	82	2.0	$23,585.00	$265.00	$24,475.00	$275.00	1.04
RCT	1,079	1082	3.0	2,999.620.00	2,780.00	(2,448,251.00)	(2,269.00)	-0.82
RCT	1,073	1,082	3.0	2,727,566.00	2,542.00	(2,350,943.00)	(2,191.00)	-0.86
RCT	192	377	0.3	—	—	25,344.00	132.00	—
Average	608	656	2.1	1,916,923.67	1,862.33	(1,187,343.75)	(1,013.25)	-0.54
Quasi-Experimental Design
CBA	3,118	3,681	2.0	1,810,000.00	580.50	4,035,689.76	1,294.32	2.23
CBA	732	4,012	5.0	—	—	598,410.00	817.50	NA
Pre-Post Studies
Pre-Post	169	NA	1.0	—	—	126,243.00	747.00	NA
Pre-Post	7,000	NA	0.9	—	—	3,696,000.00	528.00	NA
Average	3,585	—	0.9	—	—	1,911,121.50	637.50	—

NOTES: ROI is return on investment. RCT is randomized clinical trials. CBA is controlled, before and after study design. NA is not applicable.

SOURCE: Cornell University Institute for Policy Studies, 2004.

Two RCT studies, those conducted by the Diabetes Prevention Program Research Group (2003), were not technically DM program evaluations. Rather, they tested the health and economic impacts of alternative methods for preventing diabetes exacerbation for pre-diabetic patients. These studies reported the relative cost-effectiveness of alternative methods for achieving a common outcome—improved glycemic control and reduction in the prevalence of diabetes—comparing pharmacological and lifestyle modification interventions to placebo. Neither intervention was cost effective, losing $0.82 to $0.86 for every dollar invested. Thus, there were no cost savings from these interventions, and negative ROIs.

As shown, the two Diabetes Prevention Program trials were relatively costly, averaging $2,661 per participant, as compared to more typical DM program costs, such as that one reported by Laffel et al. (1998) that averaged $265 per participant.

Program savings were negative in the Diabetes Prevention Program Research Group trials (averaging a loss of $2,230). However, positive results were found for the other two clinical trials (averaging a savings of $204 per participant). Thus, while the ROIs from the Diabetes Prevention Program trials were negative, the ROI from the Laffel et al. (1998) trial was estimated to be slightly better than break even ($1.04 in savings per dollar spent on the program).

The Sidorov et al. (2002) CBA study reported average program costs as $580 and savings as $1,294, thus producing a $2.23 ROI. For the three remaining studies, the range of savings was from $528 to $818 per participant. However, since no cost data were provided, ROIs could not be calculated.

Table 6 summarizes the results across all diabetes DM studies, including the Diabetes Prevention Program studies. An average of 2,011 subjects participated in these programs over a 2.5-year period.

Table 6. Summary of Disease Management ROI Analysis for Diabetes.

	Number	Average Sample Size for Intervention	Average Evaluation Period (Years)	Average Per Participant Cost and Savings		Average ROI
						Total Benefits/Costs
Study Design				Cost	Savings
RCT	4	608	2.1	$1,862.33	$(1,013.25)	(0.54)
CBA	1	3,118	2.0	580.50	1,294.32	2.23
CBA	1	732	5.0	—	817.50	NA
Pre-Post	2	3,585	0.9	—	637.50	NA
Total	8	2,011	2.5	610.71	434.02	0.71

NOTES: ROI is return on investment. RCT is randomized clinical trials. CBA is controlled, before and after study design. NA is not applicable.

SOURCE: Cornell University Institute for Policy Studies, 2004.

Per-participant costs averaged $611, while savings were $434. For studies reporting costs and benefits, a $0.70 ROI was calculated (lower than a break even). On balance, these studies point to the potential for diabetes DM programs to break even, if treatment costs are well managed. While the CBA study by Sidorov et al. (2002) reported a positive ROI, these results are more suspect, because less rigorous methods were used to evaluate the program's financial impact.

In an earlier literature review, Klonoff and Schwartz (2000) examined the ROI for diabetes DM programs. (A summary table of their review is available from the author on request.) The researchers reported average program expenses of $271 and average gross savings of $600, producing an average ROI of $2.21 in savings per dollar spent on the program. However, since most of these studies were performed in the 1970s and 1980s using non-experimental methods, their positive results should be interpreted with caution.

Depression Disease Management Programs

All eight of the studies we examined in our literature review of depression DM programs were RCTs. Results from these trials, as well as an independent review of depression program savings as compiled by Simon, et al. (2001a), are reported in Tables 7 and 8.

Table 7. Disease Management ROI Analysis for Depression Studies.

								ROI
Study Design	Sample Size		Evaluation Period (Years)	Intervention Program Cost		Intervention Program Savings		Total Benefits/Costs
	Intervention	Control		Total	Per Participant	Total	Per Participant
Experimental Design
RCT	169	0	1.0	$201,279	$1,191.00	$(80,824.25)	$(478.25)	(0.40)
RCT	95	92	2.3	—	—	57,665.00	607.00	—
RCT	188	180	0.5	9,588	51.00	(15,792.00)	(84.00)	(1.65)
RCT	110	109	0.5	38,500	350.00	(32,560.00)	(296.00)	(0.85)
RCT	205	169	1.0	1,137,545	5,549.00	(336,200.00)	(1,640.00)	(0.30)
RCT	194	192	1.0	49,664	256.00	(13,968.00)	(72.00)	0.28
RCT	440	498	0.5	—	—	(737,000.00)	(1,675.00)	—
RCT	913	443	2.0	—	—	414,502.00	(454.00)	—
Average	289	210	1.1	239,429	1,479.40	(196,647.66)	(511.53)	(0.35)

NOTES: ROI is return on investment. RCT is randomized clinical trials. CBA is controlled, before and after study design. NA is not applicable.

SOURCE: Cornell University Institute for Policy Studies, 2004.

Table 8. Disease Management ROI Analysis for Depression, by Incremental Outpatient Costs in RCT of Depression Treatment Programs.

Intervention	Duration	Incremental Dollars Spent (Program Net Cost)
Stepped Collaborative Care	0.5	$242.00
Telephone Care Management	0.5	130.00
Psychiatric Collaborative Care	0.5	383.00
Psychologist Collaborative Care	0.5	471.00
Depression Management for High-Use Patients	1.0	675.00
Guidance-Based Psychotherapy	1.0	738.00
Interpersonal	1.0	843.00
Average	0.7	497.43

NOTES: ROI is return on investment. RCT is randomized clinical trials.

SOURCE: (Simon, G.E. et al., 2001a.)

Examining aggregate results from the eight RCTs reported in Table 7, we show an average sample size of 289 intervention subjects, and average study duration of 1.1 years. Per-participant program expenses averaged $1,479 and ranged from $51 to $5,549, signaling much variation in what was termed a DM program. Intervention program savings were all negative, averaging $512 in our analysis and $497 in the Simon and colleagues' review (2001a) (Table 8). The aggregate ROI for depression DM programs was therefore negative, averaging a loss of $0.35 per dollar spent on the program.

Multiple Condition Disease Management Programs

Four multiple condition program evaluations were examined (Table 9). Two were RCTs (Coleman et al., 1999; Wasson et al., 1992), one was quasi-experimental (Munroe et al., 1997), and one was a pre-post study (Lorig et al., 2001). The Coleman et al. (1999) intervention targeted common geriatric medical problems, including urinary incontinence, falls, depression, high-risk medication management, and functional impairment in older adults. Wasson et al. (1992) studied the effects of more frequent clinician-initiated telephone calls directed at chronic disease patients as a substitution for clinic visits. The Munroe et al. (1997) program, run by pharmacists, targeted patients with hypertension, diabetes, asthma, and/or hypercholesterolemia. Finally the Lorig et al. (2001) intervention targeted patients with heart disease, lung disease, stroke, or arthritis.

Table 9. Disease Management ROI Analysis for Studies of Multiple Conditions.

								ROI
Study Design	Sample Size		Average Evaluation Period (Years)	Intervention Program Cost		Intervention Program Savings		Total Benefits/Costs
	Intervention	Control		Total	Per Participant	Total	Per Participant
Experimental Design
RCT	96	73	2.0	—	NA	$55,776	$581.00	NA
RCT	249	248	2.0	$30,876	$124.00	205,425	825.00	6.65
Average	173	161	2.0	30,876	124.00	130,601	703.00	6.65
Quasi-Experimental
CBA	188	401	1.3	60,912	324.00	661,888	3,520.68	10.87
Pre-Post Studies
Pre-Post	683	N/A	1.0	92,205	135.00	402,970	590.00	4.37
Average	289	210	1.1	239,429	1,479.40	(196,647.66)	(511.53)	(0.35)

NOTES: ROI is return on investment. RCT is randomized clinical trials. CBA is controlled, before and after study design. NA is not applicable.

SOURCE: Cornell University Institute for Policy Studies, 2004.

Combined, these studies ran an average of 1.4 years and observed an average of 322 intervention subjects (Table 10). Intervention program expenses (from the three studies reporting costs) were $124, $135, and $224, and their savings were $825, $590, and $3,521, respectively. The ROIs for these studies were $6.65, $4.37, and $10.87. It should be noted, however, that the RCT conducted by Coleman et al. (1999) did not show statistically significant differences in costs between study and control groups. This may be attributed to small sample size, lack of power, low penetration rates, and the limited nature of the intervention, which involved half-day seminars for patients every 3 to 4 months.

Table 10. Summary of Disease Management ROI Values for Studies of Multiple Risk.

		Average Sample Size for Intervention	Average Evaluation Period (Years)	Average Per Participant Cost and Savings		Average ROI
						Total Benefits/Costs
Study Design	Number			Cost	Savings
RCT	2	96	2.0	NA	$581.00	6.65
CBA	1	683	1.0	$135.00	590.00	4.37
Pre-Post	1	188	1.3	324.00	3,520.68	10.87
Total	4	322	1.4	229.50	1,563.89	6.81

NOTES: ROI is return on investment. RCT is randomized clinical trials. CBA is controlled, before and after study design. NA is not applicable.

SOURCE: Cornell University Institute for Policy Studies, 2004.

Discussion

The literature reporting financial impact and cost-benefit for four types of DM programs, and for programs directed at multiple conditions, was reviewed. Forty-four studies were found that dealt with the economic impacts of DM programs and their potential to produce a positive ROI. Our interest was in reporting whether assumptions about the positive economic impact of DM programs correspond to actual results from well-designed studies that used rigorous methods. There was also a desire to inform public policy experts about private sector innovations in DM, and to learn whether these innovations might hold promise for Medicare and Medicaid patients.

The issue of whether DM programs are effective from a health improvement perspective was avoided in this review. We assumed that following evidence-based clinical guidelines would improve the health and functioning of patients, though it is also acknowledged that all health care interventions may produce unintended consequences. Thorough clinical reviews of these programs and the methods employed were not performed because these have been reported elsewhere (Institute of Medicine, 2001). Our primary interest was whether DM held the potential for saving money and producing a positive ROI.

From a purely financial perspective, DM programs directed at patients suffering from CHF may save more money than they cost. These programs produced a positive ROI, even in the short run, (i.e., within 1 to 2 years). In addition, programs which target multiple health and disease conditions, and which emphasize self-care and informed decisionmaking, also hold promise to be cost beneficial.

Mixed results were obtained when considering programs directed at asthma, diabetes, and depression. For example, large-scale prevention programs directed at prediabetic patients (technically not DM programs) may cost more than they save, at least in the short term. On the other hand, diabetes DM programs directed at patients with active disease may produce savings and a positive ROI, although too few studies have been performed for these results to be conclusive.

The evidence for asthma programs showed that these programs can achieve a positive ROI, but findings were not consistent, especially when examining rigorous evaluations. In the case of depression management programs, none of the studies examined found a medical cost-offset for appropriate treatment of depression patients using pharmacological agents and/or psychotherapy. Quite uniformly across the various studies examined, good treatment of depression cost more money (about $500 more a year). The story may be different when considering productivity and functionality outcomes (e.g., absence, disability, on the-job-productivity, and performing activities of daily living). Goetzel et al. (2002) noted that treating depression in accordance to evidence-based medicine may produce productivity-related savings that offset treatment costs.

Success Factors in Disease Management

Although it was not our intent to identify ingredients of successful DM programs, our review uncovered several themes common to successful programs. Many of these apply to health and DM programs and confirm previous research into this area.

For example, Heaney and Goetzel (1997) examined the impact of multicomponent health management programs and concluded effective programs offered individualized and personalized risk-reduction counseling to those at highest risk. MacKinnon et al. (1996) suggested the following success factors: developing appropriate clinical guidelines based on the best scientific evidence; educating and involving physicians and other providers on effective implementation of these guidelines; conducting repeated evaluations; sharing results with providers and patients; and updating guidelines as needed.

Gurnee and Da Silva (1997) added to the list the need to leverage medical information computer systems to identify patients for intervention and measure clinical and financial outcomes. They also advocated the use of incentives for patients and providers to participate in DM, and copromotion with local health care providers, to gain grass roots support.

Bodenheimer, Wagner, and Grumbach (2002) observed that “self-management” education, which teaches patients problem-solving skills, is critical to better outcomes. Stone et al. (2002) found that organizational change interventions, including the use of separate clinics devoted to prevention; the use of planned care visits for prevention, patient reminders; and the use of non-physician staff to carry out specific prevention activities, were among the interventions with the greatest impact. Other factors observed in successful programs include: effective screening and triage into risk-specific interventions; use of tailored materials founded on behavior change theory; and goal setting by patients.

Limitations

Our primary intent was to comment on whether certain kinds of DM programs generate a positive ROI. Thus, a first limitation is that we focused only on disease categories where economic studies were performed. Evidence from programs directed at diseases not discussed in this review should be accumulated and analyzed as well.

Second, the number of DM programs considered for each disease category was small, and some of these programs had small sample sizes. The small number of studies reviewed in each category reduced the utility of reporting variances for ROI projections, thus mean values reported should be interpreted with caution. The small sample sizes within each category (and sometimes within individual studies) may not support the notion that ROIs are significantly different from 1.0 in a statistical sense.

Third, many authors have used the term “population-based DM,” but most of the programs reviewed were not truly population based. The term “population” is used loosely, often meaning a group of patients who meet certain inclusion criteria for a study, instead of an all-inclusive group of patients with certain diseases. For example, diabetes programs may exclude those with end stage renal disease, or depression programs may exclude patients recently hospitalized for suicide attempts. Thus, programs summarized in this review should probably be viewed as sample based, not population based, as should most DM programs.

Fourth, the file drawer problem may be formidable. This term is used by meta-analysts (Rosenthal, 1991) to comment on the number of unpublished studies that would show radically different findings. It is unknown how many such studies there may be, and this review may overemphasize programs with better results since these may be more likely to be published. Conversely, it may also be true that positive program results have not yet been published. Some large DM programs are delivered by freestanding vendors or managed care organizations operating on platforms quite distinct from a traditional delivery system and academic research centers. These organizations are less likely to structure formal experiments or publish research findings because rigorous studies are difficult to perform and costly. Consequently, we may not be aware of positive results from large-scale interventions if those results have not been prepared for scientific journals.

Fifth, studies that rely on a pre-post design, commonly employed in DM program evaluations, may suffer from a common internal threat to validity, regression to the mean. Simply stated, many patients identified as very sick and costly are likely to improve over time, regardless of how they are managed clinically (i.e., they will regress toward average values on many measures). Thus, studies that only examine expenditures at one time and then again at a certain followup point may suffer from this regression to the mean phenomenon. These programs may appear to be performing better than they actually would have, if a control group of similar patients had been followed over the same period.

Sixth, many studies presented lack sufficient rigor in evaluating the financial impact of their programs. Good econometric methods are seldom used. Cost savings and ROI estimates are most often derived from secondary analyses of data and not subject to statistical testing. Sample sizes are frequently small and differences found in expenditures may be due to chance.

Seventh, study time periods differ radically, ranging from 3 months to 5 years. Although not directly examined, it is likely that longer term studies will achieve better financial outcomes, since there is a lag period between health improvement and cost savings. This may be true when comparing outcomes for CHF programs, where positive effects are likely to be realized in a short period (1-year), versus diabetes programs that may take much longer to achieve cost savings.

Finally, it is worth putting the notion of ROI in perspective. It is probably fair to say that many economists and investment analysts would be surprised to hear terms like “the ROI was only 1.08” when describing the financial impact of a DM program. While it may be true that an ROI estimate of 1.08 may have a wide confidence interval around it (especially in poorly designed studies that employ small sample sizes), a return of 8 percent, if accurate, is larger than many other investments currently available. It is important to note that the issue is not so much the absolute magnitude of the ROI, but rather the relative ROI and net present value of comparable investments.

Implications

Almost all members of the American Association of Health Plans report having one or more DM programs. However, we could find only 44 studies reporting enough detail to support the preliminary cost-benefit analyses we conducted. One may therefore argue that there are still too few studies describing the potential ROI from DM programs. More information should be published about existing programs, and ideally the financial results should be subject to the same level of statistical rigor applied to studies focused on health outcomes.

Testing DM programs in Medicare and non-Medicare populations also makes good sense. As shown, most of the relevant research has been conducted in the private sector, where a profit motive has been an important driver in decisions of which programs to implement and at what cost.

In Medicare, program managers are less concerned with profit than with solvency. In the long run, decisions concerning government-financed health care must be driven by health and economic outcomes. Medicare administrators should not passively wait for patients to get sick and then pay for acute care services, if evidence suggests that coordinated care and DM approaches are beneficial. Medicare is currently testing these approaches rigorously, before deciding whether DM programs should be the norm rather than the exception.

As shown in this review, there are many variations of DM, and not all programs may be equally practical and economically viable. There is also substantial variability in the cost of these programs, suggesting that some are far more intense, and that perhaps some are being delivered more efficiently. In particular, DM programs that leverage administrative databases and mass communication technologies such as tailored mail, telephone, and the Internet may be inherently less costly and result in more favorable ROIs than programs operated as direct extensions of outpatient clinics. More research is needed therefore, to test the assumptions surrounding DM programs, in order to determine which elements lead to the best health and financial outcomes.