Abstract
Objective
This review evaluates costs and benefits associated with acquiring, implementing, and operating clinical decision support systems (CDSS) to prevent cardiovascular disease (CVD).
Materials and Methods
Methods developed for The Community Guide were used to review CDSS literature covering the period from January 1976 to October 2015. Twenty-one studies were identified for inclusion.
Results
It was difficult to draw a meaningful estimate for the cost of acquiring and operating CDSS to prevent CVD from the available studies (n=12) due to considerable heterogeneity. Several studies (n=11) indicated healthcare costs were averted by using CDSS but many were partial assessments that did not consider all components of healthcare. Four cost-benefit studies reached conflicting conclusions about the net benefit of CDSS based on incomplete assessments of costs and benefits. Three cost-utility studies indicated inconsistent conclusions regarding cost-effectiveness based on a conservative $50,000 threshold.
Discussion
Intervention costs were not negligible but specific estimates were not derived because of the heterogeneity of implementation and reporting metrics. Expected economic benefits from averted healthcare cost could not be determined with confidence because many studies did not fully account for all components of healthcare.
Conclusion
We were unable to conclude whether CDSS for CVD prevention is either cost-beneficial or cost effective. Several evidence gaps are identified, most prominently the lack of information about major drivers of cost and benefit; the lack of standard metrics for the cost of CDSS; and not allowing for useful life of CDSS that generally extends beyond one accounting period.
Keywords: Decision support systems, clinical, Economics, Economics, medical
INTRODUCTION
The 2016 statistical update from the American Heart Association estimated that the annual cost of cardiovascular disease (CVD) and stroke in 2011–2012 was $193.1 billion in medical care and about $123 billion in lost productivity from premature death.(1) A substantial part of this burden is preventable by preventing and controlling major risk factors for CVD, including hypertension, hypercholesterolemia, and diabetes.(1) Clinical decision support systems (CDSS) are computer-based tools used to assist health providers to prevent and control these risk factors. The Community Preventive Services Task Force (Task Force) recently recommended CDSS for CVD prevention,(2) based on evidence from a systematic review conducted for The Community Guide.(3) The evidence showed CDSS improved screening for CVD risk factors and improved practices for CVD-related preventive care services, clinical tests, and treatments. The objective of the present study is to assess the economic value of CDSS interventions for CVD prevention, based on a systematic review of the literature.
METHODS AND MATERIALS
A full description of general methods for Community Guide systematic economic reviews is available online.(4). Briefly, Community Guide systematic economic reviews have the multiple objectives of providing evidence-based estimates of what it costs to implement an intervention, calculating the expected benefits from averted healthcare cost and worksite productivity losses due to reduced morbidity and mortality, and making a judgement of economic value based on estimated cost-benefit or cost-effectiveness. Estimates of economic outcomes generally vary considerably across evaluation studies. Therefore, Community Guide economic review methods attempt to account for the major elements that drive intervention costs and benefits, identified a priori based on information gained from published literature and subject matter experts.
Following the general methods, a systematic review team was constituted for the evaluation of CDSS for CVD prevention, including subject matter experts on CVD and CDSS from various agencies, organizations, and academic institutions together with expert systematic reviewers from The Community Guide branch at CDC. The team worked under the oversight of the Task Force.
CDSS for CVD prevention were defined as computer-based information systems designed to assist healthcare providers, in primary, secondary, and tertiary care settings, implement guidelines and evidence-based practices by providing tailored reminders to conduct or schedule preventive and screening services; assessments of patients’ risk for developing CVD based on their medical history; and alerts when CVD-related physiologic indicators are not at goal. The CDSS interface with the provider occurs at the point of care, based on individual patient data, and assists providers with recommendations for screening, preventive care, and treatment of their patients who have risk factors for CVD such as obesity, inactivity, smoking, hypertension, hyperlipidemia, or diabetes. Improved clinician actions should mitigate these risk factors and ultimately lead to reduced morbidity and mortality from CVD and therefore also improve economic outcomes. The complete definition and analytic framework are available in the published review of effectiveness.(3)
Search strategy and inclusion criteria
The studies for this systematic economic review were drawn from the results of three searches:
-
The primary source was the set of studies referenced in a broad systematic review published in 2012, Bright et al.,(5) and the in the accompanying report to the Agency for Healthcare Research and Quality (AHRQ).(6)
Period: January 1975–January 2011
Databases: PubMed, CINAHL, PsycINFO, PubMed NLM, and Web of Science
Scope: CDSS in all health topics
-
The Bright et al. search was updated to cover more recent studies
Period: January 2011–October 2015
Databases: PubMed, CINAHL, PsycINFO, PubMed NLM, and Web of Science
Scope: CDSS in CVD prevention
-
Search was conducted within economics-related databases using the strategy implemented in Bright et al.
Period: January 1970–October 2015
Databases: JSTOR, EconLit, Centre for Reviews and Dissemination
Scope: CDSS in CVD prevention
The three searches used terms to identify evaluation studies of CDSS to prevent CVD across healthcare settings and ages. The studies referenced in the Bright et al. review were further screened to identify those related to CVD prevention because that review considered CDSS use under all health topics. Details including the complete list of search terms for the domains of CDSS intervention, target population, CVD prevention, and the databases searched for the Bright et al. search and its update are available online,(7) as are similar details for the economics-focused search.(8)
Studies were included in this review if they:
met the intervention definition;
were in English;
were implemented in a high-income economy;(9) and
reported one or more of: the cost of intervention, change in healthcare cost, change in productivity, other economic benefits, cost-benefit, or cost-effectiveness.
CDSS interventions are generally implemented in clinics, hospitals, and other healthcare settings and the cost of implementation is borne by the same organizations. However, benefits can accrue to patients, their employers, or insurers. This review took both a health system and a societal perspective when assessing cost and benefit, so that reported estimates are meaningful for both the public and commercial perspectives of these implementers and funders.
Intervention cost
The cost to develop the CDSS is the cost of compiling evidence-based narrative guidelines and programming the guidelines and decisions into code to produce prompts for provider action. Resources are needed to then implement the system throughout the practice and for all providers. The day-to-day use and maintenance of the CDSS require staff time and other resources, and are categorized under operating cost. In summary, the components of capital cost are development and implementation and the components of operating cost are maintenance and operation. Annualized intervention cost was estimated by distributing the one-time cost of development and implementation equally over the assumed 5-year life of the system and adding this annual amortized cost, discounted at 3%, to the annual cost of operation. Details regarding the conceptualization and measurement of intervention cost is presented in the Appendix 1.
A CDSS may be embedded within an electronic health record (EHR) system containing patient demographics and all health-related data or it may be built on a registry that contains information only for patients with a specific disease or condition. The cost of collecting and populating the database with patient information, whether for an EHR or a registry, is not considered part of CDSS intervention cost.
A CDSS may also change the economic efficiency of the care process by altering the resources needed to care for a patient or by altering the number of patients cared for with the same level of resource use. The benefits of this efficiency accrue to the healthcare facility implementing the CDSS and would be observed in operation cost per patient or per patient visit.
Intervention economic benefits
Effective CDSS interventions reduce CVD risk factors, such as systolic blood pressure. The reduction in risk factors, in turn, reduces morbidity and mortality and increases the quantity and quality of years lived, measured as quality-adjusted life years (QALYs) saved.
The impact of CDSS on healthcare cost is the difference in cost of healthcare products and services used by the intervention and control groups or the pre to post change where there is no control group. The implementation of CDSS in health care can increase or decrease healthcare utilization by the patient, either owing to adherence to guidelines for care (increased cost) or to improved health (decreased cost). The sum of these two changes is observed in changes in the component cost of outpatient, inpatient, medications, labs, and emergency room (ER) visits. A full accounting of healthcare cost would include all five of these components; this review considers them to be of equal weight and an estimate that includes at least three of the five components to be reasonably complete.
Reduced illness and increased years of life lived contribute to fewer illness-related absences from work, better performance when present, and a longer period of productivity. Hence, effective CDSS implementations that prevent CVD are expected to increase worksite productivity.
Reduced morbidity and mortality also increase both the quantity and quality of life years lived. This is captured in outcomes such as QALYs saved or disability-adjusted life years (DALYs) averted.
Summary economic outcomes
Cost-benefit analysis compares economic benefit to intervention cost, where both benefit and cost are monetized and expressed in dollar terms; an intervention is cost-beneficial when economic benefit exceeds intervention cost.
Net cost (intervention cost plus healthcare cost) per QALY gained produces cost-utility, which is a type of cost-effectiveness assessment. An intervention is cost-effective when net cost per QALY gained is less than a $50,000. A threshold is applied because it is necessary to determine cost-effectiveness(10) and $50,000 is chosen for the threshold because it is a conservative estimate and the one most widely used in the literature.(11)
Measurements, metrics, and reviewer decisions
Intervention cost can differ for practices of different sizes because the scale of CDSS implementation ranges from small clinics to large health centers. The review team decided that intervention cost should be characterized by the size of practices where the CDSS is implemented. However, this review did not find any studies that fully characterized the association between cost of CDSS implementation and size of practice, whether based on number of physicians or of patients in the practice. Therefore, we used the following categories of practice size (based on number of physicians) reported in the 2012 survey of practices by the American Medical Association (AMA)(12) to classify practices: Small, 1–4 physicians (40% of respondents); Medium, 5–24 physicians (35% of respondents); Large, ≥25 physicians (25% of respondents). For studies that reported only the number of patients, we estimated the number of physicians, based on an average patient panel and workload of U.S. primary care physicians.(13). All other cost and economic benefit estimates from included studies were standardized to a per-patient per-year basis when possible.
All monetary values were converted to 2015 U.S. dollars. The Consumer Price Index from the Bureau of Labor Statistics(14) was used to adjust for inflation. Purchasing Power Parity indices from the World Bank were used to convert from foreign currencies to U.S. dollars.(15) Results are summarized using medians and interquartile intervals.
RESULTS
Twenty-one studies from the 7,508 papers screened met inclusion criteria (Figure 1). Seventeen studies(16–28) reported on EHR- or electronic medical record (EMR)-based CDSS, one study(29) piloted a CDSS residing on a stand-alone desktop, one(30) was based on a diabetes registry, and two studies(31–33) covered both EHR-based and registry-based CDSS (two papers(32, 33) reported on the same research and are counted as one study for this review). Table 1 provides an overview of additional characteristics of the included 21 studies. Most of the studies were based in the U.S. (67%) and implemented in clinics (94%), based on trials with a control group (63%) or models based on trials (25%). The studies covered CDSS implemented alone (36%) or in combination with patient reminders/education (18%), or with team-based care (23%), where primary care providers and patients worked together with other providers to improve care and self-management support for patients. Most studies focused on diabetes (47%) as a risk factor, followed by other CVD risks. More than 90% of the studies were published since 2000. Details of the included studies are available online.(34)
Figure 1.
Search Process
Table 1.
Characteristics of Included Studies and Proportion of Studies with Each Characteristic
Characteristic | No. of studies (% of all reviewed studies) |
|
---|---|---|
Country | U.S. | 14 (16, 20, 22–24, 26–28, 30–33, 35–37) (67%) |
Non-U.S. | 7 (17–19, 21, 25, 29, 38) (33%) |
|
Setting | Clinic | 18 (16–25, 27–33, 35, 37) (86%) |
Hospital | 3 (26, 36, 38) (14%) |
|
Study Type | Trial | 15 (16, 17, 19, 20, 23, 24, 26–30, 35–38) (71%) |
Model | 1 (32, 33) (5%) |
|
Trial + Model | 4 (18, 21, 22, 25) (19%) |
|
Survey | 1 (31) (5%) |
|
CDSS Risk Factor Focus | Diabetes | 10 (18, 22, 25, 28, 30–33, 36–38) (47%) |
Hypertension | 1 (24) (5%) |
|
Hyperlipidemia | 2 (17, 19) (10%) |
|
Hypertension and Hyperlipidemia | 1 (21) (5%) |
|
Including multiple CVD risk factors | 7 (16, 20, 23, 26, 27, 29, 35) (33%) |
|
Control group | Pre and post with no control group | 5 (17, 27, 36–38) (24%) |
With control group | 14 (16, 18–26, 28–30, 35) (67%) |
|
Survey, Model with no control group | 2 (31–33) (10%) |
|
Intervention added to CDSS | Team-based care | 5 (18, 24, 28, 36, 38) (23%) |
Provider incentives | 2 (22, 35) (9%) |
|
Provider audit and feedback | 1 (21) (5%) |
|
Patient phone reminders or report to patient | 4 (20, 23, 31–33) (18%) |
|
Quality improvement | 2 (35, 37) (9%) |
|
None | 8 (16, 17, 19, 25–27, 29, 30) (36%) |
|
Publication period | 1990s | 2 (20, 26) (9%) |
2000s | 9 (16, 17, 19, 21, 24, 28, 30–33) (43%) |
|
2010s | 10 (18, 22, 23, 25, 27, 29, 35–38) (48%) |
CDSS, clinical decision support system; CVD, cardiovascular disease
Intervention Cost
Twelve studies reported estimates of intervention cost which are presented in Table 2 for registry- and EHR-based CDSS implementations. The estimates are characterized further by size of the healthcare practice and whether studies included cost of development and implementation, ongoing cost of operations, or both. Nine(18, 21–23, 25, 27, 30–33) of the 12 studies reported the intervention cost of CDSS implementations specifically (Table 2). Of the remaining three studies, two studies that provided the cost of intervention did not include the cost of CDSS, with one(35) reporting the cost of adding provider incentives to the CDSS implementation and the second(36) providing only the labor cost of the staff involved in team-based care. The intervention cost from another study(29) was computed by the review team from the incremental cost per unit reduction in low density lipoprotein-c (LDL-C) reported in the study, and is therefore assumed to include change in healthcare cost of outpatient visits and medications.
Table 2.
Intervention Cost with Components and Characteristics of CDSS
Study | Sizea | Development and Implementation Cost Included | Ongoing Cost Included | Reasonably Complete Estimates | Annual Costb (5-year life) | |
---|---|---|---|---|---|---|
Per Patient | Per Practice | |||||
Registry-based | ||||||
Adler-Milstein 07 (31) | Small | Y | Y | N | $69 | $9,511 |
Adler-Milstein 07 (31) | Medium | Y | Y | Y | $23 | $20,649 |
Adler-Milstein 07 (31) | Large | Y | Y | Y | $14 | $75,964 |
Blanchfield 06 (30) | Medium | Y | Y | N | $106 | $132,438 |
Bu 07a,b (32, 33) | Small | Y | Y | Y | $55 | NR |
EHR-based | ||||||
Adler-Milstein 07 (31) | Mediumc | Y | Y | Y | $56 | $49,808 |
Bardach 13 (35) | Small | N | N | N | $9 | $18,650d |
Bu 07a,b (32, 33) | Small | Y | Y | Y | $170 | NR |
Cleveringa 10 (18) | Small | Y | Y | Y | $73 | $4,794 |
Fretheim 06 (21) | Small | Y | N | N | NR | $346 |
Gilmer 12 (22) | Medium | Y | Y | Y | $43 | $20,595 |
Khan 10 (23) | Small | Subscription fee | Subscription fee | Y | $63 | $7,318 |
Munoz 12 (36) | Large | N | N | N | $61 | $386,750e |
O’Reilly 12 (25) | Small | Y | N | N | $27 | $3,739 |
Shih 11 (27) | Small | Y | N | N | $4 | $7,053 |
Zamora 13 (29) | Small | NR | NR | N | $225f | NR |
CDSS, clinical decision support system; EHR, electronic health record; Y, Yes; N, No; NR, not reported
Size of practice based on number of providers: Small, 1–4 physicians; Medium, 5–24 physicians; Large, ≥25 physicians.
Capital cost amortized over 5 years at 3% discount rate.
Study assumes the implementation is perfectly scalable.
Cost of provider incentives only
Annual budget reported to include cost of team-based care staff
Based on reported incremental cost per mg/dL reduction in low density lipoprotein-c (LDL-C)
All three studies that assessed the cost of CDSS implemented in diabetes registries included both the one-time capital cost and ongoing operations cost. Based on a survey of users and vendors, one(31) of the studies estimated the annual cost per practice for a registry-based CDSS was about $9,500 for a small, $20,600 for a medium, and $76,000 for a large practice. The corresponding estimates for cost per patient per year were $69, $23, and $14, respectively. A U.S. study(32, 33) that modeled the cost of scaling up a registry-based CDSS nationwide estimated the annual per patient costs at $55, which corresponds closely with the survey-based estimate for a small practice. On the other hand, another U.S. study(30) estimated the annual cost of a medium-size CDSS at $106 per patient ($132,400 per practice), based on data collected during a controlled trial.
Eight studies reported the intervention cost of EHR-embedded CDSS, of which five(18, 22, 23, 31–33) included both cost of development and implementation and ongoing cost of operation and three(21, 25, 27) included only the one-time cost of development and implementation. Among the five studies that provided reasonably complete assessments of intervention cost, the mean annual costs were: $102 per patient(18, 23, 32, 33) and $6,056 per practice(18, 23) for small practices, and $49 per patient and $35,201 per practice(22, 31) for medium-sized practices.
Change in Healthcare Cost
Change in healthcare cost attributable to the intervention was reported in 15 studies: 13(16–25, 28, 37, 38) EHR-based implementations, one(30) registry-based, and one(32, 33) reporting both types (Table 3). Interventions in addition to the CDSS were present in several studies; therefore, the effect on healthcare cost and other outcomes cannot be attributed to CDSS alone. This is especially the case where intensive interventions were added, as for four CDSS interventions that included team-based care,(18, 24, 28, 38) followed by those that added less-intensive interventions, such as quality improvement,(37) provider incentives,(22) provider audit and feedback,(21) and patient reminders.(20, 32, 33)
Table 3.
Healthcare Cost: Components and Estimates
Study Type of CDSS | Additional Intervention | Time Horizon | Components | Reasonably Complete Estimate | Change in Healthcare Cost Per Patient Per Yeara | ||||
---|---|---|---|---|---|---|---|---|---|
Outpatient | Inpatient | ER | Drugs | Labs | |||||
Registry-based | |||||||||
Blanchfield 06 (30) | None | 1 Year | ✓ | ✓ | ✓ | Y | +$6 | ||
Bu 07ab (32, 33) | Patient reminders | 10 Years | ✓ | ✓ | N | −$127 | |||
EHR-based | |||||||||
Apkon 05 (16) | None | 2 Months | ✓ | ✓ | ✓ | Y | +$355 | ||
Bassa 05 (17) | None | 1 Year | ✓ | ✓ | ✓ | Y | −$133 | ||
Bu 07ab (32, 33) | None | 10 Years | ✓ | ✓ | N | −$94 | |||
Cleveringa 10 (18) | TBC | 10 Years | ✓ | ✓ | ✓ | Y | All Patients: +$148; Patients with CVD: +$98 | ||
Cobos 05 (19) | None | 1 Year | ✓ | ✓ | ✓ | Y | −$107 | ||
Frame 94 (20) | Patient reminders | 2 Years | ✓ | N | $0 | ||||
Fretheim 06b (21) | Provider audit and feedback | 1 Year | ✓ | N | −$10 | ||||
Gilmer 12 (22) | Provider incentives | 40 Years | ✓ | ✓ | ✓ | Y | −$46 | ||
Herring 13 (38) | TBC | 9 months | ✓ | N | −$539 | ||||
Khan 10 (23) | None | 32 months | ✓ | ✓ | N | −$236 | |||
Murray 04 (24) | TBC | 1 Year | ✓ | ✓ | ✓ | Y | −$2986 | ||
O’Reilly 12 (25) | None | 40 Years | ✓ | ✓ | ✓ | Y | −$23 | ||
Oxendine 14 (37) | Quality improvement | 1 Year | ✓ | ✓ | N | +$6532 during intervention −$1960 1 year post |
|||
Smith 08 (28) | TBC | 1 Year | ✓ | ✓ | N | Outpatient was −$349. Total with inpatient was −$2800 (authors noted that inpatient cost increase was primarily due to elective surgeries for musculoskeletal pain). |
CDSS, clinical decision support system; ER, emergency room; TBC, team-based care; Y, Yes, N, No
Healthcare cost for intervention versus control group or pre to post change where there is no control group.
Based on eight studies(16–19, 22, 24, 25, 30) that included at least three of five components of healthcare cost, the median change in healthcare cost per patient per year was −$35 (interquartile interval [IQI]: −$127 to $75). However, the estimated change in healthcare cost cannot be attributed to CDSS alone for two(18, 24) nterventions that included team-based care. With these studies removed, the remaining 6 studies produced a median change in healthcare cost per patient per year of −$35 (IQI: −$114 to $93).
Worksite Productivity
No studies assessed the economic benefit of worksite productivity improvements for patients whose health improved through use of a CDSS.
Cost-benefit and Cost-effectiveness
Economic benefit was compared to intervention cost in two(21, 23) studies of EHR-based CDSS and one(32, 33) study of both EHR- and registry-based CDSS (Table 4). The same time horizons were used for benefits and cost in these studies. All three studies included fewer than three components of healthcare cost and did not estimate productivity effects; they were therefore incomplete assessments of economic benefit. One study(21) that reported a benefit-to-cost ratio of 2.03:1 considered only the averted cost of medication and did not include ongoing operating cost in its estimate for intervention cost. The second study(23) provided a reasonably complete assessment of intervention cost but only considered the cost of inpatient stays and emergency room (ER) visits, estimating benefit-to-cost at 3.8:1. The third study(32, 33) estimated the benefit of averted inpatient stays and outpatient visits to a reasonably complete assessment of intervention cost at 2.3:1 for a registry-based CDSS but 0.55:1 for an EHR-based CDSS. The latter, unfavorable, ratio arose primarily because the only high-quality trial of an EHR-based CDSS for diabetes management at the time showed an increase in systolic blood pressure for the intervention group, which translated to increased CVD risk and CVD events in the economic modeling. In summary, the results from cost-benefit studies are incomplete assessments and indicate mixed conclusions on whether economic benefits exceed the cost of CDSS interventions to prevent CVD.
Table 4.
Cost-Benefit and Cost per QALY Saved Estimates
Study Type of CDSS | Benefit-to-Cost Ratio |
---|---|
Fretheim 06b (21) EHR -based |
0.42:1 over 12 months 2.0:1 over 24 months |
Khan 10 (23) EHR -based |
3.8:1 |
Bu 07a,b (32, 33) EHR -based |
0.55:1 |
Bu 07a,b (32, 33) Registry-based |
2.3:1 |
Cost per QALY saved | |
Cleveringa 10 (18) EHR-based |
All patients − $49,500 Patients with CVD − $19,600 Patients without CVD − $162,000 |
Gilmer 12 (22) EHR -based |
$16,500 |
O’Reilly 12 (25) EHR -based |
$143,000 |
CDSS, clinical decision support system; CVD, cardiovascular disease; EHR, electronic health record; QALY, quality-adjusted life year;
Three studies provided estimates of cost per QALY saved (Table 4), where the same time horizons were used for net cost and adjusted life years lived outcomes: two reported the interventions were cost-effective at $49,000(18) and $16,500(22) and one reported that it was not cost-effective at $143,000.(25) Estimates for healthcare cost from all three studies were reasonably complete but one study(25) did not include annual operating cost in the intervention cost. Further, the two studies with cost per QALY saved <$50,000 included interventions in addition to CDSS, namely team-based care(18) and provider incentives.(22) The demonstration of cost-effectiveness from these two studies is for the combination intervention and cannot be attributed to CDSS alone.
DISCUSSION
A recent symposium concluded(39) that methodologies are yet to be developed that can rigorously evaluate the economic value of health information technologies at the population and national level even though economic value is evident from individual observations of their success at the local and organizational level. The symposium noted the difficulty in transitioning from judgements of economic value at the level of specific implementations to a judgement about the aggregate of the implementations: costs and benefits have to be summed over implementations with different organizational contexts, technologies, functions, outcomes, scales, and scope. This systematic economic review of one type of health information technology, namely CDSS, encountered similar difficulties among others in synthesizing the economic evidence from various implementation instances.
The cost and economic benefit of CDSS implementations from included studies were poorly reported and many studies did not adhere to sound evaluation or accounting practices. Only a few studies provided a complete accounting of cost to develop, implement, maintain, and operate a CDSS. More complete economic evaluations are necessary to obtain reliable estimates for intervention cost across types and sizes of healthcare settings. Reported economic benefits of a CDSS are often determined or guided by the implementation’s disease or risk-factor focus (e.g., hypertension, CVD, diabetes, depression), functionality (e.g., provider prompts, management of orders, disease management), or the implementer’s objective (e.g., containing cost with cheaper drugs or averting hospital re-admissions). However, despite the heterogeneity in the research literature for the focus, functionality, and objective of CDSS implementations, certain guidelines for the evaluation of economic costs and benefits can still be described.
Starting with cost, important features of CDSS from an evaluation perspective are its useful life that generally spans multiple years, and the substantial one-time cost of development and implementation. In addition to the one-time cost, maintaining and operating the CDSS requires technical and medical staff time. The present review estimated intervention cost as the sum of these two components, based on recommended accounting practices and tax rules. Details and references are provided in Appendix 1. Because the cost of a CDSS is generally borne by the practice and may differ by practice size, it is useful to communicate the intervention cost in both per-practice and per-patient terms. Most included evaluation studies did one or the other. An additional advantage of intervention cost measured in per-patient terms is its immediate comparability to healthcare cost and health effects that are generally reported in per-patient terms.
Moving to the benefits side of the evaluation, only those benefits that likely resulted from changes in provider clinical decisions and any consequent change in patient behavior and health should be ascribed to the CDSS implementation. The societal perspective is recommended for economic evaluations to account for all healthcare, regardless of who pays for the various components. It is important to know what impact the CDSS has on total healthcare utilization; following the guidelines prompted by the CDSS may increase outpatient visits and/or medications within primary care but may avert very costly inpatient stays and ER visits in tertiary care. This is not to say a partial perspective is without merit. A primary care study evaluating a CDSS to treat hypertension may report the intervention cost per unit reduction in blood pressure ($/mmHg), a useful statistic for clinic managers.
Evaluations are more complex when one or more additional interventions occur along with the CDSS. Both the intervention cost and economic benefit (including QALY saved) must be ascribable to the CDSS when statements are made about the cost-benefit or cost-effectiveness of CDSS implementations. It is often feasible to estimate the cost of the CDSS and additional intervention(s) separately, but obtaining separate estimates for benefits from the component intervention(s) is likely to be difficult, whether analytically or by study design. For example, in an intervention where team-based care is implemented along with the CDSS, a cost-benefit analysis that included only the cost of CDSS in intervention cost and the combined effect of both team-based care and the CDSS in the benefits would be incomplete. In the absence of complete data, statements about the economic value of the CDSS would have to include appropriate caveats.
Many studies included in this review focused on a single CVD risk factor such as diabetes, high cholesterol, or high blood pressure. Outcomes for lifetime cost per QALY saved or cost-benefit ratios were often modeled based on measured improvements in a single or a few risk factors such as blood pressure, blood glucose, or cholesterol. However, a CDSS for CVD prevention and control would, in practice, be implemented with a multiplicity of functionalities including simple provider and patient reminders for screening and testing, patient risk assessments, and medication and lifestyle counseling recommendations across the range of risk factors and indicators for CVD (and diabetes).
The results presented in the current review are consistent with those found in the broader review of evidence for CDSS implementations for all diseases and conditions.(5, 6) That review found that CDSS/Knowledge Management Systems reduced healthcare cost and produced healthcare cost-savings, but it reached mixed conclusions about cost-effectiveness. Similar to the current review, that review also called for evaluations to standardize the metrics for efficiency and cost.
CONCLUSION
An overall conclusion cannot be reached about the cost-effectiveness or cost-benefit of CDSS for CVD prevention and control. The evidence on cost and benefit is limited by many estimates that do not account for major components, and mixed evidence when the estimates are reasonably complete. Further, the reported cost and benefit in many studies included the effect of interventions in addition to the CDSS.
The quality of economic evidence for CDSS can improve with more evaluations that acknowledge its capital good features and account for both development and operating cost over its span of useful life.
Supplementary Material
Acknowledgments
This authors acknowledge the Division for Heart Disease and Stroke Prevention, CDC, for support and subject matter expertise, particularly the late David B. Callahan, MD. We thank members of our coordination team in the Community Guide Branch at CDC and from other areas of CDC, and our external partners, Kimberly J. Rask, MD, PhD, Emory University, Atlanta; Daniel T. Lackland, DrPH, Medical University of South Carolina, Charleston; and Lynne T. Braun, PhD, ANP, FAAN, FACAA, Rush College of Nursing, Chicago. The authors acknowledge Randy W. Elder, PhD, Kate W. Harris, BA, and Onnalee Gomez, MS, from the Community Guide Branch at CDC for their assistance throughout the review, and John Tibbs, BA, MBA, Office of Noncommunicable Diseases, Injury and Environmental Health, CDC, for assistance with accounting concepts for capital expenditures. The authors also thank Gillian Sanders, PhD, Duke Evidence-based Practice Center, and Amy Kendrick, RN, MSN, MS-CRM, Duke Clinical Research Institute, Durham, North Carolina for the references of additional economic studies identified but not included in the Agency for Healthcare Research and Quality (AHRQ) review.
The work of Gibril Njie and Krista Proia was supported with stipends from the Oak Ridge Institute for Science and Education (ORISE).
FUNDING STATEMENT
The work of Gibril J. Njie and Krista K. Proia were supported with stipends from the Oak Ridge Institute for Science and Education (ORISE).
Footnotes
COMPETING INTERESTS SATEMENT
The authors have no competing interests to declare.
CONTRIBUTORSHIP STATEMENT
All authors contributed substantially to the conception and design of the work. VJ, ABT, SKC, GJN, KKP, and DPH drafted the manuscript, and all authors contributed to revising it critically for important intellectual content. VJ and SKC contributed to acquisition, analysis, and interpretation of data. All authors have approved the manuscript for publication and agree to be accountable for all aspects of the work.
References
- 1.Mozaffarian D, Benjamin EJ, Go AS, et al. Executive Summary: Heart Disease and Stroke Statistics—2016 Update A Report From the American Heart Association. Circulation. 2016;133(4):447–54. doi: 10.1161/CIR.0000000000000366. [DOI] [PubMed] [Google Scholar]
- 2.Community Preventive Services Task Force. Clinical Decision Support Systems Recommended to Prevent Cardiovascular Disease. Am J Prev Med. 2015;49(5):796–9. doi: 10.1016/j.amepre.2015.03.041. [DOI] [PubMed] [Google Scholar]
- 3.Njie GJ, Proia KK, Thota AB, et al. Clinical Decision Support Systems and Prevention: A Community Guide Cardiovascular Disease Systematic Review. Am J Prev Med. 2015 Nov;49(5):784–95. doi: 10.1016/j.amepre.2015.04.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Guide to Community Preventive Services. Economic Reviews. 2014 [cited September 14, 2016]; Available from: https://www.thecommunityguide.org/about/economic-reviews.
- 5.Bright TJ, Wong A, Dhurjati R, et al. Effect of clinical decision-support systems: a systematic review. Ann Intern Med. 2012;157(1):29–43. doi: 10.7326/0003-4819-157-1-201207030-00450. [DOI] [PubMed] [Google Scholar]
- 6.Lobach D, Sanders GD, Bright TJ, et al. Evidence Report No. 203. Rockville, MD: Agency for Healthcare Research and Quality; 2012. Enabling health care decisionmaking through clinical decision support and knowledge management. AHRQ Publication No. 12-E001-EF. [PMC free article] [PubMed] [Google Scholar]
- 7.Guide to Community Preventive Services. Cardiovascular Disease Prevention and Control: Clinical Decision-Support Systems (CDSS): Search Strategy. 2013 [cited September 14, 2016]; Available from: https://www.thecommunityguide.org/findings/cardiovascular-disease-clinical-decision-support-systems-cdss.
- 8.Guide to Community Preventive Services. Cardiovascular Disease Prevention and Control: Clinical Decision-Support Systems (CDSS): Search Strategy – Economic Review. 2014 [cited September 14, 2016]; Available from: https://www.thecommunityguide.org/findings/cardiovascular-disease-clinical-decision-support-systems-cdss.
- 9.World Bank Country and Lending Groups. 2010 [cited; Available from: http://data.worldbank.org/about/country-and-lending-groups.
- 10.Eichler H-G, Kong SX, Gerth WC, Mavros P, Jönsson B. Use of cost-effectiveness analysis in health-care resource allocation decision-making: how are cost-effectiveness thresholds expected to emerge? Value Health. 2004;7(5):518–28. doi: 10.1111/j.1524-4733.2004.75003.x. [DOI] [PubMed] [Google Scholar]
- 11.Grosse SD. Assessing cost-effectiveness in healthcare: history of the $50,000 per QALY threshold. Expert Rev Pharmacoecon Outcomes Res. 2008;8(2):165–78. doi: 10.1586/14737167.8.2.165. [DOI] [PubMed] [Google Scholar]
- 12.Kane CK, Emmons DW. New data on physician practice arrangements: private practice remains strong despite shifts toward hospital employment. 2013 [cited May 12, 2015]; Available from: http://www.nmms.org/sites/default/files/images/2013_9_23_ama_survey_prp-physician-practice-arrangements.pdf.
- 13.Østbye T, Yarnall KS, Krause KM, Pollak KI, Gradison M, Michener JL. Is there time for management of patients with chronic diseases in primary care? Ann Fam Med. 2005;3(3):209–14. doi: 10.1370/afm.310. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.U.S. Department of Labor, Bureau of Labor Statistics. Consumer Price Index. [cited September 14, 2016]; Available from: http://www.bls.gov/cpi/cpid1512.pdf.
- 15.World Bank. Purchasing power parities. [cited Sept. 6, 2016]; Available from: http://data.worldbank.org/indicator/PA.NUS.PRVT.PP.
- 16.Apkon M, Mattera JA, Lin Z, et al. A randomized outpatient trial of a decision-support information technology tool. Arch Intern Med. 2005 Nov 14;165(20):2388–94. doi: 10.1001/archinte.165.20.2388. [DOI] [PubMed] [Google Scholar]
- 17.Bassa A, Del Val M, Cobos A, et al. Impact of a clinical decision support system on the management of patients with hypercholesterolemia in the primary healthcare setting. Dis Manage Health Outcomes. 2005;13(1):65–72. [Google Scholar]
- 18.Cleveringa FG, Welsing PM, van den Donk M, et al. Cost-effectiveness of the diabetes care protocol, a multifaceted computerized decision support diabetes management intervention that reduces cardiovascular risk. Diabetes Care. 2010 Feb;33(2):258–63. doi: 10.2337/dc09-1232. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Cobos A, Vilaseca J, Asenjo C, et al. Cost effectiveness of a clinical decision support system based on the recommendations of the European Society of Cardiology and other societies for the management of hypercholesterolemia: Report of a cluster-randomized trial. Dis Manage Health Outcomes. 2005;13(6):421–32. [Google Scholar]
- 20.Frame PS, Zimmer JG, Werth PL, Hall WJ, Eberly SW. Computer-based vs manual health maintenance tracking. A controlled trial. Arch Fam Med. 1994 Jul;3(7):581–8. doi: 10.1001/archfami.3.7.581. [DOI] [PubMed] [Google Scholar]
- 21.Fretheim A, Aaserud M, Oxman AD. Rational prescribing in primary care (RaPP): economic evaluation of an intervention to improve professional practice. PLoS Med. 2006 Jun;3(6):e216. doi: 10.1371/journal.pmed.0030216. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Gilmer TP, O’Connor PJ, Sperl-Hillen JM, et al. Cost-effectiveness of an electronic medical record based clinical decision support system. Health Serv Res. 2012 Dec;47(6):2137–58. doi: 10.1111/j.1475-6773.2012.01427.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Khan S, Maclean CD, Littenberg B. The effect of the Vermont Diabetes Information System on inpatient and emergency room use: results from a randomized trial. Health Outcomes Res Med. 2010 Jul;1(1):e61–e6. doi: 10.1016/j.ehrm.2010.03.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Murray MD, Harris LE, Overhage JM, et al. Failure of computerized treatment suggestions to improve health outcomes of outpatients with uncomplicated hypertension: results of a randomized controlled trial. Pharmacotherapy. 2004 Mar;24(3):324–37. doi: 10.1592/phco.24.4.324.33173. [DOI] [PubMed] [Google Scholar]
- 25.O’Reilly D, Holbrook A, Blackhouse G, Troyan S, Goeree R. Cost-effectiveness of a shared computerized decision support system for diabetes linked to electronic medical records. J Am Med Inform Assoc. 2012 May-Jun;19(3):341–5. doi: 10.1136/amiajnl-2011-000371. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Overhage JM, Tierney WM, Zhou XH, McDonald CJ. A randomized trial of “corollary orders” to prevent errors of omission. J Am Med Inform Assoc. 1997 Sep-Oct;4(5):364–75. doi: 10.1136/jamia.1997.0040364. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Shih SC, McCullough CM, Wang JJ, Singer J, Parsons AS. Health information systems in small practices. Improving the delivery of clinical preventive services. Am J Prev Med. 2011;41(6):603–9. doi: 10.1016/j.amepre.2011.07.024. [DOI] [PubMed] [Google Scholar]
- 28.Smith SA, Shah ND, Bryant SC, et al. Chronic care model and shared care in diabetes: randomized trial of an electronic decision support system. Mayo Clin Proc. 2008 Jul;83(7):747–57. doi: 10.4065/83.7.747. [DOI] [PubMed] [Google Scholar]
- 29.Zamora A, de Bobadilla FF, Carrion C, et al. Pilot study to validate a computer-based clinical decision support system for dyslipidemia treatment (HTE-DLP) Atherosclerosis. 2013;231(2):401–4. doi: 10.1016/j.atherosclerosis.2013.09.029. [DOI] [PubMed] [Google Scholar]
- 30.Blanchfield BB, Grant RW, Estey GA, Chueh HC, Gazelle GS, Meigs JB. Cost of an informatics-based diabetes management program. Int J Technol Assess Health Care. 2006;22(02):249–54. doi: 10.1017/S0266462306051075. [DOI] [PubMed] [Google Scholar]
- 31.Adler-Milstein J, Bu D, Pan E, et al. The cost of information technology-enabled diabetes management. Dis Manag. 2007;10(3):115–28. doi: 10.1089/dis.2007.103640. [DOI] [PubMed] [Google Scholar]
- 32.Bu D, Pan E, Johnston D, et al. The value of information technology-enabled diabetes management. Center for Information Technology Leadership, Healthcare Information and Management System Society; Charleston, MA: 2007. p. 51. ( http://www.partners.org/cird/pdfs/CITL_ITDM_Report.pdf) Role and Potential Influence of Technologies on the Most Relevant Challenges for LTC. [Google Scholar]
- 33.Bu D, Pan E, Walker J, et al. Benefits of information technology-enabled diabetes management. Diabetes Care. 2007 May;30(5):1137–42. doi: 10.2337/dc06-2101. [DOI] [PubMed] [Google Scholar]
- 34.Guide to Community Preventive Services. Cardiovascular Disease Prevention and Control: Clinical Decision-Support Systems (CDSS) Summary Evidence Table: Economic Review. [cited September 14, 2016]; Available from: https://www.thecommunityguide.org/sites/default/files/assets/SET-CDSS-econ-2013.pdf.
- 35.Bardach NS, Wang JJ, De Leon SF, et al. Effect of pay-for-performance incentives on quality of care in small practices with electronic health records: a randomized trial. JAMA. 2013;310(10):1051–9. doi: 10.1001/jama.2013.277353. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Munoz M, Pronovost P, Dintzis J, et al. Implementing and evaluating a multicomponent inpatient diabetes management program: putting research into practice. Joint Commission journal on quality and patient safety/Joint Commission Resources. 2012;38(5):195. doi: 10.1016/s1553-7250(12)38025-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Oxendine V, Meyer A, Reid PV, Adams A, Sabol V. Evaluating Diabetes Outcomes and Costs Within an Ambulatory Setting: A Strategic Approach Utilizing a Clinical Decision Support System. Clinical Diabetes. 2014;32(3):113–20. doi: 10.2337/diaclin.32.3.113. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Herring R, Russell-Jones D, Pengilley C, et al. Management of raised glucose, a clinical decision tool to reduce length of stay of patients with hyperglycaemia. Diabet Med. 2013;30(1):81–7. doi: 10.1111/dme.12006. [DOI] [PubMed] [Google Scholar]
- 39.Payne TH, Bates DW, Berner ES, et al. Healthcare information technology and economics. J Am Med Inform Assoc. 2013;20(2):212–7. doi: 10.1136/amiajnl-2012-000821. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.