The economics of health information technology in medication management: a systematic review of economic evaluations

Abstract

Objective

To conduct a systematic review and synthesis of the evidence surrounding the cost-effectiveness of health information technology (HIT) in the medication process.

Materials and methods

Peer-reviewed electronic databases and gray literature were searched to identify studies on HIT used to assist in the medication management process. Articles including an economic component were reviewed for further screening. For this review, full cost-effectiveness analyses, cost-utility analyses and cost-benefit analyses, as well as cost analyses, were eligible for inclusion and synthesis.

Results

The 31 studies included were heterogeneous with respect to the HIT evaluated, setting, and economic methods used. Thus the data could not be synthesized, and a narrative review was conducted. Most studies evaluated computer decision support systems in hospital settings in the USA, and only five of the studied performed full economic evaluations.

Discussion

Most studies merely provided cost data; however, useful economic data involves far more input. A full economic evaluation includes a full enumeration of the costs, synthesized with the outcomes of the intervention.

Conclusion

The quality of the economic literature in this area is poor. A few studies found that HIT may offer cost advantages despite their increased acquisition costs. However, given the uncertainty that surrounds the costs and outcomes data, and limited study designs, it is difficult to reach any definitive conclusion as to whether the additional costs and benefits represent value for money. Sophisticated concurrent prospective economic evaluations need to be conducted to address whether HIT interventions in the medication management process are cost-effective.

Background and significance

The introduction of health information technology (HIT) into the medication management process holds the promise of reducing adverse drug events (ADEs), increasing efficiency of care delivery, improving quality of care, reducing costs, and saving money over the longer term. However, even if these technologies are effective, they are complex and expensive to acquire, implement, and maintain. Assessing the cost versus effectiveness is critical to determining HIT's value and ultimately its adoption. An ideal economic evaluation of these technologies would explicitly measure all direct healthcare costs (eg, capital costs) and direct non-healthcare costs (eg, home care services) as well as indirect costs (eg, productivity losses or gains) that could be affected by an intervention. Additionally, the full enumeration of the total costs needs to be synthesized with the consequences of the intervention.¹ A review of the economic literature to determine cost-effectiveness and value for money for such interventions is warranted.

Objective

The purpose of this paper is to review and synthesize the evidence on the costs and cost-effectiveness of HIT in the medication process. This review was undertaken as part of a larger study funded by the US Agency for Healthcare Research and Quality to review the evidence on the effectiveness of HIT in all phases of the medication management process.²

Methods

Literature search

Details of the search strategy have been described previously.² Briefly, electronic databases (eg, Medline, Embase, Business Source Complete) and gray literature were searched. The search strategy comprised controlled vocabulary and keywords to identify papers concerned with specific devices supporting the medication process (eg, electronic medical record (EMR)) published in all languages up to the summer of 2010. Hand searches of bibliographies were also performed.

Selection method

For inclusion, studies had to meet the following criteria: electronic systems that collect, process, or exchange health information about patients and formal care givers; medication management information technology that was integrated with at least one HIT system that processed patient-specific information and provided advice to the healthcare provider or patient or dealt with transmission or order communication between pharmacist and clinical prescriber. Any article that included an economic component was tagged and underwent further screening. For this review, full and partial economic evaluations were eligible for inclusion. A full economic evaluation is the comparative analysis of alternative courses of action in terms of both costs and consequences, and these were further classified into one of the three categories: (1) cost-effectiveness analysis; (2) cost-utility analysis; and (3) cost-benefit analysis.¹ The label partial economic evaluation indicates that the studies are restricted to either costs or outcomes or consider no alternative to the intervention being studies. However, cost analyses can provide useful information on ‘upfront’ costs compared with ‘downstream’ cost avoidance¹ and thus were included in this review. Within each of these classifications, articles were further categorized into the setting in which the evaluation took place as well as the type of HIT implemented. A checklist for assessing economic evaluations developed by Drummond et al³ was used to help guide the assessment of the literature.

In order to make the economic results more comparable, inflation adjustments⁴ and currency conversions⁵ were carried out so that all values in the text and tables reflect 2011 US dollars.

Results

Of the 35 510 articles identified in the original search, 40 contained some cost information. Of these, 31 were included in this analysis (figure 1). The nine excluded studies either did not include any cost data or did not provide an evaluation of the costs. The majority of studies evaluated computer decision support systems (CDSSs) (n=19), and 65% of the studies were conducted in hospital settings. Most of the reports (n=23, 74%) were from the USA, three (10%) were from Canada, and the rest were in Europe and Israel (tables 1 and 2). Only 16% (n=5) of the studies performed full economic evaluations. Two (7%) of the studies were conducted in pediatric populations, and eight (26%) were concerned with improving the use of, and reduction in, the costs of antibiotic therapy. Owing to the heterogeneity of studies, a synthesis of the data was not possible, and thus a narrative description of the findings is provided in the following sections.

Figure 1.

Literature flow of medication management studies. Adapted from the larger AHRQ report.² MMIT, medical management information technology.

Table 1.

Summary of full economic evaluation studies (all costs presented in 2011 USD)

Author (year) country	Type of economic evaluation	Study objective	Study design (include setting)	Population (n)	Perspective (time horizon)	Cost elements	Effectiveness measure	Intervention and alternative being evaluated	Main economic findings
Hospital-based MMIT
CPOE
Wu6 (2007) Canada	Cost-effectiveness analysis	To determine the potential incremental cost-effectiveness of an electronic MOE/MAR system	Economic evaluation comparing the mean effectiveness of: electronic MOE/MAR versus standard paper ordering for the prevention of ADEs. Setting: 3 tertiary care teaching hospitals	Hypothetical population	Healthcare institution (10 years with 5% discount rate)	Implementation costs (software, project management, clinical team involvement and training); operating costs (support for new interface, training))	Reduction of pADEs and mortality (from literature)	MOE/MAR compared with conventional paper-based system	Incremental costs for CPOE system versus paper was $15 192 per ADE averted (sensitive to the ADE rate, system effectiveness, the cost of system, and costs of increase in doctor workload)
Karnon7 (2008) UK	Cost-utility analysis	To estimate the net benefits of interventions that aim to reduce the impact of medication errors	A decision tree model to describe a series of error points and subsequent error detection points in pathways through the medication process. Assumed an acute hospital size of 400 beds model	Model populated with estimates of incidence and impacts of medication errors. The effectiveness of potential interventions on error incidence and detection rates that alter the estimated frequency of medication errors and pADEs	Payer (5-year time horizon to represent the predicted useful life of the IT-based interventions)	Interventions, efficiency savings, treatment and health effects of pADEs	Quality of life utility decrements associated with experiencing a pADE	CPOE versus additional ward pharmacists versus bar coding	Mean net benefits of $64.9 million for CPOE
Computerized reminder system
Rosser8 (1992) Canada	Cost-effectiveness analysis	To assess the cost-effectiveness of three computerized reminder systems on compliance with tetanus vaccination	Prospective randomized controlled trial (4 arms). Hospital Family Medicine Centre over 1 year	5242 randomized patients and 2369 non-randomized patients ≥20 years of age not in a hospital or institution	Healthcare practice (1 year)	Physician time, clerical and nurse time, stationary, stamps, prepaid envelopes	Proportion of patients who received tetanus toxoid during the study year or who had a claim of vaccination in the previous 10 years	Computer-generated physician reminder, versus telephone reminder to patient, versus letter reminder to patient for tetanus vaccination	The cost per additional vaccination recorded was between $0.33 and $0.64 for the physician reminders, between $6.66 and $8.16 for the telephone reminders, and $9.11 for the letter reminders
Primary care-based MMIT
Computerized reminder system
Fretheim9 (2006) Norway	Cost-effectiveness analyses	Compare costs and effects of a multifaceted intervention aimed at improving prescribing of antihypertensive and cholesterol-lowering drugs compared with usual care	Using data from a cluster-RCT of private practices, cost incurred per additional patient started on a thiazide rather than another antihypertensive drug	Intervention: 73 practices with 70 included in analysis. Control: 73 with 69 included in analysis	Perspective of the healthcare system (1 year)	Development of software; training of outreach visitors; printed material; travel; cost of pharmacists doing outreach; admin costs; opportunity cost of physician time; technical support; drug expenditure; number of consultations per patient; laboratory tests	Number of patients prescribed thiazides for hypertension	(1) Educational outreach visits; (2) audit & feedback on current adherence to guidelines & recommendations; (3) computerized reminders during patient encounter versus passive dissemination of guidelines through medical journal	$570 per additional patient started on thiazides
CDSS
Plaza10 (2005) Spain	Cost-effectiveness	To evaluate the cost-effectiveness of a CDSS to promote the recommendation of the GINA compared with standard practice	Randomized, multicenter, prospective, pragmatic study comparing CDSS offering recommendations or no CDSS	20 physicians (10 pulmonologists and 10 primary care physicians). 198 asthmatic patients followed for 1 year	Societal & national health system (ie, payer) perspectives	Direct (resource × unit cost, treatment costs) and indirect (time off work due to medical visits) costs	Difference in QOL using SGRQ, healthcare resources, medical visits, hospitalizations, asthma treatment, blood analysis, spirometry, chest radiographs	CDSS versus no CDSS	From societal perspective the CDSS was dominant. From the payer perspective the ICER was $66.64/percentage point reduction in SGRQ scale

ADE, adverse event; CDSS, computerized decision support system; CPOE, computerized physician order entry; GINA, Global Initiative for Asthma; ICER, incremental cost-effectiveness ratio; MAR, medication administration record; MMIT, medical management information technology; MOE, medication ordering entry; pADE, preventable adverse drug events; QOL, quality of life; RCT, randomized controlled trial; SGRQ, St George Respiratory Questionnaire.

Table 2.

Summary of partial economic evaluation studies (all costs in 2011 USD)

Author (year) country	Type of economic evaluation	Study objective	Study design (include setting)	Population (n)	Currency (year)	Cost elements	Effect measure	Intervention and alternative being evaluated	Main economic findings
Hospital-based MMIT
CPOE
Chisolm (2006)11 USA	Cost analysis	To assess the relationship between a computerized order set within a CPOE and processes of care pediatric asthma treatment	Before/after. Inpatient pediatric teaching hospital	Asthma patients between the age of 2 and 20 years admitted to hospital between November 2001 and November 2003. N=790 (261 ‘pre–set’; 63 ‘no set (order set not used)’; 466 ‘set’ cases)	USD (not stated)	LOS, total inpatient charges and pharmacy charges	Use of systemic corticosteroids, use of pulse oximetry, and use of metered-dose inhalers.	Computerized asthma order set within a CPOE system before and after implementation	No significant difference in costs or LOS among the three groups. Total charges were $4446, $4381, $4616 pharmacy charges were $511, $458, $527; for the ‘no set’, ‘pre-set’, and ‘set’ groups respectively
Mekhjian12 (2002) USA	Cost analysis	To evaluate the benefits of a CPOE and eMAR on the delivery of healthcare	Cohort of inpatient nursing units in an academic health system (3 sites), before-and-after POE and within post-CPOE for a period of 10–12 months across all services in the respective hospitals	Cohort of inpatient nursing units	USD (2002)	Total costs per patient	LOS, medication, radiology, and laboratory test turn-around times, medication transcription errors	Pre-CPOE and post-CPOE and, within post-CPOE (a comparison of POE and the combination of POE plus eMAR)	Some services showed a reduction in costs but overall, when all the services were combined, severity-adjusted total cost per admission did not change significantly in either phase (pre-CPOE, $7156; post-CPOE, $7111 p=0.687) or in the cancer hospital (pre-CPOE, $8073 post-CPOE, $8187; p=0.502)
Stone13 (2009) USA	Cost analysis	Evaluate the implementation of a CPOE system in the management of surgical patients	Retrospective and prospective analyses of patient-safety measures 6 months pre- and 6 months post-CPOE. Inpatients of a multispecialty hospital academic surgical practice	Number of surgical procedures pre and post: 6815 procedures in the pre period and 5963 in the first post 6 month and 6106 in the second 6 months post implementation	USD (2007/2008)	Personnel and capital costs of implementation	Patient safety, medication errors, order implementation time	CPOE compared with no CPOE	Total capital costs of $3.2 million and operating costs of $2.5 million with a savings of in personnel cost of $485512. Improvements in efficiencies noted (eg, turnaround time for orders to nursing, radiology, and laboratory)
Tierney14 (1993) USA	Cost analysis	To assess the effects on healthcare resource utilization of a CPOE that encourages cost-effective ordering. Aim of increasing cost consciousness and reducing costs	RCT in an inpatient internal medicine service of an urban public hospital over 6 months	6 medical services randomly assigned to intervention (n=1859) and control (n=3360)	USD (1990/91)	Inpatient charges (LOS, tests and drugs)	LOS, time in motion	Microcomputer workstations linked to a comprehensive EMR system for all inpatient order versus hand-written orders	Total charges per admission were significantly less ($1534, 12.7% reduction) for intervention versus control with similar differences in bed, test, and drug charges. The reductions in LOS could amount to more than $5.2 million in charges annually for that hospital's medicine service
CDSS
Evans15 (1994) USA	Cost analysis	To evaluate a CDSS to assist physicians in the selection of appropriate empiric antibiotics	Two-stage random-selection study (tertiary, private hospital and major teaching centre). 12-month time frame	28 physicians, 482 cultures	USD (1994)	Cost of antibiotics	Computer-suggested antibiotics with results of susceptibility tests of cultures and antibiotics selected by physician	Antibiotics ordered using CDSS by randomized physicians compared between crossover periods of antibiotic consultant use	The average cost for 24 h of therapy for the computer-suggested antibiotics was $62.64 per patient, compared with an average of $79.18 for antibiotics prescribed
Evans16 (1995) USA	Cost analysis	To evaluate a CDSS to improve the use of and reduce the cost of antibiotics	7-month pilot compared with 12-months previous in a 12-bed shock/trauma/respiratory ICU.	588 orders for antibiotics	USD (1994)	Cost of antibiotics	ADEs and LOS	Antibiotics ordered using CDSS during the study period compared to the control period	The mean cost of antibiotics was $132.70 (p<0. 04) less per patient during the study period compared with the control period
Evans17(1998) USA	Cost analysis	To evaluate a CDSS to improve the use of and reduce the cost of antibiotics	Prospective study in a 12 bed shock/trauma/respiratory ICU (before/after) 12 months	398 patients in intervention; 766 patients in control	USD (1995)	Cost of antibiotics and hospitalization	ADEs, days of excessive antibiotic dosage, LOS, and mortality	Antibiotics ordered using CDSS during the study period compared with the control period	The cost of antibiotics was $151 versus $504 and $633 and the total cost of hospitalization was $39 017 versus $52 314 and $66 522 for control, regimen followed, and regimen overridden, respectively
Evans18 (1999) USA	Cost analysis	To examine the effect of a computer-assisted antibiotic dose monitor to check renal function and the number of ADEs secondary to antibiotics	Descriptive epidemiologic study of a 2-year pre-intervention period and 1-year intervention period	All patients ≥18 years, admitted to hospital from April 1, 1993, to March 31, 1996, who received ≥1 of 5 targeted antibiotics with a serum creatinine or urine creatinine clearance test result before antibiotic therapy, who were never admitted to the ICU	USD (1996)	Cost of antibiotics	ADEs, days of excessive antibiotic dosage	Antibiotics ordered using CDSS during the study period compared with the control period	The intervention group had fewer mean doses of antibiotics at a lower mean cost ($116.11 vs $133.88) than patients during the pre-intervention period. This could result in a decrease in cost of more than $79210 per year
Barenfanger19 (2001) USA	Cost analysis	To assess the impact of an antibiotic therapy intervention improvement program	Quasi RCT (2 arm) of hospitalized patients prospective study in a 450-bed community teaching hospital over a 5-month time period	(i) Patients infected with a bacterial isolate without an order for antimicrobial therapy, (ii) patients infected with bacteria resistant to current antimicrobial therapy, (iii) patients on therapy who were not tested, and (iv) patients on therapy but from whom no sample for culture had been taken	USD (not stated)	Total costs, fixed costs (overhead), variable direct (pharmacy, supplies, lab tests, radiology tests), fixed indirect costs	Mortality, LOS	Compared patients whose microbiologic data were processed in the normal manual manner in the pharmacy to data processed using computer software and alerts pharmacists of potential interventions	The estimated variable cost savings annually is $3 976 749 (2000 inpatients for whom susceptibility testing was done × $1988). If the list price of software ($60 357) was subtracted, the expected annual cost savings was $3 916 393
McGregor20 (2006) USA	Cost analysis	To evaluate the effectiveness and cost effectiveness of a web-based, computerized CDSS for the management of antimicrobial utilization	RCT, in-patients, a 648-bed tertiary care, academic hospital over a 3-month period	N=4507 (n=2237 in the intervention arm and 2270 control arm)	USD (2004)	Hospital antimicrobial costs	Mortality, LOS, frequency of tests for Clostridium difficile, time spent managing antimicrobial utilization	Antimicrobial utilization managed by existing AMT using the system in the intervention arm and without the system in the control arm. The system alerted the AMT of potentially inadequate antimicrobial therapy (post-prescription review)	Hospital antimicrobial expenditures were $341 891 for intervention vs $442 605 in the control arm, for a savings of $100 714 (23%), or $45.03 per patient
Paul21 (2006) Israel, Germany, Italy	Cost analysis	To compare CDSS advice with physician performance for antibiotic treatment and antibiotic costs. In the RCT, the goal was to assess whether the CDSS improved physician performance and patient-related outcomes	Prospective 6-month cohort study comparing a CDSS for antibiotic treatment advice to physician's treatment followed by a 6-month, multicenter, cluster RCT comparing wards using the CDSS versus antibiotic monitoring without the CDSS; (3 university affiliated primary and tertiary hospitals)	Patients suspected of harboring bacterial infections in 3 hospitals (Israel, Germany, Italy)Cohort: 1203 patients RCT: 2326 patients	Euros (Cohort-2002/03; RCT-2004)	Direct drug & administration costs; ADE (rates from the literature and costs in hospital days and QALYs); ecological costs (costs to ecosystem for loss of antibiotic efficacy)	Appropriate antibiotic treatment, mortality, LOS	CDSS advises antibiotic therapy for inpatients using data available at the time of empirical antibiotic treatment, with the highest cost-benefit difference, including no antibiotic treatment versus antibiotic monitoring without CDSS	Cohort: total antibiotic costs were $344 lower per patient for CDSS compared with physicians, a relative decrease of 48%. RCT: mean total antibiotic costs per pt $900.04 (control) versus $816.57 (intervention) p=0.007.Total projected costs for the appropriate CDSS regimens were lower than control by $378.39 per patient, with the reduction originating mainly from lower ecological costs
Mullett22 (2001) USA	Cost analysis	To evaluate the impact of a pediatric anti-infective CDSS	Cohort, patients in a 26-bed pediatric ICU in an academic 232-bed hospital; 6-month pre- versus post-implementation	Pediatric patients; N=1758 (809 control, 949 intervention)	USD (1999)	Hospital costs, anti-infective drug charges	Number of anti-infective drugs used, total doses used, LOS, mortality	CDSS versus pre-CDSS (handwritten orders)	No difference in hospital costs $38 326.63 versus $33 951.71 No difference in mean anti-infective cost/patient $372.70 versus $392.79 for control and intervention respectively
Chertow23 (2001) USA	Cost analysis	To determine if a system application for adjusting drug dose and frequency in patients with renal insufficiency improves drug prescribing and patient outcomes	Four consecutive 2-month intervals consisting of control (usual CPOE) alternating with intervention (CPOE plus decision support system), conducted in a 720-bed urban tertiary care teaching hospital	Hospitalized patients with renal insufficiency, 7887 admissions during the two intervention periods, 9941 admissions in the two control periods. September 1997–April 1998	USD (not stated)	Hospital and pharmacy charges	Rates of appropriate prescription, LOS, and changes in renal function	Real-time CDSS for prescribing drugs in patients with renal insufficiency. The CDSS displayed the adjusted dose list, default dose amount, and default frequency to the order-entry user and a notation was provided that adjustments had been made based on renal insufficiency versus no recommendations in the control group	No significant differences in total costs (US$6766 versus US$6887 for the intervention and the control groups, respectively)
CPOE and CDSS
Macdonald24 (2002) Canada	Cost analysis	Evaluation of the safety and potential cost savings of a computerized, laboratory-based program to manage inpatient warfarin thromboprophylaxis after major joint arthroplasty	Consecutive-case study of adults admitted over a 54-month period (July 1994–December 1998) in a tertiary care orthopedic institution compared with patients who underwent similar procedures in the 18-month period before the program was introduced	Patients requiring joint arthroplasty who had no recent episodes of thromboembolic disease, no mechanical heart valve, atrial fibrillation, severe liver disease or baseline international normalized ratio greater than 1.3 (n=4729, intervention vs n=279, control)	CAD (not stated)	Pharmacy and comparative nursing care costs associated with the program	Test results maintained within the desired therapeutic range, severe bleeding episodes, readmission rates, episodes of venous thrombosis or pulmonary embolism	Major joint arthroplasty with warfarin therapy administered through the computerized program compared with an historical control group.	The potential savings per patient would be 11 min of nursing time or $4.62/patient daily for a total annual figure, based on 10 152 patient days/yr of $46 910
Kaushal25 (2006) USA	Cost analysis	To assess the costs and benefits associated with the implementation of a CPOE and CDSS system	Cost and benefit estimates of a CPOE system in a 720-adult bed, tertiary care academic hospital	Patients admitted to the hospital over the 10-year timeframe (1993–2002)	USD (2002, 7% discounting)	Capital, operational, drug, and hospital costs	Reductions in ADEs, LOS, proportion of appropriate prescriptions, laboratory and radiology tests (some measures from the literature)	CPOE with graduated CDSS over 10 years compared to estimates of what it might have been without the CPOE	$14.8 million to develop, implement, and operate CPOE; over 10 yrs, savings to hospital of $35.8 million. It took over 5 years to realize a net benefit and over 7 years to realize an operating budget benefit
Computerized ADE surveillance system
Evans26 (1992) USA	Quasi cost analysis	To use a hospital information system to help identify ADEs and to create a database of ADEs	Pre-post design using a computerized ADE surveillance system versus a control group with no ADEs	Hospitalized patients	USD (not stated)	Hospital costs	Reduction in ADEs and LOS	Computerized surveillance with physician notification of verified ADEs (severe) versus immediate notification of all verified ADEs. Recommendation of drug or dosage change was provided	Average cost of hospitalization was $61 213 for patients with severe ADEs, $36 196 for patients with moderate ADEs and $10 179 for patients without ADEs. The authors suggest that a reduction in ADEs would result in decreased hospital costs
Piontek27 (2010) USA	Cost analysis	Effects of an ADE alert system on cost and quality outcomes in community hospitals	Retrospective observational study of the effects of an ADE alert system in 7 hospitals in a health network. Outcomes after, and 1-year before the ADE alert system. Inpatients in 2 hospitals without an ADE alert system constituted the control group	All inpatients admitted to 1 of 7 hospitals in a health network	USD (not stated)	Pharmacy department costs, variable drug costs; total hospitalization costs	Mortality; LOS, rate of readmission, and case-mix index	Pre-post ADE alert system. Four groups: (1) pre-ADE alert system (internal control group); (2) post-implementation group; (3) external control group matching internal control time frame, and (4) external control group matching ADE post-implementation time frame	Pharmacy costs from pre–post-: $898 vs $856 per patient; external control group: $760 vs $826. Drug costs: $373 vs $349 in study group and $416 vs $445 in external control group. If the percentage decreases are applied to the control groups, the pharmacy cost savings is >$11.4 million
Primary care setting
CPOE
Weingart28 2009 USA	Cost analysis	To understand the potential benefits of medication safety alerts in ambulatory care if a commercial CPOE were used	A 6-month, multifaceted study. Estimations of the likelihood and severity of ADEs associated with an alert, the likely injury to the patient, and the healthcare utilization required to address each ADE were made using an expert panel	279476 alerted prescriptions written by 2321 ambulatory care clinicians	USD (2006)	Hospitalization, emergency room visit, office visit, filled prescription	ADEs and related injuries	Potential benefit of electronic prescribing with decision support based on expert panel estimates	Alerts potentially resulted in a cost savings of $451 277 (IQR, $158 054–$1 134 736)
CDSS
Tierney29 (2003) USA	Cost analysis	To assess the effects of an established EMR system containing a CPOE with a guideline-based CDSS for managing patients with IHD and HF	1-year, 2×2 factorial, 4 arm RCT, academic, primary care group practice	11 full time and 9 part-time outpatient pharmacists, physicians; 32 practice sessions (706 patients included)	USD (1994/1996)	Total healthcare charges (outpatient & inpatient charges)	Adherence to recommendation, QOL, exacerbation of IHD, medication compliance, satisfaction with care, physician attitude toward intervention	Evidence-based cardiac care recommendations displayed electronically to physicians, pharmacists, physician and pharmacists versus no recommendations	No difference in total costs across groups Costs: Control: $10 117 (SD $24 518); Physician only: $9076 (SD $15 738); Pharmacist only; $10 639 (SD: $19 019) Both physician and pharmacist: $11 002 (SD: $24 369)
Tierney30 (2005) USA	Cost analysis	To assess whether guideline-based care suggestions delivered via physicians' and pharmacists' computer improve the outpatient management and outcomes among patients with asthma or COPD	1-year, 2×2 factorial, 4 arm RCT, academic general internal medical practice in a hospital	246 physicians and 20 outpatient pharmacists randomized (706 patients included)	USD (1994/1996)	Total healthcare charges (outpatient & inpatient)	Adherence to treatment guidelines, QOL, pt satisfaction with physician & pharmacists, ER visits, hospitalizations	Care recommendations provided electronically to physicians, pharmacists, both physician and pharmacist versus no care recommendations	Control: $8353 (SD: $12 293); Physician only: $11 530 (SD: $26 960); Pharmacist only: $7681(SD: $13538); both physician and pharmacist: $8140 (SD: $15 236)
McMullin31 (2004) USA	Cost analysis	To evaluate the impact of a CDSS on prescription costs	Retrospective cohort study (before-after) using pharmacy claims database in primary care	Clinicians using CDSS were matched to controls with 6 month follow-up (19 physicians per group)	USD (not stated)	New and refilled prescription costs	Nil	CDSS provides evidence-based recommendations to clinicians during the electronic prescribing process	Average cost per new prescription was $4.98 lower (p=0.02) in the intervention group, and the average cost for new and refilled prescriptions was $5.97 lower (p=0.01). The 6-month savings were estimated to be $4127 (95% CI $1232 to $7013) per clinician
McMullin32 (2005) USA	Cost analysis	To evaluate the impact of a CDSS on prescription costs	Retrospective cohort study (before-after) using pharmacy claims database in primary care	Clinicians using CDSS were matched to controls. 12 months (6-month study extension of above mentioned study (19 physicians in each group)	USD (not stated)	New and existing prescription costs	Nil	CDSS provides evidence-based recommendations to clinicians during the electronic prescribing process	Average cost per new prescription decreased by $1.00 in the intervention group and increased by $4.34 in the control group. The 12-month savings was $127 152
Cobos33 (2005) Spain	Cost analysis	To assess the cost and effectiveness of a CDSS based on recommendations of the ESCHM in comparison with usual care for patients with hypercholesterolemia	A 1-year, multi-center cluster-randomized, unblinded, pragmatic trial in a primary care setting	Patients with hypercholesterolemia. Patients were excluded if they had triglyceride concentrations >400 mg/dl or were participating in another study. 44 practices, 2221 pts (1161 usual care, 1060 CDSS)	Euros (2002)	Direct costs: physician visits, lab analyses, lipid-lowering drugs	Achievement of LDL-C reduction goals in pts with CVR >20% over 10 years or keeping it <20% when pt baseline was <20%	CDSS versus usual care	The treatment costs were $254 484 in the usual care group and $149 415 in the intervention group. The adjusted means of the total costs per patient were $337 in the usual care group and $265 in the intervention group. The difference was $71 (95% CI $39 to $102; p=0.001).
E-Prescribing with CDSS
Ornstein34 (1999) USA	Cost analysis	To determine the impact of displaying prescription cost information in a CPR system on decreasing drug costs by family physicians	During a 6-month period, cost information was not displayed; during the subsequent 6-month intervention period, costs were displayed. Academic family practice setting	10 physicians, 36 residents	USD (1995/1996)	Prescription costs	Nil	CPR system that displays drug cost information at time of prescription order compared with no cost information being displayed during the control period	No difference in overall drug costs. The mean (SD) cost per prescription in the control period was $31.44 ($38.88), and $31.73 ($40.50), (P= 0.61) in the intervention period
Hospital and primary care settings
CDSS
Javitt35 (2005) USA	Cost analysis	To demonstrate the potential effect of deploying a sentinel system that scans administrative claims information and clinical data to detect and mitigate errors in care and deviations from best medical practices	RCT, members of an HMO randomly assigned to intervention or control. CCs generated by CDSS for subjects in the intervention group were relayed to treating physicians, and those for the control group were deferred to study end	All health plan enrollees between the ages of 12 and 64 years and had incurred at least one physician claim or one pharmacy claim in the 12 months before enrollment	USD (not stated)	Total charges (in-patient, out-patient, prescription, professional)	CCs generated by group; physician compliance with recommendation; and hospital utilization	CDSS tool that produces an electronic record from administrative data and runs it through a set of decision rules, identifies ‘issues’ and sends a CC message to the physician in the intervention group but not to the control group	Charges for intervention group were $90.14 pmpm lower and paid claims were $78.77 pmpm lower than control compared with the baseline. Paid claims for the entire intervention group were $9.34 pmpm lower than for the entire control group. The intervention cost $1.16 pmpm, suggesting an 8-fold return on investment
Javitt36 (2008) USA	Cost analysis	To determine whether a CDSS tool improves quality of care and the effect of the intervention on average charges pmpm	RCT, participants randomized to study group had the software turned on and not in the control group. Conducted in a large HMO	Patients all had medical charges in the previous year, all patients <65 years n=19 719 intervention group, n=19 792 control group	USD (2001)	Total charges (in-patient, out-patient, prescription, professional)	Rate at which CCs are resolved	CDSS tool that produces an electronic record from administrative data then runs the record through a set of decision rules, identifies ‘issues’ and sends a CC message to either the HMO medical director and they called the appropriate physician or to an HMO nurse and they decided whether to send message to physician. CCs turned off in control group	The intervention reduced the average total charges in study group by 6.1% of the average charge for the control group ($417.91 vs $449.52 pmpm)

ADE, adverse drug event; AMT, antimicrobial management team; CC, Care consideration; CDSS, computerized decision support system; CHF, congestive heart failure; CPOE, computerized physician order entry; CPR, computer-based patient record; CVR, cardiovascular risk; eMAR, electronic medication administration record; EMR, electronic medical record; ESCHM, European Society of Cardiology and other societies for Hypercholesterolemia Management; ER, emergency room; HF, heart failure; HMO,Health Maintenance Organization ; ICU, intensive care unit; IHD, ischemic heart disease; LOS, length of stay; MMIT, medical management information technology; pmpm, per member per month; POE, physician order entry; QALY, quality-adjusted life year.

Full economic evaluations

The following section reports the findings of the five economic evaluations that provided information on the incremental costs and the incremental effects an HIT. The HITs included computerized physician order entry (CPOE) (n=2), computerized reminder systems (n=2), and CDSS (n=1) for improving prescribing practices (table 1).

Hospital setting

Computerized physician order entry

Two studies evaluated the economic consequences of implementing a CPOE system to an existing EMR in a hospital setting. The first study used both primary data and data from various sources to obtain ADE rates as well as cost data.⁶ The new system included a menu of medications from the formulary with typical doses, drug-allergy checking, drug–drug interaction, and duplicate drug checking. The analysis included system costs (eg, implementation) and cost savings due to decreased healthcare utilization resulting from reductions in ADEs. It was estimated that the electronic CPOE would avert 261 ADEs over a 10-year time horizon compared with the standard paper ordering approach. Given the incremental cost of the new CPOE of US$3.95 million, the incremental cost-effectiveness ratio (ICER) was US$15 192 per ADE averted.

The second economic evaluation of a CPOE system developed a decision analytic model to estimate the net benefits of the intervention aimed at reducing medication errors.⁷ Monetary values were assigned to the interventions, efficiency savings, and treatment and health effects of preventable ADEs. In addition, quality-adjusted life year-based monetary values of the preventable ADEs were included in the net benefit analysis. The CPOE had a mean net benefit of US$64.9 million over a 5-year time horizon with the intervention and maintenance costs included in the model. The monetary value of lost health needed to be included in the model for the interventions to have a high probability of producing positive net benefits.

Computerized reminder system

The one hospital-based study measuring the cost-effectiveness of a CDSS reminder system conducted a trial-based economic evaluation. The 1-year, prospective trial evaluated the effect of three reminder systems (ie, computer-generated, telephone, mailed letter) on patient compliance with tetanus vaccination.⁸ The costs of contacting patients were estimated (eg, physician time). The cost per additional vaccination was US$0.64, US$8.16, US$9.11 for the computer reminder system, telephone reminders, and mailed out letter, respectively, versus usual care. The cost of setting up the computerized reminder system was not included in the analysis.

Primary care setting

Computerized reminder system

A trial-based economic evaluation compared the costs and effects of a multifaceted intervention aimed at improving prescribing of antihypertensive and cholesterol-lowering drugs in primary care.⁹ The intervention included: (1) educational outreach visit to clinics; (2) audit and feedback on adherence to guidelines; and (3) computerized reminders to physicians during patient consultations. This intervention was compared with passive dissemination of guidelines through a national medical journal. Over the 1-year study period, the ICER was US$570.25 per additional patient started on thiazides rather than another antihypertensive agent. The reduction in drug expenditures did not outweigh the costs of the intervention; however, if the effect was sustained for a second year, the intervention would be expected to lead to savings.

Computer decision support system

A multicentre, pragmatic randomized study determined the cost-effectiveness of a CDSS designed to promote guidelines for the treatment of asthma.¹⁰ Twenty physicians (198 asthmatic patients) were randomized to a hand-held CDSS that offered therapeutic recommendations based on guidelines or usual care. Effectiveness was determined by measuring the quality of life through the St Georges Respiratory Questionnaire (SGRQ). Costs were calculated from the consumption of resources over the 1-year study period. From a societal perspective, the intervention dominated standard care (ie, less costly and more effective). From the healthcare payer perspective, the ICER was US$66.42 per percentage point reduction in the SGRQ. The cost of the CDSS was not included in the analysis.

Partial economic evaluations

Most of the economic literature reported the results of partial evaluations (26 of 31, 84%). All of these evaluations took the form of cost analyses whereby the costs and effects of the alternatives were examined separately in the analysis.

Hospital setting

Computerized physician order entry

A computerized order set within a CPOE was designed to manage hospitalized pediatric patients with asthma.¹¹ A before–after study evaluated the relationship between computerized order set use and financial outcomes by studying the use of three generally recommended inpatient asthma treatments (ie, systemic corticosteroids, pulse oximetry, and metered-dose inhalers). There was no significant difference in the total inpatient costs among the pre- and post-intervention groups. Hospital charges were US$4381 and US$4616, while the pharmacy charges were US$458 and US$527 in the pre- and post-intervention groups, respectively.

Another pre/post study assessed the introduction of a CPOE system and electronic medication administration record across all inpatient clinical areas.¹² More than 450 evidenced-based order sets (eg, drug interaction), designed to meet the needs of all clinical specialties, were available to facilitate and expedite electronic order entry to support best practice. Severity-adjusted total cost per admission for all services did not change significantly in the health system.

A CPOE introduced to help with the management of surgical patients in an academic, multispecialty hospital had no effect on the rate of medication errors.¹³ However, a redistribution of workload was found leading to personnel changes resulting in a savings of US$485 512. The authors noted that considerable gains in efficiencies will likely result in long-term cost savings and improved quality of care. However, this was an expensive technology to implement (US$3.2 million) and operate (US$2.5 million).

Finally, a randomized controlled trial (RCT) estimated the effects of a CPOE system that displayed various cost information associated with an order with the aim of promoting cost-effective ordering and reduce costs.¹⁴ The total charges per admission were significantly less (US$1534) for the intervention teams than for the control teams, with similar differences in all types of charges. If these effects were extrapolated to all medicine service admissions, the projected savings would be US$5.2 million in charges per year. The network hardware costs were approximately US$33 181 per ward, with additional costs for installation and maintenance (not included in the cost savings calculation).

Computer decision support system

Various modifications to a CDSS aimed at improving the use of and reducing the cost of antibiotics have been reported in four separate studies. The evaluation of the first version of the antibiotic consultant was conducted in an academic, tertiary, private hospital. The computer displayed five antibiotic regimens most likely to be effective for an infected patient and suggested an appropriate antibiotic regimen. The average cost for 24 h of antibiotic therapy recommended by the CDSS was US$16.54 per patient less than what was actually prescribed.¹⁵ The same CDSS, with additional user options incorporated, was evaluated in two studies that took place in a 12-bed shock/trauma/respiratory intensive care unit. A 7-month pilot study revealed a mean reduction in the cost of antibiotics of US$132.70 less per patient compared with the pre-intervention period.¹⁶ Over a 1-year period, the mean cost of antibiotics was US$151 versus US$504 and US$633, while the cost of hospitalization was US$39 017 vs US$52 314 and US$66 522 for the computer regimen followed, regimen overridden, and no CDSS, respectively.¹⁷ Finally, an antibiotic-dose monitor was incorporated into the CDSS to check the renal function of patients to identify those who may be receiving excessive dosages of antibiotics.¹⁸ The intervention group received fewer mean doses of antibiotics at a lower average cost (US$116.11) than patients during the pre-intervention period (US$133.88). If this reduction in cost is summed for all 4483 patients in the intervention group, this would result in a total decrease of more than US$79 210/year. The costs associated with developing and implementing the CDSS were not included.

Another CDSS for antibiotic prescribing was introduced in a 450-bed community teaching hospital.¹⁹ The 5-month study compared patients whose microbiologic data were processed in the usual manner with patients whose data were processed using the software. There was a difference in variable healthcare costs (eg, pharmaceuticals) of US$1988 less per patient in the study group compared with the severity-adjusted control group. Using these adjusted data, the estimated annual cost savings from the intervention was US$3 976 749. If the list price of the CDSS (US$60 357) was subtracted from the expected annual cost savings, the resulting savings (US$3 916 393) would be substantial in the first year.

A third antimicrobial CDSS was evaluated in a 3-month RCT in a 648-bed tertiary care, academic hospital.²⁰ Antimicrobial utilization was managed by an existing management team using the web-based system in the intervention arm and without the system in the control arm. Expenditures for antimicrobial drugs were US$341 891 for the intervention group and US$442 605 for the control group, for a saving of US$100 714 (23%).

An evaluation of another CDSS to support appropriate antibiotic treatment used a cohort study followed by a multicenter (Israel, Germany and Italy), cluster, RCT.²¹ The trial compared hospital wards using the CDSS with antibiotic monitoring without the CDSS. Total antibiotic costs were US$344 lower per patient for the CDSS, a relative decrease of 48%, the difference originating from lower ecological costs (eg, costs associated with loss of antibiotic efficacy) in intervention wards in Israel and Italy. Direct antibiotic costs, as well as costs incurred by observed adverse events, were similar.

An anti-infective decision support tool, designed for a pediatric population, was introduced in a 26-bed intensive care unit in an academic hospital.²² During the pre-CDSS 6-month period, all patient care orders were handwritten. The study found no difference in hospital costs in the pre-CDSS period (US$38 326.63) compared with the post-CDSS period (US$33 951.71) or on anti-infective costs per patient (US$372.70 in the control group versus US$392.79 in the intervention group).

Chertow et al²³ studied the effect of adding a CDSS to an existing CPOE for prescribing drugs to hospitalized patients with renal insufficiency. The authors measured the difference between the intervention and control groups in hospital and pharmacy costs and found that there were no differences (US$6766 vs US$6887 in total costs for the intervention and the control groups, respectively).

CPOE and CDSS

A study conducted in an orthopedic institution assessed the safety and potential cost savings of using a laboratory-based computerized program to manage inpatient warfarin therapy after major joint arthroplasty.²⁴ Consecutive patients having major joint arthroplasty surgery over a 3-year period were compared with a historical cohort undergoing similar procedures in the 18-month period before the program. Financial measures considered were pharmacy and comparative nursing care costs associated with the program. The authors estimated that the potential savings per patient of US$4.62/day were due to a reduction in nursing time (ie, administration time reduced from 15 min per patient to 4 min), for a total annual figure of US$46 910.

The costs associated with the implementation of CPOE with a CDSS over a 10-year period were measured in a 720-adult bed, tertiary care academic hospital.²⁵ Reductions in items such as ADEs, drug costs, and laboratory tests were found, and it was estimated that the system saved the hospital US$35.8 million, even after including the capital and operational costs of US$14.8 million, with cost savings of US$21 million. The authors determined that it took about 6 years for the intervention to be cost saving.

Computerized ADE surveillance system

A computerized ADE surveillance system was used to help identify and prevent specific types of ADEs in hospitalized patients.²⁶ The authors compared the length of stay of patients incurring an ADE with a historical control group of inpatients with no ADEs, and showed that the average length of stay was 20 days for patients with severe ADEs, 13 days for those with moderate ADEs, and 5 days for those with no ADEs. This translated into a cost of US$61 213, US$36 196, and US$10 179 for patients with severe, moderate, and no ADEs, respectively. It is important to acknowledge that the cases were not matched for disease severity and that no direct cost analysis was made of the ADEs prevented by the system compared with before implementation of the system.

A recent publication measured the impact of an ADE alert system on cost and quality outcomes in seven community hospitals within a health network.²⁷ The ADE alerts were triggered in real time, which enabled immediate pharmacy intervention. The results showed a statistically significant decrease in average pharmacy department costs per patient from before to after implementation (US$898 vs US$856, p<0.001). In contrast, the external control group had a significant increase in pharmacy department costs (US$760 vs US$826, p=0.029). If this percentage cost decrease was extrapolated to the control groups' results, this would yield pharmacy department cost savings in excess of US$11.4 million. It was noted that these savings coincided with only modest quality improvements in projected mortality and length of stay. The costs associated with averting ADEs were not measured in this study.

Primary care setting

Computerized physician order entry

Weingart et al²⁸ designed an empirical study to understand the potential benefits of medication safety alerts generated by an e-prescribing system in ambulatory care. Using a modified Delphi technique and data from 1.8 million prescriptions, the authors estimated that e-prescribing alerts possibly averted 133–846 ADEs. An expert panel reviewed a sample of common drug interaction alerts, and estimated the likelihood and severity of ADEs associated with each alert, the likely injury to the patient and the healthcare resource utilization required to address each ADE. The analysis estimated that the cost savings due to the e-prescribing by using third-party-payer and publicly available information was US$451 277 (IQR US$158 054–US$1 134 736).

Computer decision support system

In two separate RCTs, the effect of a CDSS that provided guidelines for the treatment of ischemic heart disease and chronic heart failure²⁹ and patients with asthma or chronic obstructive pulmonary disease were evaluated.³⁰ In both studies, care recommendations were displayed to physicians or pharmacists, or both, and the results were compared with not receiving care recommendations. In the heart disease study, there were no differences between the groups in terms of adherence to guidelines or any clinical or subjective patient outcome, including outpatient, inpatient, or total healthcare costs (physician, US$9,076; pharmacist, US$10 639; physician and pharmacist, US$7639; control, US$10 117). Similarly, the CDSS had no impact on total healthcare costs across groups in the asthma and chronic obstructive pulmonary disease study (physician, US$11 530; pharmacist, US$7681; physician and pharmacist, US$8140; control, US$8353).

McMullin et al published two papers that evaluated the impact of a CDSS that provided evidence-based recommendations during the e-prescribing process on prescription costs for a range of medications used in primary care. A retrospective cohort study using pharmacy claims data found that the average cost for new and refilled prescriptions was US$5.97 lower in the intervention group with 6-month savings of US$4127 per clinician.³¹ A 6-month extension of this study showed 12-month savings on new prescriptions of US$127 152.³²

A cluster-randomized, pragmatic trial assessed the cost and effectiveness of a CDSS based on recommendations of the European Society of Cardiology and other societies for Hypercholesterolemia Management in comparison with usual care for patients with hypercholesterolemia.³³ The CDSS included recommendations on treatment, drugs, and follow-up visits according to the patient's cardiovascular risk and low-density lipoprotein goals. The CDSS did not modify effectiveness, but the treatment costs of hypercholesterolemia were markedly different in the two groups: US$254 484 in the control group and US$149 415 in the intervention group.

E-prescribing with CDSS

Ornstein et al³⁴ measured the effect of displaying cost information in an EMR at the time of prescribing by family physicians. The authors found no effect on overall drug costs (mean cost per prescription was US$31.44 and US$31.73) in the control and intervention periods.

Hospital and primary care settings

Computer decision support system

A claims-driven CDSS system was designed as a ‘rule-based artificial intelligence engine’ combined with an automatic message generator that conveyed clinical recommendations and supporting literature to physicians.³⁵ A 12-month randomized study found that charges among those whose recommendations were communicated were US$90.14 per member per month (pmpm) lower, and paid claims were US$78.77 pmpm lower than controls compared with baseline values. The intervention cost was US$1.16 pmpm and was associated with lower paid claims of US$9.34 pmpm, suggesting an eightfold return on investment from the payer perspective. However, this study was not intended as a formal cost-effectiveness analysis.

An extension of this analysis used an additional 2 years of data.³⁶ This evaluation showed a reduction in average total healthcare charges in the study group by US$31.61 pmpm compared with the control group.

Discussion

Relative to the volume of research evidence that has examined the impact of HIT on the medication management process, the amount of literature evaluating the economic impact of these systems lags far behind.² Compounding the problem is the fact that the quality of the available economic evidence is poor. For example, most of the studies reviewed here would not be considered full economic evaluations.¹ Only 16% of the papers measured the cost per successful patient outcome, and 84% simply provided cost data. Economic evaluation of health services is a comparative analysis of alternative courses of action in terms of both their costs and consequences.¹ The goal is to identify which intervention is most efficient. The main categories of costs of healthcare interventions are the costs associated with the use of resources within the health sector, those used by patients and their families, those used in other sectors, and productivity changes (figure 2).³⁷ The cost may also include downstream costs associated with ongoing treatment, or the management of adverse events or side effects of the treatment. The consequences are the relevant outcomes of interest caused by an intervention (either clinically measured or reported by the patient). The outcomes could be expressed in terms of final health outcomes such as gains in health-related quality of life, or in terms of intermediate health outcomes (eg, mm Hg in hypertension). However, in general, one should choose an effectiveness measure relating to a final outcome.³⁸

Figure 2.

Cost components of economic evaluation in healthcare.

Very few of the studies included the large cost items such as the purchase of new software (capital outlay) or implementation costs (eg, training costs, maintenance costs). Additionally, the settings where HIT programs had already been introduced had existing technology infrastructure (eg, EMRs) to support the new interventions; this may not be the case in many areas. Additionally, whether the technology is commercial or home-grown (eg, academic health center) will have implications for start-up costs and organizational savvy.

The heterogeneity (eg, cost elements) between studies was so great that combining the studies was not possible. This has meant that, while studies have been broadly grouped according to setting and type of intervention, the review has been presented on a study-by-study basis, rather than as a complete synthesis of the results. This makes the interpretation of the results somewhat complicated and commenting on the cost-effectiveness of HIT for medication management difficult. Many of the studies provided evidence of some reductions in costs in certain areas due to the intervention (eg, reduction in drug costs and hospital length of stay). The assumption is that these changes will likely result in long-term cost savings and improved quality of care. These potential cost savings are speculative and are not conclusive.

Despite the limitations of the literature, the great strength of this review lies in the fact that extensive searches were undertaken and included studies that reported any relevant information on the economics of the impact of HIT on medication management. The systematic review searches were updated in July 2010, and should therefore provide a comprehensive up-to-date review of the evidence available. Furthermore, to the best of our knowledge, this is the first systematic review conducted to combine the available research in this area.

Adoption of newer technologies needs to be based on formal evaluation of whether the additional benefit is worth the additional cost. Given the tension between the benefits of HIT for medication management and the high up-front costs, decision-makers deciding whether to implement these technologies need to better understand how and when financial benefits of such systems accrue. These types of analyses are important for well-informed decision-making. In addition, one needs to bear in mind that the effectiveness of any given system is dependent on the system's design, implementation, the user(s) of the system, and the setting into which the system is being introduced. However, because of the focus of our review (ie, systematic review of economic evaluations), we did not conduct a detailed review of implementation issues; this is left for future research.

Conclusions

In summary, the quality of the economic literature in this area is poor. A few of the studies reviewed found that HIT interventions may offer cost advantages despite their acquisition costs. However, given the uncertainty that surrounds the cost and outcomes data, and limited study designs available in the literature, it is difficult to reach any definitive conclusion as to whether the additional costs and benefits represent value for money. Analyses of the consequences of using health technology, both in terms of costs and benefits, is crucial for decisions on resource allocation. We acknowledge that the use of economic methods in this area is relatively immature (74% of the articles were published since 2001), but some of the groundwork has now been carried out for future work in this field.