Use of an automated clinical management system improves outpatient immunosuppressive care following liver transplantation

Abstract

Objective

Immunosuppressive therapy following transplantation, if not managed well, can lead to increased drug toxicity or rejection episodes. We investigated whether use of an automated clinical management system in our liver transplant program would improve clinical outcomes in managing transplant recipients' immunosuppressive medications.

Design

We performed a retrospective cohort study of two patient groups receiving liver transplants at our institution. One group of 301 patients transplanted from January 1, 2004 to November 30, 2006 received outpatient immunosuppressive management using a paper charting system. After instituting an automated clinical management system, the following group of 127 patients transplanted from December 12, 2006 to April 1, 2008 received their outpatient immunosuppressive management with that system. Only patients who received tacrolimus therapy, with or without mycophenolate mofetil or prednisone, were studied.

Measurements

Our endpoints included percentage of patients having rejection and/or tacrolimus toxicity episodes. Various recipient, intraoperative, donor, and postoperative variables, including managing the immunosuppressive therapy with a paper charting system or an automated management system, were studied to determine which factors were associated with our endpoints.

Results

Multivariable logistic regression analysis showed the automated system was significantly associated with fewer rejection episodes and fewer tacrolimus toxicity events. Formal cost-effectiveness analysis of the nurses' salaries for 1 year showed the automated system cost US$197 per patient and the paper system cost US$1703 per patient. The automated system improved quality of life years.

Conclusion

Use of an automated clinical management system for outpatient immunosuppressive management for liver transplant patients has resulted in a decrease in both tacrolimus toxicity and rejection episodes and is cost-effective.

Introduction

Following an aggregate root cause analysis of patient complications and deaths at the University of Washington Medical Center Liver Transplant Program, difficulties with immunosuppressive drug management were identified as a leading cause of patient morbidity, at times leading to subsequent mortality.¹ A subsequent quality review committee revealed that the immunosuppressive management was administered by acceptable clinical guidelines for dosages and blood levels.^2–5 Suspected causal factors included physician and nurse management of multiple patients, varying laboratory schedules, a diversity of immunosuppressive regimens, fragmentation of care, time constraints on medical personnel, the lack of consistently available records noting medication dosages, and a time lag of 1 to 3 days to make needed medication changes. Our quality committee's strategy was to enhance our electronic health record (EHR) system by developing an automated clinical management system within the EHR system to consolidate all the information needed for a comprehensive immunosuppressive medication review that could be completed in minutes, as opposed to hours, and with a physician and nurse located anywhere with computer and internet access.

Not unlike patients found in other chronic disease management programs, liver transplant patients generate large quantities of data that are needed for management of their complex immunosuppressive care. If the proper dosage is not maintained, complications can occur. A dose that is too low could cause allograft rejection, and a dose that is too high could produce drug toxicities. The mainstay of immunosuppressive therapy following liver transplantation is tacrolimus.⁶ The most common drug toxicities secondary to tacrolimus are neurotoxicity, tremors, and renal dysfunction, which has been reported in 10–80% of liver transplant recipients receiving tacrolimus.^7–12

The application of a computerized alert system by another transplant program has resulted in a quicker turnaround time for immunosuppressive medication adjustments following availability of laboratory results, but decreasing immunosuppressive drug toxicity or rejection episodes was not evaluated.¹³ We started our automated clinical management system for immunosuppressive review to allow our team to efficiently manage the care of our liver recipients after transplantation. Before supporting this system for use in our other transplant programs, questions regarding its clinical effectiveness were asked. We performed a retrospective study to determine the clinical effectiveness of our automated clinical management system compared to the paper charting system. We hypothesized that an automated clinical management system for monitoring liver transplant patients on tacrolimus therapy decreases the incidence of tacrolimus toxicity and rejection episodes while also demonstrating cost-effectiveness.

Methods

Study population and groups

After approval from the University of Washington Institutional Review Board, we conducted a retrospective cohort study on patients who received liver transplants from January 1, 2004 to April 1, 2008 who survived to discharge from the transplant admission and thus received outpatient immunosuppressive medications. Those patients initially taking tacrolimus were our study population. Patients were followed for one year. All recipients received immunosuppressive induction therapy with anti-thymocyte globulin (ATG) or basiliximab, depending on surgeon preference and drug availability. Subsequently, patients initially received tacrolimus monotherapy, with an initial target level of 10 mg/dl. The target levels decreased over the year to levels of 5 to 7 mg/dl. Patients with renal dysfunction at the time of their transplant, who need lower levels of tacrolimus, initially received mycophenolate mofetil (MMF) or prednisone or a combination of both added to their immunosuppressive regimen, depending on surgeon preference.

The study involves a before/after design, which included one group of patients before implementation of the automated clinical management system and another after implementation of the system. One group received immunosuppressive management using the paper charting system transplanted from January 1, 2004 to November 30, 2006. Another group, transplanted from December 1, 2006 to April 1, 2008, was tracked with the automated clinical management system.

Endpoints

The primary goal of our study was to determine significant recipient, intraoperative, donor, and postoperative factors associated with clinically important endpoints. The primary endpoints chosen were rejection episodes and tacrolimus toxicity episodes. We posit that if levels of tacrolimus were allowed to remain elevated for longer periods of time, that those patients would experience more side effects of tacrolimus. Likewise, if levels of tacrolimus were allowed to remain inadequate for longer periods of time, then more of those patients could experience rejection episodes. The mortality rate and readmission rate per patient following liver transplantation were also followed for secondary endpoints.

The endpoints were determined within the first year following transplantation. Rejection episodes required treatment for clinical rejection and were confirmed by biopsy. A patient was counted as having rejection if one or multiple rejection episodes occurred during the year. Tacrolimus toxicity events included seizures, tremors, mental confusion, or severe acute renal dysfunction that resolved upon discontinuation of tacrolimus. A patient was determined to have a tacrolimus toxicity if one or multiple episodes occurred during the year.

Additional secondary endpoints were the number of deaths and the number of readmissions to the transplant hospital for each patient, starting at the time of discharge from the transplant admission and continuing for 1 year following transplantation.

Factors for analysis

Baseline liver recipient data collected for our review included age, gender, race, use of interpreter, primary liver disease diagnosis including re-transplantation, receipt of exception model for end-stage liver disease points for hepatocellular carcinoma, cerebral or cardiac and vascular disease (as documented by angiograms), renal disease (as documented by creatinine clearance ≤60 ml/min or evaluation by a nephrologist for renal dysfunction prior to liver transplantation), diabetes mellitus (denoted by requirement for insulin therapy), hypertension (denoted by requirement for antihypertensive therapy), body mass index, Status 1 designation (in the intensive care unit and expected to live ≤7 days), presence of a transjugular intrahepatic portosystemic shunt, requirement for dialysis immediately prior to liver transplantation, and date of transplant. Pre-transplant laboratory data of serum creatinine, total bilirubin, and cholesterol levels were obtained.

Intraoperative and donor data obtained included blood type match between donor and recipient (identical, compatible, or incompatible), split liver versus whole liver allograft, liver transplant alone or with simultaneous kidney transplant, cold ischemia time (the time duration from placement of the liver on ice following the liver's removal from the donor to removal of the liver from ice prior to transplantation) and warm ischemia time (the time duration from removal of the liver from ice prior to transplantation to circulation of blood into the transplanted liver), and the amount of packed red blood cells (PRBC) transfused during the transplant procedure. Donor liver information collected included donor type (donation after cardiac death or donation after brain death), age, gender, race, body mass index, and percent fat in the donor liver.

Post-transplantation data included immunosuppressive induction therapy (ATG or basiliximab) and choice of maintenance immunosuppressive therapy in an intention to treat analysis (tacrolimus with or without MMF and/or prednisone). The method of post-transplant immunosuppressive management (paper charting system or automated management system) was recorded for each patient. Those patients transplanted from January 1, 2004 to November 30, 2006 were followed by the paper charting system. The pre-existing process of using the paper charting system for managing immunosuppressive therapy consisted of the following steps:

1. determining when laboratory results would be available;

2. transcribing the results into a paper spreadsheet in the recipient satellite transplant record;

3. batching the results for several patients;

4. finding a physician to review the results and write orders;

5. calling the liver transplant recipient to inform the patient to make necessary changes.

Several steps in this process could be problematic, including a delay in knowing when laboratory results were ready for review, not finding the satellite chart, and difficulty in finding a physician to review the laboratory results and prescribe medication changes.

Automated clinical management system

Patients transplanted from December 1, 2006 to April 1, 2008 were followed with the automated system. This system consists of three computer screens in the EHR system that consolidates all clinical information to expedite immunosuppressive review. The EHR system resides on a secure server with access via the internet 24 hours per day. When laboratory results are available, the patient's name is automatically added to an Immuno Daily List. This list includes not only the patient's name but also that patient's transplant coordinator's name and which physician is to review the results. By selecting the patient's name on the Immuno Daily List screen, the Immuno MD Review screen appears that includes the following fields: patient name, date of transplant, age, diagnosis, cytomegalovirus status of donor and recipient, pathology report for last biopsy, date of last rejection episode, any protocols applicable to that patient, comments for target goals for immunosuppressive medications, current immunosuppressive therapy, and all laboratory results with immunosuppressive drug levels. The physician can review this screen and type his/her orders, and the orders appear on the Immuno Daily List. The transplant coordinator reviews the orders, calls the recipient, and notes any changes. All orders and the specific nurse and physician making the medications changes are recorded automatically and authenticated. All dosages are recorded in a master list that appears on a third screen, Immuno Medications.

Cost-effectiveness analysis

Cost predictions for the formal cost-effectiveness analysis were determined through interviews with the transplant coordinators and administrative staff regarding the paper and automated systems. The average time for the nursing staff to collect the data, find a physician and present the results, and contact the patient were estimated for the paper system. The average time required in using the automated system was determined by following several patients. The probabilities for the various clinical events were determined from the two study groups. The quality of life years were determined by consensus.

The cost for developing the automated clinical system was determined by the programming costs. The cost of a transplant coordinator's salary was obtained by converting the average salary and benefits from a yearly salary to an hourly salary assuming a 40-hour work week. All costs were standardized for the year 2008.

Statistical analysis

Continuous variables were given as the mean±SD, and categorical variables were presented as percentages. After checking for normal distributions, the Student's t test was used for testing continuous variables, and the Fisher's Exact test was used for categorical variables. An autoregressive integrated moving average (ARIMA) analysis was used to determine if a change in endpoints occurred over time. Logistic regression was used to determine univariable and multivariable factors associated with the endpoints. To avoid overfitting in both univariable and multivariable logistic regression modeling, clinical reasoning was used to choose the clinical variables best associated with the endpoint. To determine how well each model fit the clinical endpoints, receiver operating characteristic (ROC) curves were developed to determine area under the curve (AUC) values for each multivariable logistic regression model. The statistical software package used was JMP V.7.0.2 (SAS Institute, Inc). p Values< 0.05 were considered significant. TreeAge Suite Pro Healthcare V.1.4.1 (TreeAge Software, Inc) was used to create a decision tree model to determine cost-effectiveness between the new automated clinical management system and the standard paper charting system.

Results

Study population

From January 1, 2004 to April 1, 2008, 512 patients underwent liver transplantation at the University of Washington Medical Center. Of these, 428 patients were discharged following their transplant admission to start on immunosuppressive management and were initially treated with tacrolimus for immunosuppressive therapy; these patients became our study population. The excluded patients were either given another immunosuppressive medication regimen (usually cyclosporine) by physician discretion or did not survive to discharge. No deaths were attributed to rejection episodes.

Study groups

The study population was divided into two groups depending on whether their immunosuppressive management was maintained using the paper charting system or the automated clinical management system after its implementation. Of the 428 patients, 301 were followed using the paper charting system and were transplanted between January 1, 2004 until November 30, 2006, and 127 patients, transplanted from December 1, 2006 to April 1, 2008, were followed using the automated clinical management system (table 1).

Table 1.

Comparison of recipient, intraoperative, donor, and post-transplant factors of patients with immunosuppressive management by the paper charting system versus the automated clinical management system

Factors	Paper charting system N=301	Automated system N=127
Recipient	Mean±SD (%)	Mean±SD (%)	p††
Age, years	53.5±8.5	52.5±8.9	0.3
Male	73%	72%	0.8
White	87%	89.%	0.7
African-American	2%	2%	0.7
Asian	9%	5%	0.2
Other races	2%	4%	0.9
Patient needs interpreter	7%	4%	0.2
Re-transplantation	3%	6%	0.3
Hepatitis B virus infection	4%	6%	0.7
Hepatitis C virus infection	56%	50%	0.2
Alcoholic liver disease	8%	12%	0.3
Cryptogenic cirrhosis	5%	6%	0.8
Acute hepatic necrosis	2%	0%	0.2
Cholestatic liver disease	11%	10%	0.6
Other liver disease diagnosis	10%	12%	0.5
HCC with MELD exception	19%	2%	<0.001
Cerebral/cardiovascular disease	4%	1%	0.07
Renal disease	7%	6%	0.8
Diabetes*	21%	21%	0.9
Hypertension	19%	21%	0.7
BMI	29±5.3	29±4.9	0.7
Status 1	0%	2%	0.6
TIPS†	7%	2%	0.1
On dialysis time of transplant	2%	2%	0.7
Pre-transplant serum creatinine, mg/dl	1.3±1.1	1.4±1.5	0.5
Pre-transplant total bilirubin, mg/dl	5±7.2	5.9±9.9	0.4
Pre-transplant cholesterol, mg/dl	141.6±54.1	144.4±61.7	0.7
Intraoperative and donor
Identical blood type	94%	94%	1
Compatible blood type	5%	5%	1
Incompatible blood type	1%	2%	0.6
Split liver	1%	0%	0.6
Simultaneous kidney transplant	4%	3%	0.8
Cold ischemia time, min‡	428.8±164.9	493±149	0.001
Warm ischemia time, min§	35.8±7.9	42±10	<0.001
Units PRBC transfused (250 ml)	3.3±2.9	3.8±3.2	0.2
Donation after cardiac death donor	13%	7%	0.09
Donor age, years	39.8±16.5	36.4±14.2	0.04
Donor male	64%	69%	0.4
Donor race white	82%	74%	0.06
Donor race African-American	3%	2%	0.5
Donor race Asian	6%	2%	0.09
Donor race other	8%	22%	<0.001
Donor BMI	26.6±6.1	27.3±5.1	0.2
Greater 20% fat in liver biopsy	3%	1%	0.3
Post-transplant
Induction with ATG¶	24%	80%	<0.001
MMF**	28%	554%	<0.001
Prednisone	33%	18%	0.001

^*Diabetes mellitus requiring insulin.

^†Transjugular intrahepatic portosystemic shunt.

^‡Time from placing donor liver on ice during the procurement until removal from ice at time of transplant.

^§Time from removal of liver from ice at time of transplant until recirculation of blood to liver.

^¶Anti-thymocyte globulin.

^**Mycophenolate mofetil.

^††Student's t-test was used for testing continuous variables, and the Fisher's Exact test was used for categorical variables.

ATG, anti-thymocyte globulin; BMI, body mass index; HCC, hepatocellular carcinoma; MELD, model for end-stage liver disease; MMF, mycophenolate mofetil; PRBC packed red blood cells; TIPS, transjugular intrahepatic portosystemic shunt.

Endpoints

Rejection episodes occurred in 17% (74) of the patients (66 (22%) of those with the paper charting system and 8 (6%) of the automated system (p<0.01)). Tacrolimus toxicity occurred in 27% (117) of the patients (94 (31%) of those treated with the paper charting system and 23 (18% with the automated system (p<0.01)). Death occurred in 2.6% (11) of the patients (9 (3%) of those treated with the paper charting system and 2 (1.6%) with the automated system (p=0.5). The readmission rate was 1.3±1.6 times per patient (1.4±1.7 for those treated with the paper charting system and 1.2±1.3 with the automated system (p=0.08)).

Time series analysis

An ARIMA analysis revealed that a significant decrease in endpoints (rejection episodes and tacrolimus toxicity episodes) occurred following institution of the automated system when compared to the time before institution of the automated system (table 2).

Table 2.

ARIMA analysis comparing best model for the paper charting system and the automated system for endpoints of tacrolimus toxicity episodes and rejection episodes

End-Points	Model*	Term	Lag	Estimate	SE	T ratio	p Value
Tacrolimus toxicity episodes
Paper system	(0,0,0)	Intercept	0	3.1	0.25	12.8	<0.01
Automated system	(0,1,1)	Intercept	0	−0.19	0.04	−4.9	<0.01
		MA1	1	1	0.17	6	<0.01
Rejection episodes
Paper system	(0,0,0)	Intercept	0	2	0.3	7.4	<0.01
Automated system	(0,1,2)	Intercept	0	−0.1	0.02	−4.6	<0.01
		MA1	1	1.6	0.49	3.3	<0.01
		MA2	2	−0.6	0.45	−1.4	0.2

^*Best model as determined by Akaike's Information Criterion and Schwarz's Bayesian Criterion rankings.

Logistic regression analysis

Patients with rejection episodes

Ten factors were chosen by clinical reasoning to be possibly associated with patients developing rejection episodes and were used for univariable analysis (table 3). In a multivariable analysis with these 10 factors, the patient care with the automated clinical management system produced lower odds of a rejection episode than management with the paper charting system (OR 0.20; p<0.01) (table 2). Patients initially treated with MMF had a lower chance of rejection, while patients initially treated with prednisone had an increased chance of having a rejection episode. These three factors—automated system, initial MMF, and initial prednisone—had an AUC of 0.73 in a ROC analysis.

Table 3.

Univariable and multivariable factors in logistic regression predicting patients with rejection episodes

	Univariable		Multivariable
Factors	OR	p Value	OR	p Value
Use of automated clinical system	0.2	<0.01	0.3	<0.01
Recipient age	0.9	0.8
Recipient male gender	0.6	0.1
Diabetes	1.2	0.7
Induction with ATG	1.1	0.8
MMF	0.4	<0.01	0.5	0.02
Prednisone	1.9	0.02	1.8	0.03
Patient needs interpreter	0.9	0.9
Incompatible blood type	1.6	0.7
Simultaneous kidney transplant	0.9	0.9

ATG, anti-thymocyte globulin; MMF mycophenolate mofetil.

Patients with tacrolimus toxicity episodes

Eleven factors were chosen by clinical reasoning to be possibly associated with patients developing tacrolimus toxicity episodes and were used for univariable analysis (table 4). Multivariable analysis using these 11 factors showed that patients managed with the automated clinical review system had lower odds of developing tacrolimus toxicity (OR 0.5, p<0.01) (table 4). Patients receiving donor livers with >20% fat had 14.3 times (p<0.01) the odds for developing tacrolimus toxicity, and for every unit of PRBC transfused, the odds for developing tacrolimus toxicity increased 1.13 (p<0.01). These three significant multivariable factors had an AUC of 0.70 in ROC analysis.

Table 4.

Univariable and multivariable factors in logistic regression predicting patients with tacrolimus toxicity episodes

	Univariable		Multivariable
Factors	OR	p Value	OR	p Value
Use automated clinical system	0.5	<0.01	0.5	<0.01
Recipient age	1.1	0.4
Recipient male gender	1	1
Patient need interpreter	1.1	0.8
Cerebral/cardiovascular disease	1.5	0.5
Log of serum creatinine, mg/dl	1.4	0.1
Units PRBC transfused (250 ml)	1.1	<0.01	1.13	<0.01
Induction with ATG	1.8	<0.01
MMF	0.8	0.2
Prednisone	0.7	0.2
Greater than 20% fat in donor liver biopsy	9.8	<0.01	14.3	<0.01

ATG, anti-thymocyte globulin; MMF, mycophenolate mofetil; PRBC, packed red blood cells.

Secondary endpoints

Due to few clinical events, we did not perform logistic regression analysis on deaths or the number of hospital readmissions per patient in the year following liver transplantation because this would have resulted in over-fitting of the model.

Cost-effectiveness analysis

Three events, including incidence of endpoint occurrence, nursing cost (salary plus benefits) to follow one patient, and quality life years were used in the formal cost-effectiveness analysis for following recipients with the paper charting system compared with the automated clinical management system for one year (table 5). Costs in the model were determined from the perspective of the salary a hospital or clinic would pay a transplant nurse coordinator to use the paper charting system compared with the automated clinical management system to follow one patient for immunosuppressive management for 1 year.

Table 5.

Events, including probabilities for endpoints occurring, nursing cost for following a patient one year for immunosuppressive management, and quality of life-years, in a cost-effectiveness analysis

Event in cost-effectiveness analysis	No endpoint	Endpoints rejection episode	Tacrolimus toxicity
Incidence
Paper system*		0.22 (0.17 to 0.27)†	0.31 (0.26 to 0.37)†
Automated clinical review system*		0.06 (0.03 to 0.12)†	0.18 (0.12 to 0.26)†
Cost
Nursing cost to follow a patient for a year‡
Paper system	$1556 (range 777 to 5055)
Automated clinical review system	$189 (range 92 to 972)
Nursing cost per event§		$142.95 (74.33 to 337.36)	$68.61 (34.31 to 171.54)
Quality of life year	1	(−0.3 (range −0.5 to −0.1))	(−0.2 (−0.3 to −0.1))

^*Probabilities determined directly for the study populations.

^†Probabilities (95% CI).

^‡Determined by using the average nurse's salary with benefits of US$57.18/h × 17 average medication reviews/year × h per review (0.2 h(0.1–1) for automated system and 1.6 h (0.8–5.2) for paper system).

^§The extra nurse's salary required to coordinate management of event.

The formal cost-effectiveness analysis revealed that the automated system cost was $197 (range $189 to $470; all costs presented in US$) to monitor one patient for 1 year with 0.96 quality life year, compared to the paper charting system cost of $1703 (range $1556 to $2350) with 0.86 quality life year. Thus, the cost savings of the automated clinic management system saved on average $1506 ($1703 minus $197) per patient per year. The cost-effectiveness analysis revealed that the automated system was more cost-effective (absolute dominance) than the paper charting system (data not shown). One-way sensitivity analysis for a nurse's salary to use the paper charting system, probability of rejection or tacrolimus toxicity episode with the paper charting system, and quality life years for a rejection or tacrolimus toxicity episode revealed within any range of selections that the paper charting system was not comparable to the automated clinical review system (data not shown). Additionally, two-way sensitivity analysis showed that no combination of ranges allowed the paper charting system to become comparable to the automated clinical review system (data not shown). Finally, a sensitivity analysis to evaluate the net health benefit on willingness to pay showed that at no time did the paper charting system become comparable to the automated clinical review system (figure 1).

Figure 1.

In a sensitivity analysis evaluating the net health benefit on willingness to pay (US$1000 to $150 000), the paper charting system never became comparable to the automated clinical management system.

The programming cost of the automated clinical review system was determined to take 1 to 2 months, or between $9 148.80 to $18 297.60, in salary and benefits. Since our transplant program already had an EHR system and other computer equipment such as servers, all the extra expense was in the form of development by the programmer and oversight while implementing the new addition. With the cost savings of $1506 per patient's review per year, the automated system cost of development was recovered in following 6 to 12 patients for 1 year. Our total projected one-year cost savings is $150 600 (range $136 700 to $188 000) (table 6). With the average salary and benefits of a transplant nurse coordinator being $118 934 per year, the automated system saves between 1.1 to 1.6 nursing equivalents per year after the initial year.

Table 6.

Nursing cost (salary plus benefits) for a year of using a paper charting system compared to an automated clinical management system for immunosuppressive management

	Paper charting system	Automated clinical management system
Nursing cost/patient/year*	$1703 (1556 to 2350)	$197 (189 to 470)
Patients/year†	100	100
Total yearly nursing cost	$170 300 (155 600 to 235 000)	$19 700 (18 900 to 47 000)
Yearly savings on nursing cost		$150 600 (136 700 to 188 000)

^*From cost-effectiveness analysis.

^†Average projected transplanted patients to follow/year.

Discussion

Benefits of automated clinical management system

To improve care to our liver transplant patients, we chose to develop an automated clinical management system in our EHR system that collected all necessary information to manage immunosuppressive care. The potential usefulness of an automated clinical management system to improve care with immunosuppressive management was not known. Kern et al¹⁴ concluded that just electronic laboratory viewing in a non-transplant ambulatory setting improved care quality but cautioned that other studies were needed. Staes et al¹³ had shown that a computerized alert system improved efficiency in providing immunosuppressive care but did not evaluate effectiveness of that care. However, others had shown that different categories of computer-based decision support systems integrated into EHR systems improved quality when compared to usual care supported by the paper medical record.^15–18 We believe we have shown that the use of an automated clinical management system is associated with improved immunosuppressive care following transplantation in an out-patient setting and is cost effective.

Automated clinical management system associated with endpoints

The percent of patients developing endpoints in our study decreased significantly after institution of the clinical automated management system. Patients who were followed by the automated system experienced an 80% lower chance of developing rejection episodes than those patients managed with the paper charting system. The odds of developing tacrolimus toxicity episodes were 48% lower in patients tracked with the automated system. This decrease was sustained for the entire follow-up period. Also, the factor of the automated clinical management system was independently associated with lower odds of developing endpoints in our study when evaluated with several other clinical factors. Both multivariable logistic models, each including the factor of automated system, predicted each endpoint of patients developing rejection episodes, and tacrolimus toxicity with an AUC of 0.73 and 0.70, respectively, as determined by ROC analysis. These values indicate the moderate clinical significance of these factors in prediction models.

Reduction of clinician response time in immunosuppressive management

Naturally, our retrospective study does not develop causality between the automated clinical management system and the improved outcomes. We posited that if immunosuppressive levels are acutely too low or too high and are allowed to persist even for a couple of days, the endpoints of rejection or tacrolimus toxicity episodes are more likely to occur. Computerized alerts of outpatient laboratory results were shown to reduce clinicians' response time for immunosuppressive management from 33 hours down to 9 hours.¹³ In a work-flow evaluation, our nurses determined that our response time with the automated clinical management system improved up to 3 days in several cases. Thus, the use of the automated system improved our response time for adjusting immunosuppressive medications from 1–3 days to only a few hours. This improved response time to adjust abnormal immunosuppressive levels appears to be a viable explanation for the association of our automated system with the decreasing odds of developing the endpoints in our study. Thus, the use of an automated system for immunosuppressive management likely improves patient care.

Study limitations

As with any time series or before/after observational study, we must point out potential difficulties with our conclusion regarding the clinical effectiveness of our automated clinical management system. All patients were formerly tracked using the paper charting system; we switched to the automated system to follow outpatient immunosuppressive management for patients transplanted starting on December 1, 2006. In direct comparison between the patients being followed by the paper charting system and the patients followed by the automated system, several factors were different between the two groups. The biological significance of any one of these factors relating to the endpoints is not directly known. These factors were controlled for by using multivariable analysis. It is possible that there were other confounding variables that we did not measure which might have also been responsible for the improved care in the later period. Also, this was during the time following our root cause analysis when we were focusing on our immunosuppressive management; the Hawthorne effect could have been a part of the improvement around the time of committee improvement work and institution of the automated system,¹⁹ but unlikely to persist for long-term follow-up. However, it is almost impossible to conduct a randomized controlled trial in this situation. Our automated clinical management system was instituted as an efficiency management tool and a possible risk-reduction strategy. The markedly improved efficiency of our automated system was evident immediately from the start. Treating some patients with a very efficient system, while randomizing others to an inefficient system after incurring the cost of developing an efficient system, precluded randomized controlled trials from being conducted.

Any formal cost-effectiveness analysis requires discussion on whether accurate probabilities, quality life years, and costs were used in the model. An advantage to our clinical study was the use of the clinical probabilities obtained directly from our study population. The number of quality life years was a best guess estimate, and the cost estimates were obtained from a discussion with the transplant coordinators and administrator. To overcome concerns with the quality life years and cost estimates, both one-way and two-way sensitivity analyses were conducted. There were no estimates or combinations of estimates in the reasonable preset ranges that allowed the paper charting system to equal the performance of the automated clinical management system. Wider, unrealistic estimates would be required to allow the two systems to become equal in cost-effectiveness analysis.

Use in chronic care management

Chronic immunosuppressive management for complex liver transplant patients is similar to chronic care management. Chronic care management of patients with diabetes mellitus or congestive heart failure requires numerous laboratory tests or other medical tests.^20–24 These values or tests should be immediately available for all caregivers, allowing any required medical care to be rapidly conducted. Also, each patient is unique and has special considerations that must be considered at the time of rendering medical care. Finally, the tracking of all medical interventions must be recorded. An automated system such as ours incorporates all these requirements. Hopefully, lessons learned from our automated system can be useful to the wider discipline of chronic medical care management.

Conclusion

As noted, the efficiency of our automated system was apparent from the start, and after demonstrating clinical effectiveness, formal cost-effectiveness and health-benefit analyses could then be conducted. Using the significant clinical events, rejection episodes and tacrolimus toxicity episodes, and from the perspective of salary and benefits paid to a nurse coordinator, the cost of the automated clinical management system was shown to be much lower than that of the paper system. Additionally, the quality life years improved with use of the automated system. The automated clinical management system demonstrated absolute dominance (both more clinically effective and less costly) over the paper charting system. Also, in a willingness-to-pay analysis, at no time was the paper charting system equal to the automated clinical management system. In other words, hiring several nurses could never equal the net health benefit obtained by use of the automated system unless drastic process improvement was achieved; however, our process for immunosuppressive management was already based on evidence-based medicine.^{2–5 25}

Our study demonstrates that an EHR system for immunosuppressive management can be efficient and clinically beneficial and cost-effective. Our prior process of clinical care was sound, but execution was difficult and delayed. We suspect that in many clinical situations, use of a paper chart and good clinical processes give excellent medical care, and management tools within the EHR systems will not improve the outcomes in these situations. However, when improved timing to deliver care leads to better care, an automated clinical management system can lead to cost-effective care and improve the quality of life for the patient.