Assessment of Readiness for Clinical Decision Support to Aid Laboratory Monitoring of Immunosuppressive Care at U.S. Liver Transplant Centers

Summary

Background

Following liver transplantation, patients require lifelong immunosuppressive care and monitoring. Computerized clinical decision support (CDS) has been shown to improve post-transplant immunosuppressive care processes and outcomes. The readiness of transplant information systems to implement computerized CDS to support post-transplant care is unknown.

Objectives

a) Describe the current clinical information system functionality and manual and automated processes for laboratory monitoring of immunosuppressive care, b) describe the use of guidelines that may be used to produce computable logic and the use of computerized alerts to support guideline adherence, and c) explore barriers to implementation of CDS in U.S. liver transplant centers.

Methods

We developed a web-based survey using cognitive interviewing techniques. We surveyed 119 U.S. transplant programs that performed at least five liver transplantations per year during 2010–2012. Responses were summarized using descriptive analyses; barriers were identified using qualitative methods.

Results

Respondents from 80 programs (67% response rate) completed the survey. While 98% of programs reported having an electronic health record (EHR), all programs used paper-based manual processes to receive or track immunosuppressive laboratory results. Most programs (85%) reported that 30% or more of their patients used external laboratories for routine testing. Few programs (19%) received most external laboratory results as discrete data via electronic interfaces while most (80%) manually entered laboratory results into the EHR; less than half (42%) could integrate internal and external laboratory results. Nearly all programs had guidelines regarding pre-specified target ranges (92%) or testing schedules (97%) for managing immunosuppressive care. Few programs used computerized alerting to notify transplant coordinators of out-of-range (27%) or overdue laboratory results (20%).

Conclusions

Use of EHRs is common, yet all liver transplant programs were largely dependent on manual paper-based processes to monitor immunosuppression for post-liver transplant patients. Similar immunosuppression guidelines provide opportunities for sharing CDS once integrated laboratory data are available.

1. Background

Liver transplantation has been a life-saving procedure for over 125,000 persons in the United States since 1988 [1, 2]. Persons that undergo liver transplantation require lifelong immunosuppressive care and laboratory monitoring to detect and prevent organ rejection, toxicity, and death following this costly [3] and complex procedure. The demand for post-transplant immunosuppressive care is increasing because each year more patients are being transplanted and living longer [2]. This poses a growing challenge for care providers at U.S. transplant centers that must continue to monitor and report on previously transplanted patients, even after patients move or use laboratories external to the healthcare enterprise [4]. Effective post-transplant immunosuppressive care requires careful laboratory monitoring because immunosuppressive medication regimens can be complicated, cause side effects, and have narrow therapeutic indices [5]. For example, blood Tacrolimus levels above a therapeutic threshold put patients at risk for toxicity or infection; levels below a threshold put patients at risk for organ rejection [5]. Traditionally, this monitoring has been performed using paper-based flow sheets but two centers have published about their efforts to address this workflow [6, 7]. Laboratory results are often received on paper by fax or mail then transcribed chronologically onto a paper flow sheet along with medication dosages, vital signs, and other patient information. Later, transplant coordinators and physicians review the paper flow sheet to identify values and trends that may indicate complications. The paper flow sheet provides a summary of patient information needed by clinicians for monitoring and clinical decision making. Paper records, however, have inherent shortcomings and are not well suited for health record management. Paper records are prone to transcription errors, are time-consuming to maintain and difficult to reproduce if lost, and are not always accessible given that they can only be in one place at one time. Experts believe that electronic health records (EHR) will help to overcome shortcomings inherent in managing paper-based healthcare information [8, 9]. While EHRs are increasingly being adopted [10, 11], partly motivated by Meaningful Use regulations, EHRs may merely coexist with paper-based systems, adding inefficiencies and cost.

Adoption of EHR technology provides an infrastructure for computerized clinical decision support (CDS)[12,13] to improve drug management for patients with chronic conditions such as diabetes, coagulation care, and kidney disease [14–17]. Computerized CDS can improve drug monitoring for solid organ transplant patients as well, although a recent case report concerning the immediate post-transplant management of a heart transplant patient highlights the need for healthcare enterprises to review thresholds used to deliver alerts and potentially customize vendor-supplied CDS [18]. Concerning use of CDS for long-term outpatient post-transplant immunosuppressive care, we identified only two studies in the literature [6, 7]. Researchers at the University of Washington demonstrated that the availability of patient information summaries when reviewing laboratory data can improve outcomes and cost [7] while researchers with Intermountain Healthcare found that computerized alerts based on program-specific guidelines can improve workflow processes and the quality of laboratory data used for clinical decisions [6]. At Intermountain Healthcare, an analysis of transplant workflow and information management gaps [19] led to the development of a system to address two critical workflow challenges: a) standardized data entry of external laboratory results to integrate laboratory results into the EHR from a continually changing set of external laboratories [20]; and b) CDS alerts to identify patients with laboratory results (internal or external) that are new, out-of-range, or overdue based on time since transplantation [6]. This system led to significant improvements in the completeness, timeliness, and lack of redundancy of laboratory reporting. The liver transplant team at Intermountain Healthcare has continued (as of 2014) to use the data entry system and CDS alerts developed in 2004 to manage the growing population of liver transplant patients.

While other U.S. transplant programs could yield similar benefits for managing a high-risk population of solid organ recipients, little is known about the capabilities of clinical information systems used by transplant programs across the United States. We identified no literature describing information management needs and challenges for laboratory monitoring of immunosuppressive care, or the prevalence of prerequisites (such as an EHR infrastructure, availability of discrete patient data, and computable logic) for implementing the types of computerized CDS mentioned above.

2. Objectives

We sought: a) to describe the current clinical information system functionality and manual and automated processes for laboratory monitoring of immunosuppressive care, b) to describe the use of guidelines that may be used to produce computable logic and the use of computerized alerts to support guideline adherence, and c) to explore barriers to implementation of CDS in U.S. liver transplant centers. The results will inform development of computerized CDS tools to improve outpatient post-transplant laboratory monitoring for immunosuppressive care.

3. Methods

3.1 Study population

The study population included liver transplant programs registered with the Organ Procurement Transplantation Network (OPTN) as of August 2013 [1]. We used publicly available data from the OPTN website to determine the number of transplants performed, the number of consecutive years performing liver transplants, the population served (adult, pediatric, or both), and the geographic region served for each liver transplant program in the United States. We excluded 45 programs that performed fewer than five liver transplant procedures in 2010, 2011, or 2012.

3.2 Survey development

We developed survey questions to describe the transplant program, approaches to paper-based and electronic information management, EHR capabilities, laboratory results monitoring guidelines, and availability of CDS. These topics were derived from the protocols used and functionality required to implement alerts at Intermountain Healthcare to improve information management processes for outpatient post-liver transplant care [6]. To gather evidence of the availability of discrete laboratory data in the EHR for CDS, we queried about the ability to graph information. The ability to graph all internal and external laboratory results is an indication that discrete laboratory data are available for alerting.

The survey instrument was iteratively refined through multiple rounds of informal pilot testing with guidance from experts in survey development and transplant care. A formal pilot study of the survey was performed with three transplant programs. Cognitive interviewing techniques [21] were used to evaluate sources of response error and led to additional refinements. These iterative refinements improved construct validity: respondents understood the questions being asked and were able to match their response to the options provided. Study data were collected using REDCap [22], a browser-based electronic data capture tool for clinical and translational research. The survey, which included logic to skip irrelevant questions, was pilot tested to ensure the correct question flow. Each respondent encountered up to 50 questions that required 10–15 minutes to complete.

3.3 Survey administration

Each liver transplant program was contacted using information from the program website. A script was used to request the participation of the clinical or operations manager in charge of the nurse transplant coordinators. When a representative was unavailable, a message was left on voicemail or with a receptionist. Potential participants were informed that the survey was online, typically took 10–15 minutes, did not involve any protected health information, and that results would be anonymized for publication. At least three calls were made to contact the targeted representative.

Once a representative was identified, an email was sent that included the purpose of the study, a consent cover letter, and an organization-specific link to the survey. Participants were encouraged to complete the survey with assistance from colleagues with a working knowledge of the clinical and information management processes used by the program. Survey administration was performed September 13, 2013 through December 31, 2013.

If surveys were not completed within two weeks, we attempted to contact the non-respondents by email or phone at least three times. We also contacted participants who partially completed the survey or whose response required clarification.

3.4 Data analysis

We performed a descriptive analysis after combining survey data with publically available data from OPTN. To assess the representativeness of the sample, we compared non-respondents to respondents using the number of transplants performed, the number of consecutive years performing transplants, population served, and geographic regional representation [1]. According to OPTN, programs performed a median of 118 liver transplantations during the three years from 2010 to 2012. For convenience, large and small transplant programs were defined as those that performed 100 or more liver transplantations and less than 100 liver transplantations, respectively, during the three years from 2010 to 2012. For graphing capabilities, we defined ‘usage’ as the proportion of those who used the capability among those for whom it was available, and ‘desire’ as the proportion of those who wanted the capability among those who did not have it. We stratified responses by program size (large, small) and population served (adult, pediatric, both). When there was a significant difference (p<0.05) in responses after stratifying by these features, we reported the p-value. We used the Wilcoxon rank-sum test for continuous variables and used the Chi-squared test or Fisher’s exact test (when more than 20% of the cells had an expected frequency less than 5) for categorical variables. Analyses were performed using R statistical software [23].

We performed a qualitative analysis of the narrative responses to a question about barriers to CDS implementation. A card sorting technique was used to organize the short narrative statements into higher-level generalizable categories [24, 25]. This technique is used to create categories based on implicit rules – a sort of folksonomy. The narrative responses were split into individual narrative phrases reflecting a single statement and printed onto individual cards. Researchers were asked to take the stack of “cards” and sort them into a set of categories that reflect their own implicit mental organization. No rules were provided as to the number of categories or the specific type of category to be generated. For this study, three of the authors completed the card sort independently followed by an iterative process of discussion and category identification through consensus. Common themes regarding barriers to CDS implementation were identified.

4. Results

4.1 Description of liver transplant programs responding to the survey

From the OPTN website, we identified 119 liver transplant programs in the United States that conducted at least five liver transplantations each year during 2010–2012. A total of 80 (67% response rate) surveys were completed by one or more transplant care team members. The remaining programs did not respond (n=35) or submitted incomplete surveys with less than 50% of available questions answered (n=4). Among the 80 completed surveys, the response rate for each question ranged from 90 to 100% (96% average).

Responding programs performed a median of 108 (range: 25–429) transplantation procedures during 2010–2012, had consecutively performed transplantations for a median of 24 years (range: 5–26), served adult (n=45; 58%), pediatric (n=12; 15%) or both adult and pediatric (n=21; 27%) populations, and were geographically distributed throughout the 11 regions of the U.S. (▶Table 1). There were no significant differences in the characteristics of responding and non-responding programs (p>0.20 for all comparisons; ▶Table 1).

Table 1.

Characteristics of the 119 liver transplant programs that met the selection criteria

	Respondents (n=80)	Non-respondents (n=39)	P Value
Total number of transplant procedures performed during the three years from 2010 to 2012			0.22a
Mean (SD)	150 (114)	170 (120)
Median (IQR)	108 (181)	154 (113)
Number of years consecutively performing transplants			0.32a
Mean (SD)	21 (6)	21 (6)
Median (IQR)	24 (9)	23 (10)
Age group transplanted			0.43b
Adult – # (%)	45 (58%)	28 (68%)
Pediatric – # (%)	12 (15%)	3 (7%)
Both – # (%)	21 (27%)	10 (24%)
Geographic distribution			0.25c
1 – CT, ME, MA, NH, RI, eastern VT	5 (6%)	2 (5%)
2 – DE, District of Columbia, MD, NJ, PA, WV, northern VA	15 (19%)	3 (8%)
3 – AL, AR, FL, GA, LA, MS, Puerto Rico	10 (13%)	4 (10%)
4 – OK, TX	9 (11%)	5 (13%)
5 – AZ, CA, NV, NM, UT	9 (11%)	8 (21%)
6 – AK, HI, ID, MT, OR, WA	3 (4%)	1 (3%)
7 – IL, MN, ND, SD, WI	10 (13%)	2 (5%)
8 – CO, IA, KS, MO, NE, WY	7 (9%)	2 (5%)
9 – NY, western VT	2 (3%)	4 (10%)
10 – IN, MI, OH	6 (8%)	2 (5%)
11 – KY, NC, SC, TN, southern VA	4 (5%)	6 (15%)

^a Wilcoxon rank-sum test;

^b Chi-squared test;

^cFisher’s exact test

Liver transplant patients require lifelong monitoring of immunosuppressive care. When the 62 programs that served an adult population were asked, “How long does your transplant team have primary responsibility for management of immunosuppression therapy?”, the majority (89%) indicated that adult patients were managed by the transplant program until death. The remaining 7 (11%) programs reported that they eventually transferred care to a community physician (n=5) or that care for some patients was managed by the transplant program while care for other patients was transferred (n=2).

4.2 Current information system functionality and processes

Among the 80 programs, 78 (98%) reported that they used an EHR to manage their liver transplant patients in the outpatient setting. To track immunosuppressant dosing and laboratory results, 33 (41%) programs used the EHR only. The remaining programs reported using paper flow sheets only (n=10, 13%), a non-EHR electronic system only (n=4, 5%), or multiple systems to manage immunosuppressive care (n=33, 41%), including a combination of a paper flow sheet with the EHR (n=24, 30%), a paper flow sheet with a non-EHR electronic system (n=1, 1%), the EHR with a non-EHR electronic system (n=1, 1%), or all three together (n=5; 6%). Two programs (3%) used both paper and electronic flow sheets but did not indicate which types of electronic flow sheet were used. Of note, seventeen different electronic systems were used and half (n=35, 44%) of the programs described using multiple electronic systems to manage information for their liver transplant patients.

Most programs had the capability of generating graphs for laboratory results, including creatinine levels (88%), liver function tests (88%), and immunosuppressive laboratory results (82%). However, less than half (42%) could integrate results from external laboratories with results from internal laboratories and view them together in a graph (▶Table 2). About half of the programs could graph prescribed dose of immunosuppressants (60%) or both immunosuppressive laboratory results and prescribed dose side-by-side (42%). The ‘desire’ for graphing features among those who did not have the functionality was always higher than the ‘usage’ among those who did have the functionality: creatinine levels (desire 78%, usage 66%); liver function tests (desire 78%, usage 67%); immunosuppressant drug levels (desire 77%, usage 59%); prescribed dose of immunosuppressants (desire 86%, usage 36%; p<0.01); and both immunosuppressant drug levels and prescribed dose in the same graph (desire 86%, usage 43%; p<0.01).

Table 2.

Needs and capabilities of electronic health records being used in outpatient post-transplant care (n=80)

	Number of respondents	Number (%) responded ‘Yes’
Does your liver program have a general guideline regarding pre-specified ranges for desired immunosuppressant drug levels for most patients?	79	73 (92%)
If “Yes”:
• Do pre-specified ranges change based on time since transplant?	70	65 (93%)
• Do pre-specified ranges change based on the presence of co-morbid conditions?	70	61 (87%)
• Are there other factors that determine the pre-specified ranges?	70	46 (67%)
Is a computer-generated alert received when an immunosuppressant lab result is outside the desired range?	77	21 (27%)
Does your liver program have a general guideline regarding pre-specified routine lab testing schedules for most patients?	79	77 (97%)
If “Yes”:
• Do pre-specified lab testing schedules change based on time since transplant?	75	73 (97%)
• Do pre-specified lab testing schedules change based on the presence of co-morbid conditions?	74	59 (80%)
• Are there other factors that determine the pre-specified lab testing schedules?	72	42 (58%)
Is a computer-generated alert received when an immunosuppressant lab result is overdue or missing?	75	15 (20%)
Is a computer-generated alert received when an immunosuppressant lab result is newly available?	77	42 (55%)
Does your electronic medical record system have the capability of graphing immunosuppressant drug levels from external labs?	78	33 (42%)
What is the status of graphing the following parameters in your electronic medical record system?
• Lab results of creatinine	73	64 (88%)
• Lab results of liver function tests	73	64 (88%)
• Lab results of immunosuppressant drug levels	72	59 (82%)
• Prescribed dose of immunosuppressants	73	44 (60%)
• Both the prescribed dose and lab results of immunosuppressant drug levels in the same graph	72	30 (42%)

Nearly all (99%) of the 80 transplant programs reported that they received laboratory results performed by laboratories outside their network (i.e., external laboratories). In fact, most programs (85%) reported that 30% or more of their patients used external laboratories for routine testing (▶Figure 1). Only one program reported that none of their patients routinely used an external laboratory. There was no significant difference in the use of external laboratories between large and small programs (p=0.64).

Fig. 1.

Distribution of routine testing from external laboratories for outpatient post-liver transplant patients (n=78)

External laboratory results were received in a variety of ways, but paper-based communication was prevalent. The majority of programs (81%) indicated that “most” or “nearly all” external laboratory results were received by fax (▶Figure 2). While half (47%) of the programs had electronic interfaces that could automatically input laboratory results into a database as discrete, computer-executable data, we found that only 19% of programs indicated that “most” or “nearly all” external laboratory results were received by this method. Most programs reported that “few” or “some” external laboratory results were received by mail (64%), electronic documents (e.g. PDF) (54%), or phone calls (59%). In addition, three programs (4%) commented that they used a website (presumably using a secure login) to receive some external laboratory results. Nearly all programs received external laboratory results by two (22%), three (34%), or four or more (42%) of the methods listed above.

Fig. 2.

Proportion of programs receiving external laboratory results by amount and method (n=80)

Multiple processes are involved with recording immunosuppressive laboratory results onto paper or electronic records. Most (n=69, 86%) programs reported at least two different processes required to get the information into the flow sheets used for post-transplant immunosuppressant care. Most (80%) programs manually transcribed individual paper laboratory results from paper reports into an EHR system. Similarly, most (75%) programs scanned and linked paper laboratory reports to an EHR system. Twenty-two (28%) programs transcribed paper laboratory results (e.g. faxed reports) to a paper flow sheet and eleven (14%) transcribed paper laboratory results to a non-EHR electronic flow sheet. Nineteen (24%) programs transcribed electronic laboratory results to a paper flow sheet.

4.3 Guideline usage and computerized alerting to support guideline adherence

Most (92%) programs reported that they used guidelines that specified target ranges for immunosuppression drug levels (▶Table 2). Among these programs, target ranges varied based on time since transplant (93%), the presence of co-morbid conditions (87%), or other factors (67%) such as renal function, a history of infections, or the diagnosis necessitating liver transplantation. The proportion of programs considering ‘other factors’ varied by the population served: pediatric (100%), adult (54%), or both (72%) populations (p<0.01). Similarly, most (97%) programs used guidelines for routine laboratory testing schedules. The testing schedules varied based on time since transplant (97%), the presence of co-morbid conditions (80%), or other factors (58%) such as history of rejections, medications, or other laboratory test results.

While half (55%) of the programs reported to receive computerized alerts for recently available immunosuppressive laboratory results, computerized alerting to support guideline adherence was limited. For example, only 21 (27%) programs received alerts for results outside of a desired range, and only 15 (19%) received alerts for overdue results based on their testing schedule guideline (▶Table 2).

4.4 Barriers to implementing clinical decision support

When asked, “What do you believe are non-financial barriers to implementing clinical decision support?”, 52 (65%) respondents provided a narrative response. In Table III, we list 10 themes identified regarding barriers to CDS implementation and present example responses for each theme. The themes span concerns about support from clinicians and administrators; changes in workflow; the need to backload and integrate data from multiple sources and have a functional EHR; and finally, despite asking about non-financial barriers, respondents mentioned financial barriers (▶Table 3).

Table 3.

Themes that emerged in comments when asked about non-financial barriers to implementing clinical decision support (n=52)

Theme/category identified	Example of narrative provided by the respondent
Physician buy-in/engagement is important for the success of any change.	“MDs sitting down setting individual pt thresholds (if need be) and then all MD sticking to these as they rotate on and off service.”
Concerns about being able to trust the accuracy of only electronic data exchange.	“Team members reluctance to rely solely on electronic communication”
Support from top administrators is lacking.	“Lack of understanding on the part of organization administrators that may not understand the need for or value of technology.”
Changes in workflow are a barrier.	“changes in workflow”
Readiness of the whole system is required in order to adopt any new electronic processes.	“Do not have a fully implemented EMR, still using a fairly manua system.”
The challenges of integrating data from all sources are significant.	“Multiple computer systems within our own organization that have tremendous difficulty interfacing with each other.”
Resistance to change or inertia is everywhere.	“systems already in place, comfort level with current workflow, computer giving a suggestion that physician may not agree with”
Backload existing data would require significant extra work.	“Anything that requires duplication of data entry”; “People to perform entry of the backload of patients who would need to be entered into a tracking database.”
The system is too complex to lend itself to any CDS (patients and processes vary all of the time).	“insufficient complexity of CDS to account for patient-specific needs”
The cost or financial investment is substantial.	“It always seems as though there is a financial barrier!”

5. Discussion

Our study is the first to describe the multiple paper-based and electronic systems concurrently used to manage the complex immunosuppressive care of over 60,000 post-transplant patients in the U.S. [2]. Despite widespread EHR adoption at medical facilities performing transplantations, less than half (41%) of the U.S. transplant programs exclusively used the EHR for outpatient post-transplant immunosuppressive care. More than half of the transplant programs used paper-based, non-EHR, or multiple systems, a situation that may expose programs to increased costs and information management problems. This situation is likely exacerbated by the finding that at least one-third of the patients in most (85%) programs had routine laboratory testing performed by ‘external’ laboratories, and only 19% of the programs received “most” or “nearly all” of the external laboratory results through an electronic interface. External laboratory results are usually received by fax, requiring manual transcription to integrate laboratory information into the flow sheet view of information used by the transplant team for decision making. The variety of methods for receiving and recording laboratory results may make it difficult for a transplant program to manage information efficiently and to integrate data into their EHR; however, once discrete data can be integrated, there are opportunities for providing the computerized CDS desired by the programs because the patterns of logic reported for identifying new, out-of-range, and overdue results were similar to those already successfully implemented at Intermountain Healthcare [6]. Even so, the respondents identified technical and sociological barriers that must be addressed before transplant programs can broadly implement CDS to support immunosuppressive care.

The difficulty with integrating external laboratory data into the EHR is a significant barrier to widespread use of CDS and the development of flow sheets and integrated views of drug levels and prescribed doses required for post-transplant care. Nearly all programs had patients receiving routine laboratory results from external laboratories, and less than half of the programs could integrate external and internal laboratory values to see trends in a graph. We found that integration required manually transcribing computerized results to paper or vice versa, or both. While it is technically feasible and preferable to establish electronic interfaces [26,27], it is not always logistically or economically feasible. Only 47% of the programs reported the use of an electronic interface to receive external laboratory results, and these interfaces only handled a subset of the laboratories from which results were received. This situation may be present for several reasons. First, the cost of establishing and maintaining electronic interfaces may be prohibitive, particularly for smaller external laboratories or when the business need is only relevant for a single transplant department within a healthcare system. Electronic laboratory exchange is more likely available if a healthcare setting has a significant business need to establish an interface or is part of a health information exchange. Second, transplant patient populations use many different laboratories. A previous analysis of the external laboratories used by a ‘small’ (84 liver transplants during 2010–2012) transplant center found that over 80 different external laboratories were reporting results, and the three most frequently used laboratories comprised only 7% of all the external results received [20]. We did not query about the number of external laboratories used by transplant patients, but we suspect that this situation is common among U.S. transplant centers. Third, since lifelong monitoring is required, the set of external laboratories used changes over time when patients move or switch health insurance providers. Care for liver post-transplant patients is unique because lifelong monitoring is required even when patients reside great distances away [4]. For other chronic conditions, care is usually transferred to a physician within a reasonable proximity to the patient. Establishing an electronic interface may not be practical in a landscape of a changing set of external laboratories. Improved strategies for sharing laboratory results across enterprises will improve integration of external laboratory results.

We found that nearly all programs had immunosuppressive care guidelines that used similar patterns of logic, such as time since transplant or presence of a co-morbid condition, to individualize the response to a given laboratory value. We queried about these patterns because they are the basis of the CDS logic used internally at Intermountain Healthcare for the past ten years to alert nurse transplant coordinators about new, out-of-range, and overdue results [6]. It is not really important to know if one transplant program uses the same target range as another transplant program because variation is expected and can be managed through configuration. It is, however, important to know that most transplant programs would want the capability to define a target range, and they would want to modify the range based on time since transplant. These patterns of logic could be implemented in a CDS tool that allows an individual transplant program to trigger alerts as indicated in their own guidelines. In fact, the CDS tool could be system-agnostic and service-based, which is “an alternative and complementary strategy for knowledge-sharing” that can facilitate implementing CDS across applications and care settings [28]. Implementing system-agnostic CDS is fruitless, though, if discrete data are not available. Currently, only a quarter of the transplant programs are using computerized alerts to identify out-of-range or overdue immunosuppression laboratory results, probably partly due to limitations in the availability of discrete data needed to trigger an alert. Thus, despite the availability of guidelines that could become computable, most transplant patients are not having their immunosuppression managed by providers with access to CDS that supports guideline adherence.

While CDS in the form of patient summaries and alerts to support the immunosuppression management workflow may benefit patients and their care givers [29], improved outcomes may also impact the financial health of a transplant program. Reimbursements are sometimes fixed for the full spectrum of transplant care, thus avoiding preventable errors is a necessary cost containment strategy [3].

The qualitative analysis of open-ended questions uncovered additional organizational and sociotechnical barriers to implementing CDS. The barriers were common change management concerns about support (the need for physician buy-in or support from top administrators), impact on workflow, readiness, resistance to change, and trust in a new system [30]. Concerns about trusting the accuracy of electronic data or CDS recommendations may reflect either poor or no previous experience with CDS systems, or frequent complaints about the usability of EHRs [31]. Discordance between system expectations and use were evident from the finding that usage of graphs to view trends of laboratory and immunosuppressive doses information (among those with the capability) was always lower than the proportion of respondents that desired the functionality (among those who did not yet have the capability). Of note, to fully implement CDS, additional technical barriers such as those concerning user interfaces and clinical validation of logic would need to be addressed [30].

Surveys have limitations, particularly the potential for selection and recall bias. Study participants were self-selected based upon their willingness to complete the survey. While respondents to an online survey may be more technically savvy or have greater expectations and desire for automated systems to support patient care than non-respondents, we don’t expect these characteristics to change a respondent’s description of the systems and guidelines used by their program. We mitigated biases by preemptively informing participants that results would be reported without identifying individual programs, following up with non-respondents, encouraging respondents to seek input from others in the transplant care team, and using formal cognitive interviewing techniques to promote high construct validity in the development of our survey instrument. We succeeded in obtaining a high response rate (67%) with no significant difference in key characteristics between responding and non-responding programs. We believe findings can be generalized to other U.S. liver transplant programs. Finally, specific guidelines were not analyzed in detail as part of this study. Doing so would serve as future work by identifying additional important similarities or differences in guidelines across transplant programs.

6. Conclusions

Despite the ubiquity of EHRs, all transplant centers must use manual, paper-based methods for part or all of their process for managing post-transplant immunosuppressive care. Across the U.S. this impacts an estimated 60,000 liver transplant patients [2]. Most external laboratory results are not automatically integrated as discrete data into the EHRs used by transplant centers and thus are not usable by CDS without manual transcription. Moreover, only a quarter of the transplant programs in the U.S. currently use computerized systems to identify overdue or out-of-range immunosuppressant drug levels even though most transplant centers have guidelines for immunosuppressive care, and these guidelines use similar patterns of logic that can be implemented using rule-based computerized CDS. In addition to challenges with using both electronic and paper-based systems for laboratory information management, there are sociotechnical and organizational barriers that impede the implementation of CDS systems. Even so, many programs have key features required for success, namely guidelines, and systems capable of storing discrete laboratory results. Pressure to improve efficiency and clinical outcomes in the face of a growing population of patients and capitated reimbursement models will further the need for CDS to support outpatient post-transplant immunosuppressive care.