Influence of user-centered clinical decision support on pediatric blood product ordering errors

Abstract

Background

Children are at increased risk from transfusion-related medical errors. Clinical decision support (CDS) can enhance pediatric providers’ decision-making regarding transfusion practices including indications, volume, rate, and special processing instructions. Our objective was to use CDS in a pediatric health system to reduce:

1. blood product-related safety events from ordering errors;

2. special processing ordering errors for patients with T-cell dysfunction, sickle cell disease (SCD), or thalassemia;

3. transfusions administered faster than 5 mL/kg/h.

Materials and methods

In this single-center before and after quality improvement study, we evaluated how user-centered design of pediatric blood product orders influenced pediatric transfusion practices and outcomes. Safety events were identified through active and passive surveillance. Other clinically relevant outcomes were identified through electronic health record queries.

Results

Blood product-related safety events from ordering errors did not change significantly from the baseline period (6 events, 0.4 per month, from 1/1/2018–3/27/2019) to the intervention period (1 event, 0.1 per month, from 3/28/2019–12/31/2019; rate ratio: 0.27 [0.01–2.25]). Packed red blood cell (PRBC) and platelet orders for patients with T-cell dysfunction that did not specify irradiation decreased significantly from 488/12,359 (3.9%) to 204/6,711 (3.0%, risk ratio: 0.77 [0.66–0.90]). PRBC orders for patients with SCD or thalassemia that did not specify phenotypically similar units fell from 386/2,876 (13.4%) to 57/1,755 (3.2%, risk ratio: 0.24 [0.18–0.32]). Transfusions administered faster than 5 mL/kg/h decreased from 4,112/14,641 (28.1%) to 2,125/9,263 (22.9%, risk ratio: 0.82 [0.78–0.85]).

Discussion

User-centered design of CDS for pediatric blood product orders significantly reduced special processing ordering errors and inappropriate transfusion rates. Larger studies are needed to evaluate the impact on safety events.

INTRODUCTION

Medical errors in blood transfusion practice are a common source of transfusion-related morbidity and mortality¹. Children are at a higher risk of adverse outcomes from such errors than adults^2,3. Due to their small size, errors in transfusion volume and/or rate are more likely to lead to inadequate treatment from under-transfusion or complications of over-transfusion such as transfusion-associated circulatory overload (TACO). Children are also more susceptible to errors in special processing of blood products. For example, preterm and term infants are relatively immunocompromised; many with immune dysfunction disorders involving T-cell immunity which manifest in the first 6 months to 1 year of life. Non-irradiated blood products administered to these populations increase the risk of rare, but severe and potentially fatal complications such as transfusion associated graft vs host disease (TA-GVHD)⁴. Additionally, administration of packed red blood cells (PRBCs) to children with sickle cell disease (SCD) without selecting phenotypically similar units (minor RBC antigen matching to a certain degree) can lead to production of minor RBC antigen alloantibodies, increasing the risk of hemolysis during future transfusions and complicating the ability to be transfused for the rest of the child’s life^5,6.

Ordering blood products is a complex task for health providers who are often not trained sufficiently in Transfusion Medicine^7–9. To make appropriate decisions, ordering providers must not only weigh the risks and benefits of a transfusion, but also decide on the appropriate volume and transfusion rate^2,10, select special processing requirements based on the patient’s medical condition(s)⁴, and effectively communicate with the blood bank medical technologists and nurses to ensure timely therapeutic interventions (e.g., preparing blood for immediate use vs future transfusion during a procedure)¹¹. In the setting of this complexity, safety events from transfusion practice errors related to human factors and information technology have been frequently reported in transfusion surveillance systems¹².

Clinical decision support systems (CDS) can reduce blood product ordering errors through just-in-time education and patient-specific contextual knowledge. Numerous studies have demonstrated the effectiveness of CDS systems to promote restrictive transfusion strategies in adults and children^13–20. However, to our knowledge, no studies have examined the role of CDS in improving transfusion volume, rate, and special processing requests in pediatric settings.

After reviewing transfusion-related safety events, a Failure Modes and Effects Analysis (FMEA)²¹ at our institution identified ordering errors as the most likely source of serious patient blood management errors. We re-designed the blood product ordering process through a combination of design by expert committee and user-centered design through formative usability testing. The resulting ordering process led to fewer severe errors in simulated scenarios during summative usability testing²². In this study, we evaluate the impact of implementing this new ordering process on pediatric transfusion practice and patient outcomes.

Primary objective

Reduce blood product safety events attributed to ordering errors (as detected from hospital incident reports plus active surveillance by a transfusion safety specialist).

Secondary objectives

1. Reduce the proportion of transfusion orders with special processing errors defined as:

   1. PRBC or platelet transfusion orders for patients with T-cell dysfunction that did not specify irradiation;

   2. PRBC transfusion orders for patients with SCD or thalassemia that did not specify to use phenotypically similar units.

2. Reduce the proportion of all flowsheet-tracked transfusions faster than 5 mL/kg/h.

Balance objective

To determine if our interventions for the primary and secondary objectives led to new problems in other parts of the system, we measured the proportion of transfusion orders with special processing (i.e., irradiation or phenotypically similar units) requested.

MATERIALS AND METHODS

Context

This study was performed at a large, urban pediatric health system in which the primary blood bank serves 2 freestanding children’s hospitals as well as high volume hematology and oncology clinics with over 10,000 blood transfusions per year. The study was conducted from 1/1/2018–12/31/2019. The study start date was chosen to provide a baseline to the interventions implemented on 3/28/2019 (see below) and the end date was selected prior to a new set of blood transfusion workflow changes implemented in early 2020 as part of Electronic Health Record (EHR) upgrades. In addition to ad-hoc transfusions, the patient blood management program supports specialized transfusion workflows for bone marrow transplant (BMT), chronic transfusion therapy, cardiac surgery, solid organ transplant, extracorporeal membrane oxygenation (ECMO), and apheresis. Transfusion orders are placed by providers using an enterprise EHR (Epic Systems^©, Verona, WI, USA) which interfaces with a separate laboratory information system containing an FDA-approved blood banking module (Sunquest Laboratory^TM, Sunquest Information Systems, Tucson, AZ, USA).

Transfusion orders consisted of Prepare and Transfuse orders. The Prepare order signaled the blood bank to allocate a specific volume of blood product(s) to a patient and also allowed the ordering provider to request special processing including irradiation, washed cells, phenotypically similar units, and cytomegalovirus (CMV) seronegative units. The Transfuse order signaled the nurse to transfuse the blood product and specified the duration. Of note, a Transfuse order was not required in workflows in which an anesthesiologist or other physician would administer the blood transfusion directly. All transfusion orders were embedded in order sets, collections of related orders to improve physician efficiency and reduce the cognitive workload of individually searching for specific orders. The majority of blood product orders were placed from dedicated blood transfusion order sets consisting of pre-transfusion testing orders, Prepare orders, and Transfuse orders. However, a substantial fraction of blood product orders originated from disease or workflow-specific order sets (e.g., post-operative cardiac surgery) in which blood product orders were a single section embedded within a larger order set.

Interventions

Blood transfusion orders and order sets were re-designed through formative usability testing described elsewhere²². Briefly, a multi-disciplinary expert committee reviewed the FMEA results and recommended a new design. The design was then adjusted through formative usability testing, in which front-line providers were provided a scenario appropriate to their clinical specialty and asked to “think aloud”²³ as they ordered blood products in an EHR test environment that had identical functionality to the production EHR environment except for re-designed blood orders. Based on comments and errors in simulated performance, iterative adjustments were made to blood product orders and order sets in the EHR test environment until 5 unique providers made no errors in 10 scenarios and had no new suggestions for design improvement. The new user-centered design was evaluated in summative usability testing and demonstrated significant reductions in severe ordering errors in simulation.

The user-centered design process led to adjustments in the Prepare and Transfuse orders as well as dedicated blood transfusion order sets (Online Supplementary Content, Figure S1). To implement these changes across the enterprise, the authors also reviewed 52 disease or workflow-specific order sets that included blood product orders and either retired or adjusted the order sets to conform to the new design. During this process, stakeholders also had the opportunity to adjust order set defaults. In particular, cardiac surgery and ECMO order sets were re-designed such that the number of units of each blood product would automatically calculate based on the patient’s weight.

Additionally, a novel workflow was developed for BMT patients. Prior to the intervention BMT providers would place a nursing communication order denoting specific parameters for transfusion (e.g. “transfuse 1 unit irradiated PRBCs if hemoglobin <8”), and then nurses would enter the Prepare and Transfuse orders themselves into a dedicated blood transfusion order set and sign under the provider’s name. After the intervention, in addition to the nursing communication order the BMT provider would place conditional blood product Prepare and Transfuse orders that specified the volume and special processing instructions. The nurse would then release the orders when the patient met criteria specified in the communication order, but the nurse did not have to enter a new order themselves for each transfusion (Online Supplementary Content, Figure S2).

Measures

All Prepare orders were extracted using EHR procedure codes associated with PRBC, platelet, FFP, and cryoprecipitate orders. Transfusion administrations were extracted from flowsheet rows entered by nurses. Of note, flowsheet data elements were not reliably entered for transfusions administered by physicians (e.g. by anesthesiology in the operating room or bedside procedures in the neonatal intensive care unit), emergency situations (e.g. massive transfusion protocol, emergency release), or for apheresis procedures. Thus, transfusions without volumes documented in flowsheet entries were excluded from measures of transfusion administrations. Additionally, no explicit link existed in the EHR during the study period between Prepare orders and Transfuse orders as this study was conducted prior to implementation of the Epic Blood Product Administration Module. Thus, order questions in the Prepare order (e.g. special processing requests) could not always be linked to the actual transfusion administration records generally created by nurses, which contained the rate of transfusion, time of transfusion, and other elements related to the administration.

Safety events related to patient blood management were detected through a combination of passive surveillance through incident reports as well as active surveillance by a dedicated transfusion safety nurse (JJ) who rotated systematically through hospital units providing nursing education and eliciting errors. Safety events were reported as all events per month and events attributed to ordering errors per month. Transfusion orders for patients with T-cell dysfunction were identified as follows:

(1) a list of the most common diagnosis codes in patients receiving PRBC or platelet transfusions was reviewed by a pediatric hematologist/oncologist and transfusion medicine specialist (MR), and diagnoses meriting irradiated PRBCs or platelets outside of emergencies were labelled (Table I); (2) patients were classified as having T-cell dysfunction if any of these diagnoses were present during the encounter where the transfusion took place; (3) all patients <6 months of age were assumed to be at risk for T-cell dysfunction. Transfusion orders for patients with SCD or thalassemia were identified by a pediatric hematologist/oncologist reviewing the most common diagnosis codes in patients receiving PRBC transfusions (Table II).

Table I.

Frequency of transfusion orders and administrations by blood product

Blood product	Baseline (1/1/18 – 3/27/19)		Intervention (3/28/19 – 12/31/19)
	Orders (%)	Administrations with documented volume	Orders (%)	Administrations with documented volume
PRBCs	14,819 (58)	9,339 (64)	8,999 (61)	6,098 (69)
Platelets	6,372 (25)	3,960 (27)	3,372 (23)	2,079 (23)
FFP	2,958 (12)	1,090 (7)	1,557 (11)	588 (7)
Cryoprecipitate	1,235 (5)	247 (2)	775 (5)	114 (1)

PRBCs: packed red blood cells; FFP: fresh, frozen plasma.

Table II.

Proportion of transfusions administered at a rate >5 mL/kg/h

Blood product	Baseline	Intervention	Fast Transfusions Averted*
PRBCs	1,262/9,339 (14)	735/6,098 (12)	89
Platelets	2,151/3,960 (54)	1,105/2,463 (45)	233
FFP	608/1,090 (56)	268/588 (46)	60
Cryoprecipitate	89/247 (36)	17/114 (15)	24

^*Fast transfusions averted was calculated by (1) estimating the expected number of fast transfusions by multiplying the baseline rate for each blood product by the number of transfusions during the intervention period and (2) subtracting the actual number of fast transfusions.

PRBCs: packed red blood cells; FFP: fresh, frozen plasma.

Transfusions administered faster than 5 mL/kg/h were identified by comparing the maximum documented rate of transfusion in mL/h per flowsheet entries to the most recently documented weight for the same patient taken prior to the transfusion start time.

Study of the interventions

Adjustments to blood product orders, dedicated blood transfusion order sets, other order sets that included blood product orders, and the new BMT workflow were all implemented simultaneously on 28 March 2019. We compared the measures described above during the baseline period (01/01/2018–03/27/2019) to the intervention period (03/28/2019–12/31/2019) after implementation. The intervention period was truncated at the end of 2019 due to unrelated interventions implemented in early 2020.

Analysis

Safety events were compared as Poisson rates per month. All other measures were compared visually using statistical process control charts (p-charts) and analytically using chi-square tests of proportions.

Statistical analyses were performed using R, version 3.5.1 (R Foundation, Vienna, Austria)²⁴.

Ethical considerations and reporting guidelines

This study was determined to be Non-Human Subjects Research as a local quality improvement study by the Children’s Healthcare of Atlanta Institutional Review Board. This study is reported according to the SQUIRE 2.0 guidelines for quality improvement reports and the Safety-related EHR Research (SAFER) Reporting Framework for safety-related EHR interventions^25,26.

RESULTS

A total of 25,384 unique Prepare orders were placed during the baseline period and 14,703 Prepare orders during the intervention period with PRBC orders comprising the majority (Table I). We identified 14,636 blood product administrations during the baseline period and 9,263 during the intervention period that had a documented transfusion rate entered in the flowsheets. The discrepancy between the number of Prepare orders and administrations with a documented transfusion rate resulted from (1) blood products ordered for the operating room (OR) in case of bleeding and never transfused, (2) apheresis orders where volume and rates were not consistently documented, (3) Prepare orders for sickle cell disease patients ordered in anticipation of potential long turnaround times in case of an acute need for transfusion that were never administered, and (4) cancelled orders. Nonetheless all Prepare orders were included in the analysis of ordering errors since they could have been administered to the patient.

Primary outcome

During the baseline period, we identified 0.9 safety events per month (13 total) related to patient blood transfusion practices, of which 0.4 per month (6 total) were attributed to ordering errors. During the intervention period, we identified 0.4 safety events per month (6 total) related to patient blood management (rate ratio 0.76, 95% confidence interval [CI]: 0.24–2.13, p=0.643), of which 0.1 per month (1 total) were attributed to ordering errors (rate ratio 0.27, 95% CI: 0.01–2.25, p=0.265).

Secondary outcomes

During the baseline period, 488/12,359 (3.9%) of PRBC and platelet Prepare orders placed for a patient with at least 1 visit diagnosis indicative of T-cell dysfunction or for a patient <6 months old did not have a special processing request for irradiation. In the intervention period, 204/6,711 (3.0%) such orders did not have a request for irradiation (risk ratio: 0.770, 95% CI: 0.66–0.90, p=0.001). While error rates peaked just prior to the intervention, the p-chart did not demonstrate special cause variation (Figure 1).

Figure 1.

Statistical Process Control Chart (p-chart) for the proportion of PRBC and platelet orders that should have been irradiated, but irradiation was not requested in the Prepare order

During the baseline period, 386/2,876 (13.4%) of PRBC Prepare orders placed for patients with SCD or thalassemia did not have a request for phenotypically similar units. In the intervention period, 57/1,755 (3.2%) such orders did not have a request for phenotypically similar units (risk ratio: 0.24, 95% CI: 0.18–0.32, p<0.001). The p-chart demonstrated special cause variation temporally associated with intervention implementation at the end of March, 2019 and sustained through the rest of the study period (Figure 2).

Figure 2.

Statistical Process Control Chart (p-chart) for the proportion of PRBC orders that should have been from phenotypically similar units, but phenotypically similar units were not requested in the Prepare order

During the baseline period, 4,112/14,641 (28.1%) of all transfusions were administered faster than 5 mL/kg/h, compared to 2,125/9,263 (22.9%, risk ratio: 0.82, 95% CI: 0.78–0.85, p<0.001) in the intervention period, with special cause variation apparent in the p-chart at the time of the intervention in March, 2019 (Figure 3). If baseline rates of fast transfusions persisted, we would have expected an additional 477 transfusions at a rate >5 mL/kg/h. The greatest reductions were seen in platelet transfusions, which accounted for 233 (49%) of the 477 transfusions >5 mL/kg/h averted (Table II).

Figure 3.

Statistical Process Control Chart (p-chart) for the proportion of blood product transfusions administered at a rate >5 mL/kg/h

Balancing outcome

We reviewed the proportion of blood product orders requesting irradiation and phenotypically similar units. The proportion of PRBCs and platelets ordered as irradiated decreased slightly from 61.6 to 57.3% (p<0.001). The proportion of PRBCs where phenotypically similar units were requested increased slightly from 17.8 to 20.1% (p<0.001).

SAFER framework

We reviewed the 8 sociotechnical dimensions (i.e. combination of technology and human elements) of patient safety for EHR interventions and described pre-intervention issues, what sociotechnical changes were made, why they were felt to be effective, and how they could be applied in other settings (Table III).

Table III.

Sociotechnical interventions described using the SAFER Reporting Framework for Safety Related EHR Research²⁶

Sociotechnical Dimension	Pre-Intervention Issues	What*	Why†	How‡
Hardware & software	Interface between EHR and transfusion management system requires additional build for each change in the user interface.	New EHR design used cascading questions to produce same outputs to interface with transfusion management system as prior EHR design.	Increased flexibility of new EHR design to match users’ needs.	Identify interfaced EHR fields early to determine how to preserve them or if new interface build is required.
Clinical content	Prepare orders ask for special processing instructions, but most ordering providers do not know indications.	Changed Prepare order to ask if indications for special processing present (see Figure 1). Defaulted “ irradiated” for children <6 months, consistent with local policy	Improved accuracy of ordering provider special processing selections.	Adapt Prepare orders to ask only for elements familiar to ordering providers.
Human-computer interface	Affordances in original system made it easy to accidentally order multiple aliquots when only one was needed. Culture to transfuse platelets as quickly as possible.	Adjusted questions to require extra clicks for multiple aliquots from same donor unit along with explicit instructions to the nurse. Changed speed buttons in platelet Transfuse orders to encourage longer durations.	Increased friction to order multiple aliquots. Easier to select platelet transfusion duration consistent with policy. Improved	Adjust interface to ensure multiple aliquots are hard to order by accident. Ensure speed buttons for transfusion duration encourage policy adherence.
Workflow & communication	If multiple aliquots prepared, no hard stop to prevent nurse from administering all available blood even if order instructions differ.	Instructions for transfusion of multiple aliquots made more visible in Transfuse orders.	Improved communication to nurse to administer 1 aliquot, check hemoglobin, and contact ordering provider.	Review workflow for multiple aliquots from same donor unit to avoid multiple transfusions when only one is needed.
People	Wide variety of expertise among ordering providers.	Designed interface for least experienced providers with defaults adjustable in hematology/oncology clinics with more provider expertise.	Less experienced providers more likely to know how to answer order questions. More experienced providers can activate shortcuts.	Usability testing of candidate interfaces with users of variable experience levels.
Internal organizational features	High volume Blood bank serving multiple complex populations with strong transfusion medicine expertise. Presence of quality improvement, clinical informatics, and human factors engineering expertise. Strong culture of safety with serious safety events reported to highest level of organizational leadership. Separate information systems for main EHR and laboratory.	User-centered design to augment clinical expertise. Technical resources to build, test, and validate complex interfaces for various workflows. Reviewed and edited large number of legacy order sets that contained blood product orders.	Early engagement of clinical experts with willingness to perform usability testing before finalizing interface design. Strong burning platform for change including regular review from executive leadership team.	Obtain executive sponsorship early. Iterative design of user interfaces based on usability testing. Review and catalog all impacted order sets and workflows early in design phase.
External rules & regulations	FDA and AABB require documentation of testing and validation of all clinical decision support and interfaces.	Extensive unit testing and regression testing for dedicated blood product order sets as well as special workflows ( e.g. BMT, ECMO).	Rigorous testing ensured design behaved as expected in production.	Allocate testing resources early for dedicated blood product order sets as well as special workflows.
Measuring & monitoring	Reliance on hospital incident reports and active surveillance by a dedicated transfusion safety specialist. No automated reporting of blood product safety metrics.	Performed usability testing in test EHR environment identical to production environment except for blood order design. Usability tests performed on hospital floors with representative providers. Developed automated metrics for special processing errors, transfusion rates.	High fidelity testing environment led to discovery of unexpected failure modes. Automated metrics more sensitive than prior approaches.	Measure user performance using simulated scenarios. Create automated reports for key blood product safety metrics.

BMT: Bone Marrow Transplant; ECMO: Extracorporeal Membrane Oxygenation; EHR: electronic health record; FDA: Food & Drug Administration; IT: Information Technology; VS: vital signs;

^*What sociotechnical changes were made to implement an EHR-related intervention to improve patient safety.

^†Why the intervention did or did not lead to safety improvements.

^‡How the intervention can be applicable or exported to others.

DISCUSSION

User-centered design of blood product orders and order sets led to reductions in ordering errors for special processing, transfusion rate, and total volume. However, there was no significant reduction in total blood product safety events, as detected from hospital incident reports and/or active surveillance by a transfusion safety specialist or those attributed specifically to ordering errors. This finding may be due to inadequate power as safety events are rare, inadequate surveillance for safety events, or because special processing, transfusion rate, and volume errors account for only a small proportion of blood product safety events. In contrast, for patients with SCD or thalassemia the new design led to dramatic improvements in the proportion of orders requesting phenotypically similar PRBCs, preventing “near-misses” that may lead to hospital safety events. Among patients at risk for T-cell dysfunction, the new design led to significantly more requests for irradiated PRBCs and platelets, although the effect size was not as large as progress seen for SCD and thalassemia patients. Nonetheless, this improvement was achieved in the context of fewer requests for irradiated blood products across the system. Finally, the proportion of transfusions administered faster than the recommended 5 mL/kg/h for non-emergent transfusions improved significantly, particularly for platelets.

The new blood product orders did not ask ordering providers to enter specific special processing requests, instead asking for indications for special processing and then cascading to the appropriate processing request. There exist fewer indications for phenotypically similar units than for irradiated blood products, and less diagnostic uncertainty for SCD and thalassemia compared to some diagnoses associated with T-cell dysfunction. This combination may explain the more dramatic reductions in ordering errors for phenotypically similar units. Many CDS systems have demonstrated improved adherence to evidence-based practices in patient blood management²⁷. In adult settings, order sets and alerts promoting restrictive transfusion strategies have repeatedly shown reductions in transfusions for patients with hemoglobin >7g/dL^13,17–20. In pediatrics, the evidence base for specific patient blood management strategies is not as well defined². Nonetheless, CDS has demonstrated improved utilization of PRBCs, plasma, and platelets in children^15,16,28,29. However, to our knowledge this is the first pediatric study demonstrating the effectiveness of CDS to improve special processing requests and transfusion rate.

Limitations

This is a single center study using one EHR instance from Epic Systems. Thus, the conclusions from this study may depend on organizational culture, behavioral habits prior to the intervention, change management structures, the existence of safety events at our institution prioritizing specific goals, and other factors that reduce its generalizability. While we have attempted to capture important sociotechnical elements using the SAFER reporting framework, these assessments are likely not comprehensive, and other factors may lead to different outcomes when applied to other health systems. Additionally, safety events associated with pediatric patients are thankfully very rare, and while the raw frequency decreased from the baseline to the intervention period, we were unable to demonstrate statistically significant improvement. Finally, our secondary study outcomes were limited to process measures. While the relevant transfusion reactions may have been detectable through incident reports, we had no reliable measures to determine if interventions changed patient outcome measures such as TA-GVHD, development of RBC alloimmunization and subsequent hemolytic reactions, or delays in transfusion.

CONCLUSIONS

User-centered design of blood product orders and order sets can improve pediatric blood transfusion practices, but its impact on safety events and patient outcomes remains unclear. Asking ordering providers for clinical indications that drive the need for special processing instead of the specific special processing requests themselves led to substantial improvements in ordering for SCD and thalassemia patients as well as improvements in patients with T-cell dysfunction. Blood product orders occur in the context of a complex sociotechnical system with changes to orders potentially affecting a wide variety of workflows. Future studies examining the impact of CDS on additional evidence-based practices and through multicenter initiatives would demonstrate the generalizability of this approach to optimize pediatric patient blood management.