Medical chart validation of inpatient diagnosis codes for Transfusion Related Acute Lung Injury (TRALI) 2013–2015

Abstract

INTRODUCTION:

Transfusion-Related Acute Lung Injury (TRALI), an adverse event occurring during or within 6 hours of transfusion, is a leading cause of transfusion associated fatalities reported to the U.S. Food and Drug Administration. There is limited information on the validity of diagnosis codes for TRALI recorded in inpatient electronic medical records (EMRs).

STUDY DESIGNS AND METHODS:

We conducted a validation study to establish the positive predictive value (PPV) of TRALI International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis codes recorded within a large hospital system between 2013–2015. A physician with critical care expertise confirmed the TRALI diagnosis. As TRALI is likely under-diagnosed, we utilized the specific code (518.7), and codes for respiratory-failure (518.82) in combination with transfusion-reaction (999.80, 999.89, E934.7).

RESULTS:

Among almost four million inpatient stays, we identified 208 potential TRALI cases with ICD-9-CM codes and reviewed 195 medical records; 68 (35%) met clinical definitions for TRALI [26 (38%) definitive, 15 (22%) possible, 27 (40%) delayed]. Overall, the PPV for all inpatient TRALI diagnoses was 35% [95% Confidence Interval (CI): 28–42]. The PPV for the TRALI specific code was 44% (95% CI: 35–54).

CONCLUSION:

We observed low PPVs (<50%) for TRALI ICD-9-CM diagnosis codes as validated by medical charts, which may relate to inconsistent code use, incomplete medical records, or other factors. Future studies utilizing TRALI diagnosis codes in EMR databases may consider confirming diagnoses with medical records, assessing TRALI ICD-10-CM codes, or exploring alternative ways for of accurately identifying TRALI in EMR databases.

INTRODUCTION

Transfusion-Related Acute Lung Injury (TRALI) is a rare but serious life-threatening adverse event occurring during or within 6-hours of transfusion, characterized by respiratory distress and pulmonary edema. TRALI has been associated with almost all blood components^1–3 and is a leading cause of reported allogeneic transfusion-related mortality in the United States. Despite measures taken by the transfusion community to reduce the risk of TRALI, 41% of transfusion-related fatalities reported to the U.S. Food and Drug Administration (FDA) between years 2010 and 2014 were attributed to TRALI.⁴ TRALI can be difficult to distinguish from transfusion-associated circulatory overload (TACO) and other acute respiratory conditions, particularly in critically ill patients who frequently have multiple acute lung injury risk factors.^3,5 Although TRALI is a rare transfusion related outcome, TRALI may be an important endpoint for studies focused on the safety of blood components and blood derived products in large databases.⁶

In 2016, the FDA expanded the Sentinel Initiative⁷ and Sentinel Distributed Database (SDD) to include inpatient electronic medical record (EMR) data from HCA Healthcare (HCA).⁸ The HCA system includes over 185 affiliated acute care hospitals and other facilities concentrated in the southern US. The partnership with HCA provides access to data from approximately two million inpatient encounters per year, as well as emergency department (ED) and outpatient visits to HCA affiliated facilities, and forms the basis of the HCA Sentinel database. The U.S. FDA has previously used administrative databases along with medical chart data to study TRALI,⁹ and the Sentinel Initiative’s partnership with HCA represents a potential opportunity for monitoring recipient safety of FDA-regulated blood components and blood-derived products. To inform the design of future studies of TRALI based on the SDD and other administrative databases, we conducted a validation study to establish the Positive Predictive Value (PPV) of TRALI diagnosis codes recorded in the HCA Sentinel database. Potential TRALI cases included in this medical record-based validation study were identified in the HCA Sentinel database as part of a protocol-based assessment focused on occurrence of TRALI after exposure to packed red blood cells, platelets or plasma.⁸

STUDY DESIGNS AND METHODS

Data sources and study population

This validation study included HCA affiliated hospitals systematically contributing data to the SDD (n=169) at the time of the study. The study population consisted of inpatient encounters from September 30, 2013 through September 30, 2015. This time-period was selected because prior to September 2013 some hospitals did not systematically collect transfusion data in a standardized format that could be aggregated across hospitals and analyzed. Within this period, encounters were included in analyses if they had a diagnosis code for TRALI (defined below) included the HCA Sentinel database.⁸ Potential TRALI cases were confirmed through medical chart review. The data presented were collected as part of a public health surveillance activity conducted under the auspices of the FDA Sentinel Initiative. Thus, the collection and analysis of these data did not qualify as human subjects research under the Common Rule and were not subject to institutional review board (IRB) review.^10–12

Case identification and chart retrieval

Potential cases of TRALI were identified with International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis codes originating from an inpatient stay. As TRALI is likely under-diagnosed, we focused not only on the TRALI specific ICD-9-CM code (Criterion A: 518.7), but also on specific respiratory failure codes (Criterion B: 518.82, Acute respiratory failure; Criterion C: 518.82, Other pulmonary insufficiency, not elsewhere classified) in combination with an ICD-9-CM code for a transfusion reaction (ICD-9-CM codes: 999.80, Other infusion and transfusion reaction; 999.89, Other transfusion reaction; E934.7, Natural blood and blood products causing adverse effects in therapeutic use). Hereafter, these criteria are referred to as potential-TRALI_A, potential-TRALI_B, potential-TRALI_C, or potential-TRALI_A,B,C (Supplement A).

The HCA Sentinel database is EMR-based but conforms to the Sentinel Common Data Model,¹³ and categorizes diagnosis codes associated with inpatient stays as principal, secondary, or present on admission. Examples of diagnoses that may be present on admission include diagnoses from a hospital transfer or occurring in an ED prior to inpatient admission. We utilized all TRALI diagnosis codes to identify potential TRALI cases and did not remove cases which had diagnosis codes with “present on admission” flags. We utilized this comprehensive TRALI definition to ensure capture of all potential TRALI cases, which may have had transfusions administered in an ED or prior to hospitalization and were subsequently admitted. Thus, inpatient encounters with a TRALI diagnosis code listed in any position (principal or secondary) as well as diagnoses labeled “present on admission” were selected for review (Figure 1). For each potential TRALI case, HCA researchers located and retrieved a medical chart corresponding to the relevant inpatient stay in which the diagnosis was recorded and provided in a secure location for physician review.

Figure 1. Disposition of potential Transfusion-Related Acute Lung Injury (TRALI) inpatient stays and transfusion information ascertained in the Sentinel Distributed Database (SDD).

*TRALI_A,B,C: TRALI_A, TRALI _B, and TRALI _C, See below; † TRALI _A: TRALI, ICD-9-CM code any position (518.7)

‡TRALI _B: Acute respiratory failure ICD-9-CM code any position (518.81), WITH ICD-9-CM code for a transfusion reaction (999.80, 999.89,E934.7)

§TRALI _C: Other pulmonary insufficiency (518.82), WITH IICD-9-CM code for a blood transfusion reaction (999.80 or 999.89 or E934.7)

ǁ ALI: Acute Lung Injury

Case adjudication

All medical charts were reviewed by at least one physician certified in both pulmonary care and critical care or certified in internal medicine and currently working in an intensive care unit. Physician reviewers were affiliated with separate institutions and were not involved in clinical care of potential TRALI cases they reviewed. Medical records of all cases with an inpatient TRALI diagnosis code were requested, regardless of transfusion status or timing of events relative to transfusion. Medical records were reviewed by physicians that verified the TRALI diagnosis. The validation study was conducted in two phases. In the pilot phase, an adjudication form was developed and tested for the review of 20 randomly chosen potential TRALI cases. At least two physicians independently reviewed two rounds of 10 charts (total n=20), blinded to the other’s decision and to the assignment of the second adjudicator. The adjudication form was revised according to clinician feedback after each round, to ensure classification rules and instructions were clear. After double adjudication of each case was complete, study staff reviewed completed adjudication forms, identified discrepancies, followed up with adjudicators as needed for clarifications. We defined minor discrepancies when adjudicators differed in understanding of when to select ‘not a TRALI case’, ‘ALI not associated with a transfusion’ or ‘unable to determine’ on the adjudication form. We defined major discrepancies as instances when adjudicators differed in their TRALI case designation (i.e., TRALI case vs unable to determine). If discrepancies persisted between adjudication decisions, discrepant cases were assigned to a third adjudicator certified in both pulmonary and critical care for a final decision. Study staff also entered de-identified data from the adjudication form into the study database and followed up with adjudicators to ensure forms were completed appropriately. Charts were provided to the adjudicator by HCA staff, and neither Sentinel staff nor study investigators had access to the medical charts themselves.

After the pilot was completed and the adjudication form finalized, adjudicators proceeded to a single adjudication phase for the remainder of cases but were given the option of requesting a second opinion. In such instances, cases were provided to a senior adjudicator, certified in both pulmonary and critical care, for a final case decision. To ensure adjudicators were familiar with TRALI, we provided training on the case definition. Project staff also reviewed abstracted clinical criteria for any discrepancies with case decisions and worked with the adjudicators to resolve them.

Although an updated TRALI definition has recently been proposed,¹⁴ this study was completed prior to the new consensus and adjudicators classified potential TRALI cases as meeting a definitive or possible clinical TRALI case definition with using the case definition in the National Healthcare Safety Network (NHSN) Manual: Biovigilance Component definition of TRALI (Table 1).^15,16 Delayed TRALI was diagnosed in the same manner, but allowed for symptom onset between 6 and 72 hours of the transfusion.^17,18 Adjudicators classified potential cases as ‘not being a case’ when essential information to rule out TRALI was available for review and ‘unable to determine’ if clinical information essential to case determination was not available. Adjudicators also classified ALI cases ‘not associated with a transfusion’ when transfusion timing information provided the ability to rule out the transfusion as the cause of ALI.

Table 1.

Transfusion-Related Acute Lung Injury (TRALI) definitions

Definition	Clinical Description
Definitive TRALI15,16	A. No evidence of acute lung injury (ALI) prior to transfusion
	AND
	B. ALI onset during or within 6 hours of transfusion
	AND
	C. Hypoxemia defined by any of these methods:
	a. PaO2 / FiO2 ≤300 mm Hg
	b. Oxygen saturation is < 90% on room air
	c. Other clinical evidence
	AND
	D. Radiographic evidence of bilateral infiltrates
	AND
	E. No evidence of left atrial hypertension (i.e. circulatory overload)
	AND
	F. No temporal relationship to an alternative risk factor* for ALI during or within 6 hours of completion of transfusion
Possible TRALI15,16	Same as above EXCEPT there is a temporal relationship to a specific ALI risk factor*
Delayed TRALI definition, defined in critically ill patients
Delayed TRALI17,18	Same as for possible TRALI except allows for symptom onset within 6 to 72 hours of blood transfusion

^*Alternative risk factors for ALI include: 1) Direct lung injury: Aspiration, pneumonia, toxic inhalation, long contusion, near drowning; 2) Indirect lung injury: severe sepsis, shock, multiple trauma, burn injury, acute pancreatitis, cardiopulmonary bypass, drug overdose

The following items were also abstracted from medical charts by adjudicators: 1) transfusions associated with the potential TRALI event, 2) mechanical ventilation before or after the potential TRALI event, and 3) transfusion indication. Description of criteria used to select potential cases is in the Supplement. The design and objectives of the parent study have been described previously.⁸ This retrospective study did not lead to additional investigations of HLA/HNA testing. We could not examine leukoreduction or blood production practices across blood centers or access reports to blood banks, the FDA or other hemovigilance systems.

Positive predictive value (PPV) calculation

We calculated the PPV of the TRALI diagnoses codes identified in the HCA Sentinel database by dividing the number of confirmed TRALI cases (definitive, possible, or delayed) by the total number of potential TRALI cases. PPVs were also calculated for each electronic TRALI criterion as compared to definitive, possible, and delayed TRALI as confirmed by adjudicators. We stratified results by age and sex where feasible. Additional exploratory analyses included: stratification of PPV results by whether electronic transfusion data were available, and examination of PPV for TRALI in relation to ‘present on admission’ or ‘principal discharge diagnosis’ flags in the Sentinel electronic data. We also calculated the PPV for mechanical ventilation procedure codes (included invasive and non-invasive mechanical ventilation) identified in the HCA Sentinel database in all potential TRALI cases (codes in Supplement). Finally, we examined the effect of excluding from analysis encounters in which adjudicators were ‘unable to determine’ the TRALI outcome. Exact binomial 95% confidence intervals (Clopper-Pearson) were calculated for the PPV estimates to quantify their precision.

RESULTS

We identified 208 potential TRALI inpatient stays (n=207 patients), at 169 hospitals included in the HCA Sentinel database; medical charts from 195 (94%) of these stays were adjudicated (Figure 1). Unanticipated information technology barriers to chart access explained the majority of unobtainable charts (Reasons charts unobtainable within the adjudication period: n=10, hospital unable to retrieve on standard review platform; n=1, key laboratory or nursing information could not be retrieved; n=1, issues with mapping from data warehouse to chart; n=1, platform access expired and could not be renewed in study timeline). The median age of the potential TRALI patients was 63 years (range=<1 year to 97 years), 54% were female, and 21% were discharged expired. There was no seasonal variation in TRALI cases. The majority of potential TRALI cases were exposed to red blood cells (78%), but many were exposed to other components (platelets: 23%, plasma: 19%, cryoprecipitate: 6%, multiple components: 19%) [Table 2].

Table 2.

Characteristics of potential TRALI cases, Sentinel Distributed Database (SDD)(n=195)

	n=# of potential inpatient stays with TRALI as defined with inpatient diagnosis codes(n=195)
Sex
Female	106 (54%)
Male	89 (46%)
Unknown	-
Age (years) at encounter admission or visit
0–19	10 (5%)
20–34	19 (10%)
35–49	27 (14%)
50–64	45 (23%)
65–79	57 (29%)
80+	37 (19%)
Year of encounter admission
2013	26 (13%)
2014	98 (50%)
2015	71 (36%)
Discharge Disposition
Discharged Alive	154 (79%)
Expired	41 (21%)
Unknown	-
Blood component transfused * †
Red Blood Cells (RBCs) any	153 (78%)
Platelets any	46 (24%)
Plasma any	37 (19%)
Cryoprecipitate any	12 (6%)
Unknown or other	4 (2%)
Multiple components	37(19%)
Transfusion indication * †
RBCs
Operative associated blood loss	26 (13%)
Trauma associated blood loss	9 (5%)
Low hemoglobin in patients with heart failure, coronary artery disease, myocardial Infarction, or shock	20 (10%)
Low hemoglobin in patients with syncope or hypotension/orthostatic hypotension not responsive to fluid resuscitation	6 (3%)
Chronic bone marrow failure (myelodysplasia, leukemia)	27 (14%)
Obstetric associated blood loss	4 (2%)
Other: Anemia, low hemoglobin/non-specific	66 (34%)
Platelets
DIC (Sepsis, trauma, obstetrics)	8 (4%)
Immune thrombocytopenia (ITP, neonatal alloimmune thrombocytopenia)	4 (2%)
Disease associated marrow failure (leukemia, lymphoma, aplasia, myeloproliferative/myelodysplastic disorders, solid tumor metastases)	19 (10%)
Chemotherapy/radiation induced marrow failure	2 (1%)
Cardiac surgery associated bleeding	5 (3%)
Bleeding or anticipated surgery in patients on anti-platelet agents	-
Trauma- or surgery associated massive transfusion	8 (4%)
Congenital thrombocytopenia/thrombocytopathy	-
Other	11 (6%)
Plasma
Abnormal coagulation studies and hemorrhage	25 (13%)
Prophylactic use for elevated PT/APTT	3 (2%)
Warfarin reversal	3 (2%)
Other	16 (8%)
Cryoprecipitate
Fibrinogen deficiency	7 (4%)
Hemophilia A, von Willebrand disease, or F XIII deficiency	-
Uremic coagulopathy	1 (<1%)
Other	12 (6%)

^*Components listed were abstracted by adjudicators as associated with the potential TRALI event; for 3 patients the exact blood component transfused was not available in the chart because transfusion of interest occurred in a different facility (hospital transfer, or outpatient transfusion); additionally, IVIg was the exposure of interest for 1 patient.

^†Please note, multiple blood components were often administered during a transfusion of interest, and patients also often had multiple indications for transfusion.

Adjudication pilot results

Twenty potential-TRALI_A,B,C cases were reviewed during a pilot phase by at least two adjudicators. Adjudicators independently reached the same TRALI case decision for 14 of 20 (70%) charts included in the adjudication pilot. Of the 6 charts with discrepant TRALI case decisions, 4 had minor discrepancies (i.e., not a TRALI case vs unable to determine), and 2 had major discrepancies (i.e., TRALI case vs unable to determine). Minor discrepancies were associated with missing chart information (e.g., illegible notes, or minimal available transfusion information for patients transferred from a non-HCA hospital, etc.) and differences in understanding of when to select ‘not a TRALI case’, ‘ALI not associated with a transfusion’, and ‘unable to determine’ on the adjudication form. When definitions were clarified, all minor discrepancies were easily resolved, and consensus was achieved. Two major discrepancies were due to case complexity. In one instance, one adjudicator did not feel there was enough available chart information to completely rule out TACO and thus listed ‘unable to determine’ while the other was confident that there was enough information to rule out TACO and the case met all TRALI criteria. In the second discrepant case, the blood transfusion of interest was administered at another hospital prior to the patient’s transfer to an HCA hospital. The two cases with major discrepancies were provided to a third adjudicator for a final decision; neither case was determined to meet clinical criteria for TRALI. After this process was complete, 7 TRALI cases met clinical definitions for definitive, possible, or delayed TRALI (35%) in the pilot.

To address these instances going forward, adjudicators recommended clarifying the adjudication form as described above, and implementing a second opinion option, which could be requested when a case was particularly complex. Both recommendations were implemented. During the remainder of the study, adjudicators requested the second opinion option in 17 instances. In 12 instances (71%), there was overall agreement between adjudicators on TRALI case status (i.e., met one of the clinical TRALI case definitions, or was not a case of TRALI), and in 5 (29%) instances the second adjudicator revised the case decision. Seven (41%) of these 17 potential-TRALI cases met clinical definitions for definitive, possible, or delayed TRALI (5 of the 12 cases with agreement, 2 of the 5 with incomplete agreement).

TRALI validation results

Of the 195 potential TRALI inpatient stays, 68 (35%)were confirmed as meeting any clinical definition of TRALI [drawn from 48 hospitals, including 26 (38%) definitive TRALI, 15 (22%) possible TRALI, 27 (40%) delayed TRALI], and 127 were not confirmed by adjudicators. Of potential TRALI inpatient stays that were not confirmed by adjudicators 79 (41%) did not meet the TRALI clinical definition, 36 (n=8%) were classified as ALI not associated with a transfusion (due to timing), and 12 (6%) were adjudicated as unable to determine (e.g., missing key clinical information). Some potential TRALI cases that were not confirmed had other potential diagnoses noted including non-transfusion related circulatory overload or pulmonary edema (n=32, 25%), Transfusion Associated Circulatory Overload (TACO) (n=15, 12%) or Transfusion Related Anaphylaxis (TRA) (n=7, 5%). We also examined reasons for lack of case confirmation among potential TRALI_A cases and observed similar patterns (data not shown). For the one patient with two potential TRALI inpatient stays, neither event was confirmed as TRALI. Instead, both events were determined to be ALI not associated with a transfusion.

PPVs for TRALI inpatient diagnosis codes were <50% in all analyses we conducted [Table 4]. The PPVs for all inpatient TRALI diagnoses recorded in the Sentinel electronic data (potential-TRALI_A,B,C) were 35% overall (68/195, 95% CI: 28–42). The PPV for potential-TRALI_A was 44% (49/111, 95% CI: 35–54) and potential-TRALI_B was 24% (19/79, 95% CI: 15–33). There were no TRALI cases meeting clinical definitions for definitive, possible, or delayed clinical definitions identified by potential-TRALI_C (0/5).

Table 4.

Positive predictive values (PPVs) associated with inpatient diagnosis codes for Transfusion-related acute lung injury (TRALI), Sentinel Distributed Database (SDD)

PPVs associated with inpatient diagnosis codes for TRALI, compared to chart review (N =195)
Stratifications	% (95% Confidence Interval)
Potential-TRALIA,B,C : All TRALI codes below	35% (68/195, 95% CI: 28–42%)
By Potential-TRALIA,B,C recorded in electronic data
Potential-TRALIA: ICD-9-CM code in any position (518.7)	44% (49/111, 95% CI: 35–54%)
Potential-TRALIB: Acute respiratory failure ICD-9-CM code in any position (518.81), WITH code for a blood transfusion reaction (999.80 or 999.89 or E934.7)	24% (19/79, 95% CI: 15–33%)
Potential-TRALIC: Other pulmonary insufficiency (518.82), WITH code for a blood transfusion reaction (999.80 or 999.89 or E934.7)	†
Potential-TRALIB,C (See above)	23% (19/84, 95% CI: 14–33%)
By sex in electronic data
Female	41% (43/106, 95% CI: 31–51%)
Male	28% (25/89, 95% 19–37%)
By age category in electronic data
0–19 years	10% (1/9, 95% CI, 0.3–43%)
20–34 years	32% (6/19, 95% CI: 13–57%)
35–49 years	35% (9/27, 95% CI: 17–54%)
50–64 years	38% (17/45, 95% CI: 24–52%)
65–79 years	39% (22/57, 95% CI: 26–52%)
80+ years	35% (13/37, 95% CI: 20–53%)
By whether any blood transfusion was recorded in electronic data
Transfusion recorded in electronic data	36% (66/182, 95% CI: 29–44%)
No transfusion recorded in electronic data	15% (2/13, 95% CI: 2–45%)
By whether Potential-TRALIA diagnosis code and ‘Principal diagnosis code’ flag was recorded in electronic data
Potential-TRALIA diagnosis code with a ‘Principal diagnosis’ flag recorded in electronic data	44% (4/9, 95% CI: 14–79%)
By whether Potential-TRALIA and ‘Present on admission’ flag was recorded in electronic data
TRALI diagnosis code with a ‘Present on admission’ flag recorded in electronic data	36% (19/53, 95% CI: 23–50%)
By whether mechanical ventilation was recorded in electronic data
Mechanical ventilation recorded in electronic data	39% (44/112, 95% CI: 30–49%)
No Mechanical ventilation recorded in electronic data	29% (24/83, 95% CI: 19–40%)

^*TRALI_A,B,C: TRALI_A, TRALI_B, and TRALI_C;TRALI_A: TRALI, ICD-9-CM code in any position (518.7).

TRALI_B: Acute respiratory failure ICD-9-CM code in any position (518.81), WITH ICD-9-CM code for a blood transfusion reaction (999.80 or 999.89 or E934.7); TRALI_C: Other pulmonary insufficiency (518.82), WITH ICD-9-CM code for a blood transfusion reaction (999.80 or 999.89 or E934.7); TRALI_{B or C}: TRALI_B, or TRALI_C as listed above.

^†0/5 patients meeting potential-TRALIC were confirmed TRALI cases.

We examined the PPV for the specific TRALI code (potential-TRALI_A) when coded as ‘Principal’ or ‘Present on admission’ in the Sentinel database, as compared to all chart confirmed TRALI cases. The PPV for the specific TRALI code (potential-TRALI_A) in principal position was 44% (4/9, 95% CI: 14–79%). The PPV for the potential-TRALI_A code flagged as ‘present on admission’ was 36% (19/53, 95% CI: 23–50%). As we were interested in learning if recording of blood transfusion in Sentinel data would modify the PPV for any TRALI inpatient diagnosis, we stratified PPVs by this information. PPVs for any TRALI diagnosis were slightly higher in encounters for which electronic transfusion records were available (transfusion recorded in electronic data, PPV=36%, 95% CI: 29–44%; transfusion not recorded in electronic data, PPV= 15%, 95% CI: 2–45%), but PPVs remained similar to those observed in primary analyses. When stratifying these PPVs by sex and age category, the PPV of the TRALI electronic algorithm remained below 50% (females: 41%, 95% CI: 31–51%, males: 28%, 95% CI: 19–37%) and in 65 to 79-year age category (65–79 years: 39%, 95% CI: 26–52%). Additional PPV estimates stratified principal diagnosis and present on admission diagnosis flags, mechanical ventilation, and TRALI Criterion are in Table 4.

Table 5 provides PPVs associated with inpatient diagnosis codes for TRALI as compared to definitive, possible, delayed clinical case definitions attained from medical chart review. The overall PPV, when potential-TRALI_A,B,C was compared to definitive TRALI was 13% (26/195, 95% CI: 9–19%). PPV results for potential-TRALI_A,B,C when compared to possible or delayed TRALI were similar to definitive TRALI comparisons [Possible: PPV=8% (15/195, 95% CI: 4–12%) and Delayed: PPV=14% (27/195, 95% CI: 9–20%)]. PPVs of <20% were also observed when each electronic TRALI criterion was compared to confirmed definitive, possible, and delayed TRALI.

Table 5.

Positive predictive values (PPVs) associated with inpatient diagnosis codes for Transfusion-related acute lung injury (TRALI_A,B,C^*) as compared to definitive, possible, delayed clinical case definitions attained from medical chart review, SDD

PPVs associated with inpatient diagnosis codes for TRALI, compared to chart review (N =195)
Stratifications	% (95% Confidence Interval)
Potential-TRALIA,B,C* compared to definitive TRALI	13% (26/195, 95% CI: 9–19%)
Potential-TRALIA,B,C* compared to possible TRALI	8% (15/195, 95% CI: 4–12%)
Potential-TRALIA† compared to delayed TRALI	14% (27/195, 95% CI: 9–20%)
Potential-TRALIA† compared to definitive TRALI	14% (16/111, 95% CI:9–22%)
Potential-TRALI †compared to possible TRALI	10% (11/111, 95% CI: 5 17%)
Potential-TRALIA† compared to delayed TRALI	20% (22/111, 95% CI: 13–28%)
Potential-TRALIB ‡compared to definitive TRALI	13%, (10/79, 95% CI: 6–22%)
Potential-TRALIB‡ compared to possible TRALI	5% (4/79, 95% CI: 1–12%)
Potential-TRALIB‡ compared to delayed TRALI	6% (5/79, 95% CI, 2–14%
Potential-TRALIC§ compared to any TRALI clinical case definition	−(0/5)

^*TRALI_A,B,C: TRALI_A, TRALI_B, and TRALI_C; TRALI_A:

^†TRALI_A, ICD-9-CM code in any position (518.7);

^‡TRALI_B: Acute respiratory failure ICD-9-CM code in any position (518.81), WITH ICD-9-CM code for a blood transfusion reaction (999.80 or 999.89 or E934.7);

^§TRALI_C Other pulmonary insufficiency (518.82), WITH ICD-9-CM code for a blood transfusion reaction (999.80 or 999.89 or E934.7).

Validation of mechanical ventilation among TRALI cases

Adjudicators also confirmed mechanical ventilation before and after the potential TRALI event. Of the 195 charts reviewed, 112 (57%) also had a code for mechanical ventilation (See Supplement for mechanical ventilation procedure codes). Adjudicators found evidence of mechanical ventilation in 95 (PPV: 85%, 95% CI: 77 to 91%) of these encounters. Reasons cited for not being able to locate specific mention of mechanical ventilation in charts included long inpatient stays with lengthy charts, or hospital transfer (i.e., limited information about mechanical ventilation). We reviewed the 17 cases in which mechanical ventilation was not confirmed by adjudicators. Adjudicators noted 4 (24%) of these patients were placed on Bilevel Positive Airway Pressure (BiPAP) during their inpatient stay, 1 (5%) was transferred from another hospital and placed on a non-rebreather O2 mask, 3 (18%) were transferred from a different hospital, and no mechanical ventilation information related to the potential ALI event was located for 9 (53%) inpatient stays. Finally, adjudicators collected information about whether mechanical ventilation occurred before or after ALI. Of the 95 patients with evidence of mechanical ventilation in Sentinel electronic data and chart data, 48 (51%) were ventilated after the ALI event of interest, 22 (23%) prior to the ALI event of interest and adjudicators were unable to determine the specific timing in relation to the ALI event in 25 (26%) patients.

DISCUSSION

In this chart validation study which relied on data from a protocol-based assessment of TRALI after RBC, platelet, and/or plasma transfusion⁸, we evaluated the validity of inpatient diagnosis codes for TRALI within the HCA Sentinel Distributed Database. Our study shows PPVs below 50% with large confidence intervals in an all analyses conducted. Use of the specific TRALI diagnosis code resulted in the highest PPV of 44%. Stratification of results by age group, sex, presence or absence of transfusion information in electronic data, TRALI as a principal discharge diagnosis or flagged as present on admission did not substantially increase PPVs. The complexity of the TRALI diagnosis and coding practices, the low prevalence of TRALI, and/or gaps in recognition or recording of transfusion reactions may have contributed to the study findings.

Given that the HCA Sentinel database includes all diagnoses captured during an inpatient stay, rather than only billable diagnoses, it is possible some potential TRALI inpatient stays were coded with TRALI when TRALI was on the differential diagnosis. Of the 127 potential TRALI cases that were not confirmed by adjudicators, approximately 17% (n=22) were suspected to be cases of TACO or TRA. Other potential diagnoses included non-transfusion related circulatory overload or pulmonary edema (n=32, 25%), or ALI not associated with a transfusion (n=36, 28%). In contrast, evaluation of the validity of inpatient procedure codes for mechanical ventilation showed a high PPV when compared to medical charts (PPV: 85%, 95% CI: 77 to 91%). However, the PPV of TRALI diagnosis codes remained low even among patients with mechanical ventilation recorded in electronic data, (PPV: 39%, 95% CI: 30 to 49%). These findings have implications for future work, as consideration of differential diagnoses may be key when designing electronic algorithms for use in Sentinel EMR inpatient data and similar data sources.

Results from this validation of TRALI inpatient diagnosis codes in a database including 169 hospitals underscores the importance of medical record review. TRALI is a rare condition with low prevalence in the transfused population and the PPV of the TRALI electronic algorithm was poor in all analyses conducted (<50%). As PPVs are influenced by disease prevalence,¹⁹ the low prevalence of TRALI in the study database may also have contributed to our findings. To our knowledge, this is one of few studies to date utilizing a large electronic database in combination with medical record review to validate diagnosis codes for TRALI. In another population-based study conducted within a large administrative database, the PPV of the TRALI specific code (ICD-9-CM 518.7) was 35%, which is similar the PPV we found in subgroup analyses examining only the TRALI specific code (n=49 confirmed cases meeting definitive, possible or delayed TRALI clinical definitions).⁹

When designing this study, we considered that potential TRALI cases might be missed if a transfusion reaction was not recognized by physicians and coded. A study by the BEST Collaborative conducted in a similar time-period as our study, described limited knowledge of transfusion reactions such as TRALI amongst internal medicine residents.²⁰ Thus, inadequate or ineffective training in transfusion medicine may also be a contributing factor to our study findings. We attempted to mitigate this limitation by focusing not only on the TRALI specific ICD-9-CM code (ICD-9-CM, 518.7), but also on specific respiratory failure codes in combination with an ICD-9-CM code for a transfusion reaction. This approach yielded an additional 19 confirmed TRALI cases (28% of cases meeting clinical definitions for definitive, possible, or delayed TRALI), but the gains in sensitivity compromised specificity, with the overall performance of our aggregated TRALI outcome (including the ICD-9-CM code for TRALI and codes for respiratory failure in combination with codes for a transfusion reaction) falling from 44% to 35%. However, this strategy would not have captured cases in which no ALI or transfusion diagnosis codes were documented and coded during an inpatient stay.

A strength of this study was the ability to identify potential TRALI cases in the HCA Sentinel database that included 169 hospitals and to review all available copies of medical records with potential TRALI codes during a 2-year period. The proportion of retrievable medical records was larger than expected based on prior Sentinel assessments (94% of potential TRALI charts retrieved, compared to 68–80% of potential cases in prior Sentinel chart validation assessments). We postulate this is likely due to the Sentinel inpatient data partner type and structure (i.e., large hospital network versus administrative claims partners).

While other studies have developed electronic screening algorithms for transfusion related pulmonary complications in EMR data²¹, unfortunately we were unable to examine these due to lack of some clinical information in the database. While the HCA Sentinel database is also EMR based, at the time of this study laboratory and other relevant clinical information such as PaO2/FiO2, timing of ALI diagnosis or chest x-ray interpretations was not available in the database (i.e., only available in medical charts). While not available in our study database, many clinical elements from underlying EMR data such as laboratory data or ALI diagnosis timing could be useful to improve the utility of TRALI diagnosis codes in administrative database studies. Although we standardized the chart adjudication process, in some instances adjudicators were unable to determine TRALI case status due insufficient clinical information.

We also only included ICD-9-CM codes, as the time-period of interest for this protocol-based assessment precedes the transition to ICD-10-CM coding in the US. Although the US transitioned to the ICD-10-CM coding system since we completed this study, the specific TRALI ICD-10-CM code (J95.84) maps directly to the ICD-9-CM code for TRALI. While we could not verify, we would expect similar PPV results for studies utilizing diagnosis codes to identify TRALI in the ICD-10-CM era.

Overall, our study’s low PPV could be due to various factors including the complexity of TRALI diagnosis, coding practices, the low prevalence of TRALI, and/or gaps in recognition or recording of transfusion reactions. While all 169 hospitals included in our study were drawn from one hospital system, we anticipate these results would generalize to other hospitals in similar time frames within the United States. Large confidence intervals could indicate the need for larger population-based investigations, which may help address potential limitations and further develop and validate TRALI coding algorithms including use of ICD-10-CM codes. Future research is needed to assess ICD-10-CM codes for TRALI, and alternative ways of accurately identifying TRALI cases in large EMR and administrative databases.

Table 3.

Disposition of potential Transfusion-Related Acute Lung Injury (TRALI) inpatient stays and transfusion information ascertained in the Sentinel Distributed Database (SDD)

TRALI case disposition	n= of potential inpatient stays with TRALI as defined with inpatient diagnosis codes (N=195)
Met clinical definition of TRALI* (n=68)
Definitive TRALI	26 (13%)
Possible TRALI	15 (8%)
Delayed TRALI	27 (14%)
Not a case of TRALI (n=127)
Multiple clinical criteria not met and/or other potential diagnoses†	69 (35%)
No evidence of bilateral infiltrates	6 (3%)
Evidence of left atrial hypertension	4 (2%)
Acute Lung Injury not associated with transfusion due to timing‡	36 (18%)
Insufficient information to determine TRALI case disposition§	12 (6%)

^*Clinical criteria included: 1) No evidence of acute lung injury (ALI) prior to transfusion for definitive TRALI, possible and delayed TRALI could have a temporal relationship to an ALI risk factor 2) ALI onset during or within 6 hours of transfusion for definitive TRALI, or onset between 6–72 hours for delayed TRALI 3) Hypoxemia 4) Radiographic evidence of bilateral infiltrates 5) No evidence of left atrial hypertension 6) For definitive TRALI, no temporal relationship to an alternative risk factor for ALI during or within 6 hours of completion of transfusion.

^†Other potential diagnoses included suspected non-transfusion related circulatory overload or pulmonary edema n=21, suspected transfusion related circulatory overload (TACO) n=5 or transfusion related anaphylaxis (TRA) n=7.

^‡Additional potential diagnoses among patients with ALI not associated with a transfusion (due to timing) included TACO (n=9) and non-transfusion related circulatory overload or pulmonary edema (n=12).

^§Reasons for meeting this designation included charts missing critical clinical information (n=11), and inability to rule out TACO with existing information (n=1).

Key Points:

• In 169 hospitals we identified 208 potential TRALI cases, reviewed 195 charts, and confirmed 68 (35%) cases met TRALI clinical definitions.

• As many potential TRALI cases identified with diagnosis codes did not meet clinical definitions, medical record confirmation may be prudent.