The accuracy of burn diagnosis codes in health administrative data: A validation study

Abstract

Background:

Health administrative databases may provide rich sources of data for the study of outcomes following burn. We aimed to determine the accuracy of International Classification of Diseases diagnoses codes for burn in a population-based administrative database.

Methods:

Data from a regional burn center’s clinical registry of patients admitted between 2006–2013 were linked to administrative databases. Burn total body surface area (TBSA), depth, mechanism, and inhalation injury were compared between the registry and administrative records. The sensitivity, specificity, and positive and negative predictive values were determined, and coding agreement was assessed with the kappa statistic.

Results:

1215 burn center patients were linked to administrative records. TBSA codes were highly sensitive and specific for ≥10 and ≥20% TBSA (89/93% sensitive and 95/97% specific), with excellent agreement (k, 0.85/k, 0.88). Codes were weakly sensitive (68%) in identifying ≥10% TBSA full-thickness burn, though highly specific (86%) with moderate agreement (k,0.46). Codes for inhalation injury had limited sensitivity (43%) but high specificity (99%) with moderate agreement (k, 0.54). Burn mechanism had excellent coding agreement (k, 0.84).

Conclusions:

Administrative data diagnosis codes accurately identify burn by burn size and mechanism, while identification of inhalation injury or full-thickness burns is less sensitive but highly specific.

1. Introduction

The care of the burn-injured patient has evolved over the past several decades, such that all but the most devastating injuries are survivable [1]. Alongside these improvements, a need to understand the long-term outcomes of burn patients has emerged to inform development of interventions to mitigate long-term morbidity following burn. However, a paucity of data regarding the long-term burden of burn exists [2,3].

Our current knowledge of outcomes after burn derives from studies hampered by small cohort sizes and loss to follow up with limited data available beyond 2 years after injury [3–5]. Many of these challenges can be overcome through population-based studies that utilize health administrative data. These data facilitate the long-term study of burn-injured individuals through the analysis of health care utilization data. Such data have been successfully utilized to study long-term mortality following major burn [6,7]. In other cohorts, these data have been used to evaluate self-harm risk, cancer risk, and to describe long-term health trajectories [7–12].

A key limitation of administrative data is the potential for misclassification bias. The identification of patients with specific conditions, such as burn, depends on the accuracy with which these conditions are coded within administrative databases. In Canadian administrative databases, diseases are coded using the tenth revision of the International Classification of Diseases (ICD-10) [13]. While the validity of these codes has been demonstrated for many conditions [14–18], the accuracy of codes for burn size, depth, body region, mechanism and inhalation injury is unknown. Prior to the use of administrative databases to study the long-term outcomes of burn-injured patients, it is important that these diagnosis codes are evaluated and proven sufficiently accurate. Therefore, the objective of this study was to determine the accuracy of ICD-10 diagnosis codes for burn and inhalation injury in a provincial health administrative database, using comprehensive clinical data from the same patients in a second regional clinical database.

2. Methods

2.1. Study design

This study is reported in accordance with the guidelines for diagnostic and validation studies of health administrative data[19]. Records derived from a clinical burn registry reflecting the acute care of burn patients admitted to the largest burn center in Canada were linked to administrative discharge data to validate ICD-10 diagnosis and external cause codes for burn. This study was approved by the Research Ethics Board of Sunnybrook Health Sciences Centre.

2.2. Data sources

Two distinct healthcare datasets were used for the study:(1) a provincial health administrative database, and (2) a comprehensive regional clinical registry. The clinical burn registry contains data regarding patient demographics, injury characteristics, and inpatient complications for all patients admitted to the burn center. Data were entered into the registry by a trained burn registrar and collected in accordance with the National Burn Repository data standard [20]. TBSA estimations were derived from the Lund and Browder chart completed by the admitting burn surgeon on admission. Presence of inhalation injury was documented on admission based on bronchoscopic evaluation performed by the admitting burn surgeon. Mechanism of injury was recorded in the medical record based on the admission history taken by the surgeon (self-reported by patient or collateral history from witnesses).

To determine the accuracy of burn diagnosis codes in administrative data, the burn registry was linked to the Registered Persons Database (RPDB). The RPDB contains demographic information on all residents of Ontario, and is linked by unique identifier to the Discharge Abstract Database (DAD), a population-based administrative database that records all discharges from acute care hospitals in the province of Ontario after the year 1991. Diagnoses responsible for admission are coded in the DAD according to a modification of the 10th revision of the International Classification of Diseases, and each admission can contain up to 25 recorded diagnoses. The Canadian Institute for Health Information coding standards require that data abstractors assign burn diagnosis codes according to each affected body area, along with codes reflecting the TBSA and external cause of injury[21]. These data were made available through the Institute of Clinical and Evaluative Sciences (ICES). ICES is a prescribed entity under the Province of Ontario’s privacy law and holds a large proportion of the administrative health data collected in Ontario. These datasets were linked using unique encoded identifiers and analyzed at ICES.

2.3. Validation cohort

We used the clinical registry to identify all patients aged 16 years and older who were admitted to the burn center for the treatment of acute burn between January 1, 2006 and December 31, 2013. Patients were excluded if data regarding their burn extent, presence of inhalation injury, or mechanism of injury were missing in the clinical burn registry. The TBSA estimation, % full-thickness burn estimation, mechanism of burn, and diagnosis of inhalation injury from the burn registry were considered the reference standard.

2.4. Administrative data and record linkage

Once the validation cohort was assembled, each patient record was linked to administrative data through both deterministic and probabilistic linkage. Deterministic linkage was performed based on hospital health record number and date of birth. Where deterministic linkage was not possible, probabilistic linkage on patient name was used.

2.5. Validation

The ICD-10 diagnosis codes related to the identification of burn, characterization of burn extent, depth and mechanism, and identification of inhalation injury were defined a priori and used to derive case definitions for validation (Table 1).

Table 1 –

Case definitions for validation.

	ICD-10CAa Diagnosis code(s)
≥10% TBSAb	T31.10-T31.99
≥20% TBSAb	T31.20-T31.99
Inhalation injury	T27.0-T27.3
TBSAb
0–10	T31.0
11–20	T31.1
21–30	T31.2
31–40	T31.3
41–50	T31.4
51–60	T31.5
61–70	T31.6
71–80	T31.7
81–90	T31.8
>90	T31.9
≥10% full-thickness TBSAb	T31.12, T31.22–23, T31.32–34, T31.42–45, T31.52–56, T31.62–67, T31.72–78, T31.82–89, T31.92–99
Burn mechanism
Flame	X00-X09
Contact	X10-X19
Electrical	W85-W87

^aICD-10CA, International Statistical Classification of Diseases and Related Health Problems, 10th revision, Canada.

^bTBSA, total body surface area.

We compared case definitions derived from the DAD to those derived from the reference standard. Diagnoses were coded as binary variables for each patient; for example, inhalation injury was coded as present or absent in the clinical registry and as positive or negative based on DAD diagnoses. For each case definition, each patient was classified as true positive (registry present, DAD positive), true negative (registry absent, DAD negative), false positive (registry absent, DAD positive), and false negative (registry present, DAD negative).

We sought to determine whether the administrative data were accurate in identifying patients with ≥10% TBSA, ≥20% TBSA, ≥10% full-thickness burn, and inhalation injury, as compared to the clinical registry. These TBSA cutoffs were chosen as they identify patients who meet criteria for referral to a burn center (≥10% TBSA) [22] and major burn (≥20% TBSA). This was assessed in two ways: (1) by determining the agreement for each case definition between the clinical registry and administrative data using the kappa coefficient, and (2) by calculating the sensitivity, specificity, and positive and negative predictive values (PPV, NPV). Accuracy in burn mechanism between the clinical registry and administrative data was assessed by exact agreement. For each mechanism (flame, electrical, contact), coding agreement was assessed using the kappa coefficient. To understand how accurately TBSA estimates are coded, we evaluated agreement between TBSA estimates within each decile of % TBSA between the clinical registry and administrative data using the kappa coefficient.

For our purposes, accuracy of 80% or greater and a kappa coefficient of 0.60 or greater were considered targets for validation [23,24]. Based on prior studies of the validity of other diagnoses codes in the same database, we expected these codes to have sensitivity in excess of 75% and specificity in excess of 90% [14,15,18,25–27].

2.6. Statistical analysis

Descriptive statistics were used to describe the validation cohort. Means and standard deviations, or medians and interquartile ranges were calculated for continuous variables, as appropriate. For discrete variables, absolute frequencies were measured. Sensitivity, specificity, PPV, NPV, accuracy (% exact agreement) and the kappa coefficient were calculated for each case definition. Agreement was considered excellent where kappa was greater than 0.81; substantial where kappa ranged between 0.61–0.80 and moderate where kappa was between 0.41–0.60 [28]. All analyses were conducted on de-identified data using SAS Version 9.4 (Cary, NC).

3. Results

We identified 1252 patients in the burn registry meeting inclusion criteria. Of these, 17 were excluded prior to linkage due to missing data. We were able to deterministically link 89% (n=1099) patients to administrative data; the remaining 11% (n=136) were linked probabilistically (Fig. 1).

Fig. 1 –

Derivation and linkage of validation cohort. ICD-10, International Classification of Diseases, 10th edition.

The patient and injury characteristics of the validation cohort are presented in Table 2. Most patients (73%, n=890) were men, and the median age of patients was 50 years. The median TBSA was 7.8%, with the most frequent mechanism being flame burns (61%, n=731).

Table 2 –

Patient and injury characteristics of validation cohort.

Patient characteristics
Median age, years (IQRa)	50 (35–62)
Male (%)	890 (73)
Injury characteristics
Median % TBSAb (IQR)	7.8 (3.1–16.5)
Median % full-thickness TBSAb (IQRa)	0.5 (0–5)
Inhalation injury (%)	195 (16)
Mechanism (%)
Flame	731 (61)
Contact	385 (32)
Electrical	90 (7)
Median APACHEc II Score (IQRa)	3 (1–7)
In-hospital mortality (%)	12 (1)

^aIQR, interquartile range.

^bTBSA, total body surface area.

^cAPACHE, acute physiology and chronic health evaluation.

In the burn registry, the prevalence of ≥10% TBSA burn was 41% (n=502); the prevalence of ≥20% TBSA burn was 20% (n=245). 17% of patients had ≥10% TBSA full-thickness burns. The prevalence of inhalation injury was 16% (n=196).

3.1. Burn size

The diagnosis codes in the administrative data identified both ≥10% TBSA and ≥20% TBSA with high sensitivity and specificity (Table 3). In identifying patients with ≥10% TBSA, administrative data were 89% sensitive and 95% specific; for ≥20% TBSA, administrative data were 93% sensitive and 97% specific. Agreement between the clinical registry and administrative data in discriminating between ≥10/20% and <10/20% TBSA injury was excellent (k=0.85 and 0.88 respectively). Accuracy (exact agreement) between TBSA estimates within each decile of % TBSA was 87%. Coding agreement by TBSA decile ranged from moderate to excellent (k, 0.58–0.86), and generally increased with increasing burn size (Table 4).

Table 3 –

Estimates of diagnostic accuracy of administrative data.

	Sensitivity (%, 95%CIa)	Specificity (%, 95% CIa)	Positive predictive value (%, 95% CIa)	Negative predictive value (%, 95% CIa)
≥10% TBSAb	89 (86–92)	95 (93–97)	93 (90–95)	93 (91–95)
≥20% TBSAb	93 (89–96)	97 (95–98)	88 (84–92)	98 (97–99)
≥10% full-thickness TBSAb	68 (61–74)	86 (83–88)	49 (43–55)	93 (91–95)
Inhalation injury	43 (36–50)	99 (98–100)	93 (88–98)	90 (88–92)
Burn mechanism
Flame	93 (87–92)	91 (88–93)	94 (92–95)	85 (82–88)
Contact	89 (85–92)	93 (91–95)	86 (82–89)	95 (93–96)
Electrical	76 (65–84)	99 (99–100)	93 (85–98)	98 (97–99)

^aCI, confidence interval.

^bTBSA, total body surface area.

Table 4 –

Prevalence and accuracy of specific injury characteristics.

	Prevalence (%)		Accuracy (%)	Kappa (95%CIa)	Agreement
	Burn registry	Administrative data
≥10% TBSAb	41	40	93	0.85 (0.82–0.88)	Excellent
≥20% TBSAb	20	21	96	0.88 (0.84–0.91)	Excellent
≥10% full-thickness TBSAb	17	23	83	0.46 (0.40–0.52)	Moderate
Inhalation injury	16	7	90	0.54 (0.47–0.61)	Moderate
Mechanism			88	0.84 (0.81–0.87)	Excellent
Flame	61	60	92
Contact	32	34	93
Electrical	7	6	98
%TBSAb			87
0–10	58	60	92	0.83 (0.80–0.86)	Excellent
11–20	21	19	92	0.74 (0.70–0.79)	Substantial
21–30	9	10	96	0.77 (0.70–0.83)	Substantial
31–40	4	4	98	0.77 (0.68–0.87)	Substantial
41–50	3	3	99	0.73 (0.60–0.86)	Substantial
51–60	2	2	99	0.72 (0.57–0.86)	Substantial
61–70	1	2	99	0.58 (0.38–0.78)	Moderate
71–80	0.9	0.8	99.5	0.70 (0.47–0.93)	Substantial
81–90	0.3	0.7	99.7	0.67 (0.36–0.97)	Moderate
>90	0.7	0.5	99.8	0.86 (0.66–1)	Excellent

^aCI, confidence interval.

^bTBSA, total body surface area.

We also evaluated the sensitivity of diagnosis codes to identify patients with full-thickness burns of ≥10% TBSA. These codes were moderately sensitive (68%) and highly specific (86%), with moderate agreement between the burn registry and administrative data (k, 0.46).

3.2. Inhalation injury

The administrative data diagnoses codes were poorly sensitive (43%) but highly specific (99%) for inhalation injury. Agreement between the clinical registry and administrative data was moderate (k=0.54).

3.3. Burn mechanism

We first sought to determine overall agreement between the clinical registry and administrative data for burn mechanism, and then to determine coding accuracy for flame, contact, and electrical burns. An external cause of injury code was missing in the administrative data for 9 patients (0.7%). Among the remaining patients, overall agreement was excellent (k, 0.84). Diagnoses codes were highly sensitive for flame and contact burns (93% and 89%), while moderately sensitive for electrical burns (76%). Specificity for all 3 mechanisms was excellent (91–99%, Table 3).

4. Discussion

We conducted a retrospective validation study of the accuracy of burn diagnosis codes in a health administrative database in Ontario, Canada. Our findings demonstrate that diagnoses codes were highly sensitive and specific for burn size and mechanism, while burn depth and inhalation injury were less sensitive and specific. The high accuracy of diagnoses codes for burn extent and burn mechanism suggests that administrative data can be reliably used to identify cohorts of burn-injured patients on the basis of either TBSA or mechanism. The diagnoses codes had the greatest sensitivity and specificity in discriminating between greater and less than 20% TBSA injury, with lower but strong agreement below this threshold. We observed greater reliability with increasing deciles of % TBSA, consistent with prior studies demonstrating that the accuracy of burn size estimations increases with increasing burn extent[29].

The diagnoses codes for inhalation injury had poor sensitivity; despite high specificity, only 43% of patients with inhalation injury were identified as such in the administrative data. The significant difference in prevalence of inhalation injury between the burn registry and administrative data (16% vs. 7%) suggests that inhalation injury is not coded as commonly during routine medical records abstraction for administrative purposes. This may reflect low clinical suspicion and confirmatory diagnostic testing, poor documentation of inhalation injury in the medical record, or incomplete coding of inhalation injury as a secondary diagnosis by the data abstractor. As a result, these codes cannot be reliably used to identify burn-injured patients with inhalation injury.

We found that diagnoses codes for ≥10% full-thickness burns were weakly sensitive and highly specific, with moderate agreement. The characterization of patients based on the extent of full-thickness burn is limited by the nature of ICD-10 diagnosis codes for burn depth. This is expressed in deciles, such that both 0% and 9% full-thickness have the same diagnosis code; this same code is used when burn depth is unspecified. As a result, for patients with <10% TBSA, it is impossible to discriminate between patients with and without full-thickness burns. This limits the ability of researchers to distinguish patients with potentially significant functional burns that will very likely undergo surgery, i.e. full-thickness burns to the face, feet, or hands, from those with more minor burns.

A key limitation of this work is its generalizability to administrative databases in other settings. Furthermore, data derived from the largest regional burn center in Canada were used as the reference standard. Diagnoses codes in administrative data are assigned based on abstraction from the medical record. Therefore, the accuracy of TBSA estimations in administrative data depends on the accuracy of estimations in the medical record. While we demonstrated high accuracy of these codes in our study, this reflects comparison to estimations performed by expert burn surgeons. TBSA estimations have previously been demonstrated to vary widely, even among burn surgeons [30–32]. It is reasonable to assume that TBSA estimations might be less accurate outside of regional burn centers. This will be a limitation of future studies utilizing health administrative data to identify burn-injured patients, and might result in misclassification of patients based on their burn extent. To the best of our knowledge, this is the first validation study of burn diagnosis codes, limiting our ability to compare our results to studies in other settings.

5. Conclusion

This validation study linked a clinical burn registry to population-based health administrative database in order to assess the validity of diagnoses codes for burn size, depth, mechanism, and inhalation injury. We have demonstrated that diagnosis codes for TBSA and burn mechanism are highly sensitive and specific, with excellent reliability between the administrative data and burn registry. Our findings suggest that patients with major burn can be reliably identified in administrative data. This should encourage investigators to consider the potential advantages of using health administrative data to further contribute to our understanding of burn and its impact at both the patient and health care system levels.