Abstract
Excess mortality risk imparted by acute respiratory failure in children is unknown. We determined excess mortality risk associated with mechanically ventilated acute respiratory failure in pediatric sepsis. Novel ICD10-based algorithms were derived and validated to identify a surrogate for acute respiratory distress syndrome to calculate excess mortality risk. Algorithm-identified ARDS was identified with specificity of 96.7% (CI 93.0 – 98.9) and sensitivity of 70.5% (CI 44.0 – 89.7). Excess risk of mortality for ARDS was 24.4% (CI 22.9 – 26.2). Development of ARDS requiring mechanical ventilation imparts modest excess risk of mortality in septic children.
BACKGROUND
Interventional trials in paediatric Acute Respiratory Distress Syndrome (ARDS) have largely been unsuccessful in improving mortality, despite improvements in hypoxemia or ventilator days.1 2 One explanation for this phenomenon is that patients are dying from factors related to the their chronic comorbidities or ARDS inciting aetiology, with sepsis being a frequent cause of ARDS, but not from acute respiratory failure (ARF) itself, making mortality less modifiable by ARDS-directed interventions.3 Efforts to dissect this relationship have examined attributable mortality, or the fraction of mortality due to ARDS in an at-risk population who did not die of ARDS. A recent adult study determined ARDS-attributable mortality, in a population of septic adults, at 27–37%.4 A comparable analysis in paediatrics is limited by the lack of available sepsis trials, small cohort sizes, accurate ARDS determination and lower mortality rates. Determination of excess mortality risk,5 defined as the increased mortality from ARDS development in children with sepsis, may be possible through the use of a large administrative dataset. The determination of ARDS in these datasets, however, is plagued by misclassification bias.6 One way to mitigate this bias is via electronic health record (EHR) confirmation of ICD-derived diagnoses. We therefore developed an ICD-10 diagnosis and procedural code algorithm maximising specificity to accurately identify an administrative database surrogate for ARDS defined as ARF from a pulmonary, sepsis or shock aetiology requiring invasive mechanical ventilation (IMV)≥24 hours. We assessed construct validity through determination of the excess mortality risk of algorithm-identified ARDS, hypothesising that ARDS would account for at least 20% of excess risk of mortality in this population.
METHODS
Study design
This retrospective study utilised a single centre cohort for algorithm-identified ARDS generation and a multicenter cohort to assess algorithm performance and to determine excess mortality risk. This study was deemed exempt and need for consent waived by the Children’s Hospital of Philadelphia (CHOP) Institutional Review Board (IRB). Data for this study were obtained from the Paediatric Health Information System (PHIS).7
Algorithm generation
We identified all children from CHOP with ICD-10 codes of interest in PHIS.7 The entire cohort was subjected to a random number generation algorithm and subjects 1–200 were used to create a reference standard through EHR review, based on power calculations to achieve 90% algorithm specificity with a hypothesised prevalence of 10%. Iterative ICD-10 algorithms (table 1) were tested against this standard initially adding ICD codes that are common risk factors for ARDS and finally excluded key neurologic diagnoses that had prolonged intubations without other features of ARDS.
Table 1.
International Classification of Disease-10 Codes and Algorithms
| Diagnosis | ICD-10 codes |
|---|---|
| Acute Respiratory Distress Syndrome | J80*, J81* |
| Asthma | J45* |
| Pneumonia | J09*, J10*, J11*, J12*, J13*, J14*, J15*, J16*, J17*, J18* |
| Bronchiolitis | J20*, J21*, J22*, J40 |
| Pneumonitis | J68*, J69* |
| Other respiratory failure | J96*, R06.03 |
| Mechanical Ventilation (Diagnosis) | Z99.11 |
| Sepsis/ Shock | A41*, R57*, R65.2* |
| Seizure and Status Epilepticus | G40.901, R56.9*, G40.001, G40.101, G40.111 |
| Procedure | ICD-10 Code |
| Invasive Mechanical Ventilation<24 hours | 5A19035 |
| Invasive Mechanical Ventilation≥24 hours | 5A19045, 5A19055 |
| Non-Invasive Ventilation<24 hours | 5A09035 |
| Non-Invasive Ventilation≥24 hours | 5A09045, 5A09055 |
| Endotracheal Intubation | 0BH17EZ |
| Algorithm | Code Composition |
| 1) ≥24 Hour Ventilation Codes | 5A19045 or 5A19055 or 5A09045 or 5A09055 |
| 2) ≥ 24 hours Ventilation Plus Respiratory and Sepsis Codes | (5A19045 or 5A19055 or 5A09045 or 5A09055) + (Any Diagnosis Code for Acute Respiratory Distress Syndrome, Asthma, Pneumonia, Bronchiolitis, Pneumonitis, Other respiratory failure, Sepsis/ Shock) |
| 3) ≥ 24 hours Ventilation Plus Respiratory and Sepsis Codes, Exclude Neurologic Codes | (5A19045 or 5A19055 or 5A09045 or 5A09055) + (Any Diagnosis Code for Acute Respiratory Distress Syndrome, Asthma, Pneumonia, Bronchiolitis, Pneumonitis, Other respiratory failure, Sepsis/ Shock) + (NONE of G40.901 or R56.9*or G40.001 or G40.101 or G40.111) |
| 4) ≥24 Hour IMV Codes | 5A19045 or 5A19055 |
| 5) ≥24 Hour IMV Plus Respiratory and Sepsis Codes | (5A19045 or 5A19055) + (Any Diagnosis Code for Acute Respiratory Distress Syndrome, Asthma, Pneumonia, Bronchiolitis, Pneumonitis, Other respiratory failure, Sepsis/ Shock) |
| 6) ≥24 Hour IMV Plus Respiratory and Sepsis Codes, Exclude Neurologic Codes (Algorithm-Identified ARDS) | (5A19045 or 5A19055) + (Any Diagnosis Code for Acute Respiratory Distress Syndrome, Asthma, Pneumonia, Bronchiolitis, Pneumonitis, Other respiratory failure, Sepsis/ Shock) + (NONE of G40.901 or R56.9*or G40.001 or G40.101 or G40.111) |
Denotes that any suffix code was allowed.
Excess risk of mortality
We determined in-hospital mortality for each diagnostic category: (1) respiratory diagnosis without sepsis or ARF; (2) severe sepsis without ARF; (3) algorithm-identified ARDS. We generated unadjusted Kaplan-Meier survival curves, stratified by underlying diagnosis category, with differences between survival curves determined using log-rank testing and log-rank test for trends. We utilised Cox proportional hazard modelling, censored at 28 days, to determine the hazard of in-hospital mortality for algorithm-identified ARDS. Multivariable Cox proportional hazard models included age, race/ethnicity, presence of complex chronic condition,8 and hospital region as confounders. We determined excess risk of mortality as an excess log hazard using the proportional hazard beta-coefficients in the following manner9:
Respiratory diagnosis without sepsis or ARF: Reference
Sepsis with ARDS: β1
Sepsis without ARDS: β2
Excess log hazard fraction: (β1 – β2)/ β1
95% CI: bootstrap with 100 simulations
Data analysis was performed using Stata 17 (StataCorp, College Station, TX) and GraphPad Prism 9 (GraphPad Software, Inc.).
RESULTS
Algorithm derivation
Utilising ICD-10 codes for respiratory diseases and sepsis, procedural codes for invasive mechanical ventilation and excluding patients with specific neurologic disease codes, an algorithm (table 1, Algorithm 6) demonstrated excellent specificity (96.7%, 95% CI 93.0 to 98.9) with good sensitivity (70.5%, 95% CI 44.0 to 89.7) to identify algorithm-identified ARDS (table 2).
Table 2.
Test Characteristics for Specified Algorithms
| Algorithm | Sensitivity | Specificity | Positive predictive value | Negative predictive value | Accuracy | |
|---|---|---|---|---|---|---|
| ARF Requiring NIV or IMV | ||||||
| 1 | ≥ 24 Hour Ventilation Codes | 95.5 (84.5–99.4) | 91.0 (85.4 to 95.0) | 75.0 (61.6–85.6) | 98.6 (95.0–99.8%) | 92.0 (87.3–95.4) |
| 2 | ≥ 24 hours Ventilation Plus Respiratory and Sepsis Codes | 95.3 (84.2–99.4) | 90.3 (84.5 to 94.5) | 73.2 (59.7–84.2) | 98.6 (95.0–99.8) | 90.5 (85.6–94.2) |
| 3 | ≥ 24 hours Ventilation Plus Respiratory and Sepsis Codes, exclude neurologic codes | 81.4 (66.6–91.6) | 94.8 (90.1 to 97.7) | 81.4 (66.6–91.6) | 94.8 (90.1–97.7%) | 91.0 (86.1–94.6) |
| ARF Requiring IMV for≥24 hours only from Pulmonary, Sepsis, or Shock Etiologies | ||||||
| 4 | ≥24 Hour IMV Codes | 76.5 (50.1–93.2) | 94.0 (89.5 to 97.0) | 54.2 (32.8–74.4) | 97.7 (94.2–99.4) | 92.5 (87.9–95.7) |
| 5 | ≥24 Hour IMV Plus Respiratory and Sepsis Codes | 76.5 (50.1–93.2) | 94.0 (89.5 to 97.0) | 54.2 (32.8–74.4) | 97.7 (94.2–99.4) | 92.5 (87.9–95.7) |
| 6 | Algorithm-Identified ARDS (≥24 Hour IMV Plus Respiratory and Sepsis Codes and exclude neurologic codes) | 70.5 (44.0–89.7) | 96.7 (93.0 to 98.9) | 66.7 (41.0–86.7) | 97.3 (93.7–99.1) | 94.5 (90.4–97.2) |
Excess risk of mortality of ARDS in sepsis
Sepsis without algorithm-identified ARDS mortality was 6.6% (1,313/19,810; 95% CI 6.3% to 7.0%). Septic patients with algorithm-identified ARDS had highest mortality, 12.7% (8,600/67,930; 95% CI 12.4% to 12.9%). In 28-day survival analysis, a significant trend in worsened survival was seen from respiratory diagnosis to sepsis to algorithm-identified ARDS (logrank Chi2 p<0.0001) (figure 1). In adjusted regression, both sepsis without ARDS (aHR=8.25, CI 7.38 to 9.21) and with algorithm-identified ARDS (aHR=16.4, CI 15.0 to 17.8) increased mortality, with algorithm-identified ARDS imparting the highest risk (p<0.0001). The excess risk of mortality of patients with algorithm-identified ARDS in this population of septic children was 24.4% (CI 22.9% to 26.2%). We performed two sub-analyses, first acknowledging that care for children with ARDS may have evolved over the course of our study we performed a sensitivity analysis accounting for year as a clustering variable and found the same excess mortality point estimate (24.4%, CI 22.6% to 26.7%). Second, 90-day survival analysis demonstrated an excess risk of mortality for ARDS of 21.4% (CI 20.1% to 23.0%).
Figure 1.
Kaplan-Meier Survival Curve for Children with Respiratory Diagnoses without Sepsis, Septic Children, and Septic Children with ARDS. The unadjusted Kaplan-Meier survival curves, censored at 28 days, for each diagnosis category are presented. The Log-rank test for trend is presented showing the significant difference between survival for each level of diagnosis category. Patient and hospital factor adjusted Cox proportional hazard ratios are presented next to the diagnostic categories’ representative survival curve. Excess risk of mortality represents the proportion of deaths in children with algorithm-identified ARDS in the septic population.
DISCUSSION
Using a novel ICD10-based algorithm to identify a surrogate for ARDS in administrative datasets, we demonstrated that ARDS is responsible for nearly one-quarter of the excess risk of mortality in patients with sepsis. Our quantification of the degree by which ARDS contributed to mortality in paediatric sepsis supports the construct validity of our ICD coding algorithm. Our estimation of excess mortality due to ARDS in a paediatric population is comparable to previous adult attributable mortality estimates.4 Our estimation of excess mortality can help inform future trials in paediatric ARF and ARDS. Trials with interventions postulated to exert a beneficial effect on mortality primarily by mitigating ARDS would require either extremely high efficacy or prohibitively large enrollment targets.
Our study has limitations. Retrospective analysis of administrative data was used in determination of the excess risk of mortality of ARDS in septic children, but does not provide inference on the causal pathway between ARDS and mortality in these patients. Utilisation of this algorithm involves both ICD-10 diagnostic and procedural codes, making algorithm-identified ARDS not applicable in datasets lacking procedural codes or using ICD-9. Administrative datasets typically lack timing of diagnoses data, making determination of ARF onset in relation to sepsis problematic. However, we and others have previously shown that ARF develops concurrently with sepsis onset.10 Sepsis and ARDS identification using administrative data and ICD codes has repeatedly been attempted with variable accuracy11–13 and optimum performance occurs in the dataset of code algorithm generation limiting the generalizability of any such algorithm, including ours. Despite these limitations, use of our derived algorithms in large single and multicenter administrative databases reproduced characteristics consistent with what was seen in the derivation cohort, suggesting validity of our algorithm. Use of our ICD algorithm offers investigators a reproducible way of accurately identifying ARF requiring IMV and an administrative database surrogate of ARDS, intended for use in retrospective research where ARDS was not otherwise identified. Future studies in EHR datasets with granular timing of sepsis and ARF onset are needed to confirm our findings. However, we are reassured that our estimates for algorithm-identified ARDS excess risk of mortality are entirely consistent with adult ARDS.
Funding
Dr. GK received support from NIH NIGMS T32GM112596 and NICHD T32HD060550. Dr. JC received support from NIH NHLBI K24HL115354. Dr. ASH received support from the NIH NHLBI K23HL153759.
Footnotes
Competing interests None declared.
Patient consent for publication Not applicable.
REFERENCES
- 1.Curley MAQ, Hibberd PL, Fineman LD, et al. Effect of prone positioning on clinical outcomes in children with acute lung injury: a randomized controlled trial. JAMA 2005;294:229–37. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Willson DF, Thomas NJ, Tamburro R, et al. Pediatric calfactant in acute respiratory distress syndrome trial. Pediatr Crit Care Med 2013;14:657–65. [DOI] [PubMed] [Google Scholar]
- 3.Ketcham SW, Sedhai YR, Miller HC, et al. Causes and characteristics of death in patients with acute hypoxemic respiratory failure and acute respiratory distress syndrome: a retrospective cohort study. Crit Care 2020;24:391. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Auriemma CL, Zhuo H, Delucchi K, et al. Acute respiratory distress syndrome-attributable mortality in critically ill patients with sepsis. Intensive Care Med 2020;46:1222–31. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Tancredi M, Rosengren A, Svensson A-M, et al. Excess mortality among persons with type 2 diabetes. N Engl J Med 2015;373:1720–32. [DOI] [PubMed] [Google Scholar]
- 6.Gavrielov-Yusim N, Friger M. Use of administrative medical databases in population-based research. J Epidemiol Community Health 2014;68:283–7. [DOI] [PubMed] [Google Scholar]
- 7.Pediatric health information system. 2021. Available: https://www.childrenshospitals.org/phis
- 8.Feudtner C, Feinstein JA, Zhong W, et al. Pediatric complex chronic conditions classification system version 2: updated for ICD-10 and complex medical technology dependence and transplantation. BMC Pediatr 2014;14:199. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Prescott HC, Sjoding MW, Langa KM, et al. Late mortality after acute hypoxic respiratory failure. Thorax 2018;73:618–25. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Yehya N, Keim G, Thomas NJ. Subtypes of pediatric acute respiratory distress syndrome have different predictors of mortality. Intensive Care Med 2018;44:1230–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Liu B, Hadzi-Tosev M, Liu Y, et al. Accuracy of international classification of diseases, 10th revision codes for identifying sepsis: a systematic review and meta-analysis. Crit Care Explor 2022;4:e0788. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Jolley RJ, Sawka KJ, Yergens DW, et al. Validity of administrative data in recording sepsis: a systematic review. Crit Care 2015;19:139. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Kerlin MP, Weissman GE, Wonneberger KA, et al. Validation of administrative definitions of invasive mechanical ventilation across 30 intensive care units. Am J Respir Crit Care Med 2016;194:1548–52. [DOI] [PMC free article] [PubMed] [Google Scholar]