Abstract
Rationale
Pulmonary complications contribute significantly to nonrelapse mortality following hematopoietic stem cell transplantation (HCT). Identifying patients at high risk can help enroll such patients into clinical studies to better understand, prevent, and treat posttransplantation respiratory failure syndromes.
Objectives
To develop and validate a prediction model to identify those at increased risk of acute respiratory failure after HCT.
Methods
Patients underwent HCT between January 1, 2019, and December 31, 2021, at one of three institutions. Those treated in Rochester, MN, formed the derivation cohort, and those treated in Scottsdale, AZ, or Jacksonville, FL, formed the validation cohort. The primary outcome was the development of acute respiratory distress syndrome (ARDS), with secondary outcomes including the need for invasive mechanical ventilation (IMV) and/or noninvasive ventilation (NIV). Predictors were based on prior case-control studies.
Measurements and Main Results
Of 2,450 patients undergoing stem cell transplantation, there were 1,718 hospitalizations (888 patients) in the training cohort and 1,005 hospitalizations (470 patients) in the test cohort. A 22-point model was developed, with 11 points from prehospital predictors and 11 points from posttransplantation or early (<24-h) in-hospital predictors. The model performed well in predicting ARDS (C-statistic, 0.905; 95% confidence interval [CI], 0.870–0.941) and the need for IMV and/or NIV (C-statistic, 0.863; 95% CI, 0.828–0.898). The test cohort differed markedly in demographic, medical, and hematologic characteristics. The model also performed well in this setting in predicting ARDS (C-statistic, 0.841; 95% CI, 0.782–0.900) and the need for IMV and/or NIV (C-statistic, 0.872; 95% CI, 0.831–0.914).
Conclusions
A novel prediction model incorporating data elements from the pretransplantation, posttransplantation, and early in-hospital domains can reliably predict the development of post-HCT acute respiratory failure.
Keywords: hematopoietic stem cell transplantation, bone marrow transplant, acute respiratory distress syndrome, acute respiratory failure, Lung Injury Prevention Score
At a Glance Commentary
Scientific Knowledge on the Subject
Pulmonary complications are the biggest contributor to nonrelapse mortality following hematopoietic stem cell transplantation (HCT). Although the outcomes of acute respiratory failure (ARF) in the wider medical, surgical, and oncological populations have improved in recent decades, the prognosis of HCT recipients with ARF have not. There is an urgent need to find ways to identify those at the highest risk of post-HCT ARF to better understand the mechanisms of disease and potentially enroll the patients at highest risk into enriched clinical trials.
What This Study Adds to the Field
A 22-point score was developed using elements from the pretransplantation, posttransplantation, and early in-hospital domains. This score incorporated predictors specific to HCT recipients (e.g., certain chemotherapy agents) or routinely collected only in a pretransplantation setting (e.g., pulmonary function indices). This score was then externally validated, reliably predicting the development of severe ARF (acute respiratory distress syndrome, the need for invasive and/or noninvasive ventilation) and modestly predicting the risk of milder ARF. This score can help facilitate mechanistic studies to better understand the spectrum of post-HCT respiratory failure, help triage patients at high risk to more closely monitored settings, and enroll the patients at the highest risk into pharmacopreventive trials.
Hematopoietic stem cell transplantation (HCT) is a curative option for hematologic malignancies. Pulmonary complications are the biggest contributors to nonrelapse mortality (1–3) and a major threat to the survivorship of HCT recipients. Complications can be infectious or noninfectious, with the latter including a spectrum of diffuse lung injury syndromes such periengraftment respiratory distress syndrome, diffuse alveolar hemorrhage, and idiopathic pneumonia syndrome (4–7). The most severe end of the spectrum of post-HCT acute respiratory failure (ARF) syndromes can be considered under the umbrella of acute respiratory distress syndrome (ARDS), which occurs in 5% of patients following HCT, with a mortality rate >60% (8). Although agnostic of pathophysiologic underpinnings, there are some advantages to grouping patients with ARF in such a syndromic manner. Most notably, post-HCT respiratory failure syndromes are typically clinically indistinguishable at the time of presentation, and the umbrella term of ARDS can help with implementation of standardized syndrome-specific care process models that have been shown to improve outcomes in other patient populations (9–12).
Case-control studies have established risk factors associated with the development of post-HCT ARDS in the pretransplantation, posttransplantation, and early-hospital domains. Some of these are shared with the general medical/surgical population, such as the presence of sepsis or pneumonia at the time of hospital admission (3). However, many are unique to the HCT population, such as low pretransplantation pulmonary function (FEV1 and DlCO), abnormal hemoglobin concentrations and platelet and white cell counts, and the use of certain chemotherapy agents (carboplatin, thalidomide, methotrexate, or cisplatin) (13).
The objective of this study was to develop and externally validate a prediction model (Lung Injury Prevention Score for bone marrow transplant patients [LIPS-BMT]) that would help identify those at the highest risk of respiratory failure requiring invasive mechanical ventilation (IMV) or noninvasive ventilation (NIV) and ARDS based on data available within the early hospital period, incorporating risk factors in the pretransplantation, posttransplantation, and early in-hospital domains. Because a mechanistic understanding of why post-HCT ARF occurs is lacking, a score could help prospectively enroll patients at risk of ARF syndromes into mechanistic studies to better elucidate the pathophysiology of these disorders. Like prior examples in the general medical/surgical population, an ability to meaningfully predict risk could also help enroll the patients at highest risk into clinical trials focused on ARF/ARDS prevention (14).
Methods
This is a multicenter cohort study. The Mayo Clinic Institutional Review Board (Rochester, Minnesota) approved the study protocol before study initiation (13-002869). Requirement for written informed consent was waived by institutional review. The Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) guidelines were used in the conduct of this study and the reporting of the results (15) (Table E1 in the online supplement).
Study Population
The study population included consecutive adult (age ⩾18 yr) patients who underwent HCT at one of three Mayo Clinic sites: Rochester, MN; Jacksonville, FL; and Scottsdale, AZ. Study enrollment occurred for patients undergoing HCT between January 1, 2019, and December 31, 2021. For those patients who had more than one HCT, the first HCT was used as the index event. Follow-up occurred until January 1, 2022. Patients who underwent HCT in Rochester, MN, formed the internal derivation/training cohort. Patients who underwent HCT at the Jacksonville, FL, or Scottsdale, AZ, sites formed the external validation/test cohort. Patients were included if they 1) consented for their medical records to be used for research purposes, 2) were hospitalized within 1 year of HCT, and 3) had a hospitalization of at least 24 hours. Each hospitalization was considered an independent event, and the probability of developing the outcome of interest was calculated based on data available up to the first 24 hours of hospital admission.
Outcome Ascertainment
The primary outcome was the development of ARDS as defined by the 2012 Berlin criteria (16). Potential patients were screened based on a requirement for positive pressure ventilation with an end-expiratory pressure >5 cm H2O and arterial blood gas demonstrating a PaO2/FiO2 ratio of less than 300. All patients were reviewed by two of three independent study investigators with expertise in ARDS diagnosis (S.H., Z.Z., and B.A.W.), with disagreements adjudicated by the senior author (H.Y.). Secondary outcomes were the need for IMV or NIV and ARF. The latter was defined as requiring IMV, NIV, a high-flow nasal cannula, or supplemental oxygen ⩾4 L/min.
Data Retrieval
Data were retrieved using data warehouses shared across the Mayo Clinic Enterprise (Mayo Data Explorer), an institutional pulmonary function test (PFT) database, and site-specific hematology databases that included details regarding patient diagnosis, chemotherapy exposures, and chemotherapy regimens. Manual chart review was used to supplement data retrieval when needed. Data collected included basic demographic and hematologic data. Purposeful variable selection was determined a priori based on existing literature specific to this patient population (3, 13). In the pretransplantation domain, this included baseline PFT results (FEV1, FVC, DlCO), pretransplantation laboratory values (hemoglobin level, leukocyte and platelet counts, and aspartate aminotransferase and albumin levels), comorbidities (tobacco use, diabetes mellitus), specific chemotherapy agents (carboplatin, cisplatin, methotrexate, thalidomide), and pretransplantation transfusion needs. If multiple PFTs were performed, the values measured closest to the time of transplantation were selected. For laboratory values, the worst (e.g., lowest cell count or albumin level, highest aspartate aminotransferase level) within 30 days were selected. In the posttransplantation domain, the use of systemic steroids was collected. For the in-hospital domain, the following data elements were collected up to the first 24 hours of hospital admission: sepsis, septic shock, presence of a community-associated respiratory virus (CARV) (influenza, respiratory syncytial virus, severe acute respiratory syndrome coronavirus 2 [SARS-CoV-2], rhinovirus, parainfluenza virus, non–coronavirus disease [COVID-19] coronaviruses, adenovirus, human metapneumovirus), pneumonia, transfusion requirements, opioid agent use, vital sign parameters (maximum temperature, median respiratory rate), laboratory values (serum bicarbonate and serum creatinine levels, platelet count). Pneumonia was diagnosed by clinician documentation of a pneumonia diagnosis. Sepsis and septic shock were diagnosed based on International Classification of Diseases, 10th Revision coding or the use of a previously validated automated search (17). If multiple values existed for any continuous variable within the first 24 hours, the worst was selected (e.g., highest creatinine level, lowest serum bicarbonate level). For a categorical variable to be positive, the test/diagnosis had to occur within the first 24 hours of hospitalization. For example, for a diagnosis of CARV, the nasopharyngeal/BAL specimen had to be collected in the first 24 hours of hospitalization.
Statistical Analysis and Model Development
Patient demographic and clinical characteristics were described using proportions (frequencies) and means or medians (IQR) as appropriate. An independent t test and χ2 test were used to determine the statistical significance for continuous and categorical variables, respectively. Standardized mean differences were used to report differences between the development and validation cohorts. A two-sided P value of less than 0.05 was considered statistically significant. Missing continuous data were handled using imputation (median imputation, caret package). Specific variables with missing pretransplantation data were smoking status (n = 4), FEV1 (n = 75), FVC (n = 75), and DlCO (n = 76). Specific variables with missing posttransplantation data were respiratory rate (n = 3), temperature (n = 13), creatinine level (n = 91), bicarbonate level (n = 467), and platelet count (n = 98).
Model development was performed exclusively with the training dataset. Variable importance was estimated using penalized regression for the primary outcome (ARDS) with all variables included (glmnet package; Figure E1). This formed a starting point for parameter tuning for the final model. Individual parameters were then sequentially adjusted in terms of threshold and points assigned, with the goal of maximizing the area under the receiver operator characteristic curve (AUROC) for predicting ARDS. The final model, optimized for predicting ARDS in the training cohort, was then tested for the secondary outcomes (ARF, need for IMV and/or NIV) in the training cohort and then for the primary and secondary outcomes in the test cohort. The bootstrap methodology was used for 95% confidence intervals (CIs) for AUROC estimation and hypothesis testing (pROC package). JMP Pro software (SAS Institute) was used for data collection and handling. Data analysis was performed in R 3.6.3 (R Foundation for Statistical Computing) using the R Studio 2022.02 integrated development environment (PBC). Specific packages used included tidyverse, caret, glmnet, tableone, and pROC (18–20). Code used to develop the model can be found at https://github.com/hemangyadav/LIPS-BMT/.
Results
Between January 1, 2019, and December 31, 2021, 2,496 patients underwent HCT. Of these, 46 declined research participation (Figure 1). Of 2,450 patients, 1,351 underwent HCT at the Rochester, MN, site (derivation/training cohort) and 1,099 underwent HCT at the Jacksonville, FL, or Scottsdale, AZ, site (validation/test cohort). Summary demographic, hematologic and pulmonary function data are outlined in Table 1. In the training cohort, 888 of 1,351 patients (65.7%) required a total of 1,718 hospitalizations. In the test cohort, 470 of 1,099 patients (42.8%) required a total of 1,005 hospitalizations. Characteristics and outcomes for hospitalizations longer than 24 hours are outlined in Table 2.
Figure 1.
Study flow diagram. Patients from Rochester, MN, formed the derivation cohort, and patients from Jacksonville, FL, or Scottsdale, AZ, formed the validation cohort. HCT = hematopoietic stem cell transplantation.
Table 1.
Baseline Patient Demographic and Clinical Data
| Characteristic | Derivation (n = 1,351) | Validation (n = 1,099) |
|---|---|---|
| Type of transplant | ||
| Allogeneic | 289 (21.4%) | 432 (39.3%) |
| Autologous | 1,062 (78.6%) | 666 (60.6%) |
| Syngeneic | 0 | 1 (0.1%) |
| Location | ||
| Rochester, MN | 1,351 (100%) | 0 |
| Scottsdale, AZ | 0 | 579 (52.7%) |
| Jacksonville, FL | 0 | 520 (47.3%) |
| Age at time of transplantation, yr | 62.70 (54.4–68.0) | 61.35 (51.5–67.3) |
| Sex | ||
| Male | 817 (60.5%) | 630 (57.3%) |
| Female | 534 (39.5%) | 469 (42.7%) |
| Race | ||
| White | 1,235 (91.4%) | 878 (79.9%) |
| Black or African American | 25 (1.9%) | 105 (9.6%) |
| Asian | 18 (1.3%) | 28 (2.5%) |
| American Indian/Alaskan Native | 14 (1.0%) | 19 (1.7%) |
| Native Hawaiian/Pacific Islander | 2 (0.1%) | 6 (0.5%) |
| Other/mixed | 40 (3.0%) | 34 (3.1%) |
| Not disclosed | 17 (1.3%) | 29 (2.6%) |
| Ethnicity | ||
| Not Hispanic or Latino | 1,290 (95.5%) | 926 (84.3%) |
| Hispanic or Latino | 37 (2.7%) | 141 (12.8%) |
| Not disclosed/unknown | 24 (1.8%) | 32 (2.9%) |
| Type of donor | ||
| Self | 1,059 (78.4%) | 662 (60.2%) |
| Unrelated | 156 (11.5%) | 270 (24.6%) |
| Related | 120 (8.9%) | 68 (6.2%) |
| Haploidentical | 16 (1.2%) | 98 (8.9%) |
| Syngeneic | 0 | 1 (0.1%) |
| Cell source | ||
| Peripheral | 1,296 (96.2%) | 1092 (99.7%) |
| Bone marrow | 49 (3.6%) | 3 (0.3%) |
| Cord blood | 2 (0.1%) | 0 |
| Hematologic disorder | ||
| Multiple myeloma | 732 (54.2%) | 470 (42.8%) |
| Non-Hodgkin lymphoma | 271 (20.1%) | 190 (17.3%) |
| Acute myeloid leukemia | 113 (8.4%) | 157 (14.3%) |
| Hodgkin lymphoma | 56 (4.1%) | 33 (3.0%) |
| Myelodysplastic syndrome | 77 (5.7%) | 79 (7.2%) |
| Acute lymphoblastic leukemia | 43 (3.2%) | 49 (4.5%) |
| Aplastic anemia | 12 (0.9%) | 4 (0.4%) |
| Chronic myeloid leukemia | 11 (0.8%) | 20 (1.8%) |
| Myeloproliferative disease | 9 (0.7%) | 57 (5.2%) |
| Other | 27 (2.0%) | 40 (3.6%) |
| Conditioning | ||
| Myeloablative | 1,192 (88.2%) | 827 (75.3%) |
| Reduced intensity/nonmyeloablative | 159 (11.8%) | 272 (24.7%) |
| Diabetes mellitus | ||
| No | 1,133 (83.9%) | 921 (83.8%) |
| Yes | 218 (16.1%) | 178 (16.2%) |
| History of smoking | ||
| Current | 120 (8.9%) | 61 (5.6%) |
| Former | 400 (29.6%) | 370 (33.7%) |
| Never | 830 (61.4%) | 667 (60.7%) |
| Unknown | 1 (0.1%) | 1 (0.1%) |
| Body mass index, kg/m2* | 28.90 (25.6–32.7) | 28.40 (25.1–32.2) |
| Hemoglobin, g/dL† | 10.00 (8.4–11.4) | 9.70 (8.1–11.0) |
| Platelet count, 109/L† | 105.00 (68.0–142.0) | 84.00 (48.0–133.0) |
| Leukocyte count, 109/L† | 3.10 (1.7–4.5) | 2.90 (1.3–4.3) |
| FEV1, L‡ | 3.12 (0.84%) | 2.93 (0.82%) |
| FVC, L‡ | 4.01 (1.04%) | 3.75 (1.05%) |
| DlCO, mL/min/mm Hg‡ | 21.01 (5.59%) | 21.87 (6.16%) |
| FEV1 z-score‡ | −0.02 (1.13%) | −0.19 (1.11%) |
| FVC z-score‡ | −0.03 (1.06%) | −0.19 (1.05%) |
| DlCO z-score‡ | −1.01 (1.13%) | −0.66 (1.26%) |
| Platelet transfusion† | ||
| No | 1,260 (93.3%) | 979 (89.1%) |
| Yes | 91 (6.7%) | 120 (10.9%) |
| Red blood cell transfusion† | ||
| No | 1,191 (88.2%) | 917 (83.4%) |
| Yes | 160 (11.8%) | 182 (16.6%) |
Data presented as median (IQR) where applicable.
At time of hematopoietic stem cell transplantation.
Within 30 d of transplantation.
Within 6 mo before hematopoietic stem cell transplantation.
Table 2.
Hospitalization Characteristics
| Characteristic | Derivation Cohort (n = 1,718*) | Validation Cohort (n = 1,005†) | SMD |
|---|---|---|---|
| Pretransplantation data | |||
| Type of transplant | 0.479 | ||
| Allogeneic | 782 (45.5%) | 687 (68.6%) | |
| Autologous | 936 (54.5%) | 315 (31.4%) | |
| Age at time of transplantation, yr | 62.3 (51.2–66.9) | 60.4 (47.9–66.7) | 0.148 |
| Methotrexate | 0.092 | ||
| No | 1,450 (84.4%) | 811 (80.9%) | |
| Yes | 268 (15.6%) | 191 (19.1%) | |
| Cisplatin | 0.025 | ||
| No | 1,603 (93.3%) | 941 (93.9%) | |
| Yes | 115 (6.7%) | 61 (6.1%) | |
| Carboplatin | 0.035 | ||
| No | 1,564 (91.0%) | 902 (90.0%) | |
| Yes | 154 (9.0%) | 100 (10.0%) | |
| Thalidomide | 0.049 | ||
| No | 1,671 (97.3%) | 982 (98.0%) | |
| Yes | 47 (2.7%) | 20 (2.0%) | |
| Current or prior smoking | 0.010 | ||
| No | 1,030 (60.0%) | 596 (59.5%) | |
| Yes | 687 (40.0%) | 406 (40.5%) | |
| Diabetes mellitus | 0.007 | ||
| No | 1,276 (74.3%) | 741 (74.0%) | |
| Yes | 442 (25.7%) | 261 (26.0%) | |
| Minimum hemoglobin, g/dL‡ | 9.5 (7.5 to 11.0) | 8.9 (6.9 to 10.5) | 0.232 |
| Minimum platelet count, K/UL‡ | 93.0 (49.0 to 135.0) | 72.5 (29.0 to 119.0) | 0.244 |
| Minimum leukocytes count, ×109/L‡ | 2.3 (1.0 to 3.9) | 1.8 (0.5 to 3.6) | 0.175 |
| Maximum aspartate aminotransferase, U/L‡ | 29.0 (23.0 to 41.0) | 32.0 (25.0 to 48.0) | 0.134 |
| Body mass index§ | 28.8 (25.7 to 32.9) | 28.2 (24.1 to 31.6) | 0.202 |
| FEV1 z-scoreǁ | 0.04 (−0.73 to 0.75) | −0.30 (−1.11 to 0.51) | 0.229 |
| FVC z-scoreǁ | 0.03 (−0.62 to 0.65) | −0.29 (−1.02 to 0.42) | 0.233 |
| DlCO z-scoreǁ | −1.06 (−1.79 to −0.38) | −0.74 (−1.60 to −0.01) | 0.248 |
| Prehospital/early in-hospital data | |||
| Oral steroid use¶ | 0.168 | ||
| No | 764 (44.5%) | 531 (52.8%) | |
| Yes | 954 (55.5%) | 474 (47.2%) | |
| Pneumonia** | 0.246 | ||
| No | 1,592 (92.7%) | 854 (85.0%) | |
| Yes | 126 (7.3%) | 151 (15.0%) | |
| Community-associated respiratory virus** | 0.005 | ||
| No | 1,679 (97.7%) | 983 (97.8%) | |
| Yes | 39 (2.3%) | 22 (2.2%) | |
| Sepsis** | 0.143 | ||
| No | 1,506 (87.7%) | 830 (82.6%) | |
| Yes | 212 (12.3%) | 175 (17.4%) | |
| Septic shock** | 0.102 | ||
| No | 1,660 (96.6%) | 950 (94.5%) | |
| Yes | 58 (3.4%) | 55 (5.5%) | |
| Red blood cell or platelet transfusion** | 0.147 | ||
| No | 1,130 (65.8%) | 729 (72.5%) | |
| Yes | 588 (34.2%) | 276 (27.5%) | |
| Opioid use** | 0.035 | ||
| No | 953 (55.5%) | 575 (57.2%) | |
| Yes | 765 (44.5%) | 430 (42.8%) | |
| Median respiratory rate, breaths/min** | 18.0 (16.0–20.0) | 18.0 (16.0–19.0) | 0.043 |
| Maximum temperature, °C** | 37.4 (37.0–38.1) | 37.3 (37.0–37.9) | 0.122 |
| Maximum creatinine, mg/dL** | 0.96 (0.75–1.30) | 0.98 (0.76–1.29) | 0.027 |
| Minimum bicarbonate, mEq/L** | 23.0 (20.8–25.0) | 23.0 (20.0–25.0) | 0.002 |
| Minimum platelet count, 109/L** | 35.0 (15.0–97.0) | 64.0 (25.5–121.0) | 0.243 |
| Outcomes | |||
| ICU admission | 0.001 | ||
| No | 1,455 (84.7%) | 851 (84.7%) | |
| Yes | 263 (15.3%) | 154 (15.3%) | |
| Total ICU length of stay, d | 1.77 (0.94–4.27) | 3.44 (1.49–8.14) | 0.372 |
| Acute respiratory distress syndrome | 0.109 | ||
| No | 1,688 (98.3%) | 970 (96.5%) | |
| Yes | 30 (1.7%) | 35 (3.5%) | |
| Acute respiratory failure (⩾4 L O2) | 0.367 | ||
| No | 1,490 (86.7%) | 725 (72.1%) | |
| Yes | 228 (13.3%) | 280 (27.9%) | |
| Mechanical ventilation | 0.042 | ||
| No | 1,627 (94.7%) | 942 (93.7%) | |
| Yes | 91 (5.3%) | 63 (6.3%) | |
| Hospital length of stay, d | 5.10 (2.9–9.0) | 4.20 (2.3–8.3) | 0.020 |
| Discharge disposition | 0.215 | ||
| Home | 1,597 (93.0%) | 875 (87.1%) | |
| Healthcare facility | 47 (2.7%) | 47 (4.7%) | |
| Hospice | 29 (1.7%) | 44 (4.4%) | |
| Against medical advice/other | 1 (0.1%) | 4 (0.4%) | |
| Died | 44 (2.6%) | 35 (3.5%) |
Definition of abbreviation: SMD = standardized mean difference.
Data presented as median (IQR) where applicable.
Total of 1,718 hospitalizations in 888 patients.
Total of 1,005 hospitalizations in 470 patients.
Within 30 d before hematopoietic stem cell transplantation.
At time of hematopoietic stem cell transplantation.
Within 6 mo before hematopoietic stem cell transplantation.
Within 30 d before hospitalization.
Within first 24 h of hospital admission.
There were considerable differences in baseline characteristics of the test and training cohorts (Table 1). For example, for demographic data, the test cohort had a higher proportion of patients with Hispanic/Latino ethnicity (12.8% vs. 2.7%) and a higher proportion of patients who identified as Black/African American (9.6% vs. 1.9%). There were differences in hematology practices as well. The test cohort had a higher frequency of allogeneic HCT (39.3% vs. 21.4%), more haploidentical HCT procedures (8.9% vs. 1.2%), and a greater use of reduced-intensity/nonmyeloablative conditioning (24.7% vs. 11.8%). Hospitalization characteristics of the cohort are outlined in Table 2. The median numbers of hospitalizations per patient were 1 (IQR, 1–2) in the derivation cohort and 1 (IQR, 1–3) in the test cohort. Outliers with multiple repeat admissions were present but relatively infrequent (Figure E2). The rate of ICU admission between the two cohorts was similar at 15.3%. There were 30 ARDS cases in the training cohort (median time to ARDS from admission, 7.95 [IQR, 3.3–20.3] d) and 35 ARDS cases in the test cohort (median time to ARDS from admission, 6.3 [IQR, 1.3–20.3] d). For the secondary outcome of ventilatory support, there were 91 (5.3%) hospitalizations during which patients required NIV or IMV in the training cohort, compared with 63 (6.3%) in the test cohort. For the secondary outcome of ARF, there were 228 (13.3%) hospitalizations during which patients required ⩾4 L/min of oxygen support in the training cohort, compared with 280 (27.9%) in the test cohort. Again, there were substantial differences between the test and training cohorts in terms of pretransplantation, prehospitalization, and early in-hospital predictors, as outlined in Table 2.
Model Development
Penalized regression was used to estimate variable importance for predicting ARDS in the training cohort (see Figure E1), serving as a starting point for model development. The five most important predictors were the presence of sepsis (n = 212; 12.3%), septic shock (n = 58; 3.4%), and pneumonia (n = 126; 7.3%); CARV positivity (n = 39; 2.3%); and allogeneic HCT status (n = 782; 45.6%). Of those who had CARV positivity, organisms identified were rhinovirus/enterovirus (n = 10), respiratory syncytial virus (n = 9), SARS-CoV-2 (n = 5), parainfluenza (n = 4), adenovirus (n = 4), non-COVID coronavirus (n = 3), influenza (n = 3), and human metapneumovirus (n = 1).
Parameters were tuned to maximize the AUROC for predicting ARDS in the training cohort, resulting in a 22-point score with 11 points assigned based on prehospitalization data and 11 points assigned based on data collected during the first 24 hours of hospitalization (Table 3). The AUROC for predicting ARDS in the training cohort was 0.905 (95% CI, 0.870–0.941; Figure 2), the AUROC for predicting the need for IMV or NIV was 0.863 (95% CI, 0.828–0.898), and the AUROC for predicting a need for ⩾4 L of supplemental oxygen was 0.738 (95% CI, 0.702–0.773) (Figure 2). The cumulative frequency of LIPS-BMT stratified by outcome is also outlined in Figure 2. The distribution of LIPS-BMT in those in whom the outcome of interest developed versus those in whom it did not is shown in Figure E3.
Table 3.
Lung Injury Prevention Score in Bone Marrow Transplantation
| Variable | Points |
|---|---|
| Pretransplant domain (max. 10 points) | |
| Allogeneic hematopoietic stem cell transplantation | 2 |
| Reduced-intensity conditioning | 1 |
| DlCO z-score <−2.5 | 1 |
| FEV1 z-score <−2.5 | 1 |
| Pretransplant aspartate aminotransferase >60 U/L | 1 |
| Pretransplant hemoglobin <8 g/dL | 1 |
| Pretransplant leukocyte count <3 × 109/L | 1 |
| Diabetes mellitus | 1 |
| Chemotherapy with thalidomide, cisplatin, methotrexate, or carboplatin | 1 |
| Posttransplant domain, prehospital (max. 1 point) | |
| Oral steroid use | 1 |
| Posttransplant domain, ⩽24 h of hospital admission (max. 11 points) | |
| Sepsis | 4 |
| With septic shock | 2 |
| Pneumonia | 2 |
| Community-associated respiratory virus* | 2 |
| Bicarbonate <21 mEq/L | 2 |
| Median respiratory rate >20 breaths/min | 2 |
| Opioid use | 1 |
Influenza, respiratory syncytial virus, severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), rhinovirus, parainfluenza virus, non–coronavirus disease (COVID-19) coronavirus, adenovirus, human metapneumovirus.
Figure 2.
Performance of the lung injury prediction model in bone marrow transplantation in the training/derivation cohort. Left: receiver operator characteristic curves for predicting acute respiratory distress syndrome, acute respiratory failure, and the need for invasive mechanical ventilation or noninvasive mechanical ventilation. AUROC is reported along with 95% confidence intervals. Right: cumulative frequency of Lung Injury Prediction Score in Bone Marrow Transplantation stratified by outcome. ARDS = acute respiratory distress syndrome; ARF = acute respiratory failure; AUROC = area under the receiver operator characteristic curve; IMV = invasive mechanical ventilation; LIPS-BMT = Lung Injury Prediction Score in Bone Marrow Transplantation; NIV = noninvasive ventilation.
Model Performance
The 22-point model was then tested in the external validation (test) cohort. In the test cohort, the AUROC for predicting ARDS was 0.847 (95% CI, 0.790–0.904), the AUROC for predicting a need for IMV and/or NIV was 0.872 (95% CI, 0.831–0.914), and the AUROC for predicting ARF requiring ⩾4 L of supplemental O2 was 0.713 (95% CI, 0.677–0.749) (Figure 3). The cumulative frequency of LIPS-BMT stratified by outcome is also outlined in Figure 3. The distribution of LIPS-BMT in those in whom the outcome of interest developed versus those in whom it did not is shown in Figure E4. For predicting ARDS, a score of 7 or more had good sensitivity (91.4%) but poor specificity (53.6%), whereas a score of 12 or more had poor sensitivity (54.2%) but good specificity (90.8%). For predicting ARF requiring ⩾4 L supplemental O2, a score of 4 or more had a good sensitivity (92.5%) but poor specificity (25.6%), whereas a score of 10 or more had poor sensitivity (35.3%) but good specificity (89.0%). For predicting a need for IMV and/or NIV, a score of 7 or more had good sensitivity (92.0%) but poor specificity (55.1%), whereas a score of 11 or greater had modest sensitivity (62.9%) but good specificity (90.0%). These are outlined in Tables E2–E4 for the training cohort and Tables E5–E7 for the test cohort.
Figure 3.
Performance of the lung injury prediction model in bone marrow transplantation in the test/validation cohort. Left: receiver operator characteristic curves for predicting acute respiratory distress syndrome, acute respiratory failure, and the need for invasive mechanical ventilation or noninvasive mechanical ventilation. AUROC is reported along with 95% confidence intervals. Right: cumulative frequency of the Lung Injury Prediction Score in Bone Marrow Transplantation stratified by outcome. For definition of abbreviations, see Figure 2.
Sensitivity Analyses
For score development and validation, each hospitalization was considered an independent event. To assess whether repeated admissions by a minority of patients could negatively affect score generalizability or performance (see Figure E2), we performed a sensitivity analysis in which only the first post-HCT hospitalization was considered. Overall score performance remained similar (Figure E5). Patient characteristics for this sensitivity analysis are outlined in Table E8. The outcomes of interest were also assessed per hospitalization. Therefore, a patient could have a second outcome during a different hospitalization. To assess the effect of this on overall score performance, we performed a sensitivity analysis in which only the first episode of ARDS (63 of 66), IMV and/or NIV (145 of 157), and ARF requiring ⩾4 L supplemental O2 (387 of 515) were considered. These results are shown in Figure E6, with overall score performance remaining similar.
Discussion
In this study, we developed and externally validated a prediction score for respiratory failure after HCT. This score performed well for predicting severe respiratory failure (ARDS or the need for IMV and/or NIV) and modestly for predicting mild ARF (⩾4 L of supplemental O2). The performance of the score when applied to a demographically and medically distinct external cohort remained close to its performance in the training cohort.
The predictors in this cohort have biologic plausibility and can be easily abstracted from the medical record. Among the pretransplantation predictors, reduced lung function has been previously associated with a range of post-HCT respiratory complications (3, 5, 21) and is routinely assessed at many centers before HCT. Allogeneic HCT recipients have a greater degree of post-HCT immunosuppression and are at increased risk of pulmonary complications. Reduced-intensity conditioning or nonmyeloablative conditioning, typically used in frailer patients, is less toxic, with shorter cytopenias than seen with myeloablative conditioning, albeit with a higher risk of disease relapse. The presence of reduced-intensity conditioning/nonmyeloablative conditioning as a predictor of ARF/ARDS may reflect overall patient frailty. In line with that, prior studies have associated Eastern Cooperative Oncology Group status with the development of ARDS (13). Because Eastern Cooperative Oncology Group status was not routinely included in clinical documentation, we were unable to include it among our predictors. Other predictors may be markers of pretransplantation organ function (e.g., aspartate aminotransferase level) and recent chemotherapy before conditioning chemotherapy leading to lower cell counts (e.g., hemoglobin, leukocyte count). A posttransplantation predictor present at the time of hospital admission was the use of oral steroids. This will most commonly be a surrogate marker for a patient with confirmed/suspected graft-versus-host disease but may be given for other indications (e.g., asthma, gout). The in-hospital predictors are closer to those we would expect from the wider medical/surgical population (22). Pneumonia and sepsis are the biggest contributors to ARF/ARDS in the HCT population and the wider medical population (3, 23). The HCT population is particularly vulnerable to respiratory viruses, and CARV positivity on admission can be plausibly linked to the development of ARF/ARDS (24). Other markers are reflections of the severity of illness (e.g., respiratory rate, metabolic acidosis) or have been previously associated with ARDS development (e.g., oral/intravenous opioid use) (3).
There are three potential ways to use this score with different thresholds selected depending on the outcome of interest and the indication for the score’s use. In a clinical setting, LIPS-BMT could determine where best to care for a patient (e.g., ICU vs. monitored non-ICU setting vs. general care). In this context, the best outcome to monitor would be the need for NIV and/or IMV, and the threshold would be chosen to strike a pragmatic mix between sensitivity and specificity, likely favoring greater sensitivity in most settings. In a research setting, patients who undergo HCT are at risk of an array of poorly understood lung injury syndromes such as idiopathic pneumonia syndrome, periengraftment respiratory distress syndrome, diffuse alveolar hemorrhage, and bronchiolitis obliterans syndrome (1, 2, 5–7). These may or may not occur alongside infectious pneumonia. Without any sort of mechanistic understanding of why these different lung injury syndromes develop, these diagnoses are often ones of exclusion, made post hoc. The LIPS-BMT could plausibly enroll patients who are at increased risk of developing these lung injury syndromes into prospective mechanistic studies collecting blood and respiratory samples for immunologic and multiomic studies. In this context, the outcome of interest could be ARF requiring ⩾4 L/min supplemental oxygen because that would capture the full array of post-HCT ARF syndromes. Because collecting blood and/or sputum samples poses a relatively low risk and is inexpensive, the threshold chosen can be lower to favor sensitivity over specificity, with only those in whom the outcomes of interest develop having their samples analyzed.
Unlike the general medical/surgical population or patients with cancer, critical care outcomes in patients who have undergone HCT have not improved meaningfully over time (25). As such, there is an urgent need for trials of potentially preventative therapies in this patient population. Indiscriminately testing preventive therapies on all HCT recipients admitted following HCT is impractical and potentially dangerous, so some sort of enrichment strategy is needed. ARDS clinical trials such as the ACURASYS (Acute Respiratory Distress Syndrome et Curarisation Systematique) trial for paralytic agent use in ARDS have used PaO2/FiO2 ratio to enrich those with severe ARDS into randomized controlled trials (12). Although this is a viable strategy when testing therapies for established ARDS, it is not well suited to ARDS prevention studies. In contrast, the LIPS-A trial was an example of a score (LIPS) being used to provide prognostic enrichment for a trial of preventive aspirin in patients at risk of developing ARDS (14). LIPS-BMT could viably do something similar in the post-HCT population, targeting enrollment after the first 24 hours of hospital admission. In this context, if the intervention is the use of a pharmacologic agent, the outcome chosen could be more severe respiratory failure (ARDS or the need for IMV and/or NIV), and the threshold chosen would be one that favors higher specificity so patients in whom the disease is unlikely to develop are not exposed to the intervention. Indeed, the performance of the LIPS-BMT is higher than other scores used previously. For example, the AUROC of the original LIPS was 0.80. The unique nature of the HCT population likely contributes to the higher score performance seen: patients have an array of premorbid data (e.g., baseline pulmonary function) that are not routinely available in the general medical population.
Study Weaknesses
Although our study has several strengths, including external validation in a geographically and demographically distinct cohort, thus increasing generalizability, it has some weaknesses as well. The second half of the study was performed during the COVID-19 pandemic. This may have affected the frequency of ARF/ARDS admissions as well as critical care and hospitalization practices. Although the absolute number of patients who had COVID-19 in our study was low, the widespread strain on healthcare systems may have had unpredictable effects on resource allocation. Another potential concern is that our study did not capture all hospitalizations. Although most patients who are hospitalized after HCT will be hospitalized in the transplant institution, it is possible that some patients are hospitalized elsewhere, and this was likely more common during the COVID-19 pandemic. Indeed, all three centers in the study had to go on hospital/ICU diversions at times as a result of capacity/staffing issues during 2020 and 2021. As such, we may not have captured all post-HCT admissions. Although it would be ideal to have complete follow-up for our cohort to capture all ARF/ARDS cases, it is not a necessity for testing a risk prediction score. Our score treats each hospitalization as an independent event. The data variables required to calculate LIPS-BMT are available before transplantation or at the time of hospitalization and are not dependent on capturing information from prior posttransplantation hospitalizations. Such prior post-HCT hospitalizations may contain valuable information: patients who experience ARF/ARDS may be more likely to have multiple hospitalizations. However, incorporating such data into a score introduces is the risk of missing hospitalizations that occur outside the transplant center and are not accessible within the electronic healthcare record. Most patients stay locally for 30 days following autologous HCT and 100 days following allogeneic HCT. After these time periods, it is more likely that we can miss these hospitalizations, and, if we were reliant on them for score calculation, that would introduce unintended bias. An analysis limited to only the immediate post-HCT period could potentially include data from post-HCT hospitalizations as predictors. Because our study likely did not capture all ARDS/respiratory failure events (i.e., those in which patients had to be admitted locally and were unable to transfer because of capacity/staffing reasons), this study likely underestimates the frequency of ARDS and respiratory failure after HCT.
Conclusions
A score incorporating elements from the pretransplantation, posttransplantation, and early in-hospital domains can reliably predict the development of severe ARF (ARDS, need for IMV and/or NIV) and modestly predict the risk of milder ARF. This scoring system has substantial promise for prognostic enrichment, allowing us to feasibly conduct mechanistic studies to better understand the spectrum of post-HCT lung injury syndromes and enroll the patients at the highest risk into pharmacopreventive trials.
Acknowledgments
Acknowledgment
The authors thank Anesthesia Clinical Research Unit Data Specialists Mr. Christopher Acker and Mrs. Erica Portner for their help with data extraction.
Footnotes
Supported by NHLBI grant K23HL151671 (to H.Y.). Study contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIH.
Author Contributions: O.G. and H.Y. contributed substantially to the conception of the work. S.H. and H.Y drafted the manuscript. All authors contributed to the design, data acquisition and analysis, and writing of the manuscript; provided intellectual contributions to the content; and made critical revisions. All authors reviewed and approved the final version of the manuscript.
This article has an online supplement, which is accessible from this issue’s table of contents at www.atsjournals.org.
Originally Published in Press as DOI: 10.1164/rccm.202308-1524OC on December 5, 2023
Author disclosures are available with the text of this article at www.atsjournals.org.
References
- 1. Afessa B, Abdulai RM, Kremers WK, Hogan WJ, Litzow MR, Peters SG. Risk factors and outcome of pulmonary complications after autologous hematopoietic stem cell transplant. Chest . 2012;141:442–450. doi: 10.1378/chest.10-2889. [DOI] [PubMed] [Google Scholar]
- 2. Astashchanka A, Ryan J, Lin E, Nokes B, Jamieson C, Kligerman S, et al. Pulmonary complications in hematopoietic stem cell transplant recipients—a clinician primer. J Clin Med . 2021;10:3227. doi: 10.3390/jcm10153227. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Herasevich S, Frank RD, Hogan WJ, Alkhateeb H, Limper AH, Gajic O, et al. Post-transplant and in-hospital risk factors for ARDS after hematopoietic stem cell transplantation. Respir Care . 2023;68:77–86. doi: 10.4187/respcare.10224. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Capizzi SA, Kumar S, Huneke NE, Gertz MA, Inwards DJ, Litzow MR, et al. Peri-engraftment respiratory distress syndrome during autologous hematopoietic stem cell transplantation. Bone Marrow Transplant . 2001;27:1299–1303. doi: 10.1038/sj.bmt.1703075. [DOI] [PubMed] [Google Scholar]
- 5. Wenger DS, Triplette M, Crothers K, Cheng GS, Hill JA, Milano F, et al. Incidence, risk factors, and outcomes of idiopathic pneumonia syndrome after allogeneic hematopoietic cell transplantation. Biol Blood Marrow Transplant . 2020;26:413–420. doi: 10.1016/j.bbmt.2019.09.034. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Wieruszewski PM, May HP, Peters SG, Gajic O, Hogan WJ, Dierkhising RA, et al. Characteristics and outcome of periengraftment respiratory distress syndrome after autologous hematopoietic cell transplant. Ann Am Thorac Soc . 2021;18:1013–1019. doi: 10.1513/AnnalsATS.202008-1032OC. [DOI] [PubMed] [Google Scholar]
- 7. Zhang Z, Wang C, Peters SG, Hogan WJ, Hashmi SK, Litzow MR, et al. Epidemiology, risk factors, and outcomes of diffuse alveolar hemorrhage after hematopoietic stem cell transplantation. Chest . 2021;159:2325–2333. doi: 10.1016/j.chest.2021.01.008. [DOI] [PubMed] [Google Scholar]
- 8. Yadav H, Nolan ME, Bohman JK, Cartin-Ceba R, Peters SG, Hogan WJ, et al. Epidemiology of acute respiratory distress syndrome following hematopoietic stem cell transplantation. Crit Care Med . 2016;44:1082–1090. doi: 10.1097/CCM.0000000000001617. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Brower RG, Matthay MA, Morris A, Schoenfeld D, Thompson BT, Wheeler A, Acute Respiratory Distress Syndrome Network Ventilation with lower tidal volumes as compared with traditional tidal volumes for acute lung injury and the acute respiratory distress syndrome. N Engl J Med . 2000;342:1301–1308. doi: 10.1056/NEJM200005043421801. [DOI] [PubMed] [Google Scholar]
- 10. National Heart, Lung, and Blood Institute Acute Respiratory Distress Syndrome (ARDS) Clinical Trials Network; Wiedemann HP. Wheeler AP. Bernard GR. Thompson BT. Hayden D. deBoisblanc B. et al. Comparison of two fluid-management strategies in acute lung injury. N Engl J Med . 2006;354:2564–2575. doi: 10.1056/NEJMoa062200. [DOI] [PubMed] [Google Scholar]
- 11. Guérin C, Reignier J, Richard JC, Beuret P, Gacouin A, Boulain T, et al. PROSEVA Study Group Prone positioning in severe acute respiratory distress syndrome. N Engl J Med . 2013;368:2159–2168. doi: 10.1056/NEJMoa1214103. [DOI] [PubMed] [Google Scholar]
- 12. Papazian L, Forel JM, Gacouin A, Penot-Ragon C, Perrin G, Loundou A, et al. ACURASYS Study Investigators Neuromuscular blockers in early acute respiratory distress syndrome. N Engl J Med . 2010;363:1107–1116. doi: 10.1056/NEJMoa1005372. [DOI] [PubMed] [Google Scholar]
- 13. Herasevich S, Frank RD, Bo H, Alkhateeb H, Hogan WJ, Gajic O, et al. Pretransplant risk factors can predict development of acute respiratory distress syndrome after hematopoietic stem cell transplantation. Ann Am Thorac Soc . 2021;18:1004–1012. doi: 10.1513/AnnalsATS.202004-336OC. [DOI] [PubMed] [Google Scholar]
- 14. Kor DJ, Carter RE, Park PK, Festic E, Banner-Goodspeed VM, Hinds R, et al. US Critical Illness and Injury Trials Group: Lung Injury Prevention with Aspirin Study Group (USCIITG: LIPS-A) Effect of aspirin on development of ARDS in at-risk patients presenting to the emergency department: the LIPS—a randomized clinical trial. JAMA . 2016;315:2406–2414. doi: 10.1001/jama.2016.6330. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP. Transparent peporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. Ann Intern Med . 2015;162:55–63. doi: 10.7326/M14-0697. [DOI] [PubMed] [Google Scholar]
- 16. Ranieri VM, Rubenfeld GD, Thompson BT, Ferguson ND, Caldwell E, Fan E, et al. ARDS Definition Task Force Acute respiratory distress syndrome: the Berlin definition. JAMA . 2012;307:2526–2533. doi: 10.1001/jama.2012.5669. [DOI] [PubMed] [Google Scholar]
- 17. Dhungana P, Serafim LP, Ruiz AL, Bruns D, Weister TJ, Smischney NJ, et al. Machine learning in data abstraction: a computable phenotype for sepsis and septic shock diagnosis in the intensive care unit. World J Crit Care Med . 2019;8:120–126. doi: 10.5492/wjccm.v8.i7.120. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Robin X, Turck N, Hainard A, Tiberti N, Lisacek F, Sanchez JC, et al. pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinformatics . 2011;12:77. doi: 10.1186/1471-2105-12-77. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Kuhn M. Building predictive models in R using the caret package J Stat Softw 2008. 28 1 26 27774042 [Google Scholar]
- 20. Friedman J, Hastie T, Tibshirani R. Regularization paths for generalized linear models via coordinate descent. J Stat Softw . 2010;33:1–22. [PMC free article] [PubMed] [Google Scholar]
- 21. Parimon T, Madtes DK, Au DH, Clark JG, Chien JW. Pretransplant lung function, respiratory failure, and mortality after stem cell transplantation. Am J Respir Crit Care Med . 2005;172:384–390. doi: 10.1164/rccm.200502-212OC. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Ahmed A, Kojicic M, Herasevich V, Gajic O. Early identification of patients with or at risk of acute lung injury. Neth J Med . 2009;67:268–271. [PubMed] [Google Scholar]
- 23. Thompson BT, Chambers RC, Liu KD. Acute respiratory distress syndrome. N Engl J Med . 2017;377:1904–1905. doi: 10.1056/NEJMc1711824. [DOI] [PubMed] [Google Scholar]
- 24. Tomblyn M, Chiller T, Einsele H, Gress R, Sepkowitz K, Storek J, et al. Center for International Blood and Marrow Research; National Marrow Donor program; European Blood and MarrowTransplant Group; American Society of Blood and Marrow Transplantation; Canadian Blood and Marrow Transplant Group; Infectious Diseases Society of America; Society for Healthcare Epidemiology of America; Association of Medical Microbiology and Infectious Disease Canada; Centers for Disease Control and Prevention Guidelines for preventing infectious complications among hematopoietic cell transplantation recipients: a global perspective. Biol Blood Marrow Transplant . 2009;15:1143–1238. doi: 10.1016/j.bbmt.2009.06.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Darmon M, Bourmaud A, Georges Q, Soares M, Jeon K, Oeyen S, et al. Changes in critically ill cancer patients’ short-term outcome over the last decades: results of systematic review with meta-analysis on individual data. Intensive Care Med . 2019;45:977–987. doi: 10.1007/s00134-019-05653-7. [DOI] [PubMed] [Google Scholar]