Skip to main content
Elsevier - PMC COVID-19 Collection logoLink to Elsevier - PMC COVID-19 Collection
. 2023 Feb 16;76:154272. doi: 10.1016/j.jcrc.2023.154272

Incidence, risk factors and pre-emptive screening for COVID-19 associated pulmonary aspergillosis in an era of immunomodulant therapy

Rebecca van Grootveld a,b,, Martha T van der Beek a, Nico AF Janssen c,d,e, Mehmet Ergün c, Karin van Dijk f, Carina Bethlehem g, Susanne Stads h, Judith van Paassen a, Leo MA Heunks f,i, Catherine SC Bouman f, Monique HE Reijers c, Roger J Brüggeman c, Frank L van de Veerdonk c, Sjoerd HW van Bree j, Charlotte HSB van den Berg k, Marnix Kuindersma l, Joost Wauters m, Albertus Beishuizen n, Paul E Verweij b,c, Jeroen A Schouten c; , on behalf of the CAPA2.0 study groupa,c,f,g,h,j,k,l,m,n
PMCID: PMC9934852  PMID: 36801598

Abstract

Purpose

COVID-19 associated pulmonary aspergillosis (CAPA) is associated with increased morbidity and mortality in ICU patients. We investigated the incidence of, risk factors for and potential benefit of a pre-emptive screening strategy for CAPA in ICUs in the Netherlands/Belgium during immunosuppressive COVID-19 treatment.

Materials and methods

A retrospective, observational, multicentre study was performed from September 2020–April 2021 including patients admitted to the ICU who had undergone diagnostics for CAPA. Patients were classified based on 2020 ECMM/ISHAM consensus criteria.

Results

CAPA was diagnosed in 295/1977 (14.9%) patients. Corticosteroids were administered to 97.1% of patients and interleukin-6 inhibitors (anti-IL-6) to 23.5%. EORTC/MSGERC host factors or treatment with anti-IL-6 with or without corticosteroids were not risk factors for CAPA. Ninety-day mortality was 65.3% (145/222) in patients with CAPA compared to 53.7% (176/328) without CAPA (p = 0.008). Median time from ICU admission to CAPA diagnosis was 12 days. Pre-emptive screening for CAPA was not associated with earlier diagnosis or reduced mortality compared to a reactive diagnostic strategy.

Conclusions

CAPA is an indicator of a protracted course of a COVID-19 infection. No benefit of pre-emptive screening was observed, but prospective studies comparing pre-defined strategies would be required to confirm this observation.

Keywords: Aspergillus fumigatus, Invasive pulmonary aspergillosis, Invasive fungal infections, COVID-19, SARS-CoV-2, Intensive care unit, CAPA

1. Background

Invasive aspergillosis (IA) has been reported in patients with COVID-19 admitted to the intensive care unit (ICU) requiring ventilatory support since the beginning of the COVID-19 pandemic [[1], [2], [3]]. During the first COVID-19 period (March 2020–August 2020) patients were not routinely treated with immunosuppressive medication. After the RECOVERY trial and a meta-analysis showed that patients with COVID-19 receiving either invasive mechanical ventilation or oxygen alone benefited from treatment with systemic corticosteroids this became the mainstay of treatment [4,5]. Subsequently, interleukin-6 inhibitors (anti-IL-6), were introduced in clinical practice after two randomized controlled trials showed lower mortality in patients with respiratory failure treated with anti-IL-6 compared to the control group. Most patients in both studies were treated with corticosteroids, so the beneficial effect could be explained by a combination of anti-IL-6 on top of corticosteroids [6,7]. From September 2020 patients with COVID-19 receiving oxygen or invasive mechanical ventilation were routinely treated with dexamethasone and from January 2021 anti-IL-6 was added to the treatment guideline in critically ill ventilated patients in the Netherlands. Anti-IL-6 was not routinely given in Belgium. Immunosuppressive treatment is classically a risk factor for IA [8]. Therefore, it was anticipated that the incidence of IA would increase. Previous studies have shown that patients with COVID-19 admitted to the ICU were at risk of developing COVID-19 associated pulmonary aspergillosis (CAPA) after approximately 7.28 days of mechanical ventilation [9] and CAPA mortality rates exceed 50% [10,11]. Any form of diagnostic screening for Aspergillus might lead to earlier detection of CAPA and thus earlier initiation of antifungal treatment and possibly reduce mortality. The current study investigated the incidence and characteristics of CAPA in patients receiving immunosuppressive treatment for COVID-19. Furthermore, the potential benefits of a pre-emptive screening strategy for CAPA were studied.

2. Materials & methods

2.1. Study design

A retrospective, observational, multicentre study was performed from September 2020–April 2021 in patients with COVID-19 admitted to the ICU who had undergone diagnostic procedures for CAPA. Clinical data were collected and patients were classified based on 2020 ECMM/ISHAM consensus criteria [12] with one minor modification: all positive Aspergillus BAL PCR results were considered relevant irrespective of cycle threshold-value. These criteria are further referred to as CAPA criteria. Patients with positive Aspergillus test results who did not fulfil CAPA criteria were classified as colonised and included in the group of patients without CAPA. Patients with CAPA were compared to patients without CAPA.

2.2. Study population

All patients with PCR confirmed SARS-CoV-2 infection admitted to the ICU who had undergone diagnostic procedures including broncho-alveolar lavage (BAL) sampling, non-bronchoscopic lavage (NBL) sampling, serum galactomannan (GM) testing and/or blood Aspergillus PCR testing.

2.3. Participating ICUs

ICUs in the Netherlands were invited to take part in this retrospective study via a newsletter issued regularly during the COVID-19 pandemic by the Dutch Intensive Care Society (NVIC). Centres that expressed interest were invited to participate and received financial support for data collection. One large Belgian centre was invited to take part as well. Funding was provided by the Dutch National Institute for Public Health and the Environment (RIVM) and the ZonMw COVID-19 Programme.

2.4. Data collection

Clinical data were collected by the participating medical centres and captured in an electronic case report form (eCRF) created in Castor EDC, and included demographics, comorbidities, EORTC/MSGERC host factors, severity of illness scores at ICU admission (APACHE IV and SOFA score), COVID-19 treatment, CAPA diagnosis, CAPA treatment, confirmed CAPA tracheobronchitis, length of stay and mortality. The cumulative dose of corticosteroids (prednisone equivalent) was calculated without collecting data about corticosteroid type, duration or dose. Participating centres were requested to report all GM results and only positive results of the other tests. The eCRF included questions about the timing of the clinical CAPA suspicion and clinical CAPA diagnosis that was documented in the electronic health record.

2.5. Ethics

All ethical considerations were in accordance with the ethical principles as posed in the declaration of Helsinki, the principles of good clinical practice, as well as current national legislation. The study was initiated by Radboudumc, Nijmegen, The Netherlands and ethical approval was granted by the medical ethical committee Arnhem-Nijmegen (CMO 2020–6339). All other participating centres received approval from their medical ethical committee before participating in the study. An opt-out system was employed, ensuring that medical data from patients who objected against use of their clinical data were not included in the study. Data were pseudonymised and data entry was performed by local physicians or research nurses.

2.6. Statistical analyses

CAPA incidence was the primary study outcome. Secondary outcomes included 30- and 90-day mortality, time to event (first positive mycological test, clinical CAPA suspicion and clinical CAPA diagnosis during ICU stay) and incidence of triazole resistance. Further analyses were performed to determine risk factors for CAPA, and to establish the effect of pre-emptive diagnostic screening for CAPA. Categorical variables were reported as numbers and percentages. Continuous variables were reported as median and interquartile range (IQR). Groups were compared with the Mann Whitney U test for continuous data and Chi-square test or Fisher's exact test for categorical data depending on sample size. Proportion confidence intervals were calculated with the Wilson score interval. Survival curves were constructed with Kaplan-Meier and compared with the log-rank test. A generalized estimating equation (GEE) logistic regression model was applied to the data to identify baseline variables at time of admission to the ICU associated with CAPA and 30-day mortality. Treatment for COVID-19 or pre-emptive screening for CAPA were considered baseline variables. A GEE model was chosen because each centre had their own protocol for patients with CAPA (centres were included as subject variables and patients as within-subject variables). Statistically significant variables from GEE univariable logistic regression analysis that were considered clinically relevant were included in the GEE multivariable logistic regression analysis. Odds ratios (OR) with their 95%-CI were reported. Variables were included in GEE logistic regression model if they were reported in >5% of patients and all EORTC/MSGERC criteria were grouped together. Two sided tests were used and a p-value of <0.05 was considered statistically significant. Statistical analyses were performed in SPSS version 25.0 and Microsoft Excel Version 2180.

3. Results

3.1. CAPA incidence

In the ten participating centres (5 university medical centres (UMCs) and 5 non-UMCs) 1977 patients with COVID-19 were admitted to the ICU in the research period (total cohort). Of these, 793 patients had undergone diagnostic procedures for CAPA and were included in our study (diagnostic cohort). CAPA incidence in the total cohort was 14.9% (295/1977) (95%-CI: 13.4–16.6%) (Fig. 1 ), with variable incidences in the centres (range: 5.4–27.3%). In the diagnostic cohort the incidence of CAPA was 37.2% (295/793) (95%-CI: 33.9–40.6%) (7 proven including 5 with histologically-confirmed tracheobronchitis, 246 probable, 42 possible). In three patients (1 probable and 2 possible) post-mortem autopsy reported evidence of IA. A clinical diagnosis of tracheobronchitis was made in 15/673 (2.2%) patients that could be classified as proven (6), probable (7) or no CAPA (2).

Fig. 1.

Fig. 1

Total number of patients with COVID-19 admitted to the ICU in the study period (total cohort) and patients undergoing CAPA diagnostics (diagnostic cohort).

CAPA classification according to 2020 ECMM/ISHAM consensus criteria (proven, probable or possible CAPA) [12].

Patients with positive Aspergillus test results who did not fulfil CAPA criteria were classified as colonised. Patients who did not have any positive Aspergillus test results were classified as no CAPA.

CAPA: COVID-19 associated pulmonary aspergillosis, ICU: intensive care unit.

3.2. Baseline characteristics and risk factors for CAPA of the study population (Table 1 and supplemental table 1)

Table 1.

Baseline characteristics of patients included in the diagnostic cohort with and without CAPA.

No CAPA (n = 498) CAPA (n = 295) Total (n = 793) Significance level
Demographics Age Median (IQR) 65 (56–72) 67 (61–72) 66 (58–72) <0.001
BMI (787) Median (IQR) 29 (25.4–33) 27.8 (25.531.5) 28.4 (25.432.4) 0.115
Sex (male) n (%) 356 71.5 239 81.0 595 75.0 0.003
Smoking Current or former smoker (493) n (%) 151 49.5 119 63.3 270 54.8 0.003
Comorbidities Any comorbidities n (%) 406 81.5 243 82.4 649 81.8 0.765
Acute leukaemia n (%) 3 0.6 2 0.7 5 0.6 1
Stem cell transplantation n (%) 4 0.8 5 1.7 9 1.1 0.305
Other haematological malignancy n (%) 18 3.6 13 4.4 31 3.9 0.578
Solid organ transplant n (%) 31 6.2 13 4.4 44 5.5 0.28
Cardiovascular disease n (%) 275 55.2 165 55.9 440 55.5 0.846
Diabetes mellitus n (%) 136 27.3 99 33.6 235 29.6 0.062
Asthma n (%) 59 11.8 28 9.5 87 11.0 0.305
COPD n (%) 48 9.6 32 10.8 80 10.1 0.585
Cystic fibrosis n (%) 2 0.4 0 0.0 2 0.3 0.532
Pulmonary tuberculosis n (%) 2 0.4 1 0.3 3 0.4 1
Multiple sclerosis n (%) 1 0.2 1 0.3 2 0.3 1
Liver cirrhosis n (%) 1 0.2 1 0.3 2 0.3 1
Inflammatory bowel disease n (%) 7 1.4 2 0.7 9 1.1 0.497
Rheumatological disease n (%) 30 6.0 19 6.4 49 6.2 0.814
Psoriasis n (%) 4 0.8 0 0.0 4 0.5 0.303
HIV/AIDS n (%) 0 0.0 1 0.3 1 0.1 0.372
Solid organ malignancy n (%) 39 7.8 27 9.2 66 8.3 0.515
Other malignancy n (%) 0 0.0 1 0.3 1 0.1 0.372
Chronic kidney disease n (%) 32 6.4 17 5.8 49 6.2 0.708
Thyroid disease n (%) 36 7.2 18 6.1 54 6.8 0.542
Other n (%) 132 26.5 84 28.5 216 27.2 0.547
EORTC/MSGERC criteria* [8] Recent history of neutropenia (748) n (%) 1 0.2 1 0.4 2 0.3 1
Active haematological malignancy (790) n (%) 9 1.8 7 2.4 16 2.0 0.577
Allogeneic stem cell transplantation n (%) 4 0.8 3 1.0 7 0.9 0.715
Prolonged use of corticosteroids (756) n (%) 28 5.9 17 6.0 45 6.0 0.946
T-cell immunosuppressants (789) n (%) 27 5.5 16 5.4 43 5.4 0.994
B-cell immunosuppressants (790) n (%) 11 2.2 6 2.0 17 2.2 0.86
Inherited severe immunodeficiency n (%) 0 0.0 0 0.0 0 0.0 1
Any EORTC criterium (791) n (%) 64 12.9 36 12.2 100 12.6 0.775
APACHE IV and SOFA scores on ICU admission APACHE IV (624) Median (IQR) 53 (28–69) 59 (38–73) 57 (31–71) 0.005
SOFA (635) Median (IQR) 6 (4–8) 7 (4–9) 6 (4–8) 0.016
Supportive care on ICU SDD (738) n (%) 335 73.1 216 77.1 551 74.7 0.226
Renal replacement therapy (786) n (%) 85 17.2 80 27.5 165 21.0 0.001
Vasopressors and/or inotropes (789) n (%) 441 88.9 277 94.5 718 91.0 0.008
Most invasive ventilatory support during ICU stay ECMO n (%) 39 7.8 16 5.4 55 6.9 0.197
Invasive n (%) 437 87.8 274 92.9 711 89.7 0.022
Non-invasive n (%) 20 4.0 4 1.4 24 3.0 0.035
None n (%) 2 0.4 1 0.3 3 0.4 1
Treatment for COVID-19 Remdesivir (792) n (%) 71 14.3 31 10.5 102 12.9 0.132
Corticosteroids (792) n (%) 479 96.2 290 98.6 769 97.1 0.047
Tocilizumab (792) n (%) 113 22.7 73 24.8 186 23.5 0.493
Corticosteroids and tocilizumab (792) n (%) 111 22.3 73 24.8 184 23.2 0.413
Time to event from ICU admission in days Performance of BAL or NBL (647) Median (IQR) 6 (3−11) 7 (4–12) 7 (3–11) 0.033
First positive BAL or NBL GM (193) Median (IQR) 10 (5–15) 10 (5–15) NA
First positive mycological test§ (318) Median (IQR) 9 (3–15) 8 (4–13) 8 (4–13) 0.95
Clinical suspicion CAPA§ (444) Median (IQR) 7 (4–11) 8 (5–12) 8 (4–12) 0.162
Clinical diagnosis CAPA§ (216) Median (IQR) 12 (9–16) 12 (7–16) 12 (7–16) 0.729
Length of ICU stay in days Length of ICU stay (729) Median (IQR) 21 (13–36) 28 (18–42) 23 (14–38) <0.001
Mortality Mortality at 30 days after ICU admission (658) n (%) 134 33.8 94 36.0 228 34.7 0.551
Mortality at 90 days after ICU admission (550) n (%) 176 53.7 145 65.3 321 58.4 0.008
Mortality during ICU stay (789) n (%) 170 34.2 138 47.3 308 39.0 <0.001

*Solid organ transplant can be found under comorbidities, Date of first GM, date of first BAL or NBL not available, If NBL GM was positive based on a GM index >1.2 twice or more, the date of the first positive NBL GM result was selected, §According to medical centres, In patients in whom the total ICU stay was known.

BAL: bronchoscopic alveolar lavage, BMI: body mass index, GM: galactomannan, ICU: intensive care unit, IQR: interquartile range, NBL: non-bronchoscopic lavage, SDD: selective digestive decontamination.

The diagnostic cohort consisted of mostly male patients with a median age of 66 years. Patients with CAPA were slightly older. No significant differences were observed in comorbidities. Presence of EORTC/MSGERC risk factors was similar (CAPA 12.2% vs no CAPA 12.9%) in both groups. The majority of patients were treated with corticosteroids (97.1%). Treatment with anti-IL-6 with or without corticosteroids was comparable in patients with and without CAPA (CAPA: 24.8%, no CAPA: 22.7%). There was no difference in the cumulative dose of corticosteroids per kg bodyweight in patients with and without CAPA from ICU admission to first diagnosis of CAPA, preadmission to hospital or preadmission to the ICU (in hospital), but the median cumulative dose of corticosteroids per kg bodyweight during total ICU stay was higher for patients with CAPA 8.33 vs 5.73 mg prednisone/kg bodyweight (p = 0.015).

Patients with CAPA had a slightly higher APACHE IV score on admission to the ICU and were treated with renal replacement therapy (RRT) and vasopressors and/or inotropes more frequently (Table 1).

Older age, male sex, higher APACHE IV and SOFA score on ICU admission and treatment with remdesivir (decreased risk) were associated with CAPA diagnosis in univariable analysis, and all remained associated with CAPA in multivariable analysis (SOFA score not included in analysis) (Table 2 ).

Table 2.

Uni- and multivariable generalized estimating equations (GEE) logistic regression model for CAPA.

Variables OR (95%-CI)
Univariable analysis
Age 1.033 (1.024−1.042)
BMI 0.981 (0.954−1.008)
Sex (male) 1.702 (1.190−2.435)
Any comorbidities 1.059 (0.636−1.765)
Cardiovascular disease 1.029 (0.873−1.214)
Diabetes mellitus 1.344 (0.954−1.895)
Asthma 0.780 (0.545−1.117)
COPD 1.141 (0.795−1.637)
Rheumatological disease 1.074 (0.640−1.802)
Solid organ malignancy 1.185 (0.756−1.859)
Chronic kidney disease 0.890 (0.586−1.354)
Thyroid disease 0.834 (0.514−1.353)
Any EORTC/MSGERC criterium [8] 0.938 (0.574−1.534)
APACHE IV at ICU admission 1.008 (1.003−1.013)
SOFA at ICU admission 1.073 (1.002−1.149)
Remdesivir* 0.709 (0.511−0.984)
Corticosteroids* 2.875 (0.795–10.402)
Tocilizumab* 1.125 (0.797−1.589)
Corticosteroids and tocilizumab* 1.151 (0.804−1.650)
Pre-emptive screening 0.742 (0.507−1.085)



Multivariable analysis
Age 1.025 (1.017−1.034)
Sex (male) 1.730 (1.257−2.380)
APACHE IV at ICU admission 1.005 (1.001−1.010)
Remdesivir* 0.668 (0.456−0.976)

*Treatment for COVID-19, Medical centres that pre-emptively screened for CAPA: Aspergillus diagnostics performed in all patients with COVID-19 admitted to the ICU irrespective if there was a suspicion of CAPA.

OR: odds ratio, 95%-CI: 95% confidence interval.

3.3. Time to event

Median time from ICU admission until the first positive mycological test, clinical CAPA suspicion and clinical CAPA diagnosis by the treating physician was eight, eight and twelve days respectively (Table 1). Fig. 2 depicts on which day the first NBL/BAL GM test became positive from ICU admission.

Fig. 2.

Fig. 2

First positive NBL or BAL GM sample after ICU admission in days.

Screening: medical centres that screened for CAPA systematically (pre-emptive screening strategy).

No screening: medical centres that performed diagnostics for Aspergillus spp. on clinical suspicion of CAPA (reactive diagnostic strategy).

BAL: bronchoscopic alveolar lavage, CAPA: COVID-19 associated pulmonary aspergillosis, GM: galactomannan, ICU: intensive care unit, ICUA: ICU admission, NBL: non-bronchoscopic lavage.

3.4. Mortality

ICU mortality in the total cohort was 500/1977 (25.3%). In the diagnostic cohort mortality during ICU stay was 39%, and was significantly higher for patients with CAPA 47.3% vs 34.2% (p 〈0.001). There was no statistically significant difference in 30-day mortality between patients with and without CAPA, whereas 90-day mortality was increased in patients with CAPA (CAPA: 145/222 (65.3%) vs non-CAPA: 176/328 (53.7%), p = 0.008) (Table 1). Mortality in patients with probable and possible CAPA was similar and was lower than patients with proven CAPA (Fig. 3 ). Ninety-day mortality in patients with tracheobronchitis was 11/14.

Fig. 3.

Fig. 3

a. Kaplan-Meier survival curve of patients with and without CAPA. Log-rank test: p = 0.020. b. Kaplan-Meier survival curve of patients classified as proven, probable/possible or no CAPA. Log-rank test: p = 0.006.

Of the 295 patients with CAPA, 192 (65%, 7 proven, 153 probable and 32 possible) were treated with an antifungal agent for ≥3 days. There was no difference in 30- or 90-day mortality between patients treated or not treated with an antifungal agent (30-day mortality: not treated 34/85 (40%), treated 60/176 (34.1%), p = 0.351; 90-day mortality: not treated 46/68 (67.6%), treated 99/154 (64.3%), p = 0.628). Also, when excluding patients with possible CAPA (data not shown).

Older age, any comorbidity, diabetes mellitus, chronic kidney disease, thyroid disease, any EORTC/MSGERC criterium and treatment with remdesivir (all increased risk), and corticosteroids or tocilizumab (both decreased risk) were associated with 30-day mortality in univariable analysis (Table 3 ). In multivariable analysis older age, any EORTC/MSGERC criterium and treatment with corticosteroids remained associated with 30-day mortality (Table 3).

Table 3.

Uni- and multivariable generalized estimating equations (GEE) logistic regression model for 30-day mortality.

Variables OR (95%-CI)
Univariable analysis
Age 1.071 (1.063−1.080)
BMI 0.976 (0.946−1.008)
Sex (male) 0.780 (0.554−1.097)
Any comorbidities 1.797 (1.133−2.846)
Cardiovascular disease 1.310 (0.583−1.811)
Diabetes mellitus 1.749 (1.218−2.512)
Asthma 1.001 (0.441−2.270)
COPD 1.564 (0.806−3.031)
Rheumatological disease 1.305 (0.710−2.401)
Solid organ malignancy 0.995 (0.650−1.523)
Chronic kidney disease 2.568 (1.132−5.824)
Thyroid disease 1.795 (1.021−3.155)
Any EORTC/MSGERC criterium [8] 2.273 (1.454−3.550)
APACHE IV at ICU admission 1.011 (0.999−1.022)
SOFA at ICU admission 1.034 (0.953−1.122)
Remdesivir* 1.531 (1.034−2.268)
Corticosteroids* 0.519 (0.411−0.656)
Tocilizumab* 0.614 (0.405−0.931)
Corticosteroids and tocilizumab* 0.628 (0.419−0.944)
Pre-emptive screening 1.578 (0.999−2.489)



Multivariable analysis
Age 1.073 (1.060−1.085)
Diabetes mellitus 1.415 (0.953−2.100)
Chronic kidney disease 1.728 (0.756−3.955)
Any EORTC/MSGERC criterium [8] 2.509 (1.468−4.284)
Remdesivir* 1.365 (0.876−2.125)
Corticosteroids* 0.492 (0.319−0.757)
Tocilizumab* 0.802 (0.495−1.298)

*Treatment for COVID-19, Medical centres that pre-emptively screened for CAPA: Aspergillus diagnostics performed in all patients with COVID-19 admitted to the ICU irrespective if there was a suspicion of CAPA.

OR: odds ratio, 95%-CI: 95% confidence interval.

3.5. Pre-emptive screening strategy for CAPA

All ICUs had their own protocol for suspected CAPA. Of the ten participating ICUs five screened for CAPA systematically (3 BAL or NBL on admission to ICU, of which 2 UMCs; 2 centres screened weekly on a tracheal or bronchial aspirate, both in UMCs) whilst the other ICUs performed diagnostics for Aspergillus spp. when CAPA was suspected (Fig. 4 ). Centres were considered screening centres if Aspergillus diagnostics were routinely performed in all patients with COVID-19 admitted to the ICU irrespective if there was a suspicion of CAPA (pre-emptive screening strategy). Centres were considered non-screening centres if Aspergillus diagnostics were only performed when there was suspicion of CAPA (reactive diagnostic strategy). The diagnostic tests that were routinely performed differed between the centres, although most often culture and BAL/NBL GM were performed. Two centres prophylactically treated patients with an antifungal agent: one systemically in patients receiving high-dose methylprednisolone and the other by inhalation therapy from initiation of invasive mechanical ventilation [13] (neither of them during the entire study period).

Fig. 4.

Fig. 4

Flow chart depicting the different CAPA protocols of centres that screened for CAPA pre-emptively.

Pre-emptive screening strategy: centres in which Aspergillus diagnostics were routinely performed in all patients with COVID-19 admitted to the ICU, Reactive diagnostic strategy: centres in which Aspergillus diagnostics were only performed when there was suspicion of CAPA.

BA: bronchial aspirate, BAL: bronchoalveolar lavage, CAPA: COVID-19 associated pulmonary aspergillosis, GM: galactomannan, ICU: intensive care unit, NBL: non-bronchoscopic lavage, TA: tracheal aspirate.

Patients in whom screening was performed were slightly younger and more often had a classical host risk factor for IA (supplemental table 2). The average time frame between ICU admission and collection of the first NBL or BAL GM was 6 days (IQR: 2–10) for centres with a pre-emptive screening strategy and 10 days (IQR: 6–14) for centres with a reactive diagnostic strategy, p < 0.001. The first positive BAL or NBL GM result was found after 9 days (IQR: 5–15) in centres that screened and 10 days (IQR: 7–14) in centres that did not screen, p = 0.362. Treatment initiation from ICU admission was 10 days in centres that screened vs 12 days in those that did not. There was no difference in time from clinical CAPA suspicion/diagnosis until treatment initiation (supplemental table 2).

CAPA incidences were similar in the centres with a pre-emptive screening strategy or a reactive diagnostic strategy (210/593 (35.4%) vs 85/200 (42.5%), p = 0.073). Both 30- and 90-day mortality were higher in hospitals that screened for CAPA (supplemental table 2). The Kaplan-Meier survival curve can be found in supplemental Fig. 1. A pre-emptive screening strategy was not associated with CAPA incidence or 30-day mortality in univariable analysis (Table 2, Table 3).

3.6. CAPA classification and mortality

Different positive test combinations of patients with probable or possible CAPA are depicted in supplemental table 3 and 4. An evaluation was performed of positive test results for CAPA classification and mortality. Test results of patients with probable and possible CAPA were ranked on the basis of the positive test results theoretically most indicative of Aspergillus invasiveness or load: 9 serum GM, 1 PCR blood, 90 BAL microscopy, 16 NBL microscopy, 51 BAL culture, 12 NBL culture, 40 BAL PCR, 55 BAL GM and 14 NBL GM. Mortality was highest in patients with at least a positive serum GM, followed by at least positive BAL/NBL microscopy (Fig. 5 and supplemental tables 5 and 6). Patients with BAL/NBL GM as maximum positive test had lower or similar mortality compared to patients with negative test results.

Fig. 5.

Fig. 5

30- and 90-day mortality in patients with positive test results ranked on the basis of positive test results theoretically indicative of Aspergillus invasiveness or load.

The estimated probability of death at day 30 (5a) or 90 (5b) with corresponding 95% confidence interval (95%-CI), calculated with the Wilson score interval.

Patients with probable or possible CAPA according to 2020 ECMM/ISHAM consensus criteria [12], colonised (patients with positive Aspergillus test results who did not fulfil CAPA criteria) or no CAPA.

Ranking: serum GM/PCR blood > microscopy: BAL/NBL microscopy > culture: BAL/NBL culture > PCR: BAL PCR > GM: BAL/NBL GM*†‡ > colonised > negative test results.

BAL: bronchoscopic alveolar lavage, GM: galactomannan, NBL: non-bronchoscopic lavage.

* NBL GM positivity according to 2020 ECMM/ISHAM consensus criteria [12].

† 30- and 90-day mortality and estimated probability of death with 95%-CI of patients with a positive BAL GM: day 30: 29.7% (31.6%; 95%-CI: 17.5–45.9); day 90: 56.7% (55.9%; 95%-CI: 39.2–72.6).

‡ 30- and 90-day mortality and estimated probability of death with 95%-CI of patients with a positive NBL GM: day 30: 14.3% (22%; 95%-CI: 4–39.9); day 90: 55.6% (53.9%; 95%-CI: 26.7–81.1).

3.7. Triazole resistance

Aspergillus species were cultured from samples of 188 patients of which 175 contained A. fumigatus (supplemental table 7) and phenotypical tests to detect triazole resistance were performed in 80/188 (42.6%) patients. Triazole resistance was detected in 11/80 (13.8%) patients. In total, 125 patients had one or more samples in which Aspergillus DNA was detected by PCR. Resistance PCR yielded results in 35/125 (28%) patients, of which 4/35 (11.4%) had resistance associated mutations. Combined culture and PCR results indicated triazole resistance in 12/93 (12.9%) patients. 30- and 90-day mortality were similar in patients with a triazole-resistant or triazole-susceptible isolate.

4. Discussion

In this large observational multicentre study incidence of CAPA, characteristics of and risk factors for CAPA and a pre-emptive screening strategy for CAPA were investigated.

In the total cohort a CAPA incidence of 14.9% was found. This finding is comparable to a previous multicentre study (Dutch CAPA1.0 study) that observed an incidence of 15% [14] with the ECMM/ISHAM consensus definition [12] although different inclusion criteria were used. The Dutch CAPA1.0 study was performed from February–May 2020, when treatment with corticosteroids and anti-IL-6 was not standard of care for patients with COVID-19 admitted to the ICU with respiratory insufficiency. We expected to find a higher incidence of CAPA because these immunomodulating drugs are now being used routinely, but CAPA incidences remained similar. This might be explained by the relatively short duration or dose of corticosteroids so that the immunomodulating effects for becoming a host for IA are minor, or possibly by the improved care for COVID-19 and prevention of SARS-CoV-2 infection by vaccination. Variable CAPA incidences have been reported, which can be explained by the use of different classifications at the start of the COVID-19 pandemic before the ECMM/ISHAM consensus definition had been published [10]. A recently published meta-analysis reported a pooled reported CAPA incidence of 10% (95%-CI: 7–14%) and showed that the incidence was lower when patients were reclassified and also differed depending on the classification used [11].

Treatment with anti-IL-6 alone or in combination with corticosteroids was not a risk factor for CAPA. Neither was there a difference in cumulative dose of corticosteroids before CAPA was suspected in patients with and without CAPA. Patients with CAPA were treated with a higher cumulative dose of corticosteroids during their total ICU stay, but this may be explained by the longer ICU stay and is therefore not a certain CAPA risk factor. An association of anti-IL-6 with occurrence of CAPA has been described before [15,16], but the number of patients treated with anti-IL-6 were small in both previous studies (39/581 (6.7%) [15] and 38/506 (7.5%) [16]) as compared to our series. Looking into other possible risk factors for CAPA, we found that EORTC/MSGERC host criteria were not associated with CAPA. This is in concordance with other reports [11]. Risk factors we observed were older age, male sex and higher APACHE IV or SOFA score on ICU admission. Treatment with remdesivir decreased the risk for CAPA, but conclusions cannot be drawn from our data because its administration may not have been random and confounding by indication may have occurred.

In the diagnostic cohort, 90-day mortality was higher in patients with CAPA compared to patients without CAPA (65.3% vs 53.7%, p = 0.008). Other reports have found similar mortality rates in patients with CAPA [10,11,15]. Interestingly, we did not find a difference in mortality at 30 days which is in line with another study that did not find any difference in early mortality at 28 days, whereas a difference was seen at 84 days [15]. ICU mortality in the total cohort (including the diagnostic cohort) was considerably lower with 25%. The higher mortality in the diagnostic cohort might be attributable to the selection of patients in whom invasive diagnostics like BAL or NBL were performed, likely because of clinical deterioration or lack of clinical improvement during ICU stay. The increase in ICU mortality in confirmed CAPA patients was relatively modest (13.1%) in comparison to the increase in mortality of those found to have had an indication for BAL or NBL diagnostics compared to the total cohort (ICU mortality in the diagnostic cohort: 39%). Possibly, CAPA occurs mainly in patients who already have a worse prognosis because of a protracted course of the COVID-19 infection and mortality is mainly determined by the course of their COVID-19 rather than by CAPA. This hypothesis is supported by the fact that antifungal treatment did not improve outcome, regardless of the strength of CAPA diagnosis. Also, it is striking that other opportunistic pathogens like CMV and HSV seem to re-activate in COVID-19 patients with a protracted course in ICU [17,18]. Whether higher mortality as observed in patients with CAPA can be attributed to IA is thus unknown. In the diagnostic cohort, autopsies were only performed in a few patients, but did confirm CAPA in some. Autopsy reports seldom reveal proven CAPA [19,20]. Van de Veerdonk et al. [21] postulate that CAPA should be seen as a disease with a continuous spectrum, from colonisation, to tissue invasion and eventually angioinvasion and dissemination, in which multiple factors contribute to Aspergillus becoming invasive. Hard criteria, such as a cut-off value for GM or PCR as suggested in the 2020 ECMM/ISHAM definition [12] would do this no justice. The hypothesis that CAPA is a disease with a continuous spectrum was underlined in a study evaluating Aspergillus test results and mortality [22]. In this study mortality was incremental in the presence and classification of CAPA. Positive serum GM and serum (1,3)-β-D-glucan were associated with increased mortality in comparison to CAPA patients who were blood biomarker negative. Of the nine patients in our study who were serum GM positive, suggestive of angio-invasive disease, one was lost to follow-up at day 90 but of the remaining eight patients seven had died. Also, mortality appeared higher in patients with diagnostic tests indicative of a higher fungal load, despite being classified as possible CAPA. The 2020 ECMM/ISHAM definition [12] may therefore not be optimal to classify patients according to the strength of diagnostic evidence, as was also shown by the similar mortality in patients with possible and probable CAPA. Furthermore, not all patients with CAPA according to the definition [12] were treated with antifungal agents and one of the reasons for this was that there was no clinical CAPA suspicion. As was recently suggested, in order to correctly identify patients with CAPA, local epidemiology and clinical details need to be taken into account [23] as well as strong diagnostic evidence demonstrated by a combination of different positive markers for Aspergillus spp. [24].

The optimal diagnostic work-flow for CAPA is still unknown, which was reflected by the variable protocols retrieved from all participating ICUs. Some ICUs performed diagnostics when CAPA was suspected, whilst others pre-emptively screened for CAPA from the start of ICU admission. The diagnostic tests that were routinely performed differed per centre as did the method of screening. In our opinion the benefit of pre-emptive screening and its optimal form remains uncertain as we found no clear benefit of screening. However, due to the retrospective observational nature of this study results should be interpreted with caution. Prospective studies comparing pre-defined strategies are required to establish an optimal form of screening. The incidence of CAPA was similar in centres that screened and did not screen for CAPA. In addition, there was no difference in time from ICU admission until first positive GM between those hospitals. Higher mortality was observed in the hospitals that screened pre-emptively, possibly because screening was mostly performed in the UMCs, where more complex patients are treated. For example, patients in the pre-emptive screening group more often had an EORTC/MSGERC host risk factor for IA.

Our study has a number of limitations. Firstly, the variable CAPA protocols in the participating ICUs may have caused inclusion bias and may have influenced outcome. Secondly, negative test results and clinical parameters were not taken into account when classifying patients according to the CAPA classification whereas these may be relevant for the final diagnosis. Thirdly, sample size was not calculated as it was considered challenging to predict the number of patients eligible to participate in the study, because the characteristics and epidemiology of the pandemic were rapidly changing. Lastly, a comparison of the total cohort with the diagnostic cohort was not possible as clinical information was not collected of patients in the total cohort.

In conclusion, treatment with anti-IL-6 with or without corticosteroids was not a risk factor for CAPA and CAPA incidence was similar to the first COVID-19 period when treatment with immunosuppressive agents was not standard of care. In general, our observations support the hypothesis that CAPA is an indicator of a protracted course of the COVID-19 infection. We did not observe a benefit of pre-emptive screening for CAPA, but prospective studies comparing pre-defined strategies would be required to confirm this observation.

The following are the supplementary data related to this article.

Supplemental Fig. 1

Kaplan-Meier survival curves of medical centres with a pre-emptive screening strategy (screening) or a reactive diagnostic strategy (no screening) for CAPA.

mmc1.docx (37.2KB, docx)
Supplementary material

Supplemental Tables 1 - 7

mmc2.docx (61.7KB, docx)

Funding

Funding was provided by the Dutch National Institute for Public Health and the Environment (RIVM) and the ZonMw COVID-19 Programme.

CRediT authorship contribution statement

Rebecca van Grootveld: Data curation, Formal analysis, Investigation, Writing – original draft, Writing – review & editing. Martha T. van der Beek: Methodology, Supervision, Writing – review & editing. Nico A.F. Janssen: Conceptualization, Writing – review & editing. Mehmet Ergün: Investigation. Karin van Dijk: Investigation, Writing – review & editing. Carina Bethlehem: Investigation, Writing – review & editing. Susanne Stads: Investigation, Writing – review & editing. Judith van Paassen: Investigation, Writing – review & editing. Leo M.A. Heunks: Investigation, Writing – review & editing. Catherine S.C. Bouman: Investigation, Writing – review & editing. Monique H.E. Reijers: Conceptualization, Writing – review & editing. Roger J. Brüggeman: Conceptualization, Writing – review & editing. Frank L. van de Veerdonk: Conceptualization, Writing – review & editing. Sjoerd H.W. van Bree: Investigation, Writing – review & editing. Charlotte H.S.B. van den Berg: Investigation, Writing – review & editing. Marnix Kuindersma: Investigation, Writing – review & editing. Joost Wauters: Investigation, Writing – review & editing. Albertus Beishuizen: Investigation, Writing – review & editing. Paul E. Verweij: Conceptualization, Funding acquisition, Methodology, Supervision, Writing – review & editing. Jeroen A. Schouten: Conceptualization, Methodology, Supervision, Writing – review & editing.

Declaration of Competing Interest

Paul Verweij: Institution contracted for research grant: F2G and Gilead Sciences; Honoraria for lectures paid to institution: F2G, Gilead Sciences, Pfizer; Participation on a Data Safety Monitoring Board or Advisory Board paid to institution: F2G, Mundipharma.

Roger Brüggeman: Institution contracted for research grant: Astellas Pharma, Inc., Gilead Sciences, Merck Sharp & Dohme Corp., Pfizer; Consulting fees paid to institution: Astellas Pharma, Inc., F2G, Amplyx, Gilead Sciences, Merck Sharp & Dohme Corp., Mundipharma, Pfizer.

Joost Wauters: Investigator-initiated grants: Gilead, MSD, Pfizer; Consulting fees: Investigator-initiated grants from Gilead, Pfizer; Payment or honoraria for lectures, presentations, speakers bureaus, manuscript writing or educational events: Speakers fees from Gilead, Pfizer; Support for attending meetings and/or travel: travel grants from Gilead and Pfizer.

No potential financial or non-financial competing interest was reported by any of the other authors.

Acknowledgements

We would like to thank everyone who contributed to this paper. In particular we want to thank the CAPA 2.0 study group, Cato Jacobs, Simon Feys, Yves Debaveye, Michiel van Lookeren Campagne, Max Melchers, Hetty Kranen, Willem Dieperink, Jantine van Holten, Renée van Ditshuizen, Jelle Goeman and the LUMC CAPA study group.

Data availability

Research data is available upon reasonable request.

References

  • 1.Alanio A., Dellière S., Fodil S., Bretagne S., Mégarbane B. Prevalence of putative invasive pulmonary aspergillosis in critically ill patients with COVID-19. Lancet Respir Med. 2020;8(6) doi: 10.1016/S2213-2600(20)30237-X. e48-e9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Koehler P., Cornely O.A., Böttiger B.W., Dusse F., Eichenauer D.A., Fuchs F., et al. COVID-19 associated pulmonary aspergillosis. Mycoses. 2020;63(6):528–534. doi: 10.1111/myc.13096. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Rutsaert L., Steinfort N., Van Hunsel T., Bomans P., Naesens R., Mertes H., et al. COVID-19-associated invasive pulmonary aspergillosis. Ann Intensive Care. 2020;10(1):71. doi: 10.1186/s13613-020-00686-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Horby P., Lim W.S., Emberson J.R., Mafham M., Bell J.L., Linsell L., et al. RECOVERY Collaborative Group Dexamethasone in hospitalized patients with Covid-19. N Engl J Med. 2021;384(8):693–704. doi: 10.1056/NEJMoa2021436. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Sterne J.A.C., Murthy S., Diaz J.V., Slutsky A.S., Villar J., Angus D.C., et al. WHO rapid evidence appraisal for COVID-19 therapies (REACT) working group. Association between Administration of Systemic Corticosteroids and Mortality among Critically ill Patients with COVID-19: a Meta-analysis. Jama. 2020;324(13):1330–1341. doi: 10.1001/jama.2020.17023. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.RECOVERY Collaborative Group Tocilizumab in patients admitted to hospital with COVID-19 (RECOVERY): a randomised, controlled, open-label, platform trial. Lancet. 2021;397(10285):1637–1645. doi: 10.1016/S0140-6736(21)00676-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Gordon A.C., Mouncey P.R., Al-Beidh F., Rowan K.M., Nichol A.D., Arabi Y.M., et al. REMAP-CAP investigators. Interleukin-6 receptor antagonists in critically ill patients with Covid-19. N Engl J Med. 2021;384(16):1491–1502. doi: 10.1056/NEJMoa2100433. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Donnelly J.P., Chen S.C., Kauffman C.A., Steinbach W.J., Baddley J.W., Verweij P.E., et al. Revision and update of the consensus definitions of invasive fungal disease from the European Organization for Research and Treatment of Cancer and the mycoses study group education and research consortium. Clin Infect Dis. 2020;71(6):1367–1376. doi: 10.1093/cid/ciz1008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Chen W., Yin C., Zhong M., Hu B., Gao X., Zhang K., et al. Incidence and outcomes of patients with COVID-19 associated pulmonary aspergillosis (CAPA) in intensive care units: a systematic review and meta-analysis of 31 cohort studies. Ann Palliat Med. 2022;11(7) doi: 10.21037/apm-21-2043. s. [DOI] [PubMed] [Google Scholar]
  • 10.Verweij P.E., Brüggemann R.J.M., Azoulay E., Bassetti M., Blot S., Buil J.B., et al. Taskforce report on the diagnosis and clinical management of COVID-19 associated pulmonary aspergillosis. Intensive Care Med. 2021;47(8):819–834. doi: 10.1007/s00134-021-06449-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Kariyawasam R.M., Dingle T.C., Kula B.E., Vandermeer B., Sligl W.I., Schwartz I.S. Defining COVID-19-associated pulmonary aspergillosis: systematic review and meta-analysis. Clin Microbiol Infect. 2022;28(7):920–927. doi: 10.1016/j.cmi.2022.01.027. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Koehler P., Bassetti M., Chakrabarti A., Chen S.C.A., Colombo A.L., Hoenigl M., et al. Defining and managing COVID-19-associated pulmonary aspergillosis: the 2020 ECMM/ISHAM consensus criteria for research and clinical guidance. Lancet Infect Dis. 2021;21(6) doi: 10.1016/S1473-3099(20)30847-1. e149-e62. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Melchers M., van Zanten A.R.H., Heusinkveld M., Leeuwis J.W., Schellaars R., Lammers H.J.W., et al. Nebulized amphotericin B in mechanically ventilated COVID-19 patients to prevent invasive pulmonary aspergillosis: a retrospective cohort study. Crit Care Explor. 2022;4(5) doi: 10.1097/CCE.0000000000000696. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Janssen N.A.F., Nyga R., Vanderbeke L., Jacobs C., Ergün M., Buil J.B., et al. Multinational observational cohort study of COVID-19-associated pulmonary aspergillosis. Emerg Infect Dis. 2021;27(11):2892–2898. doi: 10.3201/eid2711.211174. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Prattes J., Wauters J., Giacobbe D.R., Salmanton-García J., Maertens J., Bourgeois M., et al. Risk factors and outcome of pulmonary aspergillosis in critically ill coronavirus disease 2019 patients-a multinational observational study by the European Confederation of Medical Mycology. Clin Microbiol Infect. 2022;28(4):580–587. doi: 10.1016/j.cmi.2021.08.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Gangneux J.P., Dannaoui E., Fekkar A., Luyt C.E., Botterel F., De Prost N., et al. Fungal infections in mechanically ventilated patients with COVID-19 during the first wave: the French multicentre MYCOVID study. Lancet Respir Med. 2022;10(2):180–190. doi: 10.1016/S2213-2600(21)00442-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Gatto I., Biagioni E., Coloretti I., Farinelli C., Avoni C., Caciagli V., et al. Cytomegalovirus blood reactivation in COVID-19 critically ill patients: risk factors and impact on mortality. Intensive Care Med. 2022;48(6):706–713. doi: 10.1007/s00134-022-06716-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Saade A., Moratelli G., Azoulay E., Darmon M. Herpesvirus reactivation during severe COVID-19 and high rate of immune defect. Infect Dis Now. 2021;51(8):676–679. doi: 10.1016/j.idnow.2021.07.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Kula B.E., Clancy C.J., Hong Nguyen M., Schwartz I.S. Invasive mould disease in fatal COVID-19: a systematic review of autopsies. Lancet Microbe. 2021;2(8) doi: 10.1016/S2666-5247(21)00091-4. e405-e14. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Polak S.B., Van Gool I.C., Cohen D., von der Thüsen J.H., van Paassen J. A systematic review of pathological findings in COVID-19: a pathophysiological timeline and possible mechanisms of disease progression. Mod Pathol. 2020;33(11):2128–2138. doi: 10.1038/s41379-020-0603-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.van de Veerdonk F.L., Brüggemann R.J.M., Vos S., De Hertogh G., Wauters J., Reijers M.H.E., et al. COVID-19-associated aspergillus tracheobronchitis: the interplay between viral tropism, host defence, and fungal invasion. Lancet Respir Med. 2021;9(7):795–802. doi: 10.1016/S2213-2600(21)00138-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Ergün M., Brüggemann R.J.M., Alanio A., Dellière S., van Arkel A., Bentvelsen R.G., et al. Aspergillus test profiles and mortality in critically ill COVID-19 patients. J Clin Microbiol. 2021;59(12) doi: 10.1128/JCM.01229-21. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Clancy C.J., Nguyen M.H. Coronavirus disease 2019-associated pulmonary aspergillosis: reframing the debate. Open Forum Infect Dis. 2022;9(5) doi: 10.1093/ofid/ofac081. ofac081. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Dellière S., Dudoignon E., Voicu S., Collet M., Fodil S., Plaud B., et al. Combination of mycological criteria: a better surrogate to identify COVID-19-associated pulmonary aspergillosis patients and evaluate prognosis? J Clin Microbiol. 2022;60(3) doi: 10.1128/jcm.02169-21. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplemental Fig. 1

Kaplan-Meier survival curves of medical centres with a pre-emptive screening strategy (screening) or a reactive diagnostic strategy (no screening) for CAPA.

mmc1.docx (37.2KB, docx)
Supplementary material

Supplemental Tables 1 - 7

mmc2.docx (61.7KB, docx)

Data Availability Statement

Research data is available upon reasonable request.


Articles from Journal of Critical Care are provided here courtesy of Elsevier

RESOURCES