Beyond random variation: Monte Carlo analysis and predictability of COPD exacerbations

Extract

Exacerbations of COPD are associated with high symptom burden and long-term consequences for disease progression [1, 2]. These events are currently classified as severe when they result in emergency room visits or hospitalisations, and as moderate when they are treated out of hospital. Several therapies exist, and many novel treatments are being investigated for the prevention of these events. In clinical practice, decisions regarding treatment change are made based on the risk of exacerbations over the next 6–12 months [3].

Short abstract

Both moderate and severe exacerbations of COPD follow patient-specific, predictable patterns rather than occurring as random events of chance https://bit.ly/4d6TkMS

To the Editor:

Exacerbations of COPD are associated with high symptom burden and long-term consequences for disease progression [1, 2]. These events are currently classified as severe when they result in emergency room visits or hospitalisations, and as moderate when they are treated out of hospital. Several therapies exist, and many novel treatments are being investigated for the prevention of these events. In clinical practice, decisions regarding treatment change are made based on the risk of exacerbations over the next 6–12 months [3]. The predictability of exacerbations is, therefore, important for making treatment decisions regarding initiating or escalating therapy. The strongest clinical risk factor for future exacerbations is a history of previous exacerbations [4, 5]. Exacerbation frequency in individual patients, however, fluctuates year-to-year [6, 7], and prior exacerbation history is associated with a modest performance for predicting future events with C-indices ranging from 0.59 to 0.65 [8]. In addition to external exposures, which may be variable, underlying disease severity may be associated with exacerbation frequency, especially for more severe events. Continued efforts are being made to more accurately predict exacerbations. There remains, however, a lack of data on whether moderate and severe events are both predictable. If one or both exacerbation types are due to random chance, then efforts to predict them will be unsuccessful. We aimed to evaluate whether exacerbations, moderate and severe, are truly predictable.

We analysed data from participants enrolled in the Genetic Epidemiology of Chronic Obstructive Pulmonary Disease (COPDGene) study, which included current and former smokers between the ages of 45 and 80 years, enrolled at 21 clinical sites in the United States [9]. Baseline enrolment occurred between November 2007 and April 2011, and participants were contacted prospectively approximately every 6 months via an automated telephony system or by telephone call by coordinators to obtain information on exacerbations. We replicated the analyses in the NOVEL observational longiTudinal studY (NOVELTY), which enrolled participants aged 12 years or older with a physician diagnosis or suspected diagnosis of asthma and/or COPD, at 331 clinical practices across 19 countries [10]. Baseline enrolment occurred between July 2016 and May 2019, and subjects were contacted approximately every 3 months using a web-based platform or telephone calls to obtain information on exacerbations. We defined exacerbations as new or worsening cough, sputum, or dyspnoea that lasted at least two days and required therapy with systemic steroids and/or antibiotics. Exacerbations that were treated outpatient were considered moderate, and events resulting in treatment in an emergency room or in hospitalisation were considered severe [3]. In this analysis, we included only those with airflow obstruction, defined by post-bronchodilator ratio of the forced expiratory volume in 1 s to the forced vital capacity <0.70.

We conducted a Monte Carlo simulation analysis to test whether year-to-year correlations in COPD exacerbations exceeded those expected by chance. We analysed moderate and severe exacerbation counts over three consecutive years in adults with COPD with complete three year data. For each year and exacerbation type, we fitted negative binomial distributions to model the frequency of events. We then generated 10 000 simulated datasets where each year's exacerbation counts were drawn independently from these fitted negative binomial distributions, thereby eliminating any temporal correlation structure while maintaining realistic distributional characteristics. For each simulated dataset, we calculated year-to-year correlation coefficients (Years 1–2, 2–3, and 1–3) and compared the distribution of these null correlations to the observed correlations in the real data. Statistical significance was assessed using two-tailed p-values, calculated as twice the minimum of the percentile ranks of observed correlations within the simulated distributions. All analyses were performed using the R-statistical package (version 4.2.2).

Of 10 198 participants in COPDGene, we excluded 812 in whom we did not have long-term follow-up data, 5023 with no COPD, and 1328 in whom any data on moderate or severe exacerbations was missing in any of the first 3 years of follow-up, resulting in a sample size of 3035. The mean±sd age at enrolment was 59.5±9.0 years, 4696 (46.6%) were female, and 3370 (33.4%) were Black. The mean pack-years of smoking was 44.1±25.0, 4733 (47.0%) were active smokers. There were modest correlations across years for the frequencies of both exacerbation types: 0.34 (Years 1–2), 0.37 (Years 2–3), and 0.18 (Years 1–3) for severe exacerbations and 0.30, 0.33, and 0.28, respectively for moderate events (correlogram in figure 1a). The correlations for moderate exacerbations were higher than for severe exacerbations (1–3 year correlation approximately 56% higher than for severe exacerbations). All observed correlations were statistically significant (p<0.001), with none of the 10 000 Monte Carlo simulations producing correlations as high as those observed in the real data (figure 1b, c). The p-values for all simulations falling outside the observed correlations were <0.001, thus rejecting the null hypothesis that exacerbations occur independently of each other.

FIGURE 1.

Correlogram of exacerbation frequency by severity over three years in a) COPDGene, and b) NOVELTY, with Monte Carlo simulation analysis of year-to-year correlations for moderate and severe exacerbations in c, d) COPDGene and e, f) NOVELTY. Distribution of correlation coefficients from 10 000 Monte Carlo simulations testing the null hypothesis that exacerbation events occur randomly and independently across years. Each panel in c–f represents correlations between different year pairs: Years 1–2, Years 2–3, and Years 1–3. The histograms show the distribution of correlation coefficients obtained from random simulations where exacerbation counts were drawn from negative binomial distributions fitted to match the observed mean and variance of each year's data. The red dotted vertical line indicates the observed correlation coefficient from the actual patient data. The dark solid vertical line at zero represents the expected correlation under complete independence (simulation mean≈0).

We replicated these results in 2247 participants in NOVELTY, with mean age of 66.7±8.6 years, 860 (38.3%) female, and 1518 (67.6%) non-Hispanic white. The mean pack-years of smoking was 41.9±39.8, and 545 (24.2%) were active smokers. Year-to-year correlations for both moderate and severe exacerbations in NOVELTY were consistent with the patterns observed in COPDGene (figure 1b–d). Monte Carlo simulations confirmed that all observed correlations were statistically significant (p<0.001), exceeding values expected under the assumption of temporal independence (figure 1e, f).

In two cohorts of adults with COPD with prospectively-collected exacerbation data, we demonstrated that both moderate and severe exacerbations follow patient-specific, predictable patterns rather than occurring as random events of chance. Our results suggest that the occurrence of moderate events may be more predictable than severe events.

These results have important clinical implications. In clinical practice, severe exacerbations are given more importance, and treatment optimisation is recommended after even a single severe event. This practice may be partially resulting from the Global Initiative for Chronic Obstructive Lung Disease recommendations, which give equal importance to one severe and two moderate exacerbations [3]. In contrast, moderate events that can be managed with outpatient care have often been ascribed to milder exposures, and therefore a “wait and watch” approach to change in treatment is often followed. Moderate events may be harder to detect than severe events, and may often go unreported or underreported. In the prospective observational STEPINCOPD study, the proportion of patients experiencing one, two, or more than two exacerbations during the observation period prior to treatment escalation was 49.4%, 36.7%, and 11.4%, respectively [11]. With growing evidence that even a single exacerbation may increase the risk of lung function decline and future exacerbations [12, 13], our results add to data supporting changing from a watchful approach to a more active approach with the occurrence of each exacerbation.

The strengths of the study include the prospective collection of exacerbation information and the inclusion of a high proportion of Black people, who are generally underrepresented in such datasets. The study has a few limitations. We did not account for any treatment changes that may have occurred between surveys. Exacerbations were self-reported in COPDGene and were a mix of self-reported and physician-reported in NOVELTY. This limitation would, however, be expected to bias results toward reduced correlations and greater apparent randomness rather than create spurious predictability. The observation of consistent, statistically significant year-to-year correlations across two independent cohorts supports the robustness of our findings.

In conclusion, exacerbations are predictable and not chance events. Our results provide a rationale for risk-based follow-up intensity, enrichment strategies for clinical trials, and the development of future predictive models by establishing that exacerbation predictability is biologically and clinically plausible. These results support moving away from a watchful approach for moderate exacerbations.