Longitudinal passive cough monitoring and its implications for detecting changes in clinical status

Abstract

Research question

What is the impact of the duration of cough monitoring on its accuracy in detecting changes in the cough frequency?

Materials and methods

This is a statistical analysis of a prospective cohort study. Participants were recruited in the city of Pamplona (Northern Spain), and their cough frequency was passively monitored using smartphone-based acoustic artificial intelligence software. Differences in cough frequency were compared using a one-tailed Mann–Whitney U test and a randomisation routine to simulate 24-h monitoring.

Results

616 participants were monitored for an aggregated duration of over 9 person-years and registered 62 325 coughs. This empiric analysis found that an individual's cough patterns are stochastic, following a binomial distribution. When compared to continuous monitoring, limiting observation to 24 h can lead to inaccurate estimates of change in cough frequency, particularly in persons with low or small changes in rate.

Interpretation

Detecting changes in an individual's rate of coughing is complicated by significant stochastic variability within and between days. Assessing change based solely on intermittent sampling, including 24-h, can be misleading. This is particularly problematic in detecting small changes in individuals who have a low rate and/or high variance in cough pattern.

Short abstract

When compared to continuous monitoring, limiting observation to 24 h can lead to inaccurate estimates of change in cough frequency, influenced by its mean frequency and variance. This is important when evaluating small changes in cough frequency. https://bit.ly/3NtfPw1

Introduction

Cough is common in patients with respiratory diseases, and relatively rare in healthy individuals [1]. Detecting changes in its frequency is important in managing patients with cough and for evaluating novel drugs [2, 3]. An optimal objective cough monitoring period has not yet been defined [4–7], but in most clinical research settings, it remains limited to 24-h periods [8, 9].

While 24-h monitoring is the most commonly used approach in clinical research [9, 10], this strategy's ability to capture longer trends or discontinuous changes remains unclear. Estimates of shorter cough frequency can be confounded by circadian changes [11, 12].

Recent progress in the field of artificial intelligence (AI) has enabled the development of tools to unobtrusively monitor cough over protracted periods of time. This includes smartphone-based applications that employ acoustic AI to detect and track cough. We recently reported the use of one such tool to monitor cough among the inhabitants of Pamplona, Spain [13]. We now present a statistical analysis of the data collected in that cohort and explore its implications for patient care and drug development.

In this study we aim to determine the impact that the duration of cough monitoring has in the capacity to detect changes in its frequency, and hypothesise that, given the stochastic nature of cough, protracted longitudinal monitoring could provide more representative estimates of its frequency.

Methods

Study subjects

The data used for this analysis is from participants in a cohort study conducted in the city of Pamplona (northern Spain), as well as the University of Navarra's students and staff between fall 2020 and fall 2021. The aim of the study was to assess the potential correlation between the local incidence of respiratory diseases and the coughs in the cohort captured using a smartphone-based system. We targeted participants with and without respiratory diseases. Recruitment strategies included a broad information campaign carried out via social networks and through face-to-face meetings, with support from university and municipal authorities. A full protocol describing inclusion criteria has been published [14]. All participants signed informed consent, and this project was reviewed and approved by the Ethics Committee for Medical Research of the Chartered Community of Navarra (PI2020/107) and the Comité d’éthique à la recherche du Centre Hospitalier de l'Université de Montréal (2021-9247). The main study was registered in www.clinicaltrials.gov, under the number NCT04762693, and its results have been submitted separately for publication.

Study design

The overarching objective of this analysis was to assess the impact of the duration of cough monitoring on the capacity to detect changes in its frequency. Specific objectives include: 1) describing the distribution and circadian pattern of cough frequency in the studied cohort; 2) evaluating the role of effect size on the capacity to detect such changes; and 3) determining an optimal monitoring period and 4) comparing the capacity of estimates based on 24-h monitoring to longitudinal records to detect changes in cough frequency.

This is an empiric statistical analysis of a larger cohort acoustic surveillance study. Sample size calculations and the study's main results have been described elsewhere [13, 14].

Monitoring

Cough frequency was monitored using Hyfe Cough Tracker (Wilmington, DE, USA, https://www.hyfeapp.com/, henceforth referred to as Hyfe). Hyfe uses a device and cloud-based neural networks (CNN) to identify cough while preserving privacy. The CNN assigns a cough prediction score to each sound; if such a score lies above a predetermined value (in this study, 0.70 out of 1.0), the sound is labelled as a cough. The analytical performance of the algorithm was evaluated in a pilot sub-study that reported a sensitivity of 96.34% and a specificity of 96.54% among detected sounds [14], and is better characterised in a nested study of solicited sounds from 49 participants submitted jointly with this article. Additional clinical validation in specific diseases is underway (NCT05042063). All participants were instructed to install Hyfe in their personal mobile phones and use it for at least 6 h a day, during the night-time. Those willing to use it continuously or for longer periods were encouraged to do so.

Analysis

Participants with <1 h of recording data were excluded from all further analysis. To compare the capacity of 24-h monitoring with longitudinal records to detect changes in cough frequency, a randomisation routine was used to simulate the data that would have been obtained by monitoring two participants for a 24-h period. These two participants were selected for having used the monitoring system regularly throughout the study and because they received interventions that clearly modified their cough frequency. In each randomisation, a 24-h period of monitoring before and after such intervention was selected at random temporal points and the difference between both periods (after − before intervention) was calculated. This process was repeated 100 000 times to construct a distribution of after–before differences. To determine the accuracy of 24-h sampling, we calculated the proportion of randomisations that fall outside of the 95% confidence interval of the difference between the arithmetic means, and the percentage of randomisations in which the 24-h method detects a false direction for the change in cough. Similarly, the cough rates before and after the interventions were compared using a one-tailed Mann–Whitney U test, and the effect size of the reduction, measured in coughs·h⁻¹, determined using non-parametric bootstrapping (10 000 iterations). A significance level of 0.05 was used in these tests.

To characterise circadian patterns in cough, records of participants with at least 100 h of monitoring were analysed, considering only hours with at least 30 min of recording. Cohort-wide cough rates in each circadian hour (1–24 h) were summarised. The mean cough rate in each hour and for each user was calculated, keeping only users with at least 10 h of monitoring within that hour. The mean and standard error of cough rate across users for each hour was then obtained.

We then modelled an individual's cough pattern using a negative binomial distribution. Cough records from users with at least 100 h of monitoring and at least one cough were analysed, keeping only the hours with 30 continuous minutes of recording. The mean and variance of each user's cough distribution were calculated and used to determine the empirical correlation between these two parameters with a linear model. This regression was subsequently used to simulate a realistic cough rate distribution for any given mean cough rate.

To determine the influence of effect size in the capacity to detect changes, a simulation based upon the hourly cough rate distribution described above was used to model trends following certain interventions. In these simulations, the “before intervention” and “after intervention” periods were each simulated using 336 h (two weeks) of coughs and a “before intervention” cough rate of 4 coughs·h⁻¹. The same randomisation routine described above was used to calculate the fraction of differences that failed to detect the direction of the effect. This was repeated 100 times to produce distributions for the rates of failure, and implementing 19 different effect sizes, ranging from 1% to a 95% reduction in cough.

Finally, records of participants who used Hyfe regularly were used to determine how much monitoring is required to achieve accurate estimates of cough rates. Data from users with at least 240 h of recording (10 days; need not be continuous) and a mean cough rate of at least 0.5 cough·h⁻¹ or 12 coughs·day⁻¹ were used for this analysis.

For each user, a cough rate based on the full monitoring period (referred to as the “actual” cough rate) was compared to estimates based upon subsamples from 12 to 240 h. For this, the user's time series was binned using a rolling 1-h window (step=1 s), keeping all windows with at least 30 min of monitoring. These windows were sampled pseudo-randomly, ensuring that no two windows overlapped in time, and used to evaluate how the participant's “actual” cough rate differs from estimates based on shorter periods. This was repeated 100 times for each user to get an average error rate at each subsample duration. Error was defined as the absolute proportional difference between estimated cough rates based on subsamples, compared to the participant's “actual” rate.

All data analysis was carried out using R Studio version 1.3 (RStudio Team, 2020. RStudio: Integrated Development for R. RStudio, PBC, Boston, MA, USA).

Results

As reported elsewhere [13], we collected over 9 person-years of cough data and 62 325 coughs between November 2020 and August 2021 from 616 participants who recorded at least 1 h of data, and 22.4% of whom reported a history of acute or chronic cough. In total, 178 participants registered >100 h of monitoring, and 21 registered at least 240 h (figure 1). A flowchart describing participant subgroups used in individual analyses can be found in supplementary E-figure 1. There were no significant differences in the cough frequency of participants regardless of their self-reported history of respiratory disease, as can be observed in supplementary E-table 1 and E-figure 2. Similarly, the daily cough frequency of participants diagnosed with COVID-19 during the study showed a similar pattern to that of the rest of the study subjects (see supplementary material).

FIGURE 1.

Participants monitored as a function of monitored time. The number of participants as a function of the cumulative hours of monitoring they recorded. The red lines indicate participants who recorded for at least 100 h in which there was at least 30 min of recording (n=178), and those who recorded for 240 or more hours and had a mean cough rate of at least 0.5 coughs·h⁻¹ (n=21).

Longitudinal monitoring versus 24-h

Comparison between two participants with self-perceived changes and who monitored their coughs continuously during the study period revealed differences between estimates obtained longitudinally, and those derived from 24-h monitoring.

Cough follows a diurnal pattern

To understand why short-term monitoring was inaccurate, we explored the pattern of cough among users with over 100 h of recording (n=178). Their cough followed a circadian pattern, being higher during daytime and the evening, but reducing later in the night, most notably after midnight (figure 2).

FIGURE 2.

Circadian pattern of cough frequency in the cohort. The circadian pattern of cough rate for the 178 participants showing a nadir in cough in the early morning and higher rates of coughing in the morning and evening. For each hour, the relative cough rate is calculated as the ratio of an individual's cough rate and the cohort-wide average cough rate for all hours of the day.

Even accounting for diurnal changes, we observed considerable day to day and hour to hour variability in cough rates. Data from 135 participants with over 100 h of monitoring and at least one cough were used to model the distribution of cough rates. Coughing for an individual user is a stochastic process, leading to zero-inflated and over-dispersed cough rates, adequately modelled by the negative binomial distribution. The variance of cough estimates increases with higher mean cough rates. Fitting these distributions to cough patterns, however, requires knowing the mean and variance of cough rates. These statistics show a strong linear relationship after log transformation (p<0.0001, R²=0.94; supplementary E-figure 7). These parameters were used to create a functional cough rate model.

Small changes are missed often

Seeing an apparent inverse correlation between effect size and the chance of detecting a false change in cough frequency, simulations of cough time series before and after different effect sizes were analysed. While 24-h monitoring is expected to detect large effect sizes most of the time (a 65% reduction in cough frequency would be detected over 95% of the time), this is not the case when changes in cough rates are small. If the effect is a 10% reduction in cough, any reduction will be missed in 42% of trials (95% CI 21–60%). Even when the reduction is as high as 40%, a false increase might be detected 17% of the time (95% CI 6–33%, figure 3).

FIGURE 3.

Influence of effect size in the capacity to detect changes in cough frequency with 24-h monitoring. The likelihood of failing to detect a change in cough frequency decreases as the absolute magnitude of the change increases. One hundred simulations were run for each effect size using the same “before” rate (4 coughs·h⁻¹). Dots represent the failure rate from a single simulation run. The solid line represents the mean failure rate for all 100 simulations.

Longitudinal monitoring reduces variability

To explore the differences between cough rate estimates based on monitoring periods of different lengths and the overall estimate (“actual” rate), we analysed data from 21 participants with over 240 h of recording (monitoring range 270–5246 h; cough rate range 0.55–9.84 coughs·h⁻¹).

The error in estimates decreased as the monitoring period increased. With 24 h of monitoring, observed cough estimates were, on average, 47% different from its “actual” rate (range: 25–80%), reducing to an average of 14% error (range: 5–27%) after 240 h, equivalent to 10 days of monitoring (figure 4). These results remained consistent after excluding participants with known interventions described in previous sections.

FIGURE 4.

Error in cough rate estimates decrease with longer monitoring periods. Ability to measure cough is a function of the mean cough rate, its variance and the duration of monitoring. Here the monitoring records of 21 users (blue lines) were subsampled, each with different cough patterns, to see how much error can be expected (y-axis) with various monitoring durations (x-axis). Only users with 240 total hours of monitoring and a cough rate of at least 0.5 coughs·h⁻¹ were included. The red line is the mean error for all those participants.

An optimal monitoring period depends on the cough frequency of individual participants

Participants of this subgroup (n=21) with lower cough rates require longer periods of monitoring to achieve lower proportional errors. In our cohort, the proportional error of cough estimates did not drop below 10% for any participant until at least 4 days of monitoring. Users with fewer than 2.5 coughs·h⁻¹ required 6–22 days of monitoring, while users with the most coughs in our dataset (>7.5 coughs·h⁻¹) required 4–6 days, although the latter result is based on only two participants with a cough frequency of such magnitude.

Illustrative case 1: refractory chronic cough

A 56-year-old woman with chronic cough began using Hyfe on 29 January 2021 (see supplementary material for a full medical history and test results). To help reduce her cough, her primary care physician began administering gabapentin on 17 February 2021, reaching its maximum dose (900 mg twice a day) on 30 March. Comparing 13 days prior to starting gabapentin (mean 21.31 coughs·h⁻¹, 95% CI: 19.18–23.48) to the 14 days following a 5-day buffer centred on 17 February (mean 13.72 coughs·h⁻¹; 95% CI 12.19–15.19), longitudinal monitoring with Hyfe detected a decrease in cough of 7.58 coughs·h⁻¹ (95% CI 4.96–10.10), representing a 35% reduction (p=0.00002, figure 5a and table 1). In contrast, random resampling 24-h periods before and after the change demonstrates that the decrease would have fallen outside the 95% CI 57% of the time and the cough would appear to have increased 8% of the time (supplementary material).

FIGURE 5.

Changes in cough rates for two selected participants following specific interventions. a) A participant treated for a refractory chronic cough (Case 1) and b) a chronic smoker attempting to quit (Case 2). The dotted lines indicate the date of specific interventions. The shaded areas represent the periods used to calculate the pre- and post-intervention mean cough rates surrounding a buffer period.

TABLE 1.

Changes in cough frequency in two participants with chronic cough

Intervention	Cough frequency before intervention# (coughs·h−1)	Cough frequency after intervention# (coughs·h−1)	Proportional change %	p-value	Proportion of time the effect size is inaccurate % (24-h period)	Proportion of time the direction of the effect is reversed % (24-h period)
Case 1: participant with chronic cough
Gabapentin	21.31	13.72	−35	0.00002	57	8
Omeprazole	9.70	3.98	−59	0.000002	62	4
Case 2: chronic smoker
Smoking stopped	1.53	0.58	−62	0.0009	68	21
Smoking restarted	0.70	1.2	+71	0.003	43	14
Smoking stopped	1.61	0.94	−42	0.008	50	23

^#: the interventions were different for both participants – treatment with gabapentin and omeprazole in case 1, and smoking cessation/relapses in case 2.

On 28 May, omeprazole (40 mg) was added. Comparing the 2 weeks prior (mean 9.70 coughs·h⁻¹, 95% CI 8.21–11.16, n=13 days) to the 2 weeks after (mean 3.98 coughs·h⁻¹; 95% CI 2.92–5.03, n=14 days, with a 5-day buffer around 28 May), Hyfe detected a 59% reduction in cough (p=0.000002, table 1). An increase in cough frequency would have been noticed in 4% of 24-h trials, and the effect size would have fallen outside 95% CI 62% of the time (supplementary material).

Illustrative case 2: smoke cessation, relapse and cessation

On 6 November 2020, a 71-year-old woman began monitoring her cough (see supplementary material for a full medical history and test results). On 18 January 2021, she decided to quit smoking. Based on her monitoring record during the 21 days which had >12 h of recording between 8 December and 8 January, her mean cough rate prior to quitting was 1.53 coughs·h⁻¹ (95% CI 1.14–1.95). After quitting, her cough rate decreased between 28 January and 28 February to 0.58 coughs·h⁻¹ (95% CI 0.39–0.78). This 62% reduction of 0.95 coughs·h⁻¹ (95% CI 0.507–1.417) was highly significant (p=0.0009, figure 5b, table 1). Using only the 24-h monitoring simulations, 68% of the estimates fell outside of the confidence interval of the longitudinal mean and would have falsely indicated an increase in cough rates in 21% of the 24-h trials (see supplementary material).

By 20 March 2021, this participant had recommenced smoking. Comparing the 8 days prior to relapse (0.70 coughs·h⁻¹, 95% CI 0.55–0.86) to the 13 days after (1.2 coughs·h⁻¹, 95% CI 1.00–1.49), with a 10-day buffer on each side of 20 March, Hyfe detected a 71% increase in cough of 0.54 coughs·h⁻¹ (95% CI 0.270–0.836) that was statistically significant (p=0.003, figure 5b, table 1). The effect size would have fallen outside the 95% CI 43% of the time, while a false reduction in cough frequency would have been detected in 14% of 24-h trials (see supplementary material).

This patient quit smoking again during the first week of May 2021. Comparing the 13 days prior to quitting again (1.61 coughs·h⁻¹, 95% CI 1.09–2.12) with the 14 days after (0.94 coughs·h⁻¹, 95% CI 0.65–1.22) with a 10-day buffer on each side of 1 May, Hyfe measured a 42% decline of 0.68 coughs·h⁻¹ (95% CI 0.127–1.287), which was statistically significant (p=0.008, table 1). The effect size would have fallen outside 95% CI 50% of the time. A false increase in cough would have been detected in 23% of 24-h trials (see supplementary material).

All changes detected by the monitoring system matched subjective improvements reported by participants to research staff.

Discussion

This study monitored cough continuously and unobtrusively in a large cohort, allowing to compare, for the first time, the accuracy of cough estimates derived from observation periods of different length. While the lack of significant differences in the cough frequency of participants with history of different baseline respiratory conditions is surprising, it can be explained by the fact that this information was self-reported by participants, and by the fact that no specific time limits for the presence of each condition were pre-specified by the research team. This translated into many participants reporting having a certain respiratory condition despite having remained asymptomatic for years.

Despite this limitation, two participants, a chronic cougher and the smoker, had self-perceived changes in the severity of their cough during the study, and these were associated with statistically significant changes in continuously recorded measurements. We demonstrated that the magnitude of these changes was often missed, and the trends even reversed when cough is only measured for 24-h periods, as is typically done in most research settings [9]. A major limitation of this study is that the combination of long-term, continuous app usage and precise clinical information was only available for these two subjects, making these observations not generalisable. However, simulations constructed with parameters derived from the observation of over 100 participants showed similar results. This suggests that the duration of monitoring can affect the accuracy of cough frequency estimates, as can be explained by the intrinsic variability of an individual's cough within and, particularly, between days.

This variability implies that three key parameters influence our ability to assess an individual's cough precisely: the average and variance of the individual's coughs per hour and the length of the observation period. Having observed empirically that higher cough rates correspond to larger variances in coughs per hour, estimating individual cough parameters accurately demands longer observation periods.

Intermittent monitoring might be particularly problematic in a context where detecting small effect sizes is important, as for instance, in clinical trials comparing different antitussive therapies [9, 15, 16]. Furthermore, this statistical analysis suggests that recognising changes in individual cough patterns requires a deeper understanding of their dependence on both the average and variance of the individual's coughs per hour. It is interesting to speculate that recent trials demonstrating that the novel antitussives are most effective on patients with the highest cough rate may be largely a consequence of this statistical issue and not intrinsic to the drug's efficacy.

Another limitation of this study is that it was conducted with a cough monitoring tool that has not been fully validated. Although Hyfe's performance was analysed in a pilot stage of the study [14], its sensitivity and specificity for detecting cough in different practical settings are only now being formally quantified (NCT05042063). It is, however, reassuring that it did detect changes in those patients who reported changes, and found a diurnal pattern consistent with prior publications [17].

This is the first population-based description of cough epidemiology. However, while 22% of the cohort reported a history of cough, most were young and presented either mild, acute respiratory symptoms or no symptoms at all at the moment of enrolment. Thus, the cohort may not be representative of those patients who would benefit the most from objective cough monitoring, nor the target population of clinical trials that would benefit from more accurate assessment of changes in cough. Thus, this study should be replicated in a better-defined cohort using a validated device.

Given our preliminary observations, we propose that significant changes in coughs need to be defined individually, and based on the basal cough rates of patients, and that longitudinal monitoring is likely to provide more accurate estimates of a patient's basal state. The ability to devise such individualised models in real time may help patients and providers to promptly detect clinical changes. Early detection of clinical deterioration has been shown to improve outcomes in respiratory diseases such as refractory chronic cough and COPD [18–20].

Supplementary material

Please note: supplementary material is not edited by the Editorial Office, and is uploaded as it has been supplied by the author.

Supplementary material 00001-2022.SUPPLEMENT ^{(891KB, pdf)}