Use of a Digital Health Application for Influenza Surveillance in China

Abstract

Objectives. To examine whether a commercial digital health application could support influenza surveillance in China.

Methods. We retrieved data from the Thermia online and mobile educational tool, which allows parents to monitor their children’s fever and infectious febrile illnesses including influenza. We modeled monthly aggregated influenza-like illness case counts from Thermia users over time and compared them against influenza monthly case counts obtained from the National Health and Family Planning Commission of the People’s Republic of China by using time series regression analysis. We retrieved 44 999 observations from January 2014 through July 2016 from Thermia China.

Results. Thermia appeared to predict influenza outbreaks 1 month earlier than the National Health and Family Planning Commission influenza surveillance system (P = .046). Being younger, not having up-to-date immunizations, and having an underlying health condition were associated with participant-reported influenza-like illness.

Conclusions. Digital health applications could supplement traditional influenza surveillance systems in China by providing access to consumers’ symptom reporting. Growing popularity and use of commercial digital health applications in China potentially affords opportunities to support disease detection and monitoring and rapid treatment mobilization.

Influenza is a significant public health concern in China.¹ More than 570 million individuals are at risk for seasonal influenza in China each year,² with the most vulnerable groups being young children and those with underlying medical conditions.³ Recently, the Chinese government invested substantially in the development of a national influenza surveillance network⁴ to enable early detection and prevention of epidemics.⁵ The mainstay method of influenza surveillance in China has been reporting and confirmation of cases seen in health facilities. However, success of passive case reporting depends on patients having access to health care facilities, clinicians trained to recognize clinical manifestations of the disease, and available laboratory capacity, which is often difficult in a middle-income country like China.^4,5 Finally, poor-quality reporting procedures have potentially led to underreporting of influenza cases in China, which has resulted in delayed detection and response.^4,5 Therefore, it is essential for China to look beyond traditional data reporting and surveillance methods, and to seek novel data sources to better monitor influenza patterns.

Digital disease surveillance is poised to be a powerful complement to traditional epidemiology and disease monitoring approaches.^6–8 Digital surveillance methods are not limited by requirements for costly clinical infrastructure, geographic barriers, or the need for trained personnel to manually complete data collection and reporting.⁷ User posts and communication on popular social media platforms such as Twitter and influenza-related search patterns on the Chinese Web search engine Baidu have been used to demonstrate the potential for employing digital surveillance methods to predict disease patterns in China.^7,9 In the United States, participatory crowdsourcing applications such as Flu Near You have used volunteer citizen reports to collect symptom data¹⁰ and estimate influenza attack rates.¹¹ Although publicly available online data sources appear promising, researchers have yet to leverage consumer technologies such as digital health applications, which collect vast amounts of health related data, to understand population health patterns.

Two thirds of people in China use the Internet, and nearly 60% own smartphones.¹² Globally, it is estimated that more than half of all smartphone users gather some form of health-related information on their phones or have downloaded health applications.¹³ Therefore, digital health applications accessed through Web or mobile platforms could provide a rich data source to augment digital surveillance techniques. This is pertinent for China, where digital health applications potentially offer a wealth of health information because more than 90% of Internet users accessed the Web through a mobile device.¹⁴

In this study we introduced a novel approach for estimating influenza patterns in China by sourcing data from users of the Thermia educational tool, which allows parents to monitor their children’s fever and infectious febrile illnesses such as influenza through an online or mobile application. Our study extends beyond use of publicly available online resources by testing whether Thermia, a commercially available digital health application, could predict influenza outbreaks in China. Furthermore, we examined characteristics of digital health application users from China and explored how this data source could support collection of information about risks of fever and fever-related illnesses including influenza.

METHODS

Thermia is an educational tool accessed through an online or mobile application that was created by the Computational Epidemiology Group at Boston Children’s Hospital. Thermia’s objectives are (1) to educate parents about fever symptoms based on current clinical guidelines, (2) to provide at-home treatment options to enable more appropriate and supportive care, and (3) to use a bank of anonymized data to track disease and survey population health. Thermia (http://thermia.io) was launched in 2014 and is accessible worldwide. In 2015, Thermia’s fever education platform was integrated into Raiing Medical Inc’s smartphone-connected iThermonitor, a wearable thermometer with US Food and Drug Administration 510(k) clearance.

Thermia engages parents to answer questions regarding symptoms associated with febrile illnesses. Parents begin the application by entering their child’s age and temperature (taken directly from the connected thermometer or entered manually) into the Thermia application followed by answering questions related to fever symptoms. Questions include answering “yes” or “no” to whether their child (1) has an underlying chronic condition, (2) is up to date on all routine immunizations, and (3) is experiencing any of 8 possible symptoms listed (rash, vomiting, diarrhea, sore throat, cough, headache, shortness of breath, or fatigue). To retrieve educational information about fever and illness, parents were required to answer all of the preceding questions. Thermia data are automatically uploaded and presented on the aggregate level.

Data Sources

We collected data from Thermia between January 1, 2014, and July 31, 2016. We removed suspicious data with a frequency of 150 entries over 7 days, and incomplete entries without age, temperature, or symptom reports before analysis. We restricted data to include an obtainable human temperature range between 93 °F and 108 °F. We measured temperature as a continuous variable and as a categorical variable defined according to the Centers for Disease Control and Prevention (CDC; normal: temperature < 100 °F [37.8 °C]; fever: temperature ≥ 100 °F [37.8 °C]).¹⁵ We measured age continuously in years and categorically based it on CDC guidelines (infant: 0–3 years; child: 4–11 years; adolescent: 12–17 years; and adult: ≥ 18 years).¹⁶ We defined influenza as having a temperature of 100 °F or higher and having symptoms of a sore throat or cough.

We measured symptoms dichotomously as present “1” and not present “0,” and formulated a continuous scale by the addition of all subsequent symptoms given a value range between 0 and 8. We also measured underlying condition and up-to-date immunizations dichotomously, although details for both were not asked to ensure greater ease of use of the Thermia application.

Parents could choose to enter their name, e-mail address, and location, but the data for analysis was not restricted on whether parents entered personal details. We restricted Thermia data to include only observations from users in China, and anonymously aggregated the data for analysis as to not identify individuals.

Influenza Case Counts in China

We collected monthly aggregated influenza case counts from the National Health and Family Planning Commission (NHFPC) of the People’s Republic of China from January 2014 to August 2016. These data are publicly available on the NHFPC Web site (http://www.nhfpc.gov.cn) and are released 1 to 2 weeks after the end of each month. A network of physicians report laboratory-confirmed cases to the NHFPC on a daily basis but these data are released to the public monthly. The data in this study are solely for seasonal influenza and do not include swine flu (H1N1) or avian flu (H7N9). We based the denominator to calculate proportions for the reported data on China population estimates from census data obtained from the National Bureau of Statistics of the People’s Republic of China.

To model patterns of influenza over time, we compared Thermia influenza count data from China with influenza case count data from the NHFPC. We defined an influenza-like illness (ILI) case count as having a temperature of 100° F (37.8 °C) or greater and having symptoms of a sore throat or cough. Thermia case count data are available on a daily basis but we converted them to monthly counts for analysis to match the influenza case count data from NHFPC. We aggregated the data to generate a combined frequency table and to maintain user confidentiality.

Statistical Analysis

We summarized Thermia users’ demographic and health characteristics. We used multiple logistic regression models to estimate the odds of having an ILI, a dichotomous “yes” or “no” variable, among Thermia users. Independent predictor variables in logistic regression models included age, up-to-date immunizations, and underlying condition as previous literature has indicated that these variables are associated with influenza.^17–19

To determine if Thermia could be used as a digital surveillance tool, we compared influenza case counts from NHFPC to Thermia’s ILI case count data from China over the same time period. We calculated the proportion of official influenza case counts by dividing the number of influenza case counts over the total Chinese population for each month. Thermia’s target population consists of parents who are concerned about their child’s health and symptomatology; therefore, Thermia participants are not representative of the entire Chinese population, which comprises both sick and healthy individuals and individuals who are not parents. Unadjusted Thermia ILI case count data comprise both long-term temporal trends of growing user entries and seasonality as well as short-term localized outbreaks in influenza. A locally weighted scatterplot smoothing plot smooths over local fluctuations and can serve as a proxy for the long-term trends in Thermia ILI cases. To calculate local fluctuation in time series of Thermia ILI cases, we subtracted the locally weighted scatterplot smoothing plot from and the frequency of Thermia cases for each month and plotted this difference over time. We argue that by isolating the short-term fluctuations of Thermia ILI case counts we are able to control for differences in the dynamics of population composition over time and can make comparisons between the time-series of Thermia and NHFPC.

To model the temporal relationship between NHFPC case count and Thermia case count, we denote y_therm(t) the Thermia ILI case count at time t, and y_NHFCP(t) the NHFPC influenza case count, and consider a distributed lag model similar to Yang et al.²⁰:

where l is measured monthly and is the lag in time between the influenza case counts. In this model, b_l quantifies the contribution from Thermia case count at time t–l to predicting NHFPC case count at time t, in addition to the contribution from historical NHFPC case counts.

To formally test for the significant lag time between time trends of y_NHFPC and y_therm, we make the observation that within the range of lags considered (3 months), b_l is nonincreasing with respect to lag length, as the case count measurements becomes less correlated as the distance in time between them increases.

This observation leads to a sequential hypothesis testing procedure that tests for relationship between y_NHFPC(t) and y_therm(t–l) in an incremental manner. Specifically, starting from lag = 1, we test for the null and alternative hypothesis:

This hypothesis can be tested by using a likelihood ratio test with small sample adjustments²¹:

In which L_lag indicates the maximized log likelihood under the model:

If we fail to reject the null hypothesis H_{0,lag = 1}, the procedure then proceeds to test for the hypothesis H_{0,lag = 2}, etc. The procedure stops when the lag-specific hypothesis H_0,lag is rejected at certain lag (say, lag = k), and we conclude that Thermia measurements during (t–l,t) are significantly associated with NHFPC measurements at time t.

We also examined whether ILI time trend differed between child age groups. We used the age cutoff of 5 years or younger because children at that age are at highest risk for influenza¹⁷ and research shows that this age group has the longest lead time, meaning they provide earlier signals of influenza outbreaks compared with other age groups.²² We denoted y(t) the ILI case count for children older than 5 years and those aged 5 years or younger at time t, and denoted age_group the indicator for whether the observations belong to a child older than 5 years or a child aged 5 years or younger. Assuming no lag exists, to test for the similarity of ILI time series trend between a child aged 5 years or younger and a child older than 5 years, we fit the autoregressive model:

The nonlinear interaction term f(t,group) captures the difference in time series between the time series of ILI case count for children older than 5 years and children aged 5 years or younger. If b₂, the coefficient for interaction term, is significant, then there is a difference in time trend. We carried out all hypothesis tests at significance level .05, and completed all analyses with the R version 3.3.0 statistical software packages mgcv²³ version 1.8–14 and dlnm version 2.2.6 (R Foundation, Vienna, Austria).²⁴

RESULTS

We retrieved 44 999 observations from Thermia China from January 1, 2014, to July 31, 2016. We retrieved almost all observations directly through the Thermia application program interface (99.9%) using the iThermonitor, which does not require manual entry like the Web version. The application program interface method is potentially less subject to users’ data entry errors and provides a more accurate measure of ILI, which includes having a temperature of 100 °F (37.8 °C) or higher. We sourced the remaining data (0.1%) from the Web application.

Participant Characteristics and Odds of Illness

Table 1 describes Thermia participant characteristics. Participants’ mean age was 5.6 years (SD = 6.01), and more than two thirds were infants aged 0 to 3 years (68.1%). Participants recorded a mean temperature of 100.7 °F (SD = 1.79) and almost three quarters had temperatures that equated to a fever (74.2%). Nearly half of participants reported having no symptoms (48.7%) and only 1.2% reported having all 8 symptoms. Most participants (88.8%) reported not having an underlying condition and more than three quarters reported having up-to-date immunizations (77.4%). About one quarter (25.2%) of Thermia participants had ILI.

TABLE 1—

Demographic and Health Characteristics of Thermia Participants: China, January 2014–July 2016

Characteristics	No. (%)
Data source
API	44 941 (99.9)
Web	58 (0.01)
Age group
Infant (0–3 y)	30 490 (68.1)
Child (4–11 y)	6 593 (14.7)
Adolescent (12–17 y)	7 106 (15.9)
Adult (≥ 18 y)	607 (1.4)
Number of symptomsa
0	21 918 (48.7)
≥ 1	23 081 (51.3)
1	10 790 (23.9)
2	5 643 (12.5)
3	3 607 (8.0)
4	1 594 (3.5)
5	678 (1.5)
6	165 (0.4)
7	93 (0.2)
8	544 (1.2)
Underlying condition status
No underlying condition	39 946 (88.8)
Underlying condition	5 033 (11.2)
Immunization status
Up-to-date immunizations	34 804 (77.4)
Not up-to-date immunizations	10 179 (22.6)
Temperature category
Normal	11 556 (25.7)
Fever (temperature ≥ 100 °F or 37.8 °C)	33 442 (74.2)
ILI status
No ILI	33 677 (74.8)
ILI	11 322 (25.2)

Note. API = application program interface; ILI = influenza-like illness, defined as having a fever (temperature ≥ 100 °F or 37.8 °C) and having symptoms of a sore throat or cough.

^aParticipants reported any of the following 8 symptoms: rash, vomiting, diarrhea, sore throat, cough, headache, shortness of breath, or fatigue.

Table 2 illustrates Thermia participants’ odds of having an ILI by immunization status, underlying condition, and age. Participants who reported not having up-to-date immunizations had greater odds of having ILI compared with participants who reported having up-to-date immunizations, when we controlled for age and underlying condition (odds ratio [OR] = 1.11; 95% confidence interval [CI] = 1.05, 1.17). Participants who reported having an underlying condition had increased odds of having ILI compared with participants who reported not having an underlying condition, when we controlled for immunization status and age (OR = 1.26; 95% CI = 1.18, 1.34). When compared with infants, children aged 4 to 11 years had increased odds of having ILI (OR = 1.86; 95% CI = 1.76, 1.97) whereas adults had lower odds of having ILI (OR = 0.76; 95% CI = 0.62, 0.93), after we controlled for immunization status and underlying condition.

TABLE 2—

Predictors of Influenza-Like Illness Among Thermia Users: China, January 2014–July 2016

	ILI
Variable	No./Total No.	%	OR (95% CI)
Underlying condition status
No underlying condition	9 870/39 946	24.7	1 (Ref)
Underlying condition	1 444/5 033	28.7	1.26 (1.18, 1.34)
Immunization status
Immunized	8 626/34 804	25.8	1 (Ref)
Not immunized	2 868/10 179	24.8	1.11 (1.05, 1.17)
Age category
Infant (0–3 y)	7 146/30 490	23.4	1 (Ref)
Child (4–11 y)	2 376/6 593	36.0	1.86 (1.76, 1.97)
Adolescent (12–17 y)	1 632/7 106	23.0	0.96 (0.90, 1.02)
Adult (≥ 18 y)	119/607	19.6	0.76 (0.62, 0.93)

Note. CI = confidence interval; ILI = influenza-like illness; OR = odds ratio. Logistic regression models were used to determine the independent association between ILI and age, immunization status, and underlying condition status.

Surveillance

The largest increase in Thermia ILI cases occurred between January and March 2016, with the highest number of ILI cases occurring in February 2016. Data from the NHFPC showed that the greatest increase in influenza cases occurred between February and April 2016, with the highest number of influenza cases occurring in March 2016 (Figure 1). The autoregressive distributed lag model for testing the regression lag found a significant lag time of 1 month (LR_lag = 1 = 27.56; P < .001), in which Thermia appeared to predict influenza outbreaks 1 month earlier than the NHFPC.

FIGURE 1—

Monthly Estimates of Influenza Cases From Thermia and the National Health and Family Planning Commission of the People’s Republic of China: January 2014–July 2016

Note. ILI = influenza-like illness symptoms; NHFPC = National Health and Family Planning Commission of the People’s Republic of China. Thermia ILI case counts were calculated by subtracting locally weighted scatterplot smoothing plot trend from the raw Thermia ILI count. Case counts are per 10 000.

Table A (available as a supplement to the online version of this article at http://www.ajph.org) summarizes the forecasting performance for the autoregressive distributed lag model when different numbers of Thermia lags are used. Compared with the model with only lagged NHFPC terms, the model with Thermia lag 1 in addition to lagged NHFPC offered significant improvement in both out-of-sample prediction (root-mean-square error and correlation) and in Bayesian Information Criterion (P < .001). By contrast, adding more Thermia lags to the model does not seem to improve model performance any further (P value for lag 1–2 vs lag 1 = .279; P value for lag 0–3 vs lag 0–2 = .774). Therefore, we conclude that, on average, NHFPC influenza count at a specific month can be significantly better predicted by adding Thermia data at the previous month into the model.

Figure 2 displays the ILI case time series plot for Thermia participants older than 5 years and those aged 5 years or younger. There was no significant interaction between age group and the time trend (F = 0.339; P = .714) and, therefore, we saw no lag time between age groups.

FIGURE 2—

Monthly Estimates of Influenza Cases Among Thermia Participants Aged 5 Years or Younger Compared With Those Older Than 5 Years: China, January 2014–July 2016

Note. ILI = influenza-like illness symptoms. Thermia ILI case counts were calculated by subtracting locally weighted scatterplot smoothing plot trend from the raw Thermia ILI counts.

DISCUSSION

Thermia appeared to predict onset of the 2016 influenza outbreak in China 1 month earlier than the NHFPC, demonstrating the potential for a commercial digital health application to supplement an existing influenza surveillance network. This has important implications given the large health burden and difficulty identifying influenza outbreaks in a geographically vast and densely populated country such as China.

Data from NHFPC relies on health care personnel to assess and report occurrences of influenza to a centralized agency, which is not always accurately reported because of inconsistent criteria and classification.^4,5 Thermia data are primarily uploaded directly from the iThermonitor, potentially making it more consistent and less subject to manual errors. Furthermore, parents are prompted, in the moment, with straightforward questions regarding their child’s symptoms making recall bias less likely. This may have helped capture a more “real-time” snapshot of influenza patterns not typically reported in clinical settings.

Previous crowdsourced applications such as Flu Near You have documented greater prediction accuracy with increasing numbers of self-reported ILI cases.^10,11 This suggests that as the number of users increases and as individuals increasingly use their mobile devices for reporting information about their health, digital health applications such as Thermia may become more accurate as a tool for disease surveillance.

We found no statistically significant overall differences between the 2 time series plots, although small differences may have occurred as Thermia monitors self-reported ILI symptoms and may have captured additional influenza strains (e.g., swine flu or avian flu), whereas NHFPC confirms specific cases by using laboratory testing. In addition, Thermia users likely have greater access to the Internet compared with the general Chinese population, and therefore may reside in more densely populated urban areas where influenza outbreaks could spread at a faster rate and occur earlier. Moreover, Thermia users may be more affluent, and with additional resources can be more actively engaged in their child’s health allowing them to measure their child’s temperature and report influenza symptoms at an earlier onset. Therefore, more proactive symptom reporting from our sample of Thermia users attributable to unique demographic characteristics may have contributed to earlier detection compared with NHFPC.

We did not observe an earlier lag time among children aged 5 years or younger compared with those older than 5 years. The lack of significance may be because more than two thirds of Thermia participants were infants (0–3 years), making the sample size of older children too small to visualize differences in lag time. We found that parents with infants represented Thermia’s primary user demographic, and that they mainly used Thermia to monitor their child’s temperature, though nearly half reported no symptoms of febrile illness. Hence, parents may be using Thermia to monitor their infants’ fever and may be a more health-conscious cohort of parents who preemptively use the Thermia tool even if their child may not be sick. Regardless of their child’s temperature status, all Thermia users were pushed to answer the same questions related to their child’s symptoms. This consistency in measurement enabled us to retrieve an unbiased data set from Thermia users.

Most Thermia participants reported having up-to-date immunizations, suggesting that they could access a medical provider and are aware of the importance of routine immunizations for their child. Participants with ILI also had lower odds of having up-to-date immunizations compared with those without ILI. This may indicate that parents who are more diligent about obtaining routine immunizations for their child are also more conscientious about employing methods to protect their child from influenza. However, parents were not asked further details about timing or types of vaccinations that their child received, making it difficult to draw conclusions about what specific immunizations may help reduce risk of fever or ILI, although parents may have thought to “include” influenza vaccine as this is considered part of routine immunizations.

Thermia participants with an underlying condition had greater odds of having ILI, which is consistent with research showing that children with underlying medical conditions experience elevated risk of fever and influenza.²⁵ However, we are unable to examine how different medical conditions affected risk because users were not asked to provide details about the type of condition that was present. In addition, our finding that odds of having ILI decreased from the oldest age category (adults) to the youngest (infants) is also consistent with previous research.²⁶

Limitations

Several limitations should be considered. First, users’ demographic information such as ethnicity or socioeconomic status was not collected; therefore, analyses related to these factors were not possible. Second, sparse data were available from other countries, which did not permit analyses outside of China and thus our results may not generalize to other populations. Third, there was no limit to how many times a parent could use the Thermia application. Thus, Thermia ILI cases could comprise repeated entries, which were not possible to adjust for in our analyses because data were anonymously aggregated.

Parents received educational information about their child’s status from Thermia upon use, and could seek alternative advice from Thermia if a change in symptomology occurred, which could contribute to multiple cases. As well, because Thermia data are automatically uploaded, these “real-time” fluctuations in disease symptoms can be monitored. In addition, parents could use Thermia for separate occasions of illness or for different children, which would represent different cases. Lastly, given that Thermia data were anonymous, longitudinal analyses of unique individuals over time were not possible.

Public Health Implications

With 620 million mobile Internet users in China,¹⁴ digital health applications such as Thermia have potential to strengthen traditional disease surveillance systems. For Thermia to have true public health utility, Thermia data must be made accessible and widely available for public use in a timely manner. Innovative approaches are needed to monitor vulnerable subgroups in China who are potentially excluded from digital health data sources like Thermia, such as those in rural or impoverished settings without mobile Internet access. As Thermia adoption rises across other countries and in more discrete areas of China, it may be possible to predict disease outbreaks in diverse populations and areas with greater spatial resolution.