Google Flu Trends: Correlation With Emergency Department Influenza Rates and Crowding Metrics

City-level Google Flu Trends, a novel Internet-based influenza surveillance tool, shows strong correlation with influenza cases, emergency department influenzalike illness visits, and several emergency department crowding measures, validating its use as an emergency department surveillance tool.

Abstract

Background. Google Flu Trends (GFT) is a novel Internet-based influenza surveillance system that uses search engine query data to estimate influenza activity and is available in near real time. This study assesses the temporal correlation of city GFT data to cases of influenza and standard crowding indices from an inner-city emergency department (ED).

Methods. This study was performed during a 21-month period (from January 2009 through October 2010) at an urban academic hospital with physically and administratively separate adult and pediatric EDs. We collected weekly data from GFT for Baltimore, Maryland; ED Centers for Disease Control and Prevention–reported standardized influenzalike illness (ILI) data; laboratory-confirmed influenza data; and ED crowding indices (patient volume, number of patients who left without being seen, waiting room time, and length of stay for admitted and discharged patients). Pediatric and adult data were analyzed separately using cross-correlation with GFT.

Results. GFT correlated with both number of positive influenza test results (adult ED, r = 0.876; pediatric ED, r = 0.718) and number of ED patients presenting with ILI (adult ED, r = 0.885; pediatric ED, r = 0.652). Pediatric but not adult crowding measures, such as total ED volume (r = 0.649) and leaving without being seen (r = 0.641), also had good correlation with GFT. Adult crowding measures for low-acuity patients, such as waiting room time (r = 0.421) and length of stay for discharged patients (r = 0.548), had moderate correlation with GFT.

Conclusions. City-level GFT shows strong correlation with influenza cases and ED ILI visits, validating its use as an ED surveillance tool. GFT correlated with several pediatric ED crowding measures and those for low-acuity adult patients.

Influenza is a significant source of morbidity and mortality, affecting 5%–20% of the population and causing 294 000 hospitalizations and 34 000 deaths annually in the United States [1–3]. Emergency departments (EDs) supply the critical infrastructure to provide medical care in the event of a disaster or disease outbreak, including seasonal and pandemic influenza [4]. Already overcrowded and stretched to near-capacity, the increased patient volumes during a typical influenza season lead to ED crowding [5, 6], and the high ED patient volumes expected during a true influenza pandemic represent a significant threat to the US healthcare infrastructure [7]. The ability of EDs to manage both seasonal and pandemic influenza surges is dependent on coupling early detection with a graded rapid response, a fundamental principle of infectious disease outbreak control.

Although many EDs have devised influenza response measures, ranging from methods to offset patient volume (eg, ED annexes [8], new influenza clinics [9], and “drive-through” medicine [10, 11]) to new intensive ED, hospital [12, 13], and patient-administered triage systems [14], the potential utility of coupling early warning systems with the various response strategies for managing influenza outbreaks in the ED setting has not been rigorously studied. Established surveillance systems, such as the US Influenza Sentinel Provider Surveillance Network of the Centers for Disease Control and Prevention (CDC), rely on a combination of clinical symptoms, viral laboratory results, hospital admissions, and mortality statistics. Practical use of CDC data for real-time response is limited, however, because of a several-week lag in reporting [15]. For earlier detection, alternative influenza surveillance systems have been developed, including call volume of telephone triage services [16], over-the-counter drug sales [17], and volumes of ED patients with influenzalike illness (ILI) [18–24]. However, translation of these surveillance systems to inform large-scale ED-based planning has not happened, in large part because of the burden of data collection and logistics of data-reporting that cause significant time-lags between events and accessible reporting.

In comparison with other methods of surveillance reporting, Internet search data use the frequency and types of health-related Internet queries from the public at large. Innovators in information technology have begun to harness Internet search data, designing novel Internet-based surveillance systems, creating the opportunity for immediate, easily accessible surveillance methods [25–27]. One such system, Google Flu Trends (GFT), uses 45 ILI-related search queries to estimate the number of ILI-based physician visits in a particular city, state, or region, resulting in a free, easily accessible format available online [28]. A landmark study, published in Nature, demonstrated the promise of this method reporting cross-validation of GFT with estimated weekly national ILI percentages, 7–10 days before the CDC’s US Influenza Sentinel Provider Surveillance Network [25]. However, other studies have suggested that this strong correlation with ILI visits does not translate to correlation with actual cases of influenza [29]. Thus, significant gaps remain for translating this tool into practice for ED response planning, most notably, demonstration of the local GFT correlation with ED ILI activity influenza cases, and ED crowding. We assessed the temporal correlation of city-level GFT data with local cases of confirmed influenza, the volume of ED patients with ILI, and standard ED crowding indices from an inner-city ED.

METHODS

Study Population and Setting

We performed a retrospective observational study of patients presenting to an urban academic ED in Baltimore, Maryland, with an annual volume of 60 000 adult visits and 24 000 pediatric visits and with physically and administratively distinct EDs for adult and pediatric patients. All pediatric and adult ED visits during a 21-month period from 25 January 2009 through 3 October 2010 were included. Adult psychiatric visits were excluded, because of outlying prolonged length of stay associated with a lack of available inpatient psychiatric beds, with no relation to influenza or ILI.

Data Collection and Methods of Measurement

All measurements were recorded at the level of the week to allow comparison with stored GFT data. Measures of influenza activity and ED crowding were evaluated separately for pediatric and adult patients, because the hospital in this study had 2 EDs that operate using distinct administrative oversight and staff. GFT data for Baltimore were downloaded directly from http://www.google.org/flutrends in the same weekly format of the ILI surveillance for Baltimore. GFT transforms search query data to estimate weekly ILI-related physician visits for Baltimore. Influenza activity in the local population was based on the total number of positive influenza test results from the hospital virology laboratory, which reflects clinical testing of inpatients, outpatients and ED patients. Influenza peak was operationally defined as a period with ≥5 weekly influenza cases in accordance with Hospital Epidemiology and Infection Control respiratory virus definitions. The number of patients presenting to our ED with ILI (according to the CDC definition of fever and cough or sore throat) was collected from the CDC’s traditional surveillance methods reporting system (Biosense [30]; from 25 January 2009 through 18 October 2009) and an ED electronic reporting system (from 18 October 2009 through 3 October 2010) used at our site during the study period.

ED crowding metrics were chosen on the basis of previous consensus statements and included [31, 32] total number of ED visits, number of patients who left without being seen by a clinician, mean waiting time, and mean length of ED stay for admitted and for discharged patients. Mean weekly times were measured both overall (for all patients) and for each of 3 triage acuity groups to assess whether level of acuity was a modifying factor. Patients were triaged using the Emergency Severity Index according to current practice in this ED, which triages patients based on acuity into 5 levels, with the most acutely ill patients in level 1 and the lowest acuity patients in level 5 [33]. Triage acuity levels were then grouped into 3 triage acuity groups (levels 1 and 2, level 3, and levels 4 and 5).

Statistical Analysis

Initial data analysis was performed by graphically evaluating data trends over time. Cross-correlation analysis was performed to examine GFT’s correlation with ED-based influenza and crowding measures using Stata software, version 11 (StataCorp). Incremental 1-week lags (ie, 10 weeks forward and 10 weeks backward) were introduced to assess these relationships temporally. For example, a 1-week lag evaluates the correlation when GFT results are shifted 1 week backward. Correlation coefficients >0.8 were operationally defined as excellent correlation, 0.6–0.8 indicated good correlation, 0.4–0.6 indicated moderate correlation, and <0.4 indicated poor correlation.

During the study period, both the adult and pediatric ED implemented systems interventions to treat the volumes of patients with influenza. The adult ED shifted staff, as needed, from the observation unit to the main ED beginning on 25 October 2009. The pediatric ED opened an annex on 5 October 2009, with additional beds and staff to accommodate patients with ILI. It is unclear how these interventions may have affected ED crowding metrics; thus, a stratified analysis was performed using these 2 dates as intervention dates. Cross-correlation including only the preintervention data did not differ significantly from the original data analysis; thus, the analysis using all dates included in the study was used for the final analysis.

RESULTS

During the study period, there were 3 main influenza peaks: February 2009, June 2009, and October 2009. Figure 1 shows the temporal relationship between GFT and the number of positive influenza test results from the hospital virology laboratory. The sharp upslope in GFT data preceded the influenza data by 1–2 weeks in September 2009 but not in February 2009 or June 2009. Analytically, GFT had high correlation values with the number of positive influenza test results in both the adult (r = 0.876) and the pediatric (r = 0.718) populations, with the greatest correlation between no lag and 1-week lag. When a 1-week lag was introduced into the analysis, the correlation between GFT and positive influenza test results improved to 0.741 in the pediatric but not the adult population. Table 1 lists the correlation coefficients for no lag and a 1-week lag for both the pediatric and the adult EDs. For all correlations, the peak correlation, if one existed, occurred with either no lag or a 1-week lag. GFT also highly correlated with the number of patients presenting with ILI to both the adult (r = 0.885) and the pediatric (r = 0.652) EDs. As shown in Figure 2, the peak of patients presenting with ILI occurred in October 2009, with a subsequent elevated level in the pediatric population throughout the winter. Again, GFT data preceded the sharp increase in patients presenting to the ED with ILI by 1–2 weeks. The number of ILI visits also increased during the February 2009 influenza peak and, to a lesser extent, during the June 2009 influenza peak.

Figure 1.

Temporal comparison of Google Flu Trends and hospital-wide positive influenza test results.

Table 1.

Correlation Coefficient Values for Cross-Correlation With Google Flu Trends for Baltimore, Maryland

	Adult ED		Pediatric ED
Variable	No Lag	1-Week Lag	No Lag	1-Week Lag
No. of positive influenza test results	0.876	0.823	0.718	0.741
No. of ED patients with ILI	0.885	0.833	0.652	0.655
ED visit volume	<0.40	<0.40	0.649	0.707
Leaving without being seen	<0.40	<0.40	0.641	0.735
Waiting room time
Combined	<0.40	<0.40	<0.40	<0.40
Levels 1 and 2	<0.40	<0.40	<0.40	<0.40
Level 3	<0.40	<0.40	<0.40	<0.40
Levels 4 and 5	0.421	0.498	<0.40	<0.40
LOS for admitted patients
Combined	<0.40	<0.40	<0.40	<0.40
Levels 1 and 2	<0.40	<0.40	0.406	0.406
Level 3	<0.40	<0.40	<0.40	<0.40
Levels 4 and 5	<0.40	<0.40	<0.40	<0.40
LOS for discharged patients
Combined	<0.40	0.507	<0.40	0.406
Levels 1 and 2	<0.40	<0.40	<0.40	<0.40
Level 3	<0.40	<0.40	<0.40	<0.40
Levels 4 and 5	0.548	0.616	0.445	0.513

Abbreviations: ED, emergency department; ILI, influenzalike illness; LOS, length of stay.

Figure 2.

Temporal comparison of Google Flu Trends and the number of patients presenting to the emergency department with influenzalike illness. Abbreviations: ED, emergency department; ILI, influenzalike illness.

Weekly ED visit volumes showed good correlation with GFT in the pediatric ED (r = 0.649) but poor correlation in the adult ED. Figure 3 graphically depicts this temporal relationship and shows a steep upslope of GFT preceding the increase in pediatric ED volume. Of note, during the main influenza peak, the pediatric ED had a 74% increase in total patient volume, compared with the median value (median weekly volume, 513 patients; range, 330–893 patients) and with an 11% increase in the adult ED (median weekly volume, 1117 patients; range, 823–1242 patients). During that peak, patients with ILI made up 49% of the patient volume seen at the pediatric ED, but only 6% of the patient volume in the adult ED. Similarly, the number of patients who left without being seen by a clinician showed good correlation (r = 0.641) in the pediatric ED and poor correlation in the adult ED.

Figure 3.

Temporal comparison of Google Flu Trends and the volume of emergency department patient visits. Abbrevation: ED, emergency department.

For the remainder of the ED crowding metrics, GFT showed moderate to poor correlation. Length of ED stay for discharged patients had moderate correlation for both adult (r = 0.548) and pediatric (r = 0.445) level 4 and 5 patients. GFT likewise had moderate correlation in waiting room time for level 4 and 5 adult patients (r = 0.421). Length of stay for admitted patients showed a moderate correlation with GFT only in pediatric level 1 and 2 patients (r = 0.406).

DISCUSSION

EDs must be able to respond to a surge in medical need for both seasonal and pandemic influenza. Because many EDs already operate at or near-capacity, accurate and timely surveillance, coupled with planned response measures, is essential. This study validates the use of weekly city-level GFT as an ED surveillance tool because of its correlation with both positive influenza test results and volume of patients with ILI presenting to the ED. There was a less strong relationship with ED crowding measures, such as prolonged length of stay, which would be expected to occur with higher demand. This relationship was stronger in the pediatric ED than in the adult ED, where there did not appear to be as great an increase in ED visits surrounding increased influenza activity. These relationships highlight the potential usefulness of Internet-based search data on ED-based strategies to better match the supply of ED resources to surges in demand that occur during influenza outbreaks.

GFT provides near-real-time surveillance data 7–10 days before the CDC’s US Influenza Sentinel Provider Surveillance Network [25]. By harnessing health-related searches on the Internet, GFT combines focused information from a large spectrum of the population with geospatial data to create a broad-reaching yet geographically specific surveillance system. The resulting data are provided free of charge via the Internet, with options to narrow data to the state, regional, or city level.

Overall, GFT correlated well with influenza activity in the ED, but not perfectly. The smaller peak in February 2009 represents 2008–2009 seasonal influenza, the peak in June 2009 represents the initial wave of H1N1 influenza, and the main peak in October 2009 represents the major surge of the H1N1 influenza pandemic. GFT showed good to excellent correlation with both the number of positive influenza tests and the number of those patients presenting to the ED, as shown by the numbers of ED patients with ILI. Of note, during the pandemic peak, GFT preceded the actual increase in ED patients with ILI or positive influenza test results by 1 week. The influenza peak in June 2009 was not detected by GFT, possibly because of the previous month’s flurry of Internet activity surrounding the news coverage of the H1N1 outbreak. However, the overall correlation between GFT data and influenza activity remain strong. Cross-correlation analysis showed the highest correlation resulted at either 0 or 1 week, demonstrating that there was no substantial lag in GFT estimates. The combination of good correlation with influenza activity and the anticipatory increase of GFT highlight the potential of using GFT as an influenza surveillance tool for ED staffing and surge capacity planning measures.

GFT had an additional peak in late April that was not mirrored in the number of patients with ILI or positive influenza test results. This peak probably corresponds to the increasing news coverage of the H1N1 pandemic, because it began the day after the CDC declared H1N1 as a national public health emergency. McDonnell et al termed this period as “fear week” and described a surge in the Google search term “swine flu” in the absence of actual documented cases of influenza [34]. However, fear week reflected more than simply public fear of influenza, because ED patient volumes increased by 7.0% across the United States during that week, with a 19% increase in pediatric ED visits and a 1% increase in adult ED visits compared with baseline [34]. This increase is comparable to the increase seen with the actual influenza surge in June 2009, measured by the same mechanism, with a 6.6% overall increase in ED visits. This is similar to our ED volume data, where pediatric ED volumes increased during fear week despite the lack of documented influenza cases. These findings reflect one of the inherent limitations of Internet-based surveillance tools, which by definition, may identify the public perceptions of the threat of influenza as a signal and as actual symptoms or cases. However, because the increase in GFT correlates with an increase in ED patient volumes, there remains practical use of the system for surge capacity planning for EDs.

To determine how linking GFT data to a response plan would affect ED crowding, GFT data were correlated with some basic ED crowding metrics. The correlation with ED crowding measures was stronger in the pediatric than in the adult ED. This finding was not surprising, because of the significantly larger proportion of total ED patient volumes in the pediatric ED attributable to influenza and ILI that we observed—a phenomenon which has been previously described in other EDs [34, 35]. This effect was further increased by the H1N1 virus, which had a greater impact on the younger population and was the main circulating strain observed during the period of this study.

Remaining crowding measures showed that length of stay increased in both the pediatric and the adult populations, whereas in the adult population, there was little correlation with a higher number of visits. This indicates that, during episodes of influenza activity, patients may require more resources in the ED, such as intravenous fluids or other interventions, requiring longer stays. It also demonstrates insufficient supply-demand matching, which is the major problem that underlies ED crowding in general. When divided by acuity level, GFT had moderate correlation with length of stay for levels 1 and 2 admitted pediatric patients and length of stay for levels 4 and 5 discharged pediatric patients, indicating that this resource mismatch impacted both ends of the acuity spectrum and the inpatient services in addition to the ED. Overall, the only adult ED crowding measures that correlated with GFT were waiting time and length of stay of discharged patients, greatest in level 4 and 5 patients.

The main limitation of the study is the lack of regional and temporal generalizability, because this investigation was limited to 1 medical center, in 1 city, over a period of 2 years. This study used city-wide GFT data to correlate with influenza activity and crowding measures at one institution, thus assuming uniform distribution of both influenza activity and Google Flu searching throughout the city. It is additionally unclear how city-wide GFT data would correlate with suburban hospitals. This investigation was intended for demonstration purposes, however, and the GFT tool offers city- and region-specific data to allow others to evaluate regional generalizability. Furthermore, the data for this study covered 2 years, and this period was largely dominated by the 2009 novel H1N1 pandemic. Extension of this validation to more typical influenza seasons will improve the temporal generalizability. The use of cross-correlation analysis allows for validation and comparison of GFT data, but is not predictive. Further modeling will be necessary to determine the full predictive usefulness of GFT. The temporal precision of our data measurement was limited by the weekly scale with which GFT stores data. Evaluating data on a daily scale may have improved the ability to identify the lead time between GFT and measures of regional influenza activity through the use of daily lags in the cross-correlation analysis.

During the study period, we found that GFT had good correlation with both ED cases of influenza and the number of patients presenting to the ED with ILI, validating GFT as an influenza surveillance tool in Baltimore. GFT correlated well with several pediatric ED crowding measures and those for low-acuity adult patients. This highlights the potential value of linking GFT with an ED response plan. To fully use this new surveillance tool, additional analysis must be performed to incorporate GFT data in a predictive model, which can then be linked to a response plan.