Abstract
Surveillance for acute respiratory infection (ARI) and influenza-like illness (ILI) relies primarily on reports of medically attended illness. Community surveillance could mitigate delays in reporting, allow for timely collection of respiratory tract samples, and characterize cases of non–medically attended ILI representing substantial personal and economic burden. Text messaging could be utilized to perform longitudinal ILI surveillance in a community-based sample but has not been assessed. We recruited 161 households (789 people) in New York City for a study of mobile ARI/ILI surveillance, and selected reporters received text messages twice weekly inquiring whether anyone in the household was ill. Home visits were conducted to obtain nasal swabs from persons with ARI/ILI. Participants were primarily female, Latino, and publicly insured. During the 44-week period from December 2012 through September 2013, 11,282 text messages were sent. In responses to 8,250 (73.1%) messages, a household reported either that someone was ill or no one was ill; 88.9% of responses were received within 4 hours. Swabs were obtained for 361 of 363 reported ARI/ILI episodes. The median time from symptom onset to nasal swab was 2 days; 65.4% of samples were positive for a respiratory pathogen by reverse-transcriptase polymerase chain reaction. In summary, text messaging promoted rapid ARI/ILI reporting and specimen collection and could represent a promising approach to timely, community-based surveillance.
Keywords: acute respiratory infection, influenza, influenza-like illness, respiratory viruses, surveillance, text messaging
Surveillance for influenza-like illness (ILI) in the United States is conducted by state and local health departments and the Centers for Disease Control and Prevention (1, 2). These data are crucial for informing public health officials about influenza activity during seasonal epidemics and pandemics (3) and for tracking other acute respiratory infections (ARIs), which can also be associated with serious disease (4). Surveillance for ILI and ARI relies primarily on reports of outpatient visits made by primary-care providers and reports of emergency department visits and/or hospitalizations (2, 5–8). This type of surveillance, however, captures only medically attended ILI and therefore underestimates disease prevalence; it may also be biased by factors associated with seeking medical care. Additionally, while capturing more severe influenza cases that require medical attention is important from a health-care utilization perspective, from a personal, societal, and economic perspective, understanding the true burden of all influenza cases is important, since ILI leads to many lost days of school and work. For example, in 1 US study, non–medically attended influenza was estimated to cost $2 billion annually (9).
Although performance of ILI surveillance by persons in the community is rarely undertaken, this strategy has several benefits. Community surveillance is valuable, as it captures cases of non–medically attended ILI/ARI and thereby allows better estimates of the true disease burden and the spectrum of disease. Further, surveillance of whole households allows for assessment of secondary transmission. Reporting of acute illness at illness onset also facilitates more rapid reporting of ILI/ARI, since there is no lag time associated with waiting for a person to seek medical care. In previous pandemics, the time delay from illness onset to acute-care visit to provider report has been problematic (7). Finally, direct reporting may also facilitate more timely collection of respiratory tract secretions for pathogen testing within the optimal 48–72 hours after illness onset (10).
Text messaging has been used for vaccination reminders and other behavioral interventions (11, 12) and for vaccine adverse event surveillance (13). In Madagascar, outpatient providers have used text messaging to report aggregate practice-level ILI incidence data (14), and in Mexico, text messaging was pilot-tested for a one-time query of ILI symptoms in the general population during the 2009 influenza A(H1N1) pandemic (15). However, to our knowledge, text messaging has not been evaluated for longitudinal ILI/ARI surveillance in a community-based sample. Thus, we aimed to assess the feasibility of using text messaging to conduct ILI/ARI surveillance in a US community and the impact of text messaging on timeliness of illness reporting and specimen collection.
METHODS
Study design and participants
Between December 2012 and February 2013, we recruited households in New York City for a 5-year community-based study of ILI/ARI surveillance, the Mobile Surveillance for Acute Respiratory Infections and Influenza-Like Illness in the Community (MoSAIC) Study. Households were identified by contacting a random sample of participants who had taken part in a large population-based survey of an urban, primarily immigrant Latino community (http://www.wicer.org). For the current study, eligible households had 3 or more members with at least 1 member under 18 years of age, were Spanish- or English-speaking, and had a cellular telephone with text messaging. During household visits, 1 person volunteered to be the household reporter, and research staff collected sociodemographic information and health history for all household members from the household reporter. The institutional review boards of the Columbia University Medical Center (New York, New York) and the Centers for Disease Control and Prevention (Atlanta, Georgia) approved this study.
Text messages and specimen collection
The household reporter received the following text message twice weekly: “Reply with 1 or 2. Does anyone in the household have runny nose, congestion, sore throat, cough, body aches, or fever, or feels [sic] hot? 1: yes; 2: no.” Messages were sent in English or Spanish based on the reporter's preference. As an incentive to participate, household reporters received $20 each month if they responded to at least 75% of the month's text messages.
Research staff followed up on text message reports of a potentially ill person via a phone call to the household reporter to assess whether criteria for ILI/ARI had been met. If the criteria were met, staff conducted a home visit to obtain a nasal swab from persons who had been ill for 5 days or less. If, at that visit, an index case (defined as the first symptomatic individual in the household) was identified and had been sick for more than 5 days but was still symptomatic, that person was also swabbed. Additionally, households were visited monthly to promote retention. If a household member met ILI/ARI criteria at an enrollment visit or during the monthly visit, a swab was obtained. Informed consent from adults and verbal assent from appropriately aged children were obtained for collection of the sample.
The MoSAIC text messaging system is automated; messages are prescheduled to be sent, replies are precoded, and receipt of replies is also automated. Updating of messages to include new enrollees or to change phone numbers takes approximately 10 minutes for 1–5 participants. Study staff spend up to 15–20 minutes per week updating the system with changes to phone numbers and approximately 5–10 minutes in total twice a week reviewing all responses. For those responses indicating “sick,” the staff spends 5–60 minutes twice weekly to confirm ARI/ILI criteria and schedule a home visit, depending on how many sick participants there are. Each home visits takes approximately 45 minutes, including time needed to travel to the household, obtain the swab, update the study database, and deliver the swab to the laboratory.
Analysis
To evaluate the feasibility of text messaging for community-level surveillance, we determined the first year's outcomes regarding text message response rates, median numbers of days from symptom onset to swab collection, and the proportions of each pathogen detected. All analyses were conducted using SPSS, version 20 (SPSS Inc., Chicago, Illinois). Swabs were analyzed by means of a nested multiplex polymerase chain reaction assay (FilmArray Respiratory Panel 1.7) that identifies 17 viral and 3 bacterial respiratory pathogens (BioFire Diagnostics, LLC, Salt Lake City, Utah).
RESULTS
We recruited 158 of 187 (84.5%) eligible households; nearly all (98.1%) remained in the study throughout the first year (December 2012–September 2013), and 3 were replaced because they moved out of the catchment area. Therefore, the study comprised a total of 161 households. Fewer than 1% of households were ineligible to participate because they did not have a cell phone, and 5.5% were ineligible because they did not have text messaging. Households included an average of 4.9 members, and participants (n = 789) were primarily female, Latino, and publicly insured (Table 1).
Table 1.
Characteristic | Participants (n = 789) |
|
---|---|---|
No. | % | |
Age group, years | ||
<5 | 91 | 11.5 |
5–17 | 246 | 31.2 |
18–64 | 425 | 53.9 |
≥65 | 27 | 3.4 |
Sex | ||
Female | 482 | 61.1 |
Male | 307 | 38.9 |
Latino ethnicity | 784 | 99.4 |
Race | ||
White | 293 | 37.1 |
Black | 4 | 0.5 |
Asian | 1 | 0.1 |
Othera | 491 | 62.2 |
Type of health insurance | ||
Medicaid | 603 | 76.4 |
Private | 76 | 9.6 |
Uninsured | 110 | 13.9 |
Nativity | ||
Born in United States | 356 | 45.1 |
Foreign-born, ≥10 years in United States | 254 | 32.2 |
Foreign-born, <10 years in United States | 179 | 22.7 |
Language (adults) | ||
Spanish | 322 | 71.2 |
English | 124 | 27.4 |
Other | 6 | 1.3 |
Education (adults) | ||
Less than high school | 195 | 43.1 |
High school graduation | 113 | 25.0 |
Some college | 144 | 31.9 |
Abbreviation: MoSAIC, Mobile Surveillance for Acute Respiratory Infections and Influenza-Like Illness in the Community.
a In this primarily Dominican-born population, “other” represents self-reported race and includes multiracial, Latino/Hispanic, and other self-described racial groups.
During the first year, 11,282 text messages were sent, and responses were received for 8,250 (73.1%). Response rates remained high throughout the study period (range, 68.1%–83.7% per month). In the first full month of the study, the response rate was 75.1%, and in the last month, the rate was 70.1%. On average, 84.5% of households responded at least once in a given week (range, 74.7%–100% per study week). Most responses (88.9%) were received within 4 hours, and 96.2% were received within 12 hours. The rates of response within 4 hours were similar for all study months (range, 85.6%–93.5%).
There were 363 ILI/ARI episodes reported, for which 361 (99.4%) nasal swabs were obtained (2 participants refused). Most swabs (74.5%) included a report of symptoms of ARI/ILI; the remainder were collected during a monthly visit only (17.7%) or during an enrollment visit (7.8%). Approximately half (52.9%) of reported cases of ARI/ILI occurred among children. Two-thirds of swabs (69.5%) were from index cases. The median time from symptom onset to nasal swab was 2 days. Nearly half (41.6%) of samples were obtained within 1 day of symptom onset; 65.7% were obtained within 2 days, 82.3% within 3 days, and 95.8% within 5 days. Collection of specimens within 3 days of symptom onset was similar for all study months (range, 69.8%–92.3%). If messages had been sent once a week and samples had been collected on the same day as the message was sent, 47.4% of swabs would have been obtained within 3 days of symptom onset, and 68.1% would have been obtained within 5 days.
Overall, 236 (65.4%) samples were positive for a respiratory pathogen, of which 44 (18.6%) were positive for influenza A(H3N2) or influenza B. In addition, 12 other respiratory pathogens were detected, including subtypes of coronavirus (n = 39), subtypes of parainfluenza (n = 20), respiratory syncytial virus (n = 9), adenovirus (n = 2), human metapneumovirus (n = 13), rhinovirus/enterovirus (n = 117), and Chlamydophila pneumoniae (n = 3). A few of these cases (n = 10) represented co-infection with more than 1 pathogen. There was no significant difference in rates of detection of a respiratory virus in samples collected within 3 days of symptom onset and those collected after 3 days (66.0% and 62.5%, respectively; P = 0.59).
One-quarter (24.9%) of the swabs collected were obtained from children under 5 years of age; their positivity rate was 72.2%, and 8 (12.3%) were positive for influenza. Additionally detected pathogens among children under 5 included subtypes of coronavirus (n = 5), subtypes of parainfluenza (n = 8), respiratory syncytial virus (n = 5), adenovirus (n = 1), human metapneumovirus (n = 5), and rhinovirus/enterovirus (n = 42). Most (n = 8) of the instances of co-infection were in this age group. The positivity rate (55.6%) for persons aged 65 years or more was the lowest observed among the age categories. Of the 5 positive cases, 3 were cases of rhinovirus/enterovirus, 1 was coronavirus, and 1 was parainfluenza.
Overall, 85 (23.5%) of the 361 ARI/ILI episodes and 12 (27.3%) of the 44 laboratory-confirmed influenza episodes were associated with a medical visit. Among all of the ARI/ILI episodes, health care was sought only a portion of the time, with 16.6% of episodes including a visit to a primary-care provider, 4.4% including a visit to an emergency department, 1.4% including visits to both a primary-care provider and an emergency department, and less than 1% including a visit to a retail clinic or occupational health service. There were significant differences in the frequency of illness episodes involving medical visits by insurance status (42.0% commercial, 21.9% public, and 7.1% uninsured; P = 0.001) and age (41.1% <5 years, 18.8% 5–17 years, 17.4% 18–64 years, and 11.1% ≥65 years; P < 0.0001). There were nonsignificant trends based on language spoken in the household (18.9% Spanish, 12.7% English; P = 0.32) and patient's education (adults only) (22.0% less than high school, 11.1% high school, and 16.2% some college; P = 0.38).
DISCUSSION
Using this novel approach to prospective ILI/ARI surveillance in the community, a high and rapid rate of response to text messages facilitated prompt collection of respiratory specimens, which was sustained during the 10-month analysis period. This suggests that text messaging is a potentially valuable tool for performing ILI/ARI surveillance in the community.
Other household-based cohort studies in the United States and internationally have used various methods to perform surveillance for ILI/ARI, including telephone or e-mail reminders, symptom diaries and reporting hotlines, and weekly home visits (16–18). The use of text messaging as a surveillance tool has several advantages. Currently, 90% of US adults own a cell phone (19). Minority adults also often use text messaging (20), which could make text messaging a particularly effective mechanism for conducting surveillance in minority populations (20, 21). Text messaging also prompts timely reporting, which can decrease recall bias. A key feature of text messaging is scalability, allowing large numbers of participants to be monitored at one time; after the initial investment, the cost of each additional message is negligible. Messages can be automated and sent simultaneously to as many people as desired. If precoded responses are used, as in this study, screening of responses is also rapid. Additionally, while many people can be accessed simultaneously across large geographical areas and in different languages, the sending of the messages can be centralized; this allows widespread deployment with standardization of data collection.
Rapid monitoring of ARI/ILI is essential for national and international seasonal and pandemic influenza planning (22). US public health officials and international consortiums have endorsed community surveys in well-defined populations and household studies as primary methods for assessing incidence and secondary attack rates of pandemic influenza (23, 24). Surveillance of medically attended ARI/ILI underestimates illness incidence; in our study, only one-quarter of ill persons sought medical care, and this was more likely to occur for younger persons and those with commercial insurance.
Other technology has been used for non–medically attended disease surveillance at a broader community level, such as Google Flu Trends (Google, Mountain View, California) and Twitter (Twitter Inc., San Francisco, California) (25–27) or crowd-sourcing of surveillance data. However, these strategies have not been fully validated (28), and they can only collect aggregate data without any specific details about any 1 individual's illness. More importantly, these strategies monitor trends of people discussing or searching for ARI-/ILI-related subject matter, and as such there cannot be confirmation of symptoms. Additionally, samples cannot be obtained to confirm an infectious etiology; confirmation of ARI/ILI etiology is important, since there is poor correlation between ILI symptoms and actual laboratory-confirmed influenza (29). In our study, surveillance by text messaging allowed collection of individual-level data as well as collection of a respiratory specimen for confirmation of disease. The use of text messaging also facilitated rapid collection, resulting in most specimens being obtained within 48–72 hours of symptom onset, when viral shedding is highest (10). However, in this study there was no difference in the detection of a respiratory viral pathogen among samples obtained within 72 hours of symptom onset and samples obtained after 72 hours; this finding may be due to the ability of polymerase chain reaction assays to detect small amounts of virus or nonviable virus after several days of illness.
There were limitations to this study. Although participation rates were high, persons who chose to participate could have differed from the general population. For example, text-message surveillance may be difficult to perform in the elderly (19, 30). As with any surveillance study, there is a risk of underreporting, but we received 73% of expected replies. Since households reported twice weekly, we do not know how reporting would have been affected if they were only asked to report once a week. However, we have not received complaints about twice-weekly reporting, and this frequency allows for optimal timing of specimen collection. We do not know the extent to which compensation paid to participating households may have contributed to reporting rates. The issues associated with compensation for participating in ongoing active surveillance are likely to be the same regardless of the data collection method utilized. However, reaching and tracking each household by phone is more time-consuming than sending 1 batch of text messages simultaneously and following up via phone solely with those who report symptoms. It also may be more burdensome and intrusive for families to have to answer a phone call as opposed to returning a text message at a time that is convenient for them. In addition, tracking of phone calls requires more data entry than tracking of text message replies, which can be automatically retrieved from the text message platform. In future studies, investigators could assess differences in reporting and costs for phone reporting versus text messaging. We identified a respiratory pathogen for only 65% of ARI/ILI episodes, although this is consistent with other studies using reverse-transcriptase polymerase chain reaction assays (31, 32). Finally, the project was conducted in an English- and Spanish-speaking low-income urban population. Thus, our findings may not apply to other communities, and this should be assessed.
In summary, text messaging promoted rapid ILI reporting and specimen collection and represents a promising approach to timely surveillance in the community. In further work, researchers should explore the feasibility of expanding data collection to a larger population for wider surveillance of influenza or other infections. This rapid surveillance approach, coupled with specimen collection, could be useful for monitoring other seasonal and pandemic respiratory and nonrespiratory illnesses, as well as for a bioterrorism event.
ACKNOWLEDGMENTS
Author affiliations: Division of Child and Adolescent Health, Department of Pediatrics, College of Physicians and Surgeons, Columbia University, New York, New York (Melissa S. Stockwell, Celibell Y. Vargas, Stewin Camargo); Department of Population and Family Health, Mailman School of Public Health, Columbia University, New York, New York (Melissa S. Stockwell); NewYork-Presbyterian Hospital, New York, New York (Melissa S. Stockwell); Influenza Division, Centers for Disease Control and Prevention, Atlanta, Georgia (Carrie Reed, Lyn Finelli); Division of Pediatric Infectious Diseases, Department of Pediatrics, College of Physicians and Surgeons, Columbia University, New York, New York (Lisa Saiman, Aaron F. Garretson, Luis R. Alba, Philip LaRussa); Department of Nursing Scholarship and Research, School of Nursing, Columbia University, New York, New York (Elaine L. Larson); Department of Epidemiology, Mailman School of Public Health, Columbia University, New York, New York (Elaine L. Larson); and Department of Infection Prevention and Control, NewYork-Presbyterian Hospital, New York, New York (Lisa Saiman).
This study was funded by grant U01IP000618 from the Centers for Disease Control and Prevention.
We thank the other members of the MoSAIC study team: Hilbania Diaz, Dr. Yaritza Castellanos de Belliard, Dr. Maria Morban, Othanya Garcia, and Liqun Wang.
An abstract of this article was presented at the 2014 Pediatric Academic Societies Annual Meeting, Vancouver, British Columbia, Canada, May 3–6, 2014.
Investigators from the Centers for Disease Control and Prevention took part in the design and conduct of the study, the analysis and interpretation of the data, and the review and approval of the manuscript. The findings and conclusions presented in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.
Conflict of interest: none declared.
REFERENCES
- 1.Armour PA, Nguyen LM, Lutman ML, et al. Evaluation of the novel respiratory virus surveillance program: Pediatric Early Warning Sentinel Surveillance (PEWSS) Public Health Rep. 2013;128(suppl 2):88–96. doi: 10.1177/00333549131280S213. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Centers for Disease Control and Prevention. Overview of influenza surveillance in the United States. http://www.cdc.gov/flu/weekly/overview.htm. Updated October 24, 2013. Accessed June 16, 2014.
- 3.Centers for Disease Control and Prevention. Self-reported influenza-like illness during the 2009 H1N1 influenza pandemic—United States, September 2009–March 2010. MMWR Morb Mortal Wkly Rep. 2011;60(2):37–41. [PubMed] [Google Scholar]
- 4.Falsey AR, McElhaney JE, Beran J, et al. Respiratory syncytial virus and other respiratory viral infections in older adults with moderate to severe influenza-like illness. J Infect Dis. 2014;209(12):1873–1881. doi: 10.1093/infdis/jit839. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Miroballi Y, Baird JS, Zackai S, et al. Novel influenza A(H1N1) in a pediatric health care facility in New York City during the first wave of the 2009 pandemic. Arch Pediatr Adolesc Med. 2010;164(1):24–30. doi: 10.1001/archpediatrics.2009.259. [DOI] [PubMed] [Google Scholar]
- 6.Marsden-Haug N, Foster VB, Gould PL, et al. Code-based syndromic surveillance for influenzalike illness by International Classification of Diseases, Ninth Revision. Emerg Infect Dis. 2007;13(2):207–216. doi: 10.3201/eid1302.060557. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Centers for Disease Control and Prevention. Assessment of ESSENCE performance for influenza-like illness surveillance after an influenza outbreak—U.S. Air Force Academy, Colorado, 2009. MMWR Morb Mortal Wkly Rep. 2011;60(13):406–409. [PubMed] [Google Scholar]
- 8.Lewis MD, Pavlin JA, Mansfield JL, et al. Disease outbreak detection system using syndromic data in the greater Washington DC area. Am J Prev Med. 2002;23(3):180–186. doi: 10.1016/s0749-3797(02)00490-7. [DOI] [PubMed] [Google Scholar]
- 9.Molinari NA, Ortega-Sanchez IR, Messonnier ML, et al. The annual impact of seasonal influenza in the US: measuring disease burden and costs. Vaccine. 2007;25(27):5086–5096. doi: 10.1016/j.vaccine.2007.03.046. [DOI] [PubMed] [Google Scholar]
- 10.Centers for Disease Control and Prevention. Guidance for clinicians on the use of RT-PCR and other molecular assays for diagnosis of influenza virus infection. http://www.cdc.gov/flu/professionals/diagnosis/molecular-assays.htm. Updated July 9, 2013. Accessed June 16, 2014.
- 11.Stockwell MS, Kharbanda EO, Martinez RA, et al. Effect of a text messaging intervention on influenza vaccination in an urban, low-income pediatric and adolescent population: a randomized controlled trial. JAMA. 2012;307(16):1702–1708. doi: 10.1001/jama.2012.502. [DOI] [PubMed] [Google Scholar]
- 12.Free C, Phillips G, Watson L, et al. The effectiveness of mobile-health technologies to improve health care service delivery processes: a systematic review and meta-analysis. PLoS Med. 2013;10(1):e1001363. doi: 10.1371/journal.pmed.1001363. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Stockwell MS, Broder K, LaRussa P, et al. Risk of fever after pediatric trivalent inactivated influenza vaccine and 13-valent pneumococcal conjugate vaccine. JAMA Pediatr. 2014;168(3):211–219. doi: 10.1001/jamapediatrics.2013.4469. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Rajatonirina S, Heraud JM, Randrianasolo L, et al. Short message service sentinel surveillance of influenza-like illness in Madagascar, 2008–2012. Bull World Health Organ. 2012;90(5):385–389. doi: 10.2471/BLT.11.097816. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Lajous M, Danon L, López-Ridaura R, et al. Mobile messaging as surveillance tool during pandemic (H1N1) 2009, Mexico. Emerg Infect Dis. 2010;16(9):1488–1489. doi: 10.3201/eid1609.100671. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Ohmit SE, Petrie JG, Malosh RE, et al. Influenza vaccine effectiveness in the community and the household. Clin Infect Dis. 2013;56(10):1363–1369. doi: 10.1093/cid/cit060. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Klick B, Nishiura H, Ng S, et al. Transmissibility of seasonal and pandemic influenza in a cohort of households in Hong Kong in 2009. Epidemiology. 2011;22(6):793–796. doi: 10.1097/EDE.0b013e3182302e8e. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Horby P, Mai le Q, Fox A, et al. The epidemiology of interpandemic and pandemic influenza in Vietnam, 2007–2010: the Ha Nam Household Cohort Study I. Am J Epidemiol. 2012;175(10):1062–1074. doi: 10.1093/aje/kws121. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Brenner J. Mobile technology fact sheet. Internet and American Life Project, Pew Research Center http://pewinternet.org/Commentary/2012/February/Pew-Internet-Mobile.aspx. Updated January 2014. Accessed September 17, 2014.
- 20.Zichuhr K, Smith S. Digital Differences. Washington, DC: Pew Research Center; 2012. http://www.pewinternet.org/~/media//Files/Reports/2012/PIP_Digital_differences_041312.pdf. Published April 13, 2012. Accessed June 16, 2014. [Google Scholar]
- 21.Blumberg SJ, Luke JV. Reevaluating the need for concern regarding noncoverage bias in landline surveys. Am J Public Health. 2009;99(10):1806–1810. doi: 10.2105/AJPH.2008.152835. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.US Department of Health and Human Services. HHS Pandemic Influenza Plan. Washington, DC: US Department of Health and Human Services; 2005. http://www.flu.gov/planning-preparedness/federal/hhspandemicinfluenzaplan.pdf. Accessed June 16, 2014. [Google Scholar]
- 23.Reed C, Biggerstaff M, Finelli L, et al. Novel framework for assessing epidemiologic effects of influenza epidemics and pandemics. Emerg Infect Dis. 2013;19(1):85–91. doi: 10.3201/eid1901.120124. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Van Kerkhove MD, Broberg E, Engelhardt OG, et al. The Consortium for the Standardization of Influenza Seroepidemiology (CONSISE): a global partnership to standardize influenza seroepidemiology and develop influenza investigation protocols to inform public health policy. Influenza Other Respir Viruses. 2013;7(3):231–234. doi: 10.1111/irv.12068. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Bernardo TM, Rajic A, Young I, et al. Scoping review on search queries and social media for disease surveillance: a chronology of innovation. J Med Internet Res. 2013;15(7):e147. doi: 10.2196/jmir.2740. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Dugas AF, Jalalpour M, Gel Y, et al. Influenza forecasting with Google Flu Trends. PLoS One. 2013;8(2):e56176. doi: 10.1371/journal.pone.0056176. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Chew C, Eysenbach G. Pandemics in the age of Twitter: content analysis of Tweets during the 2009 H1N1 outbreak. PLoS One. 2010;5(11):e14118. doi: 10.1371/journal.pone.0014118. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Butler D. When Google got flu wrong. Nature. 2013;494(7436):155–156. doi: 10.1038/494155a. [DOI] [PubMed] [Google Scholar]
- 29.Conway NT, Wake ZV, Richmond PC, et al. Clinical predictors of influenza in young children: the limitations of “influenza-like illness.”. J Ped Infect Dis. 2013;2(1):21–29. doi: 10.1093/jpids/pis081. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Smith A. How Americans use text messaging. Pew Internet and American Life Project, Pew Research Center http://www.pewinternet.org/2011/09/19/how-americans-use-text-messaging/ Published September 19, 2011. Accessed September 17, 2014.
- 31.Hayden RT, Gu Z, Rodriguez A, et al. Comparison of two broadly multiplexed PCR systems for viral detection in clinical respiratory tract specimens from immunocompromised children. J Clin Virol. 2012;53(4):308–313. doi: 10.1016/j.jcv.2011.12.020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Loeffelholz MJ, Pong DL, Pyles RB, et al. Comparison of the FilmArray Respiratory Panel and Prodesse real-time PCR assays for detection of respiratory pathogens. J Clin Microbiol. 2011;49(12):4083–4088. doi: 10.1128/JCM.05010-11. [DOI] [PMC free article] [PubMed] [Google Scholar]