Abstract
Objectives. We investigated the feasibility of combining an online chain recruitment method (respondent-driven detection) and participatory surveillance panels to collect previously undetected information on infectious diseases via social networks of participants.
Methods. In 2014, volunteers from 2 large panels in the Netherlands were invited to complete a survey focusing on symptoms of upper respiratory tract infections and to invite 4 individuals they had met in the preceding 2 weeks to take part in the study. We compared sociodemographic characteristics among panel participants, individuals who volunteered for our survey, and individuals recruited via respondent-driven detection.
Results. Starting from 1015 panel members, the survey spread through all provinces of the Netherlands and all age groups in 83 days. A total of 433 individuals completed the survey via peer recruitment. Participants who reported symptoms were 6.1% (95% confidence interval = 5.4, 6.9) more likely to invite contact persons than were participants who did not report symptoms. Participants with symptoms invited more symptomatic recruits to take part than did participants without symptoms.
Conclusions. Our findings suggest that online respondent-driven detection can enhance identification of symptomatic patients by making use of individuals’ local social networks.
Syndromic surveillance provides information necessary to monitor trends in disease incidence and implement and evaluate response plans.1,2 To date, most efforts have focused on developing systems based on data from inpatient and ambulatory care health records.3 In a majority of high-income countries, including the Netherlands, influenza surveillance is based on a combination of reports of influenza-like illness (ILI) collected by sentinel surveillance clinics and additional microbiological testing of subgroups of symptomatic patients.4 This type of system excludes symptomatic patients who do not visit a general practitioner, and such patients are likely to account for the majority of cases in most influenza outbreaks.5
Many communicable diseases (e.g., influenza, severe acute respiratory syndrome, measles) spread largely between socially connected individuals, such as household members and schoolchildren, and they often occur in clusters.6,7 Therefore, cases of infection are expected to cluster in social networks (i.e., contacts of an infected individual are infected at a level of probability higher than that expected if the distribution was random), and clusters can be detected via local social networks of individuals reporting symptoms.
Increased Internet use facilitated the emergence of participatory surveillance (PS) systems, which enable real-time monitoring of diseases through regular submission of syndromic information by volunteers.8,9 These systems provide information that is not collected in regular surveillance, such as the proportion of symptomatic individuals who actually visit a general practitioner and the proportion who are hospitalized.
To test the feasibility of eliciting information about infections in local networks of symptomatic individuals, we combined a chain recruitment method with existing online PS platforms. Under certain conditions, such a recruitment method permits stepwise and controlled sampling of contacts of contacts, and so forth, in social networks in the general population.10 We asked PS volunteers to complete a questionnaire and to invite their contacts into the study. In this way, we collected data on chains of contacts to analyze whether other symptomatic individuals could be detected via the local social network of symptomatic respondents. Our aims were to determine whether respondents can be recruited via respondent-driven detection, to report on which individuals are reached, and to assess whether there is clustering of symptomatic patients.
METHODS
Between March and June 2014, we invited volunteers from 2 Internet PS panels in the Netherlands to complete Internet-based surveys focusing on upper respiratory tract infections. We asked them, in completing the surveys, to provide information on their symptoms and to invite 4 individuals with whom they had had face-to-face contact in the preceding 2 weeks to participate in our study (Figure 1). Participants could invite contacts via e-mail, by providing their own e-mail address and receiving 4 invitations for forwarding (indirect invitations), or by providing e-mail addresses of contacts who were then invited via the system (direct invitations). They could also invite friends via a private Facebook message. Individuals were able to opt out and provide reasons for not participating.
We use panel A to refer to a PS system collecting ILI data during the winter seasons.11 Each week, registered volunteers were reminded via an electronic newsletter to report any symptoms they had experienced since their last log-in. Panel A’s 12 957 active volunteers in the Netherlands were invited to participate in the respondent-driven detection survey, and repeated requests (a total of 3) asking them to do so were placed in the weekly newsletter they received.12 Panel B refers to a comparable system collecting data on pneumonia; the 1691 volunteers in this panel were first invited to participate in the respondent-driven detection survey with a bulletin and then were reminded once as part of their regular newsletter.13 The majority of panel A members were healthy volunteers of various backgrounds, whereas panel B consisted primarily of patients with asthma and chronic obstructive pulmonary disease.
“Seeds” indicate volunteers from the 2 panels, and “recruits” were invited contacts who completed the questionnaire. Waves denote consecutive subsamples (seeds in wave 0, recruits invited by seeds in wave 1, and so forth). Network trees denote chains of connected respondents. We refer to seeds enrolled via panels A and B as the ASeed and BSeed groups, respectively; recruits in consecutive waves as the ARec and BRec groups; and the overall samples as ASeedRec and BSeedRec. After completing their questionnaires, participants were referred to a research Web site displaying the latest results (e.g., anonymous network trees). ASeedRec participants who completed the survey had the opportunity to join a raffle for 1 of 10 gift cards of €25. More details on the survey system are provided in Appendix A (available as a supplement to this article at http://www.ajph.org) and in Stein et al.10
Questionnaire
Participants were asked to provide the number of contacts with whom they had interacted (i.e., with whom they had touched or talked within a distance of about one arm’s length) during 1 full day (“yesterday”). Contacts needed to be specified by age group and location (contact at home, work, or school; at the house of friends or family; and in other places). Participants were asked to report, from a predefined list, any clinical symptoms they had experienced during the preceding 2 weeks (Table A, available as a supplement to this article at http://www.ajph.org). Those with symptoms were asked for day of onset, symptom duration, presumed disease, whether they stayed at home, whether they visited a general practitioner, whether they had used any medicines, and whether they knew any people (not restricted to contacts from the preceding day) who had similar symptoms in the past 2 weeks. For each participant, we also collected information on age, gender, education, postal code, household size (including age groups of household members), influenza vaccination status, work or study location, and contact with groups at high risk for influenza infection during a regular workday. If they so chose, participants could complete questionnaires for their children.
Data Analysis
We excluded participants living outside the Netherlands. We compared the sociodemographic characteristics of PS participants, individuals who volunteered in our survey (seeds), and individuals recruited via respondent-driven detection. Also, we conducted the Pearson χ2 test to assess the samples’ representativeness relative to the general population. Data on the demographic characteristics of the general population were obtained from Statistics Netherlands.14 We computed marginal effects via multilevel logistic regression for the combined samples to assess which sociodemographic characteristics were associated with invitations to contacts (Appendix A). To analyze the geographical spread of recruitment, we plotted seeds and total samples separately at the 4-digit postal code level. We used the great circle distance computation (i.e., the shortest distance between 2 points on the surface of a sphere, measured along the surface of the sphere) to assess the distance participants commuted between home (postal code) and their work or study locations. Geocoding was used to convert entered locations into coordinates.
We defined ILI as a combination of fever and at a least headache or muscle pain and at least a cough or sore throat. This definition was similar to the one used for panel A, although that definition also included sudden onset of symptoms and a fever of at least 38° Celsius.12 A common cold was defined as a runny nose, cough, and sore throat. Participants with ILI or a common cold who reported symptoms that began more than 3 weeks before they participated were excluded. We computed secondary attack rates (i.e., proportion infected among assumed susceptible contacts of an infected participant) in the affected households by assuming that only one household member was a patient with a primary community-acquired case. We analyzed whether symptomatic participants recruited more other symptomatic participants than those who reported no symptoms. We compared symptomatic contacts recruited by symptomatic seeds and symptomatic contacts recruited by asymptomatic seeds with respect to whether they had “at least one symptom” or whether they had a common cold, fever, and ILI. We also compared ILI incidence rates based on regular surveillance by the Dutch Sentinel Network (NIVEL15) with estimates obtained from panel A and our survey population. Statistical analyses were performed in R (version 3.1.1).16
RESULTS
Overall, 1448 individuals (1177 in ASeedRec and 271 in BSeedRec) completed our questionnaire during March to May 2014 (Figure 2; see also Figure A, available as a supplement to this article at http://www.ajph.org). We excluded 288 responses for the following reasons: multiple participation (19), residence outside the Netherlands (115), and incomplete information (154; Appendix A). Our survey was distributed within all provinces in the Netherlands (Table B, available as a supplement to this article at http://www.ajph.org). The age distributions of both panels differed from that of the general population in that they were shifted toward older age groups. Ages varied from 3 to 97 years in ASeedRec and 17 to 82 years in BSeedRec (Figure B and Table C, available as supplements to this article at http://www.ajph.org). The age distributions of ASeedRec and BSeedRec differed significantly (Kolmogorov–Smirnov P < .001). Relative to the general population, women, those with higher levels of education, those with a household composed of 2 adults, and those who had been vaccinated against the flu (self-reported) were strongly overrepresented in ASeedRec, BSeedRec, and panels A and B (Table 1).
TABLE 1—
Panel Ab |
ASeedRec Sample |
Panel B |
BSeedRec Sample |
||||||
Indicator | Dutch Population,a % or Mean (SD) | % or Mean (SD) | P | % or Mean (SD) | P | % or Mean (SD) | P | % or Mean (SD) | P |
Female | 50.5 | 57.4 | < .001 | 66.5 | < .001 | 64.8 | < .001 | 57.6 | .023 |
Age, y | 40.2 (23.0) | 52.0 (16.2) | . . . | 53.0 (14.8) | . . . | 55.0 (13.7) | . . . | 56.8 (12.6) | . . . |
Educational level, bachelor’s degree or higher | 18.9c | 56.1 | < .001 | 58.9 | < .001 | 43.2 | < .001 | 50.6 | < .001 |
Single-member household | 16.7d | 19.5 | < .001 | 26.2e | < .001 | 17.8 | .238 | 20.7e | .095 |
Household with only adults | 25.7d | 42.8 | < .001 | 48.3 | < .001 | 51.3 | < .001 | 59.4 | < .001 |
Household with children | 24.3d | 27.9 | < .001 | 25.6 | .344 | 30.9 | < .001 | 19.9 | .104 |
Daily contact with patients | 12.2 | 9.6 | < .001 | 11.2 | .345 | . . . | . . . | 11.4 | .788 |
Vaccinated against seasonal influenza in past 12 mo | 23.8 | 36.1 | < .001 | 37.9 | < .001 | 48.4 | < .001 | 56.4 | < .001 |
Asthma or chronic obstructive pulmonary disease | 7.4 | 12.4f | < .001 | 3.6g | < .001 | 34.4 | < .001 | 20.7g | < .001 |
Allergy | 8.5 | 55.9 | < .001 | 6.6g | .024 | 32.9h | < .001 | 7.7g | .738 |
Based on data provided by Statistics Netherlands.14
Based on information from Influenzanet.12
With respect to educational level, Statistics Netherlands14 provides data on the population aged 15–64 years.
StatLine provides information only on the number of children in the household, regardless of age.
Forty ASeedRec participants and 6 BSeedRec participants did not provide information on household members and were assumed to live alone.
Also includes panel A volunteers with lung disease.
We asked participants about these risk conditions only if they reported symptoms.
Panel B volunteers were asked about hay fever and allergy.
Participants reporting symptoms were 6.1% (95% confidence interval [CI] = 5.4%, 6.9%) more likely to invite contacts to take part than were participants without symptoms. Those with a bachelor’s degree or higher were 5.2% (95% CI = 4.6%, 5.8%) more likely to invite contacts than those at lower educational levels. Men were 8.6% (95% CI = 7.6%, 9.6%) less likely than women to invite contacts, and participants recruited via panel B were 6.4% (95% CI = 5.7%, 7.2%) less likely to do so than those recruited via panel A. The latter difference might have been due to the incentive offered to panel A participants. Participants’ age did not seem to influence their probability of inviting contacts to take part.
Overall, 171 panel volunteers and invited contacts opted out via the link in the invitation or the survey Web site; 109 of these individuals provided one or more reasons, of whom 88.3% indicated that they did not want to invite or provide information about their contacts. Five (4.6%) individuals indicated that they had participated before and had received an invitation from another person in the same network tree.
Recruitment via Panel A
A total of 792 seeds were enrolled via panel A (the ASeed group; 6.1% of panel A). Overall, 385 recruits completed the questionnaire in 165 network trees (the ARec group); 30.9% of these trees had 2 or more waves, and one of the trees reached 6 waves. On average, ARec recruits invited more contacts than individuals in ASeed (Table D, available as a supplement to this article at http://www.ajph.org). In ASeedRec, a total of 1802 invitations were sent out via the recruitment page (Table E, available as a supplement to the online version of this article at http://www.ajph.org). Because all panel volunteers were invited in batches spread over 1 week, we do not know exactly how fast volunteers responded to invitations; however, 443 (55.9%) seeds participated within the first week after the initial invitation was sent. Recruits in ARec responded, on average, within 2.1 days (SD = 4.1); 43.1% did so on the day of the invitation.
Seeds lived in a total of 636 (15.7%) of a possible 4061 postal code areas; we reached another 140 (3.4%) areas through recruitment (Figure C, available as a supplement to the online version of this article at http://www.ajph.org). In the ARec group, 141 (36.6%) members had the same postal code as their recruiter. In ASeedRec, 800 (68.0%) participants provided a school or work location outside their home. On average, these participants commuted within a radius of 12.2 kilometers (range = 0.03–179.8); 73.3% commuted within a 15-kilometer radius, and 86.0% commuted within their province of residence.
Panel A’s mean age of 52.0 years was higher than that of the general population; consequently, ASeed members also had a high mean age of 54.9 years. The mean age of the ARec group was 49.0 years, indicating ASeed members’ recruitment of individuals from a younger age group. Related to the high mean age was the overrepresentation in panel A and ASeedRec of households with only 2 adults (ARec contained 9.4% more individuals with a household of 4 or more members than ASeed). The overrepresentation of women and highly educated individuals in ASeed relative to the general population decreased through peer recruitment, with 3.5% and 7.2% in ARec, respectively (Table D).
In ASeedRec, 565 (48.0%) participants reported at least one symptom; 59.5% of these participants reported that they knew at least one contact person with similar symptoms. Symptomatic participants mostly identified similar symptoms among household members, family members, and colleagues (Figure 3a). Headache, muscle pain, and common cold symptoms were most frequently identified among contacts (Figure D, available as a supplement to this article at http://www.ajph.org). ASeedRec had crude attack rates of 7.5% for common colds and 2.5% for ILI. We estimated corresponding secondary attack rates of 29.7% and 43.2% (Table F, available as a supplement to the online version of this article at http://www.ajph.org).
Overall, 17% (P = .001) more contacts with at least one symptom were recruited by ASeed members with symptoms than by members without symptoms (Table 2). Similar results (although less strong) were found for common colds (10.9%; P = .061) and fever (14.3%; P = .015). There were 28 ASeed members with ILI, and only one contact with ILI was recruited by a seed without ILI. As a result of the participation of a relatively high proportion of panel A volunteers with ILI in ASeed, the observed proportion of symptomatic individuals in ASeedRec in March 2014 was higher than the proportion in panel A by a factor of 10.4 and higher than the proportion in NIVEL by a factor of 68.8.17
TABLE 2—
Seedsa (A) |
Recruits by Seeds With Symptoms (B) |
Recruits by Seeds Without Symptoms (C) |
Pb |
||||||
Symptom Category, No. (%) | Seeds With Symptoms | Seeds Without Symptoms | Recruits With Symptoms | Recruits Without Symptoms | Recruits With Symptoms | Recruits Without Symptoms | A vs B | A vs C | B vs C |
One or more symptoms | 71 (43.0) | 94 (57.0) | 109 (69.0) | 49 (31.0) | 118 (52.0) | 109 (48.0) | < .001 | .099 | .001 |
Common cold symptoms | 16 (9.7) | 149 (90.3) | 4 (16.7) | 20 (83.3) | 21 (5.8) | 340 (94.2) | .292 | .14 | .061 |
Fever | 12 (7.3) | 153 (92.7) | 4 (18.2) | 18 (81.8) | 14 (3.9) | 349 (96.1) | .101 | .143 | .015 |
Influenza-like illness | 8 (4.8) | 157 (95.2) | 0 (0.0) | 12 (100.0) | 1 (0.3) | 372 (99.7) | > .999 | < .001 | > .999 |
Only successful seeds (n = 165) were considered in this analysis (seeds who invited a recruit who also completed the survey). Recruits in waves 1–6 were lumped together (n = 385).
We used the 2-sample χ2 test for equality of proportions with continuity correction to estimate P values. The Fisher exact test was used for contingency tables containing small values (n < 10).
Among the 29 ILI patients, 23 reported in total 73 contacts with similar symptoms. Participants in ASeedRec reported a total of 22 204 contacts (mean = 18.9 contacts, median = 10.0, SD = 27.2). By taking participants’ number of contacts and contacts with similar symptoms into account, we computed the proportions of ASeedRec members and reported contacts with symptoms during the month of March; these proportions were higher by a factor 1.8 relative to panel A and by a factor of 11.8 relative to NIVEL (Figure 3b).
Recruitment via Panel B
A total of 223 seeds were enrolled via panel B (the BSeed group; 13.2% of panel B). In total, 48 recruits completed the questionnaire in 29 network trees (the BRec group); 17.2% of these trees reached 2 waves. On average, recruits in BRec invited more contacts than seeds. A total of 340 invitations were sent in BSeedRec (Table E). The majority of BSeed members (95.5%) participated in the first week after receiving the initial invitation. BRec members responded on average within 2.6 days (SD = 5.3); 41.7% responded on the day of the invitation.
BSeedRec covered 233 (5.7%) postal code areas, indicating that almost every participant lived in a different area (Figure C). Twenty-two BRec recruits (45.8%) had the same postal code as their recruiter. In BSeedRec, 115 (42.4%) participants provided a study or work location outside their place of residence. On average, these participants commuted within a 11.4-kilometer radius (range = 0.1–83.3); 79.1% commuted within a 15-kilometer radius, and 81.7% commuted within their province of residence.
Relative to the general population, women were slightly overrepresented in the BSeed group (56.5%), and, as a result of the recruitment of predominantly women, this percentage increased to 62.5% in BRec. The mean age of panel B was relatively high, and thus BSeed also had a high mean age of 57.8 years. The mean age of BRec was 52.4 years, indicating BSeed’s recruitment from a younger age group. Although the mean household size of BSeed was similar to that of the general population, the mean size increased in BRec because a few recruits had relatively large households (Table D).
In BSeedRec, 161 (59.4%) participants reported at least one symptom, and 55.3% of these participants indicated at least one contact person with similar symptoms. Symptoms were mostly identified among household members, family members, colleagues, and school classmates. Headache, muscle pain, and common cold symptoms were again most frequently identified among contacts, although these results were less clear than with the ASeedRec sample owing to the small numbers reported (Figure D). BSeedRec had overall attack rates of 8.9% for common colds and 1.1% for ILI (Table F). The estimated secondary attack rate for common colds was 36.8%. Seeds with ILI did not report any household members with similar symptoms, and they did not recruit any contacts (Table G, available as a supplement to this article at http://www.ajph.org).
DISCUSSION
PS systems collect data in real time and include infected individuals who do not seek health care.9 Here we have described, for the first time (to our knowledge), the feasibility of sampling volunteers via PS for an online respondent-driven survey. Beginning with PS volunteers, our survey was distributed via peer-driven recruitment in several waves through all Dutch provinces and reached, within a short period, individuals from all age groups, those with a wide range of household compositions, and those at a variety of educational levels. Neither PS panel was representative of the general population in terms of basic demographic characteristics; after recruitment, however, the representativeness of the overall sample in terms of age and gender improved slightly.
Combining online communities with respondent-driven detection might enhance the identification of symptomatic patients not detected via conventional surveillance systems. Such information, combined with data derived from regular surveillance, can improve estimations of severity indices (e.g., probability of hospitalization after development of symptoms), especially for infectious disease outbreaks in which the majority of symptomatic patients do not seek health care.18–20 Through respondent-driven detection, we increased the geographical coverage of our ASeed group, in that recruits mostly resided in regions other than those of recruiters. Seeds with symptoms recruited more symptomatic contacts than asymptomatic seeds, at least with regard to experiencing general symptoms, common colds, and fever. Symptomatic participants mostly reported similar symptoms among their close contacts. This might have been due to higher transmission rates among close contacts but also higher recall rates with respect to contacts seen most often.
The findings just described indicate that recruitment of peers by symptomatic participants led to higher rates of detection of other symptomatic patients. This is supported by the fact that symptomatic participants seemed more likely to invite contacts than participants who did not report symptoms. Possibly, recent experiences of symptoms motivate individuals to recruit others; similarly, it has been observed that PS participation rates are related to illness status.8 The majority of the volunteers responded within 1 week after being invited, and many recruits responded the same day they were invited, suggesting that information on symptoms and behaviors can be quickly and efficiently obtained.
According to different criteria, the participation rate of panel volunteers can be judged as either low (considering that 90.7% of panel A members reported information on their health status 3 or more times) or high (considering that no massive communication campaign was implemented). The invitation of panel B members via a special bulletin and the difference in target groups (e.g., volunteers in panel B had more chronic health conditions) could explain the differences in participation rates between the panels.
Volunteers who enrolled were mostly women and were more highly educated than the overall population; overrepresentation of those groups relative to the general population is common in PS systems.8,9 Although we provided participants with the option of completing the survey for their children, only a few did so, and children were underrepresented in our samples. In addition, elderly individuals were overrepresented in our samples and PS panels. This high-risk group receives yearly notifications to obtain a flu vaccination and might be more interested than younger groups in influenza-related topics. This, in combination with our samples’ relatively high mean ages, might also explain the overrepresentation of vaccinated individuals and households with only adults.
Limitations
Our study involved limitations. Less than half of all seeds in the 2 samples invited a contact person, a proportion not sufficient to generate long recruitment chains. Also, this percentage was lower than during our earlier research, in which seeds were first contacted personally.21 Although, similar to earlier online respondent-driven surveys,22 we used an incentive in ASeedRec, only a slightly higher recruitment rate was observed in this group than in BSeedRec. Concerns about privacy or not wanting to bother acquaintances with a questionnaire were reported and withheld some participants from sending invitations to contacts. Even though the majority of Internet users share information with each other via social media,23 sending survey invitations specifically to a few contacts is a step many participants did not want to take.
Previously, overall attack rates of 2.5% (95% CI = 2.1%, 3.2%; based on NIVEL5) and 29.2% (95% CI = 21.6%, 37.9%; based on PS24) were estimated for a typical Dutch influenza season. In ASeedRec, only one recruit reported ILI, and a crude attack rate of 2.5% was observed. Generalization of sample estimates to the general population requires weighting (e.g., for age) to enable a proper comparison with NIVEL data. The proportion of ILI in our sample increased as a result of the participation of a select group of PS volunteers with ILI symptoms. Although we were unable to determine the true enhancement factor in the proportion of symptomatic ILI patients via respondent-driven detection, we did observe a slight enhancement in case detection relative to ILI surveillance when participants’ numbers of contacts and contacts with similar symptoms were taken into account.
The probability of identification of disease through respondent-driven detection depends on numerous factors, especially the type of disease being assessed, the incidence of the disease, and methodological aspects such as recruitment of and by symptomatic seeds. Our survey was launched after the peak period of a relatively mild influenza season and during an interval in which the number of active panel A volunteers was declining.12 Only 8 of the 28 seeds with ILI actually invited a recruit, and only 12 recruits completed the questionnaire. Motivation and participation might be much higher during the increasing phase of an influenza season and with other perceived threats of emerging infections.
Conclusions
Our findings in this novel combination of respondent-driven detection with a large PS system provide insights into which groups are reached and indicate that an increased number of symptomatic patients can be detected when the underlying connections in a local social network are used. Although online peer recruitment involves challenges, we demonstrated that respondent-driven detection through PS with large geographical coverage is possible and that timely, detailed information about respondents and their contacts can be obtained. Repeating this type of study during the epidemic peak of a more severe influenza season will provide more information on the extent to which respondent-driven detection enhances disease surveillance.
Acknowledgments
This study was conducted within the Utrecht Center for Infection Dynamics. The Swedish Research Council financed the programming of the online respondent-driven survey system.
We are grateful to Martin Camitz, Antwan Wiersma, and Ronald Smallenburg for their help with the survey launches.
Note. The funders had no role in the study design, data collection and analysis, or the decision to prepare or publish the article.
Human Participant Protection
This study was approved by the Medical Ethical Committee of the University Medical Center Utrecht. Participants provided informed consent before completing the survey.
References
- 1.Henning KJ. What is syndromic surveillance? MMWR Morb Mortal Wkly Rep. 2004;53(suppl):5–11. [PubMed] [Google Scholar]
- 2.van den Wijngaard CC, van Pelt W, Nagelkerke NJ, Kretzschmar M, Koopmans MP. Evaluation of syndromic surveillance in the Netherlands: its added value and recommendations for implementation. Euro Surveill. 2011;16:9. [PubMed] [Google Scholar]
- 3.Mandl KD, Overhage JM, Wagner MM et al. Implementing syndromic surveillance: a practical guide informed by the early experience. J Am Med Inform Assoc. 2004;11(2):141–150. doi: 10.1197/jamia.M1356. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Fourquet F, Drucker J. Communicable disease surveillance: the sentinel network. Lancet. 1997;349:794–795. [Google Scholar]
- 5.McDonald SA, Presanis AM, De Angelis D et al. An evidence synthesis approach to estimating the incidence of seasonal influenza in the Netherlands. Influenza Other Respir Viruses. 2014;8(1):33–41. doi: 10.1111/irv.12201. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Mossong J, Hens N, Jit M et al. Social contacts and mixing patterns relevant to the spread of infectious diseases. PLoS Med. 2008;5(3):e74. doi: 10.1371/journal.pmed.0050074. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Christakis NA, Fowler JH. Social network sensors for early detection of contagious outbreaks. PLoS One. 2010;5(9):e12948. doi: 10.1371/journal.pone.0012948. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Wojcik OP, Brownstein JS, Chunara R, Johansson MA. Public health for the people: participatory infectious disease surveillance in the digital age. Emerg Themes Epidemiol. 2014;11:7. doi: 10.1186/1742-7622-11-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Paolotti D, Carnahan A, Colizza V et al. Web-based participatory surveillance of infectious diseases: the Influenzanet participatory surveillance experience. Clin Microbiol Infect. 2014;20(1):17–21. doi: 10.1111/1469-0691.12477. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Stein ML, van Steenbergen JE, Chanyasanha C et al. Online respondent-driven sampling for studying contact patterns relevant for the spread of close-contact pathogens: a pilot study in Thailand. PLoS One. 2014;9(1):e85256. doi: 10.1371/journal.pone.0085256. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Marquet RL, Bartelds AI, van Noort SP et al. Internet-based monitoring of influenza-like illness (ILI) in the general population of the Netherlands during the 2003–2004 influenza season. BMC Public Health. 2006;6:242. doi: 10.1186/1471-2458-6-242. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Influenzanet. Influenzanet home page. Available at: https://www.influenzanet.eu. Accessed April 23, 2015.
- 13.De GroteLongontstekingMeting. Science in action. Available at: http://www.degrotelongontstekingmeting.nl. Accessed April 23, 2015.
- 14.Statistics Netherlands. StatLine. Available at: http://statline.cbs.nl/Statweb. Accessed April 23, 2015.
- 15.Netherlands Institute for Health Services Research. Primary care database. Available at: http://www.nivel.nl/en/nivel-primary-care-database-Sentinel-Practices%20. Accessed April 23, 2015.
- 16.R Core Team. R: a language and environment for statistical computing. Available at: http://web.mit.edu/r_v3.0.1/fullrefman.pdf. Accessed April 23, 2015.
- 17.National Institute for Public Health and the Environment. Jaarrapportage surveillance respiratoire infectieziekten 2013. Available at: http://www.rivm.nl/Documenten_en_publicaties/Wetenschappelijk/Rapporten/2014/september/Jaarrapportage_Surveillance_Respiratoire_Infectieziekten_2013. Accessed April 23, 2015.
- 18.Barboza P, Vaillant L, Le Strat Y et al. Factors influencing performance of Internet-based biosurveillance systems used in epidemic intelligence for early detection of infectious diseases outbreaks. PLoS One. 2014;9(3):e90536. doi: 10.1371/journal.pone.0090536. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Brownstein JS, Freifeld CC, Madoff LC. Digital disease detection—harnessing the Web for public health surveillance. N Engl J Med. 2009;360(21):2153–2155. doi: 10.1056/NEJMp0900702. 2157. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Anema A, Kluberg S, Wilson K et al. Digital surveillance for enhanced detection and response to outbreaks. Lancet Infect Dis. 2014;14(11):1035–1037. doi: 10.1016/S1473-3099(14)70953-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Stein ML, van Steenbergen JE, Buskens V et al. Comparison of contact patterns relevant for transmission of respiratory pathogens in Thailand and the Netherlands using respondent-driven sampling. PLoS One. 2014;9(11):e113711. doi: 10.1371/journal.pone.0113711. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Bengtsson L, Lu X, Nguyen QC et al. Implementation of Web-based respondent-driven sampling among men who have sex with men in Vietnam. PLoS One. 2012;7(11):e49417. doi: 10.1371/journal.pone.0049417. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Lehmann BA, Ruiter RA, Kok G. A qualitative study of the coverage of influenza vaccination on Dutch news sites and social media websites. BMC Public Health. 2013;13:547. doi: 10.1186/1471-2458-13-547. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Patterson-Lomba O, Van Noort S, Cowling BJ et al. Utilizing syndromic surveillance data for estimating levels of influenza circulation. Am J Epidemiol. 2014;179(11):1394–1401. doi: 10.1093/aje/kwu061. [DOI] [PMC free article] [PubMed] [Google Scholar]