Abstract
Objectives
We aimed to describe the online search trends of Zika and examine their association with Zika incidence, assess the content of Zika-related press releases issued by leading health authorities and examine the association between online trends and press release timing.
Design
Using Google Trends, the 1 May 2015 to 30 May 2016 online trends of Zika and associated search terms were studied globally and in the five countries with the highest numbers of suspected cases. Correlations were then examined between online trends and Zika incidence in these countries. All Zika-related press releases issued by WHO/Pan America Health Organization (PAHO) and Centers for Disease Control and Prevention (CDC) during the study period were assessed for transparency, uncertainty and audience segmentation. Witte's Extended Parallel Process Model was applied to assess self-efficacy, response efficacy, susceptibility and severity. AutoRegressive Integrated Moving Average with an eXogenous predictor variable (ARIMAX) (p,d,q) regression modelling was used to quantify the association between online trends and the timing of press releases.
Results
Globally, Zika online search trends were low until the beginning of 2016, when interest rose steeply. Strong correlations (r=0.748–0.922; p<0.001) were observed between online trends and the number of suspected Zika cases in four of the five countries studied. Compared with press releases issued by WHO/PAHO, CDC press releases were significantly more likely to provide contact details and links to other resources, include figures/graphs, be risk-advisory in nature and be more readable and briefer. ARIMAX modelling results indicate that online trends preceded by 1 week press releases by WHO (stationary-R2=0.345; p<0.001) and CDC (stationary-R2=0.318; p=0.014).
Conclusions
These results suggest that online trends can aid in pandemic surveillance. Identification of shortcomings in the content and timing of Zika press releases can help guide health communication efforts in the current pandemic and future public health emergencies.
Keywords: zika virus, pandemic, health communication, online systems, internet, social media, public health surveillance and communicable disease
Key questions.
What is already known about this topic?
The 2015–2016 Zika pandemic was an outbreak that aroused global public concern and media attention.
It was previously shown that online search trends can be used for communicable diseases surveillance and that health communication is critical during outbreaks.
There is however scarcity of published research on the risk communication aspects of the Zika pandemic.
What are the new findings?
This time trend study reports strong correlations between online trends and Zika incidence.
This suggests that online trends can aid in surveillance during Zika and other pandemics.
Shortcomings in the content and timing of Zika press releases were identified.
Recommendations for policy
Gaps identified in Zika pandemic press-releases and their reactive pattern with online trends can direct better communication efforts globally in order to better tackle Zika and future public health emergencies. This study also suggests that the monitoring of online trends can be used to complement traditional surveillance efforts.
Background
The 2015–2016 Zika pandemic was declared a Public Health Emergency of International Concern (PHEIC) by WHO on 1 February 2016.1 The declaration was a major communication event that attracted wide coverage and attention in the global media as this was only the fourth time a PHEIC has been declared by WHO under the International Health Regulations (IHR) 2005.2 The declaration also came at a time when the Ebola virus outbreak in Africa was coming under control after a perceived delayed response from WHO3 and the international community, and the 2016 Olympic games were scheduled to be played in Brazil.4 5
On 18 November 2016, WHO declared an end to the PHEIC. The statement communicating this decision by the Emergency Committee on Zika and Microcephaly stated that disease must be addressed as a long-term problem which required a ‘longer-term technical mechanism’.6 This declared end of the PHEIC was met with criticism by some stakeholders who feared the crisis is not yet over and this declaration may lead to a pullback in commitment and efforts to develop disease controlmethods by government and donors.7
As of 30 November 2016, 75 countries and territories have reported evidence of vectorborne Zika virus transmission since 2007—with 69 of these countries and territories reporting first-time transmission since 2015.8 About one-third of these countries have also reported increased incidence of Microcephaly and Guillain-Barré syndrome, seemingly related with the Zika virus infection.8 Over 173 000 Zika cases and 15 Zika-related deaths have been confirmed in the Pan America Health Organization (PAHO) region, from among 518 290 suspected cases.9 With the recent spread of the disease to Asia, >2 billion more people have been put at risk.10
Zika virus, a Flavivirus transmitted by the Aedes mosquitos,11 12 was first identified in rhesus monkeys in Uganda in 1947 and human infection was first reported in 1952 in Uganda.13 The clinical symptoms of the disease are usually mild with very few deaths occurring as a direct result of the disease.11 12 There are, however, serious concerns regarding the association of the disease with congenital malformations (Zika Virus Congenital Syndrome) and neurological disorders such as Guillain-Barré syndrome8 11 14
Risk research scholars have noted the challenge posed by risk uncertainty, such as the uncertainty regarding risk management and assessment that may occur during an outbreak situation.15 16 The risk uncertainty in the Zika outbreak is evident in the debate about the 2016 Brazil Olympics.17 18 The risk uncertainty is also evident in the unprecedented health policies and recommendations on reproductive health by affected countries. An example of which is the advice by the government of El Salvador (January 2016) for women to avoid getting pregnant during the coming two years.19
Health risk communication can be said to deal with the planned and unplanned communication to the public about the nature, impact and management of health threats.20 Under the IHR (2005), WHO member nations are obliged to notify the WHO about qualifying health events within 24 hours.21 The IHR framework was established to promote the dissemination of authoritative information which is of particular importance in outbreak circumstances when field conditions are often fluid, and timely and appropriate risk communication can save lives and allay public anxiety.22–24
Timely and effective communication with the public during an emerging infectious disease (EID) outbreak is critical not only for the rapid control of the outbreak but also for reducing the social, political and economic turbulence that often attend such events. The success of communication efforts by health authorities during an outbreak to mitigate adverse outcomes is contingent on various factors. WHO recommends five elements, referred to as the ‘TOTAL’ criteria,25 as key for the success (or failure) of outbreak communication: Trust-building, Operational (and advanced) planning, Transparency, Announcing early and Listening. The risk communication approach indicates that during an EID outbreak public engagement is imperative and stresses the importance of building trust under the unique prevailing conditions.26 The ability of the target audience to understand the information conveyed in outbreak communication material (such as press releases) is necessarily another important factor to consider, particularly when health literacy levels are low.
The internet has become a major source of health information for people worldwide and is a global platform for outbreak and health risk communication. Online trends (ie, the distribution of online behaviour and interactions) have been shown to be an important health surveillance tool for the detection and real-time monitoring of outbreaks.27–29 For example, Google Flu Trends (an online tool based on searches for influenza-related topics) detected influenza outbreaks in the USA 7–10 days before conventional surveillance systems.30 Online trends are also useful to assess shifts in public practice or opinion and the effects of policy.3 31 32
The current Zika pandemic is an emerging global health crisis with high levels of risk uncertainty. The online trends of this pandemic and their association with incidence are unknown. There is also no information about the impact, if any, of press releases issued by leading health authorities on these online trends, or conversely, if these trends have any influence on the timing of the press releases.
Study aim and objectives
With a goal to assess and improve digital and non-digital health communication during pandemics, the objectives of the present study were (1) to describe the online search trends of Zika virus between 1 May 2015 and 30 May 2016 globally, in the USA and in the five countries with the highest number of suspected cases; (2) to examine the association between Zika virus online search trends and the number of suspected Zika cases in the selected countries; (3) to assess and compare the content of Zika-related press releases issued by WHO/PAHO and Centers for Disease Control and Prevention (CDC); and (4) to examine the association between the timing of the press releases of WHO and CDC with the Zika-related online search volume globally and in the USA.
Methods
Study design and time frame
This time trend study used quantitative methods to explore associations between online trends, suspected cases of Zika and the timing, content and strategy of press releases between 1 May 2015 and 31 May 2016 globally and in selected countries in the PAHO region. This translated to a study period between epidemiological week 17, 2015, through epidemiological week 22, 2016. We selected epi-week 17 as the starting point of the study to encompass the first report of locally acquired Zika disease in the Americas (by Brazil) on 7 May 2015 which is regarded as the start of the pandemic.33 We did not have access to incidence data by countries beyond 31 May 2016.
Zika cases data
We downloaded data from the PAHO website on reported suspected Zika cases in PAHO-region countries (on 26 July 2016).34 These data were collated by PAHO from IHR National Focal Points reports to the WHO IHR Regional Contact Point for the Americas, and through Ministry of Health websites of the PAHO member nations. The data are available according to epi-weeks. Some countries do not monitor or report suspected cases. Guatemala, for example, reported 1415 laboratory-confirmed cases in the study period, but did not report the number of suspected cases. Using this database, we identified the five countries with the highest number of suspected cases in the study period—Brazil, Colombia, Honduras, Martinique and El-Salvador, and account for nearly 90% of suspected cases in the study period. These countries were selected for comparison of their Zika-related search terms obtained from Google Trends. Google Trends is able to compare a maximum of five countries simultaneously. Using the reported suspected Zika cases, the incidence of suspected cases was calculated using the 2015 population estimates of the reporting countries. Population data for all countries, apart from those of Saint Martin, Saint Barthelemy and Bonaire, were obtained from the PAHO Health Indicators database.35 The 2015 populations of Saint Martin and Saint Barthelemy were obtained from the Central Intelligence Agency (CIA) World Fact Book,36 and the denominator for Bonaire is based on the 2013 estimate of the Dutch Central Bureau of Statistics.37
Google Trends
Google Trends data were used as the measure of online search trends in this study and were accessed in conformity with suggested guidelines.38 Google Trends is a free online tool that can be used to study search data obtained from google.com—the most widely used search portal in the world.39 Search data for a given query are expressed as the relative search volume (RSV)—the frequency a particular search term is ‘googled’ relative to the total search volume. The RSV for a particular query (e.g., Zika virus) is obtained from a sample of all Google searches performed in a specified location and time and expressed as a normalised number reflecting the relative popularity of the query term compared with all other searches performed in that location and time. The peak popularity of a query is expressed as RSV=100, where 50 implies the term is half as popular as the peak popularity, and 0 means the query was <1% as popular as the peak.38 40
On 18 June 2016, we queried ‘all categories’, ‘web search’ search data using the ‘Topic’ feature for [Zika virus (Organism classification)] and the following four associated search topics: [‘Microcephaly (Medical condition)”, ‘Dengue fever (Disease)”, ‘Chikungunya (Disease cause)”, and ‘Aedes (Insect)”]. These associated search terms were purposively chosen from the list of breakout topics and terms associated with Zika search globally. The ‘Topic search’ feature was preferred to using the ‘search term’ method because this new feature produces search results for groups of terms that are conceptually similar in all languages. The Topic feature was accessed by typing the search term in the search box and ‘Topic’ was selected. For example, Zika was typed into the search box following which the topic search term [‘Zika Virus (Organism classification)”] was selected. These steps were repeated for the associated search topics sequentially in the available five comparative search boxes on the portal. For these search terms, global data as well as data for the USA and the five countries with the highest number of suspected Zika cases in the study period were downloaded. Our search period was set as 26 April 2015 to 4 June 2016, so as to align the search parameters with CDC epi-weeks for the above-defined study period of 1 May 2015 to 31 May 2016.
Press releases
We identified Zika-related press releases published by WHO/PAHO (n=28; 11 from WHO and 17 from PAHO) and CDC (n=27) on their respective websites, during the study period of 1 May 2015 to 31 May 2016. One press release of WHO was duplicated on PAHO's website and thus was included only once to give a total of 27 releases analysed. One PAHO press release was in Spanish and hence excluded from the Flesch-Kincaid analysis (see below) with all other analysis measures carried out on a translated-back-translated version. We initially identified also press releases published by the Ministry of Health Brazil (MHB) (n=45) but found that it was not feasible to properly analyse these Portuguese-language press releases. With the TOTAL23 criteria in mind, we developed a protocol to guide the press release coding. The protocol was pretested and revised accordingly to ensure precise operational definitions of the variables. Training and quality assurance sessions were held for the coders after which 5% of the releases from one of the agencies were independently coded by authors HL, YN and GA. Results were compared and discussed to address areas of disagreement following which all releases were coded by GA.
The press releases were analysed for content and strategy. Content constructs included transparency (title, details of outbreak, consultation with stakeholders, signature, contact details and links to other resources), uncertainty (expression of any uncertainty), segmentation (targeting of information to different people groups)26 41 and if the press release title stated that the release was advisory in nature. Strategy of the release was assessed using Witte's Extended Parallel Process Model (EPPM) constructs of self-efficacy, response-efficacy, susceptibility and severity.42 The EPPM is a sophisticated framework for examining the effectiveness of fear appeal messages and predicts how people will react when confronted with fear-inducing stimuli or information.43 The model comprises four constructs which can be grouped into two efficacy constructs (self-efficacy and response-efficacy), and two threat constructs (susceptibility and severity). According to the EPPM, a positive self-protective response or behaviour can be expected when perceptions of a threat in a message are strong and perceived levels of efficacy are high; when perceptions of a threat are strong but perceived levels of efficacy are low, the model predicts denial or rejection of protection behaviours. All parameters were scored on a dichotomous scale—present or absent.
To provide further insight and information on the press releases, the following measures of analysis were also conducted: Flesch-Kincaid44 grade level and reading-ease tests (continuous scale), word count (continuous scale) and presence/absence of figures or graphs. The Flesch-Kincaid is a commonly used measure to assess the degree of difficulty/ease to understand a given English-language passage or document.44
Statistical analysis
The correlation between Zika RSVs (in epi-weeks) and number of reported suspected cases was examined using the non-parametric Spearman's rank correlation test. Correlation was assessed from the epi-week during which the first suspected Zika case was reported. Flesch-Kincaid grade level and reading-ease tests of the press releases were assessed using Microsoft Word 2007 readability statistics. We conducted χ2 tests for independency to check for differences between WHO/PAHO and CDC press releases. Time-series regression models were developed using non-seasonal AutoRegressive Integrated Moving Average with an eXogenous predictor variable (ARIMAX (p,d,q)) to examine the behaviour of RSVs over time globally in the USA and in Brazil and to test if press releases by WHO, CDC and the MHB can explain this behaviour, respectively. These ‘independent variables’ were treated as ‘events’ in the model, in which a value of 1 indicated that one or more press releases were issued in that week, and a value of 0 indicated otherwise. An ARIMAX model is used to check and predict the dependence of values in a time series with equal periods using the past values observed in the series.45 Statistical significance was set at 0.05. The software used for analysis was SPSS V.20.
Results
Online trends
Globally, there was little public interest (as measured by RSV) in Zika until epi-week 1, 2016, when interest rose steeply to peak in epi-week 5, 2016 (figure 1A). Trends in interest in microcephaly tracks with Zika interest, although on a much lower level. Looking at the overall study period, the average interest in Zika (average RSV=9) was slightly lower than dengue (average RSV=10). There was little to no interest in Aedes. The Zika trend observed in the USA (figure 1B) is similar to the global trend. There was, however, no observable interest in dengue, chikungunya or Aedes.
Figure 1.
Zika-related online trends globally (1a), in USA (1b), Brazil (1c), Colombia (1d), Martinique (1e), Honduras (1f) and Elsavador (1g) —epi-week 17, 2015, to epi-week 22, 2016. RSV, relative search volume.
Two Zika search peaks were observed in Brazil throughout the entire study period—epi-week 49, 2015, and epi-weeks 5–7, 2016 (figure 1C). These tracked closely with interest in Microcephaly. Overall interest in Zika (average RSV=15), however, lagged behind interest in dengue (average RSV=31).
In Colombia, there was a brief period of interest in Zika in epi-weeks 22–24, 2015, and again in epi-week 41, 2015, and peaked in epi-week 3, 2016 (figure 1D). As noted for Brazil, interest in Microcephaly tracked closely with interest in Zika. Interest in chikungunya waned progressively from epi-weeks 17–37, 2015, and then flat-lined from thereon to a baseline of about RSV=10. As observed in all other countries, there was little or no interest in Aedes.
Online search trends in Martinique revealed overwhelming online interest in Zika compared with the associated terms (figure 1E). This interest peaked at epi-week 4, 2016.
The online search trends in Honduras showed little interest in Zika before epi-week 46, 2015, after which it steadily rose to peak in epi-week 5, 2016 (figure 1F). There was a period of sustained interest in chikungunya between epi-week 17, 2015, and epi-week 37, 2015.
In El Salvador (figure 1E), online search trends revealed greater interest for chikungunya (average RSV=32) than for Zika (average RSV=24) and dengue (average RSV=22). As with all other countries reviewed, interest in Microcephaly tracks closely with interest in Zika though at a much lower level.
WHO's PHEIC declaration in epi-week 5, 2015, coincided precisely with the peak interest levels globally, in the USA and in Honduras, and quite closely also in Brazil and the other countries.
Suspected Zika cases and online trends
With 159 939 cases (table 1), Brazil reported the highest number of suspected Zika cases in the PAHO region, with Colombia a distant second with 83 967 cases. Martinique, on the other hand, has the highest suspected cases incidence (7125.6/100 000) followed by French Guyana (2877.9/100 000).
Table 1.
Suspected Zika autochthonous (ie, locally vector transmitted) cases in the Pan America Health Organization (PAHO) region between epidemiological week 17, 2015, and week 22, 2016 (ordered by number of suspected cases)
| Country/territory | Suspected Zika cases* | Confirmed Zika cases* | 2015 population† | Incidence of suspected Zika cases/100 000 | |
| 1. | Brazil | 159 939 | 40 086 | 203 657 000 | 78.53 |
| 2. | Colombia | 83 967 | 8078 | 49 529 000 | 169.53 |
| 3. | Martinique | 28 930 | 12 | 406 000 | 7125.62 |
| 4. | Honduras | 22 715 | 44 | 8 424 000 | 269.65 |
| 5. | El Salvador | 10 476 | 46 | 6 426 000 | 163.03 |
| 6. | Guadalupe | 10 190 | 379 | 470 000 | 2168.09 |
| 7. | Puerto Rico | 9657 | 1352 | 3 680 000 | 262.42 |
| 8. | French Guyana | 7540 | 483 | 262 000 | 2877.86 |
| 9. | Dominican Republic | 3313 | 73 | 10 652 000 | 31.10 |
| 10. | Suriname | 2512 | 697 | 548 000 | 458.39 |
| 11. | Haiti | 2069 | 5 | 10 604 000 | 19.51 |
| 12. | Jamaica | 1619 | 24 | 2 813 000 | 57.55 |
| 13. | Panama | 1022 | 283 | 3 988 000 | 25.63 |
| 14. | Barbados | 770 | 18 | 291 000 | 264.60 |
| 15. | Saint Martin | 690 | 180 | 31 754 | 2172.95 |
| 16. | Costa Rica | 519 | 107 | 5 002 000 | 10.38 |
| 17. | Dominica | 437 | 47 | 74 000 | 590.54 |
| 18. | United States Virgin Islands | 276 | 24 | 104 000 | 265.38 |
| 19. | Paraguay | 275 | 8 | 7 033 000 | 3.91 |
| 20. | Argentina | 68 | 20 | 42 155 000 | 0.16 |
| 21. | Saint Barthelemy | 52 | 14 | 7237 | 718.53 |
| 22. | Saint Lucia | 24 | 4 | 164 000 | 14.63 |
| 23. | Anguilla | 0 | 0 | 16 000 | NA |
| 24. | Aruba | 0 | 17 | 112 000 | NA |
| 25. | Belize | 0 | 2 | 348 000 | NA |
| 26. | Bolivia | 0 | 124 | 11 025 000 | NA |
| 27. | Bonaire | 0 | 3 | 17 408 | NA |
| 28. | Cuba | 0 | 1 | 11 249 000 | NA |
| 29. | Curacao | 0 | 73 | 148 000 | NA |
| 30. | Ecuador | 0 | 244 | 16 226 000 | NA |
| 31. | Granada | 0 | 2 | 111 000 | NA |
| 32. | Guatemala | 0 | 1415 | 16 255 000 | NA |
| 33. | Guyana | 0 | 6 | 808 000 | NA |
| 34. | Mexico | 0 | 357 | 125 236 000 | NA |
| 35. | Nicaragua | 0 | 241 | 6 257 000 | NA |
| 36. | Peru | 0 | 78 | 31 161 000 | NA |
| 37. | Saint Vincent and the Grenadines | 0 | 8 | 103 000 | NA |
| 38. | Sint Maarten | 0 | 7 | 41 000 | NA |
| 39. | Trinidad and Tobago | 0 | 16 | 1 347 000 | NA |
| 40. | Venezuela | 0 | 0 | 31 293 000 | NA |
| 41. | Antigua and Barbuda | NA | NA | 92 000 | NA |
| 42. | Bahamas | NA | NA | 388 000 | NA |
| 43. | Bermuda | NA | NA | 70 000 | NA |
| 44. | British Virgin Islands | NA | NA | 33 000 | NA |
| 45. | Caiman Islands | NA | NA | 56 000 | NA |
| 46. | Canada | NA | NA | 35 871 000 | NA |
| 47. | Chile | NA | NA | 17 924 000 | NA |
| 48. | Montserrat | NA | NA | 5000 | NA |
| 49. | Saint Kitts & Nevis | NA | NA | 52 000 | NA |
| 50. | Turks and Caicos Islands | NA | NA | 50 000 | NA |
| 51. | USA | NA | NA | 325 128 000 | NA |
| 52. | Uruguay | NA | NA | 3 430 000 | NA |
*Suspected and confirmed cases from PAHO.9
†2015 population figures for all countries apart from those of Saint Martin,Saint Barthelemy and Bonaire were obtained from the PAHO Health Indicators database, 2009. http://www.paho.org.35 The 2015 population of Saint Martin and Saint Barthelemy were obtained from the Central Intelligence Agency World Fact Book.36 Population of Bonaire is based on 2013 estimate of the Dutch Central Bureau of Statistics.37
As seen in figure 2A-E strong correlation was noted between online trends and numbers of suspected cases in Brazil, Colombia, Honduras and El Salvador (r=0.748 to 0.922; p<0.001), but not Martinique.
Figure 2.
Correlation between Zika online trends and reported suspected cases in top five countries with highest number of suspected Zika cases. RSV, relative search volume.
Press releases analysis
Summary results of the press releases analysis are presented in table 2. Compared with the WHO/PAHO, CDC press releases were more likely to contain the following constructs: presence of contact details, links to other resources, use of figures/graphs and be of a risk-advisory nature (p<0.05). The Flesch-Kincaid grade-level scores of the health authorities were high with a mean grade-level score of 17.1 years for WHO/PAHO press releases and 12.4 years for CDC press releases. Overall, CDC press releases had significantly lower word counts compared with WHO/PAHO. Press releases issued by WHO/PAHO were more likely to be signed (p<0.05).
Table 2.
Comparison between WHO/Pan America Health Organization (PAHO) and Centers for Disease Control and Prevention (CDC) press releases
| WHO/PAHO (n=27) |
CDC (n=27) |
p-value | Test | |
| % | % | |||
| Content assessment | ||||
| Title | 100.0 | 100.0 | – | – |
| Current details or new information about outbreak | 96.3 | 85.2 | 0.351 | Fisher's |
| Reference to consultation | 96.3 | 88.9 | 0.610 | Fisher's |
| Signed release | 51.9 | 0 | <0.0001 | Pearson's |
| Contact details | 40.7 | 100.0 | <0.0001 | Pearson's |
| Link to other resources | 66.7 | 92.6 | 0.018 | Pearson's |
| Expresses uncertainty | 88.9 | 96.3 | 0.610 | Fisher's |
| Segmentation in release | 66.7 | 77.8 | 0.362 | Pearson's |
| Risk-advisory guidance | 11.1 | 66.7 | <0.0001 | Pearson's |
| Use of figures/graphs | 33.3 | 0 | 0.002 | Fisher's |
| Witte's Extended Parallel Process Model | ||||
| Self-efficacy | 55.6 | 77.8 | 0.083 | Pearson's |
| Response efficacy | 51.9 | 77.8 | 0.046 | Pearson's |
| Susceptibility | 92.6 | 85.2 | 0.669 | Fisher's |
| Severity | 96.3 | 92.6 | 1 | Fisher's |
| Witte efficacy construct | 51.9 | 77.8 | 0.046 | Pearson |
| Witte threat construct | 88.9 | 85.2 | 1 | Fisher's |
| All Witte constructs present | 51.9 | 77.8 | 0.046 | Pearson's |
| Readability | ||||
| Flesch-Kincaid grade level (mean±SD) | 17.06±2.95 | 12.40±3.33 | <0.0001 | Analysis of variance |
| Flesch-Kincaid reading ease (mean±SD) | 21.15±9.72 | 41.07±13.26 | <0.0001 | Analysis of variance |
| Word count (mean±SD) | 681±218.21 | 502±160.24 | 0.001 | Analysis of variance |
Press releases association with online trends
The timing of the press releases by WHO (figure 3A) and CDC (figure 3B) by epi-week revealed that nearly all were issued in 2016.
Figure 3.

Timing of press releases and online trends by epi-week. RSV, relative search volume globally (3a), in the USA (3b) and Brazil (3c).
The time-trend data were best described by an ARIMAX model (0,1,1)—autoregression component=0, integration=1 and moving-average=1. The model parameters and estimates are shown in table 3. The predicted best-fit models suggest that the online search trends of the previous week predict the timing of press releases by WHO (stationary-R2=0.345; p<0.001) and CDC (stationary-R2=0.318; p=0.014 but not by MHB (stationary-R2=0.182; p=0.003).
Table 3.
ARIMAX model for global, US and Brazilian online trends with press releases by relevant health agency as predictors
| ARIMAX model | Predictor | Stationary R2 | Parameters of the model | |||
| Estimate | p Value | |||||
| Global | (0,1,1) | Press releases by WHO | 0.345 | MA (lag 1) | −0.433 | 0.001 |
| Predictor | 13.316 | <0.001 | ||||
| USA | (0,1,1) | Press releases by CDC | 0.318 | MA (lag 1) | −0.727 | <0.001 |
| Predictor | 3.869 | 0.014 | ||||
| Brazil | (0,1,1) | Press releases by MHB not a predictor | 0.182 | MA (lag 1) | −0.391 | 0.003 |
ARIMAX, AutoRegressive Integrated Moving Average with an eXogenous predictor variable; CDC, Centers for Disease Control and Prevention; MHB, Ministry of Health Brazil; WHO, World Health Organization
Discussion
Globally, there was little online interest in Zika, as measured by online search trends, until the beginning of 2016, when interest rose steeply. Expectedly, given the diverse profiles and conditions in different country populations, some cross-country variation is noted in the online trends in this pandemic situation. We found strong correlation between online trend RSVs and the number of suspected Zika cases in four of the five most affected countries. Furthermore, the results of the regression modelling indicate that the issuance of Zika-related press releases by WHO and CDC could be predicted by an increase in online search volume in the week prior to the publication of the press release.
WHO declared Zika as a PHEIC in epi-week 5, 2016. This declaration is a rare and major communication event which attracted widespread coverage in the world media.2 From the coincidence of the timing of the PHEIC declaration and the peak in online interest globally, in the USA and in other countries (figure 3), one may conclude that the Public Health Emergency was declared in response to the steep increase in public interest and discussion about Zika in the preceding weeks. The abrupt drop in interest that immediately followed the declaration may suggest that the statement issued by WHO Director-General Dr Chan, while announcing a global emergency, had the effect of assuaging the public's anxiety about Zika. The declaration may possibly have sent a message to the public that WHO has assumed responsibility and will mobilise the necessary resources to combat the spread of the virus.
Our results also demonstrate that the interest in Microcephaly globally and in all the countries we examined closely mirrored the interest in Zika, although at a much lower level. There was no interest in Microcephaly until epi-week 45, 2015, in any of the countries studied or globally, apart from Martinique, which showed brief interest in Microcephaly already in epi-weeks 22–27, 2015. The upturn in interest in Microcephaly in Brazil coincided with the declaration by the MHB on 11 November 2015 (epi-week 45, 2015) of a national public health emergency due to an increase in suspected Microcephaly cases.13 This surge peaked 4 weeks later in epi-week 4, 2016. As observed in the online trends following WHO's PHEIC declaration, this too suggests that press releases issued by a health agency can influence online trends and information-seeking behaviour of the public. The observed changes in RSV are similar to the findings of a previous study which detected a significant increase in RSV 1–4 weeks after the introduction of a major smoking policy in the Netherlands.32
Across all the territories and countries studied, there was little or no interest in Aedes—the vector responsible for the transmission of the disease. This is an interesting finding given that there is yet no cure for Zika and vector control remains an important component of the control of the disease and is in fact the key message of WHO's Zika risk communication guidelines.46 Furthermore, the Aedes mosquito is also responsible for the transmission of chikungunya and dengue47 which were of public interest in the countries affected by the Zika pandemic as reflected in the online trends for these diseases. This lack of interest in Aedes may suggest that people are interested in the disease and not in the vector. It perhaps shows a deficit in communication by relevant health authorities to the public about this vector that needs to be corrected. The inauguration of Mosquito Awareness weeks across the PAHO region for the Caribbean47 and Spanish-speaking PAHO member nations48 are right steps in this direction and may have been initiated by PAHO in response to the low level of interest by the public in mosquito control.
We also observed differences in the intensity of interest in Zika in comparison with the associated search terms, particularly dengue and chikungunya. While Brazil experienced the highest number of suspected Zika cases, interest in dengue nearly doubled the level of interest in Zika. Likewise in Honduras and El Salvador, chikungunya attracted greater interest than Zika. This finding may be reflecting a reality that having been around longer the burden of these diseases in terms of number of cases is greater than the burden posed by Zika and they are hence more searched for. Alternatively, the public in these countries may perceive chikungunya and dengue as more severe and therefore a greater threat than Zika as alluded to in a press release by the MHB.49 The population of Martinique, with the highest incidence of suspected Zika cases, exhibited overwhelming interest in Zika compared with the other search terms.
These findings suggest that online trends may be a useful, inexpensive and rapid surveillance tool in pandemic situations in areas where internet use is widespread. The weak to moderate correlations reported between online trends and Ebola incidence in the three countries most affected by the West Africa Ebola epidemic in 201439 may have been due to poor internet penetration in the affected countries. In addition, as the internet is a major source of health information, search engines are not only a surveillance tool but also play a major role in health promotion. At times of PHEIC, health agencies such as WHO could work together with companies like Google to promote reliable sources of health information.
People's perception of, and response to risk is influenced largely by the manner in which the risk is described and explained.50 Risk management and risk communication theory emphasise the importance of creating trust between the public and health authorities51 as a key determinant in the public's reception of the risk and the extent to which they are willing to accept and act on official recommendations.26
Even in, or perhaps especially in, our world of modern telecommunication and social media, press releases remain an important tool for communicating information to the public in times of health crises such as the ongoing Zika pandemic. Press releases are the initial, and often the only, source of news for health and medical science journalists,52 and many news organisations reprint health-related and science-related press releases verbatim.53
Our press release content analyses revealed no difference in the expression of uncertainty or segmentation of information between WHO/PAHO and the CDC. CDC press releases though were more likely to provide contact details, links to other resources and to be advisory in nature, elements that have been shown to be important in a crisis risk scenario.41 CDC press releases are also more likely to have all EPPM constructs present, suggesting that they would produce a greater degree of message acceptance and protection motivation.42
Health agencies need to make greater efforts to incorporate these elements in their messages to the public during an outbreak situation, in keeping with suggested risk communication best practices.54 Admittedly, this can be especially challenging in multicultural settings where messages may need to be uniquely tailored for different sectors of the population.
Identification of shortcomings in the content and timing of Zika-related press releases can help guide health communication efforts in the current pandemic and future public health emergencies. The TOTAL criteria suggested by WHO23 may be a good benchmark against which to evaluate press releases and other communication materials, although we are unaware of any study that has done this. In keeping with TOTAL criteria,22 for example, all press releases should reiterate specific steps and behaviours people need to take to mitigate risks, and health communication should ‘announce early’. The current Zika pandemic began in 2015, yet the majority of Zika-related press releases were released in 2016. This communication time lag represents missed opportunities for risk mitigation, infection control and anxiety alleviation. A possible explanation for the lapse in communication may be the fact that the association between Zika and Microcephaly and other neurological conditions did not garner attention until late 2015, and it is the risk of these complications that has evoked the fear most people have about the disease.2 8 19
A Flesch-Kincaid reading-ease score <30 is deemed to be hard-to-read material that can be readily understood only by college students.55 The relatively high Flesch-Kincaid grade-level scores achieved by the Zika-related press releases issued by the health agencies, particularly those issued by WHO/PAHO with an average of 17 years of education needed to understand the press releases, suggest that readability by the public is a matter of concern. We are unaware of any previous study that assessed the readability of press releases issued by health agencies; however, a study on the readability of online health information found the average Flesch-Kincaid reading-ease score to be 46.1—higher than the average of the press releases of all health agencies assessed in the current study.56 Efforts to improve the readability of press releases of health agencies are warranted. Such efforts could include provision of a ‘layman summary’ and the involvement of public representatives in the assessing readability of these documents before general release.57
Limitations of the study
Our measure of online trends was based solely on Google Trends search data and did not include interest data from other search engines (e.g., Bing, Yahoo) or social media platforms such as Facebook or Twitter. We relied on this strategy since Google is by far the most popular search engine worldwide with hundreds of millions of search queries performed daily and accounts for >70% of all online searches performed globally.58 Google Trends is a free resource that has been shown to be a representative measure of online search trends.38 At the same time, Google Trends is based on certain ‘mathematical assumptions and approximations’ that suggest some caution against overinterpretation of RSV data is warranted.59 Furthermore, adding other social media channels may have presented a fuller picture of the online trends of this pandemic.
Another issue that warrants mention is the validity of the reported suspected Zika cases by PAHO member nations. The numerator used in the present study to calculate Zika incidence for each country was the number of reported suspected Zika cases that appeared in the online PAHO database. As seen in table 2, the proportion of suspected cases that are diagnostically confirmed ranges from >25% (Argentina, Suriname, Panama, Saint Martin and Saint Barthelemy) to <1% in countries such as Martinique, Honduras, Haiti and El Salvador. This wide variation may be a result of the differences in capacity of the healthcare delivery systems in these countries or differential diagnostic accuracy. False positive results may be generated from the strong serological cross-reactivity of IgM antibodies between Zika virus and other flaviviruses, while sensitivity may be compromised as antibodiesare often undetectable in serum collected within a week of illness onset.60
Strengths of the study
To the best of our knowledge, this is the first study to assess online trends of the ongoing Zika pandemic (using Google Trends) and their association with Zika incidence in a number of countries, and with the timing of Zika-related press releases published by official health agencies. We are also unaware of any studies that have applied the WHO TOTAL criteria to evaluate health press releases or other communication materials. The use of formal readability measures and an established theoretical framework to evaluate the content and readability of WHO and CDC Zika press releases enhances the ability of generalising the findings to other ongoing health and risk communication efforts.
Conclusion and recommendations
This study provides evidence to support the utility of monitoring online search trends as an additional surveillance tool during a pandemic. Given the use of press releases by many media organisations and their continued availability online, this study reinforces the need for organisations to conduct a dialogue with the public using timely press releases as a tool to impart information and tailor messages for subgroups of the population audience. In addition, health authorities should pay more attention to prompt online communication and consider the readability of published materials to the general public. This builds trust with the public sphere and encourages cooperation. Methods developed and results found in the current study can help guide health communication efforts and research in the current pandemic and future public health emergencies. Further research is needed to identify the most effective communication strategies and messages for different population segments during pandemics and global emergencies.
Footnotes
Contributors: GA designed the study. Bore responsibility for literature search, figures, data collection and analysis, data interpretation, writing and critical revision of the manuscript for important intellectual content.
YN oversaw research. Designed the study. Contributed to literature search, figures, data analysis, data interpretation, writing and critical revision of the manuscript for important intellectual content.
AG-E contributed to study design, data interpretation, writing and critical revision of the manuscript for important intellectual content.
WAA contributed to data analysis, data interpretation and critical revision of the manuscript for important intellectual content.
HL conceived the study. Oversaw research. Designed the study. Contributed to literature search, figures, data analysis, data interpretation, writing and critical revision of the manuscript for important intellectual content.
Competing interests: This research was conducted in the framework of Gbenga Adebayo's International MPH thesis at the Hebrew University-Hadassah Braun School of Public Health and Community Medicine, under the guidance of Dr Hagai Levine and Professor Yehuda Neumark. His studies were supported by a Pears Foundation scholarship.
Provenance and peer review: Not commissioned; externally peer reviewed.
Data sharing statement: Full dataset is available upon request from the corresponding author.
References
- 1. World Health Organization. Zika Outbreak: who's Global Emergency Response Plan [Internet]. 2016. http://www.who.int/emergencies/zika-virus/response/en/ (accessed 26 Apr 2016).
- 2. Belluz J. The WHO just declared a public health emergency related to Zika virus. http://www.vox.com/2016/2/1/10871562/zika-health-emergency-who (accessed 23 Aug 2016).
- 3. Fung IC, Tse ZT, Cheung CN, et al. Ebola and the social media. Lancet 2014;384:2207 10.1016/S0140-6736(14)62418-1 [DOI] [PubMed] [Google Scholar]
- 4. Iheozor-Ejiofor Z, Worthington H V, Walsh T, et al. Water fluoridation for the prevention of dental caries. Cochrane database Syst Rev 2015;6:CD010856. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. World Health Organization. Zika virus and the Olympic and Paralympic Games Rio 2016 [Internet].. 2016. http://who.int/mediacentre/news/statements/2016/zika-olympics/en/ (accessed 17 Jul 2016).
- 6. World Health Organization. Fifth meeting of the Emergency Committee under the International Health Regulations (2005) regarding microcephaly, other neurological disorders and zika virus [Internet]. 2016. http://www.who.int/mediacentre/news/statements/2016/zika-fifth-ec/en/ (accessed 3 Dec 2016).
- 7. McNEIL DGJ. Zika is no longer a global emergency, W.H.O. says. New York Times [Internet].. 2016. http://www.nytimes.com/2016/11/19/health/who-ends-zika-global-health-emergency.html?_r=0
- 8. World Health Organization. Situation Report: zika virus Microcephaly Guillian-Barré syndrome [Internet]. 2016. http://apps.who.int/iris/bitstream/10665/251811/1/zikasitrep1Dec2016-eng.pdf
- 9. Pan American Health Organization. Cumulative zika suspected and confirmed cases reported by countries and territories in the Americas, 2015-2016 [Internet]. 2016. http://www.paho.org/hq/index.php?option=com_docman&task=doc_view&Itemid=270&gid=37143&lang=en (accessed 3 Dec 2016).
- 10. Jozuka E. Singapore's Zika cases send warning signal to Asia [Internet]. 2016. http://edition.cnn.com/2016/09/05/health/zika-asia-threat/ (accessed 20 Sep 2016).
- 11. World Health Organization. WHO Zika virus Fact Sheet [Internet]. 2016. http://www.who.int/mediacentre/factsheets/zika/en/ (accessed 26 Apr 2016).
- 12. Atif M, Azeem M, Sarwar MR, et al. Zika virus disease: a current review of the literature. Infection 2016;44:695–705. 10.1007/s15010-016-0935-6 [DOI] [PubMed] [Google Scholar]
- 13. World Health Organization. The history of Zika virus [Internet]. http://who.int/emergencies/zika-virus/history/en/ (accessed 29 Aug 2016).
- 14. Rasmussen SA, Jamieson DJ, Honein MA, et al. Zika virus and birth defects--reviewing the evidence for Causality. N Engl J Med 2016;374:1981–7. 10.1056/NEJMsr1604338 [DOI] [PubMed] [Google Scholar]
- 15. Nowotny H, Scott P, Gibbons MT. Re-Thinking science: knowledge and the Public in an age of uncertainty: Wiley, 2001. [Google Scholar]
- 16. Löfstedt R. Risk Management in Post-Trust Societies. UK: Palgrave Macmillan, 2005. [Google Scholar]
- 17. Coombes R; head of features and investigations. Call to cancel 2016 Olympics because of Zika risk is not backed by WHO guidance. BMJ 2016;353:i2899 10.1136/bmj.i2899 [DOI] [PubMed] [Google Scholar]
- 18. Anon. Halting the Olympics-Zika virus bandwagon. Lancet 2016;388:437 10.1016/S0140-6736(16)31204-1 [DOI] [PubMed] [Google Scholar]
- 19. Ahmed A. El Salvador's Advice on Zika Virus: Don't Have Babies New York Times [Internet]. 2016. http://www.nytimes.com/2016/01/26/world/americas/el-salvadors-advice-on-zika-dont-have-babies.html?_r=0
- 20. Merrick J, Lemal M. Health Risk Communication. New York: Nova Scieince Publishers, Inc, 2013. [Google Scholar]
- 21. World Health Organization. International Health Regulations [Internet]. 2 edn: World Health Organization, 2005:82 http://www.who.int/ihr/publications/9789241596664/en/ [Google Scholar]
- 22. World Health Organization. Outbreak communication: best practices for communicating with the public during an outbreak - Report of the WHO Expert Consultation on Outbreak Communications held in Singapore, 2004:21–3. http://www.who.int/iris/handle/10665/69138
- 23. World Health Organization. World Health Organization Outbreak Communication Planning Guide [Internet]. 2008:1–19 http://www.who.int/ihr/elibrary/WHOOutbreakCommsPlanngGuide.pdf
- 24. Mondor L, Brownstein JS, Chan E, et al. Timeliness of nongovernmental versus governmental global outbreak communications. Emerg Infect Dis 2012;18:1184–7. 10.3201/eid1807.120249 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Heymann DL, Association APH. Control of communicable diseases manual: American Public Health Association, 2008. [Google Scholar]
- 26. Gesser-Edelsburg A, Mordini E, James JJ, et al. Risk communication recommendations and Implementation during emerging infectious diseases: a case study of the 2009 H1N1 Influenza Pandemic. Disaster Med Public Health Prep 2014;8:158–69. 10.1017/dmp.2014.27 [DOI] [PubMed] [Google Scholar]
- 27. Neumark Y, Flum L, Lopez-Quintero C, et al. Quality of online health information about oral contraceptives from Hebrew-language websites. Isr J Health Policy Res 2012;1:38 10.1186/2045-4015-1-38 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Moss R, Zarebski A, Dawson P, et al. Forecasting influenza outbreak dynamics in Melbourne from Internet search query surveillance data. Influenza Other Respir Viruses 2016;10:314–23. 10.1111/irv.12376 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Jacobsen KH, Aguirre AA, Bailey CL, et al. Lessons from the Ebola Outbreak: action items for emerging infectious disease preparedness and response. Ecohealth 2016;13:200–12. 10.1007/s10393-016-1100-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Carneiro HA, Mylonakis E. Google trends: a web-based tool for real-time surveillance of disease outbreaks. Clin Infect Dis 2009;49:1557–64. 10.1086/630200 [DOI] [PubMed] [Google Scholar]
- 31. Rosenkrantz AB, Prabhu V. Public Interest in Imaging-Based Cancer screening examinations in the United States: analysis using a Web-Based search Tool. AJR Am J Roentgenol 2016;206:113–8. 10.2214/AJR.15.14840 [DOI] [PubMed] [Google Scholar]
- 32. Troelstra SA, Bosdriesz JR, de Boer MR, et al. Effect of tobacco control policies on information seeking for Smoking Cessation in the Netherlands: a Google Trends Study. PLoS One 2016;11:e0148489 10.1371/journal.pone.0148489 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33. PAHO. Epidemiological Alert: zika virus infection [Internet]. 2015. http://www.paho.org/hq/index.php?option=com_docman&task=doc_view&Itemid=270&gid=30075&lang=en
- 34. PAHO. Zika in the Americas Epi Curve [Internet]. 2016. http://ais.paho.org/phip/viz/ed_zika_epicurve.asp (accessed 26 Jul 2016).
- 35. PAHO. PAHO Region Countries Population [Internet]. Health Indicators database 2009. http://ais.paho.org/phip/viz/basicindicatorbrowaser.asp (accessed 3 Dec 2016). [Google Scholar]
- 36. CIA. The World Factbook [Internet]. 2016. https://www.cia.gov/library/publications/resources/the-world-factbook/ (accessed 28 Aug 2016).
- 37. Central Bureau of Statistics. Caribbean Netherlands; population trends, births, deaths, migration [Internet]. 2015. http://statline.cbs.nl/StatWeb/publication/?DM=SLNL&PA=80539ned&D1=0-1,9-10&D2=a&D3=a&HDR=T&STB=G1,G2&CHARTTYPE=1&VW=T (accessed 15 Jul 2016).
- 38. Nuti SV, Wayda B, Ranasinghe I, et al. The use of google trends in health care research: a systematic review. PLoS One 2014;9:e109583 10.1371/journal.pone.0109583 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39. Alicino C, Bragazzi NL, Faccio V, et al. Assessing Ebola-related web search behaviour: insights and implications from an analytical study of Google Trends-based query volumes. Infect Dis Poverty 2015;4:54 10.1186/s40249-015-0090-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40. Google. How trends data is adjusted [Internet]. 2016. https://support.google.com/trends/answer/4365533?hl=en&ref_topic=4365599 (accessed 14 Aug 2016).
- 41. Glik DC. Risk communication for public health emergencies. Annu Rev Public Health 2007;28:33–54. 10.1146/annurev.publhealth.28.021406.144123 [DOI] [PubMed] [Google Scholar]
- 42. Witte K, Allen M. A meta-analysis of fear appeals: implications for effective public health campaigns. Health Educ Behav 2000;27:591–615. 10.1177/109019810002700506 [DOI] [PubMed] [Google Scholar]
- 43. Lavoie NR, Quick BL. What is the truth? an application of the Extended parallel process Model to televised truth® ads. Health Commun 2013;28:53–62. 10.1080/10410236.2012.728467 [DOI] [PubMed] [Google Scholar]
- 44. Kincaid JP, Fishburne RP, Rogers RL, et al. Derivation of New Readability Formulas (Automated Readability Index, fog count and Flesch Reading Ease Formula) for Navy Enlisted Personnel. Tech Train 1975;49 Research B http://www.dtic.mil/dtic/tr/fulltext/u2/a006655.pdf. [Google Scholar]
- 45. Willson TJ, Lospinoso J, Weitzel EK, et al. Effect of environmental factors on internet searches related to sinusitis. Laryngoscope 2015;125:2447–50. 10.1002/lary.25420 [DOI] [PubMed] [Google Scholar]
- 46. World Health Organization. Risk communication in the context of zika virus [Internet]. 2016;4:1 http://apps.who.int/iris/bitstream/10665/204513/1/WHO_ZIKV_RCCE_16.1_eng.pdf?ua=1 [Google Scholar]
- 47. PAHO. Caribbean Mosquito Awareness Week launched to raise awareness on preventing Zika [Internet]. 2016. http://www.paho.org/hq/index.php?option=com_content&view=article&id=11994&Itemid=135&lang=en (cited 30 Aug 2016).
- 48. PAHO. Spanish-speaking countries of the Americas celebrate the First Mosquito Awareness Week this year [Internet]. 2016. http://www.paho.org/hq/index.php?option=com_content&view=article&id=12404%3Apaises-de-america-latina-celebran-por-primera-vez-la-semana-de-accion-contra-los-mosquitos&Itemid=1926&lang=en (accessed 30 Aug 2016).
- 49. Ministry of Health Brazil. PRESS RELEASE: ministry of Health investigates cases of exanthematous diseases [Internet]. 2015. http://portalsaude.saude.gov.br/index.php/cidadao/principal/agencia-saude/17684-ministerio-da-saude-investiga-casos-de-doenca-exantematicas (accessed 28 Mar 2017).
- 50. McComas KA, Trumbo CW. Source credibility in environmental health-risk controversies: application of Meyer's credibility index. Risk Anal 2001;21:467–80. 10.1111/0272-4332.213126 [DOI] [PubMed] [Google Scholar]
- 51. Earle TC, Cvetkovich G. Social Trust: toward a Cosmopolitan Society [Internet]: Praeger, 1995. https://books.google.com.ng/books?id=z1khILCNxiwC. [Google Scholar]
- 52. Schroeder JC. Communicating science: press releases at EHP. Environ Health Perspect 2010;118:a58 10.1289/ehp.1001913 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53. Ashwell DJ. The challenges of science journalism: the perspectives of scientists, science communication advisors and journalists from New Zealand. Public Underst Sci 2016;25:379–93. 10.1177/0963662514556144 [DOI] [PubMed] [Google Scholar]
- 54. Covello VT. Best practices in public health risk and crisis communication. J Health Commun 2003;8(Suppl 1):5–8. 10.1080/713851971 [DOI] [PubMed] [Google Scholar]
- 55. Flesch R. How to write Plain English [Internet]: University of Canterbury; http://www.mang.canterbury.ac.nz/writing_guide/writing/flesch.shtml (accessed 30 Aug 2016). [Google Scholar]
- 56. Mcinnes N, Haglund BJA. Readability of online health information: implications for health literacy. Informatics for Health and Social Care 2011;36:173–89. 10.3109/17538157.2010.542529 [DOI] [PubMed] [Google Scholar]
- 57. Marston C, Renedo A. Understanding and measuring the effects of patient and public involvement: an ethnographic study. The Lancet 2013;382:S69 10.1016/S0140-6736(13)62494-0 [DOI] [Google Scholar]
- 58. Netmarketshare.com. Search Engine Market Share [Internet]. 2016. https://www.netmarketshare.com/search-engine-market-share.aspx?qprid=4&qptimeframe=Y&qpcustomd=2 (accesssed 11 Dec 2016).
- 59. Seifter A, Schwarzwalder A, Geis K, et al. The utility of “Google Trends” for epidemiological research: Lyme disease as an example. Geospat Health 2010;4:135 10.4081/gh.2010.195 [DOI] [PubMed] [Google Scholar]
- 60. CDC. Revised diagnostic testing for Zika, Chikungunya, and dengue viruses in US Public Health Laboratories [Internet]. 2016. http://www.cdc.gov/zika/pdfs/denvchikvzikv-testing-algorithm.pdf (accessed 9 Dec 2016).


