Effectiveness of 2012–2013 influenza vaccine against influenza-like illness in general population

Abstract

Most of the methods used for estimating the influenza vaccine effectiveness (IVE) target the individuals who have an influenza-like illness (ILI) rather than virologically-proven influenza and access the healthcare system.

The objective of this study was to estimate the 2012–2013 IVE in general French population, using a cohort of volunteers registered on GrippeNet.fr, an online surveillance system for ILI.

The IVE estimations were obtained through a logistic regression, and analyses were also performed by focusing on at-risk population of severe influenza, and by varying inclusion period and ILI definition. Overall, 1996 individuals were included in the analyses. The corrected IVE was estimated to 49% (20 to 67) for the overall population, and 32% (0 to 58) for the at-risk population. Three covariables appeared with a significant effect on the occurrence of at least one ILI during the epidemic: the age (P = 0.045), the presence of a child in the household (P < 10⁻³), and the frequency of cold/flu (P < 10⁻³). Comparable results were found at epidemic peak time in the hypothesis of real-time feed of data.

In this study, we proposed a novel, follow-up, web-based method to reveal seasonal vaccine effectiveness, which enables analysis in a portion of the population that is not tracked by the health care system in most VE studies.

Introduction

In most developed countries, influenza vaccination is generally recommended among individuals considered at high risk for influenza-related complications, as persons aged >65 y and those with underlying conditions. In France, individuals at high risk receive each year a voucher for free-of charge seasonal influenza vaccination from the national health insurance fund.¹

Estimating the influenza vaccine effectiveness (IVE) is necessary to evaluate every year the effect of influenza vaccination in these target groups. This evaluation is a complex problem to address, because of confounding factors and biases linked to the design of the observational studies (case-control study, cohort study, screening method), the sensitivity and specificity of the case definition used (influenza-like illness or virologically confirmed influenza), the case report system, the differences in risk exposure between vaccinated and unvaccinated individuals, and the targeted population.^2-4

Because of practical issues, studies of IVE have used a variety of non-specific outcome measures rather than virological-proven influenza, including the prevention of acute respiratory illness (ARI), influenza-like illness (ILI), pneumonia-associated hospitalizations or deaths.^5-7 Today, most of these observational studies are based on a population of patients consulting a general practitioner (GP) for ILI, which is not a randomized ILI sample,⁸ as the majority of individuals with an ILI do not systematically access the healthcare systems.^9,10 This point highlights the need to provide other IVE estimation methods, not relying on individuals tracked by the healthcare systems.

The objective of this study is to estimate the 2012–2013 seasonal IVE in the general French population, using a cohort of individuals registered on a web-based surveillance system for ILI called GrippeNet.fr.

Results

The data of GrippeNet.fr were collected between November 15, 2012 and April 21, 2013. They were asked to fill in several questionnaires along the season, whatever their health status (Fig. 1). As reported in Figure 2, 6046 individuals filled at least one intake questionnaire during this period.

Figure 1. Design of the cohort.

Figure 2. Flowchart of the study.

The inclusion and exclusion criteria enabled us to include 1996 individuals out of 6046. The 4050 excluded individuals were significantly younger (mean age = 41.8 vs. 52.9, P < 10⁻³) and less vaccinated against seasonal influenza (21% vs. 33%, P < 10⁻³) than included individuals. The percentage of women was not significantly different between included and excluded individuals (61% vs. 62%, P = 0.79). 107 individuals were excluded from the study because of missing data (5% of 2103 individuals).

Vaccinated and unvaccinated groups were statistically different on all variables tested, except the frequency of cold/flu (P = 0.32) (Table 1). Among individuals who had at least one ILI during the season, 11 individuals had two ILI (5% of individuals with at least one ILI) and two individuals had three ILI (1%).

Table 1. Baseline characteristics of all GrippeNet.fr included participants, depending on their vaccination status.

	Vaccinated individuals % (number)	Unvaccinated individuals % (number)	Overall % (number)	P value
Socio-demographic factors
Gender (women)	55% (359)	65% (873)	62% (1232)	<10−3
Age:				<10−3
0–49*	21% (140)	41% (553)	35% (693)
50–64	31% (203)	40% (530)	37% (733)
≥65	48% (314)	19% (256)	28% (570)
Presence of a child (under five years) in the household	7% (46)	14% (182)	11% (228)	<10−3
Health factors
To take regular medication for asthma	8% (51)	4% (53)	5% (104)	<10−3
To take regular medication for a lung disorder	4% (25)	1% (13)	2% (38)	<10−3
To take regular medication for diabetes, a heart disorder, kidney disorder or an immunocompromising condition	23% (151)	9% (125)	14% (276)	<10−3
Frequency of cold/flu:				0.32
- Less than one per year	48% (319)	45% (602)	46% (921)
- One or two per year	37% (241)	39% (523)	38% (764)
- At least three per year	15% (97)	16% (214)	16% (311)
Dependent variable
To have at least one ILI during the epidemic period	7% (43)	13% (170)	11% (213)	<10−3
Total	33% (657)	67% (1339)	100% (1996)

*One child under 5 y.

Estimates of IVE

The crude and adjusted IVE estimates of the 2012–2013 seasonal influenza vaccine are reported in Table 2. The corrected IVE was 49% (95% confidence interval: 20 to 67) (P < 10⁻³). Three covariables appeared with a significant effect on the occurrence of at least one ILI during the epidemic: the age (P = 0.045), the presence of a child in the household (P < 10⁻³), and the frequency of cold/flu (P < 10⁻³).

Table 2. The crude and adjusted IVE estimates of the 2012–2013 seasonal vaccine of GrippeNet.fr participants reporting an ILI defined by the presence of fever ≥38 °C and cough (significant results appear in bold).

	Crude OR (95% CI)	Adjusted OR (95% CI)	Corrected OR (95% CI)	P value (multivariate analysis, analysis of deviance)
Vaccination
Influenza vaccination during the current season	0.38 (0.24 to 0.60)	0.49 (0.31 to 0.79)	0.51 (0.33 to 0.80)	<10−3
Socio-demographic factors
Gender (women)	0.99 (0.67 to 1.46)	1.09 (0.74 to 1.61)	1.07 (0.77 to 1.47)	0.51
Age:
0–49	1	1	1	0.045
50–64	0.36 (0.25 to 0.52)	0.60 (0.39 to 0.92)	0.62 (0.41 to 0.93)
≥65	0.34 (0.22 to 0.53)	0.77 (0.45 to 1.32)	0.78 (0.47 to 1.29)
Presence of a child (under five years) in the household	3.43 (2.23 to 5.30)	2.24 (1.39 to 3.61)	1.61 (1.24 to 1.97)	<10−3
Health factors
To take regular medication for asthma	1.66 (0.76 to 3.63)	1.69 (0.71 to 4.02)	1.46 (0.76 to 2.36)	0.051*
To take regular medication for a lung disorder	0.59 (0.17 to 2.01)	0.60 (0.12 to 3.10)	0.63 (0.13 to 2.56)	0.41
To take regular medication for diabetes, a heart disorder, kidney disorder or an immunocompromising condition	0.67 (0.37 to 1.21)	1.30 (0.72 to 2.36)	1.26 (0.74 to 2.04)	0.32
Frequency of cold/flu:
- Less than one per year	1	1	1	<10−3
- One or two per year	1.78 (1.13 to 2.80)	1.49 (0.93 to 2.38)	1.39 (0.94 to 1.98)
- At least three per year	4.17 (2.56 to 6.78)	2.75 (1.63 to 4.63)	1.83 (1.38 to 2.28)

*P value of the Wald test = 0.24

No significant effect of the vaccination was observed on the subgroups of at-risk population. The estimated corrected IVE were 32% (0 to 58) (P = 0.10) for the overall at-risk population, 28% (0 to 61) (P = 0.30) for individuals over 65 yo, and 37% (0 to 71) (P = 0.21) for at-risk individuals under 65 yo (Table 3).

Table 3. The IVE estimates of the 2012–2013 seasonal vaccine evaluated in GrippeNet.fr sub-populations at risk for severe influenza reporting an ILI defined by the presence of fever ≥38 °C and cough.

	Overall at-risk population	Individuals ≥65	At-risk population <65	Individuals who have consulted a MG during the season
Number of included individuals (vaccinated; unvaccinated)	807 (429; 378)	570 (314; 256)	237 (115; 122)	574 (171; 403)
IVE (95% CI)	34% (0 to 60)	29% (0 to 63)	40% (0 to 73)	40% (0 to 67)
P value (analysis of deviance)	0.10	0.30	0.21	0.048*
Corrected IVE (95% CI)	32% (0 to 58)	28% (0 to 61)	37% (0 to 71)	35% (0 to 62)
Attack rate for unvaccinated individuals (number of ILI)	12% (44)	9% (22)	18% (22)	27% (107)
Attack rate for vaccinated individuals (number of ILI)	7% (32)	6% (20)	10% (12)	18% (31)

^*P value of the Wald-test = 0.083

When we performed the analyses targeting individuals who have consulted a GP during the follow-up of GrippeNet.fr cohort (between 15 November 2012 and 21 April 2013), the effect of the vaccination appeared at the limit of significance (P value of the analysis of deviance = 0.048, P value of the Wald test = 0.083), with a corrected IVE estimated to 35% (0 to 62).

No significant effect of the vaccination was observed outside the epidemic period (P = 0.63).

In real-time conditions, at the epidemic peak time, the corrected IVE was estimated to 49% (10 to 72) (P = 0.0043).

The corrected IVE estimated with other ILI definitions was 45% (0 to 82) (P = 0.094) by using the French Sentinelles network definition and was 21% (0 to 38) (P = 0.021) by using the ECDC definition (Table 4).

Table 4. The IVE estimates of the 2012–2013 seasonal vaccine evaluated considering several ILI definitions.

	Main analysis (cough + fever ≥38 °C)	French Sentinelles network definition⊥	ECDC definitionγ
Number of included individuals (vaccinated; unvaccinated)	1996 (657; 1339)	1996 (657; 1339)	1996 (657; 1339)
IVE (95% CI)	51% (21 to 69)	45% (0 to 82)	26% (0 to 45)
P value (analysis of deviance)	<10−3	0.094	0.021*
Corrected IVE (95% CI)	49% (20 to 67)	45% (0 to 82)	21% (0 to 38)
Attack rate for unvaccinated individuals (number of ILI)	13% (170)	3% (42)	33% (438)
Attack rate for vaccinated individuals (number of ILI)	7% (43)	1% (7)	23% (154)

^⊥a sudden fever >39 °C (102 °F) with myalgia and respiratory signs. ^γacute onset of reported symptoms and at least one of four systemic symptoms (fever/chills, feeling tired/exhausted, headache, muscle pain) and at least one of three respiratory symptoms (cough, sore throat, shortness of breath) . *P value of the Wald test = 0.050.

Discussion

In our study, we used the web-based cohort of the GrippeNet.fr surveillance system to estimate the 2012–2013 seasonal IVE on active participants who had an ILI based on the definition cough + fever ≥38 °C, during the influenza epidemic period. We estimated the IVE of the seasonal vaccine in the 2012–2013 season to be around 50%. Comparable results were found at epidemic peak time in the hypothesis of real-time feed of data (i.e., non-consolidated data).

In this study we reported an IVE of ≈50% which is in line with estimations of IVE observed for well-matched sub-types.¹¹ In mainland France, the 2012–2013 season was characterized by a prolonged influenza epidemic in the community (13 wk) with co-circulation of A(H3N2) (18%), A(H1N1) pdm09 (21%), and B influenza viruses (55%). Overall in Europe, throughout the 2012–2013 season, 99% of both the A(H1N1)pdm09 and A(H3N2) viruses showed a good match with the seasonal vaccine.¹² In France, among influenza B viruses for which the determination of lineage was performed, 92% belonging to B-Yamagata lineage and were similar to the vaccine strain “B/Wisconsin/1/2010” of the season influenza vaccine 2012–2013.¹³

Similarly to our results, early estimates of IVE measured through a test-negative case-control study in five European Union countries (week 43/2012–week 3/2013), showed a moderate VE against all influenza viruses (VE = 62.2%; 21.1 to 81.9).³ By type and subtype, the highest VE was against influenza B viruses and the lowest against influenza A(H3N2) viruses. However, due to small sample size, the precision around these estimates is low. To be noted that a discrepancy between the apparently well-matched vaccine strain/circulating strains and low VE has been reported by several studies based on virologically-proven influenza patients.^3,14-18

Our results were in line with IVE estimations previously found in an English cohort similar to GrippeNet.fr and based on ILI (2010–2011 IVE, 52%, 27 to 58).⁵ Although we did not observe a significant effect of the vaccination in at-risk population, probably because of a lack of statistical power, we can observe a tendency to have a lower IVE in the overall population (32%, 0 to 58), especially for individuals over 65 yo (28%, 0 to 61), which is inline with literature findings.^4,19-25 No significant effect of the vaccination (P = 0.63) was found outside influenza viral circulation, thus strengthening the validity of our results.

Three of the 6 covariables identified in the literature as potential risk factors for ILI/influenza and added in the model were found to have a significant effect on the occurrence of at least one ILI during the season. These effects are in line with literature findings, describing a risk of having an ILI being higher for individuals living with small children²⁶ and for individuals having frequently cold/flu,¹⁰ and being lower for older individuals.^27-30 The covariable “To take regular medication for asthma” was almost significant (p-value of the analysis of deviance = 0.051, P value of the Wald-test = 0.24), with a tendency to increase the risk of having an ILI, as found in the literature.^29-31 The covariables “To take regular medication for a pulmonary disorder” (P = 0.41) and “To take regular medication for diabetes, a heart disorder, kidney disorder or an immunocompromising condition” (P = 0.32) were not significant, while they are found in the literature to increase the risk of having an ILI.^19-23,29 The questions related to these covariables are likely not precise enough to allow for a correct identification of individuals with severe chronic diseases, and may be prone to misinterpretation.

The greatest strength of this study is its general population approach, enabling the analysis in a portion of the population that is not tracked by the health care system. The analysis we realized focusing on individuals who have consulted a GP at least one time during the winter showed that the IVE is probably not the same in this particular population as the IVE observed in general population (35%, 0 to 62 vs. 49%, 20 to 67). The number of included individuals with missing data are few (not superior to 5%). Data are collected in a prospective way, which limits recall bias.

The main limit of this approach is the absence of case validation by a healthcare professional and the absence of virological confirmation of ILI cases. Observational studies that compare less-specific outcomes such as ILI, among vaccinated populations to those among unvaccinated populations might be more subject to biases than studies using laboratory outcomes. A 2012 simulation study found that for each percentage point decrease in diagnostic test specificity for influenza virus infection, IVE would be underestimated by approximately 4%.³² The IVE estimated in the present study was not affected by this decrease, showing similar values to 2012–2013 studies based on virologically–proven influenza patients . This raises the possibility that the reported IVE estimate is caused by bias. One bias leading to IVE overestimation could be due to an under self-reporting of ILI among vaccinated subjects. The determination of vaccination status and identification of high-risk conditions in Grippenet.fr were not validated by medical records. Self-report of influenza vaccination status,^33,34 and symptoms of infectious illness, including respiratory illnesses³⁵ has been shown to be acceptably accurate and valid, and monthly and weekly follow-up intervals have been used in other studies assessing occurrences of ILI.^36,37 Even if self-report of vaccination could be subject to recall bias and overestimation of rates (especially in the elderly and social desirability bias in the answers provided),³⁸ in contrast to telephone surveys, the web self-report is less involved by the social desirability bias in the answers provided because of absence of a personal contact with the inquiring.

Furthermore, as previously explained there is no reason to expect that vaccinated and unvaccinated participants who exhibit flu-like symptoms would complete the survey in different ways. An important bias to be considered in this study is due to the so-called healthy vaccine effect, which results in overestimation of IVE. The GrippeNet.fr population is found to be more frequently employed, with a higher education level and vaccination rate with respect to the general population. People with a healthy lifestyle are more likely to accept/request vaccination and less likely to be sick. Even if we collected detailed information on chronic conditions and functional status, allowing us to correct for this potential confounding, we cannot exclude that residual confounding still biases our results.

To obtain a reliable evaluation of IVE, the definition of ILI should have a high specificity, while having also an acceptable sensitivity. In a literature review by Ebell and al,³⁹ the definition “fever ≥38°C and cough” was found to have a specificity lying between 50% and 94% (depending on the study) and a sensitivity between 30% and 78% for identifying virologically confirmed influenza. The use of the French Sentinelles network definition, which have a positive predictive value of about 40% (34 to 47) and a negative predictive value of 80% (77 to 84)¹⁰ led to an IVE estimation similar to the one of the main analysis (45%, 0 to 82 vs. 49%, 20 to 67). As expected, the use of the ECDC definition led to obtain a lower IVE estimation (21%, 0 to 38). This definition is believed to have a low specificity, although studies evaluating the specificity and sensitivity of this definition are lacking. As raised by Orenstein and al.,² the use of a cohort design studying ILI leads to underestimate real IVE. This author suggests that for an ILI definition with a sensitivity of 80%, a specificity of 90%, and with a 15% true attack rate of medically attended influenza among unvaccinated, and a 30% true attack rate of medically attended non-influenza ILI, when the estimated IVE is 55.7% the real IVE is 70%.² A lower attack rate of influenza, as well as poorer specificity or sensitivity are reflected by a higher difference between the estimated and the real IVE. Evaluating the sensitivity and specificity of these several ILI definitions in the context of GrippeNet.fr could therefore be relevant and allow us to better estimate the positive predictive value of these definitions and then the real IVE. This could be performed by sending self-swabbing kits to GrippeNet.fr participants who declare ILI symptoms, similarly to what has been done in the United Kingdom, where 42% of 294 callers to the influenza telephone hotline (NHS direct) who were sent a self-sampling kit sent back the swabs within a delay allowing for analyses.⁴⁰

While the use of such non-specific outcomes for IVE estimation is problematic, there are methods to estimate the contribution of influenza to the incidence of non-specific outcomes. It is well known that influenza epidemics are associated with increases in pneumonia/influenza hospitalizations and deaths above expected, smoothed seasonal baselines.⁶ Recently it has been suggested that in observational cohort studies of IVE, using only these ‘excess’ of influenza-associated outcomes (instead of the raw figure), during viral circulation period, could lead to more precise and less-biased VE estimates.⁷

The study of the GrippeNet.fr cohort allowed us to estimate the 2012–2013 IVE to 49% (20 to 67). A similar result was found in real-time conditions, at epidemic peak time. These results are however highly dependent on the size of the cohort.

During influenza season, this approach could complete traditional methods for estimating the IVE, providing early preliminary results, and could be very useful in a pandemic context.

Methods

Study design

This observational, prospective cohort study was conducted by using a French web-based surveillance system for ILIs called GrippeNet.fr. Between mid-November 2012 and end of April 2013, volunteers living in France (excluding overseas territories) could register on the website of the study (www.grippenet.fr) to participate in the surveillance of ILIs. They were asked to fill in several questionnaires along the season, whatever their health status (Fig. 1). Participants received every week a newsletter containing a personal link allowing them to access directly their questionnaire. Participants could fill in questionnaires for other individuals (young individuals, elderly…) on their behalf, once consent given. The system is part of a European surveillance network in the general population called Influenzanet.⁴¹ As recently reported,⁴² the Grippenet population was not representative of the general population in terms of age and gender, however all age classes were represented, including the older classes (>65 yo), generally less familiar with the digital world, but considered at high risk for influenza complications. Once adjusted on demographic indicators, the GrippeNet.fr population is found to be more frequently employed, with a higher education level and vaccination rate with respect to the general population.

Data were collected through three types of questionnaires. (1) The intake questionnaire, filled in at inclusion and that covered demographic, geographical, socio-economic, and health-related factors. (2) Several weekly questionnaires filled in during the influenza season, where participants declared the symptoms they had or not since their last connection. (3) A vaccination questionnaire, sent at the end of the GrippeNet.fr season (end of April 2013) to update influenza vaccination status of participants, if they have been vaccinated after they had filled their intake questionnaire. The full list of variables for which data were collected in the intake and weekly questionnaires is reported in Table S1.

This study was conducted in agreement with French regulations on privacy and data collection and treatment and was approved by the Comité consultatif sur le traitement de l'information en matière de recherche (CCTIRS, Advisory committee on information processing for research, authorization 11.565) and by the Commission Nationale de l’Informatique et des Libertés (CNIL, French Data Protection Authority, authorization DR-2012–024).

Sample used in the analysis

In order to include in the analyses only individuals who have participated regularly (active participants) during the influenza epidemic period, we applied the following inclusion criteria: completion of at least three weekly questionnaires separated by at least 4 d^43,44; and completion of at least one weekly questionnaire before the beginning of the epidemic period and after the end of the epidemic period; and completion of at least one weekly questionnaire / 15 d during the epidemic period. The dates considered for the epidemic period were the one determined by the French Sentinelles network (GP surveillance system) which monitors ILI, defined as a sudden fever >39 °C (102 °F) with myalgia and respiratory signs.^45,46 These dates (from week 51 of 2012 [17 December] to week 11 of 2013 [11 March]) were in agreement with the period of viral circulation defined by virological data from WHO-FluNet⁴⁷ (Fig. S1). In presence of multiple instances of intake questionnaires completed by the same participant, we considered the last one. In presence of multiple instances of weekly questionnaires for the same day completed by the same participant, we considered the last one of the day. Furthermore, we considered only participants with no change in influenza vaccination status during the epidemic period (exclusion of individuals vaccinated less than 14 d before the beginning of the epidemic, or during the epidemic) and with no missing data on studied variables.

ILI definition

ILI was defined by the presence of fever ≥38 °C and cough. According to a literature review by Ebell and al,³⁹ this definition has a specificity of at least 50% and a sensitivity of at least 30% for identifying virologically confirmed influenza. The dependant variable was the occurrence of at least one ILI during the epidemic period.

Descriptive analyses

Demographical and clinical variables of included and excluded participants were compared with χ²-test for non-continuous variables and Student t test for mean comparisons, using bilateral tests and a 5% cut-off point. The characteristics of the vaccinated group were compared with the one of the unvaccinated group with χ²-test.

Estimations of IVE

We estimated the odds ratio (OR) of the 2012–2013 influenza seasonal vaccination through logistic regression with participants reporting an ILI (not confirmed virologically) as outcome and vaccination status as explanatory variable. As age and gender structure of GrippeNet.fr population is not the same as the one of general French population, analyses were reweighted on age groups (considering the following age categories: 0–9, 10–19, 20–29, 30–39, 40–49, 50–59, 60–69, ≥70) and gender.^42,48 Logistic regression model was adjusted for gender and potential confounding factors: age (<49, 50–64, ≥65),^{19,21,22,25,27,29,30,49,50} presence of a child (under five years) in the household,²⁶ taking regular medication for asthma,^29-31 for a lung disorder,²⁹ for diabetes, a heart disorder, kidney disorder or an immunocompromising condition,^19,20,29 and the frequency of cold/flu.¹⁰

The P values were estimated through analysis of deviance and Wald test (P value < 0.05).

The odds ratio (OR) is commonly used to assess associations between exposure and outcome and can be estimated by logistic regression, which is widely available in statistics software. OR has been considered an approximation to the prevalence ratio (PR) in cross-sectional studies or the risk ratio (RR, which is mathematically equivalent to PR) in cohort studies or clinical trials. This is acceptable when the outcome is relatively rare (<10%). As ILI is not a rare disease and its incidence was superior to 10% in unexposed individuals of our study population, OR was corrected by the Zhang and Yu method⁵¹ to obtain corrected OR (OR_C), with $P_0$indicating the incidence of ILI in the unexposed group:

$$ORc=\frac{OR}{(1-P_0)+(P_0{}_{}xOR)}$$

Even if other methods are available for estimating the PR or RR in cohort studies of common outcomes,⁵² most of them did not allow to take into account many adjustment factors or face convergence issues. The simple Mantel-Haenszel risk ratio method could be used in a situation with only one or two categorical covariables have to be take into account. The use of log-binomial and Poisson methods were in turn problematic because of problems of convergence and of larger estimates.⁵³ A study measuring IVE using self-reports of ILI, showed that estimates generated using log-binomial regression and adjusting ORc by using the method of Zang and Yu gave similar results.⁵

IVE was calculated as 100 × (1− ORc) for influenza vaccination. IVE estimates less than zero were reported as zero.

Analyses in sub-groups and sensitivity analysis

For secondary analyses, we restricted the study to individuals at-risk of severe influenza (see details in Text S1).¹ The analyses were also performed outside the epidemic period (considering three weeks following the end of the epidemic), and at epidemic peak time in the hypothesis of real-time feed of data (i.e., non-consolidated data). For sensitivity analysis, the analyses were performed using the ILI definition of the French Sentinelles network (sudden fever ≥39 °C (102 °F) with myalgia and respiratory signs),⁴⁵ and the one of the European Centre for Disease Prevention and Control (ECDC): acute onset of reported symptoms and at least one of four systemic symptoms (fever/chills, feeling tired/exhausted, headache, muscle pain) and at least one of three respiratory symptoms (cough, sore throat, shortness of breath). All analyses were performed with the R software (version 2.13.2; R Development Core Team 2011).⁵⁴

10.4161/hv.27439