Abstract
Background.
Parent-reported influenza vaccination history may be valuable clinically and in influenza vaccine effectiveness (VE) studies. Few studies have assessed the validity of parental report among hospitalized children.
Methods.
Parents of 2597 hospitalized children 6 months–17 years old were interviewed from November 1, 2015 to June 30, 2016, regarding their child’s sociodemographic and influenza vaccination history. Parent-reported 2015–2016 influenza vaccination history was compared with documented vaccination records (considered the gold standard for analysis) obtained from medical records, immunization information systems, and providers. Multivariable logistic regression analyses were conducted to determine potential factors associated with discordance between the 2 sources of vaccination history. Using a test-negative design, we estimated VE using vaccination history obtained through parental report and documented records.
Results.
According to parental report, 1718 (66%) children received the 2015–2016 influenza vaccine, and of those, 1432 (83%) had documentation of vaccine receipt. Percent agreement was 87%, with a sensitivity of 96% (95% confidence interval [CI], 95%–97%) and a specificity of 74% (95% CI, 72%–77%). In the multivariable logistic regression, study site and child’s age 5–8 years were significant predictors of discordance. Adjusted VE among children who received ≥1 dose of the 2015–2016 influenza vaccine per parental report was 61% (95% CI, 43%–74%), whereas VE using documented records was 55% (95% CI, 33%–69%).
Conclusions.
Parental report of influenza vaccination was sensitive but not as specific compared with documented records. However, VE against influenza-associated hospitalizations using either source of vaccination history did not differ substantially. Parental report is valuable for timely influenza VE studies.
Keywords: discordance, influenza vaccination, immunization record, parental report, vaccine effectiveness, validity
Influenza is a common cause of acute respiratory infection in children in the United States (US) and worldwide [1–4]. To prevent and mitigate the disease with its associated complications, since 2008 the US Advisory Committee on Immunization Practices has recommended routine annual influenza vaccination for all persons aged ≥6 months [5–7].
While vaccines are currently the best tool for preventing influenza disease, the effectiveness of influenza vaccines varies depending on factors such as virus drift, vaccine match, and vaccine product, as well as the recipient’s age and underlying medical conditions [8–11]. Observational vaccine effectiveness (VE) studies are crucial for monitoring vaccine performance and identifying strategies to improve the benefits of vaccination. Case-control or test-negative designs are common approaches to estimating influenza VE. These methods infer vaccine protection based on a difference in antecedent vaccination among influenza-positive compared with influenza-negative patients. Thus, high-quality, valid vaccination data are key for accurate estimation of VE through observational studies.
Sources of influenza vaccination history in observational studies include parental or self-report and documented report from medical records or immunization information systems (IISs) [12–15]. Several challenges pertaining to influenza vaccination complicate obtaining timely and accurate vaccination history for VE studies. IISs can be useful sources of obtaining vaccination history but may not capture vaccination history from all provider sources and may not be updated in a timely fashion [16]. Unlike other routine childhood immunizations, children may receive influenza vaccines in nontraditional settings such as retail pharmacies or schools. Parental report of vaccination history is easily accessible but can be subject to recall bias and misclassification [12, 17–20]. While some outpatient studies have assessed the validity of parental report of influenza vaccination [19, 21–24], there is still a gap in the literature on vaccine validation in the inpatient setting, as well as the inability to generalize findings to other populations.
In this study, we evaluated methods of identifying influenza vaccination status among children aged 6 months–17 years hospitalized for acute respiratory illnesses or fever. Our primary objective was to compare parental report of 2015–2016 influenza vaccination with documented records. Our secondary objective was to identify potential predictors of discordance in vaccine history between parental report and documented records. Finally, the third objective was to assess how current influenza season vaccination history obtained by parental report compared with documented record affected VE estimates.
METHODS
Study Design
We included children enrolled in the New Vaccine Surveillance Network (NVSN) from November 1, 2015 to June 30, 2016. Details of this Centers for Disease Control and Prevention–funded, multisite collaborative network have previously been published [25]. In brief, after obtaining informed consent and assent in accordance with state law, children 6 months–17 years of age were eligible for study enrollment if they were admitted to the hospital within 48 hours prior to enrollment with acute respiratory illness and/or fever for a duration <14 days at study sites in Nashville, Tennessee; Rochester, New York; Cincinnati, Ohio; Seattle, Washington; Houston, Texas; Kansas City, Missouri; and Oakland, California [25]. Institutional review boards at the 7 participating hospitals and the Centers for Disease Control and Prevention reviewed and approved this study.
Data and Sample Collection
For enrolled children, demographic information, symptoms, medical history, and influenza vaccination history including location of vaccine administration were collected through parent/ guardian interview (hereafter referred as “parental report”). Parents were asked about their child’s influenza vaccination status for the 2015–2016 influenza season: “Has your child received this year’s influenza (flu) vaccine (since July 1, 2015)?” Parents who answered “Yes” were then asked to specify the number of doses received, route of administration, and timing of the most recent dose. For timing, parents were asked: “Was the vaccine given 14 days or more before your child’s illness began?” Parents unable to answer were asked to provide the date their child received the most recent dose. Parents of children aged 6 months through 8 years were also asked to provide the number of influenza vaccine doses received in past seasons.
Additional clinical information including underlying medical conditions and hospital course were obtained from standardized medical chart review. Review of vaccine records was performed for all enrolled children irrespective of parental report, and the process for obtaining documented immunization records varied by site. Documented records were obtained from medical records, IIS, primary care providers, other medical providers (eg, specialty care), public health clinics, and nontraditional providers (eg, pharmacy, supermarket) as applicable. Informed consent included approval to obtain vaccination history by contacting providers or other locations provided by parents as to where vaccine was received. For specimen collection, midturbinate nasal and throat swabs or tracheal aspirates were collected and tested for the influenza virus using molecular assays [25].
Influenza Vaccination Status
We ascertained influenza vaccination status for the 2015–2016 season by determining receipt of influenza vaccine prior to enrollment by either parental report or documented records. For the validation analysis, children were considered vaccinated per parental report if parents stated their child had received the 2015–2016 influenza vaccine and unvaccinated if parents indicated their child did not receive the vaccine. Children were excluded if parents did not know or refused to provide their child’s vaccination status for the current influenza season. Children were classified as vaccinated per documented records if immunization records indicated ≥1 dose of the 2015–2016 influenza vaccine administered from August 1, 2015 to June 30, 2016. If study staff were not able to verify the child immunization records (eg, study staff never received the immunization record from a provider after a minimum of 3 inquiries and/or were not able to find the child’s immunization records in the state IIS), or documented records indicated receipt of the first vaccine dose on the same day of the parent interview, the child was excluded from the study.
Data Analysis
We compared differences in sociodemographic characteristics between vaccinated and unvaccinated children (by parental report and documented records, separately) using χ2 tests. To assess the validity of influenza vaccination status from parental report to documented records, we estimated percentage agreement, sensitivity, specificity and Cohen’s κ coefficient [19, 23]. We considered documented records as the gold standard based on the extensive efforts placed to verify immunization records in this study. Percentage agreement was defined as the percentage of children whose parents accurately recalled the vaccination status of their children. Sensitivity was defined as the percentage of children whose parents reported receipt of influenza vaccination among those who had documentation of vaccine receipt. Specificity was defined as the percentage of children whose parents reported no receipt of influenza vaccination among those who had no documentation of influenza vaccine receipt in their immunization records. Cohen’s κ coefficient was computed to measure the agreement between parental report and documented records beyond that expected by chance. A κ coefficient of ≤0 as indicating no agreement, 0.01–0.20 indicates poor to slight agreement, 0.21–0.40 fair agreement, 0.41–0.60 moderate agreement, 0.61–0.80 substantial agreement, and 0.81–1.00 indicates almost perfect agreement [26, 27]. These validity measures were further stratified by influenza test results and were also assessed for timing of vaccination relative to illness onset.
As a second objective, we evaluated possible predictors associated with discordance between parental report and documented vaccination records. We first created a binary discordant response variable to identify children who had parental report of vaccination status agreeing with immunization records (concordant) and those who had disagreement in their vaccination status (discordant). We then fitted logistic regression models for the discordant response variable on each of the independent variables. Predictors thought to potentially affect discordance in vaccination history were evaluated [24, 28–30]. The factors included age, study site, site-specific influenza season enrollment (time period of the first and last influenza positive case at each site), sex, race/ethnicity, health insurance status (public/self-pay/unknown and private/both), underlying medical conditions (0 or ≥1), interviewee relationship to child, mother’s education, mother’s age, and household size. Covariates with P ≤ .20 from the univariate analysis were entered into a multivariable logistic regression model.
Finally, we assessed whether influenza VE estimates differed by the source of vaccination history. Feldstein et al [25] previously evaluated influenza VE in this population of children enrolled in NVSN during the same 2015–2016 study season using documented records of vaccination history. We repeated this analysis to compare VE estimates by documented records vs parental report. In brief, we used a test-negative design [25] to estimate VE against laboratory-confirmed influenza in children 6 months–17 years old, enrolled during each site-specific influenza season, and who had an illness duration of ≤10 days prior to enrollment. For this current VE analysis, children were considered vaccinated if they had received ≥1 dose of the 2015–2016 influenza vaccine ≥14 days before illness onset. Logistic regression models were used to calculate odd ratios (ORs) to compare the odds of vaccination among cases (subjects who tested positive for the influenza virus) and controls (subjects who tested negative for influenza); VE was estimated as 100% × (1 − OR). Age, study site, calendar time (enrollment month), race/ethnicity, days from illness onset to enrollment (0–2, 3–4, 5–7, and 8–10 days), and health insurance status (public/self-pay/unknown and private/both) were included a priori in the multivariate model for consistency with Feldstein et al [25]. All data analyses were performed using SAS version 9.4 software.
RESULTS
Demographics
We enrolled 2866 hospitalized children aged 6 months–17 years from November 1, 2015 to June 30, 2016, and of those we excluded 269 (9%) children who did not meet eligibility criteria for the validity analysis (Figure 1). Among the remaining 2597 children, 42% were 6–23 months old, 58% were male, 33% were white non-Hispanic/non-Latino, and 36% had private or both private and public insurance (Table 1). More than half (61%) of the children had one or more underlying medical conditions and 15% were admitted to an intensive care unit. Of the care-givers who completed the parent interview, 86% were mothers and most (83%) had an education level of high school or higher.
Figure 1.
Study enrollment for the validation analysis and vaccine effectiveness (VE) analysis, and influenza case status—New Vaccine Surveillance Network, 2015–2016.
Table 1.
Characteristics of Study Participants by Influenza Vaccination Status Obtained by Parental Report and Documented Records—New Vaccine Surveillance Network, 2015–2016
| Influenza Vaccination Status |
|||||
|---|---|---|---|---|---|
| Characteristic | Parental Report, No. (Row %) |
Documented Records, No. (Row %) |
|||
| Totala, No. (Column %) | Vaccinated | Unvaccinated | Vaccinated | Unvaccinated | |
| All children | 2597 | 1718 (66) | 879 (34) | 1485 (57) | 1112 (43) |
| Age group | |||||
| 6–23 mo | 1078 (41) | 756 (70) | 322 (30) | 683 (63) | 395 (37) |
| 2–4 y | 679 (26) | 428 (63) | 251 (37) | 382 (56) | 297 (44) |
| 5–8 y | 462 (18) | 289 (63) | 173 (37) | 219 (47) | 243 (53) |
| 9–17 y | 378 (15) | 245 (65) | 133 (35) | 201 (53) | 177 (47) |
| Study site | |||||
| Cincinnati | 432 (17) | 274 (63) | 158 (37) | 232 (54) | 200 (46) |
| Houston | 638 (25) | 459 (72) | 179 (28) | 405 (63) | 233 (37) |
| Kansas City | 304 (12) | 164 (54) | 140 (46) | 121 (40) | 183 (60) |
| Nashville | 295 (11) | 183 (62) | 112 (38) | 146 (49) | 149 (51) |
| Oakland | 385 (15) | 252 (65) | 133 (35) | 217 (56) | 168 (44) |
| Rochester | 220 (8) | 158 (72) | 62 (28) | 149 (68) | 71 (32) |
| Seattle | 323 (12) | 228 (71) | 95 (29) | 215 (67) | 108 (33) |
| Enrolled during site-specific influenza season | |||||
| No | 1020 (39) | 637 (62) | 383 (38) | 527 (52) | 493 (48) |
| Yes | 1577 (61) | 1081 (69) | 496 (31) | 958 (61) | 619 (39) |
| Sexb | |||||
| Male | 1505 (58) | 996 (66) | 509 (34) | 866 (58) | 639 (42) |
| Female | 1092 (42) | 722 (66) | 370 (34) | 619 (57) | 473 (43) |
| Race/ethnicity | |||||
| White, non-Hispanic/non-Latino | 853 (33) | 574 (67) | 279 (33) | 498 (58) | 355 (42) |
| Black, non-Hispanic/non-Latino | 686 (26) | 396 (58) | 290 (42) | 323 (47) | 363 (53) |
| Other, non-Hispanic/non-Latino | 302 (12) | 198 (66) | 104 (34) | 170 (56) | 132 (44) |
| Hispanic/Latino | 754 (29) | 548 (73) | 206 (27) | 493 (65) | 261 (35) |
| Health insurance status | |||||
| Private/private and public | 941 (36) | 676 (72) | 265 (28) | 598 (64) | 343 (36) |
| Public/self-pay | 1626 (63) | 1021 (63) | 605 (37) | 868 (53) | 758 (47) |
| Unknown | 30 (1) | 21 (70) | 9 (30) | 19 (63) | 11 (37) |
| Underlying medical conditions | |||||
| None | 1011 (39) | 611 (60) | 400 (40) | 517(51) | 494 (49) |
| ≥1 | 1586(61) | 1107 (70) | 479 (30) | 968 (61) | 618 (39) |
| ICU admission statusb,c | |||||
| No | 2218 (85) | 1453 (66) | 765 (34) | 1257(57) | 961 (43) |
| Yes | 379 (15) | 265 (70) | 114 (30) | 228 (60) | 151 (40) |
| Interviewee relationship to childb | |||||
| Mother/stepmother | 2221 (85) | 1457 (66) | 764 (34) | 1266(57) | 955 (43) |
| Father/stepfather | 334 (13) | 236 (71) | 98 (29) | 198 (59) | 136 (41) |
| Other relative | 41 (2) | 25 (61) | 16 (39) | 20 (49) | 21 (51) |
| Mothers education | |||||
| Less than high school | 393 (15) | 275 (70) | 118 (30) | 227 (58) | 166 (42) |
| High school/GED | 1051 (40) | 634 (60) | 417 (40) | 546 (52) | 505 (48) |
| College/graduate degree | 1104 (43) | 777 (70) | 327 (30) | 683 (62) | 421 (38) |
| Unknown | 49 (2) | 32 (65) | 17 (35) | 29 (59) | 20 (41) |
| Mother’s age | |||||
| 18–29 y | 1058(41) | 661 (62) | 397 (38) | 567 (54) | 491 (46) |
| 30–39 y | 1139 (44) | 790 (69) | 349 (31) | 699 (61) | 440 (39) |
| ≥40 y | 362 (14) | 238 (66) | 124 (34) | 194 (54) | 168 (46) |
| Unknown | 38 (1) | 29 (76) | 9 (24) | 25 (66) | 13 (34) |
| Household size | |||||
| Child plus 1–2 people | 618 (24) | 433 (70) | 185 (30) | 377 (61) | 241 (39) |
| Child plus 3 people | 767 (30) | 524 (68) | 243 (32) | 462 (60) | 305 (40) |
| Child plus ≥4 people | 1208 (46) | 758 (63) | 450 (37) | 645 (53) | 563 (47) |
| Influenza test result | |||||
| Positive | 133 (5) | 60 (45) | 73 (55) | 55 (41) | 78 (59) |
| Negative | 2464 (95) | 1658 (67) | 806 (33) | 1430 (58) | 1034 (42) |
Difference in characteristics between vaccinated and unvaccinated children were evaluated (by parental report and documented records, separately).
Abbreviations: GED, general equivalency diploma; ICU, intensive care unit.
The entire population of children in the validation analysis.
No significant difference in sociodemographic characteristics between vaccinated and unvaccinated children using parental report or documented record. P values from the χ2 test were >.05 for the following characteristics: sex, ICU admission status, and interviewee relationship to child.
ICU admission status. Among the children who were admitted in the ICU, 102 children were admitted the same day the interview was conducted, and 30 children were admitted after the interview.
Vaccination Receipt—Comparing Parental Report and Documented Records
Among all enrolled children (n = 2597), 66% were vaccinated based on parental report compared with 57% by documented records (Table 1). Of the 1718 children with parental report of vaccination, 1432 (83%) had documentation. Among the 879 children unvaccinated by parental report, 53 (6%) had documentation of vaccine receipt (Table 2). Of the sources used for verification of receipt or no receipt of vaccine, the IIS and/or providers were accessed or contacted for 98% of the children. Among 2213 children for whom providers were contacted, 86% of the providers were primary care providers, 5% were public health clinics, 8% were other medical providers (eg, specialty care or hospital), and <1% were nontraditional providers (eg, pharmacy).
Table 2.
Parental Report of Child’s Current Influenza Vaccination Status Compared With Documented Records—New Vaccine Surveillance Network, 2015–2016
| Documented Records, No. |
||||||
|---|---|---|---|---|---|---|
| Parental Report | Vaccinated | Unvaccinated | Percentage Agreement | Sensitivity, % (95% CI) | Specificity, % (95% CI) | κ (95% CI) |
| 2015–2016 influenza season | ||||||
| Vaccinated | 1432 | 286 | 87 | 96 (95–97) | 74 (72–77) | 0.73 (.70–.75) |
| Unvaccinated | 53 | 826 | ||||
| Influenza test resulta | ||||||
| Positive | … | |||||
| Vaccinated | 50 | 10 | 89 | 91 (83–99) | 87 (80–95) | 0.77 (.66–.88) |
| Unvaccinated | 5 | 68 | … | … | … | … |
| Negative | ||||||
| Vaccinated | 1382 | 276 | 87 | 97 (96–98) | 73 (71–76) | 0.72 (.69–.75) |
| Unvaccinated | 48 | 758 | … | … | … | … |
| Vaccinated ≥14 d before illness onsetb | ||||||
| Yes | 1233 | 32 | 94 | 96 (95–97) | 72 (64–80) | 0.62 (.55–.69) |
| No | 55 | 82 | ||||
Abbreviations: CI, confidence interval.
The 2015–2016 influenza vaccine question was stratified by influenza test results.
Among the 1432 children who were vaccinated according to parental report and documented records, 30 were excluded because of unknown timing of vaccination by both sources.
The proportion of children vaccinated according to either parental report or documented report varied by site, age, race/ethnicity, health insurance status, underlying medical conditions, mother’s education, mother’s age, and household size (Table 1). In all 7 sites, parental report indicated higher vaccination than did documented records. The percentage agreement between parental report and documented records on the child’s influenza vaccination status for the 2015–2016 season, and timing of vaccination to illness onset was 87%, and 94%, respectively (Table 2). The overall sensitivity of parental report was 96% (95% confidence interval [CI], 95%–97%) and overall specificity was 74% (95% CI, 72%–77%). The agreement between parental report of child influenza vaccination status and documented records was substantial (κ = 0.73 [95% CI, .70–.75]). The specificity of parental report was lower for children who tested negative for influenza compared with children who tested positive (73% and 87%, respectively).
Predictors of Discordance
In unadjusted analyses, study site, influenza season enrollment status, age, sex, race/ethnicity, health insurance status, mother’s age, and mother’s education were associated with discordance in children’s influenza vaccination status between parental and documented records, with P values ≤ .20 (Table 3). In the adjusted analysis, child’s age 5–8 years, and enrollment at sites other than Seattle, Rochester, or Cincinnati had significantly higher odds of discordance (P < .05).
Table 3.
Predictors of Discordance in Influenza Vaccination Status Obtained by Parental Report Versus Documented Records Among Children Hospitalized With Acute Respiratory Illness—New Vaccine Surveillance Network, 2015–2016
| Characteristic | Total No.a | Discordant Vaccination Status, No. (Row %) | Unadjusted OR (95% CI) | P Valueb | Multivariate Logistic Regressionc, OR (95% CI) | P Valued |
|---|---|---|---|---|---|---|
| All children | 2534 | 332 (13) | ||||
| Study site | <.01 | .03 | ||||
| Cincinnati, Ohio | 421 | 58 (14) | 2.35 (1.38–3.99) | 1.71 (.97–2.98) | ||
| Houston, Texas | 619 | 81 (13) | 2.21 (1.33–3.68) | 1.86 (1.10–3.14) | ||
| Kansas City, Missouri | 301 | 45 (15) | 2.58 (1.49–4.49) | 1.89 (1.06–3.36) | ||
| Nashville, Tennessee | 290 | 51 (18) | 3.14 (1.82–5.41) | 2.26 (1.28–4.01) | ||
| Oakland, California | 372 | 58 (16) | 2.72 (1.59–4.62) | 2.11 (1.23–3.63) | ||
| Rochester, New York | 217 | 19 (9) | 1.41 (.73–2.71) | 1.09 (.55–2.16) | ||
| Seattle, Washington | 314 | 20 (6) | Reference | Reference | ||
| Enrolled during influenza season | .01 | .08 | ||||
| No | 997 | 152 (15) | 1.36 (1.08–1.71) | 1.26 (.97–1.63) | ||
| Yes | 1537 | 180 (12) | Reference | Reference | ||
| Age group | <.01 | <.01 | ||||
| 6–23 mo | 1066 | 108 (10) | Reference | Reference | ||
| 2–4 y | 664 | 86 (13) | 1.32 (.98–1.78) | 1.35 (.99–1.83) | ||
| 5–8 y | 446 | 91 (20) | 2.27 (1.68–3.08) | 2.25 (1.64–3.08) | ||
| 9–17 y | 358 | 47 (13) | 1.34 (.93–1.93) | 1.41 (.93–2.12) | ||
| Sex | .10 | .09 | ||||
| Male | 1464 | 178 (12) | Reference | Reference | ||
| Female | 1070 | 154 (14) | 1.22 (.96–1.53) | 1.23 (.97–1.55) | ||
| Race/ethnicity | <.01 | .39 | ||||
| White, non-Hispanic | 839 | 86 (10) | Reference | Reference | ||
| Black, non-Hispanic | 662 | 110 (17) | 1.75 (1.29–2.36) | 1.26 (.90–1.75) | ||
| Other, non-Hispanic | 291 | 39 (13) | 1.36 (.90–2.03) | 1.25 (.82–1.91) | ||
| Hispanic | 742 | 97 (13) | 1.32 (.97–1.79) | 1.02 (.70–1.48) | ||
| Health insurance status | <.01 | .11 | ||||
| Public/self-pay/unknown | 1612 | 240 (15) | 1.58 (1.22–2.04) | 1.28 (.95–1.73) | ||
| Private/private and public | 922 | 92 (10) | Reference | Reference | ||
| Underlying medical conditionse | .38 | … | ||||
| None | 998 | 138 (14) | 1.11 (.88–1.40) | |||
| ≥1 | 1536 | 194 (13) | Reference | |||
| Interviewee relationship to child | .27 | … | ||||
| Mother/stepmother | 2181 | 277 (13) | Reference | |||
| Father/stepfather | 320 | 51 (16) | 1.30 (.94–1.80) | |||
| Other relative | 33 | 4 (12) | 0.95 (.33–2.72) | |||
| Mother’s education | .05 | .69 | ||||
| Less than high school | 392 | 60 (15) | 1.42 (1.02–1.98) | 1.18 (.81–1.73) | ||
| High school/GED | 1042 | 148 (14) | 1.30 (1.01–1.68) | 1.08 (.81–1.44) | ||
| College/graduate degree | 1100 | 124 (11) | Reference | Reference | ||
| Mother’s age | .12 | .36 | ||||
| 18–29 y | 1051 | 154 (15) | 1.30 (1.01–1.67) | 1.22 (.93–1.61) | ||
| 30–39 y | 1130 | 132 (12) | Reference | Reference | ||
| ≥40 y | 353 | 46 (13) | 1.13 (.79–1.62) | 1.10 (.75–1.60) | ||
| Household size | .38 | … | ||||
| Child plus 1–2 people | 606 | 79 (13) | Reference | |||
| Child plus 3 people | 747 | 88 (12) | 0.89 (.64–1.23) | |||
| Child plus ≥4 people | 1181 | 165 (14) | 1.08 (.81–1.45) |
Abbreviations: CI, confidence interval; GED, general equivalency diploma; OR, odds ratio (odds of discordance in vaccination status between parental report and documented records).
Sixty-three children were excluded from this analysis as they had unknown demographic characteristics, except for health insurance status.
P values from the univariate logistic regression analysis.
Regression model included covariates with P values ≤ .20 from the univariate logistic regression.
P values from the multivariate regression model.
Underlying medical conditions included chronic pulmonary/airway, cardiac, gastrointestinal, kidney, endocrine, neurologic/neuromuscular, hematologic/oncologic, genetic/metabolic, immunocompromising conditions, and prematurity (history obtained for children aged <2 years).
Vaccine Effectiveness Using Parent-Reported Vaccination History
A total of 1436 children were eligible for the VE analysis, and of those, 124 (9%) tested positive for influenza (Figure 1 and Supplementary Table 1). Influenza-positive cases differed from controls by race/ethnicity, health insurance status, duration of illness prior to enrollment, and vaccination status. Among the influenza-positive cases, 53 (43%) were vaccinated per parental report compared to 49 (40%), according to documented records (Table 4). After adjusting for age, calendar time, study site, health insurance status, race/ethnicity, days from illness onset to enrollment, and underlying medical conditions, VE against influenza A or B was 61% (95% CI, 43%–74%) using parental report, and 55% (95% CI, 33%–69%) using documented records (Table 4).
Table 4.
Influenza Vaccine Effectiveness for Prevention of Influenza A or B–Associated Hospitalizations in Children, Using Parental Report and Documented Record for Vaccination History—New Vaccine Surveillance Network, 2015–2016
| Influenza Positive |
Influenza Negative |
Unadjusted VE |
Adjusted VEa |
|
|---|---|---|---|---|
| Source of Vaccination History | Vaccinated/Total, No. (%) | Vaccinated/Total, No. (%) | % (95% CI) | % (95% CI) |
| Parental report | 53/124 (43) | 894/1312 (68) | 65 (49–76) | 61 (43–74) |
| Documented records | 49/124 (40) | 796/1312 (61) | 58 (38–71) | 55 (33–69) |
Abbreviations: CI, confidence interval; VE, vaccine effectiveness.
VE estimates were adjusted for age as a continuous variable, study site, calendar time (enrollment month), race/ethnicity, days from illness onset to enrollment (0–2, 3–4, 5–7, and 8–10 days), and health insurance status (public/self-pay/unknown or private/private and public).
DISCUSSION
Parental report of influenza vaccination is readily available and offers a resource-efficient method for assessing influenza VE in observational studies. We demonstrated that in a single influenza season, parental report was reliable and concordant with vaccination status from documented records in hospitalized children with acute respiratory illness at 7 US pediatric hospitals. When using documented records as the gold standard, parents in our study were able to recall their child being vaccinated during the current influenza season with high sensitivity (96%) and recall their child not receiving the vaccine with moderate specificity (74%). These findings were similar to previous studies that evaluated the validity of parental report for influenza vaccination [19, 24, 31] as well as other routine childhood immunizations [28]. Our study also shows high percentage agreement and substantial reliability when parents were asked about the timing of vaccination relative to illness onset.
Study site and age were associated with discordance in vaccination history reported by parents. Parents of children enrolled in Houston, Kansas City, Nashville, and Oakland had increased odds of discordant parental vaccination report compared with documented records. A prior study has demonstrated that the Washington State IIS is highly complete and accurate for receipt and dates of vaccination, although this alone does not provide an explanation for concordance with parental history [32]. It may also be that differences in parental education, socioeconomic status, or other unmeasured factors contribute to differences in concordance by site. Parents of children ≥2 years of age were more likely to be discordant in reporting their child’s vaccination status for the current influenza season, especially children aged 5–8 years. This result indicates that despite the numerous recommended childhood routine immunizations for children <15 months of age [33], parents of younger children are able to provide accurate influenza vaccination history, which contrasts with the findings of Nowalk et al [34]. It is possible that parents answer affirmatively to vaccination because of their assumption that pediatricians are following vaccination recommendations, thus resulting in high sensitivity and lower specificity, especially in communities with higher vaccination rates. In addition, there are other factors that could explain discordance in parental report such as misclassification of vaccines, social desirability, and interviewer biases.
Despite specificity of 74%, the percentage of children vaccinated among influenza cases and influenza controls were moderately similar when using parental report or documented record for vaccination history (κ = 0.73 [95% CI, .70–.75]). Our VE estimates using parental report (61%) and documented records (55%) as the source of vaccination yielded relatively similar results. In a study by Ferdinands et al [35], which assessed VE against any influenza virus among hospitalized adults ≥18 years of age using different definitions of vaccine status, VE was much lower when using documented records (34%) compared to self-report (52%). Validity of vaccination status using documented records or self-report may differ between adults and children. Older adults tend to have more underlying medical conditions, and vaccination can be administered in more nontraditional settings (eg, pharmacies, workplace, nursing homes, churches) compared with children, thus complicating vaccination ascertainment.
These data should be interpreted in the context of strengths and limitations. Our study fills a gap in recent literature on the validity of obtaining influenza vaccination history through parental report among hospitalized children. Our study also encompasses a diverse population in 7 hospitals across the US. Limitations to this study include unmeasured site variability in the vaccine verification process, as some sites relied on various sources (eg, IIS, traditional or nontraditional providers, and medical records) especially if one source was incomplete or no vaccination records were found, whereas others more routinely acquired vaccination records from the state IIS or providers alone. Thus, we were unable to evaluate the relationship between documentation source and discordance, especially when multiple sources were used. Furthermore, depending on access to provider and registry procedures, the yield of the documented verification may vary. For example, sites may have extended beyond the minimum number of attempts to obtain immunization records. The completeness and accuracy of each sites’ IIS or provider records may also vary. Documentation from other nontraditional providers may have been incomplete. Additionally, validity of our findings may not be applicable to outpatients because recall of vaccination may be different during illness requiring hospitalization. Our results for hospitalized children thus may not be generalizable to other communities or outpatient settings.
In conclusion, our study demonstrated that parental report, which is easily accessible, was a fairly reliable source for influenza vaccination status among this population of hospitalized children during the 2015–2016 influenza season. We documented that under certain circumstances, parental report can be a reasonably valid source of exposure ascertainment in VE studies of hospitalized children and in clinical settings. Studies from additional seasons and sites are needed to confirm the validity of parental report for observational studies of influenza vaccine effectiveness.
Supplementary Material
Acknowledgments.
We thank the children and parents who participated in this study. We also acknowledge Craig McGowan and Daniella Figueroa-Downing (Atlanta, Georgia); Sahle Amsalu, Brittney Cassell, Jacqueline Della Torre, Krista Doerflein, Vanessa Florian, Marilyn Rice, Chelsea Rohlfs, Michelle Roth, Joseph Sorter, Katie Weiskircher, and Michael Whalen (Cincinnati, Ohio); Gina Weddle (Kansas City, Missouri); Christina Albertin, Wende Fregoe, Joshua Aldred, Theodore Pristash, and Miranda Marchand (Rochester, New York); Bonnie Strelitz, Kirsten Lacombe, Ashley Akramoff, Jennifer Baxter, Rachel Buchmeier, Kaitlin Cappetto, Leanne Kehoe, Sarah Korkowski, Larissa Molina Huezo, Katarina Ost, Hannah Parish, Hanna Schlaack, Hannah Smith, Sarah Steele, Yekaterina Tokareva, Ariundari Tsogoo, and Emily Walter (Seattle, Washington); and all of the members of the New Vaccine Surveillance Network (NVSN).
New Vaccine Surveillance Network. The additional members of the New Vaccine Surveillance Network author group for this study are Elizabeth P. Schlaudecker (Cincinnati, Ohio); Pedro A. Piedra and Flor M. Munoz (Houston, Texas); Mary Moffat, Jennifer E. Schuster, Barbara A. Pahud (Kansas City, Missouri).
Disclaimer. The finding and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention (CDC).
Financial support. This work was supported by the Centers for Disease Control and Prevention (cooperative agreement number CDC-RFA-IP16-004).
Potential conflicts of interest. C. J. H. and B. A. P. are both investigators on pneumococcal, meningococcal, and rotavirus vaccine studies at Children’s Mercy Hospital–Kansas City, which receives grant funding from GSK, Merck, and Pfizer. B. A. P. has served as a consultant for Merck, Pfizer, GSK, and Sanofi. J. E. S. has received grant funding from Merck. J. A. E. has served as a consultant for Sanofi Pasteur and Meissa Vaccines, and has received research support from AstraZeneca, GlaxoSmithKline, and Merck. N. B. H. has received an independent investigator-initiated grant from Sanofi and Quidel; has received consulting fees from Karius; and has received vaccine donation and hemagglutination inhibition testing from Sanofi for influenza studies. J. V. W. serves on the scientific advisory board of Quidel and an independent data monitoring committee for GlaxoSmithKline. All other authors report no potential conflicts of interest.All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.
Footnotes
Supplementary Data
Supplementary materials are available at the Journal of The Pediatric Infectious Diseases Society online (http://jpids.oxfordjournals.org). Supplementary materials consist of data provided by the author that are published to benefit the reader. The posted materials are not copyedited. The contents of all supplementary data are the sole responsibility of the authors. Questions or messages regarding errors should be addressed to the author.
References
- 1.Chaves SS, Perez A, Farley MM, et al. ; Influenza Hospitalization Surveillance Network. The burden of influenza hospitalizations in infants from 2003 to 2012, United States. Pediatr Infect Dis J 2014; 33:912–9. [DOI] [PubMed] [Google Scholar]
- 2.Lafond KE, Nair H, Rasooly MH, et al. Global role and burden of influenza in pediatric respiratory hospitalizations, 1982–2012: a systematic analysis. PLoS Med 2016; 13:e1001977. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Rolfes MA, Foppa IM, Garg S, et al. Annual estimates of the burden of seasonal influenza in the United States: a tool for strengthening influenza surveillance and preparedness. Influenza Other Respir Viruses 2018; 12:132–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Chung JR, Rolfes MA, Flannery B, et al. Effects of influenza vaccination in the United States during the 2018–2019 influenza season [manuscript published online ahead of print 6 January 2020]. Clin Infect Dis 2020. doi: 10.1093/cid/ciz1244. [DOI]
- 5.Fiore AE, Shay DK, Broder K, et al. ; Centers for Disease Control and Prevention (CDC); Advisory Committee on Immunization Practices (ACIP). Prevention and control of influenza: recommendations of the Advisory Committee on Immunization Practices (ACIP), 2008. MMWR Recomm Rep 2008; 57:1–60. [PubMed] [Google Scholar]
- 6.Campbell AJ, Grohskopf LA. Updates on influenza vaccination in children. Infect Dis Clin 2018; 32:75–89. [DOI] [PubMed] [Google Scholar]
- 7.Grohskopf LA, Alyanak E, Broder KR, et al. Prevention and control of seasonal influenza with vaccines: recommendations of the Advisory Committee on Immunization Practices—United States, 2019–20 influenza season. MMWR Recomm Rep 2019; 68:1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Flannery B, Kondor RJG, Chung JR, et al. Spread of antigenically drifted influenza A(H3N2) viruses and vaccine effectiveness in the United States during the 2018–2019 season. J Infect Dis 2020; 221:8–15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Rolfes MA, Flannery B, Chung J, et al. Effects of influenza vaccination in the United States during the 2017–2018 influenza season. Clin Infect Dis 2019; 69:1845–53. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Flannery B, Chung JR, Monto AS, et al. Influenza vaccine effectiveness in the United States during the 2016–2017 season. Clin Infect Dis 2019; 68:1798–806. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Jackson ML, Chung JR, Jackson LA, et al. Influenza vaccine effectiveness in the United States during the 2015–2016 season. N Engl J Med 2017; 377:534–43. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Ferdinands JM, Olsho LE, Agan AA, et al. ; Pediatric Acute Lung Injury and Sepsis Investigators (PALISI) Network. Effectiveness of influenza vaccine against life-threatening RT-PCR-confirmed influenza illness in US children, 2010–2012. J Infect Dis 2014; 210:674–83. [DOI] [PubMed] [Google Scholar]
- 13.Zimmerman RK, Nowalk MP, Chung J, et al. 2014–2015 influenza vaccine effectiveness in the United States by vaccine type. Clin Infect Dis 2016; 63:ciw635. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Flannery B, Reynolds SB, Blanton L, et al. Influenza vaccine effectiveness against pediatric deaths: 2010–2014. Pediatrics 2017; 139:e20164244. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Poehling KA, Caspard H, Peters TR, et al. 2015–2016 vaccine effectiveness of live attenuated and inactivated influenza vaccines in children in the United States. Clin Infect Dis 2018; 66:665–72. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Stokley S, Rodewald LE, Maes EF. The impact of record scattering on the measurement of immunization coverage. Pediatrics 2001; 107:91–6. [DOI] [PubMed] [Google Scholar]
- 17.Althubaiti A. Information bias in health research: definition, pitfalls, and adjustment methods. J Multidiscip Healthc 2016; 9:211–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Kahn K, Black C, Ding H. Influenza and Tdap vaccination coverage among pregnant women—United States, April 2018. MMWR Recomm Rep 2018; 67:1055–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Brown C, Clayton-Boswell H, Chaves SS, et al. ; New Vaccine Surveillance Network (NVSN). Validity of parental report of influenza vaccination in young children seeking medical care. Vaccine 2011; 29:9488–92. [DOI] [PubMed] [Google Scholar]
- 20.Suarez L, Simpson DM, Smith DR. Errors and correlates in parental recall of child immunizations: effects on vaccination coverage estimates. Pediatrics 1997; 99:E3. [DOI] [PubMed] [Google Scholar]
- 21.Shinall MC Jr, Plosa EJ, Poehling KA. Validity of parental report of influenza vaccination in children 6 to 59 months of age. Pediatrics 2007; 120:e783–7. [DOI] [PubMed] [Google Scholar]
- 22.Irving SA, Donahue JG, Shay DK, et al. Evaluation of self-reported and registry-based influenza vaccination status in a Wisconsin cohort. Vaccine 2009; 27:6546–9. [DOI] [PubMed] [Google Scholar]
- 23.King JP, McLean HQ, Belongia EA. Validation of self-reported influenza vaccination in the current and prior season. Influenza Other Respir Viruses 2018; 12:808–13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Poehling KA, Vannoy L, Light LS, et al. Assessment of parental report for 2009–2010 seasonal and monovalent H1N1 influenza vaccines among children in the emergency department or hospital. Acad Pediatr 2012; 12:36–42. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Feldstein LR, Ogokeh C, Rha B, et al. Vaccine effectiveness against influenza hospitalization among children in the United States, 2015–2016. J Pediatric Infect Dis Soc 2021; 10:75–82. [DOI] [PubMed] [Google Scholar]
- 26.Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33:159–74. [PubMed] [Google Scholar]
- 27.McHugh ML. Interrater reliability: the kappa statistic. Biochem Med (Zagreb) 2012; 22:276–82. [PMC free article] [PubMed] [Google Scholar]
- 28.Bercovich S, Anis E, Kassem E, et al. Validation of parental reports of rotavirus vaccination of their children compared to the national immunization registry. Vaccine 2019; 37:2791–6. [DOI] [PubMed] [Google Scholar]
- 29.Binyaruka P, Borghi J. Validity of parental recalls to estimate vaccination coverage: evidence from Tanzania. BMC Health Serv Res 2018; 18:440. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Hirth J, Kuo YF, Laz TH, et al. Concordance of adolescent human papillomavirus vaccination parental report with provider report in the National Immunization Survey-Teen (2008–2013). Vaccine 2016; 34:4415–21. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Tuckerman J, Crawford NW, Lynch J, Marshall HS. Are children with special risk medical conditions receiving influenza vaccination? Validity of parental and provider report, and to a National Immunisation Register. Hum Vaccines Immunother 2019; 15:951–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Jackson ML, Henrikson NB, Grossman DC. Evaluating Washington State’s immunization information system as a research tool. Acad Pediatr 2014; 14:71–6. [DOI] [PubMed] [Google Scholar]
- 33.Committee on Infectious Diseases. Recommended childhood and adolescent immunization schedule: United States, 2020. Pediatrics 2020; 145:e20193995. [DOI] [PubMed] [Google Scholar]
- 34.Nowalk MP, Zimmerman RK, Lin CJ, et al. Parental perspectives on influenza immunization of children aged 6 to 23 months. Am J Prev Med 2005; 29:210–4. [DOI] [PubMed] [Google Scholar]
- 35.Ferdinands J, Gaglani M, Martin E, et al. Prevention of influenza hospitalization among adults in the US, 2015–16: results from the US Hospitalized Adult Influenza Vaccine Effectiveness Network (HAIVEN). J Infect Dis 2019; 220:1265–75. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.