Abstract
Background:
Despite safe and effective childhood immunizations, decreased acceptance of vaccines has become an emerging global problem. The WHO SAGE Working Group on Vaccine Hesitancy developed a common diagnostic tool, the Vaccine Hesitancy Scale (VHS), to identify and compare hesitancy in different global settings. We field tested the VHS in rural and urban Guatemala.
Methods:
We analyzed data from the enrollment visit of a study conducted at four public health clinics in Guatemala. Infants ages 6 weeks-6 months presenting for their first wellness visit were enrolled March-November 2016. Parents completed a demographic survey that included the 10 dichotomous and 10 Likert scale VHS questions. Chi-square or Fisher’s exact for categorical and ANOVA test for continuous variables were used to assess significance levels in survey differences. We conducted a factor analysis to assess the Likert scale questions.
Results:
Of 1088 families screened, 871 were eligible and 720 (82.7%) participated. No parent had ever refused a vaccination, and only eight parents (1.1%) had been reluctant or hesitated to get a vaccination for their children. However, only 40.8% (n=294) of parents said that they think most parents like them have their children vaccinated with all the recommended vaccines. Factor analysis identified two underlying constructs that had eigenvalues of 1.0 or greater and a substantive lack of variability in response across the Likert scale. There were consistent differences between how study clinics responded to the ordinal scaling.
Conclusion:
Our results suggest problems with interpretation of the VHS, especially in the presence of vaccine shortages and using a Likert scale that does not resonate across diverse cultural settings. Our factor analysis suggests that the Likert scale items are more one-dimensional and do not represent the multiple constructs of vaccine hesitancy. We suggest more work is needed to refine this survey for improved reliability and validity.
Keywords: childhood immunization, vaccine hesitancy, survey, SAGE, WHO, Guatemala
1. Introduction
Immunization is one of the greatest public health achievements, protecting children from serious illness and saving millions of lives every year. Despite a growing number of safe and effective childhood vaccines, decreased acceptance of specific vaccines or vaccination programs by both individuals and communities has become an emerging problem in high-income [1–3] as well as low-and middle-income (LMIC) countries [4]. The reasons for this are multifaceted, culture-specific, and often not completely understood. In 2012, the World Health Organization (WHO) Strategic Advisory Group of Experts (SAGE) on Immunization recognized this global challenge and established the Working Group (WG) on Vaccine Hesitancy [5]. The WG contributed to several important areas of this relatively new field of research [6–12] and created the following definition: “Vaccine hesitancy refers to delay in acceptance or refusal of vaccination despite availability of vaccination services. Vaccine hesitancy is complex and context specific, varying across time, place and vaccines. It is influenced by factors such as complacency, convenience and confidence” [7]. Vaccine hesitancy, therefore, occurs on a continuum between those who undoubtedly accept all vaccines to those who undoubtedly refuse all vaccines. The vaccine hesitant individual remains somewhere between these two extremes, including those who refuse certain vaccines while accepting others, delay vaccinations, or accept vaccinations but have concerns.
In order to inform interventions that improve vaccine coverage, it is important to understand the complex and interplaying factors that influence vaccination decisions and the determinants of vaccine hesitancy in a specific population. No uniform, global metric for quantifying vaccine hesitancy currently exists. While several cross-sectional surveys have been used to measure parental attitudes, beliefs, and behaviors surrounding vaccination [13–20], most of these surveys have only focused on a limited number of factors influencing vaccine hesitancy, have not had formal testing of validity and reliability, and have been conducted in high-income settings. One of the major tasks of the WG was to develop a common diagnostic tool, the Vaccine Hesitancy Scale (VHS), to identify and compare hesitancy in different global settings [12]. VHS questions are either closed-ended, Likert scale, or open-ended in nature. These questions were developed in conjunction with a global pilot test of indicators for vaccine hesitancy and a literature review for similar survey tools. Questions were further adapted from the Parent Attitudes About Childhood Vaccines (PACV) survey previously developed by Opel et al. [21] and found to be valid and reliable in a high-income population [22,23]. The PACV survey has also been adapted for use with adolescent vaccines [24], influenza vaccination [25], and a multi-ethnic Malaysian population [26]. In their work to develop a common survey tool that could be used globally to identify and compare vaccine hesitancy, the WG further adapted the PACV survey to have more global relevance, especially for LMICs.
While this effort has generated a useful initial tool to assess vaccine hesitancy and its relevance in delayed or incomplete vaccination, the WG encouraged further evaluation of the VHS to determine whether it offers both a valid and reliable estimate of vaccine hesitancy across diverse cultural settings. In the context of a larger randomized intervention study exploring the impact of a Short Message Service (SMS) technology to provide families with text message reminders for childhood immunizations, we conducted baseline assessments that incorporated both the closed-ended and Likert scale VHS questions. Our study team has worked in collaboration with the Ministry of Health of Guatemala [27]. The current study analyzes the reliability and validity of the VHS measure applied in urban and rural Guatemala using a factor analysis as described in more detail below. Our objective is to provide insight into the shared understanding of the VHS construct using the tool in diverse global settings.
2. Materials and Methods
2.1. Study Design
We analyzed cross-sectional data collected at the enrollment visit from a study conducted at four public health clinics of the Ministry of Public Health and Social Assistance in Guatemala. The government clinics serve a low-income population with two of the clinics located in an urban setting surrounding Guatemala City (Zona 11 and Villa Nueva) and two in the rural southwest region of the country (Colomba and Coatepeque, Quetzaltenango). Participation was voluntary and patients were not given any incentives. Of note, Guatemala experienced significant political instability during our study period, which led to considerable vaccine shortages experienced by all of our clinics. The Colorado Multiple Institutional Review Board, Universidad del Valle Ethics Committee, and Guatemala National Ethics Committee of the Ministry of Public Health and Social Assistance approved the research. Written informed consent was obtained from all parents or guardians (henceforth referred to as parent).
2.2. Participants
Eligible participants included parents of infants between the ages of 6 weeks to 6 months presenting for their first wellness visit. At least one parent needed to own an active mobile phone capable of receiving SMS, be able to use SMS, and be literate and able to decipher the messages for themselves or by a surrogate in the household. Children were excluded if they were not medically cleared to receive vaccines, the study clinic was not the patient’s primary clinic, the consenting parent was under 18 years of age, or the parent did not speak Spanish. Equal numbers of rural and urban participants were enrolled.
2.3. Data collection
Data collection occurred between March to November 2016. Parents completed a demographic survey that included the 10 dichotomous (yes/no) and 10 Likert scale (strongly agree, agree, neither agree nor disagree, disagree, or strongly disagree) VHS questions. In order to maintain the original instrument, no modifications were made and no items were added. The English language survey was translated into Spanish by a native Guatemalan. A study nurse assigned to each clinic verbally administered the surveys to participants in a quiet and confidential location. Study data were collected and managed using REDCap (Research Electronic Data Capture), a secure and web-based electronic data capture tool hosted at the University of Colorado Denver [28].
2.4. Statistical analysis
We analyzed parent demographics and survey responses using descriptive statistics. Chi-square or Fisher’s exact for categorical and ANOVA test for continuous variables were used to assess significance levels in demographic and survey differences. All statistical tests were considered to be significant at a two-tailed p value less than 0.05. To analyze reliability and validity of the Likert scale questions, we conducted an exploratory factor analysis (EFA). We first examined correlations between all survey items. We then completed a factor extraction. This is an important step to explore statistically possible linear combinations of variables and whether they represent the domains of complacency, convenience, and confidence as specified in the WG vaccine hesitancy definition [7]. We then conducted a principle component analysis, a process that helps to group those items together into factors that will maximize the variance in the data using a measure known as an eigenvalue. We retained only the factors that have eigenvalues of at least 1.0 [29]. We then explored the strength of the correlation of items within each factor and considered if any of the items within a factor might be redundant using a varimax rotation. This is a process that allows researchers to simplify factor analysis results by identifying those items that are most correlated with a factor, allowing for parsimony in the final solution. This approach also allows for the factors selected to account for a large percentage of the total variation of the measured survey questions, ideally 50% or more [30]. All analyses were conducted using SAS (SAS 9.4, SAS Institute, Cary, NC).
3. Results
Of 1088 families screened, 871 were eligible and 720 (82.7%) participated. Table 1 summarizes the demographic characteristics of the study participants, showing high literacy overall and high engagement from fathers who were also largely employed.
Table 1.
Demographic characteristics of study participants
| Rural Clinics | Urban Clinics | |||||
|---|---|---|---|---|---|---|
| Characteristic | Coatepeque n=230, n (%) | Colomba n=130, n (%) | Villa Nueva n=180, n (%) | Zona 11 n=180, n (%) | p valuea | Study Cohort n=720, n (%) |
| CHILD CHARACTERISTICS | ||||||
| Child’s gender | ||||||
| Male | 122 (53.0) | 62 (47.7) | 85 (47.2) | 89 (49.4) | 0.64 | 358 (49.7) |
| Female | 108 (47.0) | 68 (52.3) | 95 (52.8) | 91 (50.6) | 362 (50.3) | |
| MOTHER CHARACTERISTICS | ||||||
| Mother’s age (years) | ||||||
| Mean (SD) | 23.8 (±5.0) | 25.1 (±5.3) | 25.1 (±5.9) | 25.6 (±5.7) | 0.004 | 24.8 (±5.5) |
| Mother is able to read and write | ||||||
| No | 7 (3.0) | 3 (2.3) | 1 (0.6) | 5 (2.8) | 0.35 | 16 (2.2) |
| Yes | 223 (97.0) | 127 (97.7) | 179 (99.4) | 175 (97.2) | 704 (97.8) | |
| Mother’s education | ||||||
| No education | 0(0) | 6 (4.6) | 0(0) | 4(2.2) | <.0001 | 10 (1.4) |
| Completed or some primary education | 53 (23) | 57 (43.8) | 45 (25.0) | 29 (16.1) | 184 (25.6) | |
| Completed or some secondary education | 50 (21.7) | 25 (19.2) | 51 (28.3) | 47 (26.1) | 173 (24.0) | |
| Completed or some higher education | 127 (55.2) | 42 (32.3) | 84 (46.7) | 100 (55.6) | 353 (49.0) | |
| Mother’s employment status | ||||||
| Unemployed | 145 (63.0) | 112 (86.2) | 144 (80.0) | 113 (62.8) | <.0001 | 514 (71.4) |
| Employed | 85 (37.0) | 18 (13.8) | 36 (20.0) | 67 (37.2) | 206 (28.6) | |
| Mother owns a mobile phoneb | ||||||
| No | 16 (7.4) | 8 (6.2) | 10 (5.6) | 10 (5.6) | 0.93 | 44 (6.3) |
| Yes | 213 (92.6) | 122 (93.8) | 170 (94.4) | 170 (94.4) | 675 (93.8) | |
| FATHER CHARACTERISTICSc | ||||||
| Father is involved in caring | ||||||
| No | 48 (20.9) | 21 (16.2) | 27 (15.0) | 23 (12.8) | 0.15 | 119 (16.5) |
| Yes | 182 (79.1) | 109 (83.8) | 153 (85.0) | 157 (87.2) | 601 (83.5) | |
| Father’s age (years) | ||||||
| Mean (SD) | 28.6 (±7.0) | 27.0 (±5.8) | 28.1 (±6.6) | 29.3 (±7.2) | 0.04 | 28.4 (±6.8) |
| Father is able to read and write | ||||||
| No | 3(1.6) | 5 (4.6) | 1 (0.7) | 1 (0.6) | 0.07 | 10 (1.7) |
| Yes | 179 (98.4) | 104 (95.4) | 152 (99.3) | 156 (99.4) | 591 (98.3) | |
| Father’s education | ||||||
| No education | 2(1.1) | 5 (4.6) | 1 (0.7) | 1 (0.6) | <.0001 | 9(1.5) |
| Completed or some primary education | 19 (10.4) | 31 (28.4) | 29 (19) | 24 (15.3) | 103 (17.1) | |
| Completed or some secondary education | 35 (19.2) | 31 (28.4) | 38 (24.8) | 40 (25.5) | 144 (24) | |
| Completed or some higher education | 126 (69.2) | 42 (38.5) | 85 (55.6) | 92 (58.6) | 345 (57.4) | |
| Father’s employment | ||||||
| Unemployed | 5(2.7) | 0(0) | 2(1.3) | 1 (0.6) | 0.26 | 8(1.3) |
| Employed | 177 (97.3) | 109 (100) | 151 (98.7) | 156 (99.4) | 593 (98.7) | |
| Father owns a mobile phoneb | ||||||
| No | 99 (66.0) | 38 (34.9) | 40 (26.1) | 44 (28.2) | <.0001 | 221 (38.9) |
| Yes | 51 (34.0) | 71 (65.1) | 113 (73.9) | 112 (71.8) | 347 (61.1) | |
| HOUSEHOLD CHARACTERISTICS | ||||||
| Number of children living in the house | ||||||
| Mean (SD) | 1.8 (±0.8) | 2.2 (±1.3) | 1.8 (±0.9) | 1.9 (±1.1) | 0.004 | 1.9 (±1.0) |
| Family monthly income leveld | ||||||
| < Q1000 | 76 (33) | 58 (44.6) | 63 (35) | 10 (5.6) | <.0001 | 207 (28.8) |
| Q1001– 2000 | 30 (13) | 51 (39.2) | 42 (23.3) | 32 (17.8) | 155 (21.5) | |
| Q2001– 3000 | 53 (23) | 16 (12.3) | 36 (20.0) | 61 (33.9) | 166 (23.1) | |
| Q3001– 4000 | 38 (16.5) | 4 (3.1) | 18 (10.0) | 43 (23.9) | 103 (14.3) | |
| >Q4001 | 33 (14.3) | 1 (0.8) | 21 (11.7) | 34 (18.9) | 89 (12.4) | |
| Landline present in home | ||||||
| No | 123 (53.5) | 112 (86.2) | 155 (86.1) | 149 (82.8) | <.0001 | 539 (74.9) |
| Yes | 107 (46.5) | 18 (13.8) | 25 (13.9) | 31 (17.2) | 181 (25.1) | |
| Family owns a radio | ||||||
| No | 97 (42.2) | 50 (38.5) | 40 (22.2) | 30 (16.7) | <.0001 | 217 (30.1) |
| Yes | 133 (57.8) | 80 (61.5) | 140 (77.8) | 150 (83.3) | 503 (69.9) | |
| Family owns a television | ||||||
| No | 13 (5.7) | 26 (20) | 6 (3.3) | 8 (4.4) | <.0001 | 53 (7.4) |
| Yes | 217 (94.3) | 104 (80) | 174 (96.7) | 172 (95.6) | 667 (92.6) | |
| Family owns a refrigerator | ||||||
| No | 33 (14.3) | 68 (52.3) | 44 (24.4) | 58 (32.2) | <.0001 | 203 (28.2) |
| Yes | 197 (85.7) | 62 (47.7) | 136 (75.6) | 122 (67.8) | 517 (71.8) | |
| Family owns an electric or gas stove | ||||||
| No | 20 (8.7) | 54 (41.5) | 1 (0.6) | 4 (2.2) | <.0001 | 79 (11) |
| Yes | 210 (91.3) | 76 (58.5) | 179 (99.4) | 176 (97.8) | 641 (89) | |
| Family owns a car | ||||||
| No | 180 (78.3) | 108 (83.1) | 107 (59.4) | 102 (56.7) | <.0001 | 497 (69) |
| Yes | 50 (21.7) | 22 (16.9) | 73 (40.6) | 78 (43.3) | 223 (31) | |
| Number of amenitiese | ||||||
| < 2 | 30 (13) | 64 (49.2) | 18 (10) | 19 (10.6) | <.0001 | 131 (18.2) |
| 3 | 74 (32.2) | 21 (16.2) | 42 (23.3) | 40 (22.2) | 177 (24.6) | |
| 4 | 90 (39.1) | 34 (26.2) | 56 (31.1) | 58 (32.2) | 238 (33.1) | |
| 5 | 36 (15.7) | 11 (8.5) | 64 (35.6) | 63 (35) | 174 (24.2) | |
| Food insecurity scoref | ||||||
| 0 | 203 (88.3) | 121 (93.1) | 111 (61.7) | 140 (77.8) | <.0001 | 575 (79.9) |
| 1 | 13(5.7) | 2(1.5) | 4(2.2) | 13 (7.2) | 32 (4.4) | |
| 2 | 3(1.3) | 1 (0.8) | 10 (5.6) | 5 (2.8) | 19 (2.6) | |
| 3 | 6(2.6) | 0(0) | 4(2.2) | 3(1.7) | 13 (1.8) | |
| >4 | 5(2.2) | 6 (4.6) | 51 (28.3) | 19 (10.6) | 81 (11.3) | |
| Depression scoreg | ||||||
| Minimal or no depression (0–4) | 222 (96.5) | 130 (100) | 159 (88.3) | 172 (95.6) | <.0001 | 683 (94.9) |
| Mild depression (5–9)h | 8(3.5) | 0(0) | 21 (11.7) | 8 (4.4) | 37 (5.1) | |
Chi-square test or Fisher’s exact for category and ANOVA test for continuous variables.
One participant (Coatepeque = 1) left this answer blank for mothers, and thirty-three participants (Coatepeque = 32, Zona 11 = 1) responded “I don’t know” for fathers.
Father characteristics include a total of n = 601.
Q=Guatemalan Quetzal (Q1 = $0.14 USD)
Amenities scale includes ownership of radio, television, refrigerator, stove, and car.
Food insecurity is measured with USAID’s Flousehold Food Insecurity Access Scale (H FI AS) (score 0 – 27). H igher scores equal more food insecurity.
Depression is measured with the PFIQ-9 (score 0 – 27).
One individual had a PFIQ-9 score of 10 but was added into the mild depression group for analysis.
Table 2 provides parental responses to the closed-ended (Q1-Q10) and Likert scale (L1-L10) VHS questions. No parent in our study reported ever refusing a vaccination, and only eight parents (1.1%) said that they had been reluctant or hesitated to get a vaccination for their children. No parent could think of a reason why children should not be vaccinated, and only three parents (0.4%) did not believe that vaccines could protect children from serious diseases. However, a majority of those interviewed (n=426, 59.2%) thought that parents like them do not have their children vaccinated with all the recommended vaccines, with more urban versus rural parents expressing this view (69.7% vs. 48.6%; p<0.0001). Time, distance and cost to get to the clinic and/or timing of the clinic and wait at the clinic were significant factors thought to prevent immunization more in the urban compared to rural population (12.5% vs. 6.1%; p=0.0032). While 205 (56.9%) parents in the urban clinics compared to 4 (1.1%) parents in the rural clinics believed that it was more difficult for children from some ethnic or religious groups in their community to get vaccinated (p<0.0001), more rural than urban parents thought that their local leaders (religious or political, teachers, health care workers) did not support childhood vaccinations (45.6% vs. 18.6%; p<0.0001). While the overall study population had very favorable attitudes towards vaccination, there were notable differences in the Likert scale level of agreement between clinics.
Table 2.
Vaccine Hesitancy Survey (VHS) questions
| Rural Clinics | Urban Clinics | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| All Centers | Coatepeque | Colomba | Villa Nueva | Zona 11 | p | ||||||
| VHS Questions | N | % | N | % | N | % | N | % | N | % | valuea |
| Q1. Do you believe that vaccines can protect children from serious diseases? | |||||||||||
| No | 3 | 0.4% | 1 | 0.4% | 1 | 0.8% | 0 | 0.0% | 1 | 0.6% | 0.7463 |
| Yes | 717 | 99.6% | 229 | 99.6% | 129 | 99.2% | 180 | 100.0% | 179 | 99.4% | |
| Q2. Do you think that most parents like you have their children vaccinated with all the recommended vaccines? | |||||||||||
| No | 426 | 59.2% | 95 | 41.3% | 80 | 61.5% | 120 | 66.7% | 131 | 72.8% | <.0001 |
| Yes | 294 | 40.8% | 135 | 58.7% | 50 | 38.5% | 60 | 33.3% | 49 | 27.2% | |
| Q3. Have you ever been reluctant or hesitated to get a vaccination for your children? | |||||||||||
| No | 712 | 98.9% | 228 | 99.1% | 130 | 100.0% | 180 | 100.0% | 174 | 96.7% | 0.0085 |
| Yes | 8 | 1.1% | 2 | 0.9% | 0 | 0.0% | 0 | 0.0% | 6 | 3.3% | |
| Q4. Have you ever refused a vaccination for your children? | |||||||||||
| No | 720 | 100% | 230 | 100% | 130 | 100% | 180 | 100% | 180 | 100% | NA |
| Q5. Has distance, timing of clinic, time needed to get to clinic or wait at clinic, and/or costs in getting to clinic prevented you from getting your children immunized? | |||||||||||
| No | 653 | 90.7% | 208 | 90.4% | 130 | 100.0% | 163 | 90.6% | 152 | 84.4% | <.0001 |
| Yes | 67 | 9.3% | 22 | 9.6% | 0 | 0.0% | 17 | 9.4% | 28 | 15.6% | |
| Q6. Are there other pressures in your life that prevent you from getting your children immunized on time? | |||||||||||
| No | 707 | 98.2% | 229 | 99.6% | 130 | 100.0% | 169 | 93.9% | 179 | 99.4% | <.0001 |
| Yes | 13 | 1.8% | 1 | 0.4% | 0 | 0.0% | 11 | 6.1% | 1 | 0.6% | |
| Q7. Are there any reasons you can think of why children should not be vaccinated? | |||||||||||
| No | 720 | 100% | 230 | 100% | 130 | 100% | 180 | 100% | 180 | 100% | NA |
| Q8. Do you think that it is difficult for some ethnic or religious groups in your community/region to get vaccinations for their children? | |||||||||||
| No | 511 | 71.0% | 227 | 98.7% | 129 | 99.2% | 91 | 50.6% | 64 | 35.6% | <.0001 |
| Yes | 209 | 29.0% | 3 | 1.3% | 1 | 0.8% | 89 | 49.4% | 116 | 64.4% | |
| Q9. Have you ever received or heard negative information about vaccinations? | |||||||||||
| No | 650 | 90.3% | 210 | 91.3% | 115 | 88.5% | 163 | 90.6% | 162 | 90.0% | 0.8503 |
| Yes | 70 | 9.7% | 20 | 8.7% | 15 | 11.5% | 17 | 9.4% | 18 | 10.0% | |
| Q10. Do leaders (religious or political leaders, teachers, health care workers) in your community support vaccinations for infants and children? | |||||||||||
| No | 231 | 32.1% | 157 | 68.3% | 7 | 5.4% | 1 | 0.6% | 66 | 36.7% | <.0001 |
| Yes | 489 | 67.9% | 73 | 31.7% | 123 | 94.6% | 179 | 99.4% | 114 | 63.3% | |
| L1. Childhood vaccines are important for my child’s health. | |||||||||||
| Strongly agree | 543 | 75.4% | 229 | 99.6% | 130 | 100.0% | 7 | 3.9% | 177 | 98.3% | <.0001 |
| Agree | 177 | 24.6% | 1 | 0.4% | 0 | 0.0% | 173 | 96.1% | 3 | 1.7% | |
| L2. Childhood vaccines are effective. | |||||||||||
| Strongly agree | 537 | 74.6% | 227 | 98.7% | 130 | 100.0% | 5 | 2.8% | 175 | 97.2% | <.0001 |
| Agree | 182 | 25.3% | 2 | 0.9% | 0 | 0.0% | 175 | 97.2% | 5 | 2.8% | |
| Disagree | 1 | 0.1% | 1 | 0.4% | 0 | 0.0% | 0 | 0.0% | 0 | 0.0% | |
| L3. Having my child vaccinated is important for the health of others in my community. | |||||||||||
| Strongly agre | 488 | 67.8% | 223 | 97.0% | 113 | 86.9% | 4 | 2.2% | 148 | 82.2% | <.0001 |
| Agree | 189 | 26.3% | 5 | 2.2% | 4 | 3.1% | 174 | 96.7% | 6 | 3.3% | |
| Neither agree nor disagree | 5 | 0.7% | 1 | 0.4% | 1 | 0.8% | 0 | 0.0% | 3 | 1.7% | |
| Disagree | 28 | 3.9% | 1 | 0.4% | 12 | 9.2% | 2 | 1.1% | 13 | 7.2% | |
| Strongly disagree | 10 | 1.4% | 0 | 0.0% | 0 | 0.0% | 0 | 0.0% | 10 | 5.6% | |
| L4. All childhood vaccines offered by the government program in my community are beneficial. | |||||||||||
| Strongly agree | 517 | 71.8% | 228 | 99.1% | 125 | 96.2% | 4 | 2.2% | 160 | 88.9% | <.0001 |
| Agree | 198 | 27.5% | 2 | 0.9% | 5 | 3.8% | 176 | 97.8% | 15 | 8.3% | |
| Neither agree nor disagree | 5 | 0.7% | 0 | 0.0% | 0 | 0.0% | 0 | 0.0% | 5 | 2.8% | |
| L5. New vaccines carry more risks than older vaccines. | |||||||||||
| Strongly agree | 77 | 10.7% | 57 | 24.8% | 7 | 5.4% | 0 | 0.0% | 13 | 7.2% | <.0001 |
| Agree | 31 | 4.3% | 5 | 2.2% | 5 | 3.8% | 3 | 1.7% | 18 | 10.0% | |
| Neither agree nor disagree | 146 | 20.3% | 6 | 2.6% | 71 | 54.6% | 2 | 1.1% | 67 | 37.2% | |
| Disagree | 430 | 59.7% | 162 | 70.4% | 46 | 35.4% | 175 | 97.2% | 47 | 26.1% | |
| Strongly disagree | 36 | 5.0% | 0 | 0.0% | 1 | 0.8% | 0 | 0.0% | 35 | 19.4% | |
| L6. The information I receive about vaccines from the vaccine program is reliable and trustworthy. | |||||||||||
| Strongly agree | 477 | 66.3% | 217 | 94.3% | 125 | 96.2% | 2 | 1.1% | 133 | 73.9% | <.0001 |
| Agree | 215 | 29.9% | 6 | 2.6% | 5 | 3.8% | 178 | 98.9% | 26 | 14.4% | |
| Neither agree nor disagree | 13 | 1.8% | 1 | 0.4% | 0 | 0.0% | 0 | 0.0% | 12 | 6.7% | |
| Disagree | 13 | 1.8% | 6 | 2.6% | 0 | 0.0% | 0 | 0.0% | 7 | 3.9% | |
| Strongly disagree | 2 | 0.3% | 0 | 0.0% | 0 | 0.0% | 0 | 0.0% | 2 | 1.1% | |
| L7. Getting vaccines is a good way to protect my child/children from disease. | |||||||||||
| Strongly agree | 528 | 73.3% | 226 | 98.3% | 125 | 96.2% | 4 | 2.2% | 173 | 96.1% | |
| Agree | 189 | 26.3% | 1 | 0.4% | 5 | 3.8% | 176 | 97.8% | 7 | 3.9% | <.0001 |
| Neither agree nor disagree | 1 | 0.1% | 1 | 0.4% | 0 | 0.0% | 0 | 0.0% | 0 | 0.0% | |
| Disagree | 2 | 0.3% | 2 | 0.9% | 0 | 0.0% | 0 | 0.0% | 0 | 0.0% | |
| L8. Generally I do what my doctor or health care provider recommends about vaccines for my child/children. | |||||||||||
| Strongly agree | 502 | 69.7% | 219 | 95.2% | 122 | 93.8% | 3 | 1.7% | 158 | 87.8% | <.0001 |
| Agree | 200 | 27.8% | 3 | 1.3% | 6 | 4.6% | 176 | 97.8% | 15 | 8.3% | |
| Neither agree nor disagree | 10 | 1.4% | 5 | 2.2% | 2 | 1.5% | 0 | 0.0% | 3 | 1.7% | |
| Disagree | 7 | 1.0% | 3 | 1.3% | 0 | 0.0% | 1 | 0.6% | 3 | 1.7% | |
| Strongly disagree | 1 | 0.1% | 0 | 0.0% | 0 | 0.0% | 0 | 0.0% | 1 | 0.6% | |
| L9. I am concerned about serious adverse effects of vaccines. | |||||||||||
| Strongly agree | 292 | 40.6% | 76 | 33.0% | 115 | 88.5% | 0 | 0.0% | 101 | 56.1% | <.0001 |
| Agree | 214 | 29.7% | 27 | 11.7% | 11 | 8.5% | 154 | 85.6% | 22 | 12.2% | |
| Neither agree nor disagree | 11 | 1.5% | 9 | 3.9% | 0 | 0.0% | 0 | 0.0% | 2 | 1.1% | |
| Disagree | 177 | 24.6% | 117 | 50.9% | 4 | 3.1% | 26 | 14.4% | 30 | 16.7% | |
| Strongly disagree | 26 | 3.6% | 1 | 0.4% | 0 | 0.0% | 0 | 0.0% | 25 | 13.9% | |
| L10. My child/children does or do not need vaccines for diseases that are not common anymore. | |||||||||||
| Strongly agree | 21 | 2.9% | 3 | 1.3% | 8 | 6.2% | 0 | 0.0% | 10 | 5.6% | <.0001 |
| Agree | 9 | 1.3% | 0 | 0.0% | 1 | 0.8% | 6 | 3.3% | 2 | 1.1% | |
| Neither agree nor disagree | 57 | 7.9% | 4 | 1.7% | 5 | 3.8% | 0 | 0.0% | 48 | 26.7% | |
| Disagree | 558 | 77.5% | 223 | 97.0% | 108 | 83.1% | 174 | 96.7% | 53 | 29.4% | |
| Strongly disagree | 75 | 10.4% | 0 | 0.0% | 8 | 6.2% | 0 | 0.0% | 67 | 37.2% | |
Chi-square test or Fisher’s exact where appropriate
Evaluation of the scree plot of eigenvalues for the Likert scale questions only identified two underlying constructs that had eigenvalues of 1.0 or greater. The first eigenvalue represented 59.6% of the variation in the factor with the second only adding 16.1% more to the explanation of variation with a total of 76.3%. After the initial rotation, we observed that while there were clear factor loadings on two factors, several of the items had factor loadings of <0.70, suggesting that an optimum factor structure may exist when limiting the items to only those with loadings of 0.70 or greater on one of the two factors. When we optimized the rotation in this manner, we obtained a more parsimonious factor structure (Table 3). There are still two factors, but only seven items. Model fit of the two-factor solution was better than the one-factor solution, and the 7-item scale was a better fit than the 10-item scale (Table 4). There are five items loading on Factor 1, and they are primarily related to vaccine confidence and positive attitudes towards vaccines. There are two items loading on Factor 2, related to vaccine risk and complacency as well as perceptions that vaccines are not beneficial. We explored whether there were any demographic differences in factor solutions. Unemployed mothers answered slightly higher for Factor 1 questions, but no other differences were found (Table 5). While we observed differences in survey responses by clinic, these did not affect the interpretation of results.
Table 3.
Rotated and optimized rotated factor patterns
| Vaccine Hesitancy Scale (VHS) Likert Survey Question | Rotated Pattern | Optimized Rotated Pattern | ||
|---|---|---|---|---|
| VHS Factor 1: | VHS Factor 2: | VHS Factor 1: | VHS Factor 2: | |
| Confidence | Complacency/Risks | Confidence | Complacency/Risks | |
| LI: Childhood vaccines are important for my child’s health | 0.94527 | 0.11779 | 0.95850 | 0.11770 |
| L2: Childhood vaccines are effective | 0.92966 | 0.11737 | 0.94004 | 0.12354 |
| L4: All childhood vaccines offered by the government program in my community are beneficial | 0.91189 | 0.02911 | 0.91155 | 0.03995 |
| L7: Getting vaccines is a good way to protect my child/children from disease | 0.90937 | 0.08271 | 0.91662 | 0.08396 |
| L8: Generally, 1 do what my doctor or health care provider recommends about vaccines for my child/children | 0.73796 | 0.03508 | 0.75407 | 0.03676 |
| L6: The information 1 receive about vaccines from the vaccine program is reliable and trustworthy | 0.67689 | −0.07002 | — | — |
| L3: Having my child vaccinated is important for the health of others in my community | 0.47557 | 0.10477 | — | — |
| L10: My child/children does or do not need vaccines for diseases that are not common anymore | −0.05611 | 0.77924 | −0.07952 | 0.83317 |
| L5: New vaccines carry more risks than older vaccines | 0.22108 | 0.70604 | 0.22743 | 0.71048 |
| L9: 1 am concerned about serious side effects of vaccines | 0.00926 | 0.32579 | — | — |
Table 4.
EFA model fit for one and two factor models*
| χ2df | RMSEA | CFI | SRMR | |
|---|---|---|---|---|
| All 10 items, 1 factor | 4.2 | 0.067 | 0.976 | 0.037 |
| All 10 items, 2 factors | 3.4 | 0.058 | 0.948 | 0.025 |
| 7 items (without L3, L6, L9), 1 factor | 4.4 | 0.069 | 0.989 | 0.041 |
| 7 items (without L3, L6, L9), 2 factors | 2.3 | 0.043 | 0.997 | 0.014 |
| Suggested value for good fit | 2–5 | ≤0.06 | ≥0.95 | <0.08 |
EFA indices selected to report the model fit include: 1.) Wheaton et al.’s relative/normed chi-square (χ /df), 2.) Root Mean Square Error Approximation (RMSEA), 3.) Comparative Fit Index (CFI), and 4.) Standard Root Mean Square Residual (SRMR).
Table 5.
Effect sizes of the two Vaccine Hesitancy Survey (VHS) factors
| VHS Factor 1 | VHS Factor 2 | ||||
|---|---|---|---|---|---|
| Characteristics | N (%) | Mean (SD) | ANOVAa | Mean (SD) | ANOVAa |
| Child’s gender | |||||
| Male | 358 (49.7) | 1.27 (0.42) | F=0.28 | 3.69 (0.66) | F=0.18 |
| Female | 362 (50.3) | 1.29 (0.43) | p=.596 | 3.67 (0.70) | p=.675 |
| Mother’s age category | |||||
| < 20 years old | 133 (18.5) | 1.33 (0.45) | F=1.25 | 3.63 (0.64) | F=0.72 |
| 20 – 29 years old | 445 (61.8) | 1.26 (0.41) | p=.291 | 3.68 (0.68) | P=.539 |
| 30 – 39 years old | 133 (18.5) | 1.31 (0.45) | 3.70 (0.75) | ||
| > 39 years old | 9(1.3) | 1.31 (0.44) | 3.94 (0.53) | ||
| Mother’s employment status | |||||
| Unemployed | 514 (71.4) | 1.30 (0.44) | F=4.14 | 3.68 (0.66) | F=0.0 |
| Employed | 206 (28.6) | 1.23 (0.39) | p=.042 | 3.68 (0.75) | p=.984 |
| η2=0.006 | |||||
| Mother’s education | |||||
| No education | 10 (1.4) | 1.02 (0.06) | F=2.22 | 3.25 (0.59) | F=2.64 |
| Completed or some primary education | 184 (25.6) | 1.28 (0.42) | p=.085 | 3.61 (0.69) | p=.049 |
| Completed or some secondary education | 173 (24.0) | 1.33 (0.45) | 3.75 (0.63) | η2=0.011 | |
| Completed or some higher education | 353 (49.0) | 1.26 (0.42) | 3.69 (0.71) | ||
| Site location | |||||
| Coatepeque | 230 (31.9) | 1.03 (0.13) | F=1890.26 | 3.57 (0.70) | F=14.2 |
| Colomba | 130 (18.1) | 1.03 (0.10) | P<.0001 | 3.52 (0.59) | P<.0001 |
| Villa Nueva | 180 (25.0) | 1.98 (0.15) | η2=0.888 | 3.94 (0.24) | η2=0.056 |
| Zone 11 | 180 (25.0) | 1.08 (0.18) | 3.66 (0.92) | ||
| Family monthly income levelb | |||||
| ≤Q1000 | 207 (28.8) | 1.32 (0.45) | F=1.77 | 3.58 (0.69) | F=1.91 |
| Q1001–2000 | 155 (21.5) | 1.30 (0.44) | p=.133 | 3.67 (0.58) | p=.107 |
| Q2001–3000 | 166 (23.1) | 1.26 (0.41) | 3.72 (0.64) | ||
| Q3001–4000 | 103 (14.3) | 1.19 (0.37) | 3.72 (0.78) | ||
| ≥Q4001 | 89 (12.4) | 1.29 (0.42) | 3.78 (0.78) | ||
Where results were significant, effect sizes were calculated using eta squared, η2
Q=Guatemalan Quetzal (Ql=$0.14 USD)
4. Discussion
This exploratory factor analysis provides insight from an LMIC into the overall design and applicability of the VHS [12]. Our work uncovered important findings beyond an exploratory factor analysis suggesting that the closed-ended and Likert scale questions of the survey appear to have limitations in their approach to assessing vaccine hesitancy. We are one of the first studies to field test questions from this survey. While one recent study by Shapiro et al. (2018) assessed the psychometric properties and validation of the VHS in Canada [31], to our knowledge, the measure has not yet been psychometrically evaluated in an LMIC. Furthermore, while one study in Guatemala used a cross-sectional survey to explore parental attitudes and preferences specifically for pertussis and poliomyelitis vaccines in urban versus rural populations [32], there is a need for a broader, validated, and reliable tool to measure the changing prevalence of vaccine hesitancy in LMICs that explores the complex factors affecting hesitancy. Field testing the VHS is an important step in modifying and implementing such a tool.
According to our study results, the VHS applied in both rural and urban settings of Guatemala reported no prior vaccine refusal and almost no hesitancy. Interestingly, however, only 40.8% (n=294) of parents said that they think most parents like them have their children vaccinated with all the recommended vaccines. We know that in Guatemala there are important structural issues to consider that may contribute more to limited vaccine compliance than hesitancy. For example, the political instability that occurred during our study led to considerable country-wide vaccine shortages and likely played a role in vaccine timeliness and compliance, which may have impacted parental attitudes and perceptions around immunization. We believe it is critical to consider how shortages, a reality experienced by many LMICs, affect vaccine timing and completion and, subsequently, may impact parents’ vaccine hesitancy survey responses. We further posit that a failure to do so risks conflating attitudes towards vaccines with compliance, instead of making attitudes distinct from access issues as intended in the WG vaccine hesitancy definition. Additionally, we found that factors such as difficulty accessing services (such as time, distance, and cost), religious and ethnic backgrounds, and the support of local leaders had varying influences on rural verus urban VHS responses, highlighting these important socio-cultural influences on vaccine attitudes [33–37].
Our results suggest difficulty understanding and using the Likert scale format. We had very few neutral responses of “neither agree nor disagree”. We also had a consistent difference between how study clinics responded to the ordinal scaling, with one clinic almost always responding “agree” while the other three clinics almost always responded “strongly agree”. This lack of spread and stark difference between clinics seems to imply a user-bias in how the study nurses may have asked the questions and interpreted and recorded the participant’s answers. Research has previously suggested that Likert-type scale formats may be culturally biased because some populations have difficulty understanding the ordered continuum of responses [38]. Additionally, results from Likert scales may be partially dependent on the range of possible responses [39]. Different cultures may not understand these gradations of agreement and disagreement. For some populations, a graded response format measuring multiple degrees of variation may be too abstract and, in fact, meaningless. It has been suggested that a Likert-type scale format may be better understood and utilized amongst populations with a higher level of underlying education [40]. For more diverse populations with cultural differences and varying degrees of education, such as those using the survey tool, dichotomous response categories (e.g. yes/no, agree/disagree, or true/false), vignettes, or a pictorial representation of answers may be preferred and easier to interpret [40,41].
Another important finding in our study is the revelation of a two-factor structure within the VHS Likert scale items, similar to the findings from the recent study in Canada [31]. It is interesting to note how closely aligned the results are from these two studies conducted in both a high income country and an LMIC as well as with parents of both young and older children. The first component (consisting of five items) in our study represented “confidence” in vaccination, and the second component (consisting of two items) represented vaccine “risk” and “complacency”. The WG’s vaccine hesitancy definition incorporates concepts from the “3 Cs” model of vaccine hesitancy and its determinants [7]. This model highlights three categories as drivers of vaccine hesitancy: complacency (e.g. not valuing or perceiving a need for vaccination), convenience (e.g. accessibility and availability of immunization services), and confidence (e.g. trust in vaccines, vaccine delivery systems, and vaccine policies). While the WG’s vaccine hesitancy definition encompasses all three of these concepts, both our study and the Shapiro et al. study [31] found that the Likert scale questions of the VHS primarily address the issue of vaccine confidence. Additionally, both studies found that these “confidence” questions are worded positively while the questions focused on “risk” and “complacency” are worded negatively, conflating the content and direction of the items. Our factor analysis findings, therefore, suggest that the Likert scale items collectively are more one-dimensional primarily measuring the concept of confidence and do not correlate well with the other constructs of complacency and convenience included in the definition of vaccine hesitancy. Future modification of the VHS would benefit from including multiple dimensions of vaccine hesitancy, especially related to the concept of convenience, with both positively and negatively worded items.
This study has several limitations. While we field tested both the closed-ended and Likert scale questions of the survey tool, we did not use the proposed open-ended series of questions due to participant time constraints and this being a secondary objective of our larger study. These open-ended questions will likely be important to understanding the other survey responses, especially in regards to access issues, and should be assessed in future studies. Furthermore, an in-depth qualitative exploration will be needed to understand the various contextual and socio-cultural influences on vaccine hesitancy that could assist in modifying the survey tool [42,43]. We conducted only an exploratory factor analysis and not a confirmatory factor analysis. The confirmatory analysis is more common where there are two distinct samples, which would allow the exploratory analysis with one sample and confirmatory analysis with a second sample. Given that we had only one sample, we were limited to the exploratory analysis. It will be important to consider future opportunities to conduct a confirmatory analysis with another similar sample in Guatemala. Additionally, the decision to use a 0.70 limit for factor loadings to inform the final item selection for each factor could be considered overly conservative, particularly where there are items near this threshold that are clearly similar and could have been included. We erred on the more conservative side given the well accepted standards for factor analysis and to also consider minimizing respondent burden. If respondents could answer fewer questions to represent a factor, we felt that was appropriate and worthwhile. A further limitation is that while our study nurses were trained and practiced conducting the survey prior to the study commencement, they were not routinely observed administering the survey to participants. We, therefore, do not know if the study nurses were consistent in how they asked the questions and interpreted or recorded the answers. Although the VHS was translated into Spanish by a native Guatemalan, we did not perform a back-translation for validation. Most importantly, our study population included infants presenting for their initial wellness visit which includes initiation of the primary immunization series. Therefore, our sample is biased to those likely to vaccinate and may have had lower than normal hesitancy. It is also possible that parents may have been less likely to admit to vaccine hesitancy during face-to-face interviews compared to other anonymous surveys. We found low levels of hesitancy and vaccine refusal in our study, and therefore, we could not derive any meaningful vaccine hesitancy score and validate the survey in the included Guatemalan populations. In the future, it could be beneficial to follow vaccine uptake in the infants we studied to correlate intention with behavior. Future studies will be needed in broader populations with more vaccine hesitancy to test both the validity and reliability of the VHS.
5. Conclusion
Vaccine hesitancy is an evolving and important modern public health problem to consider. Understanding this complex issue from a global perspective is imperative to inform effective interventions that address the varying causes of underimmunization and counteract the global gap in immunization, especially since the majority of vaccine-preventable diseases occur in LMICs. Vaccine-hesitant parents represent a heterogeneous group, making the development of a survey tool difficult. There is a need for a standardized measurement tool to understand, evaluate, and monitor vaccine hesitancy in diverse global settings that has undergone robust psychometric assessments. The VHS offers a strong initial step towards this goal. Our study is an important field test of this survey in rural and urban Guatemala. We suggest more work is needed to refine this survey for improved reliability and validity.
Highlights.
The Vaccine Hesitancy Survey (VHS) was field tested in urban and rural Guatemala
Vaccine shortages and access in LMICs are important structural issues to consider
Participants had difficulty understanding and using the Likert scale format
Factor analysis showed a two-factor structure within the VHS Likert scale items
More work is needed to refine this survey for improved global utility
Acknowledgments:
We thank the parents and children who participated in our study. We also thank our data manager Jose G. Rivera, our project administrators Mirsa Ariano and Michelle Shiver, and our study nurses. We greatly appreciate the support and collaboration of the Ministry of Public Health and Social Assistance in Guatemala with this study.
Funding source: This work was supported by the Eunice Kennedy Shriver National Institute of Child Health & Human Development at the National Institutes of Health [Grant Number 1R21HD084115–01]. The funding source had no involvement in the study design and conduct, data collection, analysis and interpretation of the data, and manuscript preparation. REDCap was supported by NIH/NCRR Colorado CTSI [Grant Number ULI RR025780]. Its contents are the authors’ sole responsibility and do not necessarily represent official NIH views.
Footnotes
Declarations of interest: none
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- 1.Cooper LZ, Larson HJ, Katz SL. Protecting public trust in immunization. Pediatrics 2008;122(1):149–53. doi: 10.1542/peds.2008-0987. [DOI] [PubMed] [Google Scholar]
- 2.Salmon DA, Dudley MZ, Glanz JM, Omer SB. Vaccine hesitancy: causes, consequences, and a call to action. Am J Prev Med 2015;49(6 Suppl 4):S391–8. doi: 10.1016/j.amepre.2015.06.009. [DOI] [PubMed] [Google Scholar]
- 3.Butler R, MacDonald NE; SAGE Working Group on Vaccine Hesitancy. Diagnosing the determinants of vaccines hesitancy in specific subgroups: the Guide to Tailoring Immunization Programmes (TIP). Vaccine 2015;33(34):4176–9. doi: 10.1016/j.vaccine.2015.04.038. [DOI] [PubMed] [Google Scholar]
- 4.Larson HJ, Cooper LZ, Eskola J, Katz SL, Ratzan S. Addressing the vaccine confidence gap. Lancet 2011;378(9790):526–35. doi: 10.1016/S0140-6736(11)60678-8. [DOI] [PubMed] [Google Scholar]
- 5.Schuster M, Eskola J, Duclos P; SAGE Working Group on Vaccine Hesitancy. Review of vaccine hesitancy: rationale, remit and methods. Vaccine 2015;33(34):4157–60. doi: 10.1016/j.vaccine.2015.04.035. [DOI] [PubMed] [Google Scholar]
- (6).Report of the SAGE Working Group on Vaccine Hesitancy; October 2014. Available from: http://www.who.int/immunization/sage/meetings/2014/october/1_Report_WORKING_GROUP_vaccine_hesitancy_final.pdf [Accessed October 2017].
- (7).MacDonald NE; SAGE Working Group on Vaccine Hesitancy. Vaccine hesitancy: definition, scope, and determinants. Vaccine 2015;33(34):4161–4. doi: 10.1016/j.vaccine.2015.04.036. [DOI] [PubMed] [Google Scholar]
- (8).Larson HJ, Jarrett C, Eckersberger E, Smith DM, Paterson P. Understanding vaccine hesitancy around vaccines and vaccination from a global perspective: a systematic review of published literature, 2007–2012. Vaccine 2014;32(19):2150–9. doi: 10.1016/j.vaccine.2014.01.081. [DOI] [PubMed] [Google Scholar]
- (9).Dubé E, Gagnon D, Nickels E, Jeram S, Schuster M. Mapping vaccine hesitancy--country-specific characteristics of a global phenomenon. Vaccine 2014;32(49):6649–54. doi: 10.1016/j.vaccine.2014.09.039. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (10).Jarrett C, Wilson R, O’Leary M, Eckersberger E, Larson HJ; SAGE Working Group on Vaccine Hesitancy. Strategies for addressing vaccine hesitancy - a systematic review. Vaccine 2015;33(34):4180–90. doi: 10.1016/j.vaccine.2015.04.040. [DOI] [PubMed] [Google Scholar]
- (11).Dubé E, Gagnon D, MacDonald NE; SAGE Working Group on Vaccine Hesitancy. Strategies intended to address vaccine hesitancy: review of published reviews. Vaccine 2015;33(34):4191–203. doi: 10.1016/j.vaccine.2015.04.041. [DOI] [PubMed] [Google Scholar]
- (12).Larson HJ, Jarrett C, Schulz WS, Chaudhuri M, Zhou Y, Dube E, et al. ; SAGE Working Group on Vaccine Hesitancy. Measuring vaccine hesitancy: the development of a survey tool. Vaccine 2015;33(34):4165–75. doi: 10.1016/j.vaccine.2015.04.037. [DOI] [PubMed] [Google Scholar]
- (13).Allred NJ, Shaw KM, Santibanez TA, Rickert DL, Santoli JM. Parental vaccine safety concerns: results from the National Immunization Survey, 2001–2002. Am J Prev Med 2005;28(2):221–4. doi: 10.1016/j.amepre.2004.10.014. [DOI] [PubMed] [Google Scholar]
- (14).Freed GL, Clark SJ, Butchart AT, Singer DC, Davis MM. Parental vaccine safety concerns in 2009. Pediatrics 2010;125(4):654–9. doi: 10.1542/peds.2009-1962. [DOI] [PubMed] [Google Scholar]
- (15).Luthy KE, Beckstrand RL, Callister LC. Parental hesitation in immunizing children in Utah. Public Health Nurs 2010;27(1):25–31. doi: 10.1111/j.1525-1446.2009.00823.x. [DOI] [PubMed] [Google Scholar]
- (16).Stefanoff P, Mamelund SE, Robinson M, Netterlid E, Tuells J, Bergsaker MA, et al. ; VACSATC working group on standardization of attitudinal studies in Europe. Tracking parental attitudes on vaccination across European countries: the Vaccine Safety, Attitudes, Training and Communication Project (VACSATC). Vaccine 2010;28(35):5731–7. doi: 10.1016/j.vaccine.2010.06.009. [DOI] [PubMed] [Google Scholar]
- (17).Kennedy A, Lavail K, Nowak G, Basket M, Landry S. Confidence about vaccines in the United States: understanding parents’ perceptions. Health Aff (Millwood) 2011;30(6):1151–9. doi: 10.1377/hlthaff.2011.0396. [DOI] [PubMed] [Google Scholar]
- (18).Bakhache P, Rodrigo C, Davie S, Ahuja A, Sudovar B, Crudup T, et al. Health care providers’ and parents’ attitudes toward administration of new infant vaccines--a multinational survey. Eur J Pediatr 2013;172(4):485–92. doi: 10.1007/s00431-012-1904-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (19).My C, Danchin M, Willaby HW, Pemberton S, Leask J. Parental attitudes, beliefs, behaviors and concerns towards childhood vaccinations in Australia: a national online survey. Aust Fam Physician 2017;46(3):145–151. [PubMed] [Google Scholar]
- (20).Greenberg J, Dubé E, Driedger M. Vaccine hesitancy: in search of the risk communication comfort zone. PLoS Curr 2017;9. doi: 10.1371/currents.outbreaks.0561a011117a1d1f9596e24949e8690b. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (21).Opel DJ, Mangione-Smith R, Taylor JA, Korfiatis C, Wiese C, Catz S, et al. Development of a survey to identify vaccine-hesitant parents: the parent attitudes about childhood vaccines survey. Hum Vaccin 2011;7(4):419–25. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (22).Opel DJ, Taylor JA, Mangione-Smith R, Solomon C, Zhao C, Catz S, et al. Validity and reliability of a survey to identify vaccine-hesitant parents. Vaccine 2011;29(38):6598–605. doi: 10.1016/j.vaccine.2011.06.115. [DOI] [PubMed] [Google Scholar]
- (23).Opel DJ, Taylor JA, Zhou C, Catz S, Myaing M, Mangione-Smith R. The relationship between parent attitudes about childhood vaccines survey scores and future child immunization status: a validation study. JAMA Pediatr 2013;167(11):1065–71. doi: 10.1001/jamapediatrics.2013.2483. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (24).Roberts JR, Thompson D, Rogacki B, Hale JJ, Jacobson RM, Opel DJ, et al. Vaccine hesitancy among parents of adolescents and its association with vaccine uptake. Vaccine 2015;33(14):1748–55. doi: 10.1016/j.vaccine.2015.01.068. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (25).Strelitz B, Gritton J, Klein EJ, Bradford MC, Follmer K, Zerr DM, et al. Parental vaccine hesitancy and acceptance of seasonal influenza vaccine in the pediatric emergency department. Vaccine 2015;33( 15): 1802–7. doi: 10.1016/j.vaccine.2015.02.034. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (26).Mohd Azizi FS, Kew Y, Moy FM. Vaccine hesitancy among parents in a multi-ethnic country, Malaysia. Vaccine 2017;35(22):2955–61. doi: 10.1016/j.vaccine.2017.04.010. [DOI] [PubMed] [Google Scholar]
- (27).Domek GJ, Contreras IL, O’Leary ST, Bull S, Furniss A, Kempe A, et al. SMS text message reminders to improve infant vaccination coverage in Guatemala: a randomized controlled trial. Vaccine 2016;34(21):2437–43. doi: 10.1016/j.vaccine.2016.03.065. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (28).Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap) - a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009;42(2):377–81. doi: 10.1016/j.jbi.2008.08.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (29).Kaiser HF. The application of electronic computers to factor analysis. Educ Psychol Meas 1960;20:141–151. [Google Scholar]
- (30).Velicer WF, Jackson DN. Component analysis versus common factor analysis: some further observations. Multivariate Behav Res 1990;25:97–114. doi: 10.1207/s15327906mbr2501_12. [DOI] [PubMed] [Google Scholar]
- (31).Shapiro GK, Tatar O, Dube E, Amsel R, Knauper B, Naz A, et al. The vaccine hesitancy scale: psychometric properties and validation. Vaccine 2018;36(5):660–7. doi: 10.1016/j.vaccine.2017.12.043. [DOI] [PubMed] [Google Scholar]
- (32).DelaFuente JP, Castillo-Mazariegos EJ, Asturias EJ. Differential preference for pertusis and poliomyelitis vaccines in urban versus rural parents in Guatemala. Paediatr Child Health 2016;21(2):e15–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (33).Crocker-Buque T, Mindra G, Duncan R, Mounier-Jack S. Immunization, urbanization and slums - a systematic review of factors and interventions. BMC Public Health 2017;17(1):556. doi: 10.1186/s12889-017-4473-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (34).Grabenstein JD. What the world’s religions teach, applied to vaccines and immune globulins. Vaccine 2013;31(16):2011–23. doi: 10.1016/j.vaccine.2013.02.026. [DOI] [PubMed] [Google Scholar]
- (35).Ruijs WL, Hautvast JL, Kerrar S, van der Velden K, Hulscher ME. The role of religious leaders in promoting acceptance of vaccination within a minority group: a qualitative study. BMC Public Health 2013;13:511. doi: 10.1186/1471-2458-13-511. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (36).United Nations Children’s Fund: Building trust in immunization. Partnering with religious leaders and groups. New York:UNICEF;2004. [Google Scholar]
- (37).Baumgaertner B, Carlisle JE, Justwan F. The influence of political ideology and trust on willingness to vaccinate. PLoS One 2018;13(1):e0191728. doi: 10.1371/journal.pone.0191728. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (38).Flaskerud JH. Cultural bias and Likert-type scales revisited. Issues Ment Health Nurs 2012;33(2):130–2. doi: 10.3109/01612840.2011.600510. [DOI] [PubMed] [Google Scholar]
- (39).Dawes J Do data characteristics change according to the number of scale points used? An experiment using 5-point, 7-point and 10-point scales. International Journal of Market Research 2008;50(1):61–77. [Google Scholar]
- (40).Bernal H, Wooley S, Schensul JJ. The challenge of using Likert-type scales with low-literate ethnic populations. Nurs Res 1997;46(3):179–81. PMID:. [DOI] [PubMed] [Google Scholar]
- (41).King G, Wand J. Comparing incomparable survey responses: evaluating and selecting anchoring vignettes. Political Analysis 2007:15:46–66. doi: 10.1093/pan/mpl011 [DOI] [Google Scholar]
- (42).Braka F, Asiimwe D, Soud F, Lewis RF, Makumbi I, Gust D. A qualitative analysis of vaccine safety perceptions and concerns among caretakers in Uganda. Matern Child Health J 2012;16(5):1045–52. doi: 10.1007/s10995-011-0826-5. [DOI] [PubMed] [Google Scholar]
- (43).Cover JK, Nghi NQ, LaMontagne DS, Huyen DT, Hien NT, Nga le T. Acceptance patterns and decision-making for human papillomavirus vaccination among parents in Vietnam: an in-depth qualitative study post-vaccination. BMC Public Health 2012;12:629. doi: 10.1186/1471-2458-12-629. [DOI] [PMC free article] [PubMed] [Google Scholar]