Development of a survey to identify vaccine-hesitant parents

Abstract

Objective

To develop a survey to accurately assess parental vaccine hesitancy.

Results

The initial survey contained 17 items in four content domains: (1) immunization behavior; (2) beliefs about vaccine safety and efficacy; (3) attitudes about vaccine mandates and exemptions; and (4) trust. Focus group data yielded an additional 10 survey items. Expert review of the survey resulted in the deletion of nine of 27 items and revisions to 11 of the remaining 18 survey items. Parent pretesting resulted in the deletion of one item, the addition of one item, the revision of four items, and formatting changes to enhance usability. The final survey contains 18 items in the original four content domains.

Methods

An iterative process was used to develop the survey. First, we reviewed previous studies and surveys on parental health beliefs regarding vaccination to develop content domains and draft initial survey items. Focus groups of parents and pediatricians generated additional themes and survey items. Six immunization experts reviewed the items in the resulting draft survey and ranked them on a 1–5 scale for significance in identifying vaccine-hesitant parents (5 indicative of a highly significant item). The lowest third of ranked items were dropped. The revised survey was pretested with 25 parents to assess face validity, usability and item understandability.

Conclusions

The Parent Attitudes about Childhood Vaccines survey was constructed using qualitative methodology to identify vaccine-hesitant parents and has content and face validity. Further psychometric testing is needed.

Introduction

The rise in immunization resistance has spurred several investigations into parental decision-making about childhood vaccines.^1–8 Vaccine-hesitant parents (VHPs) are a heterogeneous group who tend to have beliefs that fall between those of vaccine acceptors and rejecters on an immunization acceptance continuum.¹ VHPs are defined as parents who may refuse one or two vaccines but agree to all others, delay vaccines or accept vaccines but are unsure in doing so.¹

VHPs are important for understanding and counteracting growing vaccine resistance. They comprise a much larger group than those who completely reject vaccines³ and are also potentially more amenable to behavior change because they tend to seek information from their child's provider about vaccines.^4,9 A better understanding of how to communicate with VHPs and address their concerns is vital to translating the scientific benefits of vaccines into practice.^10,11

Lack of time, materials and knowledge, however, constrain a provider's ability to communicate vaccine risks and benefits that might address parental concerns and foster trust.^12,13 Furthermore, in one study, 40% of responding pediatricians indicated that they would dismiss a child from their practice if a parent refuses one or more of the recommended immunizations.¹⁴ Other investigators have suggested that parental concerns about childhood immunization are often neglected.¹ These shortfalls may result in under-immunized children, who are consequently at risk for contracting both highly transmissible diseases (e.g., measles) and diseases currently circulating (e.g., pertussis).¹⁵

A valid method for identifying VHPs is needed to proceed with future interventions that aim to enable effective providerparent communication regarding immunization concerns and to increase immunization rates. Prior studies have used survey instruments to examine parental immunization beliefs.^{3,4,9,16–22} These surveys, however, were not explicitly designed to identify VHPs and therefore lack sensitivity. In this study, we sought to develop a survey to accurately assess parental vaccine hesitancy for use in research settings.

Results

Item selection.

The initial survey contained seventeen items within four content domains. These domains included (1) immunization behavior (six items), (2) beliefs about vaccine safety and efficacy (four items), (3) attitudes about vaccine mandates and exemptions (three items) and (4) trust (four items). Of the seventeen items, twelve were borrowed or modified from existing survey instruments and five were constructed de novo.

Augmentation of the item pool.

The themes that emerged from the focus groups both reinforced the four content domains used for the survey and revealed themes within these domains that had not yet been captured. The result was the addition of ten new items for the draft survey: one under the content domain of immunization behavior, three under beliefs about vaccine safety and efficacy and six under trust. Below is a description of the predominant themes from the focus groups and the new survey items they prompted.

In the content domain of immunization behavior, a predominant theme that was not addressed by the initial survey was that parents expressed a sense of advocacy when it came to immunizations. Parents stated that they would often rely on their own research on immunizations in order to come to an informed decision, rather than be deferential to their child's doctor. This seemed especially true as they matured as parents and began making immunization decisions for their second or third child:

(Parent) “I've done more research because I just feel like I've raised two and I just feel like, wait a minute! You know? My first one, I kind of just really followed the recommendations and then I questioned it more with my second and now I'm questioning it more with my third.”

To reflect this parent-as-advocate behavior, we added a survey item under the immunization behavior domain (“It is my role as a parent to question vaccination.”).

Three items were added to the beliefs about vaccine safety and efficacy domain. One item reflected the theme of prevalence of vaccine-preventable disease (“I am not worried about the diseases vaccines prevent because they are so rare.”). Both pediatricians and parents commented on the fact that parents perceived the risk of their child contracting a vaccine-preventable disease to be negligible in the US given the fact that many of these diseases are now so uncommon:

(Parent) “I basically just think of (immunizations) as an insurance policy, in case my child wants to travel to third world countries later on when they're teenagers. Countries where certain diseases really ravage the population, they're protected. I…think, to some degree, [immunizations are] a bit of overkill in the US.”

A second item was added to capture a recurring theme among the focus group pediatricians regarding vaccine safety. They perceived many parents' vaccine hesitancy to stem from experience with a bad outcome from an immunization:

(Pediatrician) “When I get to a point where they feel like, they know their sister's son got autism and, even though they believe in vaccines…you cannot get these kids immunized because the cousin has autism. I don't think (the parents) can face it.”

The item that was added to the initial survey to reflect this theme was: “Do you know of anyone who has had a bad vaccine reaction?”

Lastly, both parents and pediatricians stated that vaccine hesitant beliefs often were related to the fact that vaccines were unnatural. Not only did the focus group parents feel that getting several immunizations at once was unusual, but they had concerns that vaccines were filled with foreign substances:

(Parent) “You know, I was very frustrated…because I did feel like…I've always had an idea that somehow this should be a little bit natural and to be slammed with four things at once in nature seems like it would be kind of unusual, to be directly injected with these four things. Really unusual.”

(Pediatrician) “Toxins are another concern, from adjuvants and things like thimerosal, aluminum and just the fact that some people talk about it as unnatural and they'd rather it happen the natural way.”

To reflect this theme, one item was added to the survey (“I prefer my child to develop his or her immunity naturally.”).

The main theme related to trust from both parent and provider focus groups was the affirmed importance of a parent's trust in their child's provider in their decision-making about immunizations:

(Parent) “I've walked away from getting my kids vaccinated going, you know, I hope that was the right decision…but I do say, the reason I have vaccinated my children is because … I've trusted my doctor.”

(Pediatrician) “I mean, in general, I think lack of trust is the bottom of the whole picture and sometimes, trust in what I have to say does then sway them. Many times, ‘well did your daughter get them?’—‘Yes she did’—‘Well, okay.’ They trust that I'll make the right decision.”

Two additional trust themes, however, emerged that were not addressed by the initial survey questions. One of these themes was that both parents and pediatricians felt that vaccine hesitant beliefs had roots in a deep distrust of the medical system, the government and pharmaceutical companies. For instance, parents were suspicious of the information being told to them by their doctor not because they didn't trust their child's doctor but because they were suspicious of the source from which their child's doctor was getting the information:

(Parent) “You know let's face it: ‘pharma’ is profit-driven. That's just what it boils down to…I think what happens, unfortunately, is that, it's not that I don't trust my doctor. It's that I don't always know if my doctor's getting the right information from ‘pharma’”

The second trust theme that emerged involved the relationship of trust and communication. Parents with doubts about vaccines thought only open and honest communication with their child's provider could develop or maintain trust:

(Parent) “I bring up my issues (to my child's doctor) very directly… because I trust (her) and I trust that we have an equal conversation…Having a trusting relationship where the doctor is not presenting… information as, ‘I am the fount of knowledge, don't you worry, Little Mama, it's okay' makes me more open to presenting my doubts (to her).”

To incorporate these facets of trust into the survey, we added an item that addressed general distrust in vaccine information (“The information I receive about vaccines is trustworthy.”) and one item to address parent-provider communication (“I am able to openly discuss my vaccine concerns with my child's doctor”). We also replaced the four items we borrowed from the Trust in Physician Scale with modified questions from the trust subscale that is part of the Primary Care Assessment Survey (PCAS, 8 items).²³ The PCAS is a self-administered survey (as opposed to the Trust in Physician Scale, which is interviewer-administered) developed to assess primary care performance across different delivery systems and its trust subscale was designed to measure trust over the length of the provider-patent relationship. The PCAS was therefore thought to be a better choice for our survey by potentially being more pertinent to our target population of parents whose interaction with their child's provider occurred in a primary care setting.

Item reduction.

After review by the expert panel, nine survey items were deleted from the 27 item draft survey. The mean rating for the deleted items was 2.6 versus 4.1 for the retained items (with five indicating most significant in identifying VHPs). Six of the nine survey items deleted were part of the PCAS trust subscale. All expert reviewers felt that fewer items from the PCAS trust subscale would yield a shorter, less burdensome survey while maintaining the ability to discriminate between vaccine-hesitant and non-vaccine-hesitant parents. The other three items deleted were ranked low because they were felt to be redundant with other items retained in the survey: (1) Have you ever asked the doctor or nurse not to give your child a vaccination for a reason other than illness? (2) Have you ever refused a vaccination for your child for non-religious reasons? (3) Have you ever refused a vaccination for your child for religious reasons?

The final version of the survey reviewed by experts, therefore, contained 18 items and three different response formats [dichotomous (e.g., yes/no), 5—point Likert scale (e.g., strongly agree, agree, not sure, disagree, strongly disagree), and an 11—point scale (e.g., responses ranging from “0: not sure at all” to “10: completely sure”)]. The 11—point response scale was used to maximize the information obtained regarding trust and overall hesitancy (Table 2) but was restricted to just two items in order to reduce the cognitive burden associated with large scales greater than 7 points.²⁴ The response category of “not sure” was used in the Likert scale formats because we felt that this was an answer that reflected vaccine hesitancy. For instance, we felt that the “not sure” answer to the question “I am okay with the number of recommended shots children get in the first two years of life” likely represented vaccine hesitancy.

Table 2.

Descriptive characteristics and content of the PACV

Content domain	Item	Response format	Source (refs)∼
Immunization Behavior	Have you ever delayed having your child get a shot for reasons other than illness or allergy?	*Y/N/DK	18
	Have you ever decided not to have your child get a shot for reasons other than illness or allergy?	Y/N/DK	18
	How sure are you that following the recommended shot schedule is a good idea for your child?	0 (Not at all sure) to 10 (Completely sure)	Novel item
	It is my role as a parent to question shots.	#SA/A/NS/D/SD	Novel item
	If you had another infant today, would you want him/her to get all the recommended shots?	Y/N/DK	18
	Overall, how hesitant about childhood shots would you consider yourself to be?	^NAH/NTH/NS/SH/VH	Novel item
Beliefs about Vaccine Safety and Efficacy	Children get more shots than are good for them.	SA/A/NS/D/SD	3, 19, 21
	I believe that many of the illnesses shots prevent are severe.	SA/A/NS/D/SD	3, 19
	It is better for my child to develop immunity by getting sick than to get a shot.	SA/A/NS/D/SD	21, 25
	It is better for children to get fewer vaccines at the same time.	SA/A/NS/D/SD	21
	How concerned are you that your child might have a serious side effect from a shot?	&NAC/NTC/NS/SC/VC	4
	How concerned are you that any one of the childhood shots might not be safe?	NAC/NTC/NS/SC/VC	3, 4, 9, 17–20, 25
	How concerned are you that a shot might not prevent the disease?	NAC/NTC/NS/SC/VC	4, 25
	Do you know of anyone who has had a bad reaction to a shot?	Y/N/DK	Novel item
Attitudes about Vaccine Mandates and Exemptions	The only reason I have my child get shots is so they can enter daycare or school.	Y/N/DK	16, 18, 20, 22
Trust	I trust the information I receive about shots.	SA/A/NS/D/SD	20, 22
	I am able to openly discuss my concerns about shots with my child's doctor.	SA/A/NS/D/SD	3, 20, 22
	All things considered, how much do you trust your child's doctor?	0 (Do not trust at all) to 10 (Completely trust)	22, 23

^∼Bolded references indicate that the survey item was only minimally altered from this survey source.

^*Yes/No/Don't Know.

^#Strongly Agree/Agree/Not sure/Disagree/Strongly Disagree.

^{^}Not at all hesitant/Not too hesitant/Not sure/Somewhat hesitant/Very hesitant.

^&Not at all concerned/Not too concerned/Not sure/Somewhat concerned/Very concerned.

Eleven of the 18 retained items were reworded based on expert feedback. In particular, the words “vaccination” and “immunization” were replaced with “shots.” It was generally agreed that “shots” was how childhood immunizations were most often referred to by parents. Although “shots” is not a technically accurate term and even has negative connotations, its widespread use and recognition by parents in the immunization context was important for the accessibility and usability of the survey; therefore, the term was maintained. The final expert-reviewed version of the survey had a reading level of 6.6.

Pretesting.

The demographic characteristics of the 25 parents who pretested the expert-reviewed version of the survey are presented in Table 1. Eleven had at some point either delayed or decided not to give their child a vaccine. There was near unanimous agreement among parents that the survey did a good job of measuring their attitudes about childhood immunizations (N = 24), was not too long (N = 24), was easy for them to fill out (N = 21), and that the order of the questions flowed well (N = 22). Most parents commented that it took them 5 minutes or less to complete.

Table 1.

Characteristics of parents who pretested survey

Characteristic	N (%)*
Relationship to Child
Mother	22 (88)
Father	3 (12)
Age
18–29	8 (32)
30 or older	17 (68)
Number of Children in Household
1	12 (48)
2	9 (36)
3	3 (12)
4	1 (4)
Marital Status
Single	2 (8)
Married	20 (80)
Living with a partner	3 (12)
Widowed	0 (0)
Separated	0 (0)
Divorced	0 (0)
Education
8th grade or less	0 (0)
Some high school, but not a graduate	2 (8)
High school graduate or GED	0 (0)
Some college or 2 year degree	4 (16)
4-year college degree	8 (32)
More than 4-year college degree	11 (44)
Household Income
$30,000 or less	6 (24)
$30,001–$50,000	4 (16)
$50,001–$75,000	4 (16)
$75,001 or more	11 (44)
Race/Ethnicity
White	17 (65)
Black or African American	0 (0)
Hispanic/Latino	3 (12)
Asian	5 (19)
Native Hawaiian or other Pacific Islander	1 (4)
American Indian or Alaska Native	0 (0)
Other	0 (0)

^*Percentage of total (N=25 for all characteristics except for Race/Ethnicity, for which parents could check all that applied).

The majority of parents also thought the response categories for each survey item were appropriate (N = 16). Those that disagreed said they preferred all questions to be either positively or negatively worded (instead of combination of both), would have liked a comment box after some questions, or did not like the variation in response categories (N = 9). Only six parents preferred the word “vaccination” or “immunization” over “shots” when asked, including three non-white parents who, although they understood the word “shots,” were worried that others in their culture might not. Others recognized the term's inaccuracy but still were able to complete the survey because they understood its intended meaning (N = 3). The word “shots,” therefore, was retained in the survey.

Thirteen parents indicated that one or more survey items were unclear or confusing, while 12 parents had no problems understanding any survey item. The survey items that were perceived by more than one parent as confusing were: (1) “I prefer my child to develop immunity from naturally-occurring diseases instead of vaccines” (N = 2); (2) “Do you personally know of anyone who has had a bad reaction to a shot?” (N = 3); and (3) “Would you have your child get shots if they were not required to enter daycare or school?” (N = 2). Each of these survey items was reworded based on parent feedback (Table 2). In addition, one survey item was added and one item deleted. The item “I am okay with the number of recommended shots children get in the first two years of life” was deleted in favor of two separate questions that better captured parents' concerns with the timing of childhood immunizations: “Children get more shots than are good for them”^19,21 and “It is better for children to get fewer vaccines at the same time.” “I trust the judgment of my child's doctor” was deleted because the distribution of responses to this item was severely positively skewed.

We obtained at least five retrospective think-alouds from parents for each survey item. All think-alouds were considered to be accurate interpretations of the question's intended meaning (data not shown). Two questions were separately assessed for how confidently all parents were able to retrieve the information for which they were queried (“Have you ever delayed having your child get a shot for reasons other than illness or allergy?” and “Have you ever decided not to have your child get a shot for reasons other than illness or allergy?”). Most of the parents stated that they had no problem retrieving this immunization information (N = 22) and that they could do so confidently (N = 20).

The final revised survey contained 18 items under the four original content domains. The domain of immunization behavior contained six items, beliefs about vaccine safety and efficacy contained eight, attitudes about vaccine mandates and exemptions contained one and trust contained three items (Table 2). The reading level of the final survey was 5.9.

Discussion

In this study, we developed and began evaluating a tool intended to identify parents who have concerns about vaccines. The PACV is an amalgamation of de novo items as well as items borrowed or modified from existing surveys used to assess parental immunization attitudes, beliefs and behaviors and trust.^{3,4,9,16–23,25} While these existing immunization surveys have a number of features in common, they differ in their overall number of items, context and response formats. In addition, existing immunization surveys differ in their target populations, with none, to our knowledge, specifically intended for identifying VHPs. In this study, we draw on these previous surveys to produce a brief, self-administered survey that would identify VHPs. By using an iterative, qualitative approach to survey development, we constructed a final version of the PACV that contains 18 items, takes 5 minutes or less to complete, and reads at a 6^th grade level.

Data from the parent and pediatrician focus groups largely reinforced the PACV's content domains and initial survey items that had been abstracted from existing immunization surveys.^{3,4,9,16–23,25} Although the focus group data prompted the addition of 10 items to the survey, only one of these added items (“It is my role as a parent to question shots”) reflects an aspect of immunization hesitancy that, to our knowledge, has not been fully explored in existing surveys. This item explores the notion that parents feel that they need to be a stronger advocate for their child by questioning immunizations. This advocacy theme may mirror parental self-efficacy, which has been shown to be associated with childhood immunization rates.²⁶ Further characterization of this theme and its role in parental immunization decision-making is needed.

The expert panel review and pretest sample of parents provide preliminary evidence of the PACV's validity. Items rated highly by experts as being significant in identifying VHPs were retained in the PACV, conferring content validity. In addition, nearly all (N = 24) of the 25 parents interviewed stated that the PACV did a good job of measuring their attitudes about childhood immunizations, demonstrating the PACV's face validity. However, more psychometric evaluation of the PACV is warranted in order to ensure that the revisions made to the survey items based on this initial evaluation yield valid and reliable responses.

We recognize that this study has some limitations. The total number of parents (N = 4) in the two focus groups was small and is therefore likely not representative of the larger VHP population, a very heterogeneous group. This, however, might be partially mitigated by the two additional pediatrician focus groups, which involved providers (N = 7) who interface frequently with a larger number of VHPs. Also, the aim of the focus groups was simply to augment the item pool beyond the initial items selected from the published literature and not to draw broader conclusions about the attitudes or beliefs of VHPs.

Another limitation is that the pretest phase of the survey development occurred in December 2009 and January 2010, which was near the height of the H1N1 pandemic. Parental pretest responses therefore may have been influenced by H1N1 and not by the childhood vaccines that the survey intended to query. This issue was anticipated by including in the survey instructions the following sentence: “This survey is not about seasonal flu or swine flu (H1N1) shots”. However, 10 of the 25 parents sampled did not read the instructions when asked (data not shown). Lastly, the parent samples were derived from one institution in Seattle, WA, thereby limiting the generalizability of the study results. Washington State, however, has vaccination coverage levels that are lower than the national average²⁷ and therefore comprises a good population in which to study vaccine hesitancy.

In conclusion, the PACV represents a potentially useful research tool for identifying an important population of parents who have vaccine concerns. For instance, it might be used to screen parents to receive an intervention aimed at increasing parental acceptance of immunizations. A unique theme related to advocacy was identified as potentially playing a role in parental immunization behavior. Items on the PACV have face and content validity. Further psychometric evaluation will help verify the PACV's importance.

Patients and Methods

We used a four step process to develop the Parent Attitudes about Childhood Vaccines (PACV) survey (Fig. 1). First, we drafted a preliminary survey by identifying content domains in previously published studies that were relevant to parental hesitancy of immunizations.^{1,3,4,16–18,25} Individual items within these domains were borrowed or modified from existing survey instruments or constructed de novo.^{3,4,9,16–22,25,28} De novo items were written according to accepted guidelines for survey development.²⁹

Figure 1.

Summary of survey development.

In order to identify additional immunization hesitancy domains and augment the survey's item pool, we next convened two focus groups of parents who were hesitant towards vaccines (N = 4) and two focus groups of community pediatricians who had VHPs in their practice (N = 7). Parents were recruited from the primary care pediatric clinic at the University of Washington Medical Center. Parents were considered hesitant towards vaccines if they previously had communicated to clinic staff that they preferred to avoid or delay ≥1 immunizations and if they had a child aged 19–35 months who was missing ≥1 immunizations. Community pediatricians were recruited from the Puget Sound Pediatric Research Network, a regional practice-based research network. The parent focus groups were structured to identify and discuss reasons for vaccine hesitancy, identify terms that parents use in discussing their hesitancy, and to determine how parents group or categorize issues relevant to vaccine hesitancy. The pediatrician focus groups had the additional objective of understanding how physicians address parental concerns about childhood immunizations. The focus groups were audiotaped, transcribed verbatim and analyzed using an inductive coding technique for qualitative data.^30,31 Three investigators (DO, CK, CW) independently read each focus group transcript and abstracted key immunization hesitancy themes. We then met to discuss our independent analyses and negotiate a final list of themes. After this discussion, we either wrote novel items or modified items from existing surveys that reflected these themes and incorporated them into the survey.

Third, we convened a panel of six local and national immunization experts to assess content validity and facilitate item reduction of the revised survey. Experts included a researcher in vaccines and communication at the Centers for Disease Control and Prevention, a manager of a state-level immunization awareness coalition, two academic pediatricians with a research focus in vaccine policy and vaccine refusal and two pediatricians who practice in two different Washington State counties that have higher rates of children who claim non-medical exemptions from required school-entry immunizations (5.0% and 5.3% in 2009, respectively)³² than the overall national average (<2%).³³ Experts were instructed to identify any missing relevant items or domains as well as rank items on a 1–5 scale for significance in identifying VHPs (with 5 being most significant). An item's score was averaged across the expert panel and rank ordered. Items ranked in the lowest third were deleted.

Lastly, the expert panel-revised survey was subsequently formatted and pretested anonymously with a sample of 25 English-speaking parents recruited from the same University of Washington Medical Center primary care pediatric clinic used to recruit parents for the focus groups. Eligible parents had a child aged 2–36 months, a well-child visit during January 2010, and were either non-hesitant or hesitant toward vaccines. Hesitant parents were identified by the presence of documentation in the clinic's electronic database that they had previously communicated to the clinic a preference to avoid or delay ≥1 immunizations. We sought to enroll approximately 60% hesitant and 40% non-hesitant parents. Eligible parents were sent a letter prior to their child's well-child visit introducing the study, and were subsequently approached to determine their willingness to participate in the study on the day of their child's visit. All parent participants provided oral consent prior to participating.

After parents completed the survey, they completed a cognitive interview with an investigator (DO, CK). Interviews were structured to assess (1) face validity, by asking the parents whether the questionnaire domains and items measure vaccine hesitancy; (2) usability, by obtaining parent feedback on the flow of the questionnaire, response categories, the order of items and the questionnaire's length; and (3) item understandability, by performing retrospective “think-alouds” on selected survey items with each parent in order to ensure the item was being read and understood as intended.^34,35 Interviewer notes were transcribed and analyzed by three investigators (DO, CK, DM). We examined each item for missing data and plotted its response distribution. We deleted or modified items that parents found unclear, confusing, were interpreted in a way that was not intended or showed a skewed response distribution. We also modified all aspects of the survey's format that diminished its usability. The reading level of the survey was measured using the Flesch-Kincaid scale available through Microsoft Word (Microsoft Corp., Redmond, WA).

This study was approved by the Seattle Children's Hospital Institutional Review Board and the Group Health Cooperative (Seattle, WA) Human Subjects Review Committee. Quotations used in this article are representative of parent or pediatrician focus group data and have been excerpted directly from the focus group transcripts. For reading ease, additions made to the quotations are indicated by the use of brackets and excluded sections of the text are denoted by the use of an ellipsis.

Abbreviations

VHP

vaccine-hesitant parent

PACV

Parent Attitudes about Childhood Vaccines survey

PCAS

Primary Care Assessment survey

H1N1

influenza A virus subtype H1N1