Psychometric Properties of a Vaccine Hesitancy Scale Adapted for COVID-19 Vaccination Among People with HIV

Abstract

We evaluated the psychometric properties of a modified version of the Vaccine Hesitancy Scale (VHS) among people with HIV (PWH) for COVID-19 vaccination in a cross-sectional study in the US. Self-report data from an online questionnaire were collected from a sample of N = 175 PWH. Participants were surveyed in English or Spanish regarding attitudes towards COVID-19 vaccination using the adapted VHS. Participants were on average 51.55 years of age (SD = 13.90) 55% were women. The reliability of the scale was acceptable (α = 0.72). An exploratory factor analysis showed that “Lack of confidence” and “Risks” explained 45.55% and 12.31% of the variance. Related items showed expected associations with these factors, supporting construct validity. Results illustrate that the modified VHS for COVID-19 vaccination has adequate psychometric properties. We replicated the original factor structure of the VHS and demonstrated adequate internal consistency and construct validity. Validated tools are essential to guide vaccination policy and campaigns towards populations at risk.

Introduction

Since the onset of the COVID-19 pandemic and its subsequent global impact, efforts have been undertaken to rapidly develop a vaccine against SARS-CoV-2 to prevent COVID-19 (COVID-19 vaccine). Vaccines are effective in stopping the spread of disease and prevent an estimated 2–3 million deaths each year [1]. However, there has been an increase in vaccine hesitancy on a national and global level prior to and during the phased roll out of the COVID-19 vaccine [2–4]. Vaccine hesitancy refers to a reluctance or refusal to get vaccinated despite available services [5].

High rates of coverage from a COVID-19 vaccine are required to mitigate transmission. It is estimated that herd immunity to severe COVID-19 would be achieved when approximately 70% of the population has some degree of protection, either from vaccination or natural infection [6]. Failure to achieve high uptake of COVID-19 vaccination in the face of vaccine hesitancy is a potential public health challenge that could leave many individuals vulnerable during the COVID-19 pandemic. Consequently, a standardized measurement tool to identify and respond to COVID-19 vaccine hesitancy in diverse populations is essential.

In reaction to increasing resistance to vaccination, the SAGE Working Group developed a Vaccine Hesitancy Scale (VHS) to quantify vaccine hesitancy among parents [7]. The VHS uses a five-point Likert-type rating scale and in Canada has shown promising validity in identification of vaccine hesitancy. Responses on the VHS were significantly correlated to related vaccine attitudes, showing convergent validity [7]. The VHS has the potential to guide research on immunizations and vaccinations. However, the psychometric properties of the VHS among groups at increased risk of COVID-19 infection and illness due to existing comorbid conditions remains unknown. As such, people with HIV (PWH) may have multiple comorbidities, e.g., high blood pressure, obesity, cardiovascular disease [8–10], that can increase the risk of COVID-19 complications [11, 12]. PWH may also face barriers to care such as stigma, discrimination, and isolation that can interfere with treatment and lead to health disparities [13–15], highlighting the importance to accurately assess and respond to vaccine hesitancy in this population.

Vaccine hesitancy is a multidimensional construct with varied underlying factors [16]. Due to the complexity of assessment of attitudes regarding COVID-19 vaccination, reliable and valid measurement tools that can guide vaccination policy, immunization programs, community based efforts and campaigns are essential [17]. Similarly, strong psychometric properties underlie the accuracy of standardized tools to quantify and interpret vaccine hesitancy among vulnerable populations. Therefore, this study examined the psychometric properties of the VHS among PWH, specifically, the factor structure, reliability, and convergent validity of the VHS among PWH.

Methods

Ethical Approval

Institutional Review Board approval was obtained through the University of Miami (UM) Miller School of Medicine (number: 20200340). Prior to any study related activities, participants’ consent was obtained via telephone.

Participants

Participants were PWH from Miami, Florida who were enrolled in the ACTION study (A Comprehensive Translational Initiative on Novel Coronavirus). The ACTION study recruited PWH through an existing registry of PWH managed by the Miami Center for HIV Research in Mental Health (CHARM) and the Center for AIDS Research (CFAR). Community participants were also recruited using printed flyers, word of mouth, and referrals from medical professionals. Inclusion criteria were (1) English or Spanish fluency, (2) at least 18 years or older, and (3) currently residing in Southeast Florida in the Miami region. COVID-19 risk and exposure were collected from May to December 2020; vaccine data were collected from November to December 2020 using REDCap® software for data collection and storage. Participants recruited from sources outside than the CHARM/CFAR registry provided proof of identity as well as HIV test results, clinician note, or copy of laboratory results prior to enrolment.

Variables/Measures

Sociodemographic variables collected were sex, race, ethnicity, employment status, living situation, and monthly household income.

Vaccine Hesitancy Scale

Ten questions from the VHS [7] were adapted to COVID-19 and included using a five-point Likert scale ranging from 1 (strongly agree) to 5 (strongly disagree). Rather than referring to parental hesitancy regarding vaccines (e.g., “Childhood vaccines are important for my child’s health”), questions were adapted to refer to oneself (e.g., “Vaccination is important for my health”). In addition, one of the items was adapted to expand the perceived risk subscale to be specific for COVID-19: “I do not need vaccines for coronavirus as it will disappear soon” as opposed to “My child/children does or do not need vaccines for diseases that are not common anymore”. Seven additional items were generated using the Health Belief Model [18] to assess general attitudes towards the COVID-19 vaccine, vaccine-related conspiracy beliefs, trust or distrust in medical and governmental institutions, perceived risk or benefit of a COVID-19 vaccine, and willingness to receive the COVID-19 vaccine. The full adaptation of the VHS as well as seven additional items to assess are presented in Tables 2 and 3.

Table 2.

Exploratory factor analysis loadings (N = 173)

Item	Factor 1: confidence	Factor 2: risks	Communalities
Vaccination is important for my health*	0.89		0.79
Getting vaccinated is important for my health and the health of others in my community*	0.87		0.76
Vaccines are effective in preventing disease*	0.81		0.68
All vaccines offered by the government program in my community are important for good health*	0.81		0.65
Getting vaccines is a good way to protect myself from disease*	0.80		0.64
Generally, I do what my doctor or health care provider recommends about vaccines for my health*	0.74		0.56
The information I receive about vaccines from the vaccine program is reliable and trustworthy*	0.61		0.38
New vaccines carry more risks than older vaccines		0.81	0.66
I am concerned about serious adverse effects of vaccines		0.61	0.42
I do not need vaccines for coronavirus at it will disappear soon		0.43	0.25

*Denotes reversed coded item. Survey data was missing for two participants

Table 3.

Vaccine hesitancy scores by beliefs about COVID-19 vaccination (N = 173)

Items	No	Yes	t, p
	M (SD)	M (SD)
If a vaccine to prevent COVID-19 infection was available would you get vaccinated?	27.51 (7.74)	20.53 (4.37)	5.06, < 0.001
Do you believe a vaccine could be an effective way to prevent a COVID-19 infection?	29.35 (8.12)	20.77 (4.51)	4.60 < 0.001
If your doctor or another health professional recommended that you get the COVID-19 vaccine, would you get vaccinated?	29.86 (7.49)	20.99 (4.85)	5.34, < 0.001
Do you feel that your current health status or other health conditions would make it unsafe to get a COVID-19 vaccine?	21.39 (5.57)	24.75 (6.05)	3.13, 0.002
Do you feel that your current health status or other health conditions make it important for you to get a COVID-19 vaccine?	25.91 (6.74)	20.98 (4.87)	5.21, < 0.001
If getting vaccinated was a requirement to go back to your daily activities (working, traveling, public places), would you get vaccinated?	28.62 (7.65)	21.88 (5.26)	3.89, 0. 001
Do you feel that healthcare professionals, government, local authorities or leaders may force you into getting vaccinated?	22.92 (6.16)	23.05 (5.71)	0.13, 0.899

Survey data was missing for two participants

Statistical Analysis

To describe sociodemographic characteristics, univariate analyses were conducted. To examine the factor structure of the VHS, an exploratory factor analysis (EFA) was used. An orthogonal rotation (varimax) was utilized along with principal axis factoring. For factor extraction, principal axis factoring was selected given that it is (a) less likely to yield incorrect solutions [19] and (b) less sensitive to variables that are not normally distributed [20]. To evaluate internal consistency, Cronbach’s α was used. To evaluate construct validity with related vaccine hesitancy items, independent sample t-tests were performed to explore associations. All analyses were performed using IBM SPSS Version 25.

Role of the Funding Source

No funding source had any role in the design or conduct of the study, collection, analysis and interpretation of the data, or writing of the manuscript. All investigators had full access to the data and had final responsibility for the decision to submit the manuscript for publication.

Results

Sociodemographic Characteristics

Participants were on average 51.55 years of age (SD = 13.90), with a range of 21–78, and 55% were women. Racially, 59% of identified as Black/African American, 32% as White/European American, 1% Native American, and 7% other race. Ethnically, nearly half (49%) of participants were non-Hispanic, 41% Hispanic, 2% Haitian, and 7% identified as other race or ethnicity. Most participants were not employed (74%), were living in their own place (69%) and had a monthly household income of USD$1000 or less (60%). Further details on sociodemographic characteristics are presented in Table 1.

Table 1.

Sociodemographic characteristics (N = 175)

	N (%)
Mean age (SD)	51.55 (13.90)
Sex
Male	79 (45.4)
Female	95 (54.6)
Race
Black/African American	104 (59.1)
White	57 (32.4)
Native American	2 (1.1)
Other	12 (6.8)
Ethnicity
Hispanic	72 (40.9)
Non-Hispanic	87 (49.4)
Other	12 (6.8)
Haitian	4 (2.0)
Employed
Yes	44 (25.1)
No	130 (74.3)
Living situation
Own place	121 (69.1)
Someone else’s house	54 (30.9)
Monthly household income
$1000 or less	105 (60.0)
More than $1000	70 (40.0)

Data was missing for 1 person for sex and employment

Internal Consistency and Factor Structure

Cronbach’s α in this sample was 0.72 for the entire scale, 0.90 for “lack of confidence”, and 0.70 for “risks,” suggesting that the reliability of the scale was acceptable. An EFA showed that, consistent with data on parental vaccine hesitancy in the general population, a two-factor structure best explained the underlying constructs in participant responses. The two factors consisted of “lack of confidence” (M = 13.52, SD = 5.34) and “risks” (M = 9.62, SD = 2.05), which explained 45.55% and 12.31% of the variance in participant responses, respectively. For lack of confidence, the eigenvalue was 4.55, and for risks, the eigenvalue was 1.23. Therefore, both of these factors had eigenvalues greater than one. Factor loadings clearly differed between factors; that is, there were no cross-loading items. Factor loadings and communalities for the EFA are shown in Table 2.

Construct Validity

To assess convergent validity, associations were examined between total vaccine hesitancy scores and items related to COVID-19 vaccine attitudes. Participants who reported willingness to be vaccinated had significantly lower hesitancy scores. Similarly, participants who believed that a COVID-19 vaccine would be effective in preventing infection had significantly lower scores compared with participants who did not. Lower hesitancy scores were present in participants who reported that they would get vaccinated if their provider recommended it. Participants who believed that their health conditions would make it unsafe for them to receive the vaccine had greater hesitancy, as did participants who did not perceive that their health conditions made it important for them to receive the vaccine. Lastly, participants who believed that they would not receive a required vaccine showed greater hesitancy scores. Mean scores by groups are presented in Table 3.

Discussion

This study examined the psychometric properties of a modified version of the VHS adapted for COVID-19 in a group of PWH in the US. Overall, this modified VHS showed adequate reliability, internal consistency, construct validity, demonstrating strong psychometric properties to assess attitudes towards COVID-19 vaccination among adult PWH. Convergent validity was demonstrated; those believing that a COVID-19 vaccine would be effective in preventing infection and those willing to be vaccinated had significantly lower hesitancy. Similarly, hesitation to vaccinate was higher among those unwilling to receive the COVID-19 vaccine despite provider recommendations and among those anticipating being required to receive it for daily activities (e.g., work, travel).

Vaccination is an effective method to achieve herd immunity and thereby curb the catastrophic effects of the COVID-19 pandemic worldwide. In line with previous literature, in this sample, vaccine hesitancy was influenced by referent health authority of physicians and by contemporary health issues, COVID-19 beliefs, and the perception that the COVID health risk had been exaggerated [7, 21]. The complexity of attitudes underlying vaccine hesitancy and the potential for a vaccine to mitigate the COVID-19 pandemic underscore the urgent need to accurately measure and address the issues influencing vaccination uptake in populations at highest risk of severe COVID-19, such as PWH with comorbid conditions.

The VHS was originally developed by the SAGE Working Group on Vaccine Hesitancy and evaluated for psychometric properties including internal consistency, construct and criterion validity, and sociodemographic differences among Canadian parents [7]. As with the original use of the tool, the present study used an exploratory factor analysis to understand latent drivers of vaccine hesitancy that underlie responses. The tool was shown to appropriately measure the constructs as indicated with the original use of this VHS on parental vaccine hesitancy. This study replicated the original factor structure of the scale, consisting of “lack of confidence” [in vaccines] and “risks”, which explained substantial amounts of variance in the data.

Measurement and resolution of barriers underlying vaccination hesitancy among PWH could reduce health disparities that could be exacerbated by COVID-19. In previous research, factors facilitating willingness to be vaccinated among high-risk populations included perceived personal susceptibility [22]. Consistent with previous literature, in this sample of PWH, greater hesitancy was identified among those who did not perceive their health status to have an influence on their risk of illness associated with COVID-19. Similarly, those believing that a COVID-19 vaccine would not be effective in preventing COVID-19 infection reported greater hesitancy. Validated measures, such as the adapted VHS, that assess lack of confidence and perceived risk as contributors to vaccine hesitancy among PWH provide an important foundation for tailored, informational campaigns in the phased rollout of COVID-19 vaccines.

Limitations

There were several limitations in the current study. First, the timeline of data collection may have limited responses to a particular collective perception during the pandemic during which knowledge of potential side effects of the vaccine was limited and US rollout had not yet begun. However, this timing may have also increased heterogeneity in responses, which may have helped capture a wider range of perspectives regarding COVID-19 hesitancy. Ongoing assessments of attitudes towards COVID-19 vaccination in this population will be important to gauge community reactions to vaccine rollout in the coming months. In addition, study results represent responses from a sample of PWH in the Southern US and may not be extrapolated to PWH living in other areas of the US or worldwide.

Conclusions

Evaluating and responding to COVID-19 vaccine hesitancy among populations with risk factors associated with increased morbidity and mortality is essential. The use of the modified VHS among PWH indicated that the support of providers and perceived health risk may be linked to reduced vaccine hesitancy and increase the perceived importance of getting vaccinated. A validated tool to measure hesitancy towards a COVID-19 vaccine is a key element in reducing vaccine resistance and supporting the efforts of clinicians, public policy, and tailored vaccine campaigns.

Author Contributions

All authors had full access to all the data in the study and take responsibility for the accuracy of the data and the integrity of the data analysis. VJR, DJL and MLA were responsible for study concept and design. All authors were responsible for acquisition, analysis, and interpretation of data. All authors were responsible for drafting of the manuscript. VJR was responsible for statistical analysis. All authors were responsible for critical revision of the manuscript for important intellectual content. VJR, MLA and DLJ obtained funding.

Funding

National Institutes of Health, National Academies of Sciences, Engineering, and Medicine, and PEO Sisterhood.

Declarations

Conflict of interest

All authors declare no competing interests.