Abstract
Background:
Recent serogroup C meningococcal disease outbreaks led to meningococcal vaccine recommendations for Southern California men who have sex with men (MSM). Assessment of vaccine confidence is critical to improving vaccine coverage in the context of disease outbreaks wherein immunization(s) are recommended.
Methods:
We surveyed MSM using venue-based sampling and began development of the vaccine confidence index (VCI) with 30 survey items corresponding to trust- and safety-related perceptions. We performed exploratory factor analyses and computed the Cronbach’s alpha coefficient to assess internal consistency of the VCI. We created a categorical confidence variable (low, medium, and high confidence) and conducted bivariate and multivariate analyses to evaluate associations with reported confidence and immunization uptake.
Results:
Ten survey items were included in the final VCI and formed the confidence measure. Participants with low confidence had the lowest levels of reported uptake for both meningococcal vaccines. Confidence differed significantly (p ≤ 0.05) between MSM who indicated they received vaccines recommended within the context of the outbreak and those who did not.
Conclusions:
Our VCI is sensitive to a number of issues that may influence vaccine confidence. It is useful for assessing MSM trust and acceptance of recommended immunizations and may be used to inform intervention development.
Keywords: Immunization acceptance, immunization assessment, MSM, vaccine confidence, vaccine measurement
1. Introduction
Adult vaccination coverage in the United States (U.S.) has been consistently suboptimal for most routinely recommended vaccines, and to date there have been no studies that partition vaccination coverage for young adult males [1]. Recent outbreaks of serogroup C meningococcal disease in Southern California have led the California Department of Public Health (CDPH) to recommend quadrivalent meningococcal vaccine (MCV4) for all men who have sex with men (MSM) in Los Angeles, Orange, and San Diego Counties, who collectively have comprised the majority of reported clinically observed meningitis cases [2]. In addition, although serogroup B meningococcal disease was not associated with this outbreak, serogroup B meningococcal vaccines (MenB) were also made available [2]. Despite this, the few reports of vaccine uptake during the outbreak indicate low coverage (e.g., 27–40%) among this group [3-6]. The extent of vaccine confidence in the immunization recommendations among MSM under these circumstances is unknown.
A 2015 report by the National Vaccine Advisory Committee (NVAC) defined vaccine confidence as
the trust that individuals, parents, or health-care providers have (1) in immunizations recommended by the Advisory Committee on Immunization Practices (ACIP), (2) in the provider(s) who administer (s) vaccines, and (3) in the processes that lead to vaccine licensure and the recommended vaccination schedule. [7]
Vaccine confidence underpins vaccine acceptance and uptake, and it has been suggested that vaccine hesitance may stem from a lack of confidence [8,9]. Establishing and maintaining a high level of vaccine confidence is critical to achieving high levels of receipt of recommended vaccinations, yet this construct has been largely unstudied among MSM and in outbreak settings.
Many facilitators and barriers to vaccine uptake among MSM, irrespective of HIV status, have been identified [10,11]. Facilitators of vaccine uptake among MSM include provider recommendation, higher disease knowledge, more favorable attitudes, and higher perceived social norms [12,13]. Lack of a provider recommendation, lack of knowledge about the disease or vaccine, not perceiving one’s self to be at risk for a disease, and concerns over vaccine safety are often reported as primary barriers to vaccine acceptance among MSM [12,14]. These factors are not exclusively reported among MSM and have been observed in other populations [15,16]. However, other studies have identified factors related to vaccine uptake unique to MSM, such as (non)disclosure of sexual orientation to health-care providers and HPV-specific fears (e.g., contracting genital warts) [12,17].
Over the past decade, considerable momentum has been generated in defining and measuring vaccine acceptance, trust, and confidence among global populations. Expert committees including the SAGE Working Group and the NVAC spearheaded collaborations that have resulted in measurement approaches for non-MSM populations [18-22]. For example, the 2015 NVAC report both recommended creation of an index that could accurately measure vaccine confidence levels and identified a set of domains often found to be determinants of, or related to, confidence [7]. With input from experts in the vaccine field, the guidance offered by the report suggested that vaccine confidence should focus on aspects of trust in three key areas including (1) federal advisory committee immunization recommendations, (2) immunization providers, and (3) vaccine licensure processes and schedule recommendations [23].
Other measures have primarily focused on parents and childhood immunization recommendations [18,19,23,24]. The scale established by Gilkey and colleagues, originally constructed from questions on the 2010 National Immunization Survey – Teen, was later validated using data from the 2011 National Immunization Survey [20]. Its focus is on the assessment with a different population of parents related to adolescent vaccination behavior. The Parental Attitudes about Childhood Vaccines (PACV) survey has also assessed a broad range of vaccine attitudes among parents of children 19–35 months of age, and it is a measure of vaccine hesitancy [21,22,25]. A recent study found that the five clusters of vaccine opinions identified by Gust and colleagues [24] and the PACV were well correlated, but called for more comprehensive research to assess additional factors associated with vaccine hesitance [26]. While considerable work has been conducted to quantify vaccine hesitance (e.g. in terms of refused or delayed vaccinations), no measure of vaccine confidence has yet been developed specifically for use with MSM.
Precise assessment and monitoring of vaccine confidence is critical to improving vaccine coverage, especially among vulnerable populations such as MSM or in the context of vaccine-preventable diseases outbreaks wherein immunizations are recommended for special populations [27,28]. In response to this need, Emory University and the National Vaccine Program Office, in collaboration with the California HIV/AIDS Policy Research Center, initiated a project to develop a measure that would be capable of assessing vaccine confidence among MSM relative to a recommended adult immunization. This project also endeavored to create an early-stage MSM vaccine confidence index (‘MSM VCI’) that would ultimately be utilized for timely detection of changes in confidence (e.g., as a predictor of potential declines in immunization coverage or acceptance). The MSM VCI was created as a first step in the development of a potential tool that could aid researchers, clinicians, and public health practitioners to monitor vaccine confidence in this special population as a potential indicator of vaccine behavior.
2. Patients and methods
2.1. Data collection
Data were collected from November 2016 through February 2017 as part of a larger project to understand MSM vaccination response in the context of an ongoing meningitis outbreak in Los Angeles County, California [1,2]. The North Campus Institutional Review Board (IRB) at the University of California, Los Angeles, reviewed and approved the study prior to survey implementation. The Emory University IRB approved this data analysis.
2.2. Measures
Three outcomes were analyzed. The first two outcomes were self-reported uptake (yes/no/unsure) of the recommended meningococcal quadrivalent vaccine (MCV4) and the meningococcal B vaccine (MenB) (yes/no/unsure). We also measured men’s perceived importance (important/not important/unsure) of 13 vaccines for their health. The list included MCV4, MenB; human papillomavirus (HPV); hepatitis A; hepatitis B; influenza; Haemophilus Influenzae type B (Hib), measles, mumps, rubella (MMR); tetanus, diphtheria, and acellular pertussis (Tdap); varicella; herpes zoster (zoster); 13-valent pneumococcal conjugate (PCV13); and 23-valent pneumococcal polysaccharide (PPSV23). For this outcome, vaccine importance was scored as ‘yes’ for important, ‘no’ for unimportant, and ‘unsure.’ As the responses of interest was uptake and perceived importance, we collapsed ‘no’ and ‘unsure’ responses for analysis across all three outcomes. Furthermore, the intention was to measure uptake within the context of the ongoing outbreak, so we only assessed uptake of those vaccines directly relevant (i.e., MCV4 and MenB); however, we expanded vaccines considered for perceived importance in order to obtain a comprehensive landscape of vaccine perceptions among MSM.
In our final models, we sought to parcel out the influence of vaccine confidence on vaccine uptake and importance after accounting for potentially relevant covariates. These included participants’ age, race/ethnicity, education, income, employment status, health insurance, residential ZIP code, substance use, sexual risk behaviors (e.g., condomless receptive anal sex in the past 6 months), sexually transmitted infection (STI) history, health promoting behaviors (e.g., pre-exposure prophylaxis use among those who self-reported being HIV-negative), and HIV status. Age and drug use were used to develop a risk score using previously established criteria [29].
2.3. Statistical analysis
We began development of the VCI with 30 survey items identified from the vaccine confidence-related framework developed by NVAC, following similar methodology used in the development of three parallel indices for parental vaccine confidence [30]. These items were classified as they corresponded to the trust- and safety-related perceptions. We also included items that gaged perceptions of the ‘Information Environment,’ trust in their ‘Healthcare Provider,’ and general vaccine ‘Attitudes and Beliefs.’ [7] Each item was scored on a five-point Likert scale from 1 to 5, representing ‘Strongly Disagree,’ ‘Disagree,’ ‘Neutral,’ ‘Agree,’ and ‘Strongly Agree,’ respectively.
We performed an exploratory factor analysis and computed the Cronbach’s alpha coefficient to assess internal consistency. The exploratory factor analyses of 30 items resulted in a reduction to 10 items that were included as a single scale forming the current measure; all other items did not converge to meet our a priori criteria of α ≥ 0.70 for high internal consistency. We evaluated characteristics of the sample and performed descriptive analyses using SAS 9.3 (SAS Institute, Cary, NC). The summary score was created by summing scores across each of these variables, resulting in a potential range of 10–50.
We originally stratified summary scores into quartiles (10–34, 35–38, 39–43, 44–50). However, we determined that >50% of respondents fell into the second and third quartiles, so we adjusted the score partitions into tertiles to reflect the relevant distinctions between low, medium, and high vaccine confidence. After scores were stratified into tertiles, we assessed reliability by testing the association between the scores and self-reported vaccination status (i.e., MCV4 and/or MenB uptake) and vaccination importance. Self-reported receipt of MCV4 and MenB, and responses for the 13 vaccine importance questions, were compared across this categorical vaccine confidence score using the Cochran–Armitage test for trend. For each vaccine uptake/importance outcome, we conducted binary logistic regression to estimate the increase in odds of uptake/importance for a one-point increase in VCI summary score. We also included a sensitivity analyses to evaluate definitive confidence alterations that may be affected by assessing those included in both ‘no’ and ‘no/unsure’ evaluative options.
Following psychometric development and testing, we determined it was important to include additional psychosocial items consistent with a socioecological framework (i.e., taking into account the interactive effects of personal, social, and environmental factors that determine behaviors) in the final multivariable models. We included key personal, social, and contextual factors in order to be reflective of an outbreak environment and test its sensitivity to these unique socioecological conditions. All models were therefore assessed and adjusted for age-group (years), race/ethnicity, education level, employment status, income level, insurance status, high MSM residential ownership/occupancy district (e.g., West Hollywood residence vs. others), HIV status, alcohol use, marijuana use, other drug use, STI history, sex of sexual partners, number of sexual partners in the last 6 months, receptive condomless anal sex, CDC risk score, PrEP use, and whether the respondent took a multivitamin. All analyses were performed at α = 0.05 significance level.
Finally, to assess compatibility with the framework articulated in the NVAC report, we compared the factors included in the measure developed for MSM here to the factors included in three variations of another parental VCI developed by our study authors [30]. The MSM and parental measures were all developed using the 30 survey items identified from the vaccine confidence-related framework developed by NVAC and reflect highly acceptable internal validity scores.
3. Results
3.1. Sociodemographic characteristics
Using venue-based convenience sampling, we were able to approach 2,250 men about potential participation in the study. Those who expressed interest were then asked a brief series of questions to assess eligibility. Persons were eligible to enroll based on the following criteria: (a) English- or Spanish-speaking; (b) recent sexual male-to-male sexual encounters (i.e., ≥2 anal or oral experiences ≤6 months); (c) noninstitutionalized males aged ≥18 years living in Los Angeles County, California; and (d) willing and able to provide verbal consent. Eligible MSM were invited in person to take the 15–30-min survey (mean = 17.6 min) delivered by trained interviewers on iPads in field settings. All participants received $50 in cash for completing the survey. Of the men approached, 749 were screened, of which 520 provided valid responses, for an overall response rate of 69%. Of the 520 participants who provided valid responses, 24 (4.8%) had a missing response for at least 1 of the 10 items used to assess vaccine confidence. A bivariate analysis between those with VCI scores and those without showed that on all relevant factors but one (meningitis B vaccination uptake) of the two groups look indistinguishable and suggests those without VCI scores may be missing at random. Additionally, as the proportion of missing scores is less than 5%, results below are unlikely to be significantly altered. Excluding the missing scores from the analysis subsequently yields a final sample size of 496.
Respondent sociodemographic characteristics are described in Table 1. Participants were young adults (mean = 33.3 years) and racially/ethnically diverse: White (35.2%), Black/African American (15.4%), Hispanic (32.8%), and Other/Multiethnic (16.6%). Half (50.1%) were college educated, one-fifth (20.7%) made less than $20,000 per year, and the majority (88.5%) had some form of health insurance. Approximately one-third (34.7%) resided in West Hollywood, a popular gay enclave in Los Angeles County. Most (89.9%) reported sex with men exclusively, and 42.0% reported at least four sexual partners in the past 6 months. Half (52.2%) reported condomless receptive anal sex within the same time frame. Nearly 12% (11.8%) reported being HIV-positive.
Table 1.
Survey respondent characteristics.
| Mean | 95% CI | |
|---|---|---|
| Sociodemographic characteristics | ||
| Age (years) | 33.3 | (32.4, 34.2) |
| Age groups (count, percent) | ||
| 18–24 | 101 | 20.4% |
| 25–44 | 319 | 64.3% |
| 45–64 | 72 | 14.5% |
| ≥65 | 4 | 0.8% |
| Race/ethnicity (count, percent) | ||
| White | 174 | 35.2% |
| Black | 76 | 15.4% |
| Hispanic | 162 | 32.8% |
| Other | 82 | 16.6% |
| Education (bachelor’s degree or more) | 50.1 | (46.2, 55.0) |
| Employment (full time) | 61.9 | (57.6, 66.2) |
| Income US$ (≥20,000) | 79.3 | (75.7, 82.9) |
| Insured | 88.5 | (85.7, 91.4) |
| West Hollywood residence | 34.7 | (30.5, 38.9) |
| Health statistics | ||
| HIV-positive | 11.8 | (9.0, 14.7) |
| Received MCV4 | 27.0 | (23.1, 30.9) |
| Received MenB | 17.7 | (14.4, 21.1) |
| Alcohol use | 86.7 | (83.7, 89.7) |
| Marijuana use | 51.4 | (47.0, 55.8) |
| Other drugs | 48.8 | (44.4, 53.2) |
| STI | 46.8 | (42.4, 51.2) |
| Sex of sexual partners (count, percent) | ||
| Men only | 446 | 89.9% |
| Men and women | 50 | 10.1% |
| No. of partners (count, percent) | ||
| 0–3 | 286 | 58.0% |
| 4–7 | 109 | 22.1% |
| ≥8 | 98 | 19.9% |
| Receptive anal sex | 52.2 | (47.8, 56.6) |
| CDC risk score | 15.4 | (14.5, 16.2) |
| PrEPa | 17.0 | (13.4, 20.5) |
| Multivitamin | 51.5 | (47.1, 55.9) |
Calculated only for those who are HIV-negative (N = 430).
3.2. Vaccine confidence factor analysis
The standardized Cronbach’s alpha score for the 10 variables included in the scale was 0.89, demonstrating high internal consistency and reliability. Mean scores and factor analysis are reported for the 10 survey items used to form the measure in Table 2. The factor analysis demonstrated that all 10 component scores were highly associated with a single factor (i.e., ‘MSM vaccine confidence’); all factor loadings scores were ≥0.60, suggesting the 10 components were targeting the same latent factor.
Table 2.
Comparison of sociodemographic and health statistics between those with and without VCI scores.
| Not missing (N = 496) | Missing (N = 24) | p-Value | |
|---|---|---|---|
| Sociodemographic characteristics | |||
| Age (years) | 33.3 | 32.8 | 0.40 |
| Age groups (count, percent) | |||
| 18–24 | 101 (20.4) | 5 (20.8) | 0.96 |
| 25–44 | 319 (64.3) | 15 (62.5) | |
| 45–64 | 72 (14.5) | 4 (16.7) | |
| ≥65 | 4 (0.81) | 0 (0) | |
| Race/ethnicity (count, percent) | |||
| White | 174 (35.2) | 9 (39.1) | 0.53 |
| Black | 76 (15.4) | 2 (8.7) | |
| Hispanic | 162 (32.8) | 6 (26.1) | |
| Other | 82 (16.6) | 6 (26.1) | |
| Education (bachelor’s degree or more) | 50.1 | 66.7 | 0.87 |
| Employment (full-time) | 61.9 | 70.8 | 0.84 |
| Income US$ (≥20,000) | 79.3 | 70.8 | 0.31 |
| Insured | 88.5 | 82.6 | 0.15 |
| West Hollywood residence | 34.7 | 20.8 | 0.43 |
| Health statistics | |||
| HIV-positive | 11.8 | 12.5 | 0.71 |
| Received MCV4 | 27.0 | 16.7 | 0.38 |
| Received MenB | 17.7 | 4.2 | <0.01* |
| Alcohol use | 86.7 | 87.5 | 0.99 |
| Marijuana use | 51.4 | 54.2 | 0.84 |
| Other drugs | 48.8 | 58.3 | 0.89 |
| STI | 46.8 | 66.7 | |
| Sex of sexual partners (count, percent) | |||
| Men only | 446 (89.9) | 21 (87.5) | 0.70 |
| Men and women | 50 (10.1) | 3 (12.5) | |
| No. of partners (count, percent) | |||
| 0–3 | 286 (58) | 10 (41.7) | 0.28 |
| 4–7 | 109 (22.1) | 7 (29.2) | |
| ≥8 | 98 (19.9) | 7 (29.2) | |
| Receptive anal sex | 52.2 | 45.8 | 0.83 |
| CDC risk score | 15.4 | 17.0 | 0.23 |
| PrEPa | 17.0 | 28.6 | 0.17 |
| Multivitamin | 51.5 | 45.8 | 0.84 |
Calculated only for those who are HIV-negative (N = 430 and 21).
The range of scores in this sample ranged from 13 to 50 (mean = 38.9, standard deviation = 6.64). The summary scores were stratified into a three-level categorical variable representing low, medium, and high vaccine confidence: 10–30, 31–44, and 45–50. On average, scores corresponded to moderate levels of vaccine confidence, although the distribution had a slight negative skew.
3.3. Trend test and logistic regression results
Participants in the lowest vaccine confidence category (score 10–30) had the lowest levels of reported uptake for both meningitis vaccines and indicated the lowest levels of importance of other vaccines (Table 3). Reported uptake/importance of each vaccine increased with increasing measurement category for all but Hib (p-value = 0.04). For each vaccine, the Cochran–Armitage test for trend indicated statistically significant increases in reported uptake/importance with each higher measurement category.
Table 3.
Vaccination confidence component scores and factor analysis.
| Variable | Mean | SD | n | Factora |
|---|---|---|---|---|
| Vaccines recommended for men who have sex with men are safe. Y | 3.9 | 1.01 | 514 | 0.70 |
| Getting immunized is one of the best things to do to improve my health. N | 4.1 | 0.89 | 516 | 0.78 |
| Vaccines recommended for men who have sex with men are effective. N | 3.9 | 0.93 | 513 | 0.78 |
| Clinical trials result in safe and effective vaccines. | 4.0 | 0.92 | 519 | 0.75 |
| Independent vaccine advisory committees make trustworthy vaccine recommendations for the federal government. | 3.6 | 0.98 | 507 | 0.72 |
| Not getting immunized increases my chances of getting a serious disease. N | 4.1 | 0.98 | 518 | 0.60 |
| My doctor/nurse is a reliable source of trustworthy vaccine information. Y | 4.2 | 0.77 | 519 | 0.65 |
| Companies that make vaccines produce safe and effective vaccines | 3.6 | 0.96 | 517 | 0.77 |
| Vaccines are made with safe ingredients | 3.3 | 1.01 | 518 | 0.72 |
| My doctor/nurse has my best health interest in mind when making vaccine recommendations. Y | 4.1 | 0.88 | 517 | 0.68 |
| Total | 3.89 | 0.66 | 496 | |
| Cronbach alpha: α = 0.89 (standardized) |
We note the presence of a single factor; the component scores are highly associated with a single factor.
Y: yes; N: no.
We conducted 30 separate logistic regression models (15 unadjusted, 15 adjusted by sociodemographic and behavioral covariate measures) to test whether the scale predicted reported uptake/importance of each vaccine (Table 4). Results for unadjusted and adjusted models across vaccines, in terms of increased odds of vaccination uptake/importance with increasing categorical scores, were consistent; this suggested that the sociodemographic and behavioral covariates did not significantly influence its relationship on vaccine uptake/importance. We found statistically significant associations between vaccine uptake/perceived importance and scaled scores in every regression model (p < 0.05). For a one-point increase in our confidence scale, the odds that a participant reported vaccine receipt/importance increased by approximately 4–13% in the final models.
Table 4.
Percent of participants in each VCI level reporting ‘yes’ to outcomes.
| Outcome | 10–30, n = 43 | 31–44, n = 347 | 45–50, n = 106 | Chi-square | Cochran–Armitage |
|---|---|---|---|---|---|
| Vaccination uptake | |||||
| MCV4 | 14.0 | 25.9 | 35.9 | 0.02* | <0.01* |
| MenB | 11.6 | 15.0 | 29.3 | <0.01* | <0.01* |
| Vaccination importance | |||||
| HPV | 55.8 | 64.6 | 75.5 | 0.04* | <0.01* |
| MCV4 | 44.2 | 57.9 | 62.3 | 0.13 | 0.03* |
| MenB | 39.5 | 57.1 | 66.0 | 0.01* | <0.01* |
| HAV | 69.8 | 84.7 | 90.6 | <0.01* | <0.01* |
| HBV | 69.8 | 87.6 | 96.2 | <0.01* | <0.01* |
| Flu | 46.5 | 61.4 | 80.2 | <0.01* | <0.01* |
| Hib | 44.2 | 40.6 | 54.7 | 0.04* | 0.03* |
| MMR | 67.5 | 73.5 | 87.7 | <0.01* | <0.01* |
| Td/Tdap | 65.1 | 82.1 | 88.7 | <0.01* | <0.01* |
| Varicella | 55.8 | 71.2 | 74.5 | 0.07 | 0.03* |
| Zoster | 53.5 | 68.9 | 73.6 | 0.06 | 0.02* |
| PCV13 | 34.9 | 41.8 | 54.7 | 0.03* | <0.01* |
| PPSV23 | 34.9 | 39.2 | 52.8 | 0.03* | <0.01* |
Significance at α = 0.05 level.
We conducted three sensitivity analyses with cut points made to assess definitive responses among low, medium, and high confidence. In the first, we considered ‘unsure’ responses as missing instead of collapsing with ‘no’ responses, which did not significantly alter results. Cochran–Armitage trend test and logistic regression coefficient p-values remained unchanged. As expected, the proportion of ‘yes’ responses in Table 3 increased as the total number of responses considered decreased. We considered different cutoff points for the score tertiles during the second and third sensitivity analyses. We used the cutoffs 10–31, 20–43, and 44–50 for the second sensitivity analysis, and 10–29, 30–43, and 44–50 for the third one; neither variation resulted in significant deviation from the original groupings presented in Table 3.
3.4. VCI compatibility
The VCI developed in this study included 10 survey items derived from guidance provided in the NVAC report and the study authors’ previously developed 30-item vaccine confidence framework [30]. Although the measures (MSM VCI and parental VCI) were developed to target different study populations, three key constructs were consistent between the three final indices (Table 5): perceived safety of recommended vaccines (for MSM and children, respectively), perception that health-care providers make vaccine recommendations with a patient’s best health interest in mind, and trust in entities that make or recommend vaccines. The MSM VCI was originally stratified into quartiles (10–34, 35–38, 39–43, and 44–50). However, it was determined that since >50% of respondents in this MSM sample fell into the second and third quartiles, adjusting the score partitions to parallel the other VCI measures would result in a more consistent tool. This did not alter the statistical significance of any outcome and had minimal impact on the percentage of respondents falling into each vaccine confidence category (results not shown). In addition, we noted cross-compatibility with other previously developed vaccine confidence measures, which use components such as vaccine importance, perceived safety, perceived efficacy, and trust in health-care providers (Table 6) [20,31].
Table 5.
Unadjusted and adjusted logistic regression of MCV4 and MenB vaccine uptake and perceived vaccine importance with VCI score.
| Outcome | OR (95% CI) | p-Value | AOR (95% CI) | p-Value |
|---|---|---|---|---|
| Vaccination uptake | ||||
| MCV4 | 1.07 (1.04, 1.11) | <0.01* | 1.05 (1.01, 1.09) | 0.02* |
| MenB | 1.09 (1.05, 1.13) | <0.01* | 1.08 (1.03, 1.14) | <0.01* |
| Vaccination importance | ||||
| HPV | 1.05 (1.02, 1.08) | <0.01* | 1.05 (1.01, 1.09) | 0.01* |
| MCV4 | 1.05 (1.02, 1.08) | <0.01* | 1.04 (1.01, 1.08) | 0.02* |
| MenB | 1.07 (1.04, 1.10) | <0.01* | 1.07 (1.03, 1.11) | <0.01* |
| HAV | 1.08 (1.04, 1.12) | <0.01* | 1.07 (1.02, 1.13) | <0.01* |
| HBV | 1.12 (1.07, 1.17) | <0.01* | 1.13 (1.07, 1.20) | <0.01* |
| Flu | 1.08 (1.05, 1.12) | <0.01* | 1.09 (1.05, 1.14) | <0.01* |
| Hib | 1.05 (1.02, 1.08) | <0.01* | 1.06 (1.03, 1.10) | <0.01* |
| MMR | 1.08 (1.04, 1.11) | <0.01* | 1.09 (1.04, 1.14) | <0.01* |
| Td/Tdap | 1.07 (1.03, 1.11) | <0.01* | 1.07 (1.02, 1.12) | <0.01* |
| Varicella | 1.03 (1.00, 1.06) | 0.04* | 1.06 (1.02, 1.10) | 0.01* |
| Zoster | 1.04 (1.01, 1.07) | 0.01* | 1.05 (1.01, 1.09) | 0.01* |
| PCV13 | 1.05 (1.02, 1.08) | <0.01* | 1.07 (1.03, 1.11) | <0.01* |
| PPSV23 | 1.05 (1.03, 1.09) | <0.01* | 1.07 (1.03, 1.11) | <0.01* |
Multivariable models adjusted for age-group (years), race/ethnicity, education level, employment status, income level, insurance status, West Hollywood residence, HIV status, alcohol use, marijuana use, other drug use, STI history, sex of sexual partners, number of sexual partners in the last 6 months, receptive condomless anal sex, CDC risk score, PrEP status, and whether respondent takes a multivitamin.
Significance at α = 0.05 level.
Table 6.
Survey component similarity across indexes. Full color available online.
| Variable | VCI MSM α = 0.890 |
EVCI-R α = 0.857 |
EVCI-S α = 0.812 |
EVCI-C α = 0.774 |
NVAC 30 item |
|---|---|---|---|---|---|
| Vaccines recommended for [subpopulation of interest] are safe. | Yes | Yes | Yes | Yes | Yes |
| Getting immunized is one of the best things to do to [improve/protect] [subpopulation’s] health. | Yes | No | No | No | Yes |
| Vaccines recommended for [subpopulation of interest] are effective. | Yes | No | No | No | Yes |
| Clinical trials result in safe and effective vaccines | Yes | No | No | No | Yes |
| Independent vaccine advisory committees make trustworthy vaccine recommendations for the federal government | Yes | Similar | Similar | Similar | Similar |
| Not getting immunized increases [subpopulation’s] chances of getting a serious disease | Yes | No | No | No | Yes |
| My doctor/nurse is a reliable source of trustworthy vaccine information | Yes | Yes | No | No | Yes |
| Companies that make vaccines produce safe and effective vaccines | Yes | No | No | No | Yes |
| Vaccines are made with safe ingredients | Yes | No | No | No | Yes |
| My doctor/nurse has [subpopulation’s] best health interest in mind when making vaccine recommendations. | Yes | Yes | Yes | Yes | Yes |
| It is important for everyone to get the recommended vaccines for their child(ren) | No | Yes | Yes | Yes | Yes |
| Trust in: Food & Drug Administration (FDA), the federal government agency that licenses vaccines | Similar | Yes | No | No | Yes |
| Trust in: Centers for Disease Control and Prevention (CDC), the federal government agency that makes recommendations about who should get licensed vaccines | Similar | Yes | No | No | Yes |
| Trust in: Federal government agencies responsible for monitoring the safety of recommended childhood vaccines | Similar | Yes | Yes | Yes | Yes |
| Trust in: Scientists involved in developing and testing new vaccines | Similar | Yes | Yes | Yes | Yes |
| Vaccines prevent serious disease | No | No | Yes | No | Yes |
Boldface indicates measure was included in the MSM VCI.
Subpopulations of interest include ‘men who have sex with men’ and ‘young children <7 years.’
Trust was measured by five separate variables: two measured trust in non-specific government entities and three measured trust in specific entities (the FDA, CDC, and scientists). ‘Similar’ denotes the presence of a trust variable, but indicates the level of specificity may vary by index.
4. Discussion
This study responds directly to NVAC’s recommendation for the creation and evaluation of a VCI associated with the decision-making domains they identified as important to vaccine acceptance [7]. Our study is the first to examine MSM vaccine confidence among a sample of MSM during an ongoing invasive meningococcal disease (IMD) outbreak. Despite recommendations by the CDPH, the Los Angeles County Department of Public Health, and the ACIP, we found that during this outbreak, self-reported immunization among MSM was low, regardless of HIV status (37.7% among HIV-positive and 25.4% among HIV-negative respondents) [27]. In response, we developed a parsimonious 10-item index that can rapidly assess vaccine confidence among this population with a reliable internal validity score in future research.
Other recent indices have been constructed and assessed in populations with young children (≤10 years of age) or adolescents [18,20,25], but there have been no previous efforts to study vaccine confidence among MSM or to develop a tool tailored to this population until now. MSM continue to be a high-risk group for multiple health outcomes beyond HIV/AIDS, yet their inclusion in research is limited. Having a clearer accounting of both vaccine uptake and confidence about vaccines among MSM will inform public health and aid in the development and provision of effective education, outreach, and surveillance tools for maintaining high vaccine coverage.
The categories of vaccine confidence scores we identified correspond well with MSM-reported vaccine receipt of MCV4 and MenB recommended vaccines, with the percent of MSM who reported being vaccinated increasing in a stepwise fashion for each increasing confidence category. In our survey, we collected additional sociodemographic variables to be used in future analyses to gauge overall levels of MSM vaccine confidence, assess associations between sociodemographic variables and confidence, and to identify clusters of MSM with lower confidence for targeted outreach. By incorporating the sociodemographic variables, reports of individual vaccinations, and vaccine confidence scores, these analyses may provide more comprehensive explanations for MSM’s acceptance and uptake of recommended vaccines in the U.S.
The consistency of increased odds of self-reported vaccine receipt/importance with increasing the score, combined with similar trends in vaccine receipt/importance across categories, indicate the robustness of this scale in identifying vaccine confidence factors associated with vaccine receipt. Additionally, while the statistical significance of the results from the unadjusted and adjusted logistic regression models were consistent, adjustment for MSM sociodemographics led, in some cases, to slight variations in odds ratios. This is in line with prior studies identifying the association of these sociodemographics (such as younger age and education) with vaccine uptake [12,13,32].
Sexual orientation is currently not linked with vaccination data in immunization information systems (IIS). Our work is especially important to initiate standardized data collection on sexual orientation and sexual behavior in IIS for subsequent vaccine confidence monitoring that can be linked to vaccine uptake across the country [6,33,34]. Additionally, with recent IMD outbreaks elsewhere in the U.S., developing measures that will monitor any changes in MSM vaccine confidence will enhance public health efforts, especially during future outbreaks [34,35]. Use of this tool could provide a basis for ongoing surveillance that can be readily available to providers and policymakers to help inform continued vaccination recommendations and promotion.
4.1. Limitations
Study findings are subject to limitations. The study sample is a self-selected convenience sample, and as such does not enable generalizability to a broader population. Although we utilized a sampling strategy that focused on venues traditionally frequented by MSM and would theoretically result in greater generalizability among this population, we acknowledge that this sample still may not be representative of MSM in Los Angeles County. Future efforts to reach subgroups of MSM in nontraditional settings who may be socially and/or geographically isolated within Southern California would contribute to these findings and be beneficial to ongoing outreach campaigns. Additionally, survey data relied on participants’ self-report and were not verified. Via additional research, this index should be implemented at subsequent points in time, particularly in conjuncture with immunization records [36-38]. Several sources of bias may limit the ability of self-reported vaccination decisions to represent actual vaccination behavior, including recall, response, and social desirability bias. Due to the ‘rapid response’ nature of this project, we were not able to implement third party verification of vaccination status. Future studies should seek to implement more stringent protocols to verify immunization status in collaboration with health-care providers or with data from immunization registries. Our data are cross-sectional making causal inference impossible.
5. Conclusions
We developed a measure of vaccine confidence that can serve as an efficient way to identify MSM confidence in recommended vaccinations during an immunization campaign or across time. Further application of this work may provide additional insights into populations at risk for incomplete vaccination. Future validation of the scale may help public health programs and efforts better integrate the concept of vaccine confidence into their immunization interventions, including through communications and activities designed to create or increase MSM confidence in recommended vaccines.
6. Key issues.
This study was performed to respond to the recommendation to create and evaluate a VCI associated with the decision-making domains identified as important to vaccine acceptance.
It is among the first studies to examine MSM vaccine confidence among a sample of MSM during an ongoing IMD outbreak.
During the Southern California outbreak, self-reported immunization among MSM was low, regardless of HIV status and recommendations from public health alerts.
Categories of vaccine confidence scores identified correspond well with MSM-reported vaccine receipt of MCV4 and MenB recommended vaccines, with the percent of MSM who reported being vaccinated increasing in a stepwise fashion for each increasing confidence category.
A clearer accounting of both vaccine uptake and confidence about vaccines among MSM will inform public health and aid in the development and provision of effective education, outreach, and surveillance tools for maintaining high vaccine coverage.
Acknowledgments
We wish to thank the participants who responded to our survey and to the staff of partner venues for their support during the implementation of this study. We also wish to thank Drs Patrick Sullivan and Kimberley Hagen (Emory University) and the Prevention Science and Development Cores at the Emory Center for AIDS Research for support of this collaboration. Special thanks to Adam Cohen, Matthew Beymer, Adrian King, and Michelle Parra for their insights and input in assisting with the development and presentation of final results of the study.
Funding
This study was supported by a grant from the California HIV/AIDS Research Program [RP15-LA-007], a grant from the US Department of Health and Human Services, National Vaccine Program Office [1VSRNV000003], and a collaboration travel grant from the Emory University Center for AIDS Research [P30 AI050409].
Footnotes
Declaration of interest
PM Frew is the PI of a cooperative agreement to Emory University from NVPO (HHS) that provided salary/travel support to the project and received collaboration/travel support funds from the Emory University Center for AIDS Research (CFAR) and the California HIV/AIDS Policy Research Center. IW Holloway is the PI of the California HIV/AIDS Policy Research Center that provided grant (salary/travel) support to this project. C Goldbeck is a graduate student/data analyst with the California HIV/AIDS Policy Research Center that provided grant/salary support to this project. D Tan is a graduate student/data analyst with the California HIV/AIDS Policy Research Center that provided grant/salary support to this project. E Wu is a project manager with the California HIV/AIDS Policy Research Center that provided grant support to this project. J Jauregui is a graduate student/data analyst with the California HIV/AIDS Policy Research Center that provided grant support to this project. V Fenimore was a staff research associate/data analyst with the California HIV/AIDS Policy Research Center that provided grant support to this project. L Randall is the project coordinator of a cooperative agreement to Emory University from NVPO (HHS) that provided salary/travel support to the project and received additional travel support the California HIV/AIDS Policy Research Center.
C Lutz is the data analyst of a cooperative agreement to Emory University from NVPO (HHS) that provided salary support to the project. J. Mendel is the project officer of a cooperative agreement to Emory University from NVPO (HHS). A. Aikin is a technical expert to a cooperative agreement to Emory University from NVPO (HHS). G. Nowak was an external expert advisor to a cooperative agreement to Emory University from NVPO (HHS). R Bednarczyk is the PI of a grant from the National Institute for Allergy and Infectious Diseases, National Institutes of Health that provided salary/travel support to the project; R Bednarczyk is a PI of a cooperative agreement to Emory University from NVPO (HHS) that provided salary/travel support to the project. All authors declare there are no known conflicts to disclose. The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.
Reviewer disclosures
Peer reviewers on this manuscript have no relevant financial or other relationships to disclose.
Disclaimer
The findings and conclusions in this article are those of the authors and do not necessarily represent the official position of the U.S. Department of Health and Human Services.
References
Papers of special note have been highlighted as either of interest (•) or of considerable interest (••) to readers.
- 1.Williams WWL, Wan P-J, O’ Halloran A, et al. Surveillance of vaccination coverage among adult populations — United States, 2015. MMWR Surveill Summ 2017. 2017;66(11):1–28. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Mascola DL. Los Angeles County Department of Public Health alert: outbreak of meningococcal disease among adult males in Southern California. 2016. [cited 2017 Sep 21]. Available from: http://publichealth.lacounty.gov/eprp/Health%20Alerts/LACDPH%20Alert%20IMD%206%2024%2016.pdf [Google Scholar]
- 3.Nanduri S Outbreak of serogroup c meningococcal disease primarily affecting men who have sex with men—Southern California. MMWR Morb Mortal Wkly Rep. 2016;65(35):939–940. [DOI] [PubMed] [Google Scholar]
- 4.California Department of Public Health. CDPH issues health advisory for meningococcal disease. 2016. [cited 2017 Mar 3]. Available from: http://www.cdph.ca.gov/Pages/NR16-037.aspx [Google Scholar]
- 5.Koch J, Hellenbrand W, Schink S, et al. Evaluation of a temporary vaccination recommendation in response to an outbreak of invasive meningococcal serogroup C disease in men who have sex with men in Berlin, 2013–2014. Euro Surveill. 2016;21(5):12–22. [DOI] [PubMed] [Google Scholar]
- 6.Aubert L, Taha M, Boo N, et al. Serogroup C invasive meningococcal disease among men who have sex with men and in gay-oriented social venues in the Paris region: July 2013 to December 2014. Euro Surveill. 2014;20:3. [DOI] [PubMed] [Google Scholar]
- 7. Orenstein WA, Gellin BG, Beigi RH, et al. Assessing the state of vaccine confidence in the United States: recommendations from the National Vaccine Advisory Committee. Public Health Rep. 2015;130(6):573–595. •• This reference is the call to action to address vaccine confidence in the U.S. and for the development of scales to measure vaccine confidence.
- 8.Nowak GJ, Cacciatore MA. Parents’ confidence in recommended childhood vaccinations: extending the assessment, expanding the context. Hum Vaccin Immunother. 2017;13(3):687–700. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Mendel-Van Alstyne J, Nowak GJ, Aikin AL. What is ‘confidence’ and what could affect it? A qualitative study of mothers who are hesitant about vaccines. Vaccine. 2017;36(44):6464–6472. [DOI] [PubMed] [Google Scholar]
- 10.Holman DM, Benard V, Roland KB, et al. Barriers to human papillomavirus vaccination among US adolescents: a systematic review of the literature. JAMA Pediatr. 2014;168(1):76–82. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Rosenthal SL, Weiss TW, Zimet GD, et al. Predictors of HPV vaccine uptake among women aged 19–26: importance of a physician’s recommendation. Vaccine. 2011;29(5):890–895. [DOI] [PubMed] [Google Scholar]
- 12. Gerend MA, Madkins K, Phillips G 2nd, et al. Predictors of human papillomavirus vaccination among young men who have sex with men. Sex Transm Dis. 2016;43(3):185–191. • This reference is specific to understanding vaccine uptake and barriers among MSM.
- 13.Rhodes SD, DiClemente RJ, Yee LJ, et al. Correlates of hepatitis B vaccination in a high-risk population: an Internet sample. Am J Med. 2001;110(8):628–632. [DOI] [PubMed] [Google Scholar]
- 14. Wheldon CW, Sutton SK, Fontenot HB, et al. Physician communication practices as a barrier to risk-based HPV vaccine uptake among men who have sex with men. J Cancer Educ. 2017;33(5):1126–1131. •This reference is specific to understanding vaccine uptake and barriers among MSM.
- 15.Bridges CB, Hurley LP, Williams WW, et al. Meeting the challenges of immunizing adults. Vaccine. 2015;33(4):D114–D120. [DOI] [PubMed] [Google Scholar]
- 16.Jones AM, Omer SB, Bednarczyk RA, et al. Parents’ source of vaccine information and impact on vaccine attitudes, beliefs, and nonmedical exemptions. Adv Prev Med. 2012;2012:932741. doi: 10.1155/2012/932741. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Gilbert P, Brewer NT, Reiter PL, et al. HPV vaccine acceptability in heterosexual, gay, and bisexual men. Am J Mens Health. 2011;5 (4):297–305. •This reference is specific to understanding vaccine uptake and barriers among MSM.
- 18.Gilkey MB, Reiter PL, Magnus BE, et al. Validation of the Vaccination Confidence Scale: a brief measure to identify parents at risk for refusing adolescent vaccines. Acad Pediatr. 2016;16(1):42–49. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Velan B, Boyko V, Lerner-Geva L, et al. Individualism, acceptance and differentiation as attitude traits in the public’s response to vaccination. Hum Vaccin Immunother. 2012;8(9):1272–1282. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Gilkey MB, McRee AL, Magnus BE, et al. Vaccination confidence and parental refusal/delay of early childhood vaccines. PloS one. 2016;11(7):e0159087. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Opel DJ, Mangione-Smith R, Taylor JA, et al. Development of a survey to identify vaccine-hesitant parents: the parent attitudes about childhood vaccines survey. Hum Vaccin. 2011;7(4):419–425. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Opel DJ, Taylor JA, Mangione-Smith R, et al. Validity and reliability of a survey to identify vaccine-hesitant parents. Vaccine. 2011;29(38):6598–6605. [DOI] [PubMed] [Google Scholar]
- 23.Gilkey MB, Magnus BE, Reiter PL, et al. The Vaccination Confidence Scale: a brief measure of parents’ vaccination beliefs. Vaccine. 2014;32(47):6259–6265. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Gust D, Brown C, Sheedy K, et al. Immunization attitudes and beliefs among parents: beyond a dichotomous perspective. Am J Health Behav. 2005;29(1):81–92. [DOI] [PubMed] [Google Scholar]
- 25.Opel DJ, Taylor JA, Zhou C, et al. The relationship between parent attitudes about childhood vaccines survey scores and future child immunization status: a validation study. JAMA Pediatr. 2013;167(11):1065–1071. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Oladejo O, Allen K, Amin A, et al. Comparative analysis of the Parent Attitudes about Childhood Vaccines (PACV) short scale and the five categories of vaccine acceptance identified by Gust et al. Vaccine. 2016;34(41):4964–4968. [DOI] [PubMed] [Google Scholar]
- 27.Black S Recognizing the importance of vaccine confidence. EBioMedicine. 2016;12:28–29. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Jegede AS. What led to the Nigerian boycott of the polio vaccination campaign? PLoS Med. 2007;4(3):e73. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Smith DP, Pals SL, Herbst JH, et al. development of a clinical screening index predictive of incident HIV infection among men who have sex with men in the United States. Epidemiol Prev. 2012;60(4):421–427. [DOI] [PubMed] [Google Scholar]
- 30. Frew PM, Chamberlain A, Hinman A, et al. Development of a Parental Vaccine Confidence Index for monitoring and assessment of childhood vaccine decision-making. San Diego (CA): ID week; 2017. •• This reference is the formative work for this project in accordance to NVAC recommendations.
- 31. Larson HJ, de Figueiredo A, Xiahong Z, et al. The state of vaccine confidence 2016: global insights through a 67-country survey. EBioMedicine. 2016;12:295–301. •• This reference is an expansive look at vaccine confidence globally, highlighting the gaps that need to be addressed in vaccine confidence work.
- 32. MacKellar DA, Valleroy LA, Secura GM, et al. Two decades after vaccine license: hepatitis B immunization and infection among young men who have sex with men. Am J Public Health. 2001;91(6):965–971. • This reference is specific to understanding vaccine uptake and barriers among MSM.
- 33.California Department of Public Health. Meningococcal disease outbreak information. 2017. [cited 2017 Mar 3]. Available from: http://www.cdph.ca.gov/programs/immunize/Pages/MeningococcalUpdates.aspx
- 34.Folaranmi TA, Kretz CB, Kamiya H, et al. Increased risk for meningococcal disease among men who have sex with men in the United States, 2012–2015. Clin Infect Dis. 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Madewell ZJ, Wester RB, Wang WW, et al. Voluntarily reported immunization registry data: reliability and feasibility to predict immunization rates, San Diego, California, 2013. Public Health Rep. 2017;132(3):357–365. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Lieu TA, Ray GT, Klein NP, et al. Geographic clusters in underimmunization and vaccine refusal. Pediatrics. 2015;135(2):280–289. [DOI] [PubMed] [Google Scholar]
- 37.Carrel M, Bitterman P. Personal belief exemptions to vaccination in California: a spatial analysis. Pediatrics. 2015;136(1):80–88. [DOI] [PubMed] [Google Scholar]
- 38.Wolf E, Rowhani-Rahbar A, Duchin J, et al. The challenges in measuring local immunization coverage: a statewide case study. Pediatrics. 2016;137(5):e20153755. [DOI] [PMC free article] [PubMed] [Google Scholar]