Skip to main content
Papillomavirus Research logoLink to Papillomavirus Research
. 2017 Nov 8;4:99–107. doi: 10.1016/j.pvr.2017.11.001

Psychosocial correlates of HPV vaccine acceptability in college males: A cross-sectional exploratory study

Ovidiu Tatar a,, Samara Perez a,b, Anila Naz a, Gilla K Shapiro a,b, Zeev Rosberger a,b,c
PMCID: PMC5883195  PMID: 29179877

Abstract

Background

Most college males are not immunized against the human papillomavirus (HPV) and are at high risk of HPV infection. Most research of correlates of HPV vaccine acceptability in college males has assessed vaccine acceptability as a binary outcome, e.g., vaccinated or not vaccinated, without considering that some students may not even be aware that the HPV vaccine can be given to males. Our objective was to evaluate the psychosocial correlates of HPV acceptability in college males, based on multiple stages of HPV decision-making.

Methods

We used an online questionnaire to collect data from college men aged 18–26 enrolled at three Canadian universities between September 2013 and April 2014. Vaccine acceptability assessment was informed by the six-stage decision-making Precaution Adoption Process Model (PAPM). We sought information on socio-demographics, health behaviors, HPV vaccine benefits and barriers, worry, susceptibility, severity related to HPV infection and social norms. HPV and HPV vaccine knowledge was measured with validated scales. Psychosocial correlates of HPV vaccine acceptability were assessed with bivariate and multivariate multinomial logistic regression. Actual and perceived HPV and HPV vaccine knowledge scores were calculated.

Results

The final sample size was 428. Most male college students were unaware that the HPV vaccine could be given to males, unengaged or undecided about getting the HPV vaccine. Significant correlates of higher HPV vaccine acceptability were: increased HPV knowledge, having discussed the HPV vaccine with a healthcare provider, and social norms. Being in an exclusive sexual relationship was significantly associated with lower HPV vaccine acceptability. Students' actual HPV and HPV vaccine knowledge was low and positively correlated to their perception about their HPV knowledge.

Conclusions

We provided a fine-tuned analysis of psychosocial correlates of HPV vaccine acceptability in college males who are in the early stages of HPV vaccine decision-making. Interventions are needed to increase HPV and HPV vaccine knowledge in college males.

Keywords: Human papillomavirus, Vaccine, Acceptability, Psychosocial correlates, College men

Highlights

  • Most college males were unaware that the HPV vaccine can be given to males.

  • Significant correlates of HPV vaccine acceptability are provided.

  • Perceived and actual HPV knowledge are positively correlated.

  • The Precaution Adoption Process Model is appropriate for assessing HPV vaccine acceptability.

1. Introduction

Human papillomavirus (HPV) infections are considered the most common viral sexually transmitted infections (STI) [1], [2]. Epidemiological research shows that the highest incidence of HPV infection occurs among young adults 18–28 years old [3]. HPV types 16 and 18 cause approximately 70% of all cervical cancers [1]; and are also associated with oropharyngeal (25–35%) and ano-genital cancers e.g., anus (80–90%), vulva and vagina (40%) and penile cancers (40–50%) [4]. The HPV-associated oropharyngeal cancer incidence rate was recently reported as 4.5 times higher in Canadian males than females [4], [5]. In Canada, the National Advisory Committee on Immunization recommends HPV immunization for females and males 9–26 [6]. Presently, two HPV vaccines, Gardasil® and Gardasil 9® which offer protection against 4 and 9 HPV strains respectively, are being used in publicly-funded HPV vaccine programs in Canada.

Publicly-funded Canadian school-based HPV vaccine programs for boys (in grades 4–7 depending on the province) have recently begun i.e., in 2013 Prince Edward Island, in 2014 Alberta, in 2015 Nova Scotia and in 2016 Ontario, Quebec and Manitoba. The other four provinces (British Columbia, New Brunswick, Saskatchewan and Newfoundland) start in fall 2017 while the territories (Northwest Territories, Nunavut, and Yukon) have not adopted yet universal vaccination programs for boys. Across Canada, the college male population are “too old” to benefit from publicly funded HPV vaccination programs (excluding men who have sex with men in BC and Quebec). This explains why only a mere 6–8% of 18–26 year old Canadian males have reported receiving at least one dose of the HPV vaccine [7], [8]. Insufficient HPV vaccine coverage in college males (5.5–26%) has also been reported in United States [9], [10], [11], [12]. With the staggered introduction of school-based HPV vaccine programs for boys between provinces beginning in 2013, the vast majority of males who will enroll in college in the next 2–6 years (depending on the province of residence) will not have received the HPV vaccine. Some college males may only receive one dose of the HPV vaccine, as they will be reliant on their own to obtain the second dose, whereas the school-based programs remind students when the subsequent doses are scheduled. Increased sexual activity and risk taking behavior [13] among college male students leaves this population at risk for both transmission of the virus and HPV-associated diseases, highlighting the need for this group to be vaccinated against HPV. Yet, most college males are not aware of the high prevalence of HPV infection [3], are less concerned about getting infected with HPV than females [14] and more than three-quarters consider themselves at low risk for a STI [3], [9], [15]. This population is thus an important target, who can benefit from HPV vaccination (especially when many Canadian university health insurance plans offer some insurance coverage for immunizations).

Most studies related to HPV vaccine acceptability in young adult males analyzed data collected in the pre-licensure period [16] and may not accurately reflect current acceptability rates as well as the correlates of HPV vaccine acceptability given the new landscape where HPV vaccination is recommended and targeted for males. From the few pre-licensure studies, psychosocial correlates of HPV acceptability (e.g., perceived susceptibility, severity, risk, social norms, health behaviors, demographics) were shown to increase vaccine acceptability in college males [9], [10], [15], [16], [17], [18], [19], [20], [21]. Lack of HPV knowledge and vaccine awareness among college males is also considered an important barrier to HPV vaccination [22]. The literature describing which correlates are associated with HPV vaccine acceptability in college males in the post-licensure period is limited and to the best of our knowledge, there is no study examining the correlates of HPV vaccine acceptability among college males in Canada.

Recent quantitative and qualitative descriptive studies emphasized that in college males, HPV and HPV vaccine knowledge is low [3], [8], [9], [10], [18], [19], [23], [24]. Therefore, we hypothesize that a high proportion of college males are unaware that the HPV vaccine is recommended for males. Moreover, most studies of psychosocial correlates of HPV vaccine acceptability have assessed acceptability as a binary outcome: vaccinated/not vaccinated or willing/not willing to receive the HPV vaccine [9], [10], [15], [19]. These studies have not taken into consideration individuals that are unaware that the HPV vaccine can be given to males or are in other early stages (unengaged, undecided) as outlined by the Precaution Adoption Process Model (PAPM), a theoretical framework which views health decision-making as a series of categorical stages as opposed to binary (yes/no) [25]. PAPM uses awareness, intention, and past health behaviors to define six distinct stages of acceptability through which people may pass as they proceed towards the process of adopting a health behavior: Stage 1, unaware of the health behavior; Stage 2, aware of the health behavior but unengaged; Stage 3, engaged and thinking about performing the behavior but undecided; Stage 4, decided not to act; Stage 5, decided to act but not yet in action; Stage 6, acting. Thus, existing evidence related to correlates of HPV vaccine acceptability for college males is limited to individuals who reached an advanced health decision stage (i.e., decided to vaccinate, decided not to vaccinate or vaccinated) and cannot inform researchers/public health decision makers what correlates are important for college males who are in other stages of HPV vaccination decision making process (i.e., unaware, unengaged or undecided). Our study objectives were to describe the HPV and HPV vaccine knowledge and assess the correlates of HPV vaccine acceptability by PAPM stage in Canadian college males as a step in understanding how and with whom to intervene.

2. Methods

2.1. Questionnaire development

The questionnaire was developed by our research team after extensively reviewing the literature for items used in the field to measure psychosocial determinants of HPV vaccine decision making, including but not limited to knowledge, attitudes and behaviors [26], [27], [28], [29]. The questionnaire was then reviewed with a ‘think aloud’ pilot testing of the survey with 16 male undergraduates who provided feedback about how the questions were understood; they were compensated 25$. The research team then synthesized the participants’ comments, consulted with colleagues with experience in the field, and revised and improved the questionnaire.

2.2. Measures

The questionnaire measured basic socio-demographics and health related behaviors, (e.g., “Have you seen a doctor/health care provider in the past year?”). Actual HPV knowledge was measured with 16 items, e.g., “HPV always has visible signs or symptoms” and 7 items to measure actual HPV vaccine knowledge, e.g., “The HPV vaccines offer protection against all sexually transmitted infections” using Waller's et al. validated HPV knowledge scales [26]. Participants chose “true”, “false” or “I don’t know”. A total HPV (range 0–16) and HPV vaccine knowledge score (range 0–7) was generated based on correct answers and “I don’t know” were coded as incorrect. Differences exist between what people know about the HPV (i.e., actual knowledge) and how much they feel they know (i.e., perceived knowledge) [30]. Perceived HPV knowledge was measured with 1 item, e.g., “Before today, how much would you say you knew about HPV?” and perceived HPV vaccine knowledge was measured with 1 item, e.g., “Before today, how much would you say you knew about HPV vaccine?” with response options 1 = “nothing at all”, “a little”, “a moderate amount”, and 4 = “a lot’.

Four perceived benefits were measured, e.g., “I think that the HPV vaccine is effective in helping to prevent diseases caused by HPV”. Barriers were measured with 6 items, e.g., “I am concerned that the HPV vaccine costs more than I can pay”. Perceived severity (threat) was assessed with 2 items, e.g., “If I got HPV, it would be a serious threat to my health”. Perceived benefits, barriers and severity were answered on 5-point Likert scale where 1 = “strongly disagree” and 5 = “strongly agree”. Perceived susceptibility was assessed with 1 item, “Without the vaccine, what do you think is the chance that you will get a disease caused by HPV?” where 1 = “no chance”, 2 = “low chance”, 3 = “moderate chance”, 4 = “high chance”. Perceived worry was assessed with 1 item, “Before today, how much did you worry about getting a disease caused by HPV?”, where 1 = “not at all”, 2 = “a little”, 3 = “moderate amount”, 4 = “quite a lot”. Social norms were measured with 3 items, e.g., “My parents think I should get the HPV vaccine” answered on 5-point Likert scale where 2 = “strongly disagree” and 6 = “strongly agree” and 1 = “We have never discussed the HPV vaccine/ don’t know my parent's opinion”.

The HPV vaccination adoption stage was informed by Weinstein et al.'s PAPM theoretical framework [25] which is a stage-based model of health behavior change with roots in social learning approaches to health behavior and other health belief models [31]. Using the PAPM as our theoretical framework, college males chose one of following six stages: Stage 1 unaware that the HPV vaccine can be given to males, Stage 2 aware that the HPV vaccine can be given to males but unengaged, Stage 3 undecided about getting the HPV vaccine, Stage 4 decided NOT to get the HPV vaccine, Stage 5 decided to get the HPV vaccine and Stage 6 vaccinated. Thus, rather than measuring vaccine acceptability (dependent variable) as vaccinated versus not vaccinated, we used the PAPM to capture 6 stages of vaccine acceptability in a more nuanced way.

2.3. Study design and participants

We used an online survey methodology to collect data from male undergraduates aged 18–26, enrolled at three Canadian universities: McGill University (Quebec), University of Ottawa (Ontario), and University of Toronto (Ontario) between September 2013 and April 2014. Students enrolled in introduction to psychology courses who subscribed to the psychology participant pool were invited to participate. We also recruited students through online classifieds, flyers and in class announcements.

Once participants signed up for the study, they received a link to the survey hosted on a secure web platform (available at www.fluidsurveys.com). Participants provided informed consent and then completed the questionnaire. Participants recruited through psychology participant pools received one course credit. All other participants were entered in a draw to receive 1 of 3 cash prizes of $100 each. The study was approved by the Research Ethics Board-II at McGill University, the Health Sciences and Science Research Ethics Board at the University of Ottawa and by the Social Sciences, Humanities and Education Research Ethics Board at the University of Toronto.

2.4. Data cleaning and analysis

We applied data cleaning methods to remove participants who had more than 50% of missing data i.e., incomplete questionnaires. We used psychometric antonyms and synonyms, as well as variance to remove unmotivated or careless responders [32], [33]. Participants who reported that they had never heard of both HPV and HPV vaccine, as well as PAPM HPV vaccination adoption stages with small cell size (< 30) were excluded from subsequent analyses.

The percentage of correct responses at item level for actual HPV and HPV vaccine knowledge was calculated. Pearson's product-moment correlations was calculated to assess the relationship between actual and perceived HPV and HPV vaccine knowledge.

Frequencies and percentages were calculated for categorical variables, and mean and standard deviation was calculated for continuous variables for included PAPM stages.

Exploratory factor analysis (EFA, Principal Component Analysis with varimax rotation) was used to investigate the dimensionality of benefits, barriers, severity, susceptibility, worry and social norms items. We conducted parallel analysis for principal components and retained only statistically significant factors (i.e., for which the raw data Eigenvalues were higher than the 95% random data Eigenvalues) [34]. We calculated Cronbach's alpha for the items in each retained factor and eliminated items that would decrease internal consistency. For factors items measured on a Likert scale, we calculated a mean score. For actual HPV and actual HPV vaccine knowledge scales we calculated Cronbach's alpha and mean scores. Responses to the social norms questions were dichotomized into discussed/not discussed about getting the HPV vaccine with either parents, friends and current partners respectively.

We used multinomial logistic regression to model the log odds of the PAPM stages (dependent variable) as a linear combination of the independent variables. The PAPM stage unaware was selected as the reference category as it represents the initial stage of HPV decision-making. For nominal independent variables, we report the odds ratio (OR) of being in a particular PAPM stage (versus unaware) for each category versus the reference category (e.g., casually dating versus single). For continuous independent variables (e.g., actual HPV knowledge), we report the change (OR) represented by a one-unit score increase.

Bivariate analyses were first conducted for all independent variables to explore their individual relationship with PAPM stage. Independent variables significantly associated in the bivariate analyses with the PAPM stage (i.e., p ≤ 0.05) were entered in the initial multivariate model. Subsequently, independent variables significantly associated (i.e., p ≤ 0.05) with the PAPM stage were retained to produce the final multivariate model. To confirm the validity of our final multivariate model over the initial multivariate model, we used the log-likelihood test. We used the Hausman-McFadden test [35] to evaluate the final model for independence of irrelevant alternatives (IIA) which postulates that a person's choice (i.e., PAPM stage) is unchanged by other available choices (i.e., fewer PAPM stages). Thus, the final model containing all PAPM stages was compared with a restricted model containing fewer PAPM stages.

IBM SPSS v23 and R v3.3.2 was used to analyze data.

3. Results

We recruited 754 participants. We removed 64 participants as they had less than 50% of their questionnaires completed and 177 participants because they were identified as careless responders. Ten participants were removed as they reported that they never heard about both the HPV and the HPV vaccine. There were 19 participants who had decided not to vaccinate, 18 who had decided to vaccinate and 38 who reported they had received the HPV vaccine. Among those vaccinated, 10 individuals declared either they were vaccinated before 2010, (the year when the HPV vaccine became available for boys in Canada) or did not report the year of vaccination and were removed. Participants who had decided not to vaccinate, decided to vaccinate and those who identified themselves as vaccinated (i.e., in total 75 participants) were excluded from further analyses, as they were in PAPM stages with group sizes of less than 30. The final sample consisted of 428 participants classified according to PAPM stages as follows: unaware that the HPV vaccine can be given to males (n = 195), aware that the HPV vaccine can be given to males but who had not thought about getting vaccinated (i.e., unengaged, n = 172) and thought about getting the HPV vaccine but who were undecided about getting it (n = 61).

Descriptive results of continuous and categorical variables are provided in Table 1.

Table 1.

Descriptive statistics of categorical and continuous variables.

Correlate Unaware (n = 195)
Unengaged (n = 172)
Undecided (n = 61)
Whole sample (n = 428)
n % n % n % n (%)
Born in Canada 138 71 115 67 49 80 302 (71)
Not born in Canada 57 29 57 33 12 20 126 (29)


 

 

 

 

 

 

 


English 117 60 102 59 48 78 267 (62)
French 27 14 24 14 4 7 55 (13)
Other 51 26 46 27 9 15 106 (25)


 

 

 

 

 

 

 


University of Ottawa 80 41 72 42 25 41 177 (41)
University of Toronto 54 28 55 32 11 18 120 (28)
McGill University 60 31 45 26 25 41 130 (31)


 

 

 

 

 

 

 


White 101 52 92 53 38 62 231 (54)
East Asian 47 24 27 16 12 20 86 (20)
South Asian 20 10 24 14 6 10 50 (12)
Other 27 14 29 17 5 8 61 (14)


 

 

 

 

 

 

 


Living Alone 13 7 8 5 4 7 25 (6)
Living with Peers 75 38 76 44 30 49 181 (42)
Living with Family/Relatives 107 55 88 51 27 44 222 (52)


 

 

 

 

 

 

 


< 60,000$ 39 20 44 25 13 21 96 (22)
60,000–99,000$ 45 23 37 22 15 25 97 (23)
> 100,000 $ 75 39 57 33 23 38 155 (36)
Don't know 36 18 34 20 10 16 80 (19)


 

 

 

 

 

 

 


Received all childhood immunizations 186 95 162 94 57 93 405 (95)
Did not receive all childhood immunizations 9 5 10 6 4 7 23 (5)


 

 

 

 

 

 

 


Visited an HCP in the past year 159 82 133 77 55 90 347 (81)
No visit to an HCP in the past year 36 18 39 23 6 10 81 (19)


 

 

 

 

 

 

 


Single 105 54 105 61 38 62 248 (58)
Casually dating 27 14 28 16 9 15 64 (15)
Exclusive relationship 63 32 39 23 14 23 116 (27)


 

 

 

 

 

 

 


Discussed with an HCP about getting HPV vaccine 9 5 27 16 19 31 55 (13)
Did not discuss with an HCP about getting HPV vaccine 186 95 145 84 42 69 373 (87)


 

 

 

 

 

 

 


“My friends/parents/current partner think I should get the HPV vaccine” 31 16 43 25 27 44 101 (24)
Not discussed with friends/parents/current partner 164 84 129 75 34 56 327 (76)
Mean(SD) Mean(SD) Mean(SD) Mean(SD)
Age 19.49 (1.76) 19.41 (1.44) 19.52 (1.41) 19.46 (1.59)
Actual HPV general knowledge 8.36 (3.66) 9.8 (3.05) 10.7 (3.09) 9.27 (3.46)
Actual HPV vaccine knowledge 2.94 (1.82) 3.33 (1.80) 4.11 (1.46) 3.26 (1.80)
Perceived HPV knowledge 2.01 (0.68) 2.17 (0.65) 2.23 (0.62) 2.10 (0.66)
Perceived HPV vaccine knowledge 1.72 (0.67) 1.83 (0.71) 2.05 (0.67) 1.81 (0.69)
Benefits 3.94 (0.68) 4.03 (0.62) 4.15 (0.60) 4.00 (0.65)
Barriers 2.98 (0.98) 2.90 (0.96) 2.90 (0.90) 2.93 (0.96)
Threat 4.15 (0.87) 4.12 (0.86) 4.10 (0.87) 4.13 (0.86)

Note: SD = standard deviation; age range: 18–26; Actual HPV general knowledge range: 0–16; Actual HPV vaccine knowledge range: 0–7; Perceived HPV knowledge range: 1–4; Perceived vaccine HPV knowledge range: 1–4; Benefits range: 1–5; Barriers range: 1–5.

Threat range: 1–5.

Results of item-level analyses for actual HPV and HPV vaccine knowledge scales show that 21.5% of college men know that most sexually active people will get HPV at some point in their lives, 29.9% know that the HPV vaccine offers protection against genital warts and 32.7% are knowledgeable about the number of doses required to achieve adequate protection (See Figs. 1 and 2). Perceived HPV knowledge was positively and moderately correlated with actual HPV knowledge (Pearson's r = 0.42, CI: 0.34–0.50, p < 0.001). Similarly, perceived HPV vaccine knowledge was positively correlated with actual HPV vaccine knowledge (Pearson's r = 0.39, CI: 0.31–0.47, p < 0.001).

Fig. 1.

Fig. 1

Correct (%) of HPV knowledge by item.

Fig. 2.

Fig. 2

Correct (%) of HPV vaccine knowledge by item.

The EFA yielded four significant factors: benefits (3 items), social norms (3 items), barriers (2 items) and threat (2 items) for a cumulative total variance explained of 54.23%. Results of EFA, parallel analysis and internal reliability for benefits, social norms, barriers and threat are presented in Table 2. The Cronbach's alpha was 0.79 and 0.65 for actual HPV knowledge (16 items) and actual HPV vaccination knowledge (7 items) respectively.

Table 2.

Results of dimensionality (EFA) and reliability analyses.

EFA item loadings EFA raw data Eigenvalues EFA 95% random data Eigenvalues Cronbach's α
Benefits 3.11 1.43 0.86
I think that the HPV vaccine is effective in helping to prevent diseases caused by HPV 0.87
I think that the HPV vaccine may be effective in helping to prevent genital warts 0.84
I think that the HPV vaccine may be effective in helping to prevent HPV-related cancers 0.87
Social norms 2.58 1.34 0.86
My friends think I should get the HPV vaccine 0.87
My parents think I should get the HPV vaccine 0.88
My current partner thinks I should get the HPV vaccine 0.87
Barriers 2.19 1.27 0.57
I am concerned that the HPV vaccine costs more than I can pay 0.77
The fact that there is more than 1 dose required for the HPV vaccine would keep me from getting vaccinated 0.79
Threat 1.34 1.22 0.76
If I got HPV, it would be a serious threat to my health 0.87
If I got HPV, it would be a serious threat to my sex life 0.88

Note: EFA = Exploratory Factor Analysis; HPV = Human Papillomavirus.

In bivariate analyses, language, relationship status, having a discussion with the doctor about the HPV vaccine, both actual and perceived HPV and HPV vaccine knowledge, benefits of HPV vaccination and social norms (i.e., discussing with parents, friends or current partner about getting the HPV vaccine) were significantly associated with the PAPM stage i.e., being unengaged or undecided compared to unaware (Table 3).

Table 3.

Results of bivariate analyses.

Correlate Unengaged Undecided
OR (95% CI) OR (95% CI)
n = 172 n = 61
Born in Canada(Reference Not born in Canada) 1.20 (0.77; 1.87) 0.59 (0.29; 1.20)
Age(One unit increase) 0.97 (0.85; 1.10) 1.01 (0.85; 1.21)
French(Reference English) 1.02 (0.55; 1.88) 0.36 (0.12; 1.09)
Other(Reference English) 1.03 (0.64; 1.67) 0.43 (0.20; 0.94)


 

 


University of Ottawa(Reference McGill University) 1.20 (0.73; 1.98) 0.75 (0.39; 1.43)
University of Toronto(Reference McGill University) 1.36 (0.79; 2.33) 0.49 (0.22; 1.09)


 

 


East Asian(Reference White) 0.63 (0.36; 1.09) 0.68 (0.33; 1.42)
South Asian(Reference White) 1.32 (0.68; 2.54) 0.80 (0.30; 2.14)
Other(Reference White) 1.18 (0.65; 2.14) 0.49 (0.18; 1.37)


 

 


Living situation Peers(Reference Living Alone) 1.65 (0.64; 4.20) 1.30 (0.39; 4.31)
Living situation Family/Relatives(Reference Living Alone) 1.34 (0.53; 3.37) 0.82 (0.25; 2.72)


 

 


Family income $60,000–99,000 CAD(Reference < $60,000 CAD) 0.73 (0.40; 1.34) 1.00 (0.42; 2.36)
Family income > $100,000 CAD(Reference < $60,000 CAD) 0.67 (0.39; 1.17) 0.92 (0.42; 2.01)


 

 


Received all childhood immunizations(Reference Not received) 0.78 (0.31; 1.98) 0.69 (0.20; 2.32)
At least one visit to a HCP in the last year(Reference No visit) 1.30 (0.78; 2.15) 0.48 (0.19; 1.21)
Casually dating(Reference Single) 1.04 (0.57; 1.88) 0.92 (0.40; 2.14)
Exclusive relationship(Reference Single) 0.62 (0.38; 1.00)* 0.61 (0.31; 1.22)


 

 


Discussed with HCP about HPV vaccine(Reference No discussion with HCP) 3.85 (1.76; 8.44) 9.35 (3.95; 22.11)
Actual HPV knowledge(One unit increase) 1.13 (1.06; 1.21) 1.25 (1.13; 1.39)
Actual HPV vaccine knowledge(One unit increase) 1.31 (1.01; 1.27) 1.78 (1.24; 1.78)
Perceived HPV knowledge(One unit increase) 1.47 (1.07; 2.02) 1.68 (1.09; 2.60)
Perceived HPV vaccine knowledge(One unit increase) 1.28 (0.94; 1.74) 1.96 (1.30; 2.95)
Benefits(One unit increase) 1.26 (0.92; 1.73) 1.71 (1.06; 2.74)
Barriers(One unit increase) 0.91 (0.74; 1.13) 0.92 (0.68; 1.24)
Threat (One unit increase) 0.95 (0.75; 1.21) 0.93 (0.67; 1.29)
Social norms (discussed about HPV vaccine) (Reference Not discussed) 1.76 (1.05; 2.95) 4.20 (2.23; 7.92)

Note. Unaware (N = 195) was used as reference category. OR = Odds Ratio; CI = Confidence Interval; In bold are significant OR and CI at p ≤ 0.05; * p = 0.051.

We present the most parsimonious model (i.e., final model, Table 4), as the likelihood ratio test showed no difference in fit between the initial (9 correlates) and final (5 correlates) multivariate models (df = 8, χ2 = 7.75, p = 0.46). The final multinomial model exploring three PAPM stages was appropriate (compared to two PAPM stages) to study vaccine acceptability (Hausman-McFadden test df = 8, χ2 = − 57.8, p = 1). Being in an exclusive relationship (versus single) was associated with lower odds of being unengaged (OR = 0.56), having discussed with a HCP about the HPV vaccine (versus not discussed) was associated with higher odds of being unengaged (OR = 3.41) or undecided (OR = 7.45), higher actual HPV knowledge (i.e., for each unit increase) was associated with increased odds of being unengaged (OR = 1.13) or undecided (OR = 1.23) and having discussed with family, friends or current partner about the HPV vaccine (i.e., social norms) was associated with higher odds of being undecided (OR = 3.10) (Table 4).

Table 4.

Results of multivariate multinomial logistic regression.

Correlate Unaware (Reference) Unengaged AOR (95% CI) Undecided AOR (95% CI)
French(Reference English) 1.04 (0.55; 1.98) 0.31 (0.09; 1.02)
Other(Reference English) 1.16 (0.70; 1.92) 0.48 (0.21; 1.12)
Casually dating(Reference Single) 0.96 (0.52; 1.77) 0.82 (0.33; 2.03)
Exclusive relationship(Reference Single) 0.56 (0.34; 0.92) 0.47 (0.22; 1.02)
Discussed with HCP about HPV vaccine(Reference Not discussed) 3.41 (1.52; 7.62) 7.45 (2.99; 18.53)
Actual HPV knowledge(One unit increase) 1.13 (1.06; 1.21) 1.23 (1.11; 1.37)
Social norms (discussed about HPV vaccine) (Reference Not discussed) 1.48 (0.86; 2.54) 3.10 (1.55; 6.19)

Note: AOR = adjusted odds ratio; CI = confidence interval.

4. Discussion

Our results highlight the correlates of HPV vaccine acceptability in college males who were in initial stages of HPV vaccine decision making stages: unaware that the HPV vaccine can be given to males, aware that the HPV vaccine can be given to males but unengaged and undecided about getting the HPV vaccine.

Based on multinomial multivariate regression, we identified significant correlates of the early stages of HPV vaccine acceptability. Social norms and relationship status were significant correlates for being undecided or unengaged respectively. Thus, having discussed the HPV vaccine with family, friends or current partner increased the odds of being undecided (versus unaware) while college males who were in an exclusive relationship had lower odds of being unengaged (versus unaware). Other correlates were not specific for a certain stage of HPV vaccine acceptability. Thus, high levels of (actual) HPV knowledge and having discussed the HPV vaccine with a healthcare provider, increased the odds of being unengaged and undecided. These results are consistent with results from our previous studies of psychosocial correlates of HPV vaccine decision-making in parents of boys [36] where we show that there are differences in psychosocial correlates varying with HPV vaccine adoption stage and that vaccine acceptability can be better explained by a multi-stage process than with a binary outcome.

The positive role of doctor's recommendation on HPV vaccine acceptability has been substantiated by Ortashi et al. [18], Thomas et al. [15] and Barnard et al. [37]. Our results show that simply having had a discussion about the HPV vaccine with a HCP was associated with higher vaccine acceptability among college males i.e., higher odds of being both in the unengaged or undecided stages as compared to unaware. In our sample, 43% (n = 24) of students who initiated a discussion with a HCP received a positive recommendation. Therefore, encouraging college males to initiate a discussion with their HCP about the HPV vaccine could increase HPV vaccination rates in this population. In line with previous research conducted in college male populations, we found that social norms (e.g., recommendation of parents/family, peers to receive HPV vaccine, social pressure to get vaccinated against HPV) [10], [15], [16], [37] were associated with the stages closer (i.e., undecided) to HPV vaccine acceptability.

In recent studies, the association between college males’ (actual) HPV knowledge and vaccine acceptability was controversial; Oz et al. found that increased HPV knowledge was associated with higher acceptability [19] while Ratanasiripong et al. does not endorse this result [10]. Equivocal evidence exists related to the relationship status (i.e., single, exclusive relationship) and HPV acceptability as Liddon et al.'s review of HPV vaccine acceptability in males found that one of the most common reasons for men to refuse the HPV vaccine was being in a monogamous relationship [16] while Oz et al. did not find a relationship between relationship status and vaccine acceptability [19]. We believe that an association exists between the perception of college males who are in an exclusive relationship of having low risk for an STI and their perception that the HPV vaccine is not needed, which materializes in low vaccine acceptability. We recommend educational interventions to reinforce the idea that HPV vaccination is appropriate independent of relationship status as a stable relationship does not offer absolute guarantee of not getting infected with HPV.

In this study, some similar to others [30] of the analyzed psychosocial correlates were significantly associated with HPV vaccine acceptability in bivariate analysis but lost significance in the multivariate regression model, which may suggest a weaker association with HPV vaccine acceptability. Accordingly, increased HPV vaccine knowledge and perceiving the benefits of the HPV vaccine (i.e., in preventing HPV related cancers), were associated with higher vaccine acceptability. These results, albeit a weaker association with HPV vaccine acceptability, are congruent with findings of other studies. Thus, in college male populations, having more (actual) HPV vaccine knowledge [19], perceiving more HPV vaccine benefits (e.g., partner protection against cervical cancer) [15], [20], were important factors of moving closer to HPV vaccine acceptability.

In terms of socio-demographic correlates, our study found that age, ethnicity, living situation and family income were not related to HPV vaccine acceptability. Sparse evidence exists for significant associations between socio-demographic correlates (e.g., ethnicity) and vaccine acceptability [9], [15], [19], except for age. Fontenot et al. determined that for every year older (age range 18–25) the odds of having obtained HPV vaccine decreased 24% [9] whereas in Ratanasiripong et al.'s study, the odds of being vaccinated were higher for younger college males (age range 18–26) [10]. Further studies should evaluate the association between age and HPV vaccine acceptability. If confirmed, higher vaccine acceptability in younger college males could be considered a facilitator of HPV prevention in line with the goal to vaccinate males before exposure and/or transmission of HPV infection occurs (i.e., younger age) and HPV vaccination awareness campaigns could be implemented among first year university students. i.e., college freshmen.

HPV and HPV vaccine knowledge for the whole sample were quite low, which is consistent with other studies [3], [8], [9], [10], [18], [19], [24], [38]. Similar to previous research of HPV knowledge in college males, less than 60% of respondents correctly knew that “there are many types of HPV” [19], “HPV can cause genital warts” [3], [10], “HPV cannot be cured by antibiotics” [19], “Having sex at an early age increases the risk of getting HPV” [19], “Most sexually active people will get HPV at some point in their lives” [19], [37] and that “HPV usually doesn't need any treatment” [8], [19]. With respect to HPV vaccine knowledge, in line with results from other studies, less than 40% of men knew that HPV vaccination requires more than one dose [19], “[one of] the HPV vaccine offers protection against genital warts” [8], [19] and that “HPV vaccines are most effective if given to people who have never had sex” [19]. These are important knowledge gaps that should be targeted among college age students.

In this study, we did not capture HPV knowledge items that are of interest for males, i.e., causal relationship between HPV infection and oral, anal and penile cancers. Our research team has recently validated two scales for measuring HPV and HPV vaccine knowledge in parents of boys that can be easily adapted for college males [39]. We recommend these scales to be used in future research as studies have revealed that less than 60% of college males know the association between HPV and oral [8], [18], [37], anal [8], [10], [37] and penile [3], [8], [18], [19], [37] cancer.

As previously found by Krawczyk et al., in college males, low HPV and HPV vaccine (actual) knowledge was associated with low perceived knowledge [30]. In our opinion, assessing perceived HPV knowledge represents a simple and quick way (i.e., only two questions) of identifying college males that are in immediate need for HPV educational interventions.

Our study is not without limitations. We used a convenience sample of psychology pool students from three Canadian universities and they may not reflect the opinion of college students enrolled in other programs/at other Canadian universities, which may limit generalizability. Most participants were in the initial HPV decision-making stages (i.e., unaware, unengaged and undecided). Therefore, we were unable to include in our final multinomial regression model students in more advanced HPV vaccination decisional stages: decided to vaccinate, decided not to vaccinate and already vaccinated due to small cell sizes. In sensitivity analyses (amid small cell size for decided to vaccinate, decided not to vaccinate and already vaccinated) (Appendix A), discussion with an HCP about the HPV vaccine increased the odds of being “decided to vaccinate” (OR = 7.19) and “already vaccinated” (OR = 62.21). Increased social norms (i.e. discussing with parents, friends and current partner about the HPV vaccine) was associated with higher odds of being either “decided not to vaccinate” (OR = 9.44) or “decided to vaccinate” (OR = 6.64). Our results need further confirmation in future research by conducting analyses on a larger sample of college males who have reached a decision about the HPV vaccination.

As suggested by Pitts and colleagues, it would be appropriate for future research to include correlates of HPV vaccine acceptability items related to concepts of: “too late to vaccinate” because of college males’ age and sexual behaviors, as well as their dismissive perceptions of invulnerability (e.g., “it's not gonna happen to me”) [24] which aligns with our results of low worry and low perceived susceptibility. Additionally, we recommend researchers to use validated scales (where available) to increase the reliability of results.

5. Conclusions

This study offers new evidence on HPV and HPV vaccine knowledge and significant correlates of vaccine acceptability among Canadian college males. Most college males were unaware that the HPV vaccine can be given to males, unengaged or undecided about getting the HPV vaccine. HPV and HPV vaccine knowledge was low and concordant with one's perception about HPV knowledge level. HPV knowledge, discussion with a healthcare provider about the HPV vaccine, being in an exclusive relationship, and social norms were significant factors related to PAPM stages. Future research is needed to confirm if perceived benefits, barriers, severity (threat), susceptibility and worry are significantly associated with HPV vaccine acceptability in college males who are in initial HPV vaccination decision-making stages.

Acknowledgements

We would like to thank to Dr. Sophie Lebel and Georden Jones (Ottawa) and Dr. Marc Fournier (Toronto) for their invaluable assistance with data collection and recruitment.

Acknowledgments

Research funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Conflict of interest

ZR reports personal fees from Merck outside submitted work at a consultation meeting (November, 2015); and a speaker to family physicians (April, 2015).

Contributor Information

Ovidiu Tatar, Email: ovidiu.tatar@mail.mcgill.ca.

Samara Perez, Email: samara.perez@mail.mcgill.ca.

Anila Naz, Email: anila.naz@mail.mcgill.ca.

Gilla K. Shapiro, Email: gilla.shapiro@mail.mcgill.ca.

Zeev Rosberger, Email: zeev.rosberger@mcgill.ca.

Appendix A. Sensitivity analysis, multinomial logistic regression model that includes decided not to vaccinate (PAPM stage 4), decided to vaccinate (PAPM stage 5) and already vaccinated (PAPM stage 6)

Correlate Unengaged Undecided Decided NOT to vaccinate Decided to vaccinate Already vaccinated
AOR (95% CI) AOR (95% CI) AOR (95% CI) AOR (95% CI) AOR (95% CI)
N = 172 N = 61 N = 19 N = 18 N = 28
French(Reference English) 1.02 (0.54; 1.95) 0.30 (0.09; 0.97) 0.57 (0.11; 2.88) 1.70 (0.47; 6.05) *
Other(Reference English) 1.18 (0.71; 1.96) 0.50 (0.22; 1.15) 1.01 (0.32; 3.19) 1.20 (0.34; 4.29) 0.69 (0.18; 2.57)


 

 

 

 

 


Casually dating(Reference Single) 0.96 (0.52; 1.77) 0.82 (0.33; 2.00) 0.37 (0.04; 3.11) 0.45 (0.05; 3.84) 1.33 (0.34; 5.27)
Exclusive relationship(Reference Single) 0.55 (0.33; 0.91) 0.51 (0.24; 1.09) 1.14 (0.41; 3.19) 1.10 (0.38; 3.20) 0.35 (0.10; 1.27)


 

 

 

 

 


Discussed with HCP about HPV vaccine(Reference Not discussed) 3.45 (1.54; 7.71) 7.51 (3.03;18.61) 3.17 (0.82; 12.26) 7.19 (2.11; 24.56) 62.21(15.9; 242.6)
Actual HPV knowledge(One unit increase) 1.13 (1.06; 1.20) 1.23 (1.10; 1.36) 1.06 (0.91; 1.23) 1.18 (0.99; 1.40) 1.19 (0.99; 11.42)
Social norms (discussed about HPV vaccine) (Reference Not discussed) 1.44 (0.84; 2.48) 2.92 (1.47; 5.83) 9.44 (3.25; 27.40) 6.64 (2.21; 19.95) **

Note: The reference category was “unaware”, n = 195; * There were no observations for “French”; ** There were no observations in the category “Not discussed”. AOR = Adjusted Odds Ratio; CI = Confidence Interval; In bold significant AOR (p ≤ 0.05).

References

  • 1.Public Health Agency of Canada, An Advisory Committee Statement (ACS) National Advisory Committee on Immunization (NACI). Update on the recommended Human Papillomavirus (HPV) vaccine immunization schedule [Internet], 2015. Available from: 〈http://healthycanadians.gc.ca/publications/healthy-living-vie-saine/papillomavirus-immunization-schedule-2015-papillome-immunisation-calendrier/index-eng.php〉. (Accessed 14 July 2017).
  • 2.Weinstock H., Berman S., Cates W. Sexually transmitted diseases among American youth: incidence and prevalence estimates, 2000. Perspect. Sex. Reprod. Health. 2004;36(1):6–10. doi: 10.1363/psrh.36.6.04. (ISSN # 1538-6341) (accession # 12419696) [DOI] [PubMed] [Google Scholar]
  • 3.Baer H., Allen S., Braun L. Knowledge of human papillomavirus infection among young adult men and women: implications for health education and research. J. Community Health. 2000;25(1):67–78. doi: 10.1023/a:1005192902137. (ISSN# 00945145) (PMCID# 10706210) [DOI] [PubMed] [Google Scholar]
  • 4.Canadian Cancer Society, HPV not just a threat to women: mouth and throat cancers rising sharply in men [Internet], 2016. Available from: 〈http://www.cancer.ca/en/about-us/for-media/media-releases/national/2016/canadian-cancer-statistics-2016/?Region=on〉, (Accessed 14 July 2017).
  • 5.CBC News – Health, Male HPV-related oral cancer rates on the rise: Canadian cancer society [Internet], 2016. Available from: 〈http://www.cbc.ca/news/health/cancer-stats-1.3810804〉. (Accessed 14 July 2017).
  • 6.Public Health Agency of Canada, Update on Human Papillomavirus (HPV) vaccines [Internet], 2012. Available from: 〈http://www.phac-aspc.gc.ca/publicat/ccdr-rmtc/12vol38/acs-dcc-1/index-eng.php〉. (Accessed 14 July 2017).
  • 7.Government of Canada, Vaccine uptake in Canadian adults: results from the 2014 adult National Immunization Coverage Survey (aNICS) [Internet]. 2016. Available from: 〈http://www.healthycanadians.gc.ca/publications/healthy-living-vie-saine/vaccine-coverage-adults-results-2014-resultats-couverture-vaccinale-adultes/index-eng.php〉. (Accessed 14 July 2017).
  • 8.Little K., Ogilvie G., Mirwaldt P. Human papillomavirus awareness, knowledge, and vaccination status in a diverse population of male pfostsecondary students in greater Vancouver. BC Med. J. 2015;57(2):64–69. (ISSN # 0007-0556) (accession # 101143406) [Google Scholar]
  • 9.Fontenot H.B., Fantasia H.C., Charyk A., Sutherland M.A. Human papillomavirus (HPV) risk factors, vaccination patterns, and vaccine perceptions among a sample of male college students. J. Am. Coll. Health. 2014;62(3):186–192. doi: 10.1080/07448481.2013.872649. [DOI] [PubMed] [Google Scholar]
  • 10.Ratanasiripong N.T. Factors related to human papillomavirus (HPV) vaccination in college men. Public Health Nurs. 2015;32(6):645–653. doi: 10.1111/phn.12198. [DOI] [PubMed] [Google Scholar]
  • 11.Pierre-Victor D., Mukherjee S., Bahelah R., Madhivanan P. Human papillomavirus vaccine uptake among males 11–26 years in United States: findings from the National Health and Nutrition Examination Survey, 2011–2012. Vaccine. 2014;32(49):6655–6658. doi: 10.1016/j.vaccine.2014.09.044. [DOI] [PubMed] [Google Scholar]
  • 12.Fuller K.M., Hinyard L. Factors associated with HPV vaccination in young males. J. Community Health. 2017 doi: 10.1007/s10900-017-0361-4. [DOI] [PubMed] [Google Scholar]
  • 13.Arnett J.J. Emerging adulthood: a theory of development from the late teens through the twenties. Am. Psychol. 2000;55(5):469–480. [PubMed] [Google Scholar]
  • 14.Chelimo C., Wouldes T.A., Cameron L.D. Human papillomavirus (HPV) vaccine acceptance and perceived effectiveness, and HPV infection concern among young New Zealand university students. Sex Health. 2010;7(3):394–396. doi: 10.1071/SH10005. [DOI] [PubMed] [Google Scholar]
  • 15.Thomas T.L., Stephens D.P., Johnson-Mallard V., Higgins M. Young hispanic men and human papillomavirus vaccination choices. J. Transcult. Nurs. 2016;27(2):103–108. doi: 10.1177/1043659614526759. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Liddon N., Hood J., Wynn B.A., Markowitz L.E. Acceptability of human papillomavirus vaccine for males: a review of the literature. J. Adolesc. Health. 2010;46(2):113–123. doi: 10.1016/j.jadohealth.2009.11.199. [DOI] [PubMed] [Google Scholar]
  • 17.Gerend M.A., Barley J. Human papillomavirus vaccine acceptability among young adult men. Sex Transm. Dis. 2009;36(1):58–62. doi: 10.1097/OLQ.0b013e31818606fc. [DOI] [PubMed] [Google Scholar]
  • 18.Ortashi O., Raheel H., Khamis J. Acceptability of human papillomavirus vaccination among male university students in the United Arab Emirates. Vaccine. 2013;31(44):5141–5144. doi: 10.1016/j.vaccine.2013.08.016. [DOI] [PubMed] [Google Scholar]
  • 19.Oz M., Cetinkaya N., Apaydin A., Korkmaz E., Bas S., Ozgu E. Awareness and knowledge levels of Turkish college students about human papilloma virus infection and vaccine acceptance. J. Cancer Educ. 2016 doi: 10.1007/s13187-016-1116-0. [DOI] [PubMed] [Google Scholar]
  • 20.Richards K. College men and women and their intent to receive genital human papillomavirus vaccine. SAGE Open. 2016;6(1) (215824401662970) [Google Scholar]
  • 21.Mehta P., Sharma M., Lee R.C. Designing and evaluating a health belief model-based intervention to increase intent of HPV vaccination among college males. Int. Q. Community Health Educ. 2013;34(1):101–117. doi: 10.2190/IQ.34.1.h. [DOI] [PubMed] [Google Scholar]
  • 22.Hunter T., Weinstein M. Beliefs and knowledge about the human papillomavirus vaccine among undergraduate men. Health Educ. J. 2016;75(2):249–256. [Google Scholar]
  • 23.Mehta P., Sharma M., Lee R.C. Using the health belief model in qualitative focus groups to identify HPV vaccine acceptability in college men. Int. Q. Community Health Educ. 2012;33(2):175–187. doi: 10.2190/IQ.33.2.f. [DOI] [PubMed] [Google Scholar]
  • 24.Pitts M.J., Stanley S.J., Kim S. College males' enduring and novel health beliefs about the HPV vaccine. Health Commun. 2016:1–9. doi: 10.1080/10410236.2016.1196421. [DOI] [PubMed] [Google Scholar]
  • 25.Weinstein N.D., Sandman P.M., SJ B. Jossey-Bass; San Francisco, CA: 2008. The Precaution Adoption Process Model. [Google Scholar]
  • 26.Waller J., Ostini R., Marlow L.A., McCaffery K., Zimet G. Validation of a measure of knowledge about human papillomavirus (HPV) using item response theory and classical test theory. Prev. Med. 2013;56(1):35–40. doi: 10.1016/j.ypmed.2012.10.028. [DOI] [PubMed] [Google Scholar]
  • 27.McRee A.L., Brewer N.T., Reiter P.L., Gottlieb S.L., Smith J.S. The Carolina HPV immunization attitudes and beliefs scale (CHIAS): scale development and associations with intentions to vaccinate. Sex Transm. Dis. 2010;37(4):234–239. doi: 10.1097/OLQ.0b013e3181c37e15. [DOI] [PubMed] [Google Scholar]
  • 28.Krawczyk A., Knauper B., Gilca V., Dube E., Perez S., Joyal-Desmarais K. Parents' decision-making about the human papillomavirus vaccine for their daughters: I. Quantitative results. Hum. Vaccin. Immunother. 2015;11(2):322–329. doi: 10.1080/21645515.2014.1004030. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Krawczyk A.L., Perez S., Lau E., Holcroft C.A., Amsel R., Knauper B. Human papillomavirus vaccination intentions and uptake in college women. Health Psychol. 2012;31(5):685–693. doi: 10.1037/a0027012. [DOI] [PubMed] [Google Scholar]
  • 30.Krawczyk A., Stephenson E., Perez S., Lau E., Rosberger Z. Deconstructing human papillomavirus (HPV) knowledge: objective and perceived knowledge in males' intentions to receive the HPV vaccine. Am. J. Health Educ. 2013;44(1):26–31. [Google Scholar]
  • 31.Fishbein M., Ajzen I. Addison-Wesley; Reading, MA: 1975. Belief, Attitude, Intention, and Behavior: An Introduction to Theory and Research. [Google Scholar]
  • 32.Curran P.G. Methods for the detection of carelessly invalid responses in survey data. J. Exp. Soc. Psychol. 2016;66:4–19. [Google Scholar]
  • 33.Meade A.W., Craig S.B. Identifying careless responses in survey data. Psychol. Methods. 2012;17(3):437–455. doi: 10.1037/a0028085. [DOI] [PubMed] [Google Scholar]
  • 34.O’connor B.P. SPSS and SAS programs for determining the number of components using parallel analysis and velicer's MAP test. Behav. Res. Methods Instrum. Comput. 2000;32(3):396–402. doi: 10.3758/bf03200807. [DOI] [PubMed] [Google Scholar]
  • 35.Hausman J., McFadden D. Specification tests for the multinomial logit model. Econometrica. 1984;52(5):4–19. [Google Scholar]
  • 36.Perez S., Tatar O., Gilca V., Shapiro G.K., Ogilvie G., Guichon J. Untangling the psychosocial predictors of HPV vaccination decision-making among parents of boys. Vaccine. 2017;35(36):4713–4721. doi: 10.1016/j.vaccine.2017.07.043. [DOI] [PubMed] [Google Scholar]
  • 37.Barnard M., George P., Perryman M.L., Wolff L.A. Human papillomavirus (HPV) vaccine knowledge, attitudes, and uptake in college students: implications from the precaution adoption process model. PLoS One. 2017;12(8):e0182266. doi: 10.1371/journal.pone.0182266. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Kamimura A., Trinh H.N., Weaver S., Chernenko A., Wright L., Stoddard M. Knowledge and beliefs about HPV among college students in Vietnam and the United States. J. Infect. Public Health. 2017 doi: 10.1016/j.jiph.2017.06.006. [DOI] [PubMed] [Google Scholar]
  • 39.Perez S., Tatar O., Ostini R., Shapiro G.K., Waller J., Zimet G. Extending and validating a human papillomavirus (HPV) knowledge measure in a national sample of Canadian parents of boys. Prev. Med. 2016;91:43–49. doi: 10.1016/j.ypmed.2016.07.017. [DOI] [PubMed] [Google Scholar]

Articles from Papillomavirus Research are provided here courtesy of Elsevier

RESOURCES