Sensitivity of Self-reported HPV Vaccination History among 18–26 year-old Men Who Have Sex with Men — Seattle, Washington, 2016–2018

Abstract

Background

We assessed sensitivity of self-reported HPV vaccination among young adult men who have sex with men (MSM) with documented HPV vaccination.

Methods

During 2016–2018, MSM and transgender women aged 18–26 years were enrolled in Seattle, Washington. HPV vaccination history was assessed via self-administered survey, clinic electronic medical records (EMR), and the Washington State Immunization Information System (WAIIS). We assessed self-report sensitivity among participants with documented prior HPV vaccination (≥1 dose) in either the EMR or WAIIS, and used logistic regression to compare sensitivity by age, number of doses, and time since first dose.

Results

Of 292 participants with ≥1 documented HPV vaccine dose, 243 self-reported ≥1 dose (sensitivity=83.2%,95%CI:78.4%−87.3%). Compared to participants whose first dose was <1 year ago, likelihood of self-report was lower among those with ≥3 years since first dose (adjusted odds ratio (aOR)=0.2,95%CI:0.1–0.5). Furthermore, compared to participants with only 1 documented HPV vaccine dose, likelihood of self-reporting ≥1 dose was higher among those with 2 (aOR=2.4,95%CI:1.0–5.5) or ≥3 doses (aOR=6.2,95%CI:2.7–14.4). Among 115 participants with ≥3 documented doses, sensitivity for recalling ≥3 doses was 69.6% (95%CI:60.3%−77.8%).

Conclusions

Most young adult MSM with a documented history of HPV vaccination self-reported prior HPV vaccination. Although recall was highest in those with ≥3 doses, 30% of this fully-vaccinated subgroup did not correctly recall the number of doses received, highlighting limitations of self-reporting. Furthermore, results indicating reduced recall with ≥3 years since first dose suggest that sensitivity of self-report among young adult MSM may decline over time as adolescent vaccination coverage increases.

Short Summary

In a study of 18-to-26-year-old MSM with a documented history of HPV vaccination, most participants correctly self-reported having received ≥1 dose, but 30% of fully-vaccinated participants misreported number of doses.

INTRODUCTION

HPV is the most common sexually transmitted infection in the United States,¹ and certain HPV types can cause genital warts, anogenital cancers, and oropharyngeal cancers.² HPV-related cancers in males include oropharyngeal, penile and anal cancers, with men who have sex with men (MSM) at higher risk for anal HPV and anal cancer than men who have sex with women.^3,4 Given that HPV-related cancers disproportionately affect MSM, young MSM are an important population for HPV prevention efforts.

HPV vaccines were first licensed in 2006 for females in the United States.⁵ In 2009, the Advisory Committee on Immunization Practices (ACIP) issued guidance that males aged 9 through 26 years could receive the HPV vaccination series, and updated its guidance in 2011 to recommend routine HPV vaccination of males at age 11 or 12 years and catch-up vaccination through age 21 years for all males or 26 years for MSM.⁵ The most recently updated ACIP guidelines in 2019 expanded recommendations to include catch-up vaccination for all persons through age 26 years.⁶ National Immunization Survey–Teen (NIS-Teen) data from 2019 showed that 51.8% of adolescent males aged 13–17 years had completed the HPV vaccination series (2 or 3 doses, depending on age at initiation) and 69.8% had received ≥1 dose.⁷ According to National HIV Behavioral Surveillance data, the proportion of young adult MSM aged 18–26 years self-reporting ≥1 HPV vaccine dose increased from 17.2% in 2014⁸ to 32.8% in 2017.⁹

Accurate capture of HPV vaccination history is important for clinical decision-making, monitoring of population-level vaccination coverage, and assessing population-level impacts of HPV vaccination to aid in HPV vaccination programmatic and research efforts. Of the various sources for capturing vaccination history (including self-report, electronic medical records [EMR], immunization information system [IIS], and paper records), EMR or IIS documentation are considered the gold standards. However, there are several limitations to using EMR and IIS records to verify HPV vaccine receipt. In the United States, EMR data are inconsistently shared between health systems, leaving patients to keep track of their own vaccination histories either through personal records (e.g., immunization record cards) or release of information agreements for medical records to be transferred between systems. State immunization registries are not always updated by the vaccine provider, especially for adult patients, or patients may receive vaccinations out of state, again placing the burden on the patient to maintain accurate vaccination records. Self-reported vaccination status is often used in the absence of EMR or IIS data to assess vaccination status. However, self-report of vaccination history is subject to recall bias and paper records are often misplaced or lost.

A few questionnaire-based cross-sectional studies have evaluated the accuracy of self-reported HPV vaccination history compared to EMR and state IIS data. All except one focused on female populations. One study reported the sensitivity of self-reported HPV vaccination compared to medical records to be 54% among adolescent females aged 14–17 years,¹⁰ while other studies found higher sensitivity of self-reported HPV vaccination, i.e., 85% among females aged 20–22 years¹¹, 89% among females 23–38 years¹², 91% among females aged 18–49¹³, 92% among females aged 14–26 years¹⁴, and 96% among females aged 18–39 years with high-grade cervical lesions reported to a statewide surveillance system.¹⁵ Among males aged 13–26 years, 80% accurately self-reported their HPV vaccination history (i.e., either having received or not received ≥1 dose).¹⁶ In addition, in a National Health and Nutrition Examination Survey (NHANES) pilot study comparing self- or parent-reported HPV vaccination in 14–29 year-olds against provider-verified records (not stratified by gender), sensitivity of self-/parent-report was 87%.¹⁷

To our knowledge, no study has reported on the accuracy of self-reported HPV vaccination status among MSM or transgender women. We evaluated the sensitivity of self-reported HPV vaccination among MSM with documented HPV vaccination in the Seattle metropolitan area, and further analyzed the effect of age, number of vaccine doses, and time since first vaccination dose on sensitivity of self-report.

MATERIALS AND METHODS

Study Design and Procedures

This study used Seattle-based data collected as part of the Centers for Disease Control and Prevention (CDC)-funded multi-center Vaccine Impact in Men (VIM) study.¹⁸ Gay, bisexual, other MSM, and transgender women (TGW) seeking sexually transmitted disease (STD) testing, treatment or counseling services from February 2016 to September 2018 were recruited and enrolled at one of two sites: Public Health – Seattle & King County Sexual Health Clinic, or Gay City Health Project. The research study protocol was reviewed and approved by the University of Washington and Washington State institutional review boards (IRBs); CDC authorized reliance on University of Washington as the reviewing IRB.

Patients were considered eligible if they 1) were assigned male sex at birth, regardless of current gender identity or expression, 2) were 18–26 years old, and 3) had ever had anal or oral sex with a male partner, or identified as gay/homosexual or bisexual, or intended to have sex with a male partner in the future. Eligible patients were excluded from participation in the study if 1) they required language translation for clinical services, or 2) they declined or were unable to provide written informed consent. Study procedures were completed during the clinic visit. Each participant received a $40 gift card as compensation.

Clinicians or STD testers asked participants to complete paper questionnaires which included questions on demographics, sexual behaviors, and HPV vaccination history. HPV vaccination history questions included ever having received HPV vaccination, age in years at time of first dose, number of doses received, and the date of first dose. Data were also collected from clinic EMRs for key demographic, health, sexual behavior, and HPV vaccination history, including vaccination dates and number of doses. Additional HPV vaccination records were obtained via participant signed release of information from the Washington State Immunization Information System (WAIIS), a statewide, lifetime immunization registry that tracks immunization records for people of all ages, including vaccination dates and number of doses.¹⁹

Data Analysis

Documented vaccination history was defined as a clinic EMR or WAIIS record of ≥1 dose of HPV vaccine prior to the study visit. Self-reported vaccination history was defined as reporting prior receipt of ≥1 HPV vaccine dose. We also captured both self-reported and EMR/WAIIS data on time since first dose (<1 year, 1–2 years, or ≥3 years) and number of doses (1, 2, or ≥3). Although 2016 ACIP guidelines reduced the recommended number of doses from 3 to 2 for individuals starting the vaccination series at age 9–14 years²⁰, none of the participants in our study would have been young enough at the time of the enrollment period to qualify for the 2-dose recommendation, and therefore only participants with ≥3 doses would be considered fully vaccinated.

Among those with a documented record of HPV vaccination, we calculated sensitivity, with 95% confidence intervals (CI), of ≥1 dose of self-reported HPV vaccination. We also conducted sensitivity analyses stratified by age group (18–21 years versus 22–26 years), time since first documented vaccine dose, and by number of documented doses received. We also calculated sensitivity for self-reporting ≥3 doses, restricting to participants with ≥3 documented HPV vaccine doses. We performed univariate and multivariate logistic regression to estimate odds ratios (OR) of associations between age group, vaccination timing, number of doses and likelihood of correct self-report of ≥1 HPV vaccine dose; the multivariate model included age group, vaccination timing, number of doses as independent variables and self-report of ≥1 HPV vaccine dose as the dependent variable. Variables with a p-value of <.05 were considered significant. We did not calculate specificity, positivity predictive value (PPV), or negative predictive value (NPV) of self-reported vaccination status due to known incompleteness of the EMR and WAIIS. For example, EMR data would not include all vaccinations received at other clinic locations, and state IIS data would not include vaccinations that were provided out of state, or by clinics that inconsistently report immunization data to the IIS.

All calculations were performed using STATA version 13 (StataCorp, LLC, College Station, TX).

RESULTS

Descriptive statistics

A total of 751 consenting participants from Seattle were enrolled into this study. Seven participants did not authorize WAIIS records access. Among 744 authorizing WAIIS access, 293 had received ≥1 documented HPV vaccine dose according to either the EMR or WAIIS. Of these 293, one participant did not complete a study questionnaire and was excluded from this analysis. Among the 292 included in this analysis (38.9% of total enrolled), almost one-third were aged 18–21 years (31.2%) with all others being aged 22–26 years (Table 1). Most reported their gender identity as male (96.6%) with others reporting female or transgender female (1.0%) or other (2.4%). The majority of participants identified their race as white (56.9%). Most of the participants identified their ethnicity as non-Hispanic (80.5%). Most participants (68.8%) identified as gay or homosexual. The mean/median lifetime number of sex partners was 49.5/25, and the mean/median age at first sexual encounter with any partner was 16.6/17 years. The mean/median age at first HPV vaccine dose was 20/20 years, the mean/median time since first dose was 2.2/2 years, and 39.3% had ≥3 recorded doses. Notably, 11.9% of those who self-reported vaccination did not report or did not recall how many doses they received.

Table 1:

Characteristics of participating men who have sex with men and transgender women with a documented^a record of ≥1 dose of HPV vaccine (n=292), Seattle, Washington, 2016–2018

Characteristic	n (%)
Recruitment site
Gay City Health Projecst	37 (12.7)
Public Health Seattle & King County STD Clinic	255 (87.3)
Age, years
18–21	91 (31.2)
22–26	201 (68.5)
Gender Identity
Male	282 (96.6)
Female/Transgender female	3 (1.0)
Other/unknown	7 (2.4)
Race
Black	21 (7.2)
White	166 (56.9)
Asian or Pacific Islander	61 (20.9)
American Indian or Alaska Native	4 (1.4)
More than one race	19 (6.5)
Don’t know/unknown	21 (7.2)
Ethnicity
Non-Hispanic	235 (80.5)
Hispanic	55 (18.8)
Don’t know/unknown	2 (0.7)
Sexual orientation
Gay or homosexual	201 (68.8)
Bisexual	29 (9.9)
Heterosexual or straight	3 (1.0)
Queer	8 (2.7)
Other/no answer	51 (17.5)
Lifetime number of partners
(mean/median, IQR)	49.5/25 (10, 54)
Age at first sexual encounter with any partner b
(mean/median, IQR)	16.6/17 (15, 18)
Self-reported number of HPV vaccine doses c
1 dose	40 (16.5)
2 doses	50 (20.6)
≥3 doses	124 (51.0)
Unknown/no answer	29 (11.9)
Documented number of HPV vaccine doses
1 dose	104 (35.6)
2 doses	73 (25.0)
≥3 doses	115 (39.4)
Unknown/no answer	N/A
Source of documented vaccination
WAIIS and EMR	165 (56.5)
WAIIS only	91 (31.2)
EMR only	36 (12.3)
Age at first HPV vaccine, years
(mean/median, IQR)	20.0/20 (18,23)
Time since first HPV vaccine, years
(mean/median, IQR)	2.2/2 (0,4)

^a≥1 HPV vaccine dose documented in either the clinic electronic medical record (EMR) or the Washington State Immunization Information System (WAIIS) or both

^b1 participant did not report age at first sex

^cAmong 243 participants self-reporting history of HPV vaccination

Sensitivity of self-report

Two-hundred and forty-three of the 292 included participants with ≥1 documented dose of HPV vaccine correctly self-reported ≥1 dose, corresponding to a sensitivity of 83.2% (95% CI: 78.4% to 87.3%) (Table 2). When restricting to 115 participants with ≥3 documented doses, 80 participants self-reported ≥3 doses (sensitivity=69.6%, 95%CI: 60.3% to 77.8%).

Table 2:

Sensitivity of self-report of HPV vaccine compared to documented vaccine record among participating men who have sex with men and transgender women — Seattle, Washington, 2016–2018

		Self-report responsea
	n with documented HPV vaccinationb	Yes	No	Don’t know/no answer	Sensitivity %	95% Confidence Interval
Overall	292	243	15	34	83.2	78.4–87.3
Age group
18–21 years	91	72	7	12	79.1	69.3–86.9
22–26 years	201	171	8	22	85.1	79.3–89.7
Time since first documented HPV vaccine dose
<1 year ago	87	77	3	7	88.5	79.9–94.3
1–2 years ago	94	82	2	10	87.2	78.8–93.2
≥3 years ago	112	84	10	18	75.0	65.9–82.7
Documented number of HPV vaccine doses
1	104	76	12	16	73.1	63.5–81.3
2	73	63	2	8	86.3	76.2–93.2
≥3	115	104	1	10	90.4	83.5–95.1
Documented ≥3 HPV vaccine dosesc	115	80d	8d	27d	69.6d	60.3–77.8d

^aAnswer to the survey question, “Have you ever received any vaccine against human papillomavirus (HPV)?” as reported by the participant

^b≥1 dose of HPV vaccine as documented in either the clinic electronic medical records (EMR) or the Washington State Immunization Information System (WAIIS) or both

^cRestricted to those with ≥3 documented vaccine record doses

^dSelf-report agreement and sensitivity for self-reporting ≥3 doses of HPV vaccine

The sensitivity of self-report was lower, but not statistically significantly so, in participants who were aged 18–21 years (79.1%, 95% CI: 69.3% to 86.9%) versus 22–26 years (85.1%, 95% CI: 79.3% to 89.7%) (Table 2). The corresponding adjusted OR (aOR) for correct self-report was 0.6 (95% CI: 0.3 to 1.3) for participants aged 18–21 versus 22–26 years (Table 3). The sensitivity of self-report was similar in participants with documented first vaccine dose <1 year ago (88.5%, 95% CI: 79.9% to 94.3%) and those who had their first dose 1–2 years ago (87.2%, 95% CI: 78.8% to 93.2%). Sensitivity of self-report was significantly lower in those who had their first dose ≥3 years ago (78.0%, 95% CI: 65.9% to 82.7%) (Table 2). The corresponding aORs for correct self-report were 0.6 (95% CI: 0.2 to 1.5) for participants with documented first vaccination 1–2 years ago, and 0.2 (95% CI: 0.1 to 0.5) for participants with documented first vaccination ≥3 years ago, versus those with first vaccination <1 year ago (Table 3). The sensitivity of self-report of ≥1 dose increased with number of documented doses received (1 dose: 73.1%, 95% CI: 63.5% to 81.3%; 2 doses: 86.3%, 95% CI: 76.2% to 93.2%; and ≥3 doses: 90.4%, 95% CI: 83.5% to 95.1%) (Table 2). The corresponding aORs for correct self-report were 2.4 (95% CI: 1.0 to 5.5) for 2 doses, and 6.2 (95% CI: 2.7 to 14.4) for ≥3 doses, versus those with 1 dose (Table 3).

Table 3:

Odds ratios for associations between age, time since first vaccination, and number of documented vaccine doses on the sensitivity of self-report for capturing documented HPV vaccination (n=292) among participating men who have sex with men and transgender women — Seattle, Washington, 2016–2018

	Univariate OR	95% confidence interval	P value	Multivariate ORa	95% confidence interval	P value
Age group
22–26 years	1.0	N/A	N/A	1.0	N/A	N/A
18–21 years	0.7	0.4–1.3	0.21	0.6	0.3–1.3	0.21
Time since first documented HPV vaccine b
<1 year ago	1.0	N/A	N/A	1.0	N/A	N/A
1–2 years ago	1.0	0.4–2.4	0.94	0.6	0.2–1.5	0.26
≥3 years ago	0.4	0.2–0.9	0.02	0.2	0.1–0.5	<0.01
Documented number of HPV vaccine doses b
1	1.0	N/A	N/A	1.0	N/A	N/A
2	2.3	1.0–5.1	0.04	2.4	1.0–5.5	0.04
≥3	3.5	1.8–7.4	<0.01	6.2	2.7–14.4	<0.01

^aAdjusted for age, time since first documented HPV vaccine, and/or number of documented HPV vaccine doses.

^bDocumented HPV vaccine in either the clinic electronic medical records (EMR) or the Washington State Immunization Information System (WAIIS) or both.

DISCUSSION

Very few studies report the accuracy of self-reported HPV vaccination status compared to documented medical records, especially among MSM or other male populations, or among TGW. One previously published study reported on the accuracy of self-reported HPV vaccination among 13–26 year-old males (79.5% for any versus no HPV vaccination); however, a direct comparison with our results could not be made as that study did not distinguish sensitivity or specificity among males with confirmed vaccination.¹⁶ Our study is the first to our knowledge to report sensitivity of self-reported HPV vaccination status among MSM with documented HPV vaccination.

In our study, the sensitivity of self-report for ≥1 dose of HPV vaccine in 18–26-year-old MSM (83%) was higher than previously reported in a small study of female adolescents (54%), but slightly lower than in studies of adult females (85–96%) and adolescent to adult females combined (92%).^10–15 Comparisons are complicated by differences in ages and timing of vaccination across study populations. Greater historical efforts in educating females versus males about the risks of HPV infection and HPV vaccines might contribute to observed gender-based differences in HPV vaccination recall. Several studies suggest there has been gender bias favoring females over males in provider-communicated HPV vaccination recommendations.^21–23

In our study, sensitivity increased with older age, but not significantly so, and with shorter time since HPV vaccination. Other comparable studies did not report sensitivity estimates stratified by age or time since vaccination, precluding comparisons with the stratified groups in our study. The one previously published study in males reported higher accuracy (not sensitivity) for older versus younger males when stratified by age (93%, 76%, and 51% in males aged 22–26, 19–21, and 14–18 years, respectively); however, older males were also less likely than younger males to have been vaccinated, and accuracy estimates included correct reporting of either having received or not having received HPV vaccination.¹⁶ A study of females reported that age was not associated with self-report accuracy (not sensitivity) in 23–38-year-old females.¹²

We observed that ability to correctly recall any prior HPV vaccination was higher in participants who received 2 or ≥3 doses compared to only 1 dose. However, the sensitivity of self-report for ≥3 vaccine doses (70%) was lower than for any number of vaccine doses (83%), suggesting that the ability to accurately recall number of doses is more difficult than recall of any dose. Similarly, in a study of 14–17 year-old adolescent females, sensitivity of self-report was lower for ≥3 doses (35%) than for any dose of HPV vaccine (54%),¹⁰ and in a study of adult women, 89% correctly self-reported having been vaccinated, whereas only 65% accurately recalled the number of doses.¹² In the study of males, participants’ accuracy of self-reporting the correct number of doses received (including zero doses) was decreased compared to accurately reporting whether or not they received any doses (73% versus 80%).¹⁶ Furthermore, in the NHANES pilot study¹⁷, sensitivity of self-/parent-reported HPV vaccination in 14–29 year-olds with provider-verified vaccination records was higher for ≥1 dose (87%) than for 3 doses (67%). For the multi-dose HPV vaccine, self-report is likely to produce more accurate estimates of series initiation than series completion.

A limitation of this analysis is that EMR and WAIIS data may not reflect the complete vaccination history of our study participants. EMR data was only collected from the participating clinics and not from other healthcare settings that participants may have visited previously. WAIIS data for our participants captured some HPV vaccination data from other clinics in Washington State; however, this relies on the clinics to report to the state IIS. It is also unclear how complete the WAIIS is for adults, and studies are lacking for this population. Participants may have also received HPV vaccination out-of-state, and unless they produced personal vaccination records or release of information authorizations for medical records to be transferred to the clinic EMR or state IIS, documented HPV vaccination history would not be available for our analysis. Given these known issues with data completeness, we were not able to calculate specificity, PPV or NPV of self-report, or evaluate recall of exactly 1 or 2 vaccine doses.

Accurate information on HPV vaccination status is important for clinical decision-making and for vaccination surveillance. Our results indicate that while most HPV-vaccinated young adult MSM can recall prior HPV vaccination, fewer can recall the number of doses they received. In addition, sensitivity of self-report in young adult MSM populations might decrease in the future as more males are vaccinated in their adolescent years in accordance with national HPV vaccination recommendations. Our results suggest that self-reporting may be sufficiently sensitive for measuring vaccine uptake and coverage in research studies and surveillance when EMR or IIS data are not readily available, but less useful in the clinical setting where healthcare providers rely on accurate information on whether or not a person is fully vaccinated to guide clinical decision-making about the need for additional doses. Results highlight the value of the move toward greater data sharing across EMR platforms, as well as the continued need to make vaccine registries more complete and more accessible for clinical purposes. In addition, further studies are warranted to develop protocols for probing patient vaccination history that are succinct, easily clinically deployed, and require minimal qualifications and training.