A Broad Safety Assessment of the 9-Valent Human Papillomavirus Vaccine

Abstract

Parents indicate that safety is their top concern about human papillomavirus (HPV) vaccination. A data-mining method not requiring prespecification of health outcome(s) or postexposure period(s) of potentially increased risk can be used to identify possible associations between an exposure and any of thousands of medically attended health outcomes; this method was applied to data on the 9-valent HPV vaccine (HPV9) to detect potential safety problems. Data on 9- to 26-year-olds who had received HPV9 vaccine between November 4, 2016, and August 5, 2018, inclusive, were extracted from the MarketScan database and analyzed for statistically significant clustering of incident diagnoses within the hierarchy of diagnoses coded using the International Classification of Diseases and temporally within the 1 year after vaccination, using the self-controlled tree-temporal scan statistic and TreeScan software. Only 56 days of postvaccination enrollment was required; subsequent follow-up was censored at disenrollment. Multiple testing was adjusted for. The analysis included 493,089 doses of HPV9. Almost all signals resulted from temporal confounding, not unexpected with a 1-year follow-up period. The only plausible signals were for nonspecific adverse events (e.g., injection-site reactions, headache) on days 1–2 after vaccination, with attributable risks as low as 1 per 100,000 vaccinees. Considering the broad scope of the evaluation and the high statistical power, the findings of no specific serious adverse events should provide reassurance about this vaccine’s safety.

Abbreviations

CRPS

complex regional pain syndrome

HPV

human papillomavirus

HPV4

quadrivalent human papillomavirus vaccine

HPV9

9-valent human papillomavirus vaccine

ICD-10-CM

International Classification of Diseases Tenth Revision, Clinical Modification

The uptake of human papillomavirus (HPV) vaccine has lagged far behind that of other vaccines recommended for adolescents and young adults, despite the availability of HPV vaccines since 2006. Only 51.1% of US adolescents aged 13–17 years were current with the HPV vaccine series in 2018, compared with 86.6% and 88.9% who had received the recommended ≥1 dose of quadrivalent meningococcal vaccine and ≥1 dose of tetanus-diphtheria–acellular pertussis vaccine, respectively (1). In recent surveys of parents regarding HPV vaccination, safety has been cited as the top concern (2–4). A number of well-designed studies have investigated possible associations between HPV vaccination and certain health outcomes and not found any associations (5–10). However, such studies might be too focused on specific outcomes to significantly allay concerns about HPV vaccine safety.

A recently developed data-mining method known as the self-controlled tree-temporal scan statistic can evaluate whether any of thousands of health outcomes is associated with receipt of a specific vaccine or drug (11). This method, which builds on earlier work with tree-based scan statistics (12–14), differs from traditional safety study methods in that it does not require prespecifying a specific health outcome of interest or a specific postexposure period of potentially increased risk. Instead, for an exposed population, data on all diagnoses recorded within a defined postexposure follow-up period are scanned to detect any statistically unusual clustering of cases within a large hierarchy, or “tree,” of diagnoses as well as within the follow-up period. The method adjusts for the multiple testing entailed in evaluating the thousands of “branches” (e.g., nontraumatic joint disorders) and time intervals (e.g., days 19–27 after exposure) considered. Further, the method is self-controlled, eliminating confounding by fixed patient characteristics such as chronic disease status.

The method identified known vaccine-associated adverse events and produced few false signals when applied to 2 vaccines recommended for adolescents and young adults: a quadrivalent meningococcal conjugate vaccine (Menactra; Sanofi Pasteur Inc., Lyon, France) (15) and the quadrivalent HPV vaccine (HPV4) (Gardasil; Merck & Co., Inc., Whitehouse Station, New Jersey) (16). In this study, we applied it to the currently recommended 9-valent HPV vaccine (HPV9) (Gardasil 9; Merck & Co., Inc., Whitehouse Station, New Jersey). HPV9 was approved by the Food and Drug Administration for use in female persons aged 9–26 years and male persons aged 9–15 years on December 10, 2014. The Food and Drug Administration extended the indication to include male persons aged 16–26 years on December 14, 2015. The approved age range was further extended to include both female and male persons 27–45 years of age on October 5, 2018.

METHODS

Study population, enrollment criteria, and exposure

We used the IBM MarketScan Research Databases (MarketScan; IBM Watson Health, Cambridge, Massachusetts), among the largest proprietary US claims databases available for health-care research, and thus likely highly representative of the commercially insured population. The databases capture person-specific clinical utilization, expenditures, and enrollment across inpatient, outpatient, prescription drug, and carve-out services. The databases link paid claims and encounter data to detailed patient information across sites and types of providers collected from approximately 350 payers (mainly large employers and health plans, predominantly fee-for-service data).

We extracted data on female and male covered persons 9–26 years of age who were vaccinated during November 4, 2016, through August 5, 2018. To be included, an individual had to have been enrolled from 400 days prior through 56 days after HPV9 vaccination. (The prevaccination enrollment requirement is explained in “Incident diagnoses” below.) HPV9 was identified using Current Procedural Terminology code 90651 and National Drug Codes 00006411903, 00006412102, 00006411901, 00006411902, and 00006412101. HPV9 doses preceded within 365 days by a dose of HPV9 or HPV4 were excluded, effectively limiting the study to first doses, although subsequent doses could have been received during follow-up.

Hierarchical diagnosis tree

Outcomes were identified using codes from the International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM). ICD-10-CM codes have a hierarchical tree-like structure, starting with 21 broad categories of diagnoses (e.g., diseases of the circulatory system), which progressively branch into more and more specific sets of diagnoses, culminating in a highly specific diagnosis code. The ICD-10-CM tree we used has 6 levels. Table 1 presents an example of the hierarchical classification scheme; this example diagnosis does not use the sixth level.

Table 1.

Example of Hierarchical Organization in the International Classification of Diseases, Tenth Revision, Coding System

Level	Code Range or Code	Description
1	M00–M99	Diseases of the musculoskeletal system and connective tissue
2	M06	Other rheumatoid arthritis
3	M06.0	Rheumatoid arthritis without rheumatoid factor
4	M06.01	Rheumatoid arthritis without rheumatoid factor, shoulder
5	M06.011	Rheumatoid arthritis without rheumatoid factor, right shoulder

Unlike in previous studies using this method with a tree based on ICD Ninth Revision codes, we did not “prune” the tree of any codes, and the tree contained all 72,184 billable ICD-10 codes plus thousands more nonbillable (higher-order) ICD-10 codes. Thus, codes representing conditions and outcomes very unlikely to be caused by vaccination (e.g., sickle cell disease) or to manifest within a few weeks of vaccination (e.g., cancers) were retained in the tree, and we were aware that false alarms involving outcomes unlikely to be associated with vaccination could emerge.

Incident diagnoses

The study examined “incident” diagnoses observed in the inpatient or emergency department setting during the follow-up period of at least 56 days up to a maximum of 365 days. To be counted as an incident case, the patient must not have been assigned another ICD-10 diagnosis code having the same first 3 characters (i.e., in the same second level of tree) in any setting during the prior 400 days. (We chose 400 days in order to enable ascertainment of preexisting conditions that might have been recorded at a visit roughly 1 year prior, considering that some patients have annual preventive care visits.) We did not look for clustering (signals) in the broadest (first) or finest (sixth) levels of the tree. Because incidence was determined using the second level of the tree, above which no analysis of clustering was carried out, no patient could have contributed more than 1 case count to any cluster.

Risk and comparison windows

We set the analysis parameters to evaluate temporal risk windows that were between 2 and 90 days long, started between 1 and 364 days after vaccination, and ended between 2 and 365 days after vaccination. To avoid reducing power by analyzing implausible risk intervals (for example, days 330–331 postvaccination), we imposed the additional criterion that the ratio of the length of each window being evaluated to the postvaccination end day of the respective window had to be at least 20%. The comparison period used to evaluate each eligible potential risk window consisted of the days within the follow-up period that were not in the risk window (Figure 1).

Figure 1.

Examples of potential risk windows evaluated at any given instant of analysis, with their control period(s), assuming that a full 365 days of follow-up exists for the patient. A) A potential risk window that starts on day 1 after vaccination. The corresponding control period starts the day after the end of the potential risk window and extends through day 365. B) A potential risk window situated at neither end of the follow-up period but rather somewhere in between. The corresponding control period consists of the segments of the 365-day follow-up period that are not in the potential risk window being evaluated. C) A potential risk window ending on the last day of follow-up. The corresponding control period starts on day 1 after vaccination and extends through the day before the start of the potential risk window.

The tree-temporal scan statistic

With the tree-temporal scan statistic (11, 16), one performs multiple temporal scan statistics, one for each of the many clinical outcomes and groups of related clinical outcomes (i.e., leaves and branches of the tree). For each leaf and branch, one evaluates multiple potential risk windows, comparing the number of events within the risk window with what would be expected by chance if they were randomly and uniformly distributed over time. Under the null hypothesis, there is no unusual temporal clustering of events on any leaf or branch. Under the alternative hypothesis, there is at least 1 leaf or branch of the tree for which there is a temporal cluster of events during some time interval.

In using the tree-temporal tree-based scan statistic with a self-controlled design, the comparison is within-person among time periods. (The rate of any event in unvaccinated people is not measured and is not used for comparison or to standardize any other rates.) The question being asked is whether there is an elevated occurrence of cases of a particular kind of adverse event during a particular time period after exposure as compared with the rest of the period observed. Rather than prespecifying the time period of interest for a potential elevation in risk, we allow the data to tell us whether any such period exists. The formula for excess cases is as follows (11): (Actual Cases Observed in the Risk Window) – [(Length of the Risk Window) × (Number of Cases Observed Outside the Risk Window / Length of Time Outside the Risk Window)]

The second term (in square brackets) represents the number of cases that would occur in the risk window being evaluated if the cases in the risk window were occurring at the same rate as in the comparison period. The excess is what is observed beyond this value.

In this analysis, we included data for patients even if they did not have a full 365 days of postvaccination follow-up time, censoring their data at the time of disenrollment. Under the null hypothesis, the observations for any censored individual would be uniformly distributed between the time of vaccination and the time of censoring.

Multiple testing was adjusted for by means of Monte Carlo simulation. The number of Monte Carlo replications selected for this analysis, 9,999, meant the lowest possible P value was 0.0001.

Calculation of attributable risk

Without censoring, we calculated attributable risks for signals we deemed true indications of vaccine adverse events as excess cases as described above divided by the total number of doses.

Signal follow-up

We investigated 1 signal by generating and examining a list of all medical claims between 4 weeks prior to the index HPV9 vaccination date and 12 weeks after that date for each patient contributing to the signal. This “claims profile” included medical settings, types of encounters, diagnosis codes, procedure codes, and dispensings of drugs whose days’ supply overlapped this 16-week period. This approach permits investigators to assess the reasons for the visit that contributed to the signal and the recent medical history of each patient without entailing more resource-intensive medical record review.

Sensitivity analysis

In addition to the main analysis focusing on dose 1, which used the censoring method with up to 1 year of follow-up, we conducted a sensitivity analysis including all doses (if not preceded by a prior dose within 42 days), which used instead a conditional approach with a fixed 56 days of follow-up, similar to what was done previously for HPV4 (16). This analysis looked for temporal clusters 2–28 days in length that started during days 1–28 and ended during days 2–42 after vaccination.

Institutional review board approval

The study was approved by the Harvard Pilgrim Health Care Institutional Review Board.

RESULTS

A total of 493,089 doses of HPV9 vaccine were included in the main analysis. Among these, 154,652 events occurred. Of these event episodes, 59,339 (38%) were censored prior to 365 days postvaccination. The median time at censoring (calculated for censored episodes only) was 236 days.

There were 4 sets of delivery-related signals (e.g., “Encounter for full-term uncomplicated delivery”), with risk windows ranging between day 232 and day 362, inclusive. (We use “set” to refer to a group of signals with ICD-10 codes sharing the same first 3 characters.)

Other than these delivery-related signals, there were 6 sets of statistically significant signals (P ≤ 0.05). These are shown in Table 2, where they are arranged in decreasing order of the largest test statistic within the set. Rows 1 and 5–6 represent general categories of adverse event deemed related to immunization. The risk windows identified were days 1–2. The attributable risks of these signals ranged from 1.0–1.6 excess cases per 100,000 vaccinees. In contrast, rows 2–3 and rows 7–8 represent influenza outcomes, with risk windows starting on day 121 or day 156 and ending between days 210 and 245, inclusive. Finally, there was a signal for “Other injury of unspecified body region” during days 1–71 (row 4) and another for “Other specified postprocedural states” during days 7–65 (row 9).

Table 2.

Statistically Significant Signals From Tree-Temporal Scan Statistical Analysis of Events During 56–365 Days of Follow-up After 493,089 9-Valent Human Papillomavirus Vaccinations in 9- to 26-Year-Old Subjects, United States, November 4, 2016, through August 5, 2018

Row No.	Node	Text Description	No. of Node Cases	Risk Window		No. of Cases in Risk Window	Excess Cases per 100, 000 Doses	P Value
				Start Day	End Day
Set 1
1	T88.1	Other complications following immunization, not elsewhere classified	14	1	2	8	1.6	0.0001
Set 2
2	J11	Influenza due to unidentified influenza virus	461	121	210	182		0.0003
3	J11.1	Influenza due to unidentified influenza virus with other respiratory manifestations	448	121	210	178		0.0002
Set 3
4	T14.8	Other injury of unspecified body region	101	1	71	52		0.0010
Set 4
5	T50.Z95	Adverse effect of other vaccines and biological substances	8	1	2	5	1.0	0.0034
6	T50.A	Poisoning by, adverse effect of and underdosing of bacterial vaccines	10	1	2	5	1.0	0.0046
Set 5
7	J10	Influenza due to other identified influenza virus	405	156	245	151		0.0462
8	J10.1	Influenza due to other identified influenza virus with other respiratory manifestations	363	156	242	138		0.0073
Set 6
9	Z98.89	Other specified postprocedural states	79	7	65	35		0.0469

Investigation of the claims profiles of the 52 patients contributing to the “Other injury of unspecified body region” cluster from days 1 to 71 (row 4) found a variety of injuries (e.g., abrasions, contusions, lacerations, burns, puncture wounds, sprains, fractures) to various areas of the body. In approximately half (25) of the cases, there were codes for types of accidents or events recorded on the same day. These included falls (5), motor vehicle accidents (5), cycling accidents (3), overexertion of various sorts (3), dog bites (3), fights and assaults (2), and miscellaneous other accidents, all different (4).

In the sensitivity analysis of all doses, with just 56 days of follow-up, there were 1,278,548 doses and 3 sets of signals. One set was within ICD-10 code T88, “Other complications of surgical and medical care, not elsewhere classified,” and included T88.1, corresponding to row 1 of the main analysis results. One set was within T50, “Poisoning by, adverse effect of and underdosing of diuretics and other and unspecified drugs, medicaments and biological substances,” corresponding to rows 5–6 of the main analysis results. The third one was for R50.83, “Postvaccination fever.” All these signals had risk windows of days 1–2.

DISCUSSION

With this tree-temporal scan statistical method, we evaluated the tens of thousands of potential adverse events in the ICD-10-CM coding system and thousands of potential intervals of increased risk within 1 year of HPV9 vaccination. In scanning for clustering of cases within a comprehensive hierarchy of diagnoses and time, without requiring prespecification of adverse events or postvaccination time periods of concern, the method allows for a broader safety assessment than is possible with most traditional vaccine safety evaluations.

The delivery (birth)-related signals are spurious signals explained by confounding by contraindication. Because HPV vaccination is contraindicated during pregnancy, women are generally not vaccinated if they know they are pregnant. Some portion of the female population becomes pregnant at some point after vaccination (and others might be unknowingly at a very early stage of pregnancy when they are vaccinated). The pregnancy contraindication leads to the observation of a clustering of birth-related outcomes between about 8 months after vaccination and the end of follow-up, 1 year after vaccination.

Regarding the signals in catch-all categories of adverse events considered by clinician coders to be related to immunization (Table 2, rows 1 and 5–6), there was a similar “Other complications” cluster in an earlier study of HPV4, which also used tree-temporal scan statistics (16). In that study, that cluster was on days 1–3, and the attributable risk was 1.8 per 100,000 vaccinees, similar to the findings here. Review of claims data of the cases in the HPV4 “Other complications” cluster by a board-certified internal medicine physician revealed that most of those cases had codes for injection-site reactions, diffuse symptoms (e.g., nausea/vomiting, headache, dizziness), or unspecified symptoms; it was concluded that the cluster was consistent with general vaccine risk profiles and not suggestive of any previously unrecognized vaccine safety problem. On the basis of this previous experience with this catch-all diagnosis category, we would conclude the same for HPV9. The signal related to bacterial vaccines specifically, in row 6 of Table 2, might have been due to coding of adverse events presumed to be related to a bacterial vaccine given at the same time as HPV9, tetanus-diphtheria–acellular pertussis vaccine, for instance. We consider all these signals (Table 2, rows 1 and 5–6) to be consistent with true and known vaccine-associated adverse events.

The influenza signals are false alarms arising due to time-varying confounding. Most HPV first doses are given in July and August, before the start of the school year (17). This means that outcomes that have a seasonal pattern can produce spurious signals. Influenza season typically starts in December or January, roughly 4–5 months after most HPV first-dose vaccination, and typically lasts until early spring, roughly 7–8 months out. Focusing on the specific influenza seasons within the date range of our study, national surveillance showed percentages of influenza-like illness visits above baseline during December 2016 to mid-April 2017 (18) and during November 2017 through the end of March 2018 (19). The risk intervals we observed for influenza line up well with these influenza seasons.

The signal for “Other injury of unspecified body region” in days 1–71 (Table 2, row 4) was likely also due to time-varying confounding, given the tendency for HPV9 vaccine to be administered in July–August and the tendency for trauma visits to the emergency department to occur in the warmer months. There were no commonalities among the injuries of the 52 patients in the cluster and no suggestion of a vaccination-related etiology.

We suspect that the signal for “Other specified postprocedural states” (Table 2, row 9), with its quite broad risk interval ending about 2 months after vaccination and its P value of 0.047, is due to chance. This diagnosis code consists of 2 subcodes: Z98.890, “Personal history of surgery, not elsewhere classified,” and Z98.891, “History of uterine scar from previous surgery”; thus, any association with vaccination seems highly improbable.

The sensitivity analysis of all doses corroborated the finding of generally nonspecific adverse events attributed to immunization in days 1–2 after vaccination. The shorter follow-up period, 56 days, mitigated against the time-varying confounding that affected the main analysis.

This study constitutes a unique contribution to the safety record of HPV9 vaccine in 2 respects. First, unlike most vaccine safety studies, it was not limited to one or a few health outcomes of interest but rather evaluated as potential vaccine-associated adverse events all medically attended health conditions, as represented by the whole set of ICD-10-CM codes. Secondly, in using a follow-up period of up to 1 year, our study had the ability to capture adverse events with months-long latency periods. Complex regional pain syndrome (CRPS) is one such adverse event. CRPS affects one or more extremities and is characterized by persistent pain and swelling disproportionate to any known inciting event and at least 1 sign of autonomic dysfunction in the affected limb(s). A review of available studies suggests that CRPS symptom onset typically occurs within 6 months of the presumed inciting injury (20). In 2013, the Japanese Ministry of Health, Labor, and Welfare suspended its proactive recommendation of routine immunization with HPV vaccine for female persons after some postvaccination reports of serious chronic pain emerged (21), and the recommendation has not yet been reinstated. Our study found no evidence of any cluster of CRPS in the year following HPV9 vaccination.

We might have missed true adverse reactions if they did not show strong clustering in time or in the diagnosis tree. Not all plausible vaccine-associated syndromes affect just one system of the body. For instance, postural orthostatic tachycardia syndrome (POTS) is a heterogeneous and potentially debilitating autonomic disorder whose symptoms can include dizziness, nausea, fatigue, palpitations, weakness, sweating, and sleeping disorders. A case series of POTS occurring after HPV4 vaccination in Denmark raised concern about a possible association (22). Although there is an ICD-10 code for orthostatic hypotension (I95.1), some POTS cases might be coded as neurological, gastrointestinal, or other kinds of conditions and thus be less detectable as a cluster in the ICD-10 diagnosis tree.

A more important limitation of the study was the presence of time-varying confounding due to the typical summer timing of HPV vaccination, which undoubtedly produced the signals for influenza 4–8 months after vaccination. Confounding by contraindication was in evidence in the birth-related signals occurring late in the follow-up period. Although these false signals were easily understandable, less interpretable signals could emerge in other studies using this method. To minimize time-varying confounding, a shorter, fixed follow-up period can be used, as was done in our sensitivity analysis. In either case, the method should be regarded as a screening tool capable of finding signals (generating hypotheses) of possible vaccine- or other medical product-associated adverse events that merit further, more in-depth and customized study before being considered confirmed adverse reactions.

In summary, in this data-mining study of almost half a million HPV9 vaccinations that followed patients up to 1 year after vaccination, only nonspecific adverse events occurring in the first few days after vaccination emerged as plausible signals. Attributable risks as low as 1 excess case per 100,000 vaccinees were identified. One signal during days 7–65, involving history of surgery, might have been due to chance. All other signals were determined to have resulted from temporal confounding. Considering the broad scope of the evaluation and the high statistical power, the findings of no specific serious adverse events should serve to reassure those concerned about the safety of this vaccine.