Abstract
Objectives
China suffers from high burdens of human papillomavirus (HPV) and cervical cancer, whereas the uptake of HPV vaccine remains low. The first Chinese domestic HPV vaccine was released in 2019. However, collective evidence on cost-effectiveness of HPV vaccination in China has yet to be established. We summarised evidence on the cost-effectiveness of HPV vaccine in China.
Design
Systematic review and narrative synthesis
Data sources
PubMed, EMBASE, China National Knowledge Infrastructure and Wanfang Data were searched through 2 January 2021
Eligibility criteria for selecting studies
Cost-effectiveness studies using a modelling approach focusing on HPV vaccination interventions in the setting of China were included for review.
Data extraction and synthesis
We extracted information from the selected studies focusing on cost-effectiveness results of various vaccination programmes, key contextual and methodological factors influencing cost-effectiveness estimates and an assessment of study quality.
Results
A total of 14 studies were included for review. Considerable heterogeneity was found in terms of the methodologies used, HPV vaccination strategies evaluated and study quality. The reviewed studies generally supported the cost-effectiveness of HPV vaccine in China, although some reached alternative conclusions, particularly when assessed incremental to cervical cancer screening. Cost of vaccination was consistently identified as a key determinant for the cost-effectiveness of HPV vaccination programmes.
Conclusions
Implementing HPV vaccination programmes should be complemented with expanded cervical cancer screening, while the release of lower-priced domestic vaccine offers more promising potential for initiating public HPV vaccination programmes. Findings of this study contributes important evidence for policies for cervical cancer prevention in China and methodological implications for future modelling efforts.
Keywords: infection control, health economics, health policy, oncology
Strengths and limitations of this study.
The first systematic review on the cost-effectiveness of human papillomavirus (HPV) vaccination strategies in the setting of China.
A total of 14 modelling studies with disparate methodologies focusing on various HPV vaccination programmes and strategies were included for view.
We performed a review of four databases in both English and Chinese on a comprehensive set of contextual and methodological factors to identify key determinants for cost-effectiveness results and optimal vaccination programmes.
Not all aspects and assumptions of a model were evaluated in this review but only the ones we believed were most influential on cost-effectiveness results.
Introduction
As the leading cause of cervical cancer, human papillomavirus (HPV) is one of the most common sexually transmitted infections both in China and globally.1 A recent meta-analysis of nearly 200 studies on the prevalence of HPV revealed that as high as 19.0% (95% CI 17.1% to 20.9%) of women in China were infected with high-risk HPVs, while the subtypes with the highest infection rates were 16, 52, 58, 53 and 18.2 Meanwhile, cervical cancer is the forth most common cancer for women worldwide, accounting for over 100 000 new cases and 47 000 deaths each year in China.3 Among all the newly diagnosed cervical cancer cases in China, around two-thirds were found within the age group of 44–64 years.3 HPV is predominately transmitted through sexual contacts and is also responsible for many other diseases such as anal cancer, oropharyngeal cancer, vaginal and vulvar cancer, penile cancer and genital wart.4 5
Cervical cancer is preventable and curable in the early stages. To mobilise efforts to eliminate cervical cancer, WHO has set strategic targets for all countries by 2030, known as the 90-70-90 targets: 90% of girls fully vaccinated by age 15; 70% of women screened twice in a lifetime for cervical cancer (by age 35 and 45); 90% of women identified with cervical disease receive treatment.6 HPV vaccine has been endorsed as the most effective approach for preventing HPV infection and associated diseases. The USA, Australia, Canada and the UK were among the first countries to introduce HPV vaccine into national immunisation programmes.7 Population-level impact of HPV vaccine has been evidenced by many prior studies,8–10 and a recently published study following over 2000 women from Nordic countries who have received three doses of quadrivalent HPV vaccine for 14 years demonstrated a remarkable 100% effectiveness against HPV16/19-related high-grade cervical dysplasia.11
It was not until July 2016, a decade after the first HPV vaccine’s licensing in the USA, when the first commercial HPV vaccine, Cervarix, was approved to use in mainland China.12 The first Chinese domestic HPV vaccine (Cecolin), a bivalent vaccine against HPV 16 and 18, was licensed by the Chinese Food and Drug Administration in Dec 2019 and priced only half of Cervarix.13 However, HPV vaccine coverage rate remains low. As of March 2021, there are no HPV vaccination programmes in China. According to an online cross-sectional survey of 4220 female students from 136 Chinese universities in 2019, only 11% of participants reported have received the HPV vaccine.14 An even lower HPV vaccination coverage level (3.6% among females, 1.9% among males) was found in another online survey of college students in 2019.15 The low rate of HPV vaccination is attributable to many causes, including late introduction of HPV vaccines, high out-of-pocket costs, lack of awareness, and misunderstandings about HPV and HPV vaccines among the public.15 16
Cost-effectiveness analysis is widely used to evaluate the public health and economic value of health interventions and policies. A number of systematic reviews have assessed cost-effectiveness models evaluating HPV vaccination programmes across different policy settings.4 17–19 However, none of them have focused on the Chinese context. Although most suggested that HPV vaccination programmes were likely to be cost-effective in many settings, some reached discordant conclusions. Many contextual and methodological factors may affect the estimated cost-effectiveness of HPV vaccination, such as baseline risk of HPV infection, uptake of cervical cancer screening and treatment, analytical perspective, model design, cost of vaccine, vaccine efficacy and duration of protection and comparison strategies.19–21 Therefore, caution should be taken when interpreting or extending these study findings and when developing new economic models. Given the increasing awareness of HPV and availability of HPV vaccines (more options and lower price) among the public, a targeted review of latest cost-effectiveness models of HPV vaccination in the setting of China will be of substantial value for public health policy making. Furthermore, an understanding of the range of methods and assumptions used in analysing the cost-effectiveness of HPV vaccination can help guide future modelling development efforts.
We executed a systematic review of published cost-effectiveness studies of HPV vaccination in China. We comprehensively compared differences in cost-effectiveness results of various HPV vaccination programmes alone, in addition to, or in combination with other cervical cancer prevention interventions as a result of various modelling methods, designs and assumptions.
Methods
Search Strategy
We performed a systematic review of literature published in English and Chinese following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines22 for reporting of systematic reviews (online supplemental appendix table S1). The systematic literature search was conducted in databases MEDLINE (PubMed), and EMBASE for articles in English, as well as in databases China National Knowledge Infrastructure and Wanfang Data for articles in Chinese. We developed search terms using a combination of the following keywords: ‘HPV/cervical cancer’, ‘vaccine’, ‘cost-effectiveness’ and ‘China’ (see online supplemental appendix table S2, S3 for detailed search strategies). Corresponding key terms in Chinese were used in searching Chinese databases. Our searches covered all published literature up to our last search on 2 January 2021 with no limitations on publication date.
bmjopen-2021-052682supp001.pdf (109.1KB, pdf)
Selection criteria
Cost-effectiveness studies fulfilling the following selection criteria were included in review: (1) studies focusing on HPV vaccination interventions explicitly, alone or in combination with other interventions; (2) studies analysing the cost-effectiveness of HPV vaccination using a modelling approach, excluding those where costs were not assessed or using a non-modelling approach; (3) studies conducted in the setting of China (including the special administrative regions of Hong Kong and Macau), excluding those examining a broader context where China was only a subregion in the analysis and (4) studies published as full-length original research articles, excluding conference abstracts and commentaries, to ensure sufficient details were provided (in the manuscript or online supplemental appendix) for the information required for this review.
Data extraction and analysis
Information pertaining to the cost-effectiveness results and different contextual and methodological factors aforementioned that might influence cost-effectiveness results was extracted for each selected study. These factors were generally grouped in four categories: (1) study design, including both model structural design and analytical design; (2) HPV vaccine, such as type, efficacy, price and other assumptions; (3) HPV vaccination strategies compared and evaluation approach; and (4) uncertainty analysis and study quality. Data extraction was independently performed by two reviewers (WS and XC) and any differences were resolved with a third reviewer (HW or XZ). For articles published in Chinese, data extraction was conducted in the original language and translated into English for analysis.
We converted cost-effectiveness results and unit cost for vaccine from all studies in different currencies and years to 2021 USD according to the Consumer Price Index Inflation and USD/CNY exchange rate in January 2021 (US$1=¥6.5). For studies where the currency year was not stated, we assumed it to be the year of the study’s publication. We also provided an assessment of the reviewed economic evaluations using the Consensus on Health Economic Criteria (CHEC)-list23 to determine the percentage of checklist items (a total of 19) that each study met as a score for study quality. Given the heterogeneity in modelling designs and methods, study population (eg, girls/women of different ages), interventions (eg, type of vaccine), health outcomes, comparator strategies, as well as some practical challenges (eg, no sample size for weight assignment), a meta-analysis is rarely feasible for cost-effectiveness outcomes and thus was not performed in this study.24
Patient and public involvement statement
Neither patients nor the public were involved in this review.
Results
Figure 1 presents a flow diagram of the study search and selection process. A total of 136 articles (69 in English and 67 in Chinese) were identified after removing the duplicates (N=78). Following screening of titles and abstracts, 35 full-text articles were evaluated, among which 14 met the selection criteria (10 in English and 4 in Chinese) and were included for review.13 25–37 Of these 14 included studies, all were published after 2010 and 12 after 2016 (the year when the first HPV vaccine was licensed to use) (table 1). Most studies evaluated HPV vaccination at the national level (8/14), as opposed to focusing on one providence (4/14) or city (2/14). All studies except one (in Hong Kong) were conducted in the setting of mainland China.
Table 1.
Reference | Year of publication | Setting | Model type | Disease states modelled | Study population | Timeframe (cycle length) |
Perspective | Discount rate | Health measure | Calibration /validation |
Year of cost |
Canfell et al25 | 2011 | Shanxi Province (rural) | Cohort, Dynamic | CIN (3), cervical cancer | Males+females (of all ages) | Lifetime (1 year) |
Societal | 3% | Life year | Calibration | 2010 |
Choi et al26 | 2018 | Hong Kong | Cohort dynamic (for transmission)+individual-based (for disease) | CIN (3) Cervical cancer: 4 stages+asymptomatic/ symptomatic |
Males+females (10–85 years) | Lifetime (1 month) |
Societal | 3% | Life year, QALY | Calibration | 2018* |
Jiang et al27 | 2019 | China | Cohort, Static | Cervical cancer | Females of 16 years | Lifetime (unclear) |
Healthcare payer | 3% | DALY | No | 2017 |
Levin et al28 | 2015 | China | Individual, Static | CIN (3), cervical cancer | Females (9 years and older) | Lifetime (1 month) |
Government | Unclear | Deaths averted | Calibration | 2009 |
Liu et al29 | 2016 | China (rural, urban) |
Cohort, Static | CIN (3), cervical cancer | Females of 12–55 years | Lifetime (1 year) |
Healthcare payer | 3% | QALY | Calibration, Validation | 2016* |
Luo et al30 | 2020 | Wuhan City | Cohort, Static | Cervical cancer | Females of 12 years | Lifetime (unclear) |
Unclear | 3% | DALY | No | 2020* |
Luo et al31 | 2020 | Zhejiang Province | Cohort, Static | High-risk HPV infection Low-grade SIL High-grade SIL Cervical cancer |
Females of 12 years | Lifetime (1 year) |
Government | 3% | QALY | No | 2020* |
Ma et al32 | 2020 | China | Cohort, Dynamic | HPV infection (high/low-risk) CIN (3) Cervicl cancer Genital wart |
Females (of all ages) | 50 years (1 year) |
Unclear | 3% | DALY | Calibration | 2020 |
Mo et al33 | 2017 | China | Cohort, Static | HPV infection (high/low-risk) CIN (3) Cervicl cancer Genital wart |
Females of 12 years | Lifetime (1 year) |
Societal | 3% | QALY | Calibration | 2015 |
Qie34 | 2017 | Zhejiang Province | Cohort, Static | CIN (3), cervical cancer | Females of 18–25 years | Unclear (unclear) |
Healthcare sector | 3% | QALY | No | 2017* |
Song et al35 | 2017 | China | Cohort, Dynamic | CIN (3), cervical cancer | Males+females | 100 years (unclear) |
Healthcare sector | 3% | Life year | Validation | 2017* |
Sun et al36 | 2017 | Jiangsu Province | Cohort, Static | CIN (3), cervical cancer | Females of 18–25 years | Lifetime (unclear) |
Healthcare sector | 3% | QALY | No | 2017* |
Zhang et al37 | 2016 | China (rural, urban) |
Cohort, Static | CIN (3), cervical cancer | Females of 12 years | Lifetime (unclear) |
Healthcare payer | 3% | QALY | Validation | 2013 |
Zou et al13 | 2020 | China | Cohort, Static | CIN (3), cervical cancer | Females of 9–14 years | Lifetime (1 year) |
Healthcare sector | 3% | QALY | Calibration, Validation | 2019 |
*No year of cost reported, using publication year instead.
CIN (3), cervical intraepithelial neoplasia (three stages); DALY, disability-adjusted life-year; HPV, human papillomavirus; QALY, quality-adjusted life-year; SIL, squamous intraepithelial lesion.
Study design
The majority of studies (13/14) adopted a cohort-based model that stratifies the study population into groups according to each individual’s characteristics and health state, whereas only one used an individual-based model (table 1). In addition to the susceptible and death states, most models considered a similar set of disease states, including different stages of cervical intraepithelial neoplasia/squamous intraepithelial lesion and cervical cancer. Three studies included additional states for HPV infection (stratified by risk level) prior to developing cervical intraepithelial neoplasia, and only two also accounted for genital wart as possible consequence of HPV infection. Although HPV is an infectious disease that is transmitted through sexual contacts, dynamic models that capture changing risk of infection (as a function the number of infectious individuals in the population at a given time point)38 were only used in 4/14 studies. The study population simulated in each model were in line with the type of model used, where population of both males and females were considered all in dynamic models to construct transmission dynamics and potential herd immunity (other models considered females only). The majority of models assessed (12/14) considered a lifetime (or 100-year) time horizon to capture all possible long-term benefits and consequences of alternative interventions. Only eight models explicitly described the cycle length used in the model simulation, among which six used a yearly cycle and two used a monthly cycle. Although a shorter cycle may better capture the continuous-time reality and incidence of HPV infection during the period,39 yearly cycles may have limited impact on biasing cost-effectiveness results given the long incubation period but can help reduce computation time. In estimating relevant costs, over half studies (7/12) framed their analysis from the perspective of healthcare sector/payer, two from the perspective of government, and only three adopted a broader societal perspective, whereas two did not explicitly report their perspectives. Out of the 14 studies, 11 used utility-based measure for health outcomes, such as quality-adjusted life years and disability-adjusted life years, while the remaining estimated health benefits in the unit of life year saved or death averted. A 3% discounting rate was universally applied in all studies (except one whose discounting rate was unreported). Model calibration and validation are both recommended by modelling guidelines and are critical steps to establish the credibility and reliability of economic models against empirical data.40 41 However, only nine studies incorporated either calibration or validation (two performed both) in their analysis.
HPV vaccine
The type of HPV vaccine modelled and pertaining features and assumptions varied across different studies (table 2). All studies focused on the vaccination of females only. While in most studies (12/14) vaccination was considered to be implemented among preadolescent girls under age 18 (before sexual debut); two studies focused on vaccination at the age of 18–25, one examined HPV vaccination at different ages between 12 and 55, and one also considered expanded catch-up programmes among females aged 16–39 (the primary programme still focused on preadolescent girls). Different types of vaccine were considered in different studies: eight studies focused on a bivalent vaccine, one on a quadrivalent vaccine, one on a nonavalent vaccine, two compared all these three types in one study, while two did not specify the valence of vaccine. Three-dose schedule was considered in nine studies; the remaining either assumed a two-dose schedule or did not report required doses. Although most models derived estimates for vaccine efficacy against cervical cancer and other HPV-related disease states from real-world data (clinical trials or observational studies), a few studies (5/14) assumed a 100% vaccine efficacy, which might result in possible overestimation for the impact and cost-effectiveness of HPV vaccine. Regarding vaccination coverage for the modelled interventions, half studies assumed a 70% coverage in their primary analysis (which may vary in sensitivity analysis (SA)), following by a coverage level of 100% (3/14), 80% (1/14), 50% (1/14) and 20% (1/14), while one aimed to explore different coverage levels (25%, 50%, 75%). Most studies (13/14) assumed lifelong vaccine protection (in which one also explored other durations of protection) while only one considered a waning of immunity over time. Cost of vaccination, including medical cost for multiple doses and relevant administration cost, varied between studies on different types of vaccine ranging from US$54.2 to US$663. Most (8/14) chose a cost between US$300 and US$500. The first domestic vaccine was analysed in Zou et al13 study that was priced at US$99.8 per vaccination (in 2019 US$).
Table 2.
Reference | Age of vaccination | Vaccine type | No of doses | Vaccine efficacy | Vaccine coverage | Duration of protection | Unit cost reported* | Unit cost in 2021 USD |
Canfell et al25 | 15 | Unspecified | 3 | 100% | 70% | Lifelong | Varied in the analysis | Varied in the analysis |
Choi et al26 | 12 | Nonavalent | 2 | 95.5% (90.0%–98.4%) for HPV-16 95.8% (84.1%–99.5%) for HPV-18 96.0% (94.4%–97.2%) for HPV-OV 0% for HPV/NV |
25%, 50% and 75% | 20 years, 30 years and lifelong (various scenarios) | US$284 | US$303.1 |
Jiang et al27 | 16 | Bivalent Quadrivalent Nonavalent |
Unclear | 100% | 100% | Lifelong | Nonavalent: US$628 Quadrivalent: US$393 Bivalent: US$291 |
Nonavalent: US$682.5 Quadrivalent: US$427.1 Bivalent: US$316.3 |
Levin et al28 | <12 | Unspecified | 3 | 100% | 70% | Lifelong | US$46 | US$61.1 |
Liu et al29 | 12–55 | Bivalent | 3 | 93.2% against CC 64.9% against CIN2/3 50.3% against CIN1 |
70% | Lifelong | ¥1954 | US$333.2 |
Luo et al30 | 12 | Bivalent | 3 | 95% (63%–100%) | 100%† | Lifelong | ¥1999 | US$308.4 |
Luo et al31 | 12 | Bivalent | 2 | 76.78% (40%–100%) | 70% | Lifelong | ¥1040 | US$160.4 |
Ma et al32 | 9–16 | Quadrivalent | Unclear | 78.9% (74.5%–82.4%) | 50% | 5% rate of immunity waning | US$451 | US$452.3 |
Mo et al33 | 12 | Bivalent Quadrivalent Nonavalent |
3 | Bivalent: 80.7% (57.5%–98.9%) Quadrivalent: 81.5% (58.8%–98.2%) Nonavalent: 90.8% (66.5%–100%) |
20% (10%–100% in SA) | Lifelong | Bi/Quadrivalent: US$408 Nonavalent: US$452 |
Bi/Quadrivalent: US$459.6 Nonavalent: US$509.1 |
Qie34 | 18–25 | Bivalent | 3 | 100% | 80% | Lifelong | ¥1842 | US$308.0 |
Song et al35 | Primary: 15 Expanded: 16–39 | Bivalent | 3 | 100% | 70% | Lifelong | ¥1995 | US$333.6 |
Sun et al36 | 18–25 | Bivalent | 3 | 94.2% (62.7%–99.9%) | 100%† | Lifelong | ¥2000 | US$334.4 |
Zhang et al37 | 12 | Bivalent | 3 | 93.2% (78.9%–98.7%) against CC 64.9% (52.7%–74.9%) against CIN2/3 50.3% (40.2%–58.8%) against CIN1 |
70% | Lifelong | ¥301 | US$54.2 |
Zou et al13 | 9–14 | Bivalent | 2 | 94% (80%–99%) | 70% (50%–95% in SA) | Lifelong | US$99.8 | US$104.7 |
*Total cost per girl/woman vaccinated, including medical cost for multiple doses and other relevant costs (eg, vaccine administration).
†Among individuals with negative screening results.
CC, cervical cancer; CIN, cervical intraepithelial neoplasia; HPV, human papillomavirus; NV, non-vaccine high-risk HPV; OV, other five high-risk HPV targeted by the nonavalent vaccine; SA, sensitivity analysis.
Cost-effectiveness of HPV vaccination strategies
Despite no established cost-effectiveness threshold in China, almost all studies used the heuristic cost-effectiveness threshold proposed by the WHO based on local gross domestic product (GDP) per capita, even though two studies did not use a utility-based measure for health outcomes (table 3). The only exception was one that used an extended cost-effectiveness framework whose primary outcome was not incremental cost-effectiveness ratio (and thus did not specify the threshold). Various HPV vaccination strategies were assessed in the reviewed studies. Eight studies examined the impact and cost-effectiveness of HPV vaccination programmes incremental to either existing screening programmes or opportunistic vaccination programmes or none at all, among which three stratified their analysis by different vaccination coverage levels, different ages of vaccination and different income levels of target population. Although these eight studies sought to address slightly different study questions, they appeared to reach a consistent conclusion that HPV vaccination was cost-effective. One study examining the effect of vaccination age showed that vaccination was cost-effective at any age under 23 years in rural and any age under 25 years in urban areas. One study compared the value of nonavalent vaccine to quadrivalent and bivalent vaccine for the prevention of cervical cancer and found it not cost-effective unless the nonavalent vaccine could be priced lower than US$550 and US$450 for the full doses (as opposed to US$663 used in the study, in 2017 US$), respectively. The other five studies, on the other hand, analysed combination strategies for HPV vaccination with various HPV screening methods or frequencies, three of which also created cost-effective frontiers to identify an optimal strategy. However, findings of these studies were less consistent, and sometimes contradictory. Canfell et al study examined the association between cost-effectiveness of HPV vaccination strategies (in combination with screening interventions) and cost per vaccinated girl (CVG), and found strategies involving vaccination would be cost-effective only at CVGs of US$50–US$54 or less (if CVG>US$54, screening-only strategies would be more cost-effective).25 Ma et al study found that the addition of universal vaccination to screening programmes was not cost-effective unless with at least a 50% reduction on the vaccine price (from US$451 to US$226).32 The optimal combination of vaccine type and screening method identified in Mo et al study was nonavalent vaccination and visual inspection with acetic acid (VIA).33 However, another finding of this study was that quadrivalent and nonavalent vaccine both denominated bivalent vaccine regarding the cost-effectiveness, conversely to Jiang et al’s results.27 Although Song et al showed that the combination of vaccination at age 15 and screening twice in a lifetime (at age 35 and 45) was cost-effective compared with no intervention, but it was not cost-effective when compared with only screening twice in a lifetime (optimal strategy).35 Zou et al was the only study that included the domestic vaccine in their analysis (with lower price than the imported vaccines) and they identified the optimal strategy to be vaccination with careHPV screening once every 5 years.13 They also determined that adding vaccination to screening programmes would be consistently more cost-effective than screening alone when vaccination cost could be lower than US$50.
Table 3.
Reference | Intervention | Comparator | ICER reported | ICER in 2021 US$ | Threshold | Conclusion | Sensitivity analysis | Most sensitive parameters |
Canfell et al25 | Combination of: vaccination and different screening strategies (with different frequencies) | No vaccination, no screening | – | – | GDP: 3077 USD | Strategies involving vaccination would be cost-effective at CVGs of US$50–US$54 or less, but at CVGs>US$54, screening-only strategies would be more cost-effective | One-way, probabilistic | Discounting rate Cost of HPV screening Duration of protection |
Choi et al26 | Vaccination (routine) at different coverage levels | Opportunistic vaccination (12% coverage) | – | – | GDP: 40 US$099 | Cost-effective across all three vaccination coverage levels. Wil remain cost-effective if the cost of fully vaccinating one girl is no greater than US$689 646–US$734), respectively. | Probabilistic | – |
Jiang et al27 | Nonavalent vaccine | Quadrivalent, bivalent vaccine | US$35 000/DALY vs quadrivalent US$50 455/DALY vs bivalent |
US$38 040/DALY vs quadrivalent US$54 837/DALY vs bivalent |
1–3 * GDP: US$8640 | Not cost-effective compared with the quadrivalent and the bivalent vaccines. To be cost-effective, the 9-valent vaccine should be priced at US$550 and US$450 for the full doses, respectively | One-way | Discounting rate CC mortality Age of vaccination |
Levin et al28 | Vaccination (targeting different income groups)+screening | Screening only | US$10 920–US$13 277 per death averted | US$14 504–US$17 635 per death averted | – | Cost-effective across all income groups. Would remain cost-effective if the cost is less than US$50 per vaccinated girl. | One-way | Not reported |
Liu et al29 | Vaccination (at different ages)+Pap test | Pap test only | Varied by age | – | 1–3 * GDP: ¥41 908 | Vaccination is cost-effective at any age under 23 years in rural and any age under 25 years in urban areas. Catch-up vaccination to the age of 25 years in addition to routine vaccination in 12-year-old in both rural and urban can be cost-effective. | No | – |
Luo et al30 | Vaccination | No vaccination | ¥83 496/DALY | US$12 881/DALY | 1–3 * GDP: ¥52 000 | Very cost-effective | One-way | Discounting rate Cost of vaccine Cost of CC treatment |
Luo et al31 | Vaccination | No vaccination | ¥12 472/QALY | US$1924/QALY | 1–3 * GDP: ¥92 100 CNY | Very cost-effective | One-way | Cost of vaccine Discounting rate QALY estimates |
Ma et al32 | Combination of: universal vaccination (coverage : 0% to 90%) and screening (coverage: 20% to 70%) | Status quo (0% vaccination coverage and 20% screening coverage) | – | – | 1–3 * GDP: US$10 264 | The addition of universal vaccination to screening programmes is not cost-effective. The vaccine requires at least a 50% price reduction to be cost-effective. | Probabilistic | – |
Mo et al33 | Combination of: different types of vaccination and screening methods | No vaccination, no screening | – | – | 1–3 * GDP: US$7960 | Optimal: nonavalent vaccination+VIA screening. Quadrivalent/nonavalent vaccine, in combination with current screening strategies, is highly cost-effective and dominates bivalent vaccine | One-way | Vaccine efficacy Cost of vaccine Discounting rate |
Qie34 | Vaccination+Pap test | Pap test only | ¥43 490/QALY | US$7272/QALY | 1–3 * GDP: ¥52 000 | Very cost-effective | No | – |
Song et al35 | Combination of: different vaccination (at different ages) and VIA/VILI screening (with different frequencies) strategies | No vaccination, no screening | – | – | 1–3 * GDP: ¥50 696 | Vaccination (at age 15)+ screening twice in a lifetime (at age 35 and 45) is cost-effective compared with no intervention. Optimal: screening twice in a lifetime* |
One-way | Discounting rate Cost of vaccine Vaccine coverage |
Sun et al36 | Vaccination+Pap test | Pap test only | ¥43 489/QALY | US$7272/QALY | 1–3 * GDP: ¥52 000 | Very cost-effective | One-way | Discounting rate Vaccine efficacy Cost of vaccine |
Zhang et al37 | Vaccination+screening | Screening only | Rural: ¥11 365/QALY Urban: ¥6124/QALY |
Rural: US$2047/QALY Urban: US$1103/QALY |
1–3 * GDP: ¥41 908 | Very cost-effective. Would remain very cost-effective if vaccine cost is below ¥630 in rural and ¥750 in urban; and remain cost-effective if below ¥1700 in rural and ¥1900 in urban | One-way, two-way, probabilistic | Cost of vaccine Discounting rate HPV infection rate |
Zou et al13 | Combination of: vaccination and various screening methods with different frequencies | No vaccination, no screening | – | – | 1–3 * GDP: US$10 276 | Optimal: vaccination +careHPV screening every 5 years. Strategies that combined vaccination and screening would be more cost-effective than screening alone strategies when the vaccination cost was less than US$50 |
One-way, probabilistic | – |
*Based on analysis of the reported cost-effectiveness frontier outcomes (rather than what the authors reported).
CC, cervical cancer; CNY, Chinese yuan; CVG, cost per vaccinated girl; DALY, disability-adjusted life year; GDP, gross domestic product per capita; ICER, incremental cost-effectiveness ratio; QALY, quality-adjusted life year; VIA, visual inspection with acetic acid.
Uncertainty analysis and study quality
To assess model uncertainty, many studies explicitly incorporated SA, including one-way SA (in 10 studies), two-way SA (in 1 study) and probabilistic SA (in 5 studies) (table 3, some studies incorporated multiple types of SA). Among those performed one-way SA, the parameters that cost-effectiveness results were most sensitive to included discounting rate, cost of vaccine and vaccine efficacy. Quality assessment of the reviewed studies against the CHEC-list suggested that most of them upheld a high level of quality in reporting, with an average score of 85 and ranging from 53 to 100 (where 100 represented 100% of checklist items were complied with) (online supplemental appendix table S4).
Discussion
To our knowledge, this study provides the first systematic review on the cost-effectiveness of introducing HPV vaccination programmes in the setting of China. In this review, we performed a comprehensive and in-depth assessment of 14 model-based cost-effectiveness studies regarding their findings, study design, and assumptions for HPV vaccine and vaccination programmes. Despite considerable heterogeneity in the methodologies used in different models, our findings show that HPV vaccination is estimated to have substantial potential to be a cost-effective addition to existing/other cervical cancer prevention interventions in China. However, the cost-effectiveness of HPV vaccination is likely to depend on considerations such as cost of vaccination, age of vaccination, vaccine efficacy, as well as complementary and/or competing strategies (eg, cervical cancer screening).
Among all the influential factors, cost of vaccine was consistently identified as a key determinant for the cost-effectiveness of HPV vaccination. Cost estimates varied considerably across studies for different vaccines and years; acquiring more reliable evidence on vaccine cost will help reduce uncertainty surrounding cost-effectiveness results. Six of the reviewed studies performed additional threshold analysis to determine the cost at which adding HPV vaccination to cervical cancer screening programmes would become/remain (more) cost-effective. While three studies suggested disparate thresholds for the cost per fully vaccinated girl/woman ranging from US$226 to US$689, findings of the other three were more consistent showing a lower threshold of US$50. The Zou et al study assessed the first domestic bivalent HPV vaccine at a unit cost of US$99.8, substantially cheaper than the imported vaccines.13 Given more domestic vaccines under development and growing initiatives to include HPV vaccine into national immunisation programme, further reduction in vaccine price and improved cost-effectiveness is attainable in the foreseeable future. Furthermore, some other characteristics and assumptions of HPV vaccine and vaccination programmes were also found to be associated with increased cost-effectiveness of HPV vaccination, such as higher vaccine efficacy, longer duration of vaccine immunity, younger age being vaccinated and higher vaccination coverage (although most models did not account for herd immunity).
Findings of the reviewed studies were generally consistent with other systematic reviews focusing on cost-effectiveness of HPV vaccination in low-income and middle-income countries.19 42 43 Most of these studies concluded that vaccination was likely to be cost-effective, particularly in contexts without organised cervical cancer screening programmes. On the contrary, HPV vaccination, regardless of the type of vaccine and modelling design, was more consistently found in high-income countries,44 45 due in large to higher willingness to pay thresholds and vaccine uptake. Based on the summary of evidence, a few recommendations may be provided for implementing HPV vaccination programmes to enhance its cost-effectiveness. HPV vaccine is most recommended for routine vaccination for girls at younger age (before 16) while will still remain valuable for women of older age (under 23 years in rural and under 25 years in urban areas) according to one reviewed study that explored different vaccination ages.29 The US Centers for Disease Control and Prevention recommended vaccination for everyone (including men) at age 11 through age 26 years.46 In the UK, men and women aged 12–13 years are routinely offered HPV vaccination and can access free vaccination up until their 25th birthday.47 Regarding the type of vaccine for recommendation, two studies reached contradicting conclusions about the relative cost-effectiveness between nonavalent, quadrivalent and bivalent vaccines. This difference is likely attributable to disparate costs applied for different vaccines in the two models. In Jiang et al’s model,27 nonavalent vaccine was assumed to be 60% and 116% more expensive than quadrivalent and bivalent vaccines, respectively, while it costed only 11% more than the other two vaccines in Mo et al’s study.33 Future investigations of different vaccines and their pricing, efficacy and population impacts may be required for more rigorous recommendation strategies. Meanwhile, the reviewed studies demonstrated strong synergies between HPV vaccination and cervical cancer screening that the greatest public health benefits, and sometimes also the optimal strategy, could be achieved only when these two interventions were implemented simultaneously. However, in identifying the optimal combination strategies, two studies indicated that screening alone might outperform strategies with the addition of HPV vaccination, while there was less consistency regarding the screening methods (pap, VIA or careHPV test) and testing frequencies. Given current low uptake of screening in China, establishing appropriate strategies to substantially expand cervical cancer screening should be prioritised prior to or simultaneously with implementing HPV vaccination programmes.
From the methodological point of view, a few recommended model design and practice may be highlighted for future modelling efforts. First, a key finding of this review was that the majority of reviewed studies applied a static model in simulating HPV infection that was unable to capture potential herd immunity when HPV vaccination reached a high level of coverage. According to the modelling guideline, dynamic design is important to consider when an intervention affects a pathogen’s ecology or when the intervention affects disease transmission.38 Incorporating dynamic design will ensure capturing the indirect effects of HPV vaccination that arise from averted infections, that is, individuals not reached by the vaccination programme can still benefit by experiencing a lower infection risk. However, applying such a dynamic model may require modelling the population of men (who are non-recipients of HPV vaccine) as well as additional model parameters. Second, cost-effectiveness models are built on various input data and assumptions and are inevitably subject to uncertainty. Handling model uncertainty is important and can help assess the robustness of model results and enhance our confidence in a chosen course of action. Model calibration and SA are both recommended practices40 to address uncertainty but were not performed in all models (calibration in 7/14 models, SA in 12/14 models). For the conduct of uncertainty analysis, we also recommend carefully choosing uncertainty ranges for parameters to meaningfully reflect their plausible values (rather than imposing an arbitrarily range) and explicitly reporting the rationale. Third, although cervical cancer is the primary disease following HPV infection, it is also important to account for other possible consequences and diseases, without which the impact and cost-effectiveness of HPV vaccination may be underestimated.
Our review may have some limitations. First, we did not attempt to exhaustively include all aspects and assumptions of a model in this review (such as utility estimates, force of infection, disease progression) but only the ones we believed were most influential on cost-effectiveness results. Second, the quality of evidence used to support a model is another central factor in ensuring credibility and reliability of model inferences but was not assessed in this review. Third, we were unable to perform a meta-analysis due to the variability across studies in the strategies evaluated and outcomes reported. Nevertheless, all studies have compared the estimated cost-effectiveness with the WHO-CHOICE cost-effectiveness benchmark using local (national, provincial or city-level) GDP per capita, providing a consistent criterion across studies.
The body of evidence from this systematic review of cost-effectiveness modelling studies on HPV vaccine suggests that implementing HPV vaccination programmes for young girls is likely warranted in China and should be paired with expansion of cervical cancer screening to maximise their impact. Cost of vaccination was found to significantly affect the cost-effectiveness estimates and policy recommendations. As domestic vaccines become available and their prices continue to drop, HPV vaccination will become a more viable option in designing cervical cancer prevention programmes. Future modelling studies following established best-practice standards are needed to reduce decision uncertainty and definitively establish the cost-effectiveness of HPV vaccination in combination with screening programmes.
Supplementary Material
Footnotes
Contributors: WS and TC conceptualised the study. WS and XC conducted data collection. WS, HW and XZ performed data analyses. WS and TC wrote the first draft of the article. XC, HW and XZ helped to interpret results and critically revise the manuscript. TC is responsible for the overall content as guarantor. All authors approved the final draft.
Funding: This work was supported by the State Administration of Traditional Chinese Medicine of the People’s Republic of China (grant number: GZY-GCS-2017–36).
Disclaimer: The funder had no role in study design, data analysis, or decision to publish.
Competing interests: None declared.
Provenance and peer review: Not commissioned; externally peer reviewed.
Supplemental material: This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.
Data availability statement
All data relevant to the study are included in the article or uploaded as online supplemental information. There are no additional unpublished data associated with this study. All data are available in the manuscript and online supplemental appendix.
Ethics statements
Patient consent for publication
Not applicable.
References
- 1.Zhu B, Liu Y, Zuo T, et al. The prevalence, trends, and geographical distribution of human papillomavirus infection in China: the pooled analysis of 1.7 million women. Cancer Med 2019;8:5373–85. 10.1002/cam4.2017 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Li K, Li Q, Song L, et al. The distribution and prevalence of human papillomavirus in women in mainland China. Cancer 2019;125:1030–7. 10.1002/cncr.32003 [DOI] [PubMed] [Google Scholar]
- 3.Bruni L, Albero G, Serrano B. ICO/IARC information centre on HPV and cancer (HPV information centre). human papillomavirus and related diseases in China. Summary Report 2019. https://hpvcentre.net/statistics/reports/CHN.pdf [Google Scholar]
- 4.Gervais F, Dunton K, Jiang Y, et al. Systematic review of cost-effectiveness analyses for combinations of prevention strategies against human papillomavirus (HPV) infection: a general trend. BMC Public Health 2017;17:1–18. 10.1186/s12889-017-4076-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Bansal A, Singh MP, Rai B. Human papillomavirus-associated cancers: a growing global problem. Int J Appl Basic Med Res 2016;6:84. 10.4103/2229-516X.179027 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.World Health Organization . Global strategy to accelerate the elimination of cervical cancer as a public health problem. Geneva: World Health Organization, 2020. https://www.who.int/publications/i/item/9789240014107 [Google Scholar]
- 7.Markowitz LE, Tsu V, Deeks SL, et al. Human papillomavirus vaccine introduction--the first five years. Vaccine 2012;30 Suppl 5:F139–48. 10.1016/j.vaccine.2012.05.039 [DOI] [PubMed] [Google Scholar]
- 8.Mix JM, Van Dyne EA, Saraiya M, et al. Assessing impact of HPV vaccination on cervical cancer incidence among women aged 15-29 years in the United States, 1999-2017: an ecologic study. Cancer Epidemiol Biomarkers Prev 2021;30:30–7. 10.1158/1055-9965.EPI-20-0846 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Steben M, Tan Thompson M, Rodier C, et al. A review of the impact and effectiveness of the quadrivalent human papillomavirus vaccine: 10 years of clinical experience in Canada. J Obstet Gynaecol Can 2018;40:1635–45. 10.1016/j.jogc.2018.05.024 [DOI] [PubMed] [Google Scholar]
- 10.Leval A, Herweijer E, Ploner A, et al. Quadrivalent human papillomavirus vaccine effectiveness: a Swedish national cohort study. J Natl Cancer Inst 2013;105:469–74. 10.1093/jnci/djt032 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Kjaer SK, Nygård M, Sundström K, et al. Final analysis of a 14-year long-term follow-up study of the effectiveness and immunogenicity of the quadrivalent human papillomavirus vaccine in women from four Nordic countries. EClinicalMedicine 2020;23:100401. 10.1016/j.eclinm.2020.100401 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Chen R, Wong E. The feasibility of universal HPV vaccination program in Shenzhen of China: a health policy analysis. BMC Public Health 2019;19:1–10. 10.1186/s12889-019-7120-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Zou Z, Fairley CK, Ong JJ, et al. Domestic HPV vaccine price and economic returns for cervical cancer prevention in China: a cost-effectiveness analysis. Lancet Glob Health 2020;8:e1335–44. 10.1016/S2214-109X(20)30277-1 [DOI] [PubMed] [Google Scholar]
- 14.You D, Han L, Li L, et al. Human papillomavirus (HPV) vaccine uptake and the willingness to receive the HPV vaccination among female college students in China: a multicenter study. Vaccines 2020;8:31. 10.3390/vaccines8010031 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Deng C, Chen X, Liu Y. Human papillomavirus vaccination: coverage rate, knowledge, acceptance, and associated factors in college students in mainland China. Hum Vaccin Immunother 10.1080/21645515.2020.1797368 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Ji M, Huang Z, Ren J, et al. Chinese vaccine providers' perspectives on the HPV vaccine. Glob Pediatr Health 2020;7:2333794X20967592 10.1177/2333794X20967592 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Seto K, Marra F, Raymakers A, et al. The cost effectiveness of human papillomavirus vaccines: a systematic review. Drugs 2012;72:715–43. 10.2165/11599470-000000000-00000 [DOI] [PubMed] [Google Scholar]
- 18.Pink J, Parker B, Petrou S. Cost effectiveness of HPV vaccination: a systematic review of modelling approaches. Pharmacoeconomics 2016;34:847–61. 10.1007/s40273-016-0407-y [DOI] [PubMed] [Google Scholar]
- 19.Fesenfeld M, Hutubessy R, Jit M. Cost-effectiveness of human papillomavirus vaccination in low and middle income countries: a systematic review. Vaccine 2013;31:3786–804. 10.1016/j.vaccine.2013.06.060 [DOI] [PubMed] [Google Scholar]
- 20.Ekwunife OI, O'Mahony JF, Gerber Grote A, Grote AG, et al. Challenges in cost-effectiveness analysis modelling of HPV vaccines in low- and middle-income countries: a systematic review and practice recommendations. Pharmacoeconomics 2017;35:65–82. 10.1007/s40273-016-0451-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Natunen K, Lehtinen T, Torvinen S. Cost-effectiveness of HPV-vaccination in medium or low income countries with high cervical cancer incidence—a systematic review. Journal of Vaccines and Vaccination 2013;4:2–10. [Google Scholar]
- 22.Moher D, Liberati A, Tetzlaff J, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Int J Surg 2010;8:336–41. 10.1016/j.ijsu.2010.02.007 [DOI] [PubMed] [Google Scholar]
- 23.Evers S, Goossens M, de Vet H, et al. Criteria list for assessment of methodological quality of economic evaluations: consensus on health economic criteria. Int J Technol Assess Health Care 2005;21:240–5. 10.1017/S0266462305050324 [DOI] [PubMed] [Google Scholar]
- 24.Shields GE, Elvidge J. Challenges in synthesising cost-effectiveness estimates. Syst Rev 2020;9:1–7. 10.1186/s13643-020-01536-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Canfell K, Shi J-F, Lew J-B, et al. Prevention of cervical cancer in rural China: evaluation of HPV vaccination and primary HPV screening strategies. Vaccine 2011;29:2487–94. 10.1016/j.vaccine.2010.12.085 [DOI] [PubMed] [Google Scholar]
- 26.Choi HCW, Jit M, Leung GM, et al. Simultaneously characterizing the comparative economics of routine female adolescent nonavalent human papillomavirus (HPV) vaccination and assortativity of sexual mixing in Hong Kong Chinese: a modeling analysis. BMC Med 2018;16:1–13. 10.1186/s12916-018-1118-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Jiang Y, Ni W, Wu J. Cost-effectiveness and value-based prices of the 9-valent human papillomavirus vaccine for the prevention of cervical cancer in China: an economic modelling analysis. BMJ Open 2019;9:e031186. 10.1136/bmjopen-2019-031186 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Levin CE, Sharma M, Olson Z, et al. An extended cost-effectiveness analysis of publicly financed HPV vaccination to prevent cervical cancer in China. Vaccine 2015;33:2830–41. 10.1016/j.vaccine.2015.02.052 [DOI] [PubMed] [Google Scholar]
- 29.Liu Y-J, Zhang Q, Hu S-Y, et al. Effect of vaccination age on cost-effectiveness of human papillomavirus vaccination against cervical cancer in China. BMC Cancer 2016;16:1–11. 10.1186/s12885-016-2207-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Luo P, Zhang C, Liy D. Rapid evaluation of health economics of bivalent HPV vaccination in Wuhan based on prime tool. China Pharmacist 2020;23:1115–8. [Google Scholar]
- 31.Luo Y, He H, Tang X, et al. Cost-effectiveness of 2-dose human papillomavirus vaccination for 12-year-old girls in Zhejiang Province: implications for China's expanded program on immunization. Hum Vaccin Immunother 2020;16:1623–9. 10.1080/21645515.2019.1711299 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Ma X, Harripersaud K, Smith K. Modeling the epidemiological impact and cost-effectiveness of a combined schoolgirl HPV vaccination and cervical cancer screening program among Chinese women. Hum Vaccin Immunother 2020:1–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Mo X, Gai Tobe R, Wang L, et al. Cost-effectiveness analysis of different types of human papillomavirus vaccination combined with a cervical cancer screening program in mainland China. BMC Infect Dis 2017;17:1–12. 10.1186/s12879-017-2592-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Qie S. Pharmacoeconomic evaluation of cervical cancer vaccine in the prevention of cervical cancer in Chinese women aged 18-25. J Clin Med 2017;4:9342–5. [Google Scholar]
- 35.Song XB, Zhao QJ, Zhou Z, et al. [Health economic evaluation of bivalent human papilloma virus vaccine in China: based on the dynamic model]. Zhonghua Yu Fang Yi Xue Za Zhi 2017;51:814–20. 10.3760/cma.j.issn.0253-9624.2017.09.008 [DOI] [PubMed] [Google Scholar]
- 36.Sun Y, Liu Y, Liu T. Economic evaluation of human papillomavirus vaccine versus Chinese women aged 18 to 25 for treating cervical cancer. Chin J Evid-Based Med 2017;17:102–7. [Google Scholar]
- 37.Zhang Q, Liu Y-J, Hu S-Y, et al. Estimating long-term clinical effectiveness and cost-effectiveness of HPV 16/18 vaccine in China. BMC Cancer 2016;16:1–12. 10.1186/s12885-016-2893-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Pitman R, Fisman D, Zaric GS, et al. Dynamic transmission modeling: a report of the ISPOR-SMDM modeling good research practices task force working Group-5. Med Decis Making 2012;32:712–21. 10.1177/0272989X12454578 [DOI] [PubMed] [Google Scholar]
- 39.O'Mahony JF, Newall AT, van Rosmalen J. Dealing with time in health economic evaluation: methodological issues and recommendations for practice. Pharmacoeconomics 2015;33:1255–68. 10.1007/s40273-015-0309-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Briggs AH, Weinstein MC, Fenwick EAL, et al. Model parameter estimation and uncertainty: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force--6. Value Health 2012;15:835–42. 10.1016/j.jval.2012.04.014 [DOI] [PubMed] [Google Scholar]
- 41.Eddy DM, Hollingworth W, Caro JJ, et al. Model transparency and validation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force--7. Value Health 2012;15:843–50. 10.1016/j.jval.2012.04.012 [DOI] [PubMed] [Google Scholar]
- 42.Silas OA, Achenbach CJ, Murphy RL, et al. Cost effectiveness of human papilloma virus vaccination in low and middle income countries: a systematic review of literature. Expert Rev Vaccines 2018;17:91–8. 10.1080/14760584.2018.1411195 [DOI] [PubMed] [Google Scholar]
- 43.Okeah BO, Ridyard CH. Factors influencing the cost-effectiveness outcomes of HPV vaccination and screening interventions in low-to-middle-income countries (LMICs): a systematic review. Appl Health Econ Health Policy 2020;18:641–54. 10.1007/s40258-020-00576-7 [DOI] [PubMed] [Google Scholar]
- 44.Marra F, Cloutier K, Oteng B, et al. Effectiveness and cost effectiveness of human papillomavirus vaccine: a systematic review. Pharmacoeconomics 2009;27:127–47. 10.2165/00019053-200927020-00004 [DOI] [PubMed] [Google Scholar]
- 45.Mahumud RA, Alam K, Keramat SA, et al. Cost-effectiveness evaluations of the 9-valent human papillomavirus (HPV) vaccine: evidence from a systematic review. PLoS One 2020;15:e0233499. 10.1371/journal.pone.0233499 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Meites E, Szilagyi PG, Chesson HW. Human papillomavirus vaccination for adults: updated recommendations of the Advisory Committee on immunization practices: Wiley online library, 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Government of Canada . Overview of national data on opioid-related harms and deaths, 2018. Available: https://www.canada.ca/en/health-canada/services/substance-use/problematic-prescription-drug-use/opioids/data-surveillance-research/harms-deaths.html [Accessed 15 Jan 2019].
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
bmjopen-2021-052682supp001.pdf (109.1KB, pdf)
Data Availability Statement
All data relevant to the study are included in the article or uploaded as online supplemental information. There are no additional unpublished data associated with this study. All data are available in the manuscript and online supplemental appendix.