Proceedings of the Modeling Evidence in HPV Pre-Conference Workshop

Abstract

All published and active HPV modelers from around the world were invited to Malmo, Sweden for the inaugural Modeling Evidence in HPV (MEHPV) Pre-Conference Workshop, May 9 and 10, 2009. This workshop—held for the 2 days preceding the 25th International Papillomavirus Conference—provided a unique opportunity for open discussion on HPV modeling and the related scientific and methodological issues. From over a dozen countries and a variety of settings, 34 participants (representing 82% of the HPV modeling literature) exchanged ideas on the field’s fundamental questions. The proceedings, based on the 217-page transcript, was assembled by the Scientific Committee for the explicit purpose of objectively summarizing these ideas in a de-identified, readable fashion. It represents the work and opinions of session participants as recorded and does not constitute official positions of the participants as a whole or individually, the scientific committee, or any sponsoring organization or entity. Through transparency and broad dissemination, this introspective proceedings further characterizes prominent, contemporaneous issues in the state of HPV modeling.

INTRODUCTION

The establishment of human papillomavirus (HPV) as a major cause of human cancer led to a wave of technological advances in cancer prevention, including HPV vaccination and testing (1). However, the lack of awareness of the established knowledge, the new technologies, and the optimal ways to organize preventive strategies are recognized as a major bottleneck to improved public health gains. On May 9 and 10, 2009, 34 HPV modelers, clinicians, and epidemiologists from around the world gathered in Malmo, Sweden, for the inaugural Modeling Evidence in HPV (MEHPV) Pre-Conference Workshop. The MEHPV Workshop—held for the 2 days preceding the 25^th International Papillomavirus Conference—provided a unique opportunity for open discussion on HPV modeling and the related scientific and methodological issues (2). The objective was to develop an international network of investigators engaged in HPV modeling and to facilitate open discussion relating to structure, parameterization, and other methodological issues commonly faced within our community.

The workshop was patterned after the Mount Hood Challenge Meetings, (3) which substantially advanced the field of diabetes modeling, and the EuroQol Group, which regularly assembles diverse scientists from around the world interested in health measurement and valuation. Without the efforts of a stand-alone meeting, the pre-conference workshop affords an occasion to discuss a number of important issues in HPV modeling and to exchange ideas on some fundamental questions. An in-depth and detailed literature review of all applicable research was performed, analyzing results and narrowing down to the most prolific modelers. Invitations containing workshop details were sent to potential attendees. Efforts were undertaken to invite all active HPV modelers, many of whom were planning to attend the general conference; requests were made for invitees to submit any potential modelers who inadvertently did not receive an invitation. Among the achievements of the workshop, several authors and coauthors of the bulk of the published literature (82% or 75 of 92 articles that were identified to be about HPV modeling) attended this unique meeting (see Appendix 1). The participants came from a variety of settings and over a dozen countries, which facilitated a true and meaningful exchange on the full breadth of the scientific literature.

Composed of academic, public, and private sector HPV modelers, the 34 participants held focused high-level discussions based on their collective HPV modeling experience. Attendees endeavored to remain impartial, setting aside interests of affiliations and constituent institutions (e.g., GlaxoSmithKline, Merck & Co., Sanofi Pasteur, US Centers for Disease Control and Prevention, US National Cancer Institute, and UK Health Protection Agency) to cultivate an open dialogue among contributing practitioners.

The sessions were developed by the MEHPV Workshop Scientific Committee in cognizance with the need to open channels of communication within the HPV modeling community. To synthesize knowledge, it was imperative to organize a neutral forum in which ideas and information could be shared. Thorough planning set the stage in anticipation of providing a stimulating, fertile climate that fostered new ideas and creative solutions. Indeed, as the workshop progressed, concepts evolved into innovative and forward-looking ideas.

The following summary is arranged by discussion in chronological order and all attempts have been made to retain the accuracy of discussions and to identify references where possible. The agenda consisted of seven topic-specific discussion sessions, as well as general introductory and closing summary discussions (see Appendix 1). Each session was introduced and moderated by an MEHPV attendee who had expertise in the area being discussed. No slide presentations or formal handouts were admitted.

At the onset of each session, attendees were reminded that the discussion was being recorded and would later be transcribed for potential publication. Anonymity was assured. Transcripts were reviewed and edited by all Scientific Committee members to improve readability and to protect participant identities only. This summary represents the work and opinions of session participants as recorded and does not constitute official positions of the participants as a whole or individually, the scientific committee, or any sponsoring organization or entity. Content was maintained, even when disputed by Committee members. The 25th International Papillomavirus Conference was supported by the International Papillomavirus Society (IPVS) through conference grants from GlaxoSmithKline, Merck & Co., and Sanofi Pasteur. In addition to IPVS funds, the pre-conference was also supported through conference and pre-conference registration fees. No honoraria were received by any participant or Scientific Committee member.

DAY 1

Objectives of Modeling Research

Three main issues comprise the objectives noted by HPV modelers:

1. Policy-related issues

2. Theoretical issues

3. Model issues

While issues may differ, two overriding themes emerged: the need to continue forward motion and the need for modeling transparency, both necessary to advance to the next level of understanding.

Policy-Related Issues

General policy-related issues include the effectiveness and cost-effectiveness of HPV vaccination strategies in both females and males. For example, HPV modeling can inform policy-makers (e.g., the Advisory Committee on Immunization Practices [ACIP]) to guide their development of HPV vaccine recommendations.

The issue of the cost-effectiveness of vaccinating males warrants further investigation into both the reduction of HPV-related female disease due to herd immunity and the reduction of HPV-related male disease. Current models are being re-examined to understand discordant results; models under development are observed closely to evaluate the findings.

Female Vaccination

In the United States, the ACIP routinely recommends the HPV vaccination for 11-to 12-year-old girls and a “catch-up” vaccination for 13- to 26-year-old females (who were not previously vaccinated). Workshop attendees noted that studies modeling a “catch-up” vaccine for females over the age of 26 are needed to inform the ACIP on future vaccine recommendations—assuming licensure of an appropriate HPV vaccine.

Male Vaccination

To inform policy-makers on the effectiveness and cost-effectiveness of male vaccination—thus guiding development of vaccine recommendations—further research is necessary to understand and account for the disparate results across available models. Refer to “Sexual Behavior and the Question of Male Vaccination” section for detailed discussion.

Bivalent and Quadrivalent Vaccination

An important, fundamental policy-related issue is the choice between the quadrivalent HPV vaccine (with protection against HPV 6, 11, 16, and 18) and the bivalent HPV vaccine (with protection against HPV 16 and 18). Modeling to inform policy-makers is dependent on knowing the HPV types in question.

Impact of vaccination on cervical cancer screening

With the ever-changing technological landscape of screening and vaccination, cervical cancer screening models appear to be at the forefront, evolving as needs change. The ACIP focuses mainly on HPV vaccine recommendations within the context of current cervical cancer screening in the US. However, the Centers for Disease Control and Prevention (CDC), as well as others, are showing interest in creating models to explore how cervical cancer screening strategies might evolve over time following the onset of HPV vaccination in the US—particularly for medically underserved women.

Further policy issues concern data from several randomized controlled trials of different HPV screening protocols and technologies. In the near future, it is likely that several countries will use HPV testing either as stand-alone primary screening or in conjunction with cytology and/or supplemental tests.

In addition, it may be helpful to model the impact of screening tests before and after the impact of the vaccination programs.

Theoretical Issues

Vaccination Coverage

Exploring the impact of different levels of vaccination coverage is important both to policy-making and to epidemiological theory. In the US, HPV vaccination coverage levels are relatively low, in terms of not only people who receive just the initial dose of the vaccine, but also those who receive all three doses (4-7). A poster presentation at the 25^th International Papillomavirus Conference showed that only about 50% of individuals who received the first dose went on to receive all 3 doses (8). This has major implications in terms of how well the population is protected. It also increases the importance of assumptions made about efficacy of one, two, and three doses. While good disease data exists, modelers are still grappling with the issue of degree of protection as it relates to transmission and infection, specifically, assumptions that protection against infection leads to protection against transmission. Currently, conflicting data exists.

Coverage assumptions are also likely to vary across settings. In the Netherlands, which has a good screening program, vaccine coverage is currently only about 50% (compared to an anticipated coverage of 80%-90%) (NR). In the US, there are states with vaccine uptake rates as low as 10%, where the price of the vaccine is not the only factor contributing to the low coverage (NR).

Vaccine Efficacy

Vaccine efficacy can be incorporated differently across models. For example, a model can include efficacy against infection with HPV vaccine types, efficacy against disease associated with the HPV vaccine types, or both. Many published models have assumed that protection against infection thereby precludes transmission. Although reasonable, this assumption needs to be better supported.

It was also noted that differences exist in modeling the waning of vaccine protection. For instance, if the duration of vaccine protection is 20 years, this could be modeled by (1) assuming 100% protection for all vaccinated individuals for 20 years, then 100% susceptibility after that, or (2) assuming an average of 20 years protection, such that the duration of protection is longer than average for some and shorter than average for others. Refer to “Modeling Transmission” and “Natural History, Transmission and Behavior” sections for detailed discussions.

Uncertainty in HPV Natural History

Gaps in data often lead to a reliance on unsupported assumptions. There is a considerable gap of in-depth data on the natural history of HPV infection—particularly for males and for noncervical sites in females. The focus of most previously published HPV models has mainly been on the heterosexual transmission of HPV and the natural history of cervical infection. Several models have explored male and noncervical female HPV endpoints, but only in an approximate way—mainly because of lack of data. To illustrate, more data exists on the natural history of cervical cancer than on the natural history of anal cancer. Moreover, the data on the quality of life implications of noncervical disease is limited. Refer to “Natural History of HPV Infection” section for detailed discussion.

Models also need to address the extent to which male vaccination can prevent disease in men who engage in sex with men (MSM) —assuming the vaccine is efficacious in males—and what vaccination strategies are appropriate. Refer to “Sexual Behavior and the Question of Male Vaccination” section for detailed discussion.

An additional area of uncertainty for investigation is the distinction between new infections, reactivation of latent infections, and re-infection. The importance of this distinction was highlighted by Goldie and colleagues (2004), who found that the estimated impact of HPV vaccination was quite sensitive to assumptions regarding the proportion of persistent HPV infections that were newly acquired versus latent (9). Refer to “Natural History, Transmission, and Behavior” section for detailed discussion.

HPV Disease and Comorbidities

Life events (such as immunosuppressive disorders, chronic illnesses, diabetes and HIV) have the potential to alter the progression of HPV infection to HPV disease in large subgroups within populations. For individuals with chronic illness, quality of life is further degenerated when HPV infection moves to disease. While this important supposition makes sense, further modeling is necessary. Refer to “Comorbidity” section for detailed discussion.

Coinfection and Interaction Between HPV Types

Data on coinfection and potential interaction between high-risk HPV types is severely lacking, yet must be taken into account to avoid overestimating the benefit of vaccination. In Australia, for example, cross-sectional studies have shown that the prevalence of coinfection is high, although the extent to which this is a cohort effect has yet to be determined (10).

Uncertainty in HPV Transmission Probability

While there are limited data describing the incidence of HPV-related disease, there are few data sets describing the probability of HPV transmission. Refer to “Modeling Transmission” and “Natural History, Transmission and Behavior” sections for detailed discussions.

HPV and HIV

Workshop attendees noted that it is necessary to explore the possibility that HPV might increase susceptibility to HIV (Human Immunodeficiency Virus). Data have been published addressing HIV incidence among a cohort of MSM (11) and a cohort of women in Zimbabwe (12). This is particularly relevant for developing countries.

Model Uncertainty

Characterization of uncertainty across different possible models is also of concern. An understanding is needed of how the functional forms used to represent relationships between variables in different models effect outcomes. Assumptions of models should be outlined and used consistently to be aware of their impacts. While it could be useful to achieve consistency within the models, it may be difficult to do this in practice. Models involve different populations and methodologies, so there is no one-size-fits-all method that incorporates the breadth of information, uses it in a way that is meaningful to audiences, and makes it interpretable and usable. Refer to “Models and Challenges” and “Open Discussion” sections for detailed discussion.

Model Issues

Varied Parameter Sets

It was remarked that perhaps the largest challenge facing modelers is the use of varied parameter sets, which makes it difficult to compare models. One approach would be to standardize parameter values; however, it may be even more fruitful to try to fit models to epidemiological data from multiple studies to achieve robustness. It is important that such fitting is done at interim points, rather than simply disease endpoints. HPV incidence is likely to vary in different populations because sexual behavior is likely to vary.

It was also noted that one approach is to attempt to match assumptions of key parameters (such as the utility loss associated with HPV-related health outcomes, e.g., genital warts) to differences in model structures. By harmonizing parameters, modelers are able to standardize model inputs and reconcile differences resulting from differing structural assumptions in models. Refer to “Pivotal Parameters and Evidence” and “Model Transference, Homogeneity, and Comparative Modeling” sections for detailed discussions.

The Need for Pilot Models, Consistency, and Transparency

Individual-level data from epidemiological studies are often not available publicly. Work is underway to compile such data from large trials; additional funding is needed to raise the priority.

In cost-effectiveness evaluations, there are large differences in reference-case assumptions between countries, which have a correspondingly enormous impact on cost-effectiveness conclusions. Country-by-country parameter values are limited, making it difficult for modelers to have concrete values (10). Refer to “Country-Specific Parameters” section for detailed discussion.

Existing models predict significant variations in uptake data between time of initial introduction of vaccination and subsequent years. Uncertainty in coverage dynamics leads to differences between models. Once a national vaccination program is initiated, it is difficult to influence the design and input a monitoring mechanism to gather information. A comment was made that instead of looking at just one country, a pilot model can be developed based on the past experiences of a variety of countries. Furthermore, for developing countries, there is a window of opportunity in which to construct such a pilot model of vaccination coverage, wherein the design could be used to improve understanding of the dynamic effects of vaccination. Refer to “Data Availability and Quality” and “Open Discussion” sections for detailed discussion.

Review of Models and Challenges

Geographic and Economic Differences

Models have been developed with different levels of sophistication and applied to different countries and circumstances. Much of the model development has been driven by the locations where the models were to be applied. Therefore, the screening strategies for HPV-related disease differ between countries, which have an impact on model development, and in turn on model parameters. To guide future model development, several factors to consider are: country to country categorization of models, application circumstances, and resolution of challenges.

Economic models will always be in flux, due to additional factors, in comparison to purely epidemiological models, which can be more constant. For example, patents expire and prices change, so the costs of screening and vaccination are not fixed in time. For modelers, it is prudent to take into account that future vaccines may prove to be more effective, of broader spectrum, and lower in price (NR).

Structural Differences

HPV types are modeled with various methods. Some models group different high-risk HPV types together and use progression and regression rates, which represent an amalgam of the individual rates for each type. Other models use separate rates for each HPV type or for various combinations of types (e.g., 16 and 18 combined). Some models assume progression from HPV infection to cervical cancer is more likely among older individuals, whereas others assume progression rates are not age-specific. All of these variations inevitably lead to disparate results.

More broadly, there are differences in parameter assumptions and units of outcome measurement, which further complicate disentangling structural differences. Models measure outcomes in various parameters: natural units (such as cancers prevented or life years lost) or composite outcome measures (such as Quality-Adjusted Life Years [QALYs]; refer to “Quality of Life” section for detailed discussion); some parameters are not directly comparable. To illustrate, a proportion of females progressing from CIN1 (cervical intraepithelial neoplasia) to CIN2 over a certain period of time can be observed; yet, the proportion from CIN3 to cancer cannot be observed. Hence, a model is required to be fitted to data to estimate the parameter, which involves the imposition of a model structure. Thus, a parameter fitted for one model will not necessarily be applicable for another model. The issues of differences in parameters and structures are inextricably linked together. One approach that was discussed was to examine structurally similar models and the impact on results of assumed parameter values.

Vaccine Decision Models or Vaccine/Screening Decision Models

A central difference between models is the comparators of the decision analysis: incorporate both cervical cancer vaccination and screening assumptions or only one of the two. With regards to cervical cancer screening, some countries may have existing screening recommendations. Models can be used to assess the marginal costs and benefits of adding HPV vaccination to either (1) the recommended screening program or (2) the screening program as it exists in the real world. For the latter, empirical evidence of screening rates within different age groups can be used.

To determine assumptions regarding vaccine coverage, modelers rely mostly on country-specific experience for previously introduced vaccines. Models can utilize different scenarios of compliance to screening and vaccination recommendations. For example, some models evaluate the effectiveness and cost-effectiveness of screening and vaccination for the entire population, while others evaluate only the people who attend screenings at the recommended intervals. The incorporation of screening in compartmental, deterministic models is sometimes achieved by applying average rates of screening to the entire screened population. Such an approach can approximate real-world screening compliance, but nonetheless is imperfect because some females are never screened, while some have intermittent screenings.

Many models assume that screening has always been in effect and, therefore, static. As a result, the predicted incidence of cancer may not accurately reflect real-time data due to historical trends in screening uptake. In the US, for example, it is likely that cancer incidence today is higher among older females than it will be among future groups of older females, in part, because the oldest cohorts today were screened less when they were younger. The decision whether to take into account cohort effects in screening can affect predictions.

Case Examples

In the US, surveillance of the national screening program indicates that screening uptake is comparable for non-Hispanic blacks and non-Hispanic whites (13). However, the rate of invasive cervical cancer is two to three times higher in older non-Hispanic black females compared to similarly aged non-Hispanic white females. Given the similarities in screening rates across these two populations, the differences in cervical cancer incidence rates may be due to factors other than screening.

Japan is a good example of a country for which HPV models would benefit by taking into account the generational differences in screening uptake. Screening coverage runs from about 10% in younger females to 50%-60% in older females (14). As a result, mortality due to cervical cancer has increased in the younger generation, while the incidence of cervical cancer in the older generation has decreased.

For some countries, registry data can be used to provide information on screening behavior as well as vaccine coverage. For example, Australia has high-quality registry data, which provides comprehensive information on screening behavior and follow-up.

Model Transferability, Homogeneity, and Comparative Modeling

Observations were made that country-specific models have been applied to other countries (model transferability). This brings up several questions:

• Is simply updating the parameters for these countries sufficient?

• What parameters should be adjusted?

• Should the structure of the models also be adapted?

• If so, which structures: natural history and transmission, natural history of disease, vaccination, and/or screening?

Issues arise around comparative modeling. From one perspective, having a common structure for modeling the natural history of disease may be useful (model homogeneity). This could then be adapted for various screening and transmission interventions (i.e., tailored to the specific country). Conversely, a case can be made for the value of analyzing models with uncommon structures for modeling natural history (comparative model). Of note is the possibility that an effort to compel modelers to use the same parameters could have unintended consequences (e.g., selected parameter values could be inappropriate).

Although different models may use different epidemiological parameters, one advantageous exercise would be to build models fitted to a common set of data, and then subject each model to the same examination. An example of a common data set would be age-specific HPV and cancer incidence. Taking that a step further, identifying a set of common outcome measures to apply could help identify disparities between models. Since reporting cost-effectiveness ratios would add an additional layer of complexity due to differing costs and utilities between countries, outcomes would ideally include units, such as the population percentage needed to vaccinate to prevent one particular disease or the population percentage needed to vaccinate to obtain an organic quality.

Comparative modeling can be fruitful. A participant cited the Cancer Intervention and Surveillance Modeling Network (CISNET) as an example of a successful implementation of comparative modeling, as well as a 2004 colorectal cancer model from the Institute of Medicine. In an effort to assess comparative modeling, population-based inputs and several data sources were shared, while each investigator modeled natural history parameters independently. After several sessions analyzing the divergent data, CISNET successfully reconciled several of the discrepancies between models (15). The 2004 model was published and a white paper was produced (16). However, many of the parameters used in HPV models are highly dependent on model structure and, hence, difficult to apply to other models.

To further illustrate the need for comparative modeling, the effectiveness of screening also differs across countries. For example, in Mexico, a high proportion of women participate in routine Pap screening. However, the effectiveness of screening is poor and varies across states within Mexico (NR).

Addressing Challenges Through Model Transparency

More meetings of HPV modelers, like the MEHPV, would also be instrumental in advancing comparative modeling. Constructive disagreement and presentations could result in more robust models, rather than a “one-size-fits-all” structure. Variations can lead to more comprehensive analyses; rather than agreeing, it can be better understood how to leverage the strengths of particular models. Identifying which models are best-suited to address which research questions may be helpful. Some attendees agreed that in terms of a decision-making framework, transparency is necessary as well as more published information about models in terms of intermediate outputs—not just costs, discounts, or life years gained, but inputs such as incidents over time by types. Because policy-makers regularly make decisions based upon models, the associated model structure and parameters should be accessible and transparent for review and comparison—for both policy-makers and modelers.

The most practical vehicle or mechanism to offer that level of transparency would be a modeling website where modelers could run others’ models and see codes, parameters, equations, and assumptions. The discussion also included a suggestion of an online members-only blog for modelers to discuss issues and post supplemental material. A type of online forum for these types of exchanges and discussions was suggested. Several modelers felt reluctant about the exchange of models, and one modeler expressed the need for caution in releasing models without offering support on its usage. Models can be “abused” through misuse and, consequently, produce inaccurate results.

Modeling Transmission

Factors influencing infectivity (often captured by the transmission probability model parameters) are both physical and behavioral. Physical factors include barriers (i.e., condoms), types of sex acts, immunity and re-infection, HPV type-specificity, and gender specificity. Behavioral issues include types of relationships and sexual activity, as well as data accuracy of sexual behavior surveys (e.g., subjective responses, level of detail, and honesty). Refer to “Transmission Parameters” for more discussion.

Deterministic or Stochastic Models of Transmission

To model transmission, the number of partnerships may be stratified by gender, age, and sexual-activity groups. For example, a compartmental, deterministic model may estimate transmission probabilities by fitting a model to serological data or partnership and/or per-sex-act transmission probabilities. Some modelers have instead attempted to develop more detailed models of sex partner networks. However, introducing this level of complexity can be difficult to justify because the difficulty of determining the network of sex partnerships and parameterizing the model may overly complicate the interpretation of the results.

A crucial concern that models must address is the assumptions made about heterogeneity in risk behavior and what drives the epidemiology of the infection. More specifically, is the epidemiology dominated by high- or moderate-risk individuals? This will likely be determined by parameters like transmission probability and the duration of infectiousness. These assumptions can have enormous implications for the impact of a vaccine upon the epidemiology of the infection.

In response, a stochastic, individual-based model can capture more of the heterogeneities in screening behavior and provide insight into re-infections in partnerships. For example, individual-based models can more easily track who is regularly screened for cervical cancer, as well as those never screened. Moreover, it is more practical when modeling a large number of HPV types and coinfection. An individual-based model more easily captures progression of infection toward disease for different types of HPV and is able to take into account the competing risks towards cancer. Individual-based models can be more adept at incorporating core groups and hubs into a sexual data network and should be scale-free—providing a validity test confirming the network is modeled correctly. In a compartmental model, where HPV prevalence is ranked by level of sexual activity, it is difficult to fit the epidemiological data when there are more limited possibilities for the probability of transmission per partnership. With an individual-based model that incorporates partnership formation and dissolution, the resulting probabilities for partnerships vary according to the risk groups and the lengths of the partnerships. In addition, another facet of heterogeneity is the mixing behavior between people in different risk groups. For example, do high-risk men have sex with high-risk women? Close examination could aid in the determination of the cost-effectiveness of vaccinating males and how the vaccine programs are implemented.

In conclusion, the benefit of transparency and heterogeneity must be weighed in deciding between compartmental, deterministic models and stochastic, individual-based models. In model development, it is often ideal to start simple and then add complexity. Once the impact of the complexity becomes known, it is often useful to take that knowledge and work back to a simpler model. That is, additions to the model that increase model complexity without improving the model’s usefulness can be removed. However, some decisions in HPV modeling may be made because of the demands of policy-makers for health/economic recommendations; unfortunately, there may not be sufficient time or analytic resources to explore the impact of the different patterns of transmission heterogeneity.

The Need for Sexual Behavior Data

It is important to retain flexibility in the definition of sexual activity and the sites of HPV infection. Vaginal intercourse is certainly not the only form of sexual activity that can lead to HPV transmission. HPV infection of other sites can influence the ability of the vaccine to provide protection to the other mucosal sites. Data from several studies shows that anal HPV infection is more common than cervical HPV infection in certain populations of women, although the implications of this on disease are unclear.

Although modelers have attempted to incorporate national data on HPV epidemiology into models, the same cannot always be said for country-specific sexual behavior data and should be kept in mind when applying models. To achieve greater accuracy, models should use such data to reparameterize whenever possible (and when data sets are available). The accuracy and usefulness of models would benefit from having gender- and age-specific sexual behavior data to reproduce the expected disease prevalence. Currently, models are based on various assumptions related to mixing patterns. Having more insight on this model component would greatly improve models. (See “Uncertainty in HPV natural history” for discussion on MSM vaccination.)

The subject of having more detailed sexual network data for models was also discussed. While this “wish list” item could have great value, the practicality of getting these data may be highly unrealistic. Furthermore, adding this level of detail and complexity to the model might not change the model results. On the other hand, the scientific value of a better understanding of sexual behavior extends beyond the immediate practical questions (e.g., the benefit of male vaccination). In the longer term, more emphasis should be placed on understanding the transmission dynamics of infection. A research agenda should be devised to accomplish more than just answering the policy questions of the day.

The Question of Male Vaccination

Choosing between a “boys-only” or “girls-only” program has less value than identifying the incremental benefit of adding boys to a girls-only vaccination program at different levels of coverage. To inform policy-makers on how male vaccination affects female disease, country-specific dynamic models are needed. To illustrate this, in a country where there is 90% coverage of girls, vaccinating boys is unlikely to result in a large incremental gain. But in other countries, where there is relatively low (e.g., under 40%) vaccination coverage, vaccinating males in addition to females will likely have a substantial impact on HPV-associated disease in both male and females, which could potentially be cost-effective. Assessments of the impact of vaccinating boys could change if diseases other than cervical cancer (such as anal and oral cancers) are included.

It becomes necessary to understand the extent to which the boys are contributing to the spread of infection, as well as the mixing behavior between those who are at high risk and those who are at low risk for infection. Even if only girls are vaccinated, there will still be indirect benefit from herd immunity if vaccine coverage is sufficiently high (17).

Furthermore, the incidence of HPV is thought to increase rapidly after sexual debut in MSM; thus, if the vaccine is effective for males, the vaccine benefits to males might be maximized by vaccinating them prior to sexually activity. This correlates to another issue: cost effectiveness of vaccinating all males to prevent HPV-associated health outcomes in MSM (NR).

Natural History of HPV Infection

Disease Infection and Progression

HPV models must capture the natural history of HPV infection and the associated disease progression. Natural immunity must be taken into account as well. Many early models assumed that once an infection had cleared, susceptibility resumed; therefore, no natural immunity existed. Subsequent models took the opposite approach, assuming that once an infection had cleared, immunity was established for life. Ongoing models are exploring the assumption that once an infection clears immunity slowly wanes.

Disease progression involves addressing the disease processes that occur during infection and disease, how processes are detected and measured, how processes should be described, and the level of detail with which they should be described. The processes can be described in terms of virological, cytological, or histological changes (or combinations of these), based on the results of testing and screening. These changes are partly dependent on the effect that different descriptions will have on results of predictive models.

The topic of processes of infection and disease progression raises a number of questions to be considered:

• Should stages be represented by discrete states or by a continuous process?

• Do different stages of the disease share the same level of infectiousness?

• How is progression affected if someone is infected with multiple types of HPV?

• Does one HPV type overgrow another type or do both types progress in parallel?

For future modeling of disease processes, it is imperative to have access to accurate data sets. However, it was emphasized that input data are only as good as the tests that are chosen. For example, DNA screening is only as good as the number of HPV types in a given probe. In the last 20 years or so, the number of HPV types that can be tested has gradually increased and the list continues to grow. One noncommercial, laboratory-developed test found that 50%-60% of the HPV infections (mainly in the A3 subgroup) were not detected by commercial tests. This virtually doubles the numbers of people who are HPV-positive—although not all types are oncogenic. Also, many models merge different HPV types together, with potential effects on their predictions that are not completely understood.

Natural History Modeling Challenges

Generalizability and Specificity

Generalizability to other population subgroups is a central consideration in natural history. Model specificity is also imperative: models can apply to heterosexual men, heterosexual women, MSM or women who have sex with women (WSW). In terms of the natural history of infection in men, more questions arise:

• What is the natural history process in terms of lesion progression and duration of infection?

• Is there natural immunity and how should this be modeled?

Furthermore, the topic of immunity brings additional questions:

• Is there acquired aimmunity?

• Is there cross-immunity?

• How can this information be captured?

Natural history models may also differ from country to country. The issue of heterogeneity in the population (in terms of progression, regression, and exposure) is a consideration for modelers:

• How much of that heterogeneity is captured in the models?

• Is it worth capturing in the models?

• To what extent do interventions such as screening (cytological and virological); vaccination; as well as the promotion of condoms, microbicides and lower-risk behavior alter natural history processes?

Case Example: Natural History Consideration in the Netherlands

The natural history model developed in the Netherlands to inform policy-makers is a compartmental, deterministic model calibrated by using data from a clinical HPV screening trial in Amsterdam and sexual activity data from the Netherlands (NR). There is current interest in expanding the model to handle events such as lesion transformation, tumor maturation, and the effect of tumor suppressor genes. However, the data were not available; hence, a pragmatic approach was taken by focusing on epidemiological events.

Natural History, Transmission, and Behavior

Future work in modeling natural history needs to incorporate relationships between natural history, transmission, and behavior. The highest rate of transmission per partnership may occur among couples who engage in regular sexual activity with one partner having an undetectable, low-grade lesion. At the same time, the probability of transmission from male to female may be different from the probability of transmission from female to male. However, few studies have explored this possibility even though it potentially affects HPV vaccination model predictions. While cross-sectional data exist on the prevalence of low-grade cervical lesions in the female population at any given time, far less data exist on infection and disease outcomes in the male population for lack of active screening.

Human leukocyte antigen (HLA) factors, immunological factors, and cell-specific factors modulate the course of an HPV infection. Studies have also shown how behavioral factors (e.g., smoking) influence the duration of infection in females (18), yet quantifying it is not easily achieved. Modelers attempted to study the effect of smoking on persistence and progression (19) (20) in different circumstances (e.g. country, population), but no empirical data existed on the effect of smoking status on infection progression to disease. As a result, modelers used the prevalence of smoking at the time of infection and made assumptions about the effect of smoking. Because smoking behaviors differ from country to country (as seen in the declining smoking rates in developed countries compared to those in developing countries), differences may cause difficulties when calibrating to cervical cancer incidence because historical patterns will be reflected.

While cofactors may not necessarily affect the efficacy of a vaccine, they could affect the magnitude of indirect benefits of vaccination. For example, smokers might be at higher risk of having a persistent infection if they are not vaccinated. Likewise, female smokers may have a greater likelihood of choosing a male partner who smokes. In this scenario, there may be greater benefit from a model vaccinating a male smoker than a model with a homogeneous population. Understanding these factors in transmission may be important, even if they do not directly affect the efficacy of vaccination.

In conclusion, admittedly, there are a large range of less-than-ideal simplifications made in most models. It would be erroneous to state that it is essential to increase the complexity in all areas. It is more vital to determine which of these issues truly have a significant impact on results and which are superfluous. When issues are identified, efforts could be focused on conducting suitable epidemiological studies to obtain the data necessary to create more realistic models. Participants felt that close collaboration between modelers and epidemiologists, in applying for funding in natural history studies, is a vital part of this process. Overall, many modelers agreed that meaningful model development would be aided by a supporting epidemiological research agenda.

Modeling HPV-Related Disease

Prioritization

Attendees noted several HPV-related diseases that modelers might choose to include when estimating the costs and benefits of HPV vaccination:

• Cervical cancer

• Anal cancer

• Oropharyngeal cancer

• Penile cancer

• Ano-genital warts

• Vulvar cancer

• Vaginal cancer

• Non-melanoma skin cancer

• Recurrent respiratory papillomatosis (RRP)

While cervical cancer is generally categorized (from a burden of disease perspective) as the one of the most critical HPV-related disease in this list, ranking the other diseases was less obvious. A case was made to rank genital warts after cervical cancer, particularly from a decision-making perspective since its health outcome helps differentiate between the bivalent and quadrivalent vaccine. Vaccine efficacy data are also available against these endpoints. If mortality is a more important consideration than cost and morbidity, then genital warts would drop in ranking and cancers would rise.

It was mentioned that while RRP is rare, it is extraordinarily costly (the total cost burden of RRP in the United States may be comparable to that of genital warts)(21). Global estimates could be even higher, especially in locations where there are significantly increased risk factors. Participants felt that more research regarding the incidence and cost of RRP would improve estimates of the cost-effectiveness of HPV vaccination.

The prioritization of these conditions also depends on geographic area. For example, in sub-Sahara Africa, genital warts are not commonly diagnosed and treated (NR). Rates of HPV-related cancers such as oropharyngeal and penile cancer vary by country as well. Thus, the ranking of diseases to be included in an HPV model can therefore vary across countries.

Data Availability and Quality

Another important issue with modeling HPV-related diseases revolves around availability and quality of natural history data. More data have been published about cervical cancer than other HPV-related cancers, particularly regarding the relationship between HPV and disease progression. This is to be expected considering the policy relevance of models on the cost-effectiveness of cervical cancer screening. However, despite the relatively large amount of cervical cancer data, participants felt that more are still needed. For example, there are discrepancies in the values used in modeling the rate of CIN3 progression to invasive cancer.

Data on HPV-related, noncervical cancers are sparse and can be difficult to obtain. To illustrate, while oropharyngeal cancer is a source of HPV-related morbidity and mortality, precancerous predecessors are difficult to identify, resulting in the lack of empirical evidence on which to base natural history assumptions in models. As a result, some models have instead used “back of the envelope” estimates of the impact of HPV vaccination on these noncervical cancers.

Comorbidity

Comorbidity is a factor to consider when modeling HPV-related diseases—specifically, the question of whether diseases should be modeled as if an individual can have multiple diseases or modeled exclusively. For example, warts and cervical cancer could logically be modeled concurrently. While it might be ideal to model diseases together, participants felt that it might also be a challenging task. Dealing with the different type-specific natural histories and the interaction with the disease-specific treatments can complicate models. Even though modeling diseases separately may be easier, such models will not be able to factor in competing risks, such as death.

Quality of Life

Quality of life, in addition to mortality, is an important consideration in modeling HPV-related diseases given that they occur relatively early in adult life. For example, according to data from Surveillance, Epidemiology and End Results (SEER), deaths attributed to cervical cancer in the US result in an average life loss of 30 years—much higher than an average life loss of 8-10 years from breast cancer. Children with RRP can require surgical treatment several times a year for many years (21). This has a tremendous effect on families and can cause family-related social issues (such as dysfunction and divorce). Modelers often do not incorporate psychosocial outcomes among affected individuals (e.g., family members) when dealing with outcomes in terms of burden of disease.

One option is to use a composite outcome measure that takes into account all HPV-related diseases and weight them in relation to the impact of severity of disease or quality of life over time, producing a combined value such as a QALY. Competing sources of disease burden could be taken into account using either a multiplicative or additive approach. Many participants thought that cost-effectiveness studies of HPV vaccination should report (or make available) the cost per life-year saved as well as the cost per QALY gained.

In addition to difficulties in accounting for family burden, one attendee mentioned that problems also lie in the collection of quality-of-life data, often resulting in inconsistent application (applied to some models and not others). This stresses need for meta-analyses of the targeted literature to evaluate the use of QALYs. Potential shortcomings of QALYs led participants to emphasize the need to provide economic results in terms of clinical outcomes, such as cost per life-year saved, as well as cost per QALY gained.

General Discussion

Insufficient Availability of Parameter Estimates

One modeler provided an example of obtaining quality-of-life data for genital warts. The degree of disutility associated with genital warts was not well known and not thought likely to be a key parameter in the cost-effectiveness of HPV vaccination programs, so a clinic-based study to collect data was set up (NR). It was suggested that modelers may advise, as part of the modeling process, primary studies to collect currently unavailable utility information. The same can be suggested for unavailable parameters of epidemiological studies, although such studies are likely to involve a greater commitment in terms of time and expenses than utility studies. Poor quality assessment of utilities was reiterated. Even if you have a study that quantifies the utility, the issue of duration exists. Therefore, sensitivity analysis is even more important when quoting quality.

Many of the attendees thought that it is usually better to incorporate diseases into models—even when the quantity and quality of data available are not ideal—rather than leave them out entirely. The need for other types of intermediary outputs was expressed—clinical endpoints, infections, various precursor lesions, etc—so modelers could get a sense for how others calibrate models and how vaccines are impacted. It was stated that a source of frustration was that effectiveness is presented in natural and discounted units when they should be presented in life years and in QALYs. Similarly, although there are issues with QALYs and discounting, there are accepted guidelines for methodology that health economists follow that provide some consistency between models.

In Germany, guidelines for economic evaluations have recently changed, such that outcomes are no longer measured in terms of QALYs. Instead, a group of clinical outcomes is used within the same disease class, and the relative effects of different interventions are compared. Since Germany’s approach is new and different, there was a concern that comparison and model transferability to fit other countries may be difficult (NR).

Future Collaborations

Three potential means of collaboration were discussed:

• Create HPV modeling “best practices”

• Develop online forum to create transparency and share models

• Partner with modelers in developing countries to strengthen intellectual infrastructure

These three activities seemed more within communal purview for the immediate future than augmenting the paucity of available evidence.

The first group effort discussed is to establish HPV modeling “best practices.” The best practices guidelines will enable modelers to be more effective and deliver high-quality evidence. However, the purpose of the guidelines would not be to try to appraise modeling techniques. While general in nature, these guidelines would be specific enough to help inform future models in an efficient manner. An example of a specific HPV modeling guideline is to include a table of intermediate outcomes (such as cervical neoplasias) in the results of a model.

An overarching theme of the workshop was the importance of transparency. The establishment of an online forum could be a place to post existing models. This means of sharing existing models would allow for better understanding as to how different results are obtained.

To compare models, other than cost-effectiveness analyses, a set of general outputs could be identified. There could be value in standardizing various types of output (such as age-specific incidence of HPV infection and disease outcomes) in the absence of vaccination, as well as after the onset of vaccination. Creation of a set of metrics such as this may also help identify where other differences may exist.

For example, as a group, modelers could publish comparative results on parameters and intermediate disease endpoints, such as the age-, sex- and type-specific prevalence of HPV infection and transmission probabilities. While it is understood that there would be variations from country to country, the comparative results would still be informative, as would be the differences between models within the same country.

Furthermore, it would be helpful to interpret or summarize results in a way that is user-friendly for policy-makers who are sometimes unfamiliar with health economics concepts, such as QALYs and discounting. By focusing less on the technical aspects of the models and providing background in more easily understood terms, the results could more easily be used to inform decision-makers.

The final discussion dealt with the issue of resource-poor settings. Countries with ample resources often have multiple HPV models that address the impact and cost-effectiveness of HPV vaccination. These models are usually discussed in detail and in a scientific manner. However, a number of countries do not have access to relevant models. The same countries often lack the epidemiological data required to parameterize models, as well as the infrastructure to collect data. In some cases, these countries have to reconcile using models with simpler approaches. Consequently, it was suggested that, in the absence of more sophisticated models, these countries could parameterize a more generic model with country-specific data.

Given that the burden of disease is greatest in the developing world, there is an ethical and professional obligation to intellectually engage colleagues in a range of activities including modeling, to empower them to be part of the solution. To accomplish this, the effort would have to be supported by a sanctioned international body of experts. Some decision-makers in developing countries have suggested that the results of existing models are difficult to apply in a developing-country context because they present outcome measures that are less useful in resource-poor settings, such as QALYs.

It would be beneficial to encourage scientists in developing countries to learn modeling techniques to advise local policy-makers. One way to foster the model development process is to support training grants. A partnership with a developing country to produce an intellectual infrastructure around a particular topic could reap long-term benefits.

DAY 2

Pivotal Parameters and Evidence

Transmission Parameters

To date, information regarding the transmission of HPV is limited. As a result, modelers are forced to parameterize models based on assumptions or estimations by fitting models to epidemiological endpoints. One attendee noted difficulty in procuring funding to explore the transmission probability of HPV.

In one model, a pivotal parameter is the probability of transmission from an infected individual to a susceptible individual, which drives HPV prevalence and cervical cancer incidence rates. While there are various ways to estimate this probability, one group used results from serologically tested blood samples taken from pregnant women who had at least one previous pregnancy and more than one sample taken between pregnancies. Doing so enabled the modeling of women who were initially sero-negative and later sero-positive. In addition, the sexual behavior data collected at more than one time point allowed for the calculation of the per-partnership transmission probability.

Other models have applied the per-sex act (rather than per-partnership) HPV transmission probability, which could be a more useful measure in some cases. Some modelers found that insight gained from their models suggested that future models might be best served by deriving estimates from empirical evidence.

Screening Parameters

Participants observed that no two papers used the same estimates for screening sensitivity and specificity determination. Studies were based on different screening algorithms, such as performing biopsies on all women with a positive cytological result; performing colposcopies on all women with positive results; performing biopsies only on women with abnormal colposcopy; performing an HPV test on all women and performing biopsies on those with positive results. Results from a Swiss study comparing HPV testing and liquid-based cytology to detect cervical cancer precursors in 13,842 women revealed that the probability of a Pap test reading abnormal is directly proportional to HPV viral load (22). It would thus appear that studies that conduct biopsies only on a subset of individuals may overestimate the sensitivity of the Pap test (or any other test).

HPV-Related Natural History Parameters

Current vaccines provide protection against vulvar, vaginal, and cervical lesions (23). Since the vaccines appear to protect against persistent HPV infection and HPV-related endpoints, it is highly plausible that they could also prevent anal lesions; however, no studies exist currently. While there are natural history data on the cervix, literature reviews show that little or none exist for vulvar, vaginal, and anal lesions for females, much less related sites for males (24).

It may not be appropriate to apply natural history and transmission rates from one site (the cervix) to other sites. Caution was advised when modeling the progression of disease at different anatomical sites in the different sexes. For example, the incidence of oropharyngeal cancer in females is five times lower than in males.

Age-Specific Parameters

Another parameter issue is the use of age-structured progression and regression rates. According to one participant, a literature review showed insufficient evidence to support the assumption that rates differ by age; however, many models follow the assumption (25). The impact can vary. If there are shifts in the average age of infection due to HPV vaccination, the use of age-specific (vs. age-constant) progression rates could have a major impact on modeling results, especially with regards to the impact of waning vaccine protection.

Country-Specific Parameters

Country- and region-specific data can affect the results of parameterization of progression and regression rates in models. An example was discussed regarding the parameterization of a model using Australian data (NR). To match observed rates of CIN 1-3 in the population, modelers had to apply upper-bound estimates of progression rates from the literature. The application of higher progression rates was the only option that could be taken to fit the data on high-grade lesions in younger females. The resulting model suggested that HPV vaccination in Australia was more cost-effective than previously thought; however, the burden of disease in younger females was not captured prior to parameterizing the model to data.

Likewise, a disparity in screening compliance patterns exists between countries. The UK has an organized “call and recall” system, with disincentives for early rescreening, which results in excellent 3-year screening coverage—the recommended screening interval (NR). Screening incidence by age behaves more like a step function, so in the UK scenario, a simpler model of screening compliance may be appropriate. But in Australia, there is underscreening at the 5- and 10-year level, which means the simple model is not as appropriate (NR).

In the US, the most nationally representative data are self-reported, which has been shown to overestimate the recency and frequency of screening (26). Evidence of actual screening utilization can be derived from claims data and related information. This is not nationally representative because it is collected from a sample in which privately insured individuals are overrepresented. One other important consideration is to separate routine screening from follow-up smears, cervical smears, and post-hysterectomy vaginal smears in cytopathological data. Furthermore, a compartmental model can be extended to incorporate a variable for females who are never screened during their lifetimes. While it is more difficult to consider intermittent screening intervals in a compartmental model, it is often easier to separate out the group that has never been screened.

Economic Parameters

Modelers have found pivotal economic parameters to include costs and discount rates. Finding estimates of direct medical costs of all HPV-related diseases can be difficult. The literature for indirect costs associated with follow-up cancer screening is limited. The discount rate (the rate at which future costs and benefits are “discounted” to present value) can vary from country to country. One attendee noted that in the Netherlands it is recommended that cost-effectiveness studies use a 1.5% discount rate for health outcomes and a 4% discount rate for costs (NR). This differs from most other countries, making it difficult to compare model results from the Netherlands to the cost-effectiveness results from elsewhere.

Attendees noted that it would be beneficial if journal articles and presentations to decision-makers include more information than cost-effectiveness ratios. It may be useful if modelers also present the estimated impact of vaccination on health (such as QALYs or life years gained, as well as outcomes averted) and costs over time, both in discounted and undiscounted terms.

Parameterization vs. Calibration

Instead of searching for primary data to inform transition parameters (direct parameterization), an alternative approach is to find parameters that make models fit outcomes, such as HPV prevalence and disease incidence (calibration). One group that used this approach concluded that it was difficult to fit the HPV prevalence with a single probability of transmission per partnership.

Sensitivity Analyses and Calibration

HPV modeling is complex and involves numerous parameters, complicating sensitivity analyses and calibration (SAC). Infection can precede associated disease by years; therefore, few prospective data exist on the progression of the disease to a cancer endpoint. Because parameter estimation in models largely depends on setting and data sources, it is difficult to compare models built for different countries due to differences in screening and testing procedures. Furthermore, there is a general lack of data on type-specific cancer incidence. As a result, modelers often find it difficult to know what data to use.

Three SAC Approaches

To better address these difficulties, a variety of SAC approaches have been introduced to HPV modeling. One new approach is to shift from manual calibration (validation by visual inspection) and one-way sensitivity analysis to multi-parameter calibration models, which can use thousands of different parameter sets (also known as a parameter grid). Modelers start with a wide range of possible parameter sets and then choose a select number of sets (or grid points) that fit certain disease endpoints.

Instead of fixed parameter sets, a second approach is to use a decision algorithm as a way of estimating the optimal parameters. Modelers use the starting point of examining various possibilities of what the data might reveal (i.e., various possibilities of interpreting screening data compared with various possibilities of a model structure) and proceed from there.

A third approach is to search locally within a parameter space using the Markov chain Monte Carlo (MCMC) method. Instead of examining an array of fixed parameter sets, this approach samples from probability distributions based on constructing a Markov chain (all necessary information about the future exists in the present and there is no need to examine the past). Multiple chains are used for a probabilistic sensitivity analysis, with attention to convergence in distribution across different chains. The state of the chain after a large number of steps is then used as a sample from the desired distribution.

The science of searching parameter space and fitting to models is one that is developing quickly. While this can be a thorough approach, problems with convergence evolve as the models get more complex.

Regardless of approach, results can be overinterpreted because of data source features that are used, each of which is subject to different levels of uncertainty and bias. Careful consideration may need to be given as to how modelers weigh the varied fits to the varied data sources. It may also be important to consider model structures and how parameters are determined by the structure of the model in which they are fitted. Inferring parameter values from a different model structure may not give the same set of answers.

Lack of Evidence on Uncertainty

As discussed on Day 1, handling uncertainty is a key issue within the models. For example, sensitivity analyses might focus on distributions of parameters such as cost and utilities of health outcomes, but ignore the uncertainty in the natural history parameters. The resulting probabilistic sensitivity analysis—though consisting of a large number of samples and presented in terms of cost-effectiveness acceptability curves or points in the cost-effectiveness plane—may underestimate the degree of uncertainty in the results by ignoring uncertainty in model structure and taking into account only the uncertainty in the economic parameters.

A similar point can be made with regards to the development of quality of life weights for HPV-associated health outcomes. The full degree of uncertainty associated with the quality of life weights will not be reflected in the sensitivity analyses if point estimates (rather than ranges) are applied or if the ranges applied exclude plausible lower- or upper-bound values.

There are few systematic literature reviews to inform the selection of parameter values for models of HPV. While there have been a few meta-analyses, many of them pre-date current modeling work by several years. Thus, for models that have a base case parameter set, it is often unclear which base case data sets were selected for models and how they were aggregated to obtain a particular point estimate. The same is true for the multi-parameter probabilistic sensitivity analysis approach, since this requires distributions to be imposed on the parameters. It may be helpful to understand how the distributions were derived and what criteria were used for selecting relevant studies that determined the distributions of values.

Presentation of Outcomes

Several participants expressed concerns about the presentation of outcomes. Distributions should be interpreted cautiously, because peculiar assumptions are often made to fit models. Point estimates tend to provide a binary outcome to decision-makers (e.g., cost-effective or not cost-effective) when uncertainties surrounding models and associated data are unclear. To truly and accurately inform policy-makers, modelers may need to report results in simpler, more understandable terms. The information needs to be “translated.” A better approach to a binary outcome may be to provide some measure of the degree of certainty that an intervention is cost-effective given various cost-effectiveness thresholds (such as the number of samples from distributions that fall under the threshold) or to show the most plausible range of cost-effectiveness estimates.

Lack of SAC Transparency Leads to Lack of Standardization

Participants voiced concerns about the lack of clarity regarding the applied SAC methods. It is a challenge to formulate a standard approach to SAC because modelers are still trying to understand how to interpret the available data and calibrate models to them. Most models are calibrated to the primary endpoint of cervical cancer incidence, which brought up the issue of whether this was sufficient. A more reasonable goal may be for modelers to be more transparent with the methods used to calibrate models and even provide likelihood values for the best fitting functions. If it cannot be deduced what modelers have done and how well the models fit, then the results are problematic to interpret.

The issue of transparency is a common thread that ran through topics discussed in the workshop. Several participants believe many HPV models are not presented in sufficient detail to be completely transparent and reproducible. However, many modelers voiced concerns. There was apprehension that the first group to submit an entirely transparent model for peer review would likely face exhaustive scrutiny—and perhaps destructive, rather than constructive, criticism—because aspects of the model are delineated. Furthermore, there is the issue of to whom the models should be transparent. For instance, it may be troublesome to make the methodology of complex models transparent to end-users (i.e., policy-makers).

Open Discussion

Participants discussed charting a path forward. Three topics emerged as the most pressing to address: best practices in HPV modeling, comparative modeling, and modeling in developing countries.

Call for Action

Best Practices in HPV Modeling

Initiating a project to create and publish HPV Modeling Best Practices (refer to “General Discussion” at end of Day 1 for more detail) is of the utmost importance. Tentative suggestions for best practice guidelines included:

• Show how models were fitted to data

• Present model results in both natural and discounted units, in both life years saved and QALYs gained

• Include health outcomes and costs associated with each vaccine strategy, not just cost-effectiveness ratios

• Justify choice of which diseases are used in the model (attendees expressed a preference for including all relevant diseases when feasible, at least in the sensitivity analyses)

• Include intermediate outputs and a variety of clinical endpoints

• Include mixing assumptions and data used to inform the mixing patterns in transmission models

• List parameter sources, and, if the parameters are based in calibration, how calibrations were conducted

Comparative Modeling

A comparative modeling project would offer valuable observations that can be useful to other modelers by sharing information, including data and algorithms, structural issues, and methodological and epidemiological insights generated by model construction (i.e., transparency).

Distributing knowledge greatly assists the refinement of models. However, the means of accomplishing a platform for comparative modeling is not clear. One suggestion was to create a form on which to list the parameters used. Another was to evaluate intermediate inputs. Yet another was for groups with related models to produce papers—natural history parameters or intermediate outputs could be an example for other groups to follow.

Due to the content and space limitations of journals/papers, important depth of detail is sacrificed; hence, a complete and transparent “appendix 1” would help others improve models. Therefore, a suggestion was made for authors to create an adjunct document (such as an appendix 1 that contains modeling details) that would be published as a volume. This volume would contain a collection of the adjunct documents (either as a stand-alone or along with the associated article). Contents could include:

• Detailed descriptions of parameter sets (e.g., equations and sexual behavior)

• Age-specific prevalence or incidence of different disease stages within models and other intermediate outcomes

• Country-specific costs and registry data

• Calibration metrics

• Postvaccination dynamics (such as time-series)

• Detailed model structure (with basic flow diagrams)

• Approaches to screening algorithms

The following questions (and possible answers) arose around the concept of appendix guidelines:

• Will modelers follow the guidelines?

  • ○
    If modelers spend significant amounts of time with appendices, would time be better spent working on and improving models?
• Will appendix guidelines be updated on an annual/regular basis?

  • ○
    As new models are developed and new approaches used, the guidelines need to change.
• Should appendices be created for articles retroactively?

  • ○
    Much of the existing literature is no longer applicable to current issues, so it may have limited value.
  • ○
    More recently published articles and those soon-to-be published are more likely candidates for appendices.
• Should appendices be submitted for print publication along with the paper?

  • ○
    Due to space limitations, it is unlikely that appendices will appear in print, although some journals may publish them online.
  • ○
    If an attempt is made to get one published in a reasonably high profile journal, it would be a “message” to editors that such detail is an integral part of a model description.
• Are any models currently being worked on and/or near completion that would be suitable for appendices?

  • ○
    As many as seven groups have papers that are or will be “appendix-ready.”
• Should the appendix guidelines only be for HPV models?

  • ○
    To come to a consensus, the guidelines should be targeted at HPV models.
  • ○
    Other modelers, such as those working with colorectal, lung, prostate and breast cancers, may find the guidelines helpful but should consider creating similar groups and compose applicable guidelines.
• When should the appendix guidelines be ready for distribution?

  • ○
    A peer-reviewable draft could come together quickly.
  • ○
    It could take significantly longer for groups to compare what others are producing and evaluate them to produce a “final” set of guidelines.
  • ○
    At the 2010 group meeting, modelers could review appendix material, making updates as necessary.

Modeling in Developing Countries

Many group members expressed interest in HPV modeling within developing countries and the means of supporting this work. It was suggested that it would be beneficial to have a basic model available to developing countries: straightforward with a robust and simple interface. While this would require funding for model development and interface design, new web technologies could make dissemination feasible.

A subset of modelers from this workshop agreed to form an international group to discuss and design a useable tool with a user-friendly interface that could be internet accessible. Ideally, this “developing countries” group should also include modelers unable to attend the workshop, as well as relevant collaborators from developing countries.

APPENDIX

PARTICIPANTS

Modeling Evidence in HPV Pre-Conference Workshop Malmo, Sweden, May 9-10, 2009

Scientific Committee:

• Marc Brisson

• Harrell Chesson

• Benjamin M. Craig

• Anna R. Giuliano

• Mark Jit

Participants:

• Kari Auranen

• Ruanne Barnabas

• Johannes Berkhof

• Marie-Claude Boily

• Thomas Broker

• Karen Canfell

• Yoon Hong Choi

• Lynne Conway

• Veerle Coupe

• Erik Dasbach

• Inge De Kok

• Donatus Ekwueme

• Elamin Elbasha

• Geoffrey Garnett

• Ralph Insinga

• Michelle Kohli

• Ryo Konno

• Denise Kruzikas

• Shalini Kulasingam

• Nathalie Largeron

• Angela Mariotto

• Pontus Naucler

• Joel Palefsky

• David Philp

• Lucia Pirisi-Creek

• Heini Salo

• Wayne Thompson

• Nicolas Van de Velde

• Simopekka Vanska

• Cathal Walsh

MODELING EVIDENCE IN HPV (MEHPV) PRE-CONFERENCE WORKSHOP

AGENDA

Saturday, May 9th, 2009

Day 1, Session 1: Opening Introductions

This section gave attendees an opportunity to introduce themselves, their work, and their affiliations.

Day 1, Session 2: Objectives of Modeling Research

Led by Marc Brisson, this session focused on current and future issues that HPV models need to address.

Day 1, Session 3: Review of Models and Associated Objectives

Led by Mark Jit, this session reviewed current models and provided an opportunity for the group to discuss similarities and differences.

Day 1, Session 4: Modeling Transmission

Led by Marie-Claude Boily, this session provided the group with a forum to discuss their views and share their experiences of modeling the transmission process of HPV.

Day 1, Session 5: Natural History of HPV Infection

Led by Geoff Garnett, this session dealt with modeling issues related to the natural history of HPV infection.

Day 1, Session 6: Modeling HPV-Related Disease

Led by Dr. Craig, this session gave attendees the opportunity to list and rank HPV diseases, as well as discuss the concept of comorbidities and the issue of incorporating quality measures.

Day 1, Session 7: General Discussion

This session was a general review of the seven topics discussed on Day 1 of the workshop.

Sunday, May 10th, 2009

Day 2, Session 1: Pivotal Parameters and Evidence

Led by Shalini Kulasingam, this session focused on data, screening, vaccination, cost and quality of life issues.

Day 2, Session 2: Sensitivity Analyses and Calibration

Led by Hans Berkhof, this session focused on the importance of data setting and source, the difficulties of country-to-country comparison, lack of data, and multi-parameter calibration models.

Day 2, Session 3: Open Discussion

This session was a general review of Day 1 and 2 discussions.