The OncoSim model: development and use for better decision-making in Canadian cancer control

Abstract

The Canadian Partnership Against Cancer was created in 2007 by the federal government to accelerate cancer control across Canada. Its OncoSim microsimulation model platform, which consists of a suite of specific cancer models, was conceived as a tool to augment conventional resources for population-level policy- and decision-making. The Canadian Partnership Against Cancer manages the OncoSim program, with funding from Health Canada and model development by Statistics Canada.

Microsimulation modelling allows for the detailed capture of population heterogeneity and health and demographic history over time. Extensive data from multiple Canadian sources were used as inputs or to validate the model. OncoSim has been validated through expert consultation; assessments of face validity, internal validity, and external validity; and model fit against observed data. The platform comprises three in-depth cancer models (lung, colorectal, cervical), with another in-depth model (breast) and a generalized model (25 cancers) being in development. Unique among models of its class, OncoSim is available online for public sector use free of charge. Users can customize input values and output display, and extensive user support is provided.

OncoSim has been used to support decision-making at the national and jurisdictional levels. Although simulation studies are generally not included in hierarchies of evidence, they are integral to informing cancer control policy when clinical studies are not feasible. OncoSim can evaluate complex intervention scenarios for multiple cancers.

Canadian decision-makers thus have a powerful tool to assess the costs, benefits, cost-effectiveness, and budgetary effects of cancer control interventions when faced with difficult choices for improvements in population health and resource allocation.

INTRODUCTION

Against the backdrop of an aging population, new technologies and drugs, and constrained health care budgets, the Canadian Partnership Against Cancer (hereinafter, the Partnership) was created in 2007 with a mandate from the federal government to accelerate cancer control across Canada. From the outset, a key strategy for the Partnership was to mobilize the best evidence and deliver it into the hands of policy- and decision-makers. The OncoSim microsimulation model platform^a, with its suite of specific cancer models, was conceived as a tool to augment conventional information resources for population-level Canadian decision-making, within the reality of thirteen jurisdictional health care systems. With funding from Health Canada, OncoSim is led and supported by the Partnership, with model development by Statistics Canada, to help answer complex questions facing Canada’s cancer control system.

Here, we describe the rationale, development, and application of the OncoSim platform, providing the cancer control community with foundational knowledge about a Canada-specific analytical tool for population-level decision-making.

METHODS

Objectives of the OncoSim program

The objectives of the OncoSim program are fourfold:

• ■To develop an accessible, comprehensive, Web-based platform that projects the future burden of cancer and its economic effects
• ■To provide an ability to simulate the effects of past, current, and future cancer control interventions on both the population and the economy
• ■To inform resource allocation in cancer control
• ■To maintain transparency in the development and use of OncoSim

OncoSim Specification and Management

The Canadian Partnership Against Cancer initiated the OncoSim program through a request-for-proposals process. Applicants were not limited to Canada, but the process required that the models be based on Canadian data for disease burden, treatment patterns, outcomes, and costs. Furthermore, the models were intended for a multiuser environment in which users could explore scenarios and not be limited to pre-specified options. Models were to be able to plausibly project outcomes, resource use, and costs for simulated scenarios. Applicants were also asked to facilitate access for general users. Statistics Canada was the successful applicant, and their statisticians, mathematicians, economists, and programmers continue to serve on the team of model developers.

The Partnership’s health economics team manages and promotes OncoSim and provides access to the model for internal and external users. Technical advisory panels composed of external specialists and Statistics Canada and Partnership staff provide strategic advice and clinical, epidemiologic, computing, and economics expertise for model development, update, and progress review. An oversight committee provides direction to the program, with its membership consisting of a pan-Canadian group of cancer control specialists, executive Partnership staff, and program staff. These management and oversight structures are accountable to Health Canada through its funding of the Partnership.

OncoSim Approach

The microsimulation modelling approach allows for the simulation of cases at the level of individuals, and captures the heterogeneity of the population’s health and demographic history over time. Some facets of the model allow for interactions between “individuals” that can affect cancer outcomes, such as infectious disease transmission and hereditary genetic profiles. A high degree of flexibility is available in the construction and use of the components.

The OncoSim suite of models is built on a common framework (Figure 1), with users being given the capability to control input parameters. Examples include human papillomavirus (hpv) vaccination; risk factors such as smoking and use of hormone replacement therapy; screening characteristics such as modalities and associated costs, test sensitivities and specificities, participant eligibility, and follow-up protocols; and treatment characteristics such as procedure types, probabilities, and costs. Users also have a limited ability to modify the natural history of a cancer, duration of health states, and underlying demographic and economic characteristics. A specific alternative population cohort can be defined by the user. Similarly, outputs can be customized to stratify results, present data in tabular or graphic form, and export results for further customization or analysis. Outputs include incidence, mortality, resource utilization (for example, follow-up procedures, treatments), direct health care costs, life expectancy, quality-adjusted life-years, incremental cost-effectiveness ratios, and macroeconomic factors (for example, employment, taxes, labour productivity), most of which can be stratified by age, sex, year, and province or territory. Details of the OncoSim approach and its user functions have been described in previous publications^1–3.

FIGURE 1.

Conceptual diagram of the OncoSim platform.

Data Sources

Inputs to the model, as well as the targets that the model must replicate, were obtained from multiple Canadian sources: data for cancer-specific rates of incidence (Canadian Cancer Registry); cancer survival (multiple sources, including provincial sources); cancer and other-cause mortality (Statistics Canada); past and projected demography (Statistics Canada); costs of procedures (multiple provincial sources); costs of cancer management (multiple provincial sources); patterns of practice [Institute for Clinical Evaluative Sciences, provincial administrative databases, data from randomized controlled clinical trials (rcts)]; health behaviours (Canadian Community Health Survey, National Population Health Survey); screening performance (pan-Canadian screening networks); measures of health-related quality of life (Statistics Canada, publications using Canadian sources); and environmental radon data (Health Canada). Certain parameter estimates derived from the published literature are adjusted to provide projections consistent with observed Canadian data.

Validation

Rigorous assessments are performed to validate OncoSim.

• ■External consultation and evaluation

  Experts not involved in model development are engaged in assessing components or the model in whole—for example, pan-Canadian expert reviews of treatment pathways and case-study evaluations. Those assessments ensure conformity with clinical and modelling best practice and knowledge, with the case-study evaluations generating parameter sensitivity analyses from independently specified scenarios.

• ■Face validity

  Simulated individual life trajectories are inspected for plausibility, and extreme scenario analyses are conducted.

• ■Internal validity

  Model inputs (for example, incidence rates) are determined to be consistent with outputs (for example, incident cases), and model construction logic is verified by both internal and external experts.

• ■External validity

  Outputs are compared against data not used to build the model—for example, observed data, rcts, and results from other comparable models such as the U.S. Cancer Intervention and Surveillance Modeling Network (cisnet) models.

• ■Calibration

  Each model is required to demonstrate acceptable close fit to targets derived from Canadian sources (for example, incidence and staging in the Canadian Cancer Registry) and published sources. Typically, several thousand estimations are made to meet hundreds of targeted outcomes in individual subcomponents such as screening, natural history, and cancer incidence and mortality, and then again to meet key prioritized targets in whole-model calibration.

Additionally, peer-reviewed publication of model results, including comparisons with other modelling results, is encouraged from both internal and external users^2–10. As newer data emerge, the model is used to compare past projections to current real-world data, thus continually assessing its ability to accurately project future events. Recalibration is undertaken as new inputs or new functionalities are added.

RESULTS

The OncoSim platform currently comprises in-depth models for lung, colorectal, and cervical^b cancer. An in-depth breast cancer model and a generalized cancers model (the 25 most common cancers) are under development.

Unique among models of its class, OncoSim is available for public sector use without charge. It is accessible online, freeing users from purchase and maintenance of computing resources. Extensive user support, tailored to the technical expertise of the user, is provided by the Partnership and by Statistics Canada. Users can run their own simulations or request that the program team do the work on their behalf. Default settings reflecting the Canadian status quo are available, and users can customize those settings for their own jurisdiction, subpopulations, or particular intervention application. Users have access to full documentation about the internal model structures, assumptions, and parameter descriptions.

OncoSim has been used to support decisions at the national and jurisdictional level. For example, after an independent evaluation, the Canadian Task Force on Preventive Health Care used OncoSim modelling to support its 2016 guidelines for colorectal and lung cancer screening^11,12. Various provincial health ministries and cancer agencies are using OncoSim to study the effects of hpv testing in cervical cancer screening, to develop a business case for introducing lung cancer screening, and to explore potential levers to reduce wait times in colorectal cancer screening. OncoSim has also supported the Canadian Cancer Society in its annual statistics reports by making projections for special-topics chapters^13,14. Similarly, the Partnership’s signature System Performance reports have relied on OncoSim analyses to examine future-state scenarios for key indicators of cancer control¹⁵. OncoSim is additionally playing a significant role in a Genome Canada–funded research project that is exploring the potential use of genetic information to tailor breast cancer screening in women primarily to breast cancer risk rather than to age. Table i presents other examples of OncoSim applications.

TABLE I.

Use of OncoSim to support policy- and decision-making

Partner	Type of work	Use of OncoSim
Canadian Task Force on Preventive Health Care; Public Health Agency of Canada; Canadian Agency for Drugs and Technologies in Health	■ Analysis supporting development of clinical guidelines for colorectal and lung cancer screening	Publications to support policy decision-makinga,b
Canadian Cancer Society	■ Cancer projections highlighting the future burden of cancer in Canada	Publications to support policy decision-makingc,d
Institute of Health Economics	■ Economic analysis of computed tomography screening for lung cancer in Alberta	Informed the economic analysis to support a policye
Canadian Partnership Against Cancer: System Performance	■ Analysis showing the future impact of selected cancer control interventions	Publication to support policy decision-makingf,g
Various research institutions	■ Case study evaluations: Partners were funded to use the OncoSim platform to evaluate relevant cancer policies of current interest related to interventions in human papillomavirus and cervical cancer ■International knowledge exchange: cancer control experts participated in a knowledge transfer and exchange event with the OncoSim team to learn about the applications of OncoSim and its application to the Canadian Strategy for Cancer Control ■Evaluation of the cost-effectiveness of lung cancer screening and assessment of the costs of lung cancer care using high-quality health administrative data ■Exploration of the potential use of genetic information to tailor breast screening in women to breast cancer risk	Research; consultation and education
Screening networks:   Canadian Breast Cancer Screening Network, Pan-Canadian Cervical Cancer Screening Network, National Colorectal Cancer Screening Network, Pan-Canadian Lung Cancer Screening Network	■ Projections for various strategies in lung, cervical, and colorectal cancer screening	Support policy decisions; publications to support policy decision-making
Various cancer agencies and ministries of health	■ Evaluate and monitor the downstream effects on colposcopy utilization of the human papillomavirus test as a proof of cure ■Assess primary human papillomavirus DNA testing compared with Pap testing ■Inform human papillomavirus vaccination strategies and cervical cancer screening programs ■Assess the feasibility of lung cancer screening ■Assess the effects of tobacco on the burden of lung cancer ■Understand levers to reduce wait times for colorectal cancer screening ■In-depth training and support was provided to assess the feasibility of using OncoSim to support policy decisions for colorectal cancer screening programs	Support policy decision-making; consultation

^aBacchus et al.,201611. See also https://canadiantaskforce.ca/guidelines/published-guidelines/colorectal-cancer/.

^bLewin et al.,201612. See also https://canadiantaskforce.ca/guidelines/published-guidelines/lung-cancer/.

^cCanadian Cancer Society’s Advisory Committee on Cancer Statistics, 2015¹⁴.

^dCanadian Cancer Society’s Advisory Committee on Cancer Statistics, 2016¹³.

^eInstitute of Health Economics, 2014¹⁶.

^fCanadian Partnership Against Cancer, 2016¹⁷.

^gCanadian Partnership Against Cancer, 2017¹⁵.

DISCUSSION

OncoSim was developed to aid evidence-based policy- and decision-making in cancer control in Canada. Published hierarchies of medical evidence usually recognize systematic reviews, principally based on rcts, as the highest level of evidence, and continue down through decreasing levels of evidence^18,19. Simulation studies have not traditionally been included in such hierarchies. Nevertheless, simulation studies in which intervention effects are estimated for populations are integral in informing cancer control policy when clinical studies are not feasible or desirable²⁰.

In clinical studies, only a small, finite number of interventions can typically be compared at one time, even though, in practice, multiple options might have to be considered and compared. For example, cervical cancer screening can be implemented using the Pap test or hpv test, both with varying follow-up protocols, and with or without prior hpv vaccination. Furthermore, the eligible age ranges, frequencies of screening, vaccine choice, and choice to vaccinate or not vaccinate boys as well as girls are variables that lead to multiple scenarios. Simulation studies are a feasible approach to evaluate the full range of potential choices. Likewise, outcomes of interest for real-world decision-making such as cost-effectiveness are not provided by most clinical cancer studies, given that extended time horizons are not possible. Simulation studies allow for the projection of costs and effects into the future and thus can take into account important trends such as population aging and declining smoking rates.

OncoSim has several limitations.

First, OncoSim is a complex set of models that might appear to be a “black box.” To shed light, all inputs are viewable with the published model and are accompanied by a data dictionary and supporting documentation (visit http://www.oncosim.ca).

Second, OncoSim has the capacity to incorporate and project the effects of parameter uncertainty, but because of the computational demands involved in handling the model’s more than 100,000 inputs, it does not yet easily or fully support probabilistic analysis, as recommended by guidelines^21–23. However, assessment and implementation steps are being taken to enable probabilistic analysis.

Third, the models require continual updates to refresh their data inputs. Some components of OncoSim were initially developed using 2008 data and, in some cases, relied on expert opinion. Those components are currently being updated as access to more recent administrative data has become available. An update cycle is ongoing for all components of the model.

Fourth, external validation is a challenging but critical contributor to the plausibility of model projections. The colorectal cancer screening module has recently undergone an extensive validation exercise that replicates four rcts (submitted for publication), and work has begun to evaluate cervical cancer screening against rcts. To ensure the external validity of lung cancer low-dose computed tomography screening, OncoSim results were compared with those from the U.S. National Lung Screening Trial (nlst) and the relevant cisnet models²⁴. Using simulated trial characteristics and a cohort matched to the nlst, outcomes similar to those reported from the trial were achieved³. Compared with the averaged results of the five cisnet models for scenarios created in an identical manner, OncoSim produced higher estimates of overdiagnosis and lower estimates of mortality reduction (Table 5 in Flanagan et al.³). Those discrepancies might be attributable to the demographic and smoking prevalence differences between Canada and the United States as well as to the underlying model assumptions, but further investigation is needed. In a comparison with the nlst primary outcome of mortality reduction, OncoSim results fell well within the range of those from the individual cisnet models, with all model results converging to the nlst results by the trial median follow-up of 6 years (Figure 2).

FIGURE 2.

Ratio of cumulative number of lung cancer deaths in control arm to screen arm: comparison of OncoSim to CISNET (U.S. Cancer Intervention and Surveillance Modeling Network) and NLST (U.S. National Lunch Screening Trial) results. Source: OncoSim version 2.1 and Appendix Figure 2 in de Koning et al.²⁴

Finally, OncoSim requires extensive and comprehensive data that are sometimes not available—perhaps because of non-collection or protection under privacy legislation. Consequently, although data are gathered from the best available sources, they might not be fully representative across all jurisdictions. Nevertheless, OncoSim can be tailored to inform decision-making within relevant contexts using local costs or patterns of practice. Model inputs, projections, and validation—and consequently decision-making—could be enhanced by innovative methods to leverage existing data holdings through data linkage and reduced barriers to accessing data.

SUMMARY

OncoSim, available online and free of charge for public sector use, is the only model of its kind in Canada and is devoted to policy-pertinent pan-Canadian decision-making. Built on an extensive set of epidemiologic and economic data and rigorously validated, OncoSim can evaluate complex cancer control scenarios for multiple cancers. Policy- and decision-makers thus have a powerful tool to assess the costs, benefits, and cost-effectiveness of cancer control interventions when faced with difficult choices for improvements in population health and resource allocation.

CONFLICT OF INTEREST DISCLOSURES

We have read and understood Current Oncology’s policy on disclosing conflicts of interest, and we declare that we have none.