Skip to main content
PMC home page
Journal of the American Medical Informatics Association : JAMIA logoLink to Journal of the American Medical Informatics Association : JAMIA
. 2020 May 17;27(6):908–916. doi: 10.1093/jamia/ocaa022

Simulation modeling validity and utility in colorectal cancer screening delivery: A systematic review

Heather Smith o1,o2,✉, Peyman Varshoei o1, Robin Boushey o2, Craig Kuziemsky o3
PMCID: PMC7309251  PMID: 32417894

Abstract

Objective

This study sought to assess the impact and validity of simulation modeling in informing decision making in a complex area of healthcare delivery: colorectal cancer (CRC) screening.

Materials and Methods

We searched 10 electronic databases for English-language articles published between January 1, 2008, and March 1, 2019, that described the development of a simulation model with a focus on average-risk CRC screening delivery. Included articles were reviewed for evidence that the model was validated, and provided real or potential contribution to informed decision making using the GRADE EtD (Grading of Recommendations Assessment, Development, and Evaluation Evidence to Decision) framework.

Results

A total of 43 studies met criteria. The majority used Markov modeling (n = 31 [72%]) and sought to determine cost-effectiveness, compare screening modalities, or assess effectiveness of screening. No study reported full model validation and only (58%) reported conducting any validation. Majority of models were developed to address a specific health systems or policy question; few articles report the model’s impact on this decision (n = 39 [91%] vs. n = 5 [12%]). Overall, models provided evidence relevant to every element important to decision makers as outlined in the GRADE EtD framework.

Discussion and Conclusion

Simulation modeling contributes evidence that is considered valuable to decision making in CRC screening delivery, particularly in assessing cost-effectiveness and comparing screening modalities. However, the actual impact on decisions and validity of models is lacking in the literature. Greater validity testing, impact assessment, and standardized reporting of both is needed to understand and demonstrate the reliability and utility of simulation modeling.

Keywords: colorectal cancer screening, simulation modeling, decision making, model validation

INTRODUCTION

Background and significance

A simulation model is a computer-generated representation of a real-world system or process used to analyze the evolving behavior of a system over time or to predict results of various “what if” scenarios. Using trial data, population statistics, and probability estimates, simulation models have been developed in a broad range of areas in health care. They have been used to predict outcomes, resource requirements, feasibility, and costs of proposed interventions. Simulation models offer an invaluable decision aid for policymakers and healthcare leaders in a variety of healthcare applications, including colorectal cancer (CRC) screening.1–6 CRC screening uses fecal tests, diagnostic imaging, or endoscopic examination to assess for possible CRC among asymptomatic individuals at increased risk of developing CRC. Screening has been shown to be effective at detecting CRC earlier, as well as at reducing CRC incidence, morbidity, and mortality.7–10 There are many variations for CRC screening protocols with different modalities (ie, fecal testing vs colonoscopy) and delivery methods (ie, testing kit mailed to individual vs in person at a clinic visit). Determining the optimal modality, delivery method, and interval follow-up between tests in a population of a particular geographic region or jurisdiction can be complex and difficult to assess.11 Simulation modeling offers an opportunity to address this gap by providing a method to test interventions in a simulated environment to inform decision making by health leaders, policymakers, and other end users.1,3

From our review of the literature, we identified 2 gaps in the literature that we aim to address with this systematic review: (1) no systematic review has looked at the application of simulation in CRC screening within the last 10 years, as the most recent review included articles until 2007 inclusively5; (2) multiple systematic reviews have looked at the utility, strengths, and opportunities for simulation modeling in health care but have not evaluated the impact of models on health system and policy decision making.1,12–14

Therefore, we aim to address these knowledge gaps by assessing the validity and impact of simulation modeling in health system and policy decision making for CRC screening delivery as reported in the literature. Recognizing the risk of underreported impact, we also aim to assess the potential impact of a model on a healthcare or policy decision by assessing the utility of the evidence generated by the simulation model using the GRADE EtD (Grading of Recommendations Assessment, Development and Evaluation Evidence to Decision) framework: an internationally recognized set of criteria for evidence-informed decisions in health systems and policy.15

Objective

This study aims to assess the validity and impact of simulation modeling in the health system and policy decision making, where it has been frequently applied in health care: CRC screening delivery.

MATERIALS AND METHODS

Protocol and registration

This systematic review was conducted in accordance with the Cochrane Handbook for Systematic Reviews and the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analysis) statement, and was registered in PROSPERO (no. 130823).16,17 A detailed description of the study protocol including the development process and rationale has been published elsewhere.18 Here, we provide a summary of the methods and minor adjustments from the published protocol.

Search strategy and eligibility criteria

We searched for articles published between January 1, 2008, and March 1, 2019, from 10 electronic databases (Medline, Embase, Cochrane Central, Scopus, National Health Service Economic Evaluation Database, ACM Digital Library, Econolit, Health Technology Assessment Database, IEEEXplore, and Cost-Effective Analysis Registry) using controlled vocabulary (MeSH [Medical Subject Headings]) supplemented by citation searches of all included articles. The final inclusion and exclusion criteria used to select articles are outlined in Table 1.

Table 1.

Inclusion and exclusion criteria

Inclusion criteria Full-text articles published in English that meet all 3 criteria:1. Developed a simulation model derived from clinical data2. Focused on the delivery of colorectal cancer screening to average-risk individuals3. Simulated screening using only the following modalities of screening recommended by Canadian guidelines within the last 10 years:• Fecal occult blood test• Fecal immunohistochemical testing• Flexible sigmoidoscopy• Colonoscopy
Exclusion criteria Excluded if any of the following 4 criteria are met:

  1. Used other screening modalities not recommended in Canadian screening guidelines as identified previously

  2. Commentary or review articles

  3. Simulation model including screening of other cancers

  4. No mention of colorectal cancer screening delivery
Open in a new tab

Selection of studies for review

All article titles and abstracts were screened by 3 independent reviewers (H.S., P.V., and C.K.) using the Abstrackr abstract screening program followed by a screening of selected full-text studies. All were reviewed by H.S., and 50% were each reviewed by C.K. and P.V. to assess for compliance with the eligibility criteria as mentioned in Table 1 using DistillerSR, version 2019 (Evidence Partners, Ottawa, Ontario, Canada).19 Conflicts were resolved by discussion among all 4 authors until a consensus was reached.

Data extraction

All included studies were reviewed by H.S., C.K., and P.V. who each reviewed 50% of the included studies. Using DistillerSR, data were extracted regarding the model description, validation, and impact as outlined in Tables 1 and 2 of the published research protocol (Tables 1 and 2).18 Discrepancies were identified and conflicts resolved through discussion to obtain consensus among the authors. Missing or incomplete data were requested from the study authors.

To assess model accuracy, all models were assessed in accordance with the guidelines of the International Society for Pharmacoeconomics and Outcomes Research-Society for Medical Decision Making (ISPOR-SMDM) good practice guidelines for modeling in health care, including recommendations on model validation. We also assessed for the involvement of “end users”: decision makers, clinicians, and administrators who would use the model for decision making.

Studies were then assessed for the extent to which the study has or could potentially have contributed evidence toward informed decision making. For contribution, each article was searched in its entirety for statements referring to the simulation model results informing decision making. Recognizing that the impact on decision making is often not communicated at the time of publication, we also assessed the potential contribution a simulation model could have made to evidence-informed decision making based on whether the results align with important factors for making informed decisions as outlined in the GRADE EtD (Grading of Recommendations Assessment, Development and Evaluation Evidence to Decision) framework.15,63

Corresponding authors of each study were also contacted to provide further information on model validation, end-user involvement, and impact of the simulation model. Results were analyzed using Microsoft Excel (Microsoft Corporation, Redmond, WA) and RStudio version1.1463 (RStudio, Boston, MA). In contrast to the protocol report, no subgroup analysis was conducted due to the heterogeneity of included articles.18

RESULTS

A total of 492 relevant articles were identified through database and reference searching. Of these, 43 met inclusion criteria for data extraction (Figure 1, Table 2). Cohen’s kappa interrater reliability was 0.82 for title and abstract screening and 0.93 for full-text screening. Modeling methods were identified in all articles. Several used multiple modeling methods (n = 9 of 43 [20%]), and the most frequently encountered combination of techniques was Monte Carlo simulation with Markov modeling (n = 6 of 9 [67%]) (Table 2). The most frequently used methods, including when they were used in combination with other techniques, were Markov modeling (in 31 of 43 studies [72%]), Monte Carlo simulation (in 7 of 43 studies [16%]), and microsimulation (in 6 of 43 studies [14%]) (Figure 2). A total of 33% were based on preexisting models, with the MISCAN-Colon (MIcrosimulation SCreening ANalysis Colorectal Cancer) model the most frequently cited as the base model. A majority of models had multiple applications (39 of 43 studies [90%]). The most common applications were to assess cost-effectiveness of screening (in 32 of 43 studies [74%]), compare screening modalities (in 18 of 43 studies [42%]), or assess resource utilization of screening (in 15 of 43 studies [35%]) (Table 2). Models often included multiple screening modalities, with colonoscopy being the most frequently included (26 [63%]). Inputs were mostly generated from published trials, registries, and population statistics databases within the 5 years of publication (Figure 3). Occasionally, authors conducted primary data collection to populate the model (6 [14% of models]). Model outcomes most often included cost-effectiveness (30 [70% of included articles]), mortality (19 [44%]), cancer detection (13 [30%]), resource utilization (6 [14%]), or screening participation (7 [16%]).

Figure 1.

Figure 1.

Open in a new tab

PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analysis) flow diagram. CRC: colorectal cancer.

Table 2.

Included article summary

 
 
graphic file with name ocaa022ilf1.jpg
Open in a new tab

GRADE EtD: Grading of Recommendations Assessment, Development, and Evaluation Evidence to Decision . “X” denotes validation efforts were described but not explicitly performed.

Figure 2.

Figure 2.

Open in a new tab

Popularity of simulation model methods.

Figure 3.

Figure 3.

Open in a new tab

Model inputs.

Details of the simulation models were not consistently described. The model entities, attributes, and variables were often identified; in contrast, 29 (67%) articles included a figure of the conceptual model, and 23 (54%) identified the software program used to generate the model. Of those that mentioned a software program, TreeAge Pro (TreeAge Software, Williamstown, MA) and Microsoft Excel were most frequently used (48% and 26%, respectively). Few articles described the role of end users in developing or evaluating the simulation model (4 [9%]). We contacted all authors for further information, and among the 12 who responded, 11 (92%) reported end-user involvement in both model conceptualization and development and 8 (67%) reported involvement in model validation.

Model validation was not consistently reported (Figure 4). A majority of authors mentioned steps in their methods that suggest model validation (42 [98%]), in contrast to 25 (58%) that reported performing any form of model validation, and no article addressed all 4 aspects of model validation recommended by ISPOR-SMDM. Internal and predictive validation were most often reported, whereas face validation was rarely reported (34% and 37% vs 7%). All 12 authors who responded to our correspondence reported conducting face validation, and 4 (33%) reported conducting all types of validation recommended by ISPOR-SMDM.

Figure 4.

Figure 4.

Open in a new tab

Model validation.

A majority of simulation models were developed to address a particular health system or policy decision in CRC screening delivery (91%), and all articles contributed some evidence important to informed decision making as outlined in the GRADE EtD framework (Figure 5). A total of 12% of articles described the study as having an impact on decision making, whereas 7 of the 12 (58%) authors who provided feedback reported that the model did have an impact on health system or policy decision making. All 7 of these “impactful articles” also described having end users involved in the research question formulation and model conceptualization. Articles with a reported impact on decision making showed no significant difference in the relative contribution to the GRADE EtD framework (published research protocol, Table 2).18

Figure 5.

Figure 5.

Open in a new tab

Impact on decision making.

DISCUSSION

Simulation modeling informing decision making

This review demonstrates that simulation modeling is a valuable tool for informed decision making in CRC screening delivery by providing evidence that meets all 9 criteria for informed health system and policy decisions outlined by the GRADE EtD framework. In particular, simulation models were found to be useful for estimating the long-term outcomes and cost-effectiveness of different screening strategies in a specific population or geographic region, which is consistent with its application in other areas of health care.5,64 However, the translation of these models to evidence-informed decisions appears to be lacking or underreported. Studies rarely describe an impact on decision making, in contrast to the majority of the 12 authors who responded to our correspondence (16% vs 58%). The generalizability of these 12 responses is limited, given that they only represent 28% of all contacted authors and may have been more inclined to respond given their impact on decision making. Nonetheless, this gap in reporting of impact has been identified in multiple reviews and appears to be an issue unique to simulation modeling in health care, unlike in other sectors.1,5,65,66 Reasons for this may include the journal focus, word count limitations, or timing between publication and policy decision. While we cannot confirm the actual impact of models in the context of these limitations, we find that the potential for impact is very strong based on the GRADE EtD criteria. As clinicians and experts in health systems research, we find this represents a critical gap. The fact that a majority of models were developed to address a specific decision in CRC screening delivery further supports the argument that these models may be impactful, but their impact remains underreported. Other studies not included in this review suggested that simulation modeling is very useful in decision making, as demonstrated by the use of multiple iterations of the MISCAN-Colon model in the Netherlands.37,63 If the purpose of simulation modeling is to inform decision makers, then this needs to be reported. Only then can we understand how to enhance methods of developing impactful simulation models in CRC screening.

End-user engagement is critical to the success of impactful simulation modeling.67 A majority of authors with whom we corresponded indicated that end users were involved in the study and that the model influenced their decision making, but they did not report this in publication. Only 1 included article provided a thorough description of end-user involvement in model development and evaluation.23 Various articles, outside the inclusion criteria of this review, have advocated for greater end-user participation, termed participatory simulation modeling, whereby end users collaborate with researchers in all phases from conceptualization to validation of the model.63 The extent to which similar collaborations actually occur in the development, analysis, and validation of “nonparticipatory” simulation models of CRC screening delivery remains unknown. If our correspondence with authors is representative of all articles included in this review, then end users are frequently involved but their participation is not reported in the published article. The goal of simulation modeling is to serve as a decision support technique for end users. Therefore, it is important to report on the methods used to engage with end users and the extent to which the model supports their decisions, recognizing their importance and that they may vary between projects.68

Model validation and reporting

For simulation models to be useful for decision makers, models must be accurate and valid for application.69 Few articles in this systematic review describe model validation. This concern has been raised by many other reviews of simulation modeling in health care.72–74 In further discussion with the authors, we found that validation was often conducted but not reported. The validation process is not an isolated set of procedures, but rather is an integral part of model development. It provides the model with higher levels of credibility and ensures that the model’s behavior is close enough to that of the actual system so that the model can be a good substitute.39 Reporting the validation process is highly important in model development. There are methods available for testing the validation of simulation models using both subjective comparisons and statistical procedures.38 Multiple guidelines for validation have been developed but are not consistently adopted. A potential next step to address this issue is to require model registration, as is done with clinical trials to reinforce standardization of model validation, promote collaboration, and minimize redundancy of one-off model building.70

Model components and model conceptualization were also not consistently reported. Only 28 (68%) articles included a figure of the conceptual model. To provide a verifiable simulation model, the model entities, attributes, states, activities involved, and the events a system will encounter should be identified to be able to develop a verified simulation model. Providing a conceptual model allows readers to understand the simulation model functions, and see that it mimics the actual system in a way that is accessible to both modelers and end users. Reporting the model components and the conceptual model are also important for assessing the credibility and reproducibility of a model. Templates such as those presented by Banks et al71 can be used to categorize the model components and to construct an efficient conceptual model.

An initial aim of this study was to assess the validity and utility of simulation modeling in CRC screening delivery. However, our final observations are limited due to potentially incomplete reporting by authors at the time of publication. Few studies reported the validity and utility of the simulation model; however, the majority of the 12 authors who responded to our correspondence indicated that their simulation models had been validated and were impactful for end-user decisions. Further research with a larger cohort of author responses is needed to conclude if these authors are representative of all included studies.

CONCLUSION

In this study, we found that simulation models can be a powerful adjunct to empirical research in understanding optimal CRC screening delivery and have been used to generate evidence critical to informing decision making for a broad range of health system and policy decisions in average-risk CRC screening delivery. However, reports of impact on decision making and the validity of simulation models are limited in the literature. Greater validity testing, impact assessment, and standardized reporting of both is needed to understand and demonstrate the validity and utility of simulation modeling.

FUNDING

This work was supported by a research grant from the University of Ottawa Telfer School of Management and a Discovery Grant from the Natural Sciences and Engineering Research Council of Canada.

AUTHOR CONTRIBUTIONS

HS is the guarantor of this review. HS, CK, and PV performed the screening, data extraction, and analysis outlined in this manuscript. All authors participated in the conceptualization and design of this review, and have read and approve of this manuscript.

ACKNOWLEDGEMENTS

We would like to thank Alexandra Davis from the Ottawa Hospital Library for her assistance in conducting the search.

CONFLICT OF INTEREST STATEMENT

All authors declare no competing interests.

REFERENCES

  • 1. Fone D, Hollinghurst S, Temple M, et al. Systematic review of the use and value of computer simulation modelling in population health and health care delivery. J Public Health Med 2003; 25 (4): 325–35. [DOI] [PubMed] [Google Scholar]
  • 2. Xue H, Slivka L, Igusa T, et al. Applications of systems modelling in obesity research. Obes Rev 2018; 19 (9): 1293–308. [DOI] [PubMed] [Google Scholar]
  • 3. Salleh S, Thokala P, Brennan A, et al. Simulation modelling in healthcare: an umbrella review of systematic literature reviews. Pharmacoeconomics 2017; 35 (9): 937–49. [DOI] [PubMed] [Google Scholar]
  • 4. Freebairn L, Atkinson J, Kelly P, et al. Simulation modelling as a tool for knowledge mobilisation in health policy settings: a case study protocol. Health Res Policy Syst 2016; 14 (1): 71. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5. Katsaliaki K, Mustafee N.. Applications of simulation within the healthcare context. J Oper Res Soc 2011; 62 (8): 1431–51. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6. Gauvreau CL, Fitzgerald NR, Memon S, et al. The OncoSim model: development and use for better decision-making in Canadian cancer control. Curr Oncol 2017; 24 (6): 401–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Benson AB, Hoffe S, Nurkin S, Overman MJ. NCCN Guidelines Colorectal Cancer. 2018:186. Available at: https://www.nccn.org/default.aspx. Accessed November 14, 2018.
  • 8. Swartz AW, Eberth JM, Strayer SM.. Preventing colorectal cancer or early diagnosis: which is best? A re-analysis of the U.S. Preventive Services Task Force Evidence Report. Prev Med 2019; 118: 104–12. [DOI] [PubMed] [Google Scholar]
  • 9. Rabeneck L, Rumble RB, Thompson F, et al. Fecal immunochemical tests compared with guaiac fecal occult blood tests for population-based colorectal cancer screening. Can J Gastroenterol 2012; 26: 131–47. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10. Meester RG, Doubeni CA, Lansdorp-Vogelaar I, et al. Colorectal cancer deaths attributable to nonuse of screening in the United States. Ann Epidemiol 2015; 25 (3): 208–13.e1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11. Champion C, Alvarez GG, Affleck E, et al. A systems perspective on rural and remote colorectal cancer screening access. J Cancer Policy 2017; 14: 27–32. [Google Scholar]
  • 12. Sobolev B. Linking operations and health services research. Clin Invest Med 2005; 28 (6): 305–7. [PubMed] [Google Scholar]
  • 13. Zhang X. Application of discrete event simulation in health care: a systematic review. BMC Health Serv Res 2018; 18 (1): 687. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14. Standfield L, Comans T, Scuffham P.. Markov modeling and discrete event simulation in health care: a systematic comparison. Int J Technol Assess Health Care 2014; 30 (2): 165–72. [DOI] [PubMed] [Google Scholar]
  • 15. Moberg J, Oxman AD, Rosenbaum S, et al. The GRADE evidence to decision (EtD) framework for health system and public health decisions. Health Res Policy Syst 2018; 16 (1): 45. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.PRISMA. Available at: http://www.prisma-statement.org/. Accessed December 10, 2018.
  • 17. Higgins JPT, Green S, eds. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0. http://handbook-5-1.cochrane.org/Accessed December 10, 2018.
  • 18. Smith H, Varshoei P, Boushey R, et al. The use of simulation modeling to inform health system and policy decision-making in colorectal cancer screening: a systematic review protocol. JMIR Res Protoc 2019. Nov 26 [E-pub ahead of print]. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19. Wallace BC, Small K, Brodley CE, et al. Deploying an interactive machine learning system in an evidence-based practice center: abstrackr. In:Proceedings of the 2nd ACM SIGHIT International Health Informatics Symposium, New York, NY: ACM Digital Library; 2012: 819–24. [Google Scholar]
  • 20. Tsoi KK, Ng SS, Leung MC, et al. Cost-effectiveness analysis on screening for colorectal neoplasm and management of colorectal cancer in Asia. Aliment Pharmacol Ther 2008; 28 (3): 353–63. [DOI] [PubMed] [Google Scholar]
  • 21. Rodriguez-Moranta F, Trapero-Bertran M, Castells A, et al. Endoscopic requirements of colorectal cancer screening programs in average-risk population. Estimation according to a Markov model. Gastroenterol Hepatol 2008; 31: 405–12. [DOI] [PubMed] [Google Scholar]
  • 22. Macafee DA, Waller M, Whynes DK, et al. Population screening for colorectal cancer: the implications of an ageing population. Br J Cancer 2008; 99 (12): 1991–2000. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23. Berg B, Denton B, Nelson H, et al. A discrete event simulation model to evaluate operational performance of a colonoscopy suite. Med Decis Making 2010; 30 (3): 380–7. [DOI] [PubMed] [Google Scholar]
  • 24. Subramanian S, Bobashev G, Morris RJ.. Modeling the cost-effectiveness of colorectal cancer screening: policy guidance based on patient preferences and compliance. Cancer Epidemiol Biomarkers Prev 2009; 18 (7): 1971–8. [DOI] [PubMed] [Google Scholar]
  • 25. Lejeune C, Dancourt V, Arveux P, et al. Cost-effectiveness of screening for colorectal cancer in France using a guaiac test versus an immunochemical test. Int J Technol Assess Health Care 2010; 26 (1): 40–7. [DOI] [PubMed] [Google Scholar]
  • 26. Ventura L, Zappa M, Carreras G, et al. What is the best screening strategy to detect advanced colorectal adenomas? Simulation from ongoing Italian screening experiences. Tumori 2011; 97 (5): 547–50. [DOI] [PubMed] [Google Scholar]
  • 27. van Rossum LG, van Rijn AF, Verbeek AL, et al. Colorectal cancer screening comparing no screening, immunochemical and guaiac fecal occult blood tests: a cost-effectiveness analysis. Int J Cancer 2011; 128 (8): 1908–17. [DOI] [PubMed] [Google Scholar]
  • 28. Tran B, Keating CL, Ananda SS, et al. Preliminary analysis of the cost-effectiveness of the National Bowel Cancer Screening Program: demonstrating the potential value of comprehensive real world data. Intern Med J 2012; 42 (7): 794–800. [DOI] [PubMed] [Google Scholar]
  • 29. Whyte S, Walsh C, Chilcott J.. Bayesian calibration of a natural history model with application to a population model for colorectal cancer. Med Decis Making 2011; 31 (4): 625–41. [DOI] [PubMed] [Google Scholar]
  • 30. Wang VW, Koh PK, Chow WL, et al. Predictive genetic testing of first degree relatives of mutation carriers is a cost-effective strategy in preventing hereditary non-polyposis colorectal cancer in Singapore. Fam Cancer 2012; 11 (2): 279–89. [DOI] [PubMed] [Google Scholar]
  • 31. Sharp L, Tilson L, Whyte S, et al. Cost-effectiveness of population-based screening for colorectal cancer: a comparison of guaiac-based faecal occult blood testing, faecal immunochemical testing and flexible sigmoidoscopy. Br J Cancer 2012; 106 (5): 805–16. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32. Sharp L, Tilson L, Whyte S, et al. Using resource modelling to inform decision making and service planning: the case of colorectal cancer screening in Ireland. BMC Health Serv Res 2013; 13 (1): 105. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33. Sharaf RN, Ladabaum U.. Comparative effectiveness and cost-effectiveness of screening colonoscopy vs. sigmoidoscopy and alternative strategies. Am J Gastroenterol 2013; 108 (1): 120–32. [DOI] [PubMed] [Google Scholar]
  • 34. Hosking M, Roberts S, Uzsoy R, et al. Investigating interventions for increasing colorectal cancer screening: insights from a simulation model. Socioecon Plann Sci 2013; 47 (2): 142–55. [Google Scholar]
  • 35. Dinh T, Ladabaum U, Alperin P, et al. Health benefits and cost-effectiveness of a hybrid screening strategy for colorectal cancer. Clin Gastroenterol Hepatol 2013; 11 (9): 1158–66. [DOI] [PubMed] [Google Scholar]
  • 36. Cronin P, Goodall S, Lockett T, et al. Cost-effectiveness of an advance notification letter to increase colorectal cancer screening. Int J Technol Assess Health Care 2013; 29 (3): 261–8. [DOI] [PubMed] [Google Scholar]
  • 37. van Hees F, Zauber AG, van Veldhuizen H, et al. The value of models in informing resource allocation in colorectal cancer screening: the case of the Netherlands. Gut 2015; 64 (12): 1985–97. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38. Wilson FA, Villarreal R, Stimpson JP, et al. Cost-effectiveness analysis of a colonoscopy screening navigator program designed for Hispanic men. J Can Educ 2015; 30 (2): 260–7. [DOI] [PubMed] [Google Scholar]
  • 39. Li Y, Zhu M, Klein R, et al. Using a partially observable Markov chain model to assess colonoscopy screening strategies – a cohort study. Eur J Oper Res 2014; 238 (1): 313–26. [Google Scholar]
  • 40. Chauvin P, Josselin J-M, Heresbach D.. The influence of waiting times on cost-effectiveness: a case study of colorectal cancer mass screening. Eur J Health Econ 2014; 15 (8): 801–12. [DOI] [PubMed] [Google Scholar]
  • 41. Cenin DR, St John DJ, Ledger MJ, et al. Optimising the expansion of the National Bowel Cancer Screening Program. Med J Aust 2014; 201 (8): 456–61. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42. Sekiguchi M, Igarashi A, Matsuda T, et al. Optimal use of colonoscopy and fecal immunochemical test for population-based colorectal cancer screening: a cost-effectiveness analysis using Japanese data. Jpn J Clin Oncol 2016; 46 (2): 116–25. [DOI] [PubMed] [Google Scholar]
  • 43. Goede SL, Kuntz KM, van Ballegooijen M, et al. Cost-savings to Medicare from pre-Medicare colorectal cancer screening. Med Care 2015; 53 (7): 630–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44. Fitch K, Pyenson B, Blumen H, et al. The value of colonoscopic colorectal cancer screening of adults aged 50 to 64. Am J Manag Care 2015; 21 (7): e430–8. [PubMed] [Google Scholar]
  • 45. Coldman A, Flanagan W, Nadeau C, et al. Projected effect of fecal immunochemical test threshold for colorectal cancer screening on outcomes and costs for Canada using the OncoSim microsimulation model. J Cancer Policy 2017; 13: 38–46. [Google Scholar]
  • 46. Brenner H, Kretschmann J, Stock C, et al. Expected long-term impact of screening endoscopy on colorectal cancer incidence: a modelling study. Oncotarget 2016; 7 (30): 48168–79. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47. Wong MC, Ching JY, Chan VC, et al. Colorectal cancer screening based on age and gender: a cost-effectiveness analysis. Medicine (Baltimore) 2016; 95 (10): e2739. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48. Song L-P, Wang H-Y.. Modeling and control of colorectal cancer. PLoS One 2016; 11 (8): e0161349. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49. Pil L, Fobelets M, Putman K, et al. Cost-effectiveness and budget impact analysis of a population-based screening program for colorectal cancer. Eur J Intern Med 2016; 32: 72–8. [DOI] [PubMed] [Google Scholar]
  • 50. Comas M, Mendivil J, Andreu M, et al. Long-term prediction of the demand of colonoscopies generated by a population-based colorectal cancer screening program. PLoS One 2016; 11 (10): e0164666. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51. McLeod M, Kvizhinadze G, Boyd M, et al. Colorectal cancer screening: how health gains and cost-effectiveness vary by ethnic group, the impact on health inequalities, and the optimal age range to screen. Cancer Epidemiol Biomarkers Prev 2017; 26 (9): 1391–400. [DOI] [PubMed] [Google Scholar]
  • 52. Hassmiller Lich K, Cornejo DA, Mayorga ME, et al. Cost-effectiveness analysis of four simulated colorectal cancer screening interventions, North Carolina. Prev Chronic Dis 2017; 14: E18. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53. Chiu SY, Malila N, Yen AM, et al. Predicting the effectiveness of the Finnish population-based colorectal cancer screening programme. J Med Screen 2017; 24 (4): 182–8. [DOI] [PubMed] [Google Scholar]
  • 54. Aronsson M, Carlsson P, Ekbom A, et al. Health effects and costs due to postcolonoscopy colorectal cancer. United Eur Gastroenterol J 2016; 4: A70–1. [Google Scholar]
  • 55. Idigoras I, Arrospide A, Portillo I, et al. Evaluation of the colorectal cancer screening Programme in the Basque Country (Spain) and its effectiveness based on the Miscan-colon model. BMC Public Health 2018; 18 (1): 78. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56. Rice K, Sharma K, Li C, et al. Cost-effectiveness of a patient navigation intervention to increase colonoscopy screening among low-income adults in New Hampshire. Cancer 2019; 125 (4): 601–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57. Senore C, Hassan C, Regge D, et al. Cost-effectiveness of colorectal cancer screening programmes using sigmoidoscopy and immunochemical faecal occult blood test. J Med Screen 2019; 26 (2): 76–83. [DOI] [PubMed] [Google Scholar]
  • 58. Melnitchouk N, Soeteman DI, Davids JS, et al. Cost-effectiveness of colorectal cancer screening in Ukraine. Cost Eff Resour Alloc 2018; 16 (1): 20. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 59. Ladabaum U, Mannalithara A, Brill JV, et al. Contrasting effectiveness and cost-effectiveness of colorectal cancer screening under commercial insurance vs. Medicare. Am J Gastroenterol 2018; 113 (12): 1836–47. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 60. Arrospide A, Idigoras I, Mar J, et al. Cost-effectiveness and budget impact analyses of a colorectal cancer screening programme in a high adenoma prevalence scenario using MISCAN-Colon microsimulation model. BMC Cancer 2018; 18 (1): 464. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 61. Chen C, Stock C, Hoffmeister M, et al. How long does it take until the effects of endoscopic screening on colorectal cancer mortality are fully disclosed? A Markov model study: long-term effects of endoscopic screening. Int J Cancer 2018; 143 (11): 2718–24. [DOI] [PubMed] [Google Scholar]
  • 62. Areia M, Fuccio L, Hassan C, et al. Cost-utility analysis of colonoscopy or faecal immunochemical test for population-based organised colorectal cancer screening. United European Gastroenterol J 2019; 7 (1): 105–13. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 63. Wheeler SB, Leeman J, Hassmiller Lich K, et al. Data-powered participatory decision making: leveraging systems thinking and simulation to guide selection and implementation of evidence-based colorectal cancer screening interventions. Cancer J 2018; 24 (3): 136–43. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 64. Koleva-Kolarova RG, Zhan Z, Greuter MJW, et al. Simulation models in population breast cancer screening: a systematic review. Breast 2015; 24 (4): 354–63. [DOI] [PubMed] [Google Scholar]
  • 65. van Lent WA, VanBerkel P, van Harten WH.. A review on the relation between simulation and improvement in hospitals. BMC Med Inform Decis Mak 2012; 12 (1): 18. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 66. Brailsford SC, Bolt T, Connell C, et al. Stakeholder engagement in health care simulation In: WSC ’09: Winter Simulation Conference Proceedings. New York, NY: ACM Digital Library; 2009; 1840–9. [Google Scholar]
  • 67. Tako AA, Kotiadis K. Participative simulation (PartiSim): a facilitated simulation approach for stakeholder engagement. In: 2018 Winter Simulation Conference (WSC). Piscataway, NJ: IEEE; 2018: 192–206
  • 68. Weinstein MC, Toy EL, Sandberg EA, et al. Modeling for health care and other policy decisions: uses, roles, and validity. Value Health 2001; 4 (5): 348–61. [DOI] [PubMed] [Google Scholar]
  • 69. Eddy DM, Hollingworth W, Caro JJ, et al. Model transparency and validation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-7. Value Health 2012; 15 (6): 843–50. [DOI] [PubMed] [Google Scholar]
  • 70. Sampson CJ, Wrightson T.. Model registration: a call to action. Pharmacoeconomics Open 2017; 1 (2): 73–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 71. Banks J, Carson J, Nelson B, et al. Discrete-Event System Simulation. 5th ed. Upper Saddle River, NJ: Prentice Hall; 2010. [Google Scholar]
  • 72. Haji Ali Afzali H, Gray J, Karnon J.. Model performance evaluation (validation and calibration) in model-based studies of therapeutic interventions for cardiovascular diseases: a review and suggested reporting framework. Appl Health Econ Health Policy 2013; 11 (2): 85–93. [DOI] [PubMed] [Google Scholar]
  • 73. de Boer PT, Frederix GWJ, Feenstra TL, et al. Unremarked or unperformed? Systematic review on reporting of validation efforts of health economic decision models in seasonal influenza and early breast cancer. Pharmacoeconomics 2016; 34 (9): 833–45. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 74. Kolovos S, Bosmans JE, Riper H, et al. Model-based economic evaluation of treatments for depression: a systematic literature review. Pharmacoeconomics Open 2017; 1 (3): 149–65. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Journal of the American Medical Informatics Association : JAMIA are provided here courtesy of Oxford University Press

RESOURCES