Addressing Challenges of Economic Evaluation in Precision Medicine Using Dynamic Simulation Modeling

Abstract

Objectives:

The objective of this article is to describe the unique challenges and present potential solutions and approaches for economic evaluations of precision medicine (PM) interventions using simulation modeling methods.

Methods:

Given the large and growing number of PM interventions and applications, methods are needed for economic evaluation of PM that can handle the complexity of cascading decisions and patient-specific heterogeneity reflected in the myriad testing and treatment pathways. Traditional approaches (eg, Markov models) have limitations, and other modeling techniques may be required to overcome these challenges. Dynamic simulation models, such as discrete event simulation and agent-based models, are used to design and develop mathematical representations of complex systems and intervention scenarios to evaluate the consequence of interventions over time from a systems perspective.

Results:

Some of the methodological challenges of modeling PM can be addressed using dynamic simulation models. For example, issues regarding companion diagnostics, combining and sequencing of tests, and diagnostic performance of tests can be addressed by capturing patient-specific pathways in the context of care delivery. Issues regarding patient heterogeneity can be addressed by using patient-level simulation models.

Conclusion:

The economic evaluation of PM interventions poses unique methodological challenges that might require new solutions. Simulation models are well suited for economic evaluation in PM because they enable patient-level analyses and can capture the dynamics of interventions in complex systems specific to the context of healthcare service delivery.

Background

Economic evaluations are now commonly used by health technology assessment (HTA) organizations and payers to assess the value of new healthcare interventions (including medicines, pharmaceuticals, devices, and healthcare programs) and to inform decision makers about the efficient allocation of healthcare resources.¹ Many countries have developed guidance on the key considerations in economic evaluations such as the target population, analytical perspective, choice of comparator, analytical methods, outcome measures, measures of utility, costs to be included, time horizon, discounting, and sensitivity analysis.² These economic evaluation guidelines commonly also consider the use of economic modeling.^3,4 Traditional modeling approaches, such as decision trees or Markov cohort models, typically examine the value of 1 test compared with standard care, where this test is conducted for a specific clinical reason, and provide 1 result (eg, diagnosis) and a single trajectory of costs and outcomes for cohorts of “average” patients.^5,6 Nevertheless, the notion of the “average” person is challenged in the evaluation of precision medicine (PM) interventions.^7–9

From a broad perspective, personalized approaches in health applications can be defined as those using any type of patient-specific information (diagnostic tests, patient-reported outcomes, risk estimates, functional performance) to inform therapy tailored to the patient.^7,10 Ginsburg and Phillips¹¹ differentiated between personalized and precision medicine with precision medicine going beyond genomics, preferences, beliefs, attitudes, knowledge, and social context.¹¹ They described PM as a model for healthcare delivery that relies on data, analytics, and information with the ability to guide healthcare decisions tailored to a specific patient. It is generally acknowledged that PM interventions have the ability to guide healthcare decisions tailored to a specific patient, with the aim of improving the effectiveness and quality of care, reducing related adverse events, and reducing the need for unnecessary testing and treatment.¹¹

Although advanced diagnostic tests used to target treatments can drive improved outcomes, the result of using these tests is a complex and dynamic set of treatment pathways that varies for each individual and where the downstream consequences and outcomes are difficult to predict. To support decision making in PM, models need to reflect the dynamic treatment pathways, the underlying clinical evidence for improved outcomes, and the uncertainties around the evidence.¹²

In the past decade, HTA organizations have considered a large and growing number of PM interventions for approval and reimbursement. For example, pembrolizumab was the first solid cancer therapy approved for use by the Food and Drug Administration in 2017 based on the presence of a specific biomarker rather than the location of a particular tumor.¹³ Methods are needed for economic evaluation of PM that can handle this complexity of cascading decisions, reflected in the myriad treatment pathways, while presenting results that provide meaningful information to support health policy decisions.

Given that many of the current health economic modeling approaches are not likely able to capture this complexity, the objective of this article is to present dynamic simulation modeling as another modeling paradigm that can manage this complexity. The article first describes the challenges of economic evaluation in PM and how simulation modeling methods can potentially address some of these challenges. Then we provide an overview of common simulation modeling methods in health. Finally, empirical examples are used to illustrate how simulation modeling methods could be and have been used to capture the individual care pathways as seen in PM.

Specific Technical Challenges of Economic Evaluation in PM

The economic evaluation of PM interventions epitomizes the need for methods that reflect a system view to healthcare that captures the upstream and downstream consequences of changes in healthcare with multiple decision points based on patient-level characteristics. PM typically involves multiple diagnostic tests and treatments over time, with the sequence of these tests and treatments differing between individual patients.¹¹ These pathways or clinical trajectories are complex, dynamic, and specific to the context of the delivery of healthcare services.⁸ Cohort and state-transition models have limited ability to deal with such pathways and typically report results for an “average” patient.^6,14

The challenges of conducting economic evaluations of PM have been synthesized by Phillips et al⁹ using next-generation sequencing (NGS) as an example. NGS tests including targeted gene panels, whole-exome sequencing, and whole-genome sequencing, which simultaneously examine multiple genes and can contribute to making treatment recommendations. The top-priority challenges include (1) the requirement for a complex model structure that incorporates multiple pathways, results, and testing uses to reflect the fact that NGS tests evaluate multiple genes and to consider interactive effects across multiple conditions and (2) the need to specify a time frame that captures both the upstream and downstream costs and outcomes specific to NGS and the need to capture downstream consequences for patients and families.

Additional conceptual and methodological challenges in PM relate to the valuation of outcomes including preferences for genomic information, multiple layers of uncertainty, and risks.¹⁵ Patient heterogeneity is critically important in the context of economic evaluation in PM as individual care pathways and decisions about treatment and care need to be considered in the analysis. Although current guidelines for health economic modeling recognize the importance of patient heterogeneity,^3,16 there is little specific guidance on how to incorporate heterogeneity in cost-effectiveness models.

Nonetheless, in a systematic literature review of health economic models specifically in personalized and PM (in general and including patient stratification by other means than genetics) in all disease areas, Degeling et al⁸ reported that decision tree models and Markov models remain the most frequently used approach (n = 21 of 31 publications, 66%). When analyzed over time, there was an observed increase in patient-level simulation modeling methods such as discrete event simulation (DES) since 2011. This review also provides a checklist of the 10 main challenges for health economic modeling in PM, and these challenges are summarized in Table 1 along with a description of how these challenges can potentially be addressed by simulation modeling methods rather than traditional health economic modeling methods. Only a minority of these health economic models of PM addressed the 10 items in the checklist or otherwise identified and reported the challenge. The most commonly addressed challenge was the diagnostic performance of the test (n = 20 of 31 studies, 65%) and the different combinations of test(s) and treatments(s) (n = 22 of 31 studies, 71%). Although 11 of the models (35%) included combinations of tests, all assumed a fixed sequence of tests. Only 6 of the models (19%) were defined on a patient level. These findings suggest that economic models of PM have not yet fully embraced these more advanced simulation modeling methods to address these challenges.

Table 1.

Checklist of challenges for health economic modeling in the context of personalized medicine and how these challenges can potentially be addressed by simulation modeling.⁸

Challenge	Specification of challenge in the checklist	How these challenges are addressed by simulation modeling and limitations compared with traditional health economic modeling
1. Modeling patient-level processes	Is the model defined on a patient level?	Patient-level models reflecting care pathways considering context of delivery
2. Modeling patients’ preferences	Are patients’ preferences modeled to take their effect on the outcomes into account?	Incorporate at decision nodes the probability of uptake based on patient preferences; issue of availability of the data and the attributes from preferences need to align with variables in the model/care pathway
3. Modeling physicians’ preferences	Are physicians’ preferences modeled to take their effect on the outcomes into account?	Incorporate at decision nodes the probability of uptake based on physician preferences; issue of availability of the data and the attributes from preferences need to align with variables in the model/care pathway
4. Taking into account the diagnostic performance of tests	Is the effect of the sensitivity, specificity, positive predictive value, and/or negative predictive value on the outcomes taken into account?	Include compound probabilities based on patient-specific pathways considering context of care delivery
5. Modeling combinations of tests	Does the modeled process include combinations of tests and/or prediction models?	Include compound probabilities based on patient-specific pathways
6. Modeling companion diagnostics	Does the modeled process include combinations of test(s) and treatment(s)?	Include compound probabilities based on patient-specific pathways
7. Study-specific outcome measures	Does the modeled process include study-specific outcomes, such as disease-specific adverse events?	Patient-level models reflecting care pathways and patient-specific outcomes based on patient characteristics
8. Data gaps	Do the authors mention any evidence gaps? If so, do they mention that these evidence gaps exist because of stratification of patients based on risk models and/or test results?	Simulation models offer greater flexibility to include patient-specific pathways and account for stratification of patients based on risk models and/or test results
9. Greater uncertainty due to more complex analysis	Do the authors mention greater uncertainty with respect to the outcomes, due to more complex analysis, as a result of personalization of the model?	Simulation models offer greater flexibility to include patient-specific pathways and account for uncertainty at a patient level; there remain challenges to aggregate these findings
10. Absence of guidelines	Do the authors mention any difficulties related to the absence of guidelines for health economic modeling in the context of personalized medicine?	There is guidance for simulation modeling from the operations research literature and emerging in health

Overview of Simulation Modeling Methods in Health Economic Evaluation

Health economic evaluation models have commonly used cohort-based Markov or state-transition modeling to reflect the clinical evidence and estimate the cost-effectiveness of an intervention compared with usual care in the form of an incremental cost-effectiveness ratio. For example, of 58 publications of economic evaluations modeling therapies for rheumatoid arthritis identified between 1996 and 2012, 38 (66%) were decision trees (n = 13) or Markov models (n = 25).¹⁷ In these models, a limited number of health states represent the care pathway for a group of patients. These models typically aggregate the complexity in the real world into a few distinct health states and therefore heavily rely on assumptions about how patients move from one health state to another and how overall outcomes are synthesized or extrapolated, and importantly, they neglect the heterogeneity at the patient level.

The use of cohort models has been partly driven by the use of economic evaluation in the context of HTA, which has historically focused on single interventions (often a new medicine).¹⁸ Further, many of these models are trial based, and they therefore model the relatively straightforward head-to-head comparison of 2 interventions.

Nevertheless, a different paradigm is needed as HTA increasingly breaks out of the “adoption addiction” and moves toward health technology management with a broader systems view of innovation, adoption, and disinvestment throughout the technology life cycle to support healthcare decision making.^18,19 Dynamic simulation modeling is a collection of methods developed primarily in the context of business operations that may be better suited than cohort-based models to reflect the dynamics of the health system and delivery of services.²⁰

Dynamic simulation models use mathematical representations of complex systems with multiple intervention scenarios and evaluate their consequences over time from a systems perspective.²⁰ Complex systems consist of tasks that are relationally dependent events with unpredictable outcomes.²¹ As a consequence, simulation models are generally nonlinear and described implicitly through rules or equations, whereas state-transition models are typically linear and defined by transition matrices between health states that are indexed by time. The nonlinearity in simulation models implies that the individual must be simulated in context of the broader context of other individuals and the healthcare system across time such that emergent behavior reflects the system rather than individuals.^20,22

The literature on the applications of simulation modeling in healthcare is growing rapidly, but given its roots, most applications of these methods remain in the traditional areas of operations research including scheduling, transportation, and allocation of resources. In this article, we focus on 2 types of dynamic simulation models, DES and agent-based models, and discuss their potential for use in PM (see Box 1 and Table 2).^5,23–28

BOX 1. Descriptive overview of discrete event simulation and agent-based modeling methods.²².

Discrete event simulation (DES)	Agent-based model
DES is a simulation modeling method used to represent processes at an individual level where individuals may be subject to events, whether they be decisions or occurrences over time. DES captures individual-level heterogeneity and is used to characterize and analyze queuing processes and networks of queues where there is an emphasis on the utilization of resources.	ABM is a simulation modeling method used to represent individual objects called “agents” and describe their local behavior with local rules. Agents are social and interact with others and their environment, and they may learn and adapt themselves on the basis of experience. ABM is useful to discover patterns of emergence in dynamic and adaptive systems by using deductive and inductive reasoning.

Table 2.

Description of simulation model characteristics.^*

	Discrete event simulation (DES)	Agent-based modeling (ABM)
Type of problem	Operational, tactical	Strategic at the policy level (eg, to inform program implementation) Operational at the management level (eg, tactical at the level of logistics, such as scheduling)
Perspective	Process oriented, emphasis on detail complexity (top down)	Individual oriented, dynamic and detail complexity (bottom up)
Handling of time	Discrete	Discrete
Approach	Explanatory	Exploratory and explanatory
Basic building blocks	Entities, events, queues	Autonomous agents, decision rules
Data sources	Numerical with some judgmental elements	Broadly drawn: qualitative and quantitative
Unit of analysis	Queues, events	Decision rules, emergent behavior
Mathematical formulation	Mathematically described with logic operators	Mathematically described with logic operators and decision rules
Outputs	Point predictions, performance measures	Detailed and aggregate key indicators, understanding of emergence due to individual behavior, point predictions
Advantages	Flexible, which facilitates updating Useful for problems for which it is particularly relevant to be able to capture the changing attributes of entities (eg, patients) and for which the processes to be characterized can be described by events	Unique feature of ABM is the ability to capture relationship networks among individuals Well suited to addressing public health planning and policy needs, as well as healthcare infrastructure investment decisions Can help address problems that involve both deterministic and stochastic processes ABM allows testing assumptions about human behavior in response to new information, incentives, or penalties
Disadvantages	Compared with traditional health economic models, DES models are data intensive and require more time to obtain data and data analysis to prepare model inputs compared with traditional health economic models; programming and calibration are usually time-consuming	Compared with traditional health economic models, ABM models are data intensive and require more time to obtain data and data analysis to prepare model inputs; programming and calibration are usually time-consuming

^*Adapted from Marshall et al²² and Caro and Moler.²⁴

Discrete Event Simulation

DES is the most commonly used dynamic simulation modeling approach in the healthcare context.²⁵ In DES models, the behavior of a system is captured in an ordered sequence of defined events (eg, a test is done, or a treatment is provided). Other events include points in time with respect to changes in health such as the detection or recurrence of a disease or delivery of services such as admission to a health facility and receipt of medical or surgical treatment. DES has the flexibility to map care pathways that incorporate different testing strategies and services that may affect uptake of treatments and downstream patient outcomes.²⁰

There is a considerable body of literature on the application of DES in health to address a variety of healthcare issues.^5,29,30 A recent systematic review by Zhang³¹ identified 211 studies using DES as the main modeling technique in the context of healthcare delivery or public health scenarios (not specific to PM). The authors noticed a significant expansion of publications in this area especially after 2010. The studies were categorized into 4 main classes of applications of DES: health and care systems operations, disease progression modeling, screening modeling, and health behavior modeling. Zhang³¹ reported that most DES models (93%) belong to the first 2 categories (ie, health and care systems operations and disease progression modeling) and aimed to evaluate the effects of operational changes, health economic evaluations, and patient scheduling.

In comparison with state-transition models that do not capture dynamic interactions in the delivery of care, DES is more suited to modeling complex systems by reflecting patient flows through the system.²⁴ For instance, DES models can incorporate attributes of individuals that can affect or even determine responses to events, including age, sex, health status, illness history, duration of disease, and other demographics. These attributes can also vary over time.

Agent-Based Modeling

Agent-based modeling (ABM) is another dynamic simulation method for modeling dynamic, adaptive, and autonomous systems in which the agent (for example, the patient or doctor) serves as the entity of simulation.²⁸ The agents in an ABM model interact within an environment based on a set of decision rules describing the agent’s behavior and operationalized using mathematical logic operators.²⁶ An agent’s behavior is typically nonlinear and dependent on previous interactions. For instance, the likelihood of seeing a doctor increases after contamination with a virus resulting in a fever. Agents can also be programmed to change health states based on an interaction with the health system, which implies we can accumulate costs and outcomes attached to an agent’s state as long as the simulation is running. ABM emphasizes agent–agent (for example, patient and care providers) and agent–environment (for example, patient and hospital clinic) interactions and is therefore well suited to examining patterns of health and behavior of populations over time, for example, modeling infectious disease outbreaks.^20,22

In summary, simulation models such as DES and ABM can enable decision makers to better understand the behavior of complex systems—characterized by these nonlinearities and spatial relationships among entities, multiple agents, feedback loops, and variables that evolve dynamically over time—and predict their response to changes with intended and unintended consequences.

Applications and Case Examples of Simulation Modeling for PM

In this final section, we present several examples to illustrate how complex genomic testing affects clinical pathways that would be better addressed in dynamic simulation modeling. These case examples represent a range of applications of simulation models that address some of the 10 challenges identified in the checklist, including clinical risk stratification (which may not be based on genetics) at a patient level, diagnostic performance of the test, combinations of tests, and different testing and treatment sequences and study-specific outcome measures.

Molecular Profiling to Inform Treatment Decisions in Patients With Cancer

One of the most prominent changes in the treatment landscape in cancer is the availability of complex genomic tests in combination with targeted treatments or immunotherapy. Nevertheless, the diagnostic component is usually not considered in health economic models.³² For instance, women with early-stage breast cancer are faced with challenging treatment decisions determined by stage of disease and the use of neoadjuvant or adjuvant systemic therapy. Patients with overexpressed HER2 (HER2 positive) are eligible for anti-HER2 targeted therapies, such as trastuzumab.³³ Likewise, testing for estrogen receptor will identify eligibility for hormone therapy. Although these tests, either through immunohistochemistry or fluorescence in situ hybridization, have been available for quite some time, health economic models have rarely considered the diagnostic performance of the assays,³² despite the differences in test accuracy between methods to assess HER2 status reported in a recently published meta-analysis.³⁴ Likewise, clinical guidelines recommend determination of HER2, estrogen receptor, and progesterone receptor expression status on tissue biopsy and excision material after surgery. Recent studies have shown discordance of up to 15% for progesterone receptor between both samples, which affects optimal treatment decisions.³⁵

Although these examples illustrate only how a simple immunohistochemistry or fluorescence in situ hybridization test can change treatment pathways, it will soon become more complex in terms of building the clinical and health economic evidence base. Several international studies have tested complex genomic profiling (NGS panels, whole exome sequencing, and whole genome sequencing) in metastatic cancers to direct treatment. Although earlier work on HER2 was for only 1 specific treatment, complex gene panels (140 genes) used in lung cancer can guide decisions for a dozen of targeted treatments, including ALK-TKI and EGRF-TKIs, and immunotherapy. The recently published PROFILER study presented the evidence for using complex genomic testing using 2 gene panels in nearly 2600 patients in France, showing that 27% were recommended a molecularly based therapy, whereas only 6% actually received a molecularly based therapy.³⁶ Modeling the health economic impact in this study is far more complex and may require more granularity than possible even through simulation modeling.

Several examples using DES in personalized oncology have been published, including comparisons to state-transition models in case of colorectal cancer.³⁷ Although the previous study was a direct head-to-head economic evaluation of 2 treatment strategies based on a clinical trial, DES has also been used to model the health economic impact of using circulating tumor cells to monitor progressive metastatic castration resistant prostate cancer. The model accounted for 2 lines of treatment in which several biomarkers could be used to optimize treatment strategies.³⁸

Simulation modeling can also be useful in situations inwhich there is decisional conflict, such as when guidelines recommend adjuvant chemotherapy but chances of recurrence are estimated to be low. In this scenario, personalized prediction tools such as gene expression profiling (eg, OncotypeDX [Genomic Health Inc., Redwood City, CA] or Mammaprint [Agendia Inc., Amsterdam, the Netherlands]) can calculate the likelihood of cancer recurrence and can help to risk stratify women who may not benefit from chemotherapy, sparing them from associated toxicity. Jahn et al³⁹ developed a DES model to evaluate the cost-effectiveness of OncotypeDx used both with and without the Adjuvant! Online (AO) (Peter M Ravdin, University of Texas, Austin, TX) score, a web-based decision aid to guide decisions about the use of adjuvant chemotherapy. The combination of OncotypeDx and AO results resulted in 12 distinct risk groups of patients. A DES model was selected for this analysis for multiple reasons: (1) individual patient pathways were influenced by multiple characteristics and test results, (2) there were time-dependent functional relationships, (3) OncotypeDx and AO could be modeled as companion tests, and (4) the desired to track and report individual patient pathways. In another publication using the same model, the authors moved even further toward personalized treatment strategies by evaluating the cost-effectiveness of adjuvant chemotherapy for women in specific risk groups according to the joint results of gene expression profiling and AO. Using the DES model, the authors developed a flexible tool for the evaluation of several test-treatment strategies.⁴⁰

PM Treatment Options in Chronic Obstructive Pulmonary Disease

As noted earlier, state transition models have a limited ability to deal with patient-specific pathways of diagnosis and treatment. The article by Hoogendorn et al⁴¹ explores the suitability of models using more traditional approaches to evaluate PM treatment options for chronic obstructive pulmonary disease (COPD). The authors assessed COPD models to determine which types of patient heterogeneity (risk stratification by patient characteristics) were included. The authors concluded that all of the currently available models are capable of running simulations for different age and COPD severity classes, and most models also have the ability to run analyses separately by sex and smoking status. Nevertheless, the validity of subgroup analyses within the models was questioned because important input parameters were not specified by sex, age, or smoking status. Thus, the challenge for the evaluation of PM treatment options in COPD is that treatment is more likely to be personalized on the basis of clinical parameters rather than age, sex, and smoking status, such that the existing models are likely of limited usefulness. Information on the effectiveness and cost-effectiveness of treatment options for these subgroups is needed to guide clinical guideline development and decisions for reimbursement. It was recommended that future models should include all clinical patient characteristics considered to influence disease severity, prognosis, and treatment response in COPD.⁴²

To address PM in COPD, for which treatment is tailored to specific clinical phenotypes, such as patients with frequent exacerbations, Hoogendoorn et al⁴² subsequently developed a patient-level DES simulation model for COPD that was able to estimate the incremental costs and effects of different treatments for many subgroups of patients. The model included 14 patient characteristics (eg, current smoking status, the number of pack-years smoked, history of heart failure, presence of asthma, bronchodilator responsiveness, diabetes, history of depression), 10 intermediate outcomes (eg, exacerbations, pneumonias, lung function), and 3 final outcomes (death, quality-adjusted life-years, and costs). This new model was unique because none of the previously published models had modeled time to events simultaneously with changes in clinical variables for individual patients.

Other Considerations

Although there are some distinct advantages of simulation modeling over traditional state-transition models for the economic evaluation of PM, this approach can also present challenges for both the analysts who construct these models (eg, data requirements, computational requirements, requirements for modeling skills, model reproducibility) and the decision makers who use these models (eg, additional expertise required by evidence review groups) (Table 2).

The additional complexity that can be captured using these modeling methods come at the cost of more complex structures and level of detail.²⁴ Consequently, analysts with specialized modeling and biostatistical skill sets are required to design, develop, program, and conduct analyses using simulation modeling software. These complex models require more data, both in terms of the number of parameters that need to be estimated and the volume of observations needed to inform these estimates. Simulation models require a great deal of data that may not be available in the detail required to populate the model, especially for early stage analyses of PM interventions.

This complexity can also present challenges for reviewers and users of these models. The complexity and transparency of cost-effectiveness modeling have been long-standing concerns because decisions about the use of drugs and other therapeutic interventions were first influenced by economic analyses using modeling approaches.^43,44 More recently, transparency in modeling has been promoted through reporting standards, reference models, collaboration, model registration, peer review, and open-source modeling.⁴⁵ With the additional complexity associated with dynamic simulation models, further challenges in communicating the structure, assumptions, and outcomes from these models may be encountered. The uptake of such methods may therefore be lower than is optimal, and it may be more difficult to comprehend, assess the validity, and interpret the findings from these models.

Summary

In this article, we discuss how simulation modeling methods such as DES may be better suited than traditional Markov or state-transition cohort models to address the complexity and specific challenges of economic evaluation of PM interventions. Among the core advantages of simulation models are the ability for patient-level analyses of care pathways and the ability to deal with system complexity of multiple tests, diagnostic performance and testing, and treatment sequences that present challenges for PM. Although the use of simulation modeling in health is growing in general, there remain few examples in PM to date.

Another modeling approach that we have not addressed in this article and that is potentially relevant for PM is constrained optimization. Constrained optimization is a mathematical approach to finding the best solution to a problem, subject to constraints.⁴⁶ Optimization methods may be useful for designing optimal treatment pathways that are relevant in the context of PM, where a specific treatment may be used in a specific subset of patients because of their genotypic or clinical phenotypic profile. Optimization modeling approaches are specifically suited to address constraints in the system (most commonly, fixed budgets and time), which have resource allocation implications for decision makers.^46,47 An example of constrained optimization is the work by the Alliance for Paired Donation to make improved matches between kidney disease sufferers and potential kidney donors.⁴⁸ The Alliance developed and implemented an innovative technique called nonsimultaneous extended altruistic donor chains that permits better-optimized matching of potential donors to patients, greatly increasing the number of possible transplants.

Although there are benefits from applying dynamic simulation modeling methods for PM interventions, implementing these models to support policy decisions may be challenging. Dynamic simulation models are not a “one size fits all” solution, and continuing research, education, and testing of these methods is required to understand when these methods should be applied. We offer 3 suggestions regarding the application of simulation modeling in economic evaluation of PM interventions:

1. Simulation modeling methods should be considered part of the tool kit for economic evaluations in PM given the need to model a cascade of testing and treatment sequences.

2. Although simulation modeling may be an appropriate modeling approach for economic evaluation in PM, in general, models should aim to represent the decision problem and the decision context in which the results will be interpreted and applied in as simple a manner as possible.

3. Providing sufficient transparency to achieve understanding by decision makers and reflecting the robustness of the model may be an even greater challenge for simulation modeling than other types of models. Modelers should explicitly document the rationale for applying simulation modeling, the modeling assumptions, and the strength of the data used to populate the model, and they should conduct an appropriate exploration of the uncertainty around the model.