Incorporating Resource Constraints in Health Economic Evaluations: Overview and Methodological Considerations

Abstract

It is well known that healthcare resource constraints influence the capacity to deliver care, affecting both the costs and outcomes of medical interventions. If these constraints are not adequately accounted for in economic evaluations, there may be a lack of understanding regarding the full impact of implementing health technologies, leading to decisions being made with suboptimal information. This paper offers an overview of the types of healthcare resource constraints and their potential effects, and introduces a framework grounded in operations research and health economics principles, outlining the methodological considerations for incorporating resource constraints into economic evaluations. Drawing from a literature review and advisory group feedback, three categories of resource constraints were identified: single-use resource constraints, reusable resource constraints and patient throughput constraints. The proposed framework outlines a comprehensive set of steps necessary for effectively incorporating constraints into health economic evaluations and details specific approaches and methodological considerations for each stage to ensure a more accurate and realistic assessment of health interventions. This paper also aims to raise awareness among payers and decision-makers with regards to the limitations of technology evaluations in a resource-constrained health system. Specifically, it suggests that health technology assessment agencies ought to offer guidance on incorporating constraints into the submissions they receive. Moreover, it advocates for a more comprehensive economic evaluation in economic assessments to fully capture an intervention’s value.

Key Summary Points

The prevailing working assumption in conventional health economic evaluations is that all physical and human resources are instantly accessible and utilised. For many resource allocation questions, this approach needs to be rapidly shifted towards a methodology that accurately mirrors real-world resource constraints.

Failing to adequately account for resource constraints in economic evaluations may lead to a suboptimal estimate of the impact of the implementation of an intervention and provide an incomplete dataset for decision-makers.

This paper gives an overview of the types of healthcare resource constraints and their possible effects and presents a framework to support their implementation in economic evaluations based on operations research and health economics principles.

Introduction

Healthcare resource constraints (i.e. factors that limit or restrict access to healthcare resources) are an important issue, causing long waiting lists for various medical services. This issue was in the spotlight during the coronavirus disease 2019 (COVID-19) pandemic, when the lack of medical equipment (e.g. facemasks, ventilators), staff and facilities highlighted the detrimental and broad impact of constrained resources across multiple healthcare services.

Economic evaluations in healthcare typically take a long-term perspective and assume that all physical and human resources (e.g. beds, doctors, computed tomography scanners) are immediately available and consumed, regardless of actual resource constraints. Nonetheless, there is growing recognition that neglecting resource constraints in economic evaluations may have unintended deleterious consequences in many resource allocation situations [1, 2].

Although it is accepted that resource constraints could impact the costs or health outcomes of an intervention (and thus the economic evaluation), these constraints are not frequently accounted for when assessing the incremental impact and value of an intervention. Even though resource constraints can influence the number of patients being treated or cause delays in treatment that can result in lower health outcomes, these impacts are neither captured routinely in economic evaluations nor considered in most technology assessments.

Modelling resource impact (‘resource modelling’) entails estimating the physical resources needed for an intervention over time to assess whether capacity constraints can meet the resource demand. This field is relatively new, and there are a limited number of economic evaluations that have explored resource modelling [1, 2]. Because these evaluations have a variety of objectives, methods and terminology, there is a lack of clarity in this area regarding best practice.

Existing studies address the issue of resource constraints either from an operations research perspective (focussing on the impact of process outcomes such as queue size and waiting time) or from a health economic perspective (using assumptions/expert opinion, optimisation techniques and the modelling of opportunity costs to incorporate the impact of constraints on the costs and health outcomes of interventions) [1, 3]. However, these studies have been mostly theoretical exercises or conducted in an informal or narrative way [4, 5].

This paper aims to offer a comprehensive overview of the different types of resource constraints and their potential effects on process, cost and health outcomes, combining operations research and health economic perspectives. While health technology assessments (HTAs) and economic evaluations are increasingly being used in low- and middle-income countries (LMICs), this manuscript specifically focusses on the context of high-income settings. The aim is to address the unique challenges and considerations pertinent to these environments, where resource constraints may manifest differently compared with LMICs. Subsequently, this paper presents a framework outlining the methodological considerations for incorporating resource constraints into economic evaluations. This includes selection of the setting, identification of relevant constraints, selection of the modelling technique, the assessment of data requirements, the estimation of impacts and the presentation of results.

Literature Review

Methods

To support the development of the framework, a pragmatic literature review was conducted. The search, executed via PubMed on 25 May 2022, aimed to find studies related to economic evaluation that incorporated resource modelling or capacity constraints from either an operations research or health economic perspective. A detailed search strategy is available in the appendix. The primary objective of the review was to understand the work that has been done in the area thus far, identify definitions established by other authors and gather existing evidence and methodologies to support the creation of a comprehensive framework for integrating resource constraints into economic evaluations.

The data extraction process involved identifying key characteristics of each paper included in the review, including but not limited to the focus area, type of decision model, model validation, quantified constraints, sources of information and impact of constraints on economic models’ inputs and outcomes. A data extraction table summarising the characteristics is provided in the appendix.

Furthermore, a review of guidelines for health technology assessment (HTA) and the economic evaluation of health technologies was carried out to assess the recommended approaches for addressing resource constraints in economic evaluations. Our investigation focussed on guidelines from six countries: the United Kingdom, France, Canada, Australia, the Netherlands and Sweden.

The authors, who include leading experts in resource modelling, health economics and HTA, reviewed data from the selected studies and guidelines. The analysis of the data was conducted collectively by all authors through two committee meetings and several rounds of review, where opinions and interpretations were shared and discussed. They developed terminology, explored the potential effects of constraints on economic evaluations, and provided methodological considerations for incorporating capacity constraints in economic evaluations.

Results

Resource Constraints in the Literature

The search initially resulted in 641 papers, with 43 duplicates excluded. The screening of abstracts of 598 papers led to the full-text review of 43 papers. Following this full-text review, and after an additional manual search of the references of relevant articles, we identified 17 publications that met the inclusion criteria. The examined literature concentrated on four principal areas impacted by supply constraints: the functioning, organisation and optimisation of health systems; therapeutic interventions (either pharmaceutical or procedural); diagnostic methodologies; and policy. The most frequently quantified constraint was human resources (noted in 11 studies) [3, 4, 6–8, 10–14, 17], with surgical resources [8, 11] and diagnostic testing capabilities [4, 13] each addressed in two studies. Among the 17 papers reviewed, discrete event simulation emerged as the most prevalent type of decision model (used in eight papers) [1, 2, 8, 10, 11, 13–15]. Other decision model types were used infrequently; each was cited no more than three times. These include the Markov model [2, 8, 12], a composite of the Markov model and decision tree [4, 7] and a mixed model combining discrete event simulation and agent-based modelling [13]. System dynamics modelling (SDM) was used in one case to identify the existing constraints [17].

A total of 59% of the studies (10 of 17) [2–4, 6–8, 10, 11, 14, 15] evaluated the effect of resource constraints on incremental cost-effectiveness ratios (ICERs). The impact on net monetary benefit (NMB) was investigated in 18% of the studies (3 of 17) [4, 10, 11], and all of these 3 studies examined NMB along with ICER [4, 10, 11]. In a unique study [12], resource constraints were considered within the framework of a HTA submission focussed on the implementation of a national screening program. Another paper discussed the case from the perspective of a HTA submission [4], although no actual HTA submission occurred.

In all applicable studies, information on data sources for modelling constraints and the impact of constraints on the model was provided. Additionally, information on the impact of constraints on model inputs was provided in 91% of the reviewed papers, while the performance of model verification was mentioned in 67% of the studies (Table 3 in the appendix).

Table 3.

Extraction table

Reference	Area of impact	Intervention	Healthcare constraints	Resource constrains (non-financial)	Modelling Method	Impact on ICER?	Impact on NMB?	Inclusion in HTA decision-making?	Information on the impact of constraints on model inputs?	Information on data sources for modelling constraints?	Information of how constraints effect the model?	Model verification
Salleh, 2017 [1]	SLR	Systematic review	Yes	N/A	DES	No	No	No	N/A	N/A	N/A	N/A
Wright, 2019 [2]	SLR	Systematic review	Yes	N/A	MM, DT, DES	Yes	No	No	N/A	N/A	N/A	N/A
van Baal, 2018 [3]	Therapeutic	Non-robot versus robot supported surgery	Yes	Human resources	Cohort state-transition models	Yes	No	No	+	+	+	−
Wright, 2022 [4]	Therapeutic	Crizotinib	Yes	Test commissioning awareness Degree of centralisation of testing Ability to conduct testing (human + financial resources)	MM + DT	Yes	Yes	Noa	+	+	+	+
Bozzani, 2020 [6]	HC organisation	Health system functioning and optimisation	Yes	Financial constraint Human resources a Policy constraints	Model combining transmission model outputs with unit costs	Yes	No	Nob	+	+	+	−
Miquel-Cases, 2016 [7]	Therapeutic	RG-NACT with MRI for ER-positive/HER2-negative breast cancer	Yes	Implementation rates Human resources Equipment	MM + DT	Yes	No	No	+	+	+	−
Standfield, 2017 [8]	HC organisation	Usual orthopaedic care versus OPSC	Yes	Human resources Surgical resources	MM, DES, DES-DQ , DES - CAL	Yes	No	No	+	+	+	+
Fuertes-Guirò, 2018 [9]	Therapeutic	Laparoscopic cholecystectomy: conventional versus single-incision	Noc	Operating time	N/A	No	No	No	N/A	N/A	N/A	N/A
Stahl, 2004 [10]	HC organisation	Reorganise surgical & anaesthesia care	Yes	Human resources	DES	Yes	Yes	No	+	+	+	+
Standfield, 2016 [11]	HC organisation	OPSC versus traditional services with and without extra surgical resources	Yes	Human resources Surgical resources	DES-DQ	Yes	Yes	No	+	+	+	+
Sharp, 2013 [12]	Diagnostic	Screening programmes	Yes	Human resources Facilities	MM state-transition model	No	No	Yes	+	+	+	+
van de Ven, 2022 [13]	Diagnostic	Genomic testing: whole Genome Sequencing versus targeted gene panels	Yes	Number of centre Biopsy suitability Human resources	DSM, SD, DES, ABM, hybrid model (DES + ABM)	No	No	No	−	+	+	+
Crane, 2013 [14]	HC organisation & technology	Clinical pathways & medical, laser, or surgical techniques	Yes	Human resources	DES	Yesd	No	No	+	+	+	+
Jahn, 2010 [15]	Therapeutic	Stents: drug-eluting versus bare-metal	Yes	Daily stent placements	DES	Yes	No	No	+	+	+	−
Thokala, 2015 [16]	Policy	Policy (resource modelling included in HTA)	Yes	N/A	N/A	No	No	No	N/A	N/A	N/A	N/A
Bozzani, 2022 [17]	HC organisation	Health system functioning and optimisation	Yes	Human resources	SDM	No	No	No	N/A	+	+	+
Verguet, 2019 [18]	Policy	Quantifying interactions between health system components and incorporating constraints in models	Yes	Yes	N/A	No	No	No	N/A	N/A	N/A	N/A

+ information available, − information not available

ABM agent-based modelling, DES discrete event simulation, DES-CAL discrete-event simulation calibration version, DES-DQ discrete-event simulation with dynamic queuing, DSM dynamic simulation modelling, DT decision tree, N/A not applicable, MM Markov model, OPSC orthopaedic physiotherapy screening clinic and multidisciplinary treatment service, SD system dynamics, SDM system dynamics modelling technique, SLR systematic literature review

^aOn the basis of an HTA submission, ^bPolicy focus, ^cOpportunity costs of resources utilisation, ^dBetween scenarios

Resource Constraints in Health Technology Assessment and Economic Evaluation Guidelines

A review of guidelines from six countries—the United Kingdom, France, Canada, Australia, the Netherlands and Sweden—was performed [5, 19–27]. Our analysis explored how these guidelines integrate resource constraint considerations, examined the clarity with which these constraints are acknowledged in economic evaluations and identified the strategies recommended for their management. It was observed that half of the guidelines—those from the United Kingdom, France and Canada—explicitly recognise and tackle the issue of resource constraints in economic evaluations [5, 19–22]. In contrast, the guidelines from Australia, the Netherlands and Sweden are not explicit about resource constraints in their economic evaluation frameworks [23–27].

The Canadian Agency for Drugs and Technologies in Health (CADTH) recognises that resources may vary across different health interventions or programs. CADTH assesses the system’s ability to accommodate the new technology, considering factors related to organisational feasibility (such as operational elements and human resources). These factors, which could either facilitate or hinder the new technology’s implementation, might lead to a CADTH recommendation with a reimbursement condition [21]. In its economic evaluation guidelines, CADTH calls for a thorough identification and analysis of resources related to the implementation, operation, maintenance and repair of the new intervention, as well as staffing and training needs. The need to account for capacity issues that may affect existing infrastructure is also highlighted. Additionally, it is mentioned that the choice of modelling technique may need to consider the model’s ability to incorporate competition for constrained resources and the capacity to develop waiting lists or queues [22].

Similarly, the French HTA authority, Haute Autorité de santé (HAS), advises that costs linked to organisational changes should be included in evaluations across the entire time horizon. This also includes additional expenditures required for the transition from the prevailing situation to that in which the intervention becomes routine. The coexistence of several interventions during the build-up of the new intervention should also be accounted for. Moreover, HAS provides guidance on modelling choices, stipulating that they should accommodate resource constraints where they exist, allowing for stochastic outcomes and interactions, particularly if these constraints may delay interventions and thereby impact costs or health outcomes [20].

In the United Kingdom, the National Institute for Health and Care Excellence (NICE) recommends the integration of all additional costs associated with interventions into the reference case of economic evaluations. If changes in practice or service delivery in the National Health Service are necessary, these should be included in a separate non-reference-case analysis. NICE also emphasises the importance of highlighting potential resource constraints that could influence the timeline for implementing health technologies [19].

Comprehensive Overview of Resource Constraints

Approaches to Estimating the Impact of Resource Constraints

Broadly, approaches to estimating the impact of resource constraints can be considered either qualitative or quantitative. Qualitative approaches are used in situations in which identification of the existence of resource constraints and their potential impact might be sufficient [28]. Following this approach, potential health and economic impacts are described in a qualitative manner, and the impact of the constraints on costs and health outcomes can be assumed, without explicitly modelling the constraints or their impact. Quantitative approaches are applied in situations where the consequence of the resource constraints is estimated, allowing for assessment of the constraints’ impact on the healthcare system [29]. In this approach, resource constraints are incorporated in economic models to assess their impact on the healthcare system and on the costs, quality-adjusted life year (QALY) outcomes or economic assessment of the intervention.

Although a qualitative approach can be used, where neither the constraints nor their impacts are explicitly estimated but instead are considered in a deliberative manner, this approach provides limited information for decision-makers. However, we acknowledge that in situations where quantification is difficult, such as in low- and middle-income countries where data availability may be more of a challenge, qualitative approaches can still provide valuable information. This manuscript primarily focusses on quantitative approaches and provides considerations for integrating resource constraints into economic models to inform decision-makers in high-income settings.

Categories of Resource Constraints

Informed by the literature review and on the basis of the feedback provided by the advisory group, we identified three categories of resource constraints: single-use resource constraints, reusable resource constraints and patient throughput constraints (see Fig. 1). These categories can be understood through two primary approaches for modelling resource constraints: explicitly quantifying physical constraints and implicitly estimating their impacts.

Fig. 1.

Overview of the categories of constraints. MRI magnetic resonance imaging

Explicit Quantification of Physical Constraints

This approach involves directly estimating the constraints on specific resources, such as single-use and reusable resources:

• Single-use resources: Also known as one-off resources, which are defined as items that can be used only once, such as pharmaceuticals, reagents for diagnostic tests, disposable tests and equipment such as syringes, plasters or masks.

• Reusable resources: Also called occupied resources, which are defined as resources that are occupied for a given period and can be redeployed, such as staff (e.g. doctors, nurses, consultants) and equipment (e.g. hospital beds, ambulances, scanners). Constraints on these resources can lead to delays in treatment, limit the number of patients receiving the intervention and impact overall patient outcomes.

Implicit Estimation Using Patient Throughput

This approach uses patient throughput as a proxy to estimate the potential effect of resource constraints without explicitly modelling the specific resources:

• Patient throughput constraints: Defined as the limited throughput of patients in an intervention, such as a certain number of patient appointments per day. This method reflects the cumulative impact of various resource constraints on the healthcare system’s capacity to treat patients.

It is important to note that this classification is focused on service-level, physical resource constraints. However, we acknowledge that above-service level constraints, such as political or policy considerations, also exist and can impact resource allocation.

Impact of Resource Constraints

Understanding the plausible impacts of resource constraints on healthcare delivery is crucial for devising effective management strategies. These impacts can vary considerably, affecting patient care and the efficiency of healthcare systems. The identification of these impacts is as essential as recognising the constraints themselves. Some potential effects of resource constraints are listed below:

• Delayed treatment: Resource constraints often lead to longer waiting times for patients, delaying essential treatments. This delay can adversely affect patient outcomes, increasing the severity of conditions, and in some cases, leading to increased mortality rates

• Diversion of resources: To manage constraints effectively, resources may be diverted from one area to another, prioritising certain treatments or services over others. While this can be an effective short-term strategy, it may lead to underservice in the areas from which resources are diverted

• Task shifting: Particularly in specialised fields such as ophthalmology, task shifting – reallocating tasks to healthcare professionals who do not traditionally perform them—can mitigate the impacts of resource constraints. For example, training nurses to administer intra-vitreal injections can alleviate the burden on ophthalmologists, improving patient throughput. However, this approach requires careful consideration of training and quality assurance to maintain high standards of care

• Use of the next-best alternative: In scenarios where the optimal intervention cannot be provided due to constraints, healthcare providers may resort to alternative treatments, procedures or facilities. While this allows for continued care, it may not offer the same level of safety and/or efficacy, potentially leading to suboptimal health outcomes or compromising patients’ safety

• Ethical and equity concerns: Resource constraints often necessitate prioritisation decisions, which can raise ethical and equity concerns. Deciding who gets access to limited resources can highlight or exacerbate existing inequalities in healthcare access and outcomes

Each of these impacts can have profound implications for the healthcare system, affecting not only patient outcomes, but also the economic efficiency of healthcare delivery. The early identification and explicit acknowledgment of these impacts in economic models is paramount; it enables decision-makers to better understand the full scope of resource constraints and the consequences of implementing a new technology, and they can consider appropriate corrective actions or adjustments in policy and practice to mitigate adverse effects.

Modelling Resource Constraints

We propose a taxonomy of the potential impact of constraints on the basis of their impact on healthcare systems, costs, QALYs and economic evaluations.

The impact of resource constraints can be modelled in two ways: the impact on the healthcare system’s production process can be estimated as the starting point, which can subsequently affect the intervention outcomes, such as QALYs, costs or other relevant outcomes of an economic evaluation; or the impact on costs and QALYs can be directly estimated, without considering the resource constraints’ impact on the healthcare system’s production process (Fig. 2).

Fig. 2.

Taxonomy of the potential impact of constraints. ICER incremental cost-effectiveness ratio, NHB net health benefit, NMB net monetary benefit, QALY quality-adjusted life year. ^aNot modelling constraints explicitly but modelling their impact directly on costs or QALYs, ^bPopulation-level

When constraints are explicitly estimated for single-use and reusable resources, their impact first manifests as a process impact on the production process (or function) of the healthcare system (e.g. fewer patients receiving the intervention, increased delays to treatment, longer waiting lists).

These process impacts can be used to model the health and cost impacts of the constraints, depending on whether there is any corrective action in place. For example, in the absence of corrective action, the impact of delayed treatment can be estimated as a lost utility (or even, in case of severe diseases such as cancer, length of life) without modelling the costs. Alternatively, it can be assumed that patients with delayed treatment are transferred to premium out-of-hours/outsourced clinics (corrective action), resulting in increased costs but no loss in QALYs. Other corrective actions might involve increasing system capacity (increasing costs and QALYs for the intervention) or transferring some patients to the comparator(s). Constraints that impact the estimated cost or the estimated benefit per patient receiving the intervention have also been called quality-limiting capacity constraints [4].

For patient throughput constraints, if it is assumed that all participants receive either the intervention or the comparator, there is no impact on patient-level costs or QALYs (and therefore in the ICER). However, the impact on costs and QALYs can be estimated at the population level (population-level NMB or NHB). This type of constraint (also known as an access-limiting capacity constraint), which reduces the number of patients who receive an intervention, will influence the population-level NMB or NHB of the intervention. Despite previous studies suggesting that these constraints will not affect the ICER [4], they can affect the ICER in a less obvious way: in a situation with a partial intervention arm (i.e. some patients receive the intervention and others the comparator), if fewer people have access to the new intervention, then the QALYs and costs will be more weighted to those produced by the comparator; with fewer constraints (more access), QALYs and costs will be more weighted to the intervention arm.

When the impact of resource constraints on costs/QALYs is estimated directly, using assumptions such as changing costs or quality of life gains because of uptake, the impact on costs, QALYs and cost-effectiveness results can be assessed. However, the impact of the resource constraints on the healthcare system is not estimated or assessed.

It is important to acknowledge that resource constraints can be time sensitive and temporary, as seen in cases such as genomic testing, and this dynamic nature, along with the evolving roles of staff, task shifting and telemedicine, needs to be accurately reflected in economic models to address the temporal impact of these constraints.

Framework for Incorporating Capacity Constraints in Economic Evaluations

A framework proposed for incorporating capacity constraints into economic evaluations is summarised in Table 1, and methodological considerations for each of the steps are described in detail in the subsections below. This framework is designed with a focus on high-income settings, reflecting the specific resource allocation challenges and health system structures typical of these environments. The framework proposed in this manuscript was developed on the basis of the collective experience and expertise of the authors in operations research and health economics. It aims to provide guidance for integrating resource constraints into economic evaluations, recognising that this is an emerging and evolving field. The iterative development process involved two initial workshops with all authors, where the majority attended in person and the rest participated online. Key perspectives and the manuscript’s foundation were agreed upon during these sessions. This was followed by several rounds of offline reviews, with additional comments incorporated after consensus was reached through a collaborative exchange of authors ideas using various communication tools, including document comments and emails, ensuring a comprehensive consideration of various perspectives and refinements.

Table 1.

Framework for incorporating constraints in economic evaluations [4, 10–13]

Step	Description
Choosing the setting	Decide whether the constraints should be estimated at the national level or at local organisational level
Identifying and selecting the constraints	Identify all relevant constraints and select which constraints should be estimated (and which constraints can be ignored or dealt with quantitatively)
Choosing the modelling technique	Determine whether traditional economic evaluation modelling techniques are adequate or if there is a need for alternative techniques
Selecting data for the constraints	Determine whether existing intervention data or data from published literature can be used to parameterise the constraints or whether additional data are needed (e.g. expert opinion, primary research, hospital database, claims data)
Estimating the process impacts	Understand how constraints manifest as process impacts in the health system
Estimating the impact on QALYs	Estimate the impact of constraints on QALYs either directly or by linking the effect of process impacts on health outcomes
Estimating the impact on costs	Estimate the impact of constraints on costs either directly or by linking the effect of process impacts on costs
Performing sensitivity analyses and model validation	Describe any sensitivity analyses performed and how the model results are validated
Incorporating into economic evaluation	Determine whether the process, cost and QALY impacts are quantitatively incorporated into ICER/NMB/NHB calculations or considered qualitatively

It is important to emphasise that the steps can be performed in a different sequence; the process of resource modelling is iterative rather than a strictly sequential set of steps.

In situations where the resource constraints are not estimated explicitly, it is also possible to use a subset of steps in the framework. By skipping steps 4 and 5, the impact of the constraints on costs and QALYs can be assumed directly in steps 6 and 7 without modelling the constraints or their impact on the healthcare system. Similarly, qualitative approaches can involve conducting steps 1, 2 and 9 (i.e. skipping steps 3–8) where the impact of constraints on the healthcare system is described qualitatively.

This framework is designed for situations where the resource constraints are estimated explicitly (i.e. those outlined earlier as quantitative approaches). It further assumes that the typical economic evaluation considerations, including treatment alternatives, population, outcomes and perspective, are predefined. Thus, it solely focusses on the additional factors required for integrating resource constraints into economic models. In cases where data are not available in published literature, expert input should be considered to ensure that the estimations and assumptions are accurate.

Step 1: Choosing the Setting (Local Versus National)

The selection of the setting for estimating constraints, whether at a national or local organisation level, should depend on the targeted decision-maker or budget holder in combination with the feasibility of estimating the resource constraints’ impact at local, regional or national levels.

When resource constraints are at a national level (e.g. capacity for cancer screening), modelling can be done at a national level, and data can be generalised for local settings. A local approach can provide more robust results that are relevant to that specific healthcare setting and can be verified by comparison with the data in that setting. However, such results may not be easily generalised to the national level, and their usability for a traditional national HTA will be limited. A nationwide approach might involve identifying and developing models for key representative setting types (e.g. small hospitals, mid-sized hospitals, large hospitals). If the patient pathway is similar between settings and the differences can be captured as parameters (e.g. the number of beds), a single, generic model can be developed, which can be amended to represent the different settings. It is important to note that selecting the base case is necessary, and expert opinion should drive this selection.

Step 2: Identifying and Selecting the Constraints

Healthcare systems involve a wide array of resources, each with their own potential for constraints that can affect overall efficiency and effectiveness. The challenge lies in distinguishing which of these constraints should be quantitatively estimated and which can be managed qualitatively.

This process involves two stages: first, the identification of potential constraints in the healthcare system; and second, the careful selection of those constraints that are most likely to influence the cost-effectiveness of any given intervention.

To identify resource constraints in healthcare systems, it can be beneficial to apply implementation science frameworks, such as the Consolidated Framework for Implementation Research [30]. These frameworks offer a structured approach to analysing various factors influencing the implementation of healthcare interventions, facilitating a comprehensive identification of potential resource constraints.

A large number of potential constraints are likely to be identified in an initial search. Because of modelling limitations, it is unlikely to be possible to model the impact of every constraint, and many of the constraints are unlikely to have a meaningful impact and do not need to be estimated. It has been suggested that the modelling of constraints could be useful when (1) a new technology needs considerable additional resources compared with the current situation; (2) the new technology is reliant on occupied resources, or the occupied resources related to the new technology are specialised; (3) specific resources are known to be in short supply; or (4) the new technology has a positive effect on already existing queues [16].

To narrow down the number of constraints to be modelled, a process of constraint selection must be undertaken. One potential approach for resource constraint selection is scenario drafting, which is a method used to create and analyse various plausible future scenarios, especially in situations with high uncertainty, to aid in decision-making and strategic planning [31].

In both stages—identification and selection of resource constraints—it is important to seek opinions from clinical experts, as well as from the professionals responsible for the running of the services, to understand and validate the key resource constraints involved in an intervention. Methods such as the Delphi technique, structured interviews, and nominal group techniques can be used to effectively gather and synthetise experts’ opinion. Attention should be directed to resources facing constraints that are not transitory in nature but will span an extended period (at least the medium term).

As previously mentioned, sometimes multiple resources can be combined into a single unit of resource constraint. For example, a clinic visit can be used as a resource constraint, encompassing various resources, such as doctors, nurses, space and diagnostic services.

Step 3: Choice of Modelling Technique

The choice of modelling technique is linked to the type of resource constraints. For single-use and patient throughput resource constraints, it is possible to link the health states to resources using static traditional economic evaluations cohort modelling techniques (such as semi-Markov models and decision trees) to estimate the overall resources required and understand whether there is enough capacity. Advanced simulation techniques, such as discrete event simulation or system dynamics modelling, can also be used for these constraints, but the simplest model possible for each situation should be used where possible.

For reusable resources, advanced simulation techniques are required. These techniques help comprehend resource availability fluctuations and estimate whether there is adequate capacity to accommodate the rate of demand (e.g. patient arrival rate) and occupancy duration (e.g. hospital length of stay) [1]. Discrete event simulation models are more flexible than cohort models because they allow for the modelling of alternative patient pathways or management of waiting lists subject to pre-defined characteristics while providing more detailed results (e.g. queue size/waiting time) [32]. Additionally, SDM can be used to identify and understand constraints within the healthcare system. SDM explores interactions and feedback loops within the system, helping to uncover how constraints in one area may impact others. This approach is useful for identifying constraints that may not be immediately apparent and understanding their potential long-term effects on system performance.

Step 4: Selecting Data for the Constraints

For single-use resources and patient throughput, the constraints are included in the model directly. Reusable resources (e.g. beds) will require additional data, such as availability (e.g. schedules/shift times), patient arrival rate, and time of occupancy (e.g. length of stay). For resources with multiple uses (e.g. magnetic resonance imaging scanners), the use of the resource in ways unrelated to the modelled intervention may also need to be considered.

Published literature, if available, can be used as a data source but needs to be checked for face validity. If no data are available, primary research or expert opinion could be used to parameterise the constraints. For patient throughput constraints, it is important to carefully consider the data and check the rationale behind the assumptions to ensure that throughput used has face validity.

Step 5: Estimating process impacts on the healthcare system

As described earlier (Fig. 2 Taxonomy of the potential impact of constraints), the impact of the constraints manifests as an effect on the production process of the healthcare system, such as fewer patients receiving the intervention, increased delays in treatment or longer queues/higher waiting times. For single-use resource constraints or patient throughput constraints, these service impacts can be modelled in the short or long term. However, for reusable resource constraints, these process impacts are always short term; potential organisational changes, treatment pathway changes or corrective actions cause uncertainty in predicting long-term process impacts.

In addition, it is possible to directly model the impact of constraints on costs and QALYs using assumptions (e.g. changing the cost or changing the quality-of-life gains because of uptake); however, this option does not consider the resource constraints’ impact on healthcare system operations.

Step 6: Estimating the Impact on Quality-Adjusted Life Years

The impact of resource constraints on QALYs can be estimated by modelling their effect on the production process of the healthcare system. As previously mentioned, in some situations the impact of the constraints on QALYs can be assumed directly without modelling the impact of the constraints on the healthcare system.

The impact on QALYs depends on the type of process impact of the constraints. For reusable resource constraints, the impact of treatment delay can be estimated as reduced life expectancy and/or lost utility on the basis of the clinical condition. However, single-use constraints or patient throughput constraints can place a limit on the number of patients treated with the intervention, which can be estimated as a net loss in the NMB. This overlaps somewhat with the concepts around the value of implementation [33].

Step 7: Estimating the Impact on Costs

The impact of constraints on costs, sometimes reflected in charges or distorted prices, depends on the type of production process impact the constraints have and the available corrective actions.

Although estimating the cost and impacts of constraints using general equilibrium theory or opportunity costs is a complex task requiring detailed data and extensive time for analysis, it should not be dismissed as unfeasible. Indeed, the challenges involved, such as the need for granular data and the difficulty of estimating opportunity costs, make it a particularly intriguing subject for in-depth study. See Section 6.3 for further discussion about this issue.

Estimating the opportunity cost is also challenging because assumptions need to be made about the use of free resources. For example, if there is a day bed free, we need to make assumptions about the likelihood that it would be used by a sick patient or by a recovered patient who could not travel home that day.

As such, the cost impacts can be estimated by considering the unit costs of resources and the liquidity of the resources (e.g. assuming patients with delayed treatment can be transferred to premium out-of-hours/outsourced clinics, which results in increased costs). Cost impacts of other corrective actions, such as an increase in capacity, could be estimated using a bottom-up approach or by approximation using unit costs, along with assumptions about any increased costs.

In some situations, the impact of resource constraints on costs can be assumed directly (e.g. changing the cost) without modelling the impact of the constraints on the healthcare system.

Step 8: Sensitivity Analyses and Model Validation

Standard tools used in cost-effectiveness models to characterise parametric uncertainty (such as deterministic or probabilistic sensitivity analysis [PSA] or scenario analyses) can also be used to understand the uncertainty around the impact of constraints. For example, assessing the impact of small increases in constraints would be useful.

For model validation, it is first important to compare the results of modelling without constraints with any similar models to ensure that the model has face validity. Models with constraints can be validated by reducing the inputted constraints to zero and comparing them with the model without constraints.

Step 9: Incorporation into Economic Evaluations

Methodologically, there are different ways in which constraints could be incorporated into economic evaluations.

One option is to model the effect of constraints on costs and QALYs quantitatively and incorporate them directly into the estimation of the cost-effectiveness of an intervention (e.g. ICER). This option would provide information regarding the impact of constraints on the outcomes of an intervention. Estimating the outcomes of the intervention (e.g. costs, QALYs) without modelling the constraints might introduce more uncertainty into the results because of the assumptions required, which can reduce the robustness of the information provided.

In situations where the constraints impact the ICER in a less obvious way, such as when modelling patient throughput constraints, it is important to consider outcomes other than ICER, such as the population-level NMB or population-level ICER.

Alternatively, in situations in which the resource constraints are not estimated explicitly, thus not contributing to the economic assessment, the impact of constraints on the healthcare system outcomes (health and costs) can be described qualitatively.

Case Example Overview

To provide a practical context for the steps outlined in the framework, we present an overview of an example [4] to illustrate how each step in the framework can be addressed. This example focusses on a NICE single technology appraisal assessing the cost-effectiveness of crizotinib for patients with ALK mutation-positive non-small-cell lung cancer in comparison with standard chemotherapy (docetaxel) [34]. An existing model was used to evaluate the cost-effectiveness of crizotinib with and without capacity constraints.

Step 1: Choosing the Setting

The first step involves deciding whether the constraints should be estimated at the national level or at the local organisational level. In our example, the impact of the intervention was assessed at a national level (the United Kingdom) [4].

Step 2: Identifying and Selecting the Constraints

The next step is to identify all relevant constraints and select which constraints should be quantitatively estimated. This process begins with the initial identification of potential constraints, followed by a careful selection of those most likely to influence the cost-effectiveness of the intervention. For the crizotinib example, the capacity constraints for ALK mutation testing implementation were identified through semi-structured telephone interviews with ten clinical experts (medical oncologists, pathologists, geneticists) experienced in precision medicine for non-small-cell lung cancer [4, 35]. The key constraints to be modelled were selected on the basis of the opinions of the research team, although it was highlighted in the limitations of the paper that a more robust method for prioritising the constraints was required. These constraints were a lack of provision of ALK testing due to uncertainty in reimbursement, the localisation of testing leading to lower-quality testing and a lack of capacity in pathology laboratory staffing.

Step 3: Choice of Modelling Technique

Selecting the right modelling technique is essential and depends on the specific resource constraints being addressed. It is necessary to evaluate whether traditional economic evaluation modelling techniques suffice or if alternative methods are required. In the crizotinib example, the authors’ model sought to replicate the model submitted by the manufacturer of crizotinib for evaluation by NICE [4]. To account for the ALK testing pathway, where the constraints would have an impact on the care pathway, a custom decision tree was added to the Markov model used in the manufacturer submission.

The resource constraints were modelled by adding additional branches to the decision tree. The lack of reimbursement awareness was reflected in the possibility that patients would not be offered the test and instead be given standard chemotherapy. Separate branches were added such that some patients received localised and some centralised testing, with different costs and outcomes associated with the location of testing. Deficiencies in both locations of testing were assumed to be a product of understaffing, and it was assumed that the costs would be lower and benefits higher in scenarios where staffing could be improved.

Step 4: Selecting Data for the Constraints

Choosing suitable data to parameterise the constraints is critical. It is important to assess whether existing intervention data or data from published literature can be used, or whether additional data are required. Within our case example, focussed searches, using the Ovid search tool in the Embase and MEDLINE databases along with key search terms, were conducted to identify constraint values [4]. Additionally, grey literature reports, including the National Lung Cancer Audit and Cancer Research UK reports, were consulted to gather potential parameter values.

Data were identified to represent the probability that an individual would receive testing, the probability of their test being processed at local or centralised laboratory and the degree of pathology laboratory understaffing.

Step 5: Estimating Process Impacts on the Healthcare System

Estimating the process impacts involves understanding how constraints manifest as effects on the healthcare system's production processes, such as fewer patients receiving the intervention, increased delays in treatment or longer waiting times. In the crizotinib example, for the identified constraints (lack of awareness by commissioning, degree of centralisation of immunohistochemistry testing [locally or central] and level of laboratory capacity), the impact on the health system, costs, QALYs and ICER/total NMB was estimated [4]. The expected loss in total NMB arising from each constraint was estimated separately and then for all the constraints combined.

Step 6: Estimating the Impact on Quality-Adjusted Life Years

The impact of resource constraints on QALYs can be modelled by understanding their effect on the healthcare system’s production processes. For the crizotinib example, individual capacity constraints, such as the lack of awareness in testing commissioning (impacting patient throughput), considerably affected patients’ QALYs but had a smaller influence on the intervention’s ICER [4]. Nonetheless, this constraint resulted in an annual NMB shortfall of approximately £5 million.

Step 7: Estimating the Impact on Costs

The impact of constraints on costs can be estimated either directly or by linking the effect of process impacts on costs. This involves considering the type of production process impact and the available corrective actions. In the presented example, centralised labs offered better quality but could lead to treatment delays due to delayed test results and additional appointments needed [4]. The procedure for centralised ALK testing was estimated to incur an additional cost of £101 per patient on the basis of the cost of additional oncologist appointments. The cost of immunohistochemistry testing at local level was assumed 17% higher than centralised testing costs on the basis of a study of the cost of pathology laboratories.

Step 8: Sensitivity Analyses and Model Validation

Sensitivity analyses and model validation are crucial to understanding the uncertainty around the impact of constraints. In the crizotinib example, PSA addressed parameter uncertainty related to the estimated impact of capacity constraints [4]. The model validation in this example was primarily technical, aiming to verify the accuracy and correctness of the model when operated in the same way as the manufacturer’s model. This involved checking for any errors in the model’s execution. Additionally, it incorporated an element of convergent validity, where the objective was to replicate the results achieved by the original model—in this case, the one developed by the manufacturer. This focus on technical precision and outcome replication was crucial for assessing the model’s reliability.

Step 9: Incorporation into Economic Evaluations

Finally, incorporating constraints into economic evaluations can be done by quantitatively modelling costs and QALYs or by describing the impact qualitatively. The case example incorporated the impact of constraints by quantitatively modelling costs and QALYs [4]. These quantifications were then integrated into the estimation of the economic impact, specifically measuring the impact on ICER and NMB.

Further Considerations

How to Consider the Impact of Constraints?

The most robust way to incorporate the impact of constraints is to model their impact on the production process of the healthcare system as a starting point. This approach will provide detailed information about how the use of a technology will affect resource constraints and the impact of implementing (or not implementing) potential corrective actions. In comparison, directly modelling the cost and QALY impact without modelling the constraints requires less time and effort but may not provide an accurate picture of the impact because this method is based on assumptions.

A qualitative approach does not measure any of the impacts of constraints; however, it can be useful to provide some qualitative information on constraints not selected to be included in the model.

When to Consider Resource Constraint Modelling?

Resource constraint modelling is likely to add considerable time and effort to the modelling process, especially when using advanced modelling techniques such as discrete event simulation. As such, resource constraint modelling should be considered at the time of research commissioning and should only be used when there is a clear need.

Selecting data sources and validation methods strategically is key to minimising added uncertainty in model results. This involves choosing credible data sources and robust validation techniques for accuracy. Given the frequent limitations of data quality, employing expert elicitation and PSA with broad distributions is vital. Such strategies effectively manage data uncertainties, enhancing the model’s reliability.

How to Assess the Opportunity Cost of Resources?

Economic evaluations can adopt both alternative perspectives—a healthcare system perspective versus a broader societal perspective [36]—and alternative aims in terms of being normative (recommending what should be done to maximise an objective function) or positive (or descriptive): trying to explain how specific stakeholders are behaving or why they are behaving as they are [37].

For example, an emergency department is a monopoly owner of a scarce resource to be rationed, and it would be in a position to raise prices to allocate the scarce supply. That price would be the relevant price for the short-term allocation of the scarce resources. However, in terms of planning for future capacity from a societal perspective, it would make more sense to estimate and use the true societal opportunity costs of providing additional units of the resource.

The general theory of the second best says that [38], if prices are distorted from true societal opportunity costs, analysts taking the long-term societal perspective should no longer necessarily follow a simple comparison of marginal benefit versus marginal cost—as is implicit in standard cost-effectiveness analysis. Adjustments would be needed; for example, see Garrison et al. [39] on adjusting for a very high-cost standard of care for a haemophilia gene therapy. Instead, an analyst should attempt to estimate the shadow price or societal opportunity cost of the resources and then use that estimate for resource allocation and planning from a normative societal perspective. Operations research simulation models of production processes and cost functions could be used to estimate these shadow prices.

How to Incorporate the Effects of Constraints into Economic Evaluations?

The review of economic evaluation guidelines in six countries (Australia, Canada, France, the Netherlands, Sweden and the United Kingdom) revealed that, although some HTA agencies (CADTH, HAS and NICE) do specify that resource constraints should be taken into consideration, there is no standard method for formally incorporating the constraints into economic evaluations.

Highlighting the potential health or economic impacts qualitatively can be beneficial; however, information regarding the impact of constraints on the healthcare system or intervention outcomes is limited. Consequently, this approach may have a low impact on decisions made at a national HTA level.

The approach that provides the most valuable information to decision-makers is the incorporation of the impact of constraints quantitatively in economic evaluations. This approach is useful to explore why investing in overcoming these constraints could be advantageous. For instance, an intervention might not initially appear cost-effective in the current health system context, but investment in mitigating these constraints could unlock considerable benefits. To evaluate these potential investments, the value of implementation methods can be applied.

However, as described in Fig. 2 Taxonomy of the potential impact of constraints, the inclusion of resource constraints directly into the model and/or modelling their impact on outcomes (costs, QALYs), and the economic outcomes assessed (ICER, NMB or NHB), should be tailored to each type of constraint.

In addition, the preference of HTA agencies and those using economic evaluations for the use of simpler models, rather than complex models such as discrete event simulation, could be a barrier to modelling reusable resources.

Finally, it is important to recognise that, if the new therapy alleviates existing queues or generally liberates constrained resources, this will have implications beyond the direct comparison between the new therapy and its comparator. In such cases, the ideal approach would be to account for the broader impact on the entire population relying on these constrained resources. It is advisable to conduct subsequent studies to validate predicted impacts and discover new ones.

Conclusions

Explicit resource constraints are not routinely incorporated into health economic models, and this may result in a lack of understanding of the full impact of adopting new health technologies, leading to suboptimal decisions. This paper provides a comprehensive overview of the different types of healthcare resource constraints and the potential impact of these constraints (on the healthcare system, costs and health outcomes of interventions). Our aim is to stimulate discussion on when and how resource constraints should be incorporated into economic evaluations. Additionally, we suggest that there is a need for HTA agencies to provide more guidance on the inclusion of constraints—or adjustments for them—in HTA submissions. The use of a broader economic assessment in economic evaluations that estimate NMB or NHB, in which resource constraints are considered, will facilitate full assessment of the value of an intervention and enable a more informed decision-making process.

Appendix

In the pragmatic literature review, a PubMed search (Table 2) was conducted using key terms to identify relevant publications. A manual search was performed to capture relevant publications not captured in the previous steps, as shown in the PRISMA flow diagram (Fig. 3). The data extraction table is presented on Table 3 .

Table 2.

PubMed search strategy

Search terms	Cost-effectiveness OR cost-utility OR economic evaluations OR cost benefit OR net monetary benefit OR net benefit OR budget impact Capacity limitation* OR capacity constraint* OR supply limitation* OR supply constraint* OR opportunity cost* OR resource limitation OR resource constraint* OR resource capacity OR treatment capacity OR service capacity OR resource modelling
Time period covered in the research	No restriction
Scope markets	No restriction
Selected disease areas	No restriction
Inclusion criteria	Healthcare constraints: physical resource (non-financial) Inclusion of resource constraints in economic models

Fig. 3.

Pragmatic literature review PRISMA flow diagram

Author Contributions

Concept and design: PT Acquisition of data: HD, RP and SW Analysis and interpretation of data: HD and RP Drafting of the manuscript: PT, HD and RP Critical revision of the paper for important intellectual content: LG, SW, DH, ID-Z, PL and GM Supervision: PT and LG All authors read and approved the final manuscript.

Funding

This research was funded by F. Hoffmann-La Roche AG. Authors received consultancy fees for their contribution to the research leading to this article.

Data Availability

Not applicable.

Code Availability

Not applicable.

Declarations

Conflict of Interest

Henrique Duarte and Roelien Postema are full-time employees of Putnam Associates. Gerardo Machnicki is a full-time employee of F. Hoffmann-La Roche, Ltd. Praveen Thokala is the director of PT Health Economics Ltd and has been paid consultancy fees by Putnam/PHMR Ltd to advise on this study. He has also performed paid consultancy in the past with IQVIA, RTI-HS, Roche, Pfizer, Daiichi Sankyo, Exact Sciences, Abbvie, Novo Nordisk and Novartis. Stuart Wright is supported by a Wellcome Trust Early Career Award (226922/Z/23/Z). Don Husereau has received fees from F. Hoffmann-La Roche AG for unrelated projects. Isabelle Durand-Zaleski reports fees for speaking engagements and advisory board participation from BMS, Janssen, MSD, NovoNordisk and Hoffmann-La Roche AG for unrelated projects. Peter Lindgren reports institutional grants from AstraZeneca, Biogen, Jansen, MSD and Novartis and fees for speaking engagements and advisory board participation from Astellas, AstraZeneca, Biomarin, Daiichi Sankyo, Jansen, MSD, NovoNordisk, SOBI, Takeda and Vertex. Lou Garrison has received consulting and speaking fees unrelated to this project from Pfizer, Novartis Gene Therapy, Roche, Merck, J&J and BioMarin.

Ethics Approval

Not applicable.

Consent to Participate

Not applicable.

Consent for Publication (From Patients/Participants)

Not applicable.