Table 2.
Example of the application of the adapted EVIDEM MCDA framework to appraising a hypothetical intervention in two contexts: ‘population priorities’ not explicitly established and thus not part of the quantitative model, and ‘population priorities’ established and thus part of the quantitative model
| Domains | Domain weights | Criteria | Criteria weights | Normalized criteria weights | ||||
|---|---|---|---|---|---|---|---|---|
| No explicit priorities | Priorities established | No explicit priorities | Priorities established | |||||
| (a) Relative hierarchical weighting of the domains and criteria of the quantitative MCDA model (independent of the hypothetical intervention), representing the value system and trade-offs of the committee | ||||||||
| Normative universal criteria | ||||||||
| Need for intervention | 30 | 30 | Disease severity | 30 | 0.09 | 0.09 | ||
| Size of affected population | 20 | 0.06 | 0.06 | |||||
| Unmet needs | 50 | 0.15 | 0.15 | |||||
| Comparative outcomes of intervention | 30 | 28 | Comparative effectiveness | 50 | 0.15 | 0.14 | ||
| Comparative safety/tolerability | 30 | 0.09 | 0.08 | |||||
| Comparative patient-perceived health/PROs | 20 | 0.06 | 0.06 | |||||
| Type of benefit of intervention | 10 | 10 | Type of preventive benefit | 30 | 0.03 | 0.03 | ||
| Type of therapeutic benefit | 70 | 0.07 | 0.07 | |||||
| Economic consequences of intervention | 20 | 17 | Comparative cost consequences—cost of intervention | 50 | 0.10 | 0.09 | ||
| Comparative cost consequences—other medical costs | 25 | 0.05 | 0.04 | |||||
| Comparative cost consequences—nonmedical costs | 25 | 0.05 | 0.04 | |||||
| Knowledge on intervention | 10 | 5 | Quality of evidence | 50 | 0.05 | 0.03 | ||
| Expert consensus/CPGs | 50 | 0.05 | 0.03 | |||||
| Contextual criteria | Country policy | |||||||
| Population priorities | – | 10a | Rare diseases | Second priority of national health | 40a | Assessed qualitatively | 0.04 | |
| Other priorities | Cancer and mental disorders first and third priority | 60a | 0.06 | |||||
| Sum | 100 | 100 | – | 1.00 | 1.00 | |||
| Domains | Criteria | Evidence on intervention X for condition Y | Performance scoresb | Value contributionsc | |
|---|---|---|---|---|---|
| No explicit priorities | Priorities established | ||||
| (b) Assessment of hypothetical intervention X for condition Y using above weights | |||||
| Normative Universal Criteria (Quantitative Assessment) | |||||
| Need for intervention | Disease severity | Y causes severe disability, reduces life expectancy and patients’ and caregivers’ QoL | 4 | 0.07 | 0.07 |
| Size of affected population | Prevalence of Y: 4/10,000 (rare) | 1 | 0.01 | 0.01 | |
| Unmet needs | Current interventions have low efficacy and affect patients’ QoL | 4 | 0.12 | 0.12 | |
| Comparative outcomes of intervention | Comparative effectiveness | Disability delayed by 2 years; no data on impact on survival | 3b | 0.09 | 0.08 |
| Comparative safety/tolerability | Doubles the risk of heart disease; triples the incidence of nausea and skin changes | −3b | −0.05 | −0.05 | |
| Comparative patient-perceived health/PROs | Inconclusive evidence | 0a | 0.00 | 0.00 | |
| Type of benefit of intervention | Type of preventive benefit | X does not reduce risk of condition Y | 0 | 0.00 | 0.00 |
| Type of therapeutic benefit | X delays disability but does not cure Y or prolong life | 3 | 0.04 | 0.04 | |
| Economic consequences of intervention | Comparative cost consequences—cost of intervention | X costs substantially more than current therapy | −4b | −0.08 | −0.07 |
| Comparative cost consequences—other medical costs | Model: potential savings due to reduced disability slightly outweigh additional costs for monitoring and AE treatments | 1b | 0.01 | 0.01 | |
| Comparative cost consequences—nonmedical costs | Model predicts moderately reduced patient and caregiver productivity losses | 2b | 0.02 | 0.02 | |
| Knowledge on intervention | Quality of evidence | One small RCT; economic model uncertain | 2 | 0.02 | 0.01 |
| Expert consensus/CPGs | Generally recommended by experts but with caveats | 3 | 0.03 | 0.02 | |
| Contextual criteria | |||||
| Population priorities | Rare diseases | X fully aligned with rare disease priority | 5 | Positive impactd | 0.04 |
| Other priorities | X not aligned with other priorities | 0 | 0.00 | ||
| Value estimate (quantitative output)e | 0.28 | 0.30 | |||
| Contextual Criteria (Qualitative Assessment) | Impact on value estimate of X | ||||
| Contextual normative criteria | Alignment with the mandate and scope of the healthcare system | Severe disease requiring healthcare intervention | Positive impact | ||
| Alignment with the common goal rather than special interests | Many conflicting interests | No impact | |||
| Environmental sustainability | Intervention X has no environmental consequences | Positive impact | |||
| Contextual feasibility criteria | Affordability and opportunity costs | Low budget impact because of small population | Positive impact | ||
| System capacity and appropriate use | Some risk of inappropriate use | Negative impact | |||
| Political, historical, and cultural context | Innovative treatment approach | Positive impact | |||
For simplicity, subcriteria are omitted in this example
MCDA multicriteria decision analysis, AE adverse event, CPGs clinical practice guidelines, PROs patient-reported outcomes, QoL quality of life, RCT randomized controlled trial
aThe weights reflect the established priorities in the country
bScale: −5 to 5 for comparative criteria; all other criteria (noncomparative) are scored on a scale of 0–5
cValue contribution = normalized weight (W x) × standardized score (S x, assigned score divided by maximum 5)
dAssessed qualitatively in this context
eValue estimate = ∑(W x × S x). The maximum (1) corresponds to a hypothetical intervention that prevents and cures severe endemic diseases with significant unmet needs and which, compared with existing approaches, has demonstrated large improvements in efficacy, safety, and PROs, as well as positive economic consequences. (When specific healthcare system priorities are included in the quantitative model, this definition also includes full alignment of the intervention with these priorities.)