. 2023 Mar 28;30(4):3776–3786. doi: 10.3390/curroncol30040286

Table 2.

Current MCDA Rating Tool.

Criteria	Rating Scale			Weight
Criteria	1	2	3	Weight
Group A—Criteria to Assess the Importance of the RWE Question
Drug’s perceived incremental benefit: The objective of this criterion is to evaluate the perceived clinical benefit of the therapy compared to existing options. The ‘perceived’ clinical benefit is based on the currently available objective evidence (including primary clinical trial data and indirect comparisons ¹) and expected long-term outcomes (in the setting of immature data).	Rated using the European Society of Medical Oncology (ESMO) Magnitude of Clinical Benefit Scale (MCBS) v1.0. [15]			17.65
	Minimal to low clinically meaningful incremental benefit, as evidenced by: (a) overall survival benefit demonstrated through a hazard ratio > 0.65 and/or gain in median overall survival of <2.5 months, as compared to current standard(s) of care, for therapies with a median overall survival of <1 year; or (b) overall survival benefit demonstrated through a hazard ratio of >0.70 and/or gain in median overall survival of <3 months, as compared to current standard(s) of care, for therapies with a median overall survival > 1 year; or (c) median progression free survival benefit demonstrated through a hazard ratio of > 0.65, as compared to current standard(s) of care; or) clinical benefit demonstrated in an alternative outcome (including improvements in response rate, quality-of-life or other clinically meaningful outcome).	Moderate clinically meaningful incremental benefit, as evidenced by: (a) survival benefit demonstrated by a hazard ratio < 0.65 and gain in median survival of 2.5–2.9 months, as compared to current standard(s) of care, for therapies with median overall survival < 1 year; or (b) overall survival benefit demonstrated by a hazard ratio < 0.70 and gain in median overall survival of 3-4.9 months, as compared to current standard(s) of care, for therapies with median overall survival > 1 year; or (c) progression-free survival benefit demonstrated by a hazard ratio < 0.65 and gain > 1.5 months, as compared to current standard(s) of care.	Substantial clinically meaningful incremental benefit, as evidenced by: (a) overall survival benefit demonstrated through a hazard ratio of <0.65 and gain in median overall survival of 2.5–3.0 months, as compared to current standard(s) of care, for therapies with median overall survival < 1 year; or (b) overall survival benefit demonstrated through a hazard ratio < 0.70 and gain in median overall survival of >5 months, as compared to current standard(s) of care, for therapies with median overall survival >1 year.	17.65
Magnitude of uncertainty: The objective of this criterion is to assess the degree of uncertainty in question (the uncertainty can be about toxicity, clinical effectiveness, quality-of-life, treatment sequence, generalizability of benefits, costs or other).	Minimal uncertainty: This can be based upon either a qualitative assessment or quantitative assessment (the latter can be conceptualized as a <10% variation in either of the following: (a) the confidence intervals around the survival estimates; (b) the upper and lower range of ICERs from the pCODR assessment ¹).	Moderate uncertainty: This can be based upon either a qualitative assessment or quantitative assessment (the latter can be conceptualized as a 10–25% variation in either of the following: (a) the confidence intervals around the survival estimates; (b) the upper and lower range of ICERs from the pCODR assessment ¹).	Substantial uncertainty: This can be based upon either a qualitative assessment or quantitative assessment (the latter can be conceptualized as a >25% variation in either of the following: (a) the confidence intervals around the survival estimates; (b) the upper and lower range of ICERs from the pCODR assessment ¹).	10.6
Relevance of uncertainty: The objective of this criterion is to assess the relevance of resolving the uncertainty to decision-makers (i.e., what is the likelihood that resolving the uncertainty with new evidence will alter the funding status or clinical treatment recommendations).	Indirect relevance: As assessed by expert opinions, there is an expected low likelihood for new evidence to facilitate a change in funding status (i.e., facilitate drug price re-negotiations) and/or change in clinical treatment recommendations (i.e., indicated patient populations or treatment sequence).	Moderate relevance: As assessed by expert opinions, there is uncertainty in the likelihood for new evidence to facilitate a change in funding status (i.e., facilitate drug price re-negotiations) and/or change in clinical treatment recommendations (i.e., indicated patient populations or treatment sequence).	Substantial relevance: As assessed by expert opinions, there is an expected high likelihood for new evidence to facilitate a change in funding status (i.e., facilitate drug price re-negotiations) and/or change in clinical treatment recommendations (i.e., indicated patient populations or treatment sequence).	18.8
Group B—Criteria to Assess the Feasibility of the RWE Project
Comparator: The objective of this criterion is to assess the likelihood that a relevant historical or contemporaneous Canadian comparator population, of sufficient size, can be identified within a reasonable time frame (i.e., within time to be relevant to the funding decision, for contemporaneous control). A ‘relevant’ comparator is a group that has been treated according to current Canadian standard of care regimen.	Substantial concern: Unlikely to identify an appropriate comparator population within a reasonable time due to absence of clear standard-of-care therapy (i.e., >2 relevant standard-of-care treatments currently available or evolving standard-of-care treatment) and/or low-volume patient population.	Moderate concern: Moderate concern for the identification of an appropriate comparator population due to absence of clear standard-of-care therapy (i.e., 2 relevant standard-of-care treatments currently available) and/or moderate-volume patient population.	Low concern: Appropriate comparator population will be easily identified due to a well-defined standard of care therapy and high-volume patient population.	11.8
Cases: The objective of this criterion is to assess the likelihood that there will be enough patients receiving the treatment in question to have a sufficient sample size within a reasonable time frame (i.e., within time to be relevant to the funding decision).	Substantial concern: Unlikely to establish a sufficient sample size (with appropriate follow-up for relevant outcome(s)) within a reasonable time ² based upon expected incidence of disease (using Canadian provincial estimates) and required sample size for analysis.	Moderate concern: Likely to establish a sufficient sample size, based upon expected incidence of disease (using Canadian provincial estimates) but unlikely to have follow-up for relevant outcome(s) within a reasonable time ² based upon expected incidence (using Canadian provincial estimates) and required sample size for analysis.	Low concern: Very likely to establish a sufficient sample size (with appropriate follow-up for relevant outcome(s)) within a reasonable time ² based upon expected incidence of disease (using Canadian provincial estimates) and required sample size for analysis.	14.1
Data: The objective of this criterion is to assess the quality of data available in at least one Canadian province to address the uncertainty. This requires an assessment of the availability and completeness of data for both the exposed and comparator cohorts pertaining to: (a) data for relevant patient and disease characteristics to account for important co-variates, ensure un-biased comparability between groups and measure relevant outcomes +/− (b) data for relevant costing inclusive of total health care costs accrued during treatment (ex. systemic treatment, planned and unplanned health care resource utilization).	Substantial concern: Substantial concern for the availability of high-quality and complete data for both exposed and comparator cohorts in known real-world databases (as assessed by an absence of ≥1 of the following: (a) patient and/or disease characteristics required to define current funding eligibility; (b) >2 relevant patient and/or disease co-variates; (c) ability to identify primary systemic treatment, inclusive of line-of-therapy).	Moderate concern: Moderate concern for the availability of high-quality and complete data for both exposed and comparator cohorts in known real-world databases (as assessed by an absence of ≥1 of the following: (a) 1–2 relevant patient and/or disease co-variates; (b) ability to identify prior or subsequent treatment inclusive of line-of-therapy).	Low concern: No expected issues in accessing high-quality and complete data in known real-world databases.	17.65
Expertise and Methodology: The objective of this criterion is to evaluate the availability of required expertise (ex. clinical experts, data analysts and methodologists) and methodology to conduct the study.	Substantial concern: Expected challenges to find the necessary expertise and need to develop new methods to conduct the study, with above limitations in data taken into consideration (if applicable).	Moderate concern: Expected challenges to find the necessary expertise or need to develop new methods to conduct the study, with above limitation in data taken into consideration (if applicable).	Low concern: No expected issues with the availability of the necessary expertise and no new methods required to conduct the study.	9.4

¹ Please refer to the pCODR economic guidance report for details on pCODR’s reanalysed ICERs. ² Reasonable time: time required is less than half the time of remaining patency duration. Abbreviations: ICER: incremental cost-effectiveness ratio; pCODR: pan-Canadian Oncology Drug Review.