Table 2.

Classification of terms reported in the literature

Categories of classification system	Criteria of classification system	Terms used in articles
A-Health outcomes and benefits of intervention	Number of criteria: 6	Number of terms: 44
	A1: Health benefits: 7 terms, cited 10 times	· A1 – health benefits [13,31,38,50], potential health gain [44], enhanced health outcomes [44], relative advantage [51], health effects [30], additional effects [22], incremental health gain [43]
	A2: Efficacy/effectiveness: 11 terms, cited 29 times	· A2 – efficacy [13,47], efficacy/effectiveness [10,19,20,25,27,28,44,48], effectiveness [14,22,26,32-34,48], clinical benefit [19,22,24,42,47], clinical impact [45], clinical merit [22], relative clinical benefit in relation with current standards [16], determine relative value for degree of benefit against benchmarks [16], magnitude of treatment effect [22], response rate [43], onset and duration of treatment/program effect [43]
	A3: Life saving: 4 terms, cited 5 times	· A3 – prolongation of disease-free survival [42], saving life [19], life expectancy gains [13], average life-year benefit per patient [13,33]
	A4: Safety: 11 terms, 19 times	· A4 – side effects [33,41,47], unintended consequences [40], safety [9,22,26,31], safety and tolerability [10,19,20], risks [20,22], risk management [44], harm [42], adverse effects [32], inconvenience [22], risk of event [22], reduction in symptomatic toxicity compared with standard therapy [42]
	A5: PRO: 10 terms, 17 times	· A5 – patients reported outcomes [10], quality of life [19,42,44,52], impact on quality of life [22,43], number of QALYs gained per patient [36,39], disability adjusted life years [13], likely impact on patient [16], patient preference [25], patient autonomy [26,35,40], relative value to patient [16], best for patient [38]
	A6: Quality of care: 1 term, 1 time	· A6 – overall gain in quality of care [44]
B-Type of health benefit	Number of criteria: 2	Number of terms: 12
	B1: Population effect (prevention): 6 terms, 11 times	· B1 – public health interest [10], population effects [19], prevention [19,28], prevention of ill health [44], social impact [13,22,33], social benefit [13,22,33]
	B2: Individual effect (medical service): 6 terms, 7 times	· B2 – type of medical service [10], relief/prevention of symptoms/complications of disease [42], health gain or maintenance [44], individual effects [19], individual impact and benefit [13,33], the composition of the health gain [39]
C-Impact of the disease targeted by intervention	Number of criteria: 4	Number of terms: 21
	C1: Disease severity: 2 terms, 9 times	· C1 – severity of disease [9,10,13,19,30,37,39,47], impact of the disease/condition on quality of life [43]
	C2: Disease determinants: 2 terms, 2 times	· C2 – determinants (the factors responsible for the persistence of the burden) [17], characteristics of target condition [22]
	C3: Disease burden: 7 terms, 13 times	· C3 – burden of disease [9,13,22,33], disease burden [17,25,45,48], burden of illness [22], burden of therapy [22], cost to treat disease [33], cost to prevent disease [33], national cost of the disease/condition to the healthcare system [43]
	C4: Epidemiology: 10 terms, 16 times	· C4 – prevalence [9,13], number of potential beneficiaries [35,37,40], indirect beneficiaries [40], size of population [10,19], prevalence and incidence of disease [23,25,43], number of residents benefiting [44], number of clients served [43], number of patients [47], social/demographics [22], incidence [22]
D-Therapeutic context of intervention	Number of criteria: 4	Number of terms: 18
	D1: Treatment alternatives: 5 terms, 13 times	· D1 – treatment alternatives [13,22], availability of alternatives [16,19,25,42,44,47], availability of effective intervention and preventable [13], alternatives [35,40,45], benchmark comparators [16]
	D2: Need: 8 terms, 16 times	· D2 – comparative interventions limitations (unmet needs) [10], need [19,22,28,38,42,44,49], clinical impact (need and trends) [24], emergencies and need [13], apparent need [14], clinical need [36,41,50], desirability of effects [40], meets patient’s basic need [38]
	D3: Clinical guidelines & practices: 4 terms, 7 times	· D3 – evidence-based guidelines [13,33,36], best practice [14], clinical guidelines [10,23], academic health center research (establishing/or using best practice) [24]
	D4: Pre-existing use: 1 term, 1 time	· D4 – pre-existing prescribing of the drug [47]
E-Economic impact of intervention	Number of criteria: 9	Number of terms: 36
	E1: Cost: 3 terms, 11 times	· E1 – cost per patient [19], costs [19,20,22,27,32,42,44,47,51], unit cost [22]
	E2: Budget impact: 6 terms, 11 times	· E2 – budget impact on health plan [10,19,25,47], total budget impact [30], budget impact [32,45,47], usage and cost implications of competing new drugs if approved [16], affordability [25], operating and start-up costs [43]
	E3: Broad financial impact: 7 terms, 7 times	· E3 – impact on other spending [10], financial impact on government [13], economic impact [45], economics [22], national medical costs per-year [39], cost-saving [33], national saving in costs of absence per year [39]
	E4: Poverty reduction: 1 terms, 3 times	· E4 – positive poverty reduction [13,30,37]
	E5: Cost-effectiveness: 5 terms, 23 times	· E5 – cost-effectiveness [9,10,13,14,17,20,22,25-27,30,34,37,39,41,44], economic evaluations [27], cost and consequences [9,13,14,41], pharmacoeconomic analysis [23], cost utility expressed as cost per QALY [22]
	E6: Value: 2 terms, 3 times	· E6 – value for money [32,44], financial value [44]
	E7: Efficiency and opportunity costs: 6 terms, 10 times	· E7 – efficiency of intervention [31], efficiency [10,19,22,23,44], opportunity costs [10], opportunity costs to the population/society [16], best within available resources [38], interdependencies [50]
	E8: Resources: 5 terms, 6 times	· E8 – resources [17,51], variation in rate of use [45], available resources [13], resources implications [50], volume of activity [13]
	E9: Insurance premiums: 1 term, 1 time	· E9 – impact on health insurance premiums [9]
F-Quality and uncertainty of evidence	Number of criteria: 6	Number of terms: 34
	F1: Evidence available: 7 terms, 9 times	· F1 – evidence [22,42,45], proof [22], scientific evidence [47], current level of knowledge [17], time of assessment in technology development [35], timelines of review [45], therapy mechanism of action [23]
	F2: Strength of evidence: 14 terms, 20 times	· F2 – strength of evidence [16,44], quality of evidence [47], quality of data and past decisions [47], quality of data [22], quality [26], validity of evidence [10,19], related degree of knowledge certainty [23], certainty [48], consistency [19,22,44], consistent [38], completeness and consistency of reporting evidence [10], openness [26,44], selection of studies [35,40], precision of treatment effect [22]
	F3: Relevance of evidence: 5 terms, 8 times	· F3 – relevance of evidence [10,19], representativeness of users (studies vs. real world) [35,40], level of generalization [35,40], effectiveness in real practice [22], evidence of effectiveness [44]
	F4: Evidence characteristics: 5 terms, 7 times	· F4 – normative characteristics of study [35,40], choice of endpoints [35,40], clinical trial data [47], multiple randomized trials or meta-analysis/single randomized trial of reasonable size/small randomized trial [42], phase II [53]
	F5: Research ethics: 2 terms, 4 times	· F5 – research ethics [35,40], informed consent [26,40]
	F6: Evidence requirements: 1 term, 1 time	· F6 – adherence to requirement of decision making body [10]
G-Implementation complexity of intervention	Number of criteria: 9	Number of terms: 57
	G1: Legislation: 6 terms, 6 times	· G1 – legal arrangements [40], legislative issues [22], medical liability [40], human rights legislation [23], legal implications [45], conformity of programs [22]
	G2: Organizational requirements and capacity to implement: 15 terms, 17 times	· G2 – system requirements [25], physical environment [44], environment [22,26], system capacity [10], local capacity [17], ability to implement [38], implementation [22], organization’s structure [51], organizational burden [49], logistics [36], process [28], well-organized [38], organizational feasibility [22,25], feasibility of delivery [16], deliverability [48]
	G3: Skills: 6 terms, 6 times	· G3 – knowledge and skills [51], nature of staff [51], clinical education and training [44], human resources availability [17], recruitment and retention of staff [44], attracting/retaining scarce clinical staff [44]
	G4: Flexibility of implementation: 7 terms, 8 times	· G4 – flexibility [51], reversibility [51], trialiability [51], revisability [51], ability to evaluate [22], provision for revision/appeals [38], engagement [26,48]
	G5: Characteristics of intervention: 6 terms, 8 times	· G5 – characteristics of intervention [22], complexity of the intervention [51], components of technology [35], autonomy of the intervention [38], autonomy [17,26,46], convenience [42]
	G6: Appropriate use: 3 terms, 3 times	· G6 – appropriate use of intervention [10], appropriateness [44], appropriate setting/level of service [43]
	G7: Barriers and acceptability: 3 terms, 4 times	· G7 – acceptability [22,48], responsiveness [44], controversial nature of proposed technology [45]
	G8: Integration and system efficiencies: 9 terms, 9 times	· G8 – system integration (best use of elements of healthcare system) [34], integration into local community [44], ease of integration [22], impact on other services [40], links to other services [44], compatibility [22], reduction of the monitoring [33], reduction of waiting list size [33], impact [22]
	G9: Sustainability: 2 terms, 4 times	· G9 – sustainability [23,24,26], longevity [19]
H-Priorities, fairness and ethics	Number of criteria: 7	Number of terms: 55
	H1 Population priorities: 5 terms, 5 times	· H1 – perspective and current priority [13], target and priority-setting [14], known priorities [44], population priority [10], coverage of selected conditions [13]
	H2 : Access: 10 terms, 17 times	· H2 – population access [10], access [19,27,47,49], equity of access improvement [13], access to care easier [31,33,34], distribution and access to healthcare [35,40], accessibility [22,44], equity of access [44], access to health system [22], geographical equity [43], timeliness of access [43]
	H3 : Vulnerable and needy population: 9 terms, 11 times	· H3 – vulnerable population [37,38], potential victims [40], particular social groups with high risk and/or increased vulnerability [23], compassion for the vulnerable [19], particularly needy/vulnerable groups [44], age of targeted group [13,30], maternal mortality [13], quality of maternity care services [13], population equity [43]
	H4: Equity, fairness and justice: 12 terms, 32 times	· H4 – equity [8,13,14,19,22,25,27,40,44,46,48], fairness [10,14,40,44,47], health equity [23,26], equality [19,26,38], distributive justice [23,25], formal justice [23], social justice [23], justice [26,46], social injustice [40], addressing health status inequalities at a population level [44], human integrity and dignity [35,40], basic human rights [35]
	H5 : Utility: 2 terms, 3 times	· H5 – utility [10,26], utilitarism [25]
	H6: Solidarity: 6 terms, 8 times	· H6 – solidarity [19,25,26], collectivism [26], mutuality [26], reciprocal trust [40], diversity [26], cohesion [26]
	H7: Ethics and moral aspects: 11 terms, 14 times	· H7 – ethics [14,22], ethical values [22], values [22], values and beliefs [51], consistency with societal values [22], ethical implications [45], moral obligation to implement a technology [35,40], rule of rescue [25], priority to basic and necessary care [38], moral consequence of HTA [35,40], moral challenges related to certain components of HTA [35]
I-Overall context	Number of criteria: 11	Number of terms: 83
	I1: Mission and mandate of health system: 13 terms, 19 times	· I1 – goals of healthcare [52,53], goals [21], beneficence [28], non-maleficience and justice [28], beneficence/non-maleficience [17,26,53], strategic fit [9,23], medical and social worth [45], relevance [22], present social consensus, [17,49] consensus regarding public funding of a therapy [17,53], government mandate [17], national standards [24], healthcare context positioning [23]
	I2: Overall priorities: 6 terms, 6 times	· I2 – national priorities [45], national or board priority [14], local and national priorities [8], international priorities [45], alignment with external directives [9], strategic direction [43]
	I3: Financial constraints: 8 terms, 13 times	· I3 – budget constraints [13,33,45], cost-containment [42,49], budget level [13,19,45], social economical context [16], limited provincial health resources [17], budget implementation challenges [17], economic feasibility [37], reliance of other services/sectors(on investment) [14]
	I4: Incentives: 4 terms, 5 times	· I4 – financial incentives [28,45], organizational support [16], donor involvement [31], incentives for compliance [20]
	I5: Political aspects: 5 terms, 7 times	· I5 – political pressure [13,19,45], political components [52], politically and legally defensible decisions [42], politics [37], political impact [37]
	I6: Historical aspects: 3 terms, 3 times	· I6 – historical components [52], past experiences [16], historical budgets [19]
	I7: Cultural aspects: 7 terms, 10 times	· I7 – culture and religious convictions [19,28,47], stigma [28], compatibility with values [16], challenge of social and values arrangements [28,47], conception of certain persons or disease [47], psychosocial implications [34], public preference [14]
	I8: Innovation: 3 terms, 3 times	· I8 – perceived benefits of change [16], innovativeness [37], generation or application of knowledge [43]
	I9: Partnership and leadership: 8 terms, 9 times	· I9 – partnership and networking [16], partnerships [9], maintaining relationship [42], leadership [16], community development [53], academic commitments: research and education [9,23], partnership and collaboration across organizations [43], contribution to position as a learning organization [43]
	I10: Citizen involvement: 3 terms, 3 times	· I10 – citizenship [53], ownership [53], enabling health literacy (empowerment) [53]
	I11: Stakeholders interests and pressures: 23 terms, 28 times	· I11-stakeholders pressure [52], advocacy [16,45], pressure from physician and patients groups and past decisions [32], clinical expert opinions [37], patient representative group opinions [37], power relations among stakeholders [28], user of the technology interests [47], challenge the relationship between patient and physician [47], professional prestige [28,47], clinicians excitement and decisions in other hospitals [32], public reaction and public accountability [28], HTA’s producer interest [28,47], company activities [32], researchers ethics interests [28,47], third party agents involved [47], recommendations made by other countries [13], status in other jurisdictions [49], current status of public funding in other jurisdictions [17], drugs used in other hospitals [32], expressed demand [14,37], patient demand [32], expected level of interest (patient and medical) [34], entitlement [28]

This table is reporting all the terms (338) extracted from the selected articles and tabulates them using the classification system developed for this study, which is based on a hierarchical approach clustering 58 criteria into 9 categories.