Table 4.

Main characteristics of the four selected HEE quality appraisal instruments

	QHES instrument[21]	BMJ guidelines[22]	CHEC list[23]	Philips guidelines[24]
Author; year; journal	Chiou et al. 2003; Medical Care	Drummond, Jefferson; 1996; British Medical Journal (BMJ)	Evers et al. 2005; International Journal of Technology Assessment in Health Care	Philips et al. 2006; Pharmacoeconomics
Affiliation of authors	Academia and industry	Academia	Academia	Academia
Published in a peer-reviewed journal	Yes	Yes	Yes	Yes
Number of references	35	48	30	22
Purpose	To provide a grading system for assessing the quality of health economic evaluations	To improve the transparency of reporting	To develop a generally accepted criteria list for assessing the methodology of economic evaluation studies in systematic reviews	To identify, review, and consolidate currently available guidelines in order to establish a synthesized and consistent quality assessment framework for decision analytic models
Development process	Selection of criteria from 19 existing guidelines; Use of a conjoint analysis survey of 120 international experts to estimate weights for each criterion	Not specified	Selection of items from 15 existing guidelines; Use of a Delphi panel consisting of 23 international experts to generate the final criteria list	Selection and formulation of items by reviewing and consolidating 15 existing guidelines for good practice in decision-analytic modeling in HTA
Time frame	Before, during and after peer review	Before, during, and after peer review	After peer review	Before, during, and after peer review
Target audience	Producers and consumers of economic evaluations	Producers and consumers of economic evaluations	Consumers intending to conduct a systematic review of trial-based economic evaluations	Producers and consumers of model-based economic evaluations
Preferred analytical technique	Full economic evaluations: cost-minimization-, cost-effectiveness-, cost-utility-, cost-benefit-analysis	Full economic evaluations: cost-minimization-, cost-effectiveness-, cost-utility-, cost-benefit-analysis	Full economic evaluations based on clinical trials: cohort studies, case–control studies, randomized controlled clinical trials	Full economic evaluations based on decision-analytic models
Standard reporting format included	16 questions which should be asked when appraising the quality of health economic evaluations	Ten sections under the three headings of study design, data collection, and analysis and interpretation of results: study question, selection of alternatives, form of evaluation, effectiveness data, benefit measurement and valuation, costing, modeling, adjustments for timing of costs and benefits, allowance for uncertainty, and presentation of results	19 questions which should be asked when appraising the quality of health economic evaluations in systematic reviews	15 sections under the three key themes of structure, data, and consistency: statement of decision problem/objective, statement of scope/ perspective, rationale for structure, structural assumptions, strategies/comparators, model type, time horizon, disease states/pathways, cycle length, data identification, pre-model data analysis, data incorporation, assessment of uncertainty, internal consistency, and external consistency
Number of questions/criteria	16	35	19	61
Operationalization of the questions/criteria	Yes/No	Yes/No/Not clear/Not appropriate	Yes/No	Yes/No/Unclear/Not applicable
Use of a quality score	Yes	No	No	No
Assessment instruction	No	Yes	Yes: http://www.beoz.unimaas.nl/chec/	Yes
Pilot test of the guideline	Yes: Ofman et al. [31]	Not specified	Yes, but no details given	Yes