Maniadakis 2006 |
Item |
Dimension of quality |
Question for critical appraisal |
Yes |
No |
N/A |
Comment |
Structure |
S1 |
Statement of decision problem/objective |
Is there a clear statement of the decision problem? |
x |
|
|
|
Is the objective of the evaluation and model specified and consistent with the stated decision problem? |
x |
|
|
|
Is the primary decision‐maker specified? |
x |
|
|
|
S2 |
Statement of scope/perspective |
Is the perspective of the model stated clearly? |
x |
|
|
|
Are the model inputs consistent with the stated perspective? |
x |
|
|
|
Has the scope of the model been stated and justified? |
x |
|
|
|
Are the outcomes of the model consistent with the perspective, scope and overall objective of the model? |
x |
|
|
|
S3 |
Rationale for structure |
Has the evidence regarding the model structure been described? |
x |
|
|
Referencing original study; Oostenbrink 2005
|
Is the structure of the model consistent with a coherent theory of the health condition under evaluation? |
|
x |
|
The model does not include a health state of death |
Have any competing theories regarding model structure been considered? |
|
x |
|
|
Are the sources of data used to develop the structure of the model specified? |
x |
|
|
Referencing original study; Oostenbrink 2005
|
Are the causal relationships described by the model structure justified appropriately? |
x |
|
|
|
S4 |
Structural assumptions |
Are the structural assumptions transparent and justified? |
x |
|
|
|
Are the structural assumptions reasonable given the overall objective, perspective and scope of the model? |
x |
|
|
|
S5 |
Strategies/comparators |
Is there a clear definition of the options under evaluation? |
x |
|
|
|
Have all feasible and practical options been evaluated? |
x |
|
|
|
Is there justification for the exclusion of feasible options? |
|
x |
|
|
S6 |
Model type |
Is the chosen model type appropriate given the decision problem and specified causal relationships within the model? |
x |
|
|
|
S7 |
Time horizon |
Is the time horizon of the model sufficient to reflect all important differences between options? |
|
x |
|
|
Are the time horizon of the model, the duration of treatment and the duration of treatment effect described and justified? |
x |
|
|
|
Has a lifetime horizon been used? If not, has a shorter time horizon been justified? |
|
x |
|
|
S8 |
Disease states/pathways |
Do the disease states (state transition model) or the pathways (decision tree model) reflect the underlying biological process of the disease in question and the impact of interventions? |
x |
|
|
|
S9 |
Cycle length |
Is the cycle length defined and justified in terms of the natural history of disease? |
x |
|
|
|
Data |
D1 |
Data identification |
Are the data identification methods transparent and appropriate given the objectives of the model? |
|
x |
|
|
Where choices have been made between data sources, are these justified appropriately? |
|
|
x |
Not described |
Has particular attention been paid to identifying data for the important parameters in the model? |
x |
|
|
|
Has the process of selecting key parameters been justified and systematic methods used to identify the most appropriate data? |
|
x |
|
|
Has the quality of the data been assessed appropriately? |
x |
|
|
Referencing original study; Oostenbrink 2005
|
Where expert opinion has been used, are the methods described and justified? |
|
|
x |
|
D2 |
Pre‐model data analysis |
Is the data modelling methodology based on justifiable statistical and epidemiological techniques? |
x |
|
|
|
D2a |
Baseline data |
Is the choice of baseline data described and justified? |
x |
|
|
Described but not justified |
Are transition probabilities calculated appropriately? |
x |
|
|
|
Has a half‐cycle correction been applied to both cost and outcome? |
|
x |
|
|
If not, has this omission been justified? |
|
x |
|
|
D2b |
Treatment effects |
If relative treatment effects have been derived from trial data, have they been synthesised using appropriate techniques? |
|
|
x |
For each comparison data were derived from a single trial |
Have the methods and assumptions used to extrapolate short‐term results to final outcomes been documented and justified? |
x |
|
|
|
Have alternative assumptions used to extrapolate short‐term results to final outcomes been explored through sensitivity analysis? |
|
x |
|
|
Have assumptions regarding the continuing effect of treatment once treatment is complete been documented and justified? |
|
|
x |
|
Have alternative assumptions regarding the continuing effect of treatment once treatment is complete been explored through sensitivity analysis? |
|
|
x |
|
D2c |
Costs |
Are the costs incorporated into the model justified? |
x |
|
|
|
Has the source for all costs been described? |
x |
|
|
|
Have discount rates been described and justified given the target decision‐maker? |
|
x |
|
|
D2d |
Quality of life weights (utilities) |
Are the utilities incorporated into the model appropriate? |
x |
|
|
|
Is the source for the utility weights referenced? |
x |
|
|
|
Are the methods of derivation for the utility weights justified? |
x |
|
|
Referencing original study; Oostenbrink 2005
|
D3 |
Data incorporation |
Have all data incorporated into the model been described and referenced in sufficient detail? |
x |
|
|
|
Has the use of mutually inconsistent data been justified (i.e. are assumptions and choices appropriate)? |
|
|
x |
|
Is the process of data incorporation transparent? |
x |
|
|
|
If data have been incorporated as distributions, has the choice of distribution for each parameter been described and justified? |
x |
|
|
Referencing original study; Oostenbrink 2005
|
If data have been incorporated as distributions, is it clear that second order uncertainty is reflected? |
x |
|
|
|
D4 |
Assessment of uncertainty |
Have the four principal types of uncertainty (D4a ‐ D4d below) been addressed? |
|
x |
|
|
If not, has the omission of particular forms of uncertainty been justified? |
|
x |
|
|
D4a |
Methodological |
Have methodological uncertainties been addressed by running alternative versions of the model with different methodological assumptions? |
x |
|
|
|
D4b |
Structural |
Is there evidence that structural uncertainties have been addressed via sensitivity analysis? |
x |
|
|
Referencing original study; Oostenbrink 2005
|
D4c |
Heterogeneity |
Has heterogeneity been dealt with by running the model separately for different subgroups? |
|
x |
|
|
D4d |
Parameter |
Are the methods of assessment of parameter uncertainty appropriate? |
x |
|
|
|
Has probabilistic sensitivity analysis been done? If, not has this been justified? |
x |
|
|
|
If data are incorporated as point estimates, are the ranges used for sensitivity analysis stated clearly and justified? |
|
|
x |
|
Consistency |
C1 |
Internal consistency |
Is there evidence that the mathematical logic of the model has been tested thoroughly before use? |
x |
|
|
|
C2 |
External consistency |
Are the conclusions valid given the data presented? |
x |
|
|
|
Are any counterintuitive results from the model explained and justified? |
|
|
x |
|
If the model has been calibrated against independent data, have any differences been explained and justified? |
|
x |
|
|
Have the results of the model been compared with those of previous models and any differences in results explained? |
|
x |
|
|