Skip to main content
. 2012 Sep 12;2012(9):CD009157. doi: 10.1002/14651858.CD009157.pub2
Maniadakis 2006
Item Dimension of quality Question for critical appraisal Yes No N/A Comment
Structure
S1 Statement of decision problem/objective Is there a clear statement of the decision problem? x      
Is the objective of the evaluation and model specified and consistent with the stated decision problem? x      
Is the primary decision‐maker specified? x      
S2 Statement of scope/perspective Is the perspective of the model stated clearly? x      
Are the model inputs consistent with the stated perspective? x      
Has the scope of the model been stated and justified? x      
Are the outcomes of the model consistent with the perspective, scope and overall objective of the model? x      
S3 Rationale for structure Has the evidence regarding the model structure been described? x     Referencing original study; Oostenbrink 2005
Is the structure of the model consistent with a coherent theory of the health condition under evaluation?   x   The model does not include a health state of death
Have any competing theories regarding model structure been considered?   x    
Are the sources of data used to develop the structure of the model specified? x     Referencing original study; Oostenbrink 2005
Are the causal relationships described by the model structure justified appropriately? x      
S4 Structural assumptions Are the structural assumptions transparent and justified? x      
Are the structural assumptions reasonable given the overall objective, perspective and scope of the model? x      
S5 Strategies/comparators Is there a clear definition of the options under evaluation? x      
Have all feasible and practical options been evaluated? x      
Is there justification for the exclusion of feasible options?      
S6 Model type Is the chosen model type appropriate given the decision problem and specified causal relationships within the model? x      
S7 Time horizon Is the time horizon of the model sufficient to reflect all important differences between options?      
Are the time horizon of the model, the duration of treatment and the duration of treatment effect described and justified? x      
Has a lifetime horizon been used? If not, has a shorter time horizon been justified?   x    
S8 Disease states/pathways Do the disease states (state transition model) or the pathways (decision tree model) reflect the underlying biological process of the disease in question and the impact of interventions? x      
S9 Cycle length Is the cycle length defined and justified in terms of the natural history of disease?      
Data
D1 Data identification Are the data identification methods transparent and appropriate given the objectives of the model?   x    
Where choices have been made between data sources, are these justified appropriately?     x Not described
Has particular attention been paid to identifying data for the important parameters in the model? x      
Has the process of selecting key parameters been justified and systematic methods used to identify the most appropriate data?   x    
Has the quality of the data been assessed appropriately? x     Referencing original study; Oostenbrink 2005
Where expert opinion has been used, are the methods described and justified?     x  
D2 Pre‐model data analysis Is the data modelling methodology based on justifiable statistical and epidemiological techniques? x      
D2a Baseline data Is the choice of baseline data described and justified? x     Described but not justified
Are transition probabilities calculated appropriately? x      
Has a half‐cycle correction been applied to both cost and outcome?   x    
If not, has this omission been justified?   x    
D2b Treatment effects If relative treatment effects have been derived from trial data, have they been synthesised using appropriate techniques?     x For each comparison data were derived from a single trial
Have the methods and assumptions used to extrapolate short‐term results to final outcomes been documented and justified?  x      
Have alternative assumptions used to extrapolate short‐term results to final outcomes been explored through sensitivity analysis?   x    
Have assumptions regarding the continuing effect of treatment once treatment is complete been documented and justified?     x  
Have alternative assumptions regarding the continuing effect of treatment once treatment is complete been explored through sensitivity analysis?     x  
D2c Costs Are the costs incorporated into the model justified? x      
Has the source for all costs been described? x      
Have discount rates been described and justified given the target decision‐maker?   x    
D2d Quality of life weights (utilities) Are the utilities incorporated into the model appropriate? x      
Is the source for the utility weights referenced? x      
Are the methods of derivation for the utility weights justified? x     Referencing original study; Oostenbrink 2005
D3 Data incorporation Have all data incorporated into the model been described and referenced in sufficient detail? x      
Has the use of mutually inconsistent data been justified (i.e. are assumptions and choices appropriate)?     x  
Is the process of data incorporation transparent? x      
If data have been incorporated as distributions, has the choice of distribution for each parameter been described and justified? x     Referencing original study; Oostenbrink 2005
If data have been incorporated as distributions, is it clear that second order uncertainty is reflected? x      
D4 Assessment of uncertainty Have the four principal types of uncertainty (D4a ‐ D4d below) been addressed?   x    
If not, has the omission of particular forms of uncertainty been justified?   x    
D4a Methodological Have methodological uncertainties been addressed by running alternative versions of the model with different methodological assumptions? x      
D4b Structural Is there evidence that structural uncertainties have been addressed via sensitivity analysis? x     Referencing original study; Oostenbrink 2005
D4c Heterogeneity Has heterogeneity been dealt with by running the model separately for different subgroups?   x    
D4d Parameter Are the methods of assessment of parameter uncertainty appropriate? x      
Has probabilistic sensitivity analysis been done? If, not has this been justified? x      
If data are incorporated as point estimates, are the ranges used for sensitivity analysis stated clearly and justified?     x  
Consistency
C1 Internal consistency Is there evidence that the mathematical logic of the model has been tested thoroughly before use? x      
C2 External consistency Are the conclusions valid given the data presented? x      
Are any counterintuitive results from the model explained and justified?     x  
If the model has been calibrated against independent data, have any differences been explained and justified?   x    
Have the results of the model been compared with those of previous models and any differences in results explained?   x