A Systematic Review of Model-Based Economic Evaluations of Treatments for Venous Leg Ulcers

Abstract

Objective

The aim of this review was to identify, and assess the quality of, published model-based economic evaluations relating to treatments for patients with venous leg ulcers to help inform future decision-analytic models in this clinical area.

Methods

A systematic literature search was performed on six electronic databases, from database inception until 21 May 2018. Search results were screened against predefined criteria by two independent reviewers. Data was then extracted from the included studies using a standardised form, whilst the decision-analytic model-specific Philips Checklist was used to assess quality and to inform model critique.

Results

A total of 23 models were identified, 12 studies used a Markov modelling approach, five used decision trees and six studies did not detail the model type. Studies were predominantly from the National Health Service (NHS)/payer perspective, with only two taking a societal perspective. Interventions were wide ranging, but dressing technologies (11/23) were most common. The intervention studied was found to be dominant in 22/23 studies. The reporting quality of papers was mostly low, with evidence behind model structures, time horizons and data selection consistently underreported across the included papers.

Conclusions

This review has identified a sizeable literature of model-based economic evaluations, evaluating treatments for venous leg ulcers. However, the methods used to conduct such studies were generally poorly reported. In particular, the reporting of evidence surrounding the model structure, justification of the time horizon used and the rationale for selecting data inputs should be focused on in any future models developed.

Electronic supplementary material

The online version of this article (10.1007/s41669-019-0148-x) contains supplementary material, which is available to authorized users.

Key Points for Decision Makers

Twenty-three models evaluating the cost effectiveness of treatments for venous leg ulcers were found within this systematic review.

The most common modelling approach was a Markov model, with studies predominantly taking an NHS/payer perspective and evaluating wound dressing technologies.

The reporting quality of models was appraised using the Philips Checklist. This found that most studies did not adequately report all aspects of the model used. Particularly, limited information was given surrounding the modelling techniques and structure.

Introduction

Venous leg ulcers (VLU) are long-lasting wounds of the lower leg. They are usually formed after an injury, with slow healing due to increased blood pressure in the leg veins [1]. Typically, ulcers take 3–4 months to heal with appropriate treatment [2]. VLUs often cause pain, malodour and fluid leakage [3]. These symptoms affect mobility, sleep and daily living, impairing quality of life [3].

VLUs affect 2% of over 80-year-olds in the UK and account for 60–85% of all leg ulcers [4]. The incidence and prevalence are increasing due to ageing populations and the global trend of obesity [5–7]. With their high prevalence and prolonged healing time, VLUs were estimated to cost the UK NHS £921 million in the price year 2012/2013 [8].

Treatment for VLUs involves wound care and sustained graduated compression therapy [4], with most of the direct costs of treatment attributable to community nurse visits [9]. There are also newly emerging advanced treatment modalities for VLUs. These include bioengineered tissue, electric stimulation and a wide variety of dressing variations [10]. A recent Cochrane systematic review concluded that there is low-certainty evidence for the clinical efficacy of novel dressings and topical agents in VLUs [4, 7, 11].

However, due to limited healthcare budgets and the increasing demand for services, it is now no longer enough to demonstrate a treatment as clinically effective, it also must be shown to be value for money, or cost effective [12]. Economic evaluations seek to address this question, comparing alternative courses of actions in terms of their costs and consequences, to determine which represents the most effective use of resources. Often such evaluations are conducted alongside clinical trials; however, the time horizons of such trials are usually dictated by the primary clinical outcome and therefore may end before all of the costs and outcomes, which are of economic interest, have been fully observed. Instead, decision-analytic models can be used [13]; such models do not have a restriction on the time horizon over which they can model, can take into account multiple treatment options, use data from multiple sources and perhaps most importantly, can assess the uncertainty surrounding a decision.

There are several studies published using decision-analytic models to conduct economic evaluations of treatments for VLUs. However, there have been no reviews of these studies to collate and critically appraise the methods used. Given this and the likelihood that newly emerging treatment modalities will need to be evaluated in the future, this review aimed to identify, evaluate and critically appraise all published model-based economic evaluations relating to the treatment of VLUs. In doing so, it is hoped that it will facilitate researchers developing future VLU models in the common methodologies used as well as areas that may require particular attention.

Methods

This systematic review was conducted and reported according to PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) guidelines [14]. The review has been registered on PROSPERO, the international prospective register of systematic reviews, under the registration number CRD42018102852.

Eligibility Criteria

Studies were considered eligible if they described a full economic evaluation using a decision-analytic model to evaluate any intervention for VLUs. A decision-analytic model was considered to be one that used mathematical techniques to define a series of possible consequences depending on alternative options [13, 15]. Where a study considered other wounds as well as VLU, it was made a condition that the results for the VLU population be presented separately. Patients of all ages, geographical locations and sexes were considered eligible for inclusion. Notably, studies detailing preventions for venous leg ulcers were excluded. In addition to the above constraints, only articles available in full text in English were considered due to resource limitations. All abstracts and reviews were excluded.

Search Strategy

A systematic search of electronic databases was conducted, from database inception until 21 May 2018, when the search was conducted. The search was applied to MEDLINE, Embase, Cumulative Index to Nursing and Allied Health Literature (CINAHL), Econlit, Web of Science and NHS Economic Evaluation Database (NHS EED). Database selection was structured from research and recommendations from Thielen et al. [16]. The search strategy itself was formed with the help of a biomedical information specialist, and was specialised to detect economic evaluations through the use of published search filters [17–19]. The full search strategy is available in the Electronic Supplementary Material 1 (Appendix 1), with search terms including ‘venous ulcer*’, ‘varicose ulcer*’ and ‘cost*’.

Study Selection

After duplicates were removed, study selection occurred in two stages and was performed independently by two reviewers (AL and EM). Initially, titles and abstracts were screened, with full texts then obtained for studies that appeared to meet the inclusion criteria. These full texts were then again screened against the inclusion criteria to assess eligibility for inclusion in the review, with reasons for exclusion noted down. All conflicts were resolved by discussion between the two reviewers. All eligible studies had their references screened to ensure that no papers had been missed from the search strategy.

Data Extraction

Data extraction was performed independently by two reviewers (AL and EM) using a standardised form. This form was piloted in an initial study to allow it to be refined. Data collected pertained to the study characteristics, details of the decision-analytic model and the results and conclusions of the study. Any disagreements were resolved by discussion, with a template of the data extraction form found in the Electronic Supplementary Material 1 (Appendix 2) and the populated data extraction form included within Electronic Supplementary Material 2.

Quality Assessment

Quality assessment was carried out using the Philips Checklist [20], an extensive checklist specifically designed for the assessment of modelling studies and recommended by both the National Institute for Health and Care Excellence (NICE) and the Cochrane Collaboration [21–23]. The checklist was completed independently by two reviewers (AL and EM), with any disagreements regarding answers resolved by discussion. Possible responses for the items on the checklist were ‘yes’, ‘no’, ‘not applicable’ (for items that were not relevant to the study) and ‘partial’ (for items that had multiple elements, of which only some were satisfactorily fulfilled by the study).

Results

The literature search returned a total of 2515 studies. After removing duplicate papers and initial screening of titles and abstracts, 130 full texts were retrieved and assessed for eligibility. 23 studies were included within the review. This study selection process is summarised in Fig. 1.

Fig. 1.

PRISMA diagram of study selection process

Study Characteristics

Table 1 summarises the included studies.

Table 1.

Summary of characteristics of included studies

Study	EE type	Intervention	Comparator	Country, perspective	Currency, price year	Model type (cycle length, time horizon)	Model states/tree summary	Outcome measures	Sensitivity analysis	Result
Markov models
Carr et al. 1999 [35]	CEA	Four-layer bandaging system (Profore)	Charing Cross System of four-layer bandaging	UK, not stated	GBP (£) 1997/98	Markov model (1 week, 1 year)	Healed and unhealed	Median time to heal	Univariate deterministic	Intervention dominant
Schonfeld et al. 2000 [42]	CEA	Skin substitute (Apligraf)	Zinc-impregnated gauze (Unna’s boot)	USA, commercial health plan	USD ($) 1996	Markov model (1 month, 1 year)	Unhealed, healed, recurrence, no therapy	Number of months during the year in a healed ulcer state,  % of patients with healed ulcers	Univariate deterministic	Intervention dominant
Scanlon et al. 2005 [29]	CEA	Silver-releasing dressing (Contreet Foam)	Aquacel Ag, Actisorb Plus, Iodoflex	UK, societal	GBP (£) 2003/04	Markov model (1 week, 26 weeks)	Healed, non-healed	Percentage reduction in wound area, number of healed wounds	Univariate deterministic	Intervention dominant
Iglesias and Claxton 2006 [46]	CEA + CUA	Oral pentoxifylline	Placebo with standard care (compression therapy and wound dressings)	UK, NHS	GBP (£) 2004	Markov model (1 week, 1 year)	Healed, unhealed, death	Ulcer-free weeks, QALYs	Univariate deterministic	Intervention dominant
Clegg and Guest 2007 [44]	CEA + CUA	Bio-electric stimulation therapy (Posifect)	Standard care (compression therapy and wound dressings)	UK, NHS	GBP (£) 2005/06	Markov model (1 week, 16 weeks)	Healed, improved, unchanged, worsened, dead	% of healed patients at 16 weeks, QALYs	Univariate deterministic, probabilistic	Intervention dominant
Guest et al. 2009 [39]	CEA + CUA	Amelogenin	Standard care (compression therapy and wound dressings)	UK, NHS	GBP (£) 2006/07	Markov model (1 month, 1 year)	Unchanged worsened, improved, healed, recurrence, dead	% of healed patients, QALYs	Univariate deterministic, probabilistic	Intervention dominant
Taylor et al. 2011 [43]	CEA + CUA	Electric stimulation (Accel-Heal)	Standard care (compression therapy and wound dressings)	UK, NHS	GBP (£) 2008/09	Markov model (1 month, 5 months)	Unchanged, worsened, improved, healed, recurrence	Probability of healing, QALYs	Univariate deterministic, probabilistic	Intervention dominant
Carter et al. 2014 [41]	CEA	Extracellular matrix (OASIS)	Human skin equivalent, living skin equivalent, standard care (compression therapy and wound dressings)	USA, payer	USD ($) 2012	Markov model (1 week, 1 year)	Unhealed, healed, death	Healed wound weeks, expected wound closure rate	Univariate deterministic, probabilistic	Intervention dominant
Ashby et al. 2014 [38]	CUA	Two-layer bandage	Two-layer hosiery, four-layer bandage, short-stretch bandage, paste bandage	UK, NHS	GBP (£) 2011	Markov model (1 month, 12 years)	Unhealed, healed, death	QALYs	Probabilistic	Intervention dominant
Kaspar et al. 2015 [28]	CEA	Polyacrylate-based hydrogel (HydroClean)	Amorphous hydrogel	Germany, payer	Euro (€) not stated	Markov model (1 day, 14 days)	Wound bed covered with ≤ 50% granulation tissue, wound bed covered with > 50% granulation tissue	Mean days with > 50% granulation tissue	None	Intervention dominant
Nherera et al. 2016 [34]	CEA + CUA	Iodine dressing (Cadexomer iodine)	Standard care (compression bandaging with debridement)	USA, payer	USD ($) 2014	Markov model (1 week, 1 year)	Healed, unhealed, infected, dead	Ulcer-free weeks, QALYs	Univariate deterministic, probabilistic	Intervention dominant
Walzer et al. 2018 [24]	CEA + CUA	Hydration response technology dressing (Cutimed Sorbion Sachet S)	Four superabsorbent dressings (Zetuvit Plus, KerraMax Care, Eclypse, DryMax extra)	UK, NHS	GBP (£) not stated	Markov model (1 week, 1 year)	Healed, improving, unchanged, deteriorating, infected or complications	Time taken to heal wound, QALYs	Univariate deterministic	Intervention dominant
Decision tree models
Guest et al. 2005 [30]	CEA	Carboxymethylcellulose dressing (Aquacel hydrofiber)	Gauze dressing	Germany and USA, payer	Euro (€) and USD ($) 2002/03	Decision tree model	Healed or unhealed	Ulcer successfully healed at 18 weeks	Univariate deterministic, probabilistic	Intervention dominant
Guest et al. 2012 [45]	CEA + CUA	Cavilon No Sting Barrier Film	Cavilon durable barrier cream, standard care	UK, NHS	GBP (£) 2009/10	Decision tree model	Not detailed	% of healed patients at 6 months, QALYs	Univariate deterministic, probabilistic	Intervention dominant
Jemec et al. 2014 [31]	CEA	Silver dressing (Biatain Ag)	Non-silver dressings	UK, NHS	GBP (£) 2012/13	Decision tree model	Healed vs healing vs no improvement + specialist care	Time to healed wound	Univariate deterministic	Intervention dominant
Guest et al. 2015 [36]	CUA	Two-layer cohesive compression bandage (Coban 2)	Other compression systems (Ktwo, Profore)	UK, NHS	GBP (£) 2012/13	Decision tree model	Not detailed	QALYs	Univariate deterministic, probabilistic	Intervention dominant
Guest et al. 2018 [27]	CEA + CUA	Collagen-containing dressing	Standard care (compression therapy and wound dressings)	UK, NHS	GBP (£) 2015/16	Decision tree model	Healed or unhealed	Probability of healing, QALYs	Univariate deterministic, probabilistic	Intervention dominant
Model not detailed
Harding et al. 2000 [32]	CEA	Hydrocolloid dressing (Granuflex)	Apligraf, saline gauze	UK, not stated	GBP (£) not stated	Not detailed—“cost-effectiveness model”	Not detailed	% healed in 12 weeks	None	Intervention dominant
Kerstein et al. 2001 [33]	CEA	Hydrocolloid dressing (DuoDERM)	Apligraf, saline gauze	USA, not stated	USD ($) 2000	Not detailed	Not detailed	Proportion of ulcers completely healed	None	Intervention dominant
Sibbald et al. 2001 [40]	CEA	Skin substitute (Apligraf)	Profore	Canada, societal	Canadian dollars ($) 1996/97	Not detailed—“computer-based model”	Not detailed	Number of ulcer days, days to heal	Univariate deterministic	$14 per ulcer-day averted
Meaume and Gemmen 2002 [26]	CEA	Hydrocolloid dressing (DuoDERM)	Apligraf	France, health insurance plan	Euro (€) not stated	Not detailed	Not detailed	% patients healed	None	Intervention dominant
Panca et al. 2013 [25]	CUA	Sodium carboxymethyl-cellulose dressing (Aquacel)	Four superabsorbent dressings (DryMax Extra, Flivasorb, Kerramax, Sachet S)	UK, NHS	GBP (£) 2003/04	Not detailed—“decision model”	Not detailed	QALYs	Univariate deterministic, probabilistic	Intervention dominant
Guest et al. 2017 [37]	CEA + CUA	Two-layer cohesive compression bandage (Coban 2)	Other compression systems (Ktwo, Profore)	UK, NHS	GBP (£) 2014/15	Not detailed	Not detailed	% of healed patients, QALYs	Univariate deterministic, probabilistic	Intervention dominant

CEA cost-effectiveness analysis, CUA cost-utility analysis, EE economic evaluation, GBP British pound, NHS UK National Health Service, QALY quality-adjusted life-years, USD United States dollar

The most common intervention type across the studies was dressings for VLUs, the subject of 11 studies (48%) [24–34]. There were also four studies each focusing on compression bandaging [35–38] and extracellular matrices [39–42]. The other interventions included in papers in the review were electric stimulation therapies [43, 44], barrier creams [45] and pentoxifylline oral medication [46]. Comparators varied greatly across studies depending on intervention type, with ten studies using more than one comparator [24, 25, 29, 32, 33, 36–38, 41, 45]. Eight studies used ‘standard care’ as a comparator, with all of these papers giving a definition of what ‘standard care’ involved at some point in the text [27, 34, 39, 41, 43–46].

The publication year of these studies ranged from 1999 to 2018. Ten of the studies (43%) were published in the Journal of Wound Care [24–27, 35–37, 39, 43, 45], with no other journal responsible for more than one paper. There were 11 cost effectiveness analyses (CEAs) [26, 28–33, 35, 40–42] and three cost utility analyses (CUAs) [25, 36, 38], with the remaining nine studies a combination of both CEA and CUA [24, 27, 34, 37, 39, 43–46]. For the CEAs, the most common clinical outcome related to percentage of patients healed (12 outcomes) [26, 27, 30, 32, 33, 37, 39, 41–45], with time taken for the ulcer to heal used as an outcome in ten studies [24, 25, 29, 31, 34–36, 41, 42, 46], and reduction in wound area size used twice [28, 29]. It should be noted that whilst these outcome measures may be clinically meaningful within VLUs, they do not have a willingness-to-pay threshold attached to them [such as with the quality-adjusted life-year (QALY)], and they do not facilitate comparison with other economic evaluations conducted within other disease areas.

For the CUAs, QALYs were the only measurement of utility used. Nine decision-analytic models sourced utilities for their QALY measurements from the 2007 study by Clegg and Guest [25, 27, 34, 36, 37, 39, 43–45]. This paper used standard gamble methodology on a sample of 200 participants, including both the general population and VLU patients, to allow for utilities to be assigned to different VLU states [44]. The remaining three QALY measurements obtained utility values from randomised controlled trials that used the EuroQol 5D and/or Short Form 6D questionnaires [24, 38, 46].

The payer perspective was taken in 18 papers [24–28, 30, 31, 34, 36–39, 41–46], with only two studies (9%) stating that they had taken a societal perspective [29, 40]. Three papers did not state the perspective used [32, 33, 35].

Markov models were the most common decision-analytical approach, being used in 12 (52%) studies [24, 28, 29, 34, 35, 38, 39, 41–44, 46]. The most common time cycle length used for these models was 1 week with a 1-year time horizon, found in five studies [24, 34, 35, 41, 46]. Other time horizons varied greatly, from 14 days [28] to 12 years [38]. Decision-tree models were used in five studies [27, 30, 31, 36, 45], and the remaining six studies did not explicitly state their model type [25, 26, 32, 33, 37, 40]. For these six studies, no figure was included to describe the model structure, making it difficult to appreciate the type of model used. Transition probabilities for models were described in 12 of the 23 studies (52%), with two studies stating that probabilities could be “obtained directly from the corresponding author” [39, 43]. Nine studies did not state any of the transition probabilities used in their models [25, 26, 31–33, 35–37, 45].

Of the 23 studies included in the review, 22 reported that the intervention was dominant. The one study without a dominant intervention, evaluating a skin substitute (Apligraf), had an incremental cost effectiveness ratio (ICER) of $14 per ulcer-day avoided [40]. Almost all studies (91%) in the review were funded by medical companies, other than the studies by Ashby et al. (HTA programme fund) [38] and by Iglesias and Claxton (funded by the University of York) [46].

Quality Assessment

The reporting and methodological quality of studies was evaluated using the Philips Checklist [20]. The responses given for each study are included in Electronic Supplementary Material 1 (Appendix 3). To allow for analysis of these results, a measurement system calculated the percentage of criteria fulfilled. A yes (Y) response counted as one point, a no (N) response as no points, a partial (P) response as half of a point, and criteria given an ‘N/A’ response were discounted from the calculation. Although this scoring system could be criticised for assuming equal weighting to all criteria, it allows for an estimate of number of items fulfilled. The scores from this ranged from 27% up to 89%, with the scores for included studies found in Electronic Supplementary Material 1 (Appendix 3).

The following discussion of study quality follows the three domains of the Philips Checklist, ‘model structure’, ‘data’ and ‘uncertainty and consistency’.

Model Structure

Selecting an appropriate model structure is key for decision-analytic models. Appropriate models should best reflect the disease process, relevant healthcare system and available evidence for the decision problem [47].

In the majority of studies, relatively little explanation was given for the selection of the type and structure of decision-analytical model. Only two studies (9%) gave full evidence as to how the model structure was formed [38, 39], whilst other possible model structures were considered by only one study [39]. Model structures should align with the biological and clinical theory of the condition studied, and should be driven by the study question rather than data availability [20, 48]. By not detailing the rationale for the model structure used, it cannot be determined whether the model was the best fit for the study objective, or whether it was chosen due to data limitations. This reduces the perceived quality of the study.

The Markov models used in the included studies often had a simplistic structure. Commonly, the following states were used, ‘healed’, ‘unhealed’ and ‘dead’ [38, 41, 46], whilst some studies included more detailed states such as ‘improved’, ‘worsened’ and ‘complications’ [24, 44]. The cycle lengths used in the Markov models ranged from 1 day to 1 month, with this length mostly determined by the frequency of outpatient appointments. This is despite it being recommended that cycle length is based on the natural history of disease and not local treatment patterns [49].

Thirteen studies (57%) were considered to have a time horizon that was insufficient to reflect important differences between the options [25–27, 29, 30, 32, 33, 36, 37, 40, 43–45]. Although most VLUs are cured within 3–4 months, treatment can go on to take much longer [2]. The time horizon was often determined from available trial data [28, 29, 39, 42–44], with no attempts to extrapolate this data to a more suitable economic time horizon. Guidelines suggest that a lifetime horizon is considered optimal for all decision-analytic models to ensure that no associated costs or benefits are missed [13, 50]. No study used a lifetime horizon within this systematic review and only four papers (17%) fully justified using a shorter time horizon [28, 29, 38, 46].

Only two papers (9%) adopted a societal perspective [29, 40]. Societal perspectives are recommended, as narrower perspectives may lead to important costs and benefits being ignored [13, 50, 51]. Of these two studies, Scanlon et al. did not include any indirect costs despite reporting the use of a societal perspective, reasoning that “due to the relatively high average age of people with venous leg ulcers, no costs of lost workdays were applied” [29]. Whilst Sibbald et al. included loss of work as the only indirect cost considered [40], excluding other potential indirect costs such as patient transport or time lost from daily activities. This review found an absence of a true societal perspective when evaluating the cost effectiveness of VLU treatments.

Half-cycle corrections were not used by any papers using a Markov model within this review. Reasoning for their exclusion was only given by Nherera et al., who stated “for the study model, the cycle length was considered to be small enough not to require a half-cycle correction” [34]. Whilst half-cycle correction is a criteria on the Philips Checklist, the need for such a correction is contested [52, 53].

Data

An important strength of model-based economic evaluations in comparison with trial-based evaluations is that they allow authors to source all relevant evidence to help answer the study objective [13, 50]. Despite this, only nine (39%) of the studies used systematic literature reviews to inform model parameters [24, 26, 29, 30, 32–34, 38, 41]. The details of what these reviews involved and the results from them were embedded within the studies, with no literature reviews published separately. The remaining studies used single sources of data, with nine models using randomised controlled trials for data inputs [28, 31, 35, 39, 40, 42–44, 46] and five using The Health Improvement Network (THIN) database [25, 27, 36, 37, 45]. There is potential when using only a single source of data that the generalisability of the results is reduced.

Expert panels were used in 12 (52%) of the studies [26, 29, 30, 32–34, 37, 39–42, 44]. These panels mostly consisted of healthcare professionals whose estimations were used to compensate for gaps in research data. Whilst the use of expert opinion is acceptable, it was adjudged by Coyle and Lee to be at the bottom of the hierarchy for data sources [54], and should only be used as a last resort when no other data can be obtained [55]. Two studies did not detail the methods used in eliciting expert opinion or who the experts were [34, 41].

Uncertainty and Consistency

Using decision-analytic models in economic evaluations allows authors to assess and analyse the uncertainty associated with decision making [13, 49, 50]. The Philips Checklist divides uncertainty into four dimensions: methodological, model structure, heterogeneity and parameter uncertainty.

Uncertainty regarding methodology, model structure and heterogeneity were poorly assessed by the studies included in the review. Methodological uncertainties were evaluated by only five studies (22%) [29, 35, 38, 42, 46]. These studies ran their sensitivity analyses with different weighting of data used [46], different time horizons [29] and with different clinical scenarios [35]. Structural uncertainties were not addressed by any studies in the review. Heterogeneity was assessed by only two studies (9%), which examined populations from a different country [30] and with a different type of VLU [27].

Parameter was the most commonly addressed dimension of uncertainty, which was completed by 16 studies (70%) [25, 27, 29–31, 34, 36–39, 41–46]. Sensitivity analyses were run on parameter estimates including resource use, unit costs and clinical outcomes. It is suggested that parameter uncertainty should be tested using probabilistic analysis [20], which was performed by six out of the 16 papers (38%) [27, 30, 34, 38, 41, 46]. The majority of studies instead conducted univariate deterministic analysis [25, 27, 29, 30, 36, 37, 39, 42, 44, 45]. Univariate analysis allows for the impact of individual parameters to be assessed but is not recommended for testing uncertainty and should instead be used in the model development process [50]. It is also recommended that when testing uncertainty, the range of values for a parameter should be well justified [20]; however, many models used arbitrary percentage variation without reasoning (e.g. ± 20% base case) in their sensitivity analysis [25, 27, 29, 30, 36, 37, 39, 42–45].

There were some flaws in how the results of the modelling studies were presented. Kerstein et al. [33] and Meaume and Gemmen [26] reported average cost-effectiveness ratios (ACERs) rather than ICERs, which are considered as the standard summary measures of economic evaluations [7, 50]. ACERs should not be used to report the results of analyses, as they assume that all outcomes are produced at equivalent cost and spread additional costs from an intervention over all previous outcomes, which is not the case [56, 57]. Negative ICERs were reported by Walzer et al. [24], despite describing the intervention (Sachet S) as dominant. It is unconventional to report negative ICERs, given that, presented without context, they do not show if an intervention offers a decreased cost for an increased benefit or an increased cost for a decreased benefit [50].

The 2012 study by Guest et al. [45] stated that cost effectiveness would be determined by the clinical outcomes of QALY gain and percentage of healed patients. However, after the results from the study showed that there were no differences in costs or clinical outcomes for all interventions, the study then reported on a previously unmentioned outcome (wound size reduction) as the reasoning behind recommending one of the interventions as the preferred strategy. Using post hoc outcome measures has the potential for bias, given that they may be selected on the basis of clinical or statistical significance of the observed results. The validity of this model is therefore reduced, with the potential to mislead decision makers.

Discussion

This systematic review has shown several common themes in published decision-analytic models for VLU treatment. The most common model types were Markov models, with new dressing technologies the most common intervention investigated. The majority of studies were funded by the health technology companies whose product was being evaluated, and were found to be the dominant treatment strategy.

The major themes when critically appraising the reporting and methodological quality of the decision-analytic models were the limited data sources and the limited length of time horizons over which interventions were evaluated. The most significant finding from this review was the large number of papers that reported little or no details of the modelling techniques or structure used for the study [25, 26, 32, 33, 37, 40]. The lack of reporting detail may impact on how usable the results of these studies are. Although it should be noted that these criticisms are not unique to VLU-based decision-analytic models. A systematic review of all decision models in UK health technology assessments by Cooper et al. in 2005, found that only 10% of studies across all disease areas suitably reported the process involved in developing the model structure [55].

This review is believed to be the first to focus on decision-analytic models for VLU treatment but has found similar results to other systematic reviews in the domain of wounds. A 2018 review by Cheng et al. reviewed all types of economic evaluations for chronic wounds [58]. The 12 decision-analytic models appraised in the Cheng review had similar methodological quality issues as the models included in this review, with short time horizons and few studies conducting analyses from a societal perspective. The researchers in this paper also criticised the absence of model validation, and models using single trials as their data source [58]. There were no decision-analytic models relating to VLU treatment in this study, meaning that none of the models evaluated in this review had been previously appraised by Cheng et al. Similar methodological shortcomings were commented on in a systematic review by Langer and Rogowski in 2009 [59]. This review focused on human-cell-derived wound care products for both VLUs and diabetic foot ulcers, and found that some of the models included had unsuitably short time horizons, used deterministic rather than probabilistic sensitivity analysis and methodological issues with ICER calculations and reporting [59].

There are several limitations of this review, which may have affected the results. The study selection only included published articles available in full text in English, meaning potentially relevant studies would have been omitted. The use of the Philips Checklist also has limitations. For example, the authors of the Philips Checklist acknowledge that the framework cannot guide appraisal on the suitability of individual model structures and structural assumptions themselves [20]. As well, many of the questions in the checklist require subjective interpretation, and whilst the error resulting from this would have been minimised by the use of two researchers, this may still be implicit within the results. Furthermore, the Philips Checklist was published in 2004, after a number of studies analysed in this review [26, 32, 33, 40, 42]. It may therefore be deemed inappropriate or unfair to judge such studies on criteria that were formed after their publication, given how modelling techniques and standards are likely to have changed over time. Word limitations in journals may also limit the detail authors can include in their studies regarding model structure. This should not be an issue for more modern papers, however, due to the increased likelihood of supplementary materials, but older papers may have been restricted when describing the methodology of their studies.

This review strengthens the calls of previous reviews for future decision-analytic models to improve their reporting quality [58, 59]. Despite systematic evidence by Cooper et al. showing that model-based evaluations are not being sufficiently described in studies regardless of disease area, reporting thoroughness does not appear to be improving with time [55], a finding echoed within this review. Future decision-analytic modelling studies should make use of the increasing ability to include supplementary online materials, as well as other transparency initiatives such as sharing of model code, to greater facilitate reporting clarity. There is also value in publishing the results of any systematic literature reviews conducted to identify the baseline data for modelling studies. This review highlights eight studies that performed their own systematic reviews to identify baseline data inputs [24, 26, 29, 30, 32, 33, 38, 41], with Nherera et al. the only paper to utilise a previously published review [34]. Since none of these studies published the methodology or results of their reviews separately, this may have resulted in a duplication of researcher effort, which could have been focused on other areas of the modelling process, such as model validation. There is also potential for future research to develop guidelines specifically for VLU-based models, similar to guidelines developed by the Mt Hood Diabetes Challenge Network [60]. A similar review focusing on trial-based economic evaluations for VLU treatments would also be of worth, as there is still a significant gap in the literature for this area. The review by Cheng et al. covered only three trial-based evaluations for VLUs [58], finding all three of these to be cost-saving, whilst a review by Weller et al. only focused on one treatment option (compression therapy), finding five relevant, within-trial studies [61].

Conclusion

Venous leg ulcers are an ever-increasing burden on the health of the ageing population and on the budget of healthcare providers. This review has shown that there is a sizeable number of decision-analytic model-based economic evaluations available to decision makers, but with a large variance of methodological quality across the studies. Common issues related to improper evidence for model structure, inadequate time horizons and limited data sources. These limitations in study methodology provide inferior quality evidence for decision makers to evaluate the cost effectiveness of potential interventions. By acknowledging and avoiding the shortcomings of the evaluations included within this review, future work should be of a higher reporting quality, thus providing better quality evidence to inform the cost effectiveness of different venous leg ulcer interventions.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Compliance with ethical standards

Data Availability

The search strategy, data extraction template along with quality appraisal responses are provided in Electronic Supplementary Material 1, attached to this manuscript. The completed data extraction form is included as Electronic Supplementary Material 2.

Author Contributions

AL contributed to the design of the study, carried out the searches, screening and data extraction, contributed to interpretation of data, and drafted and critically reviewed the paper. EM contributed to the design of the study, carried out the screening and data extraction, contributed to interpretation of data, and drafted and critically reviewed the paper. NL contributed to the design of the study and critically reviewed the paper.

Funding Source

No funding was received for the writing of this manuscript.

Conflict of interests

The authors, AL, EM and NJL, declare that they have no competing interests.