Letter: Evidence‐Based Wound Care in the UK: A Response to David Leaper's Editorial in International Wound Journal April 2009 6 (2)

In David Leaper's editorial published in April 2009 (1), some important points are made. These include the necessity for evidence‐based practice and standards, the limited evidence for some wound care management strategies, and the difficulty this poses for making recommendations for clinical practice. However, other points made about the usefulness of observational evidence for the evaluation of interventions and that in the field of wound care ‘further Cochrane systematic reviews are not likely to achieve much’ (p. 90) are debatable (1).

Professor Leaper confuses the primary functions of Cochrane reviews and clinical guidelines, the former being to address the question of the effectiveness of particular interventions and the latter being to provide recommendations to guide clinical practice. He criticises Cochrane reviews for failing to admit all the available (experimental and observational) evidence, unlike guidelines, and then producing recommendations that further research is needed, while failing to provide clinical guidance.

Professor Leaper also argues that Cochrane reviews do not benefit from the input of an expert panel and that review authors have scientific skills with no topic‐related knowledge. The Cochrane Collaboration is an organisation in large part driven by the efforts of those carrying out systematic reviews on a voluntary (unpaid) basis. Many of these volunteer review authors are clinicians who are very keen to further the evidence base in topic areas where they have a high level of knowledge, experience and expertise. In reviews of wound management, these people include physicians, surgeons, nurses and podiatrists. In addition, each Cochrane review is subject to a very rigorous peer review process by both clinical and methodological experts, and so overall there is ample opportunity for clinical input, helping to ensure that Cochrane reviews are methodologically sound but also relevant in terms of topic coverage and interpretation of findings. In commenting on a systematic review on antibiotics and antiseptics used with venous leg ulcers, Professor Leaper describes ‘the lack of expertise on the panel’. We are pleased to confirm that this review team represented a wealth of clinical knowledge and experience including medicine, nursing, pharmacology and clinical research (2).

Cochrane reviews in the field of wound care (and other areas) are important because in addition to summarising the evidence on the effectiveness of interventions for the benefit of patients, health care practitioners and policy‐makers, they highlight gaps in the evidence, make recommendations for further research to address these gaps and identify design flaws in published randomised controlled trials (RCTs) so that future researchers can address methodological weaknesses by improving trial design and conduct. Unfortunately, it is often the case that published trials may have methodological inadequacies. There is therefore room for methodological improvement but the answer does not lie in using observational research designs to evaluate effectiveness. That the Cochrane Collaboration has highlighted that many wound care treatments have been inadequately evaluated using inappropriate designs, poorly conducted trials and small samples should spur wound care researchers to address these issues.

An ‘unhelpful systematic review’ (p. 90) of topical negative pressure therapy (TNP) is used to illustrate the point in Professor Leaper's editorial that alerting clinicians of the shortcomings and weaknesses in this evidence in a recent Cochrane review may be ‘unacceptable’ to practitioners, particularly as ‘…many experienced practitioners… are aware that TNP has saved lives and limbs' (p. 90) (1). However, the dangers of relying on clinical experience alone have been well documented and includes ‘bad’ decisions that are not in the best interests of the patient (3). Leaper then presents an argument for considering lower levels of evidence and ‘time‐honoured practice’ to guide clinical practice. However, it is unclear why aiming for rigorous evaluations of wound care treatments should pose any greater difficulties than for other healthcare treatments and why we would want to disadvantage patients by relying on inferior evidence to make decisions about optimal care. In the absence of well‐conducted research, it is also worth asking how practitioners know that TNP, for example, has resulted in positive outcomes and whether the perceived positive effects are not because of some other factor.

Related to this, it is useful to highlight how some ‘time‐honoured practices' that are assumed to be the best for the patient may not be. It is only when RCTs (or systematic reviews of RCTs) are conducted that this becomes evident. For example, use of compression stockings in the surgical setting is known to reduce the risk of deep vein thrombosis (DVT). However, a recent trial (4) has shown that the assumption (as made in clinical guidelines on the topic) that graduated compression stockings would also be beneficial for other patient populations with reduced mobility (in this case patients unable to walk following stroke) was inaccurate. In fact, the large RCT showed that there was no difference in the incidence of DVT between the group with no stockings and those with stockings. The only significant differences between the groups was the greater numbers of skin problems (skin ulcers, blisters and irritation) in the group that were wearing stockings. Clearly, ‘obvious’ assumptions can be false.

Given that the primary purpose of a Cochrane review is to summarise the evidence for questions of cause and effect, reviewers are correct in being cautious about providing definitive conclusions from less than reliable evidence. Admission of observational evidence would precipitate even more caution in Cochrane reviews as health care research is littered with examples where observational evidence can seriously mislead practitioners and the public. Some of these examples include diethylstilbestrol for recurrent miscarriage, class 1 c anti‐arrhythmics for preventing death post‐myocardial infarction, beta‐carotene for preventing lung cancer in smokers and hormone replacement therapy (HRT) for preventing cardiovascular disease in women.

It is disingenuous to argue that the case has been settled, as Professor Leaper does, that there is little difference in treatment estimates from well‐designed observational research and trials–the methodological research that argues such a position benefits from hindsight and is not widely accepted. There is plenty of evidence that non randomised studies exaggerate treatment effects 5, 6, 7. The examples offered above show that real harm can cause observational evidence–diethylstilbestrol (DES) did not reduce recurrent miscarriages (and had severe intergenerational effects) (8), suppressing arrhythmias with class 1 c anti‐arrhythmics increased deaths 9, 10, beta‐carotene increased mortality amongst smokers (11) and HRT did not prevent cardiovascular disease in women (12). These examples show that appraisal of other sources of evidence in respect of the effectiveness of treatment is fraught with problems. The use of observational studies for evaluating treatments is only recommended in very specific circumstances such as studying rates of diseases or harmful effects such as fatal events (13). However, further research is needed to determine whether causality can be soundly based on observational evidence (14).

Professor Leaper emphasises the importance of expert opinion; we agree that this can assist clinical guideline development in areas where research evidence is lacking. However, it is important to understand that opinion is not synonymous with evidence and the hazards of failing to take account of research findings have been well documented. In 1992, Antman et al. published a series of cumulative meta‐analyses of clinical trials evaluating interventions used with patients suffering myocardial infarction, the primary outcome of interest being survival (15). Retrospectively generated meta‐analyses were presented for different time points and were matched with recommendations from contemporaneous expert reviews of the literature (non systematic). The authors concluded that because the clinical experts had not used scientific and systematic methods when conducting their reviews, advice on some life‐saving therapies was delayed for more than a decade, whilst other treatments were recommended long after the research evidence showed them to be harmful (15).

Professor Leaper also compares a systematic review on the effects of parachute failures written for the BMJ Christmas edition (and its obvious all‐or‐none effect), with the situation in wound care management. All‐or‐none effects are rare–skin grafting for burns might be an example in wound care, but the marginal benefits (if any) of silver dressings on healing and similar products show that well‐designed randomised trials are necessary to remove the confounding effects of selection bias that are always present even in the very best of cohort studies. Most treatments in health care do not produce all‐or‐none effects. Confounding, whereby the apparent effect of the intervention is distorted because of the effect of an extraneous factor related to both the intervention and outcome, explains why the observational studies quoted above produced misleading results. Those examples also show that the study design most likely to produce a reliable estimate of effect is not an observational one. It is a randomised design.

When considering basing practice on the evidence from lower levels of evidence, practitioners need to consider the ethical imperative ‘first do no harm’. Harm need not be a clinical event. Harm can be created by choosing treatments for which there is little evidence of effect. Choosing one treatment over another represents an opportunity cost–an expenditure on an ineffective treatment means money may not be available for effective treatments, thus further disadvantaging patients. Harm also includes patients' negative experiences such as discomfort.

Professor Leaper is right to point out we do not have complete knowledge of the effectiveness of interventions in wound care and that guidelines can assist clinical decision making in the absence of perfect knowledge. However, guideline recommendations for treatment, intervention or prevention that are not based on high‐quality experimental studies can have serious shortcomings for patients as illustrated by the examples above. Furthermore, it is still the case that much wound care practice remains based on expert opinion and low‐level research, in part because most treatments are devices where there is not the same requirement for efficacy evidence for licensing (16). How should we explain to our patients that they are being treated with interventions that are not supported by the latest and best available scientific evidence? In the meantime, there is no sound reason for researchers choosing to use designs other than an RCT to evaluate wound care strategies.

Cochrane Wounds Group

Sally Bell‐Syer, MSc ¹ Marian Brady, PhD ² Julie Bruce ³ Nicky Cullum, PhD ⁴ Ruth Foxlee ⁵ Andrew Jull, RN, PhD ⁶ David Margolis, MD PhD ⁷ Liz McInnes, PhD ⁸ Andre Nelson, PhD ⁹ Susan O’Meara, PhD, RN ¹⁰ Dirk Ubbink, PhD, MD ¹¹