Invited Editorial presents an accurate summary of the results of two randomised placebo-controlled trials of vertebroplasty

Abstract

Our recent editorial in the Journal presents an accurate summary of our two randomised trials of vertebroplasty, which found no benefit of vertebroplasty over placebo. Participants in both trials are representative of patients seen in clinical practice and who would qualify for government-subsidised funding of vertebroplasty in Australia. Clinical experience and previous published literature are likely to have overestimated the treatment benefit of vertebroplasty for many reasons. This is why randomised placebo-controlled trials are required to determine the efficacy of treatment interventions, particularly when the condition being treated is self-limiting and the primary end point is improvement of symptoms. Based on the best evidence currently available, the routine use of vertebroplasty outside of the research setting for painful osteoporotic vertebral fractures appears unjustified.

Our editorial (1), written at the request of the Editor of the Medical Journal of Australia, presents an accurate summary of our two randomised placebo-controlled trials of vertebroplasty published in the New England Journal of Medicine (2, 3). We discussed the negative findings of our high-quality trials in relation to the published literature and explained why clinical experience and trials with inadequate methods can exaggerate treatment benefits.

Vertebroplasty has been used to treat painful osteoporotic vertebral fractures for ten years and has been publicly funded in Australia for over four years despite the absence of robust evidence to support its use. It is well accepted that biased uncontrolled observations and results from poorly designed studies can mislead decision making in health care, from treatment decisions for the individual patient to formulation of national public health policies (4). It might therefore be more appropriate for proponents of the procedure to stop searching for spurious reasons to dismiss our trial results and reflect upon how it was possible for vertebroplasty to become the standard of care in the absence of appropriate evaluation in the first place (5).

We have made our methods and results accessible to public scrutiny by publishing both the protocols and results in peer- reviewed journals. According to well-established criteria (6), the results of both trials are at low risk of bias (Table 1). We have also scrupulously responded to erroneous criticisms of our trials made by Dr Clark and others (7–10). Most, if not all of the concerns raised by Clark et al in their current critique are readily addressed. Clark et al also present what may be pertinent observations based upon their collective but unpublished experience of having treated 2,500 patients with vertebroplasty. However, other than a quasi-experimental open controlled before-after observational study that compared the outcome of volunteers who agreed to undergo vertebroplasty to those who declined (11), and which was at high risk of bias (Table 1), they provide no data and scarce references in support of their assertions.

Table 1.

Comparison of risk of bias of our randomised placebo-controlled trials and the Sydney observational study

	Buchbinder et al (2)	Kallmes et al (3)	Diamond et al (11, 18)
Randomisation adequate?	Yes	Yes	No
Allocation concealed?	Yes	Yes	No
Groups similar at baseline?	Yes	Yes	Data not reported for symptom duration, opioid analgesia, hospitalization
Patients blinded?	Yes	Yes	No
Care providers blinded?	Yes	Yes	No
Outcome assessors blinded?	Yes	Yes	No
Sample size calculation described and acceptable?	Yes	Yes	Not reported
Primary endpoint specified?	Yes	Yes	No
Drop out rate described and acceptable?	Yes	Yes	Not reported
Timing of outcome assessment similar?	Yes	Yes	Not reported for 6 to 12 month assessment (mean 215 days, range, 57 to 399 days)
Intention-to-treat analysis?	Yes	Yes	Not reported
NHMRC hierarchy of levels of evidence (I highest to IV lowest level (22)	II	II	III-2

Drs. Diamond and Clark were valued collaborators throughout the implementation of one of the two trials, the INVEST trial (3). Their enthusiasm for and experience in spine augmentation were key factors in allowing successful completion of that trial. Their plans to carry out a future, similar trial in Sydney are laudable and hopefully will add important information to our understanding about augmentation. It is well accepted that randomised placebo-controlled trials are the only truly valid means of evaluating the efficacy of treatment interventions particularly when the condition is self-limiting and the primary endpoint is improvement of symptoms.

Patient selection

As stated by Clark and colleagues, the natural history of acute osteoporotic vertebral fractures is generally favourable with pain gradually subsiding over 6 to 12 weeks, while a minority present with more severe symptoms often requiring opioid analgesia and sometimes hospitalization. We also agree that those with more severe and persistent symptoms are likely to derive the most benefit from effective treatment. Examination of the eligibility criteria and baseline characteristics of study populations in both trials confirm that these are the patients we studied. For example, nearly half of the patients in both the INVEST and Australian trials had overall or average pain severity of 8 or greater on a 0–10 point scale at baseline and the majority of participants in both trials were taking opioids (Table 2).

Table 2.

Comparison of baseline characteristics of our randomised placebo-controlled trials and the Sydney observational study

	Buchbinder et al (2)	Kallmes et al (3)	Diamond et al (11, 18)
N	N = 78 (V: 48; P: 50)	N = 131 (V: 68; P: 63)	N = 79 (V: 55; UC: 24) (‘Follow-up’ study N = 126; V: 88; UC: 38)+
Mean Age (years)	V: 74.2	V: 73.4	V: 76.5
	P: 78.9	P: 73.3	UC: 76.3
Female (%)	V: 82	V: 78	V: 64
	P: 78	P: 73	UC: 83
Symptom duration (weeks)	V: 9 weeks (median) P: 9.5 weeks (median)	V: 16 weeks (mean) P: 20 weeks (mean)	Not reported for total study sample or by treatment group†
Number with short symptom duration	32% < 6 weeks	20% < 6 weeks 41% ≤13 weeks	100% 1–6 weeks
Pain rating Mean (SD)	V: 7.4 (2.1)# P: 7.1 (2.3)#	V: 6.9 (2.0)* P: 7.2 (2.0)*	V: 19 (4)^ UC: 20 (5)^
Modified Roland-Morris	V: 17.3 (2.8)	V: 16.6 (3.8)	Not measured
Disability (0–23) Mean (SD)	P: 17.3 (2.9)	P: 17.5 (4.1)
Percent taking opioids at baseline	V: 79 P: 85	V: 56 P: 63	Not reported

V vertebroplasty, P placebo, UC usual care

⁺ Only baseline data from the original study are presented as the follow up study included patients recruited during a second year

^†Participants reported to have a 1–6 week history of pain

^#Overall pain over the course of the previous week on a scale of 0 to 10, 0 = no pain to 10 = maximum imaginable pain

^*Average pain intensity during past 24 hours on a scale of 0 to 10, with higher scores indicating more severe pain

^{^}Pain score out of 25 calculated by assessing pain associated with five activities: walking, climbing in and out of a chair, bathing, dressing, and at rest each on scales from 0 (no pain) to 5 (maximal pain; score recorded immediately on awakening, before the administration of the morning dose of analgesia)

The study populations in both trials have been deemed by others to be typical of those seen in clinical practice (5, 12). Our participants also share comparable baseline characteristics, including levels of pain and disability, with participants in other vertebral augmentation trials including both the VERTOS (13) and FREE (14) trials. Importantly, both trials included patients similar to those who would qualify for interim government-subsidized funding of the procedure in Australia specifically for patients whose pain is not controlled by conservative medical therapy (15). While duration of medical therapy is not specified, historically, this has ranged from at least 4 to 6 weeks.

While Clark et al suggest that we enrolled patients very different to the ones they treat, other than a longer duration of symptoms in some participants, our trial population also appears similar with respect to baseline characteristics to patients in the Diamond study (Table 2)(11). While participants in their study were required to have acute symptoms (1 week to 6 weeks), mean duration of symptoms within each treatment group was not reported making it unclear whether or not the treatment groups were comparable at baseline. It is not possible to directly compare baseline pain scores with the Diamond study as they assessed maximal pain for five activities to derive a score out of 25 (10). However it is likely that participants in our trials had comparable or greater baseline levels of pain as assessment of maximal pain with activity likely generates higher mean pain rating scores than assessments of resting, average or overall pain (16).

We agree that the decision to proceed to vertebroplasty should be informed by both the clinical and radiological assessment of the patient. Contrary to Clark et al.’s claims, an interventional radiologist was involved in confirming eligibility of all participants in the Australian trial and both outpatients and hospitalized patients were included.

Selection Bias

There is no evidence that either trial was affected by selection bias. Participation rates of 36% and 30% of eligible patients, in the Australian and INVEST trial respectively, is considered more than acceptable by usual trial standards, particularly considering both trials included a sham procedure. Furthermore, eligible patients who declined enrollment in the INVEST trial had similar levels of pain and disability to those who participated (17).

On the other hand, the study by Diamond et al was at high risk of selection bias (Table 1). Assignment of treatment was not random; it is well established that volunteers who agree to undergo intervention are likely to have better health outcomes than those who refuse (volunteer bias); the participation rate was not reported in the original published report (11); and the follow up publication reported the combined outcomes of the original cohort as well as a further 47 participants recruited over a second year (18).

While it is true that many patients will not consent to be randomised into placebo-controlled trials, and the rate may be higher for trials assessing invasive treatments, this will not affect the internal validity of the trial. Furthermore, the assertion that participants with severe pain would more often opt out of the trial and proceed with vertebroplasty is not supported by data from the Diamond study (11). When offered vertebroplasty, 30% of patients preferred usual medical care and their baseline pain levels were comparable to those who agreed to undergo the procedure. This assertion is also without foundation for the Australian trial, as the availability of vertebroplasty outside of the trial was limited in our setting, particularly prior to government subsidized funding. In fact, most of those who declined participation in the Australian trial did not wish to undergo any procedure also preferring to adopt a ‘wait-and-see’ approach.

Timing of vertebroplasty

Current evidence does not support the claim made by Clark et al that the results of vertebroplasty is better in their hands and/or when the procedure is restricted to those with symptoms for 6 weeks or less. The original study report by Diamond et al failed to demonstrate any difference in clinical outcomes between treatment groups at 6 weeks or at 6 to 12 months (11) and exploratory analysis of the approximately 20% of patients in the INVEST trial who had fractures less than 6 weeks duration did not find any evidence of benefit of vertebroplasty over placebo. There was also no evidence that symptom duration was a treatment effect modifier in the Australian trial.

While we acknowledge that power was suboptimal for subgroup analysis when performed for both trials individually, the suggestion that vertebroplasty might be more effective for a subgroup of patients with very recent symptoms (i.e. less than 6 weeks) is improbable for several reasons. As Clark et al and others (19) have outlined, most osteoporotic spinal fractures heal quickly; this implies that most people would be unlikely to benefit from early invasive intervention and a subgroup with more persistent symptoms may be more likely to derive benefit. Secondly, the net overall effect of vertebroplasty in both trials was close to zero, making it unlikely that there would be subgroups that would benefit from the procedure (20). The only way that a proportion of patients could receive a large effect from vertebroplasty would be if a substantial proportion were made worse, a scenario that doesn’t reflect the available data (7).

Finally, as Clark et al acknowledge, the results of an open randomised controlled trial of vertebroplasty versus usual care that included only patients with very recent symptoms also failed to demonstrate any benefit of vertebroplasty over usual care at 3 months (21), further refuting the contention that benefits are more likely if the treatment is given early. While Rousing et al did find a significant reduction in pain at 24 hours following vertebroplasty no comparative data was presented so this finding is of uncertain significance.

Vertebroplasty technique

As we have stated previously (7), the decision by one centre to withdraw from the Australian trial once government approval for reimbursement became available does not diminish the internal validity of the Australian trial. The other centre did not formally withdraw and contributed patients over the first 2 years. Clark et al also cast doubt on the validity of our trials by questioning the vertebroplasty techniques used in our trials. Both trials performed vertebroplasty according to standard methods and we are not aware of any evidence to support claims made by Clark et al about cement volume or distribution.

Power and expected size of the treatment benefit

Although neither trial reached their large pre-specified sample sizes, calculated to test hypotheses regarding safety, including incident vertebral fracture rate in the Australian trial, and to detect very small differences in outcome in the INVEST trial, both trials had more than adequate power to detect clinically important differences between groups with respect to their primary efficacy end points (5).

As we stated in our Editorial, anecdotal evidence, uncontrolled and open studies are well known to overestimate treatment benefit for many reasons particularly when studying a self-limiting condition. In the case of vertebroplasty, reasons include the favourable natural history of vertebral fractures; regression to the mean; the placebo effect; unblinded outcome assessment; and loss to follow up which is generally not random and is biased towards a more favourable outcome. The Vertos2 trial is an open randomised controlled trial that compares vertebroplasty to usual care. As such it is also likely to have exaggerated treatment benefit for many of the reasons we have outlined.

Conclusion

Clark et al appear to suggest that a lower standard of evidence should apply for procedural medicine. New drug treatments are not considered to be of proven efficacy until, at the very least, they have been evaluated in randomised controlled trials. Even then, their safety is not guaranteed, particularly for uncommon adverse effects or those for which there is a long delay between exposure and clinical manifestation. At the present time in Australia, it appears that new interventions are not always afforded the same degree of scrutiny as new drug therapy, although we would argue that in the interests of our patients, the same onus of proof should be applied.

All patients should be honestly informed about the evidence for and against vertebroplasty. Not only is its efficacy unproven, the procedure poses some immediate, albeit small risks of cement leakage, infection and injury to the spinal cord, and it is not yet established whether or not it increases the risk of further vertebral fracture. Based upon the results of our two trials that do provide the highest level of evidence we have to date regarding the value of vertebroplasty, routine use of this procedure outside of the research setting now appears unjustified. If there is any remaining uncertainty about the value of vertebroplasty for specific subgroups of patients this should be addressed by the performance of further high quality randomised placebo-controlled trials. In the meantime, primum non nocere, or first, do no harm (Hippocrates ~410 BC).

This is a commentary on article Buchbinder R, Osborne RH, Kallmes D. Vertebroplasty appears no better than placebo for painful osteoporotic spinal fractures, and has potential to cause harm. Med J Aust. 2009;191(9):476-7. and Clark WA, Diamond TH, McNeil HP, Gonski PN, Schlaphoff GP, Rouse JC. Vertebroplasty for painful acute osteoporotic vertebral fractures: recent Medical Journal of Australia editorial is not relevant to the patient group that we treat with vertebroplasty. Med J Aust. 2009;192(6):334-7.