Adverse Event Recording and Reporting in Clinical Trials Comparing Lumbar Disk Replacement with Lumbar Fusion: A Systematic Review

Abstract

Study Design Systematic review.

Objectives (1) To compare the quality of adverse event (AE) methodology and reporting among randomized trials comparing lumbar fusion with lumbar total disk replacement (TDR) using established AE reporting systems; (2) to compare the AEs and reoperations of lumbar spinal fusion with those from lumbar TDR; (3) to make recommendations on how to report AEs in randomized controlled trials (RCTs) so that surgeons and patients have more-detailed and comprehensive information when making treatment decisions.

Methods A systematic search of PubMed, the Cochrane collaboration database, and the National Guideline Clearinghouse through May 2015 was conducted. Randomized controlled trials with at least 2 years of follow-up comparing lumbar artificial disk replacement with lumbar fusion were included. Patients were required to have axial or mechanical low back pain of ≥3 months' duration due to degenerative joint disease defined as degenerative disk disease, facet joint disease, or spondylosis. Outcomes included the quality of AE acquisition methodology and results reporting, and AEs were defined as those secondary to the procedure and reoperations. Individual and pooled relative risks and their 95% confidence intervals comparing lumbar TDR with fusion were calculated.

Results RCTs demonstrated a generally poor description of methods for assessing AEs. There was a consistent lack of clear definition or grading for these events. Furthermore, there was a high degree of variation in reporting of surgery-related AEs. Most studies lacked adequate reporting of the timing of AEs, and there were no clear distinctions between acute or chronic AEs. Meta-analysis of the pooled data demonstrated a twofold increased risk of AEs in patients having lumbar fusion compared with patients having lumbar TDR at 2-year follow-up, and this relative risk was maintained at 5 years. Furthermore, the pooled data demonstrated a 1.7 times greater relative risk of reoperation in the fusion group compared with lumbar TDR, although this risk decreased to 1.1 at 5-year follow-up. However, given the lack of quality and consistency in the methods of recording and reporting of AEs, we are unable to make a clear recommendation of one treatment over the other.

Conclusions Based on the currently available literature, lumbar TDR appears to be comparable in safety to lumbar fusion. However, due to lack of consistency in reporting of AEs, it is difficult to make conclusions regarding the true safety profile of lumbar TDR. Standardization in AE reporting will significantly improve the reliability of the current literature.

Introduction

Lumbar total disk replacement (TDR) was developed as an alternative to lumbar spinal fusion for the treatment of painful degenerative disk disease. Theoretically, lumbar TDR has the potential to decrease the impact of spinal stiffness and adjacent segment degeneration/stenosis associated with lumbar fusion.^{1 2 3 4 5 6 7 8 9} Lumbar TDR experienced an initial groundswell of support following its introduction, but recent studies have shown decreased enthusiasm, in part due to unpredictable results and long-term complications.^{10 11 12 13 14}

Similar to other new medical technologies, lumbar TDR prostheses undergo an extensive Food and Drug Administration (FDA) approval process, which is intended to establish safety and efficacy of novel medical devices. Although time-consuming and costly, this process represents an important safeguard for patients. Approval for devices begins with preclinical studies evaluating mechanical function, biocompatibility, neurotoxicity, and other factors. In motion-sparing devices such as lumbar disk arthroplasty, wear properties of the bearing materials must also be tested. If these initial studies demonstrate that a device is safe for implantation in patients, clinical trials to evaluate efficacy and adverse event (AE) occurrence is performed, typically in the form of a randomized controlled trial (RCT).

As a part of the FDA's evaluation, a summary of safety and effectiveness data (SSED) is generated. This process includes a detailed analysis of all AEs in the pivotal trial, whether they appear related to the procedure or not. Given this approach, as well as the follow-up of these patients for as much as 10 years, it seems likely that the information gathered from the SSED provides an upper limit of the AE profile of the device.

Recently, concern has been raised regarding inconsistent reporting of AEs related to cervical TDR and bone morphogenic protein.^{15 16} Currently available peer-reviewed RCTs of lumbar TDR describe study methodology and report primary and secondary clinical outcome variables, such as pain reduction, functional improvement, health-related quality of life, the ability to work, and pain medication use. Although nearly all trials report AEs as well, the degree of standardization of AE reporting in lumbar TDR RCTs has not been assessed.

Carragee et al questioned whether current peer-reviewed publications provide adequate reporting of AEs in the use of new spinal technologies.¹⁵ This concern was based on the discovery of substantial differences between the AEs reported in SSED trials involving bone morphogenic protein and those reported in peer-reviewed literature, with important AEs having been omitted in the published studies.¹⁵ This analysis has now been verified by two further independent reviews.

As a relatively new and unproven technology, a thorough analysis of the literature can provide both patients and clinicians with an opportunity to make an informed decision regarding the risks and benefits of lumbar TDR. In this article, we will compare the AEs associated with lumbar fusion with those associated with lumbar TDR to address three goals: (1) evaluate the quality of AE reporting; (2) compare the AE risk associated with lumbar fusion versus lumbar TDR; and (3) provide recommendations on how to improve quality of reporting in peer-reviewed publications as it relates to AE.

Study Rationale and Context: Clinical Questions and Objectives

1. To compare the quality of AE methodology and reporting among randomized trials comparing lumbar fusion with lumbar TDR using established AE reporting systems.

2. To compare the AEs and reoperations of lumbar spinal fusion with those from lumbar TDR.

3. To make recommendations on how to report AEs in RCTs so that surgeons and patients have more-detailed and comprehensive information when making treatment decision.

Materials and Methods

In this systematic review, PubMed, Cochrane collaboration database, and National Guideline Clearinghouse databases and bibliographies of key articles and previously published systematic reviews were searched through May 2015.

Inclusion criteria

The inclusion criteria were as follows: (1) randomized controlled trials with at least 2 years of follow-up comparing lumbar artificial disk replacement with lumbar fusion; (2) patients with axial or mechanical low back pain of ≥3 months' duration due to degenerative joint disease defined as any of the following: degenerative disk disease, facet joint disease, and spondylosis (Table 1).

Table 1. Inclusion and exclusion criteria.

	Inclusion	Exclusion
Patients	• Age ≥ 18 y • Axial or mechanical LBP ≥ 3 mo due to degenerative joint disease defined as any of the following:  − Degenerative disk disease  − Facet joint disease  − Spondylosis  − Degenerative spondylolisthesis if grade 1 or less and if back pain greater than leg pain (at least 85% of patients in a study must have the diagnosis of DJD)	• <18 y old • Unspecified chronic LBP without definitive dx • Spondylolisthesis grade 2 or greater • Isthmic spondylolisthesis • Neurogenic claudication associated with stenosis • Deformity • Fracture, cancer, inflammatory disease, infection, trauma, pregnancy-related LBP • Radiculopathy • Signs of neural compression
Intervention	Lumbar: • Anterior fusion • Posterior fusion (PLF/PLIF) • Circumferential fusion	• Revision surgery
Comparator	• Lumbar TDR	• Revision surgery
Outcome	• Surgery-related adverse events • Types of complications • Reoperations • Overall adverse events • Quality of adverse event methodology • Quality of adverse event reporting	• Patient reported or clinical outcomes • Follow-up rate <70%
Study design	• RCTs with 2-y follow-up or longer	• RCT that includes subjects from another study at the same follow-up time (i.e., duplicate or overlapping studies) • Nonrandomized comparison studies • Case reports • Nonclinical studies • Case series • Cost effectiveness studies • Prognostic study

Abbreviations: DJD, degenerative joint disease; dx, diagnosis; LBP, low back pain; PLF, posterior lumbar fusion; PLIF, posterior lumbar interbody fusion; RCT, randomized controlled trial.

Exclusion criteria

The exclusion criteria were as follows: patients <18 years of age or patients with any of the following: isthmic spondylolisthesis, neurogenic claudication associated with stenosis, radiculopathy, deformity, cancer, inflammatory disease, infection, trauma, pregnancy-related low back pain.

Outcomes

Outcomes studied were: (1) overall score of the quality of AE acquisition methodology and reporting; (2) overall score of AE results reporting (the scoring systems are published elsewhere and included in the online supplementary material [supplementary Table 1]¹⁶); (3) AEs defined as those secondary to the procedure and including medical and surgery-related events; and (4) reoperations.

Analysis

Overall scores for AE acquisition quality and reporting and the risk of AEs and reoperations are reported for each study. For individual studies, we calculated relative risks (RRs) and their 95% confidence intervals (CIs) comparing lumbar TDR with fusion. To test whether the differences in risks are statistically significant, we used the chi-square test. Where possible, we combined studies and calculated pooled risks and RRs using a random-effects model to account for heterogeneity. Forest plots were used to present pooled comparisons when possible. We performed the statistical analyses using Stata 9.1 (StataCorp LP, College Station, Texas, United States).

Results

A total of 18 RCTs were identified comparing lumbar TDR with lumbar fusion (Fig. 1). Eleven of these were excluded for various reasons (see online supplementary Table 2). Seven met the inclusion criteria and formed the basis for this report (see online supplementary Table 3). Two of these studies were continuations of prior RCTs with a longer follow-up of 5 years. These are reported separately and not pooled with the original five studies.

Fig. 1.

Flowchart showing results of literature search.

Clinical Question 1

We compared the quality of AE methodology and reporting among RCTs comparing lumbar fusion with lumbar TDR using established AE reporting systems.

• The description of methods used to assess AEs was poor, with a mean score of 2.3 on a 5-point scale. The majority of the studies scored a 2. Only one study scored a 5 (Table 2) (see online supplementary Table 4 for itemized scores).

• Most studies described how and when the AEs were collected; however, few studies provided clear definitions of AEs, provided a grading of the severity of AEs, or described the methods of analysis for AEs.

• The reporting of AEs and reoperations was slightly better with a mean score of 3.3 (Table 3; see online supplementary Table 5 for itemized score). All studies reported AEs and reoperations and most provided some level of categorization regarding those related to surgery.

• Although most studies had specific follow-up times where AEs or reoperations were measured, few reported the timing of when they actually occurred so it was unclear whether they were acute or more chronic in nature.

Table 2. Summary table of RCTs comparing fusion with lumbar TDR with risk of adverse events and quality of AE reporting.

Study (year)	Devicea	Industry funded	Fusion			TDR			RR	95% CI		Methods clarity score	Reporting results score
			AEs (n)	Subjects (n)	Risk (%)	AEs (n)	Subjects (n)	Risk (%)		Lower	Upper
2-y studies
Berg (2009)	Charité ProDisc Maverick	No	15	72	20.8	14	80	17.5	1.2	0.62	2.3	5	4
Blumenthal (2005)	Charité	Yes	10	99	10.1	20	205	9.8	1.0	0.50	2.1	2	3
Delmarter (2011)	ProDisc-L	No	7	72	9.7	5	165	3.0	3.2	1.1	9.8	2	3
Zigler (2007)	ProDisc-L	Yes	5	75	6.7	4	161	2.5	2.7	0.73	9.6	2	3
Gornet (2011)	Maverick	Yes	22	172	12.8	17	405	4.2	3.0	1.7	5.6	2	4
Total/mean			59	490	12.0	60	1,016	5.9	2.0	1.4	2.9
5-y studies
Skold (2013)	Charité ProDisc Maverick	No	15	71	21.1	14	80	17.5	1.2	0.63	2.3	0	2
Zigler (2012)	ProDisc-L	Yes	8	75	10.7	5	161	3.1	3.4	1.2	10.1	3	4
Total/mean			23	146	15.8	19	241	7.9	2.0	1.1	3.5	2.3	3.3

Abbreviations: AE, adverse event; CI, confidence interval; RCT, randomized controlled trial; RR, relative risk; TDR, total disk replacement.

^aCharité, Depuy Spine, Raynham, Massachusetts, United States; ProDisc, Synthes Spine, West Chester, Pennsylvania, United States; Maverick, Medtronic, Memphis, Tennessee, United States.

Table 3. Summary table of RCTs comparing fusion with lumbar TDR with risk of reoperations and quality of AE reporting.

Study (year)	Devicea	Industry funded	Fusion			TDR			RR	95% CI		Methods clarity score	Reporting results score
			AEs (n)	Subjects (n)	Risk (%)	AEs (n)	Subjects (n)	Risk (%)		Lower	Upper
2-y studies
Berg (2009)	Charité ProDisc Maverick	No	7	72	9.7	8	80	10.0	0.97	0.37	2.5	5	4
Blumenthal (2005)	Charité	Yes	9	99	9.1	11	205	5.4	1.7	0.72	4	2	3
Delmarter (2011)	ProDisc-L	No	6	72	8.3	4	165	2.4	3.4	1.0	11.8	2	3
Zigler (2007)	ProDisc-L	Yes	2	72	2.8	6	161	3.7	0.75	0.15	3.6	2	3
Gornet (2011)	Maverick	Yes	12	172	7.0	15	405	3.7	1.9	0.90	3.9	2	4
Total			36	487	7.4	44	1016	4.3	1.7	1.1	2.6
5-y studies
Skold (2013)	Charité ProDisc Maverick	No	7	71	9.9	8	80	10.0	0.99	0.37	2.6	0	2
Zigler (2012)	ProDisc-L	Yes	9	75	12.0	7	71	9.9	1.2	0.48	3.1	3	4
Total			16	146	11.0	15	151	9.9	1.1	0.57	2.1	2.3	3.3

Abbreviations: AE, adverse event; CI, confidence interval; RCT, randomized controlled trial; RR, relative risk; TDR, total disk replacement.

^aCharité, Depuy Spine, Raynham, Massachusetts, United States; ProDisc, Synthes Spine, West Chester, Pennsylvania, United States; Maverick, Medtronic, Memphis, Tennessee, United States.

Clinical Question 2

We compared the AEs and reoperations of lumbar spinal fusion with those from lumbar TDR.

• The 2-year risk of surgery-related AEs varied among studies, ranging from 6.7 to 20.8% in the fusion group and 2.5 to 17.5% in the lumbar TDR group (Table 2). The 5-year risks were 10.7 to 21.1% and 3.1 to 17.5%, respectively (Table 2).

• When combining the data in a meta-analysis, the 2-year pooled risk was 12% in the fusion group and 5.9% in the lumbar TDR group, which translates to an approximately 2 times greater risk of a surgery-related AE in the fusion group compared with the lumbar TDR group (RR = 2.0; 95% CI = 1.4 to 2.9; Table 2 and Fig. 2). The relative risk was similar for the 5-year follow-up (RR = 2.0; 95% CI = 1.1 to 3.5; Table 2 and Fig. 2).

• The 2-year risk of reoperation varied among studies, ranging from 2.8 to 9.7% in the fusion group and 2.4 to 10% in the lumbar TDR group (Table 3). The 5-year risks were 9.9 to 12% and 9.9 to 10.0%, respectively (Table 3).

• When combining the data in a meta-analysis, the 2-year pooled risk was 7.4 in the fusion group and 4.3% in the lumbar TDR group, which translates to a 1.7 times greater risk of a reoperation in the fusion group compared with the lumbar TDR group (RR = 1.7; 95% CI = 1.1 to 2.6; Table 3 and Fig. 3). The risks for reoperation were similar between groups for the 5-year follow-up (RR = 1.1; 95% CI = 0.57 to 2.1; Table 3 and Fig. 3).

Fig. 2.

Surgery related to adverse events comparing lumbar fusion with lumbar TDR. Abbreviations: CI, confidence interval; M-H, Mantel-Haenszel; TDR, total disk replacement.

Fig. 3.

Reoperations comparing lumbar fusion with lumbar TDR. Abbreviations: CI, confidence interval; M-H, Mantel-Haenszel; TDR, total disk replacement.

Clinical Question 3

We recommend the following changes to reporting AEs in RCTs so that surgeons and patients have more-detailed and comprehensive information when making treatment decisions.

• A designated committee independent of the principal investigator and industry sponsor, such as a Clinical Event Committee, can improve transparency and accuracy of the information.¹⁷

• Separate publication focused on detailed description of AEs in high-quality studies can provide a venue through which AEs can be separately reported and analyzed.

• Simple questionnaires that are reliable and easy to use and enable documentation of type, severity, and timing of AEs can be utilized.

• Guidelines from other specialties should be modified and incorporated for use in orthopedic surgery trials.

• Future trials should use the evaluation systems reported in this article as a checklist for their methods and results.

• A standardized reporting method should be used, such as the CONSORT statement, for reporting harm in clinical trials.¹⁸

Discussion

Our first goal was to evaluate the quality of AE reporting in lumbar fusion and lumbar TDR literature. Critical analysis of the currently available RCTs demonstrated a generally poor description of methods of assessing AEs. Although most studies did describe the method and timing of AE collection, there was a consistent lack of a clear definition or grading for these events. Furthermore, a high degree of variation in reporting of surgery-related AEs was seen in the reviewed studies. Although there was a somewhat better score of AE reporting, most studies lacked adequate reporting of the timing of AEs, and there were no clear distinctions between acute or chronic AEs.

Accepting these limitations, analysis of AEs and reoperations between lumbar spinal fusion and lumbar TDR demonstrated consistently higher risks of both for fusion as compared with lumbar TDR. A meta-analysis of the pooled data demonstrated a twofold increased risk of AEs in patients having lumbar fusion compared with patients having lumbar TDR at 2-year follow-up, and this relative risk was maintained at 5 years. Furthermore, a similar analysis of the pooled data demonstrated a 1.7 times greater RR of reoperation in the fusion group compared with lumbar TDR, although this RR decreased to 1.1 at 5-year follow-up. These findings were consistent across all RCTs reviewed and appeared to favor lumbar TDR over fusion. However, given the lack of quality and consistency in the methods of recording and reporting of AEs, we are unable to make a clear recommendation of one treatment over the other.

Given the absence of clear protocols and assessment tools for recording and reporting of AEs in clinical trials, it is perhaps not surprising that there remains a high degree of variability in the literature. Unfortunately, the most consistent approach has been the summary of SSED used by the FDA, which may include unrelated AEs, potentially leading to an overestimation of the risk associated with a given device. Other systems to report AEs recommend that events related to the disease or disease progression should be excluded from AE analysis.

Several features appear to be important in improving the consistency and effectiveness of AE reporting, including established definitions of AEs, documentation of timing method of discovery of AEs, and appropriate grading of the severity of each complication. This review demonstrates that even clinical studies of the highest level of medical evidence failed to provide this information. Thus, despite the strength of RCTs with regard to reducing the impact of confounding variables associated with patient diversity, surgical intervention, and surgeon related issues, we are unable to comment on this important aspect of comparative safety of these two surgical approaches to lumbar degenerative disk disease.

Although multiple studies have alluded to the fact that retrospective collection of data associated with morbidity and mortality significantly underestimates the risk of complications associated with complex spine surgery,^{19 20} substantial efforts at improving this situation are underway. Street et al proposed the Spine Adverse Event Severity (SAVES) system to document the occurrence of an AE at the time of surgery, at discharge, and later.¹⁹ This simple questionnaire requires minimal effort to complete, has good reliability, and is available in paper and electronic formats. Additionally, this questionnaire enables documentation of the type, severity, and timing of AEs and their effect on patients' length of stay. We recommend using the systems published in this article and the article by Anderson as a checklist when planning the methods and reporting of AEs alongside clinical results.¹⁶

A more-standardized approach has been outlined by Ioannidis, adding to the well-accepted CONSORT method of reporting RCTs.¹⁸ In this method, a systematic approach to reporting the methods used to capture AEs and the best reporting practices are outlined. Using this method as a standard, almost all studies even in highest-impact journals have been shown to have inadequately reported and documented AEs.^{21 22}

Additionally, modification and incorporation of the recommendations from other specialties may prove to be beneficial in establishing more uniform guidelines for orthopedic surgery trials. The National Cancer Institute and the World Health Organization have established guidelines to assist in reporting AEs in oncological clinical trials; these recommendations could potentially be modified for use in orthopedic and spine surgical trials.^{23 24}

We conclude that the lack of standardized definitions, absence of classification and collection protocols, and inconsistent reporting methodologies have led to a significant variation in AE recording and reporting in the peer-reviewed literature related to lumbar TDR. This deficiency creates a significant gap in the utility of clinical data, even among studies conducted using the highest level of sophistication from a medical evidence viewpoint. Efforts to standardize assessment and reporting of AEs in surgical trials appear warranted.

Summary and Conclusion

In short-term follow-up, there is a higher risk of AEs and reoperation in lumbar fusion compared with lumbar TDR, although differences in reoperation risks appear to diminish at longer follow-up. However, the lack of consistency in published RCTs prevents strong recommendations based on differences in AEs relating to these two surgical options. Because peer-reviewed publications lack adequate consistency in AE reporting, the impact of the conclusions presented in those reports is diminished. Going forward, improvements in methods to collect, assess, and report AEs are needed.

Editorial Perspective

Evidence-Based Spine-Care Journal/Global Spine Journal reviewers enthusiastically congratulate Hiratzka and coauthors on their study idea and execution. They found that lumbar disk replacements did indeed seem to have a reported lower rate of complications compared with fusion cohorts, at least in the short term. On the topic of AE reporting in our medical literature, this article raises two important issues:

1. Is there an inherent positive results reporting bias affecting us as clinicians, researchers, journal editors, and reviewers?

2. Are our current methods of collecting complications and AEs satisfactory in terms of reflecting an accurate picture of perioperative occurrences?

Over the previous decade, the introduction of major new technologies in spine, for instance in the form of minimally invasive surgery, motion-preserving surgery, and osteobiologics have all invoked a fair bit of sustained controversy despite a fairly impressive body of high-grade research in the arenas of disk replacements and recombinant bone morphogenic protein. The thought that there might some form of inadvertent observer bias involving the surgeons participating in sentinel prospective RCTs is unsettling as this literature not only is used to provide device approvals through regulatory agencies but also heavily influences the clinical decision making of practitioners as well as patients. A thought-provoking study by Singh et al compared reoperation rates for adjacent segment degeneration in cervical spine fusions outside of a prospective RCT comparing cervical artificial disk replacement (ADR) surgery and fusions.¹ The authors found a statistically lower rate of reoperations for adjacent segment disease in patients having cervical fusion not involved in the formal prospective RCT trials, raising the open-ended question of the potential reasons for such a difference.² An interesting perspective on conflict of interest was raised by Matsen et al, who found a significant correlation of academically highly productive orthopedic surgeons and the number of their device industry disclosures.² Another aspect of selection bias affecting what is published in our academic literature was raised by Emerson et al, who showed a significantly higher acceptance rate and less-critical assessments by reviewers of two leading orthopedic journals for sham articles with positive results compared with articles with negative or indifferent findings.³ Finally, we also should be aware that our patients in this information age are potential sources of bias. Of course, there is no such thing as perfect research, but being aware of our inherent human biases and learning to deal with them more openly in our representation and interpretation of research as well as improving our efforts at formulating research questions and methodology will go a long way to further improve the quality of our research.

The second question pertains to the adequacy of AE reporting using the lumbar ADR example compared with fusions. The mainstays of the prospective RCTs referenced in the Hiratzka article were very thoroughly performed and hitherto unparalleled in their scope and depth (as well as expense). The length of follow-up of the ADR FDA trials is now beyond 5 years and exceeds most previously published prospective RCTs in the field of spine surgery. The concept of trying to catch all complications—related or not—through the universal summary of SSED as mandated by the FDA has been criticized for the sheer data volume of chaff it invariably accumulates, but it represents an attempt at removing observer judgment by being inclusive in the selection of what constitutes a meaningful complication. The subject of underreporting AEs/complications/occurrences remains a major shortcoming of our literature. Hiratzka et al rightfully reference the award-winning study by Street et al, who identified an 87% perioperative complication rate in 942 patients with major spine surgery by using a novel surveillance system.⁴ All bias aside, this study clearly highlights the general inherent inability of our current day health care systems to identify and track complications. Underreporting is a logical consequence of such systems limitations. The SAVES model suggested by Street et al is definitely a worthwhile undertaking to consider. Hiratzka et al also present very useful checklists on reporting of complications for studies, which add dimensions of timing and grading of severity as desirable feature for future clinical studies. A logical further consequence of the discovery of inherent AEs underreporting is to insist that future researchers make source data along with an AE chart accessible for public secondary review. This enhanced data transparency would allow for independent determination if investigators inadequately collected or just did not publish complications.