External Validity of the HOPE-Trial

Abstract

Background:

Randomized controlled trials (RCTs) are the most reliable way of evaluating the effect of new treatments by comparing them with previously accepted treatment regimens. The results obtained from an RCT are extrapolated from the study environment to the general health care system. The ability to do so is called external validity. We sought to evaluate the external validity of an RCT comparing the results of total hip arthroplasty with those of hemiarthroplasty for the treatment of displaced femoral neck fractures in patients ≥80 years of age.

Methods:

This prospective, single-center cohort study included 183 patients ≥80 years of age who had a displaced femoral neck fracture. All patients were screened according to the inclusion and exclusion criteria for an RCT comparing total hip arthroplasty and hemiarthroplasty. The population for this study consisted of patients who gave their informed consent and were randomized into the RCT (consenting group, 120 patients) as well as those who declined to give their consent to participate (non-consenting group, 63 patients). The outcome measurements were mortality, complications, and patient-reported outcome measures. Follow-up was carried out postoperatively with use of a mailed survey that included patient-reported outcome questionnaires.

Results:

We found a statistically significant and clinically relevant difference between the groups, with the non-consenting group having a higher risk of death compared with the consenting group. (hazard ratio, 4.6; 95% confidence interval, 1.9 to 11.1). No differences were found between the groups in terms of patient-reported outcome measures or surgical complications.

Conclusions:

This cohort study indicates a higher mortality rate but comparable hip function and quality of life among eligible non-consenters as compared with eligible consenters when evaluating the external validity of an RCT in patients ≥80 years of age with femoral neck fracture.

Level of Evidence:

Therapeutic Level II. See Instructions for Authors for a complete description of levels of evidence.

There is often a question as to whether the results from a randomized controlled trial (RCT) can be extrapolated from the study environment to the general health care setting¹. Active participation of patients in an RCT is not always easily achieved, and some refusal to participate and dropout is inevitable.

The authors of one study on fast-track surgery for patients undergoing total hip arthroplasty reported on the external validity of their RCT², but generally this issue has not been addressed in the orthopaedic literature. In the present study, we evaluated the external validity of an RCT comparing the results of total hip arthroplasty with those of hemiarthroplasty for the treatment of displaced femoral neck fractures in patients ≥80 years of age. The aim was to compare mortality, complications, and patient-reported outcome measures between those who were included in an RCT and those who fulfilled the inclusion criteria for the RCT but declined to participate.

Materials and Methods

Study Design and Setting

This observational prospective cohort study was performed between 2009 and 2015 at Danderyd Hospital in Stockholm, Sweden. Danderyd Hospital is a teaching hospital affiliated with the Karolinska Institute. It is 1 of the 4 major emergency hospitals in Stockholm, providing medical care for a population of approximately 500,000 inhabitants. The guidelines of the STROBE (STRrengthening the Reporting of OBbservational studies in Epidemiology) statement were followed³.

Participants

All patients ≥80 years of age who were admitted to Danderyd Hospital with a displaced femoral neck fracture during the inclusion period were screened for participation in the HOPE clinical trial (Hemiarthroplasty Compared to Total Hip Arthroplasty for Displaced Femoral Neck Fractures in the Elderly-elderly [HOPE], clinicaltrials.gov NCT02246335)⁴_.

The inclusion criteria were an acute displaced femoral neck fracture (Garden 3 and 4), an age of ≥80 years, the ability to walk independently with or without walking aids, and intact cognitive function as indicated by a Short Portable Mental Status Questionnaire (SPMSQ) score of 8 to 10 points⁵. Patients with osteoarthritis or rheumatoid arthritis in the fractured hip, those with pathological fractures, and those who were non-walkers or who were deemed unsuitable for participation in the study for any reason were excluded (Table I).

TABLE I.

Inclusion and Exclusion Criteria for the RCT Comparing Total Hip Arthroplasty and Hemiarthroplasty

Inclusion criteria

Age ≥80 years

Acute (<36 hr) displaced femoral neck fracture

Independent walker

Absence of cognitive impairment (SPMSQ score, 8-10 points)

Exclusion criteria for RCT

Osteoarthritis or rheumatoid arthritis in fractured hip

Pathological fracture

Non-walker

Substance abuse

Other reason deemed unsuitable for participation (i.e., medical condition)

The primary assessment was performed by a research nurse who established that the patient fulfilled the inclusion criteria and identified any comorbidities. Patients who gave their informed consent to be randomized were included in the HOPE trial (the consenting group), and those who declined to participate in the RCT were, after a separate informed consent was obtained, included in the non-consenters group.

The HOPE trial was analyzed with the intention-to-treat principle, thus including all randomized patients in the groups to which they were randomly assigned, regardless of the treatment that they actually received. This principle is used to avoid overoptimistic estimates of the efficacy of an intervention resulting from the removal of patients whose intervention deviated from the treatment protocol⁶.

Follow-up and Data Collection

A research nurse and the first author (S.M.) interviewed the patients and obtained baseline data. The American Society of Anesthesiologists (ASA) score was used as a proxy for comorbidity.

The functional outcome scores were self-reported by the patients. Follow-up was carried out 1 year after surgery in the form of a visit to the outpatient clinic and/or a mailed survey consisting of the patient-reported outcome questionnaires. Patients in both groups had a follow-up visit at the orthopaedic clinic at 3 months. Mortality and reoperations (revision of prosthetic components, open reduction and internal fixation due to periprosthetic fracture, hip replacement as a secondary or tertiary procedure, excision arthroplasty, closed reduction, and surgical debridement) up to 2 years after surgery were identified with use of the unique Swedish civic identity number. We examined the digital medical records at Danderyd Hospital and conducted searches in the Swedish Hip Arthroplasty Register (SHAR), the Swedish Patient Registry, and the Swedish Death Register. All hip-related complications in the study patients were treated and registered at our department, and no other reoperations or complications were found to have occurred at other hospitals in Sweden. The study data were collected and managed in a digital case report form with use of Research Electronic Data Capture (REDCap) tools provided by the Karolinska Institute. REDCap is a secure, web-based application designed to support data capture for research studies⁷.

Patient-Reported Outcome Measures

Hip function was rated with use of the Harris hip score⁸ and was assessed as the primary outcome. The disease-specific score has a maximum of 100 points (no disability) and covers the domains of pain, function, and range of motion. Health-related quality of life was assessed with use of the EuroQol-5 Dimensions (EQ-5D)⁹, a generic instrument with 5 dimensions (mobility, personal hygiene, usual activities, pain, and anxiety/depression) and 3 possible responses for each item (no problems, some problems, severe problems). The EQ-5D visual analog scale, ranging from 0 (worst possible health status) to 100 (best possible health status), was used to register the patients’ perceived health status.

Surgery

Patients who were included in the RCT were randomized to treatment with either total hip arthroplasty or hemiarthroplasty. Two types of femoral stems were used: (1) a cemented CPT stem (Zimmer) with a modular unipolar head (VerSys Endo; Zimmer) or (2) a cemented Lubinus SPII stem and a unipolar head (Waldemar Link). The CPT stem is a collarless, polished femoral stem made from cobalt-chromium alloy with a 12/14 head taper. The Lubinus SPII stem is a cemented, anatomical, collared femoral stem made from matte cobalt-chromium alloy. For total hip arthroplasty, a modular 32-mm cobalt-chromium femoral head was used together with a cemented highly cross-linked polyethylene acetabular component (Marathon Cemented Cup; DePuy). The direct lateral surgical approach was used.

Patients in the non-consenters group were managed in accordance with the routine practice at our department. Patients ≥80 years of age are normally managed with hemiarthroplasty, but the final decision of whether to perform a hemiarthroplasty or total hip arthroplasty is ultimately determined by the patient’s level of activity and the surgeon’s preference. Patients who have a deteriorating medical condition or an infection between admission and surgery and thus are deemed unsuitable for arthroplasty are managed with closed reduction and internal fixation with use of cannulated screws. All procedures were performed on the day of or the day after admission. All patients received antibiotic and anticoagulant prophylaxis (3 doses of 2-g cloxacillin and low-molecular-weight heparin for 30 days postoperatively). All patients were mobilized to full weight-bearing on the first postoperative day under the supervision of a physiotherapist. Patients who underwent surgery with the direct lateral approach had no restrictions imposed on their mobilization.

Statistical Methods

The Student t test was used for continuous normally distributed data, and the chi-square test was used for ordinal data. The Mann-Whitney U test was used for data that were not normally distributed. All tests were 2-sided. For the Harris hip score outcome variables, we used a generalized linear regression model to detect a difference between the 2 groups. We used the multivariate Cox proportional hazards method to evaluate factors associated with mortality and reoperation. A multivariate model that was adjusted for age, sex, type of surgery, and ASA classification (1 or 2 versus 3 or 4) were included in the analysis as covariates. The associations are presented as hazard ratios (HRs) with 95% confidence intervals (CIs). A p value of <0.05 was considered significant. The statistical analyses were performed with use of SPSS (version 22.0; IBM).

Ethics and Registration

The study was conducted in accordance with the Declaration of Helsinki and was approved by the Ethics Committee of the Karolinska Institute. The trial was initiated, designed, and performed as an academic investigation and is registered at ClinicalTrials.gov (number NCT02362971).

Results

Study Subjects and Descriptive Data

Between October 2009 and June 2016, 963 patients ≥80 years of age with a displaced femoral neck fracture were screened for participation in the cohort study at the Orthopedic Department at Danderyd Hospital, Stockholm. Of these, a total of 183 patients (78% female; median age, 85 years [range, 80 to 97 years]) fulfilled the inclusion criteria and were included in the cohort study of external validity, with 120 patients in the consenting group and 63 patients in the non-consenting group (Table II, Fig. 1). The group characteristics at baseline are presented in Table II. The mean duration of follow-up was 22 months (range, 0 to 24 months). All patients were followed for 2 years or until death.

Fig. 1.

Patient flowchart. HA = hemiarthroplasty, and THA = total hip arthroplasty.

TABLE II.

Study Population Characteristics

	Consenters (N = 120)	Non-Consenters (N = 63)
Age* (yr)	85 (80-94)	87 (80-97)
Sex (no. of patients)
Male	29 (24%)	12 (19%)
Female	91 (76%)	51 (81%)
Involved side (no. of patients)
Right	52 (43%)	25 (40%)
Left	68 (57%)	38 (60%)
ASA score (no. of patients)
1-2	52 (43%)	20 (32%)
3-4	68 (57%)	43 (68%)
Procedure (no. of patients)
Total hip arthroplasty	56 (47%)	5 (8%)
Hemiarthroplasty	62 (52%)	56 (89%)
Internal fixation†	2 (2%)	2 (3%)

^*The values are given as the median, with the range in parentheses.

^†Two patients in each group were managed with internal fixation because of a deteriorating medical condition or the development of a severe infection between admission and surgery.

Six patients in the consenting group did not receive their allocated treatment because of a decline in medical status between randomization and surgical treatment. Two of these patients were managed with closed reduction and internal fixation with use of cannulated screws.

In the non-consenting group, the main treatment (56 patients) was hemiarthroplasty, according to the practice at our department. The other patients were managed with total hip arthroplasty (5 patients) or closed reduction and internal fixation using cannulated screws (2 patients) because of a decline in medical status between screening and surgical treatment (Fig. 1).

Outcomes

Patient-Reported Outcome

One hundred and one patients in the consenting group and 27 patients in the non-consenting group completed the follow-up period with patient-reported outcome measures. We did not find any significant differences between the groups in terms of the primary outcome variable (the Harris hip score), even after adjusting for confounders (Table III). There were no significant differences between the groups in terms of the EQ-5D score (p = 0.96).

TABLE III.

Harris Hip Score^*

		Harris Hip Score
Variable	EM	Coef.†	95% CI	P Value
Group
Consenter	78	—	—
Non-consenter	81	–3	–10 to 4	0.4
Surgical treatment
Internal fixation	94	—	—
Total hip arthroplasty	73	–21	–65 to 22	0.3
Hemiarthroplasty	71	–23	–67 to 20	0.3
Age	86	0.02	–0.8 to 0.8	1.0
Sex
Male	82	—	—
Female	77	−5	–12 to 3	0.2
ASA score
3-4	77	—	—
1-2	82	5	–1 to 11	0.13

^*Generalized linear model regression, including the adjustment in the Harris hip score, with 95% Wald confidence interval (CI), for each variable in the multivariate analysis. EM = estimated marginal mean for the covariate.

^†Coef. = the difference in the marginal means between groups.

Mortality

At 1 year, the overall mortality rate was 9%, with non-consenters having a higher risk of death than consenters (19% compared with 3%). In the regression model, after adjusting for differences in demographic characteristics (age, sex, ASA category) and type of surgical treatment, the risk of death remained significantly higher in the non-consenting group (HR, 5.3; 95% CI, 1.5 to 18.6).

At 2 years, the overall mortality rate was 16%, with non-consenters having a significantly higher risk of death during the study period compared with consenters (33% compared with 7%) (HR, 4.6; 95% CI, 1.9 to 11.1) (Table IV, Fig. 2). Other factors associated with an increased mortality risk included advanced age, male sex, and treatment with internal fixation (Table IV).

Fig. 2.

Cox proportional hazards regression for survival in the different groups, adjusted for age, sex, ASA category, and type of surgery.

TABLE IV.

Two-Year Mortality During the Study Period^*

Variable	No. of Patients (N = 183)	Mortality Rate	Hazard Ratio (95% CI)	P Value
Group
Consenter	120 (66%)	7% (8 of 120)	—
Non-consenter	63 (34%)	33% (21 of 63)	4.6 (1.9-11.1)	0.001
Age (80-97 yr)	183	NA	1.1 (1.0-1.2)	0.02
Sex
Female	142 (78%)	13% (18 of 142)	—
Male	41 (22%)	27% (11 of 41)	3.1 (1.4-6.8)	0.01
ASA score (no. of patients)
1-2	72 (39%)	10% (7 of 72)	—
3-4	111 (61%)	20% (22 of 111)	1.7 (0.7-4.1)	0.3
Surgical treatment
Total hip arthroplasty	61 (33%)	3% (2 of 61)	—
Hemiarthroplasty	118 (65%)	21% (25 of 118)	2.9 (0.6-13.3)	0.2
Internal fixation	4 (2%)	50% (2 of 4)	10.3 (1.3-79.3)	0.03

^*Hazard ratios showing the effects of the group and other variables in the multivariate analysis, according to Cox proportional hazard modeling.

Hip Complications and Reoperations

Twelve patients (7%) required at least 1 reoperation; this number includes those who required closed reduction because of dislocation. Complications included periprosthetic femoral fracture (5 patients), dislocation (3 patients), postoperative infection (2 patients), and mechanical failure of internal fixation (2 patients). The rate of reoperation was higher in the non-consenting group compared with the consenting group (10% vs. 5%), but this difference did not reach significance when adjusting for confounders (adjusted HR, 1.9 [95% CI, 0.5 to 7.5]; p = 0.3).

Outcomes in the RCT Group (HOPE-Trial)

We found no clinically relevant or statistically significant differences between patients managed with total hip arthroplasty and those managed with hemiarthroplasty in terms of patient-reported outcomes (Harris hip score, EQ-5D scores) or the prevalence of hip-related complications¹⁰.

Discussion

In the present prospective cohort study of patients with a femoral neck fracture, we found a higher mortality rate but comparable hip function when eligible non-consenters were compared with patients who were included in an RCT. Our findings emphasize the notion that patients included in RCTs might belong to a selected patient population.

The treatment of femoral neck fractures in elderly patients has been under debate for decades. Most surgeons seem to prefer hemiarthroplasty for elderly patients with low functional demands in the absence of arthritic changes in the hip. For healthy, cognitively intact, and active patients, studies have favored total hip arthroplasty^11-13. Few studies have included large numbers of patients ≥80 years of age¹⁴, raising the question of whether active and lucid octogenarians would benefit from surgery with total hip arthroplasty rather than hemiarthroplasty. In the consenting group in the present study, there were no differences in functional outcome between patients managed with total hip arthroplasty and those managed with hemiarthroplasty. The short-term follow-up period might not have been sufficient time for the development of acetabular erosion in patients managed with hemiarthroplasty, especially in those ≥80 years of age, who may have limited activity.

The discrepancy between RCTs and nonrandomized studies was previously addressed in the meta-analysis by Bhandari et al.¹⁵. Those authors concluded that nonrandomized studies underestimated the relative benefit of arthroplasty by indicating higher mortality and revision rates than their randomized counterparts. However, this conclusion could be the result of selection bias caused by the inclusion of healthier patients in the RCT. Petersen et al. evaluated the differences in baseline data in an RCT of patients managed with total hip arthroplasty for the treatment of osteoarthritis². The authors reported significant differences in baseline data among those who gave consent and those who did not. Studies from other fields of medicine also have indicated that non-consenters have been associated with both poorer health status and higher mortality than those included in RCTs^16,17. The reporting of data related to external validity in reports from RCTs varies, and thus it is incumbent on the reader to consider the question of generalizability^18-20.

Low enrollment rates can compromise the external validity of RCTs. In the present study, 1 of 3 patients declined to give their informed consent, and our results indicate that the patients who declined had an increased mortality rate. These results were adjusted for age, sex, type of surgery, and ASA classification, suggesting that this subgroup of non-consenting patients was more fragile than the group of patients who were included in the RCT. A poorly explained rationale, particularly in settings involving elderly patients in a stressful, traumatic, and acute situation, increases the risk of a lack of patient understanding of the study aims, which may increase the risk of refusal to participate²¹.

Our results and those of previous studies indicate that study participants are generally healthier than non-participants^2,17,20. Less restrictive inclusion criteria for RCTs may increase the generalizability of the results to the target population for a specific disease. The issue of non-consenters should be addressed in future studies to more easily facilitate the reader’s interpretation of the external validity of the study results.

Some of these issues can be addressed by establishing a link between a prospective randomized trial and a nationwide clinical registry. Conducting registry-based randomized controlled trials (rRCTs), which include a randomization module in a large, all-inclusive clinical registry with unselected consecutive enrollment, can combine some of the most important features of a prospective randomized trial with the inclusiveness and efficiency of a large-scale all-comers clinical registry. The consecutive enrollment combined with patient identification and automated linked registry-based follow-up allows for a cost-effective model with analysis of those who are lost to follow-up. The external validity is analyzed by allowing the results for excluded patients and non-consenters to be analyzed²². Reasons for not participating in RCTs are multifactorial and include the unwillingness to be randomized or to undergo additional follow-up. Our results reinforce the need for researchers in the field of orthopaedics to improve the quality of the reporting of RCTs by providing additional information about the recruitment process, specifying the reasons for the exclusion of patients, and reporting on the results of non-consenters. This should be done to better establish the study’s generalizability, to avoid a lack of reproducibility, and to prevent selection bias.

We found no significant difference between the groups in terms of functional outcomes. The loss of a large number of patients in the non-consenter cohort as a consequence of the higher mortality rate may have limited our ability to detect a meaningful difference.

The sample size could not be altered because of the predetermined sample size for the RCT and the observational design of the present study. The sample size of the non-consenters was further limited because of the high mortality rate. To detect a difference between the groups in terms of the Harris hip score would have required a much larger sample size, which would not have been feasible for the eventual completion of the RCT. The limitations are counterbalanced by the strength of this study, which, to our knowledge, is the first study to evaluate the outcome of non-consenters in an orthopaedic RCT including patients with hip fractures.

Conclusions

In this cohort study evaluating the external validity of an RCT of octogenarians with femoral neck fractures, there was a higher mortality rate but comparable hip function among eligible non-consenters compared with eligible consenters.

Data Sharing

A data-sharing statement is provided with the online version of the article (http://links.lww.com/JBJSOA/A99).