Arthroplasty studies with greater than 1000 participants: analysis of follow-up methods

Abstract

Background

The use of patient-reported outcome measures (PROMs) has become a mainstay of orthopedic joint arthroplasty research. Large studies with >1000 participants are vital to orthopedic research, as they allow for comprehensive multivariable analysis. Achieving high follow-up rates minimizes potential response bias. Maintaining adequate follow-up rates becomes more challenging as sample size increases. We aimed to systematically review the present literature to determine the follow-up rates of large cohorts/registries of total joint arthroplasty patients and to identify factors associated with successful collection of PROMs.

Methods

A comprehensive literature search of PubMed, EMBASE, and the Cochrane Central Register of Controlled Trials was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Inclusion criteria were: ≥1000 participants, ≥6 months of postoperative follow-up, and use of validated PROMs postoperatively.

Results

Of 720 abstracts screened, 21 studies met inclusion criteria. Only 2 studies reported achieving a PROM follow-up rate ≥80%, but neither collected PROMs preoperatively. The median rate of follow-up was 70%, and the median number of patients was 2970. Only 38% (8 of 21) of studies collected baseline PROMs prior to surgery.

Conclusions

Very few studies in the present literature have collected validated PROMs on ≥1000 patients with ≥80% follow-up; these parameters are conducive to comprehensive multivariable analysis, while maintaining study validity and avoiding follow-up bias. Federal funding and a central coordinating site may be helpful in achieving follow-up in studies of this magnitude.

Level of Evidence

Level III, systematic review of studies with Level of Evidence I-III.

Introduction and background

The American healthcare system is in the midst of a paradigm shift as it transitions from a volume-based system to a value-based system [1], [2]. Value in healthcare is defined as quality, which is synonymous with outcome, relative to cost [3]. As such, validated patient-reported outcome measures (PROMs) have become a key component to clinical research and, in the future, may very well be used to determine reimbursements for orthopedic procedures such as total joint arthroplasty. Therefore, the collection and documentation of these clinically relevant outcomes (with the use of validated PROMs) prior to surgery, as well as at defined postoperative intervals, is essential to measure the effect a surgical procedure has on patients’ overall and joint-specific health.

A number of registries and cohort studies currently are collecting PROMs [4]; however, collecting PROMs at clinically relevant time points from an adequate proportion of patients is challenging. There is no agreed-upon threshold that defines adequate follow-up rate, but current literature favors a minimum of 60%-70% [2].

In contrast to primary care, in which patients are required to return to the office for long-term medical treatment (ie, refill of prescriptions, annual physical, etc.), there is no guarantee that patients who are doing well after successful joint arthroplasty surgery will return to the office. Furthermore, return visits for asymptomatic patients doing well after surgery can contribute to increased healthcare costs without clear benefit [5]. Additionally, patients doing poorly after an orthopedic procedure may choose to see another provider, and their outcome and/or complication may be missed. Thus, the use of PROMs, which are self-administered and can be completed by patients from the comfort of their own home, provides an important avenue by which outcomes can be assessed. However, for the aforementioned reasons, collection of postoperative outcomes in orthopedics is challenging, and loss of patients to follow-up can introduce significant response bias that then impacts the final conclusions of a study [6], [7]. To complicate things further, large sample sizes, typically in excess of 1000 participants, are needed to adequately control for the myriad of patient- and disease-specific factors that may significantly impact the outcome of interest [8], such as demographic characteristics, socioeconomic status, disease severities, treatments rendered, and implants placed.

The purpose of the present study is to systematically review the current literature to determine the enrollment and follow-up rates (percentage of patient responders) in published studies utilizing PROMs following knee or hip arthroplasty surgery with at least 1000 patients (ie, large enough for comprehensive multivariable analysis). In addition, we sought to identify relevant characteristics, such as funding source and follow-up methodology, for studies that successfully achieved high PROM follow-up rates (≥80% as defined by previous studies [9]).

Material and methods

Literature search

The systematic review was initiated with a comprehensive search of articles published prior to August 21, 2016, in 3 electronic databases: PubMed, EMBASE, and the Cochrane Central Register of Controlled Trials according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. The search terms and methodology utilized are shown in Appendix A. The electronic search was supplemented with a manual search reviewing all bibliographies of retrieved review articles for potentially relevant citations. Articles found in the searches were then evaluated for inclusion based on criteria defined below. The initial literature search also included surgical procedures relevant to sports medicine (eg, anterior cruciate ligament reconstruction, knee/hip/shoulder arthroscopy, etc.). These were ultimately removed from this publication as there were too few studies that fulfilled the inclusion criteria.

Search inclusion criteria

A study was eligible for inclusion in the systematic review if it (1) had a study population of at least 1000 patients who had (2) undergone knee or hip arthroplasty (3) with at least 6 months of follow-up postoperatively [10] and (4) the study had collected PROMs after surgery. Studies that collected PROMs prior to surgery but not after surgery were included in supplemental data (Appendix B).

Data abstraction

Data abstracted from the studies that met inclusion criteria included the following: (1) how many patients underwent the specified procedure, (2) how many of those patients completed PROMs preoperatively (enrollment), (3) how many patients completed PROMs after surgery (follow-up), (4) list of specific PROMs collected, (5) primary funding source, (6) study design, and (7) the time interval between surgery and final PROMs collection. Percentage enrollment and follow-up rate of PROMs was calculated based on initial number of patients included in study with subsequent number of patients completing preoperative and/or postoperative PROMs, respectively. In addition, basic demographic data of patient populations in studies such as average age and male-to-female ratio also were abstracted as measures of individual study characteristics. Level of evidence was assigned according to journal designation or, if not reported, then according to Elsevier criteria (Table 1). All data were extracted by 2 authors (M.B.T., R.W.). In situations where there was disagreement between the 2 authors, a senior author (K.P.S.) was consulted to determine the final outcome.

Table 1.

Total knee arthroplasty studies meeting inclusion criteria.

Author (y)	Study design	Registry or dataset (time period)	LOEa	Mean age (y)	% Female	Initial N	Preop PROM collected	Preop enrollment % (N)b	Postop follow-up % (N)c	Length of follow-up (y)	Contact method	Primary funding source	PROM utilized
Jiang et al (2017) [11]	Prospective Cohort	Knee Arthroplasty Trial (multicenter RCT) (1999-2003), UK	II	71	56.4%	2252	Yes	95% (2131)	76% (1707)d	1	☒Mail ☐Web-based ☐Phone	NIH	OKS
Williams et al (2013) [12]	Retrospective Cohort/Registry Review	NHS Elective Orthopedic Center (2006-2008)	III	71.4	60.8%	3002	Yes	82% (2456)	71% (2121) 34% (1008)	0.5 2	NR	NIH and Arthritis Research UK	OKS, EQ-5D
Hawker et al (1998) [13]	Retrospective patient-based survey (sent 1992)	Medicare Provider Analysis and Review Files (1985-1989)	III	73	71%	1750	No	(No preop)	68% (1193)e	2-7	☒Mail ☐Web-based ☐Phone	Agency for Health Care Policy and Research	SF-36, WOMAC
Singh and Lewallen (2013) [14]	Retrospective Cohort/Registry Review	Mayo Clinic Total Joint Registry (1993-2005)	III	68	56%	10,957 7404	NR	NR	65% (7139) 57% (4234)	2 5	☒Mail ☐Web-based ☒Phone ☒Other: clinic visits	Mayo Clinic Orthopedic Surgery Department	Mayo Knee Questionnaire
Singh and Lewallen (2014) [15]	Retrospective Cohort/Registry Review	Mayo Clinic Total Joint Registry (1993-2005)	III	69	40%	2695 1842	NR	NR	57% (1533) 48% (881)	2 5	☒Mail ☐Web-based ☒Phone ☒Other: clinic visits	Mayo Clinic Orthopedic Surgery Department	Mayo Knee Questionnaire
Judge et al (2012) [16]	Retrospective Cohort	Elective Orthopedic Center Database UK (2005-2008)	III	71.3	62.1%	3608	NR	NR	55% (1991)f	0.5	NR	NHS Institute for Innovation and Improvement and University of Oxford	OKS, EQ-5D, VAS

EQ-5D, EuroQoL-5D; NHS, National Health Service; NIH, National Institutes of Health; NR, not reported; OKS, Oxford Knee Score; RCT, randomized controlled trial; SF-36, Thirty-Six Item Short Form Health Survey; VAS, visual analog scale; WOMAC, Western Ontario and McMaster Universities Osteoarthritis Index.

^aDesignated according to Elsevier publishing definitions (https://www.elsevier.com/__data/promis_misc/623124los.pdf), unless marked with *, which is journal article designated LOE.

^b% Calculated from preop completion N divided by initial N.

^c% Calculated from postop completion N divided by initial N.

^dN calculated based on post-op % completion given in article.

^e% Calculated does not adjust for death/incapacitated.

^fCompleted both preop and postop.

All the outcomes in this analysis were continuous; their means and measures of dispersion were extracted from each study. Two independent reviewers extracted the data.

Results

Of the 720 abstracts identified by the literature search, 669 did not meet the inclusion criteria upon screening of abstracts and titles. The remaining 51 articles were retrieved for full text review; of these, 2 were review articles and an additional 2 were ongoing randomized controlled trials (Fig. 1). Furthermore, 22 studies were excluded because of incomplete data, or because multiple papers reported data from the same cohort or dataset and only the paper with the largest enrollment number was included (Appendix B). Due to the low number of sports medicine studies that met inclusion criteria (n = 4) compared to studies pertinent to arthroplasty, the sports medicine studies were ultimately removed from the current systematic review.

Figure 1.

Literature search flowchart of article identification process.

Therefore, 21 articles remained for data abstraction by 2 independent reviewers (Table 1): 5 for knee arthroplasty, 14 for hip arthroplasty, and 2 for knee or hip arthroplasty. Only 38% (8 of 21) of studies reported on preoperative PROMs prior to surgery. Meta-analysis could not be conducted due to significant heterogeneity between studies.

PROM enrollment and follow-up rates

Overall analysis revealed that the highest reported postoperative PROM follow-up rate was 86% at 1-3 years in a study population of 1476 patients undergoing total hip arthroplasty [17]. The lowest PROM follow-up rate was 11% at 1 year in a study population of 6861 patients undergoing knee or hip arthroplasty [18]. The median postoperative PROM follow-up rate was 70%, and the median number of patients was 2970. Overall, only 8 studies reported preoperative PROMs [11], [12], [18], [19], [20], [21], [22], [23], 4 studies did not collect [13], [17], [24], [25], and 9 studies did not report whether preoperative PROMs were collected [14], [15], [16], [26], [27], [28], [29], [30], [31].

Two studies, one by Amlie et al and the other by Paulsen et al, reported ≥80% PROM follow-up rates, with 1476 and 5747 patients, respectively [17], [24]. Neither collected preoperative PROMs at the time of enrollment. Funding sources were reported for both studies. Amlie et al contacted participants via mail, while Paulsen et al did not report a contact method [17], [24]. Both studies were multicenter cross-sectional (Level of Evidence [LOE] III) studies.

Nine studies reported PROM follow-up rates between 70% and 79%, with a median of 2391 patients (range 1233-97,487) [11], [12], [19], [20], [21], [22], [25], [26], [31], of which 6 studies reported collection of preoperative PROMs [11], [12], [19], [20], [21], [22]. Of the 3 studies without preoperative PROM data, 2 did not report whether such data were collected [26], [31], and 1 did not collect [25]. Funding was reported by all 9 studies, 2 of which reported no funding. Patients were contacted via mail in 8 of 9 studies, with 1 study failing to report a contact method [12]. One study used both mail and clinic visits [20]. There were 5 multicenter studies. Five studies were prospective (LOE II/III) and 4 were retrospective (LOE III) (Table 1, Table 2, Table 3).

Table 2.

THA studies meeting inclusion criteria.

Author (y)	Study design	Registry or dataset (time period)	LOEa	Mean age (y)	% Female	Initial N	Preop PROM collected	Preop enrollment % (N)b	Postop follow-up % (N)c	Length of follow-up (y)	Contact method	Primary funding source	PROM utilized
Amlie et al (2014) [17]	Cross-sectional study/Registry Review	Norwegian Arthroplasty Register (2008-2010)	III	66	64%-69%	1476	No	(No preop)	86% (1273)	1-3	NR	Norwegian Ministry of Health and Social Affairs15	VAS, HOOS, EQ-5D
Paulsen et al (2012) [24]	Cross-sectional study/Registry Review	Danish Hip Arthroplasty Registry (THA in the past 1-2, 5-6, and 10-11 y)	III	71	57%	5747	No	(No preop)	83% (4784)	(1-2, 5-6, or 10-11 postop)	☒Mail ☐Web-based ☐Phone	Local regions	EQ-5D/SF-12, HOOS/OHS
Rolfson et al (2011) [19]	Prospective observational study/Registry Review	Swedish Hip Arthroplasty Register (2008)	II	68	58%	12,300	Yes	NR	79% (9727)d	1	☒Mail ☐Web-based ☐Phone	Swedish Association of Local Authorities and Regions and the National Board of Health and Welfare	Charnley’s Functional Categories, VAS, EQ-5D
Haase et al (2016) [20]	Retrospective Cohort/Registry Review	Dresden Registry (2006-2011)	III	60.8	52.8%	2970	Yes	84% (2496)	79% (2343)e	0.5	☒Mail ☐Web-based ☐Phone ☒Other: clinic visits	None	WOMAC, EQ-5D, UCLA Activity Score, Harris Hip Score
Quintana et al (2012) [26]	Prospective Cohort (1999-2000; 2003-2004)	Single center	II	NR	NR	1233	NR	NR	73% (897)	0.5	☒Mail ☐Web-based ☐Phone	Fondo de Investigación Santaria and Department of Health of the Basque Government	WOMAC
Gould et al (2012) [25]	Retrospective patient-based survey; sent 2001	1 Elective UK Orthopedic Center (1993-1996)	III	(Median 78)	62%	1727	No	(No preop)	72% (1240)	5-7	☒Mail ☐Web-based ☐Phone	North Bristol Trust Research and Innovation Small Grant Scheme	OHS
Lim et al (2015) [21]	Retrospective Cohort/Registry Review	NHS PROMs and Hospital Episode Statistics Hip Data Set (2009-2011)	III	67.8	59%	97,487	Yes	96% (93,253)	71% (69,361)	0.5	☒Mail ☐Web-based ☐Phone	Oxford University and Oxford University Hospital Trust	OHS
Paulsen et al (2014) [22]	Prospective Cohort	16 Departments in Denmark (2010-2011)	II	(Median 68)	54%	1837	Yes	73% (1335)	70% (1288)	1	☒Mail ☐Web-based ☐Phone	Local regions	HOOS, EQ-5D
Smith et al (2012) [27]	Retrospective Cohort	UK Center Experience (2004-2008)	III	68f	63%	1315	NR	NR	69% (911)	1-3	☒Mail ☐Web-based ☐Phone	None	WOMAC, Self-Administered Patient Satisfaction Scale for Primary Hip and Knee Arthroplasty
Judge et al (2010) [28]	Retrospective Cohort/Registry Review	European Collaborative Database of Cost and Practice Patterns of Total Hip Replacement at 20 Orthopedic Centers	III	NR	56%	1327	NR	NR	68% (908)	1	☒Mail ☐Web-based ☐Phone	The European Collaborative Database of Cost and Practice Patterns of Total Hip Replacement	WOMAC
Pennington et al (2013) [23]	Retrospective Cohort/Registry Review	NHS England (2008-2011)	III	67.7-72.6g	35.4%-44%	226,805	Yes	78.8% (178,723)	67% (152,808)h	0.5	NR	English Department of Health	OHS, EQ-5D
King et al (2016) [29]	Retrospective Cohort/Registry Review	NJR England and Wales (2003-2012)	III	(Median 53-69)i	59-60%	216,265	NR	NR	57% (124,111)	0.5	NR	National Joint Registry	OHS, EQ-5D
Greene et al (2015) [30]	Retrospective Cohort/Registry Review	Swedish Hip Arthroplasty Registry (2002-2007)	III*	68	57%	120,891	NR	NR	17% (21,103)	1	NR	Swedish Hip Arthroplasty Register	EQ-5D, VAS

EQ-5D, EuroQoL-5D; HOOS, Hip disability and Osteoarthritis Outcome Score; NHS, National Health Service; NJR, National Joint Registry; NR, not reported; OHS, Oxford Hip Score; SF-12, Twelve Item Short Form Health Survey; THA, total hip arthroplasty; UCLA, University of California at Los Angeles; VAS, visual analog scale; WOMAC, Western Ontario and McMaster Universities Osteoarthritis Index.

^aDesignated according to Elsevier publishing definitions (https://www.elsevier.com/__data/promis_misc/623124los.pdf), unless marked with *, which is journal article designated LOE.

^b% Calculated from preop completion N divided by initial N.

^c% Calculated from postop completion N divided by initial N.

^dCompleted both preop and postop.

^eFor WOMAC only.

^fDemographic data for Responders Cohort.

^gStratified data into groups.

^hN calculated based on initial N, preop completion %, and postop % completion given in article.

ⁱStratified data into groups.

Table 3.

Mixed THA and TKA studies meeting inclusion criteria.

Author (y)	Study design	Registry or dataset (time period)	LOEa	Mean age (y)	% Female	Initial N	Preop PROM collected	Preop enrollment % (N)b	Postop follow-up % (N)c	Length of follow-up (y)	Contact method	Primary funding source	PROM utilized
Wylde et al (2009) [31]	Retrospective Cohort	Orthopedic Center, Australia	III	67.5-70	65%	2391	NR	NR	72% (1725)	5-8	☒Mail ☐Web-based ☐Phone	None	OHS or OKS
Patel et al (2015) [18]	Retrospective Cohort/Registry Review	California Joint Registry (2011-2014)	III	NR	57%	6861	Yes	30% (2070)	11% (719)	1	☒Mail ☒Web-based ☒Phone	NR	SF-12, WOMAC, UCLA Activity Index

NR, not reported; OHS, Oxford Hip Score; OKS, Oxford Knee Score; SF-12, Twelve Item Short Form Survey; THA, total hip arthroplasty; TKA, total knee arthroplasty; UCLA, University of California at Los Angeles; WOMAC, Western Ontario and McMaster Universities Osteoarthritis Index.

^aDesignated according to Elsevier publishing definitions (https://www.elsevier.com/__data/promis_misc/623124los.pdf), unless marked with *, which is journal article designated LOE.

^b% Calculated from preop completion N divided by initial N.

^c% Calculated from postop completion N divided by initial N.

Five studies reported PROM follow-up rates between 60% and 69%, with a median of 1750 patients (range 1315-226,805) [13], [14], [23], [27], [28]. Only 1 study reported collection of preoperative PROMs [23]. Funding was received by 4 of the 5 studies; 1 study reported no funding [27]. Patients were contacted via mail in 3 studies, 1 study did not report a contact method [23], and 1 study used combination of mail, phone calls, and clinic visits [14]. There were 3 multicenter studies with all 5 studies being retrospective (LOE III) (Table 1, Table 2, Table 3).

Five studies reported a PROM follow-up rate of <60%, with a median of 6861 patients (range 2695-216,265) [15], [16], [18], [29], [30], of which only 1 reported collection of preoperative PROMs [18]. Funding was reported by 4 of the 5 studies. Patients were contacted via mail, phone call, and/or web-based system in 1 study [18], and another study used a combination of mail, phone calls, and clinic visits [15]. The remaining 3 studies did not report a contact method. All 5 studies were multicenter studies and either retrospective or cross-sectional (LOE III) (Table 1, Table 2, Table 3).

Discussion

In our systematic review of large studies with at least 1000 participants and the use of validated PROMs following total joint arthroplasty, we found a total of 21 studies involving total hip and/or knee arthroplasty. No randomized controlled trials with more than 1000 patients were identified that included PROMs as a primary outcome. Thus, all of the studies we analyzed were cohort studies or registry studies. The studies with highest follow-up rates were multicenter studies with federal funding sources.

A paucity of studies report outcomes after total joint arthroplasty with more than 1000 participants. Of the 21 studies included in this systematic review, only 38% (8 of 21) collected preoperative PROMs to establish a baseline. Obtaining baseline measures of joint pain and function are important as these are highly predictive of postoperative PROMs [18], [32]. Studies with a minimum of 1000 patients allow for the simultaneous adjustment for several potential confounders (using multivariable analysis) that may influence the outcome of a patient including age, gender, education or socioeconomic status, and smoking. This study design allows for controlling important confounders of outcome and can determine the treatment effect of the given surgical procedure. Additionally, our experience with comprehensive multivariable analysis in large observational outcome studies has taught us that a minimum of 1000 patients is needed to adequately control for demographic data, disease severities, treatments rendered, and implants placed [33], [34]. This is particularly true in the world of total joint arthroplasty, where new implants and surgical techniques are being developed constantly. Therefore, following outcomes of at least 1000 patients after total joint arthroplasty is key to conducting a well-controlled observational study. This is not to imply that smaller studies are of lesser scientific validity or merit, as there are certainly instances when utilizing cohorts of >1000 participants is impractical or unnecessary. However, when considering the multitude of variables that may impact the outcome of a total joint arthroplasty aside from the usual demographics (eg, implant manufacturer, surgeon volume, surgeon experience, surgical technique utilized, severity of preoperative illness, degree of preoperative deformity, etc.), large studies such as those included in this systematic review have the power to be used for comprehensive multivariable analysis. Nonetheless, studies of this size are the overwhelming minority of what is reported in current literature.

It is difficult to achieve a follow-up of more than 80% of study participants in large cohorts or registries after orthopedic surgery. Only 2 studies identified in the present review were able to achieve a follow-up rate of 80% or greater [11], [19], [20], [24]. However, neither of these studies collected baseline PROMs at the time of enrollment, which makes assessing the impact of a given procedure nearly impossible (ie, one can conclude that after procedure X, patients “do well,” but one cannot gauge how much patients improve or if patients improve more when undergoing procedure X vs procedure Y).

Maintaining high follow-up rates is crucial, as losing participants can introduce significant response bias. The possibility of significant response bias is particularly worrisome when those who remain in the study (ie, those who are not lost to follow-up) are different than those who did not [6]. Murray et al [35] reported that total hip replacement patients who became “lost to follow-up” had significantly worse pain, range of motion, opinion of their progress, and radiographic features at last recorded assessment. As the proportion of patients lost to follow-up increases, so does the likelihood of study invalidation [36], because patients who are lost to follow-up are likely different than those not lost, and, as a result, the data obtained from patients who continue to participate cannot be generalized to patients lost to follow-up [6]. Additionally, a growing body of evidence suggests that socioeconomic differences exist between those who follow-up compared to those who fail to respond [37], [38]. This makes achieving high follow-up rates important to preserve both study quality [10] and the validity of the study results.

The present review teaches us important lessons regarding resources and techniques used to successfully follow large groups of patients. It is worth noting that studies with the highest follow-up rates tended to receive funding from large, national funding sources such as the National Institutes of Health, the Norwegian national agencies [39] (Ministry of Health and Social Affairs, Norwegian Medical Association, and Norwegian Research Council), and the Danish regional governments [40].

Interestingly, all studies included in this systematic review utilized traditional mail to obtain follow-up, and only 1 study utilized a web-based follow-up method [38]. Our personal experience conducting a prospective cohort study of total joint arthroplasty patients supports the notion that older individuals (ie, those most likely to undergo total joint arthroplasty) are responsive to requests for follow-up via mail, while younger individuals (ie, those most likely to undergo anterior cruciate ligament reconstruction) are significantly less responsive to mail (unpublished data). This is a trend worth observing over the coming years, as generations that are more inclined to rely on the internet reach their 60s and 70s and begin to utilize the services of total joint arthroplasty surgeons.

Another commonality shared by studies with the highest follow-up rates was the use of a central site for follow-up coordination. Thus, it seems that receiving federal funding (or financial support from a large funding source) and the use of a centralized site to coordinate follow-up are both associated with successful follow-up of large cohorts of patients following total joint arthroplasty.

The present study has several strengths. It is the first of its kind, to our knowledge, that has evaluated the ability of study groups to obtain outcomes on large cohorts or registries of patients after total joint arthroplasty. Furthermore, we have helped identify key factors that were associated with high follow-up rates including federal funding sources and a centralized coordinating site. Also, we have identified areas in orthopedics that could benefit from large (≥1000 patients) studies statistically powered for comprehensive multivariable analysis.

The results of the current systematic review can aid future researchers who hope to receive grant funding and/or improve the quality of future research studies in many ways. First, this study has demonstrated that conducting a large study is feasible, but almost always requires a multicentered approach so that data can be collected in a timely fashion. The authors would highly recommend that any researcher hoping to conduct future studies aimed at understanding outcomes following total joint arthroplasty would either develop partnerships with other high-volume centers to collect prospective data, or less desirably, use data from existing databases. Second, the importance of collecting prospective data cannot be understated. At the very least, the authors suggest that any further research involving total joint arthroplasty outcomes should include preoperative (or “baseline”) data against which postoperative outcomes can be compared. As this systematic review has demonstrated, collection of preoperative PROMs is not the norm for large total joint arthroplasty research. Thus, future researchers hoping to secure grant funding should be sure to design their study to include the collection of preoperative PROMs. Third, this systematic review has demonstrated that the most successful studies (in terms of follow-up rates) have utilized multiple follow-up methods including phone calls and mail. Nonetheless, as the longevity of total joint arthroplasty implants grows and younger technologically savvy generations age, the authors anticipate that electronic forms of follow-up data collection, such as e-mail and text messaging, will become increasingly prevalent and more successful. However, the conclusion remains the same—multiple forms of follow-up are crucial to maximizing follow-up rates and minimizing missing data. Finally, it is worth acknowledging that there likely exists a direct relationship with the quality of study design and the likelihood of receiving grant funding. Although this might seem obvious, the authors feel it is worth stating directly. Therefore, studies that are designed in such a way that will allow for successful collection of both preoperative and postoperative PROMs (including both general health and joint-specific PROMs) are more likely to receive federal funding, and those that receive federal funding are likely to be more successful at achieving high follow-up rates. The authors hope that future researchers will benefit from these recommendations as they are the result of evidence (as demonstrated by this systematic review) as well as years of experience (oftentimes with failure).

Our review does have some weaknesses worthy of discussion. Several large studies were identified but had to be excluded as the methodology regarding patient selection and follow-up rates were unclear. This limitation is inherent to study design. Also, most studies found in this review were of level III evidence with only 4 studies of level II evidence and no level I studies. The included studies had high variability on the reporting of baseline scores and enrollment, follow-up PROMs, and time frames for outcome collection after surgery. These factors were prohibitive to performing a meta-analysis. Additionally, some studies included in this systematic review collected only general health outcome measures (such as the EuroQoL-5D or the visual analog scale) [30], while others collected only joint specific outcome measures (such as the Oxford Hip Score or the Western Ontario and McMaster Universities Osteoarthritis Index) [11], [21], [25], [26], [28], [31]. Because both are technically patient-reported outcomes, they fulfill the criteria for inclusion in this study. However, an ideal study includes both a general health outcome measure as well as a joint-specific outcome measure, as general health outcome measures are typically not sensitive enough to detect meaningful changes in patient outcomes, and, when a change is detected, it is unknown whether it is due to the surgery of interest (total joint arthroplasty, in the case of the studies included in this systematic review). On the other hand, studies that utilize only joint-specific outcome measures may detect changes in patient outcomes that are due to changes in overall health status (eg, development of depression or new illness such as symptomatic coronary artery disease). Thus, both types of outcome measures should be utilized in large studies going forward in order to avoid these issues.

The authors would also like to note that significant consideration was given to assessing the quality of each individual study included in this current systematic review with the use of Grading of Recommendations, Assessment, Development and Evaluations or another study quality assessment tool. However, the authors ultimately felt that doing so is beyond the scope of this systematic review. Unlike most systematic reviews, the current study did not set out to assess the outcomes of total joint arthroplasty nor did it aim to use the available evidence to make recommendations regarding how total joint arthroplasty should be performed. Instead, the current study set out to better understand what aspects of study design lend themselves to successful collection of PROMs from large study populations that have undergone total joint arthroplasty. Thus, it would be inappropriate and add little to the current study to have included an assessment of the quality of each study included in this systematic review.

Conclusions

Obtaining baseline measures of joint pain and function during study enrollment, as well as follow-up PROMs after orthopedic surgery, is challenging for large cohort studies or registries of patients. Few studies in the present literature were able to obtain validated PROMs on more than 1000 patients with greater than 80% follow-up rate; these are the parameters conducive to comprehensive multivariable analysis, while maintaining study validity and avoiding follow-up bias. Future research in our field should strive to maximize both sample size and follow-up rate, as both are necessary for conducting high impact research that is scientifically valid. Factors that appear to be conducive to performing large studies (≥1000 participants) with high follow-up rates include a federal funding source and multicenter design with a central follow-up coordination site. Although the most successful studies reported in this systematic review utilized traditional methods of achieving follow up, such as mail or phone call, future studies are likely to rely more heavily on electronic methods of follow-up such as e-mail or text messaging.