Risk Factors for Loss to Follow-Up in 3,202 Patients at Two Years after ACL Reconstruction: Implications for Identifying Health Disparities in the MOON Prospective Cohort Study

Abstract

Background:

Understanding the risk factors for loss to follow-up in prospective clinical studies may allow for a targeted approach to minimizing follow-up bias and improving the generalizability of conclusions in ACL reconstruction (ACLR) and other sports-related interventions.

Purpose:

The purpose of this study was to identify independent risk factors associated with failure to complete (i.e., loss to follow-up) patient-reported outcome measures (PROMs) at two years after ACLR within a well-funded prospective longitudinal cohort.

Study Design:

Prognostic cohort study

Methods:

All patients undergoing primary or revision ACLR enrolled in the prospectively collected database of the multicenter consortium between 2002–2008 were included. Multivariate regression analyses were conducted to determine which baseline risk factors were significantly associated with loss to follow-up at a minimum of 2 years post-surgery. Predictors assessed for loss to follow-up were as follows: consortium site, sex, race, marital status, smoking status, phone number provided (home or cell), email address provided (primary or secondary), years of school completed, average hours worked per week, working status (full time, part time, homemaker, retired, student, disabled), number of people living at home, and pre-op PROMs (KOOS subscales, Marx, IKDC).

Results:

A total of 3,202 ACLR patients were enrolled. The 2-year PROM follow-up rate for this cohort was 88% (2,821/3,202). Multivariate analyses showed that patient sex (male [OR 1.8]), race (Black [OR 3.6], other non-Caucasian [OR 1.8]) were independent predictors of 2-year loss to follow-up of PROMs. Education level was a non-confounder.

Conclusion:

While education level did not predict loss to follow-up, patients who are male and non-Caucasian are at increased risk of loss of PROMs follow-up at two years. Capturing patient outcomes with minimal loss depends on equitable, not equal, opportunity to maximize generalizability and mitigate potential population-level health disparities.

Social Media Summary:

Patients who are male and non-Caucasian are at increased risk of loss of PROMs follow-up at two years. Capturing patient outcomes with minimal loss depends on equitable, not equal, opportunity to maximize generalizability and mitigate potential population-level health disparities.

Introduction

Outcomes research, by way of patient-reported outcome measures (PROMs), helps guide the clinical practice of orthopaedic surgery^1,3. Collecting and documenting these clinically relevant outcomes through validated PROMs in the perioperative period is important¹⁵. They aid in measuring the response to orthopaedic surgery on a patient’s overall and joint-specific health as well as in determining the value of an orthopaedic intervention. By extension, the composition of patients represented in these high-level, multicenter prospective outcomes-related research studies warrants careful consideration prior to drawing insights. Conversely, the patients not included in the study from loss to follow-up are an equally important consideration.

“Adequate” follow-up of orthopaedic PROMs is important to limit bias. A high loss-to-follow-up rate introduces study response bias and undermines the validity of the observations. Unlike in the medical setting whereby patients are naturally “incentivized” to return to the office for continued management of long-term chronic diseases (i.e., prescription refills, insulin, etc.), orthopaedic patients who undergo surgery do not routinely return to the orthopaedic surgery office at one year to determine treatment response. They may be doing superbly, poorly, or anywhere in between the two extremes^14,18. Some have suggested a loss of less than 5% leads to little bias and greater than 20% poses serious methodological concerns⁴. Despite the insights drawn from well-funded and impactful prospective longitudinal cohort studies²¹, collection of outcomes in orthopaedics remains a challenge and premature loss to follow-up introduces marked response bias that affects the final conclusions in outcomes research^{2, 8, 12}.

Beyond the risk of short-term study bias, patients underrepresented in the sports medicine literature broaches the possibility of potential health disparities in the long term. If racial, sex, or socioeconomic disparities belie loss to follow-up among our highest-level ACLR studies, we may be neglecting a significant portion of the patient population and foregoing health equity for equality. If the characteristics of the enrolled cohort and the follow-up cohort are different, we cannot assume that research findings are generalizable to the group that does not follow up. By identifying which patients are at risk for loss to follow-up, more attention may be directed to this population to acquire PROMs and promote representation in the literature. Despite enormous effort in obtaining PROM information, limited literature exists on which patients are likely to follow up or are likely to be lost to follow-up for PROMs²⁰. Identifying patients lost to follow-up will improve validity and generalizability in PROMs research. Furthermore, if specific populations are at risk of being lost to follow-up, investigators can funnel additional resources to those areas. The purpose of this study was to identify the risk factors associated with failure to complete patient-reported outcome measures (PROMs), or loss to follow-up, after ACLR within a well-funded prospective multicenter longitudinal cohort²¹. We hypothesized that younger age, male sex, and fewer years of education would be associated with greater loss to follow-up in multivariate analysis.

Methods

Multicenter Orthopaedic Outcomes Network (MOON) Cohort

The study was initiated in 2002, and all patients undergoing unilateral primary or revision ACLR were eligible to be enrolled. Workman’s compensation patients were an exclusion criterion. All subjects gave their informed consent to participate, and the study was approved by each site’s Institutional Review Board prior to commencement of patient enrollment.

There were 3,202 patients who met this study’s inclusion criteria, consented to the study, and completed a baseline questionnaire. This questionnaire included baseline demographics, injury descriptors, sports participation level, comorbidities, knee surgical history, and PROMs, including: the International Knee Documentation Committee (IKDC)⁶ subjective form, the 5 subscales of the Knee injury and Osteoarthritis Outcome Score (KOOS)¹⁹, and the Marx activity rating scale¹¹. Enrollment questionnaire was completed within 2 weeks of surgery date with the majority done before surgery.

Additionally, each surgeon completed a questionnaire documenting the results of examination under anesthesia, surgical technique utilized, and the surgical findings as well as treatments of any concomitant meniscus and cartilage injuries. The modified Outerbridge articular cartilage injury classification system¹⁰ was used. All data were compiled and sent to a central coordinating site for processing, data cleaning and logical error checks, data reduction, and storage.

Outcome Measure

The primary outcome measure of interest was loss to follow-up, as defined by any patients who did not return PROMs two years after surgery. Follow-up was prospectively managed at the central coordinating site beginning two weeks prior to surgery. The central site was responsible for contacting and sending the same paper questionnaire with PROMs to each patient. The patient’s completed PROM questionnaires were sent (self-addressed stamped envelope) back to the central site for data quality checks and then scanned into the database. After the central site completed their attempts to contact and obtain return PROMs, the site and surgeon were sent a call list of their patients to improve follow-up rates for those who did not respond. Each surgeon’s individual follow-up rate was tracked and transparent to all surgeons in the group.

Predictors of Interest

Predictors assessed for significance in loss to follow-up were as follows: consortium site, sex, race, marital status, smoking status, phone number provided (home or cell), email address provided (primary or secondary), years of school completed, average hours worked per week, working status (full time, part time, homemaker, retired, student, disabled), number of people living at home, and pre-op PROMs (KOOS subscales, Marx, IKDC).

Statistical Analysis

To determine the association between patient variables and whether or not PROMs follow-up was obtained (completed questionnaire vs. did not complete), a multivariate analysis was conducted. Wilcoxon Sign Rank tests were performed for continuous variables, while Chi-Square tests were used for categorical variables. If small sample sizes occurred in the categorical groups, the Fisher’s exact test was used.

Multivariate analyses (multiple logistic regression modeling) were conducted to establish predictors of loss to follow-up. Each model started with the full dataset, and variables were removed if they had a high amount of standard error in the model or had issues with colinearity. From the full model, a step-down procedure was performed to identify the most parsimonious model.

Results

A total of 3,202 patients were eligible in the cohort (Table 1). The 2-year follow-up rate was 88% (2,821/3,202).

Table 1:

Patient demographics

	Study Cohort
N	3,202
Median Age (Range)	23 years (10–68)
% Male (N)	56% (1,791)
% Caucasian (N)	83% (2,645)
% Non-Smokers (N)	80% (2,548)
Education level1	14 (11–16)
Surgeries:
➢ACLR-Primary	2987 (93%)
➢ACLR- Revision	215 (7%)

¹[median # (interquartile range) of school years completed]

Predictors of PROMs follow-up

Table 2 is the classification of and relationship to follow-up in the study cohort. The consortium site, patient sex, race, marital status, smoking status, education level (years of school), and presence of disability were significantly different based on completion of follow-up. In addition, other factors significantly different based on follow-up were living with a partner, whereas living with parents, children, siblings, or alone were not (Table 2). A patient-provided email contact was also found to be significant, whereas a patient-provided phone contact was not. All five subscales of the preoperative KOOS and the IKDC were found to be significantly associated with PROMs follow-up; however, Marx activity score was not significantly associated.

Table 2:

The Relationship of Risk Factors to the Classification of and Relationship to Follow-up in the Study Cohort

Risk Factors	Study Cohort
	N = 3,202	Completed Follow-up	Lost to Follow-up	p-value
		2,821 (88%)	381 (12%)
Consortium Site				0.0029
#1	684 (21%)	610 (22%)	74 (19%)
#2	185 (6%)	165 (6%)	20 (5%)
#3	869 (27%)	737 (26%)	132 (35%)
#4	130 (4%)	124 (4%)	6 (2%)
#5	139 (4%)	122 (4%)	17 (4%)
#6	757 (24%)	682 (24%)	75 (20%)
#7	438 (14%)	381 (14%)	57 (15%)
Sex
Male	1,791 (56%)	1,539 (55%)	252 (66%)	<0.0001
Female	1,411 (44%)	1,282 (45%)	129 (34%)
Race
White	2,645 (83%)	2,394 (85%)	251 (66%)	<0.0001
Black	278 (9%)	194 (7%)	84 (22%)
Other	279 (9%)	233 (8%)	46 (12%)
Marital Status
Married	882 (29%)	803 (30%)	79 (22%)	0.0041
Single	2,090 (68%)	1,827 (67%)	263 (73%)
Separated	112 (4%)	93 (3%)	19 (5%)
Missing	118 (4%)
Smoking Status
Current	292 (9%)	236 (9%)	56 (15%)	0.0002
Never	2,548 (80%)	2,264 (81%)	284 (76%)
Quit	325 (10%)	292 (10%)	33 (9%)
Missing	37 (1%)
Home Phone Number Provided
Yes	2,984 (93%)	2,635 (93%)	349 (92%)	0.1891
No	218 (7%)	186 (7%)	32 (8%)
Cell Phone Number Provided
Yes	1,136 (36%)	998 (81%)	138 (78%)	0.257
No	265 (8%)	226 (19%)	39 (22%)
Missing	1,801 (56%)
Primary Email Provided
Yes	2,070 (65%)	1,859 (66%)	211 (55%)	0.0001
No	1,132 (35%)	962 (34%)	170 (45%)
Secondary Email Provided
Yes	273 (9%)	251 (9%)	22 (6%)	0.0405
No	2,929 (91%)	2,570 (91%)	359 (94%)
Years of School Completed	14 (11, 16)	14 (11, 16)	13 (11, 16)	0.0013
Missing	17 (1%)
Average Hours Worked per Week	25 (0, 44)	25 (0, 45)	30 (0, 40)	0.9272
Missing	564 (18%)
Works Full Time
Yes	1,181 (37%)	1,050 (37%)	131 (34%)	0.2812
No	2,021 (63%)	1,771 (63%)	250 (66%)
Works Part Time
Yes	314 (10%)	278 (10%)	36 (9%)	0.8026
No	2,888 (90%)	2,543 (90%)	345 (91%)
Works as Homemaker
Yes	99 (3%)	88 (3%)	11 (3%)	0.8058
No	3,103 (97%)	2,733 (97%)	370 (97%)
Retired
Yes	24 (1%)	23 (1%)	1 (0%)	0.3499
No	3,178 (99%)	2,798 (99%)	380 (100%)
Student
Yes	1,504 (47%)	1,335 (47%)	169 (44%)	0.2761
No	1,698 (53%)	1,486 (53%)	212 (56%)
Disabled
Yes	100 (3%)	79 (3%)	21 (5%)	0.0043
No	3,102 (97%)	2,742 (97%)	360 (95%)
Working Other
Yes	131 (4%)	102 (4%)	29 (8%)	0.0002
No	3,071 (96%)	2,719 (96%)	352 (92%)
How many people live at home with patients	2 (1, 3)	2 (1, 3)	2 (1, 3)	0.4558
Missing	330 (10%)
Lives Alone
Yes	292 (9%)	259 (9%)	33 (9%)	0.7408
No	2,910 (91%)	2,562 (91%)	348 (91%)
Lives with Parents
Yes	1,301 (41%)	1,153 (41%)	148 (39%)	0.4496
No	1,901 (59%)	1,668 (59%)	233 (61%)
Lives with Partner
Yes	1,012 (32%)	909 (32%)	103 (27%)	0.0409
No	2,190 (68%)	1,912 (68%)	278 (73%)
Lives with Children
Yes	717 (22%)	635 (23%)	82 (22%)	0.6643
No	2,485 (78%)	2,186 (77%)	299 (78%)
Lives with Other
Yes	930 (29%)	818 (29%)	112 (29%)	0.8719
No	2,272 (71%)	2,003 (71%)	269 (71%)
Lives with Siblings
Yes	332 (24%)	297 (24%)	35 (20%)	0.1891
No	1,069 (76%)	927 (76%)	142 (80%)
Not Applicable	1801	1597	204
Pre-OP KOOS Symptom Score	68 (54, 82)	68 (54, 82)	64 (50, 79)	0.0004
Missing	5 (0%)
Pre-OP KOOS Pain Score	75 (61, 86)	75 (61, 86)	69 (53, 83)	<0.0001
Missing	7 (0%)
Pre-OP KOOS ADL Score	87 (71, 96)	87 (71, 96)	82 (63, 93)	<0.0001
Missing	3 (0%)
Pre-OP KOOS Sports and Rec Score	50 (25, 75)	50 (25, 75)	45 (20, 70)	0.0234
Missing	63 (2%)
Pre-OP KOOS QOL Score	38 (19, 50)	38 (19, 50)	31 (19, 44)	0.0006
Missing	8 (0%)
Pre-OP MARX Score	12 (8, 16)	12 (8, 16)	12 (6, 16)	0.3665
Missing	26 (1%)
Pre-OP IKDC Score	51 (39, 62)	51 (39, 63)	46 (33, 59)	<0.0001
Missing	47 (1%)

To determine which risk factors are independent predictors of loss to follow-up, a multivariate analysis was performed controlling for confounding factors. Utilizing multivariate analysis, only patient sex (male [OR 1.8]), race (black [OR 3.6]), and preoperative KOOS pain scores (lower [OR 0.98]) remained significantly associated with PROMs follow-up (Table 3). Interactions between age and education (years of school), race and education (years of school), and race and sex were assessed and found not to be significant. Therefore, being at high risk for loss to follow-up were males, those of non-Caucasian race, and those with lower preoperative KOOS pain scores.

Table 3:

Multivariate Analyses

	Odds Ratio	95% Confidence Interval	p-value
Male Sex	1.80	(1.41,2.30)	<0.001
Age	0.98	(0.96,1.00)	0.065
Race -- Caucasian (reference value)	-	-	-
Race -- Black	3.64	(2.67,4.96)	<0.0001
Race -- Other	1.81	(1.25–2.64)	0.002
Marital Status – Single (reference value)	-	-	-
Marital Status -- Married	0.99	(0.67,1.45)	0.946
Marital Status -- Separated/Divorced/Widowed	1.84	(0.98, 3.48)	0.060
Education Level (Years of School Completed)	0.98	(0.94,1.03)	0.488
Pre-op KOOS pain score	0.98	(0.98,0.99)	<0.001

Discussion

Using the prospective longitudinal MOON cohort of ACLR primary and revision patients, we identified characteristics of patients who are at risk for loss to follow-up in terms of PROMs and, thus, may be underrepresented in the ACLR literature. Patients who are male and non-Caucasian are significantly less likely to complete PROMs follow-up (Table 3). While lower preoperative KOOS pain scores was statistically significant, an odds ratio of 0.98 is likely clinically insignificant. To our knowledge, this is the first study to demonstrate in a multivariable analysis of a prospective sports cohort on patient risk factor for loss to follow-up of PROMs.

These findings underscore the role of disparities related to sex, age, race, social support, and mental health state in the ACLR literature. Men are at increased risk of loss to follow-up, suggesting additional efforts should be made to better engage them pre- and post-operatively to obtain their PROMs follow-up. Black and other non-Caucasian races were associated with higher risk of loss to follow-up,. Certainly, the data that highlight race as a risk for loss to follow-up should not be ignored altogether. Recognizing that non-Caucasians may experience health disparities in the United States is not a novel reveal, and this study underscores that even in the ACLR literature such underrepresentation exists. The concept of health equity suggests particular populations require more resources to achieve a desired outcome than others due to immutable patient-specific factors. While the most pragmatic denominator between the described patients at risk for loss to follow-up is socioeconomic status, this was unable to be directly addressed from this study. However, health equity reverses the narrative of health equality. Health equality would place the onus of blame on those who fail to complete the PROMs. Health equity recognizes the actual limitation possibly being the method by which we connect with our patients, which may be inaccurate, ineffective, or both. To address potential follow-up biases and combat health inequities in the literature, these data can be used to target the affected populations and maximize the follow-up. Further research is needed to mitigate this problem.

Interestingly, whether patients were willing to provide email addresses or phone numbers had no impact on their follow-up; it would be expected that patients willing to provide additional contact information would be more willing to follow up.¹⁶ One potential future avenue to increase patient follow-up in an increasingly connected world may be through digital means such as smartphone-based messaging or mobile applications, particularly among the inclined younger patients who may be averse to mailing in PROMs. While there is an argument that increased digitization of PROMs may exacerbate disparities, 81% of Americans now own a smartphone with Internet access, representing an increase of 11% in just the past 3 years.²² This steady increase, with federal endeavors seeking to improve connectivity in underserved areas, suggests more people will be “online”.

Our prospective cohort of ACLR patients had 381 patients who failed to complete their 2-year PROMs, which we could subsequently analyze for predictors of loss to follow-up. Given the fact that arthroplasty patients differ from ACLR patients, only one other study in the literature has examined these predictors in a similar population. Reinholdsson et al.¹⁷ reported on ACLR patients in the Swedish Knee Ligament Registry, which had a follow-up of 52% at two years, and similarly found younger age and male sex to be significant factors in loss to follow-up. However, this 2017 study has a high loss to follow-up rate, which limits generalizability, and has an absence of sufficient non-Caucasian patients to account for differences. We found that our findings from ACLR patients are comparable to those from larger previous studies on arthroplasty patients. The patient variables of age and race have been found to be significantly associated with PROMs follow-up in multiple studies as reported by Hutchings et al.⁵, Schamber et al.²⁰, and Patel et al.¹³ on the England National Joint Registry, University of California San Francisco’s retrospective cohort, and California Joint Registry, respectively. Hutchings et al.’s multivariable logistic regression of 37,961 total hip arthroplasties (THAs) and 44,422 total knee arthroplasties (TKAs) found younger patients (those under 55 years: OR 3.01– 6.05) and non-white patients (OR 1.24–2.08) to be significantly less likely to follow up to PROMs at 9 months.⁵ In addition, Hutchings et al. also found men (OR 1.03–1.35), patients in the lowest quintile of socioeconomic status (OR 1.47–1.86), those who lived alone (OR 1.11–1.27), and those with poorer preoperative health (3 or more comorbidities: OR 1.14–1.45) to be less likely to follow up to PROMs.⁵ Judge et al. reported on 1,991 TKAs and found that those with greater preoperative pain were a higher risk for loss to follow-up, although this was unlikely a clinically meaningful difference in our study.⁷ In the trauma literature, Madden et al.⁹ studied the Fluid Lavage of Open Wounds (FLOW) Trial and similarly found younger age and male sex to be significant factors in loss to follow-up.

This recurring theme of patient factors affecting follow-up in both our study and those from the sports, arthroplasty, and trauma literature underscore the importance of addressing these patient-level factors introducing selection bias in the short term and potentially exacerbating health disparities in the long term. While the aforementioned studies made no mention of race, socioeconomic status was central to those studies able to report on this metric.

Our study is not without limitations. The primary limitation is the inability to control for socioeconomic status, which certainly affects the conclusions related to health disparities. While sex and race are associated with loss to follow-up, the patient’s financial status may certainly impacts his or her ability to follow-up. The closest metric related to annual income available is educational level but was a non-contributor in the multivariate analysis. Furthermore, patients lost to follow-up may possibly represent the floor and ceiling in terms of outcomes; as such, patients doing very well may see follow-up as futile and those doing poorly enough may seek care elsewhere. Examining overall location of each consortium site with surrounding patient population, as well as preoperative activity level, would have been a valuable addition to the study that we were unable to account for or perform. While our study consortium sites serve patients from major metropolitan cities, suburban areas, and rural areas, the ideal metric would be the patients’ zip code, which may confer socioeconomic status. Level of competition may certainly impact a player’s likelihood to follow-up, as a college athlete with higher performance expectations may be more diligent with physician follow-up, including PROMs. Workman’s compensation patients were excluded from the study due to additional incentives, yet they represent a population worth examining in the future for comparison with the present findings. The postoperative clinical course or experience was unable to be accounted for due to absence of PROMs data and may inherently confound loss to follow-up.

In our study, patients who are male and non-Caucasian are significantly less likely to complete PROMs follow-up. Therefore, these patients are likely underrepresented in sports medicine outcomes research and capturing their outcomes would increase the value and generalizability of outcomes measurement. Patients fitting these demographics may either be inherently less engaged or require other resources to achieve PROMs follow-up. Regardless, equitable opportunity should be prioritized over equal opportunity when it comes to PROMs follow-up if we are to gather insights that will be generalizable to these populations. Further research is needed on effective means to reach these patients, and the advent of mobile devices that offer immediate, ubiquitous connectivity offers hope to reach an increasingly online and connected patient population. As can be seen from the data presented in this study, capturing patient outcomes with minimal loss to follow-up in the ACL and sports literature represents not just scientific rigor of curtailing selection bias but also mitigating population-level health disparities.

What is known about the subject:

Outcomes research guides patient care, and the value of prospective level one clinical studies represents the highest form of evidence-based medicine. However, even these studies are limited by patient selection.

What this study adds to existing knowledge:

In the sports and ACL literature, specifically, it is unknown whether health disparities due to social, racial, or gender differences belie differences in high level prospective studies. Better understanding of risk factors for loss to follow-up may improve the generalizability of such studies and guide improved representation in the literature.