Abstract
Background:
Patient-reported outcome (PRO) surveys have become increasingly important in both improving patient care and assessing outcomes.
Purpose/Hypothesis:
The purpose of this study was to evaluate which variables are associated with compliance with completing PRO surveys in patients who have undergone anterior cruciate ligament reconstruction (ACLR). The authors hypothesized that older patient age and longer time since ACLR would be associated with lower completion rates of PRO surveys preoperatively and at postoperative time points.
Study Design:
Case series; Level of evidence, 4.
Methods:
All patients who underwent ACLR by a sports medicine fellowship–trained orthopaedic surgeon at a single institution and were electronically assigned PRO surveys through a data collection system preoperatively between December 2013 and March 2015 were included. Postoperatively, PRO surveys were sent to patients’ email addresses at 6, 12, and 24 months. Demographics, history, and operative and postoperative information were evaluated for an association with survey completion rates.
Results:
A total of 256 patients met the inclusion criteria. There were 140 (54.7%) male and 116 (45.3%) female patients, with an overall mean age of 28.6 ± 11.9 years. Only 19 (7.4%) patients completed all preoperative and postoperative surveys at all time points. Less than half of the patients (n = 104; 40.6%) completed both the preoperative survey and at least 1 postoperative survey. There was a steady decrease in the completion rate of PRO surveys postoperatively over time (Pearson r = –0.995, P = .005). Male patients had significantly worse compliance with completing PRO surveys preoperatively and at a minimum of 1 time point postoperatively (P = .044). Patients who did not identify as athletes or report frequent exercise (≥3 times/wk) had significantly worse compliance with completing any PRO surveys (P = .046). Lower body mass index was associated with greater odds of compliance with completing the preoperative survey and 24-month postoperative survey (odds ratio, 0.902; P = .029).
Conclusion:
An inverse relationship was found between the surgery-to-survey period and percentage of those completing PRO surveys, with poor overall compliance. Male sex and not self-identifying as an athlete or performing frequent physical exercise were associated with lower completion rates of PRO surveys, while lower body mass index was associated with a greater rate of completion.
Keywords: ACL, patient-reported outcome surveys, ACL reconstruction, electronic data collection system
Anterior cruciate ligament (ACL) tears are one of the most common and most studied orthopaedic injuries, with an annual incidence of 68.6 ACL tears per 100,000 person-years.25 In addition, ACL reconstruction (ACLR) is consistently reported as one of the most frequently performed orthopaedic procedures.4,5,9,30 Given the high prevalence of injuries and surgical management, an influx of clinical research has focused on measuring postoperative success in a reliable, reproducible fashion.17 Recently, patient-reported outcome (PRO) surveys have become important metrics as practitioners seek to improve the quality of health care and subjective experience for patients and to obtain useful outcome data for research.3,6,16,20,22 PRO surveys are administered at defined time points, allowing providers to capture and quantify patients’ experiences of their care in a comparable way. Furthermore, the change in PRO scores from preoperatively to postoperatively is gradually becoming a standard metric by which “success” of a procedure is defined, and it has a very real impact on medical decision making.2,11,31 For research purposes, high-impact orthopaedic journals have defined a postoperative time point of 24 months as the minimum short-term follow-up.1 Orthopaedic practices and research centers have thus been working to improve patient compliance with completing PRO surveys. While not all institutions are inclined to publish, improving the ease of administering outcome metrics can not only increase compliance rates but also allow for collaboration with other institutions and a more comprehensive assessment of patient outcomes.
As PRO surveys become increasingly important in improving both patient care and physician reimbursement, it is essential to evaluate what patient- and injury-specific factors are related to compliance with completing PRO surveys, so as to achieve the highest possible compliance. To be able to apply surveys to all patients, data collection systems must be efficient and inexpensive and must minimize the use of additional personnel to manage patient compliance. In addition, for the outcomes data to be of any value with regard to assessing the physician, facility, or procedure, data sets that are representative of the given population are necessary. If compliance is inadequate, the data do not have any meaningful use.
PRO scores may be collected through a variety of vehicles, including email, telephone, and in person, with variable success. There is a growing interest in electronic PRO collection because of the proposed benefits of improved response time, ease of data organization and analysis, and cost efficiency.18,21,24 However, each vehicle utilized to collect PRO scores has distinct advantages and disadvantages and demonstrates variable success depending on the specific patient. For example, in patients undergoing arthroplasty, previous studies have identified decreased compliance with completing PRO surveys in older patients.23,26
The purpose of this investigation was to determine which factors are associated with completion rates of PRO surveys administered via an electronic data collection system preoperatively and at 6, 12, and 24 months postoperatively after ACLR. We hypothesized that there would be lower completion rates in older patients both preoperatively and postoperatively and that completion rates for all patients would decline the farther patients are out from surgery.
Methods
After institutional review board approval, a database of prospectively collected information for all board-certified sports medicine orthopaedic surgeons at our institution was queried for patients who underwent arthroscopically aided ACL repair/augmentation or ACLR (Current Procedural Terminology code 29888) from December 2013 until March 2015. This time frame was chosen because in December 2013 our institution began using an electronic survey portal (OBERD; Universal Research Solutions) to distribute and manage PRO surveys through an online email interface, allowing for conformity in PRO collection methods. All patients who underwent ACLR received the same series of PRO surveys (termed an order set) and at all postoperative time points. Five PRO surveys were administered as part of the order set: the Lysholm score, Marx activity scale, International Knee Documentation Committee (IKDC) score, Knee Injury and Osteoarthritis Outcome Score (KOOS), and Short Form–12. In our experience, it takes approximately 20 minutes to complete all 5 surveys. This order set was assigned preoperatively within the electronic survey portal by either a research assistant or administrative assistant 3 to 7 days before the surgical date.
After identifying all patients who underwent ACLR during the time period, the electronic survey portal (OBERD) was queried for the PRO order set assigned to all these patients. The resultant cohort of patients was then cross-referenced with the patients identified from the electronic health record to obtain a final cohort that underwent ACLR and were assigned the same order set. Inclusion criteria were patients who underwent ACLR, either primary or revision, at our institution and who were preoperatively assigned the PRO order set through the electronic survey portal. Exclusion criteria were any patients who were contacted outside of routine electronic portal emails for other research investigations.
All patients were automatically sent the PRO surveys via email through the electronic survey portal preoperatively and at 3 time points after surgery: 5 months, 11 months, and 23 months. For each of these postoperative time points, each survey stays open for 2 months, meaning that patients may log into the electronic survey portal at any time during this period and complete the surveys. These time points are designed to capture PRO data at approximately 6 months, 12 months, and 24 months postoperatively. Patients were considered to have completed preoperative and postoperative PRO surveys if they completed at least 1 knee survey: Lysholm, KOOS, or IKDC. The surveys are accessible for completion by computer, tablet, and mobile devices.
Data Collection
For each patient, we obtained demographic information (age, body mass index [BMI], race, ethnicity, sex, having traveled more than 100 miles from home address to surgery center, laterality, workers’ compensation, insurance status, chronicity [>3 months from date of injury to preoperative office visit], comorbidities [psychiatric diagnosis, heart disease, diabetes, immunocompromised], narcotic and nonsteroidal anti-inflammatory medication use, athlete or frequent recreational physical activity [>3 times/wk], level of athletic activity [recreational, competitive high school or middle school, collegiate, professional], smoking status, alcohol use, employment status, primary language, and having an active email address on file), operative data (primary or revision ACLR, type of graft used, concomitant procedures, and complications), and postoperative data (total number of office visits to our institution, number of office visits for ACLR, time to last follow-up visit, reoperation, if the patient returned to our institution for a different orthopaedic complaint, additional orthopaedic surgery at our institution on a different joint, postoperative PRO scores, and number of email reminders sent before PRO surveys were complete).
Statistical Analysis
Statistical analysis was performed using descriptive statistics, with chi-square or Fisher exact testing for categorical variables and Student t and Mann-Whitney U testing for comparison of parametric and nonparametric continuous variables, respectively. Patients were classified as completing PRO surveys both preoperatively and at least 1 postoperative time point, completing no surveys at all, completing both the preoperative survey and 24-month postoperative survey, or completing all surveys. Binomial logistic regression analysis was performed to evaluate any predictive associations between continuous variables and compliance with completing the preoperative survey and 24-month postoperative survey. All reported P values are 2-tailed, with an α level of .05 detecting significant differences (SPSS Statistics version 23.0; IBM).
Results
Of the 313 patients who underwent ACLR from December 2013 to March 2015 at our institution, 256 (81.8%) were assigned the knee PRO order set containing the 5 standardized surveys. All patients had an active email address on file. There were no patients contacted outside of standard time points for follow-up during this time period. Eighty-seven patients (34.0%) underwent a mean 1.32 ± 0.64 (range, 1-4) prior ipsilateral knee surgeries. Thirty-five patients (13.7%) had 1 comorbidity (hypertension, diabetes, psychiatric diagnosis, immunocompromised/malignancy). Complete demographic information is presented in Table 1.
TABLE 1.
Demographic Information of Cohort (N = 256)a
| Variable | Value |
|---|---|
| Age, mean ± SD, y | 28.6 ± 11.9 |
| Body mass index, mean ± SD, kg/m2 | 25.9 ± 5.1 |
| Race | |
| White | 190 (74.3) |
| African American | 17 (6.6) |
| Hispanic | 22 (8.6) |
| Asian | 18 (7.0) |
| Other | 9 (3.5) |
| Sex | |
| Male | 140 (54.7) |
| Female | 116 (45.3) |
| Insurance status | |
| Private/employer | 229 (89.5) |
| Medicare | 1 (0.4) |
| Medicaid | 7 (2.7) |
| Self-pay | 1 (0.4) |
| Traveled >100 miles for surgery | 22 (8.6) |
| Laterality | |
| Right | 123 (48.0) |
| Left | 133 (52.0) |
| Chronic tear | 83 (32.4) |
| Psychiatric diagnosis | 14 (5.5) |
| Taking NSAIDs for knee pain | 88 (34.3) |
| Taking narcotics for knee pain | 13 (5.1) |
| Athlete | 215 (84.0) |
| Level of athletic activity | |
| Recreational | 130 (60.5) |
| Competitive | 53 (24.7) |
| Collegiate | 27 (12.6) |
| Professional | 5 (2.3) |
| Employment status | |
| Employed | 127 (49.6) |
| Unemployed | 18 (7.0) |
| Student | 111 (43.4) |
aData are presented as n (%) unless otherwise indicated. NSAIDs, nonsteroidal anti-inflammatory drugs.
Operative and Postoperative Variables
Of the 256 ACLRs performed, 208 (81.3%) were primary reconstructions. Bone-tendon-bone grafts were most commonly utilized (n = 224; 87.5%), and 68.0% of all grafts were autografts. A majority (55.4%) underwent at least 1 concomitant procedure, with partial meniscectomy (73.6%) being the most common. There were 19 complications (7.4%), and 21 patients (8.2%) underwent reoperation on the ipsilateral knee. Seven (2.7%) patients were deemed failures based on diagnostic imaging evidence of a retear or revision ACLR.
Compliance With Completing PRO Surveys
Only 19 (7.4%) patients completed all preoperative and postoperative surveys at all time points. Less than half of the patients (n = 104; 40.6%) completed both the preoperative survey and at least 1 postoperative survey. Compliance with completing postoperative surveys steadily decreased over the 2-year time frame (Figure 1).
Figure 1.
Compliance percentages for the 4 standard-of-care time points for measuring patient-reported outcomes: preoperatively and 6 months, 12 months, and 24 months after index anterior cruciate ligament reconstruction.
This steady decrease in the completion rate over time was shown to be statistically significant (Pearson r = –0.995 [95% CI, –0.999 to –0.765], P = .005). A complete breakdown of survey completion is presented in Table 2.
TABLE 2.
Compliance With Completing PRO Surveysa
| Variable | n (%) |
|---|---|
| Completed preoperative survey | 159 (62.1) |
| Completed 6-month postoperative survey | 113 (44.1) |
| Completed 12-month postoperative survey | 76 (29.7) |
| Completed 24-month postoperative survey | 47 (18.4) |
| Completed preoperative survey only | 55 (21.5) |
| Completed preoperative survey and at least 1 postoperative survey | 104 (40.6) |
| Completed no surveys at all | 57 (22.3) |
| Completed all surveys | 19 (7.4) |
| Completed preoperative survey and 24-month postoperative survey | 32 (12.5) |
aPRO, patient-reported outcome.
Variables Affecting Survey Completion
Male patients were found to have significantly worse compliance with completing PRO surveys preoperatively and at a minimum of 1 time point postoperatively (P = .044). Male patients were also found to be more likely to not complete any surveys (P = .008); however, this sex difference approached but did not reach significance when comparing patients who completed both the preoperative survey and 24-month postoperative survey (P = .088). Patients who did not identify as athletes or as participating in frequent exercise (>3 times/wk) had significantly worse compliance with completing any PRO surveys (P = .046). However, the level of athletic activity had no significant influence on survey completion. On regression analysis, older age demonstrated greater odds of compliance with completing both the preoperative survey and 24-month postoperative survey, which approached but did not reach significance (odds ratio [OR], 1.028; P = .073). Lower BMI was associated with greater odds of compliance with completing the preoperative survey and 24-month postoperative survey (OR, 0.902; P = .029) (Table 3).
TABLE 3.
Binomial Logistic Regression Evaluating the Association Between Continuous Variables and Compliance With Completing PRO Surveysa
| Variable | Odds Ratio (95% CI) | P Value |
|---|---|---|
| Age | 1.028 (0.997-1.061) | .073 |
| Body mass index | 0.902 (0.822-0.989) | .029 |
| Comorbidities | 1.529 (0.660-3.891) | .373 |
| No. of previous ipsilateral knee surgeries | 1.134 (0.698-1.843) | .611 |
| Total No. of office visits at our clinic | 0.984 (0.857-1.131) | .896 |
| No. of office visits specific to ACLR | 1.089 (0.921-1.288) | .317 |
| Time to last office visit | 1.102 (0.574-2.117) | .771 |
aIncludes preoperative and 24-month patient-reported outcome (PRO) surveys. Bolded value indicates statistical significance (P < .05). ACLR, anterior cruciate ligament reconstruction.
There was no association between the preoperative scores for all 5 PRO surveys and compliance with completing PRO surveys postoperatively (P > .05 for all). There was also no association between survey completion and patient race, insurance status, traveling greater than 100 miles from home address for surgery, smoking status, alcohol use status, taking narcotics preoperatively, employment status, acute nature of ACL tear (<3 months), English as a primary language, having a psychiatric diagnosis, employment status, ACL graft type, or if index surgery was revision ACLR (Table 4).
TABLE 4.
Variables Associated With Compliance With Completing Patient-Reported Outcome Surveysa
| No Surveysb | Preoperative Survey and Any Postoperative Survey | Preoperative Survey and 24-Month Postoperative Survey | ||||
|---|---|---|---|---|---|---|
| Variable | % | P Value | % | P Value | % | P Value |
| Age, y | .640 | .436 | .394 | |||
| <20 | 16.4 | 45.6 | 11.4 | |||
| 20-29 | 26.3 | 32.9 | 7.9 | |||
| 30-39 | 22.4 | 42.9 | 14.3 | |||
| 40-49 | 25.6 | 43.6 | 17.9 | |||
| ≥50 | 23.1 | 46.2 | 23.1 | |||
| Sex | .008 | .044 | .088 | |||
| Male | 28.6 | 35.0 | 9.3 | |||
| Female | 14.7 | 47.4 | 16.4 | |||
| Race | .251 | .688 | .999 | |||
| White | 21.2 | 43.4 | 13.2 | |||
| African American | 41.2 | 35.3 | 0.0 | |||
| Hispanic | 9.1 | 36.3 | 18.2 | |||
| Asian | 27.8 | 27.8 | 5.6 | |||
| Other | 33.3 | 33.3 | 22.2 | |||
| Traveled >100 miles for surgery | 18.2 | .792 | 31.8 | .379 | 22.7 | .168 |
| Nature of tear | .635 | .728 | .143 | |||
| Acute tear | 30.1 | 40.0 | 10.4 | |||
| Chronic tear | 20.4 | 42.2 | 16.9 | |||
| Insurance status | .401 | .484 | .061 | |||
| Private/employer | 22.3 | 41.5 | 15.1 | |||
| Medicare | 100.0 | 0.0 | 0.0 | |||
| Medicaid | 14.3 | 42.9 | 0.0 | |||
| Self-pay | 0.0 | 100.0 | 100.0 | |||
| Workers’ compensation | 22.2 | 27.8 | 5.6 | |||
| Taking narcotics | 17.4 | .494 | 30.8 | .569 | 7.7 | .999 |
| Smoker | 3.5 | .734 | 2.9 | .365 | 6.3 | .564 |
| Athlete | 20.0 | .046 | 70.6 | .565 | 13.2 | .334 |
| Level of athletic activity | .413 | .904 | .115 | |||
| Recreational | 23.1 | 38.5 | 13.1 | |||
| Competitive | 16.7 | 44.4 | 3.7 | |||
| Collegiate | 14.8 | 44.4 | 14.8 | |||
| Professional | 33.3 | 50.0 | 33.3 | |||
| Psychiatric diagnosis | 14.2 | .741 | 57.1 | .196 | 28.6 | .082 |
| Employment status | .659 | .805 | .717 | |||
| Employed | 23.6 | 40.9 | 13.4 | |||
| Student | 43.4 | 41.4 | 10.8 | |||
| Unemployed | 27.8 | 33.3 | 16.7 | |||
| Revision ACLR (n = 48) | N/A | N/A | 45.8 | .415 | 12.5 | .999 |
| Concomitant procedures (n = 142) | N/A | N/A | 44.4 | .174 | 12.0 | .776 |
| ACL graft | .786 | .845 | .341 | |||
| BTB autograft | 20.4 | 41.5 | 9.5 | |||
| BTB allograft | 25.3 | 40.5 | 17.7 | |||
| Hamstring autograft | 24.1 | 37.9 | 13.8 | |||
| Achilles allograft | 0.0 | 0.0 | 0.0 | |||
| Complications (n = 19) | N/A | N/A | 42.1 | .891 | 10.5 | .999 |
| Failed ACLR (n = 7) | N/A | N/A | 28.6 | .704 | 0.0 | .601 |
aBolded values indicate statistical significance (P < .05). ACL, anterior cruciate ligament; ACLR, anterior cruciate ligament reconstruction; BTB, bone-tendon-bone; N/A, not applicable.
bN/A = Not calculated for revision ACLR, concomitant procedures, complications, or failed ACLR under the “no surveys” category, as these are all postoperative variables and thus should have no effect on the ability to complete preoperative surveys.
Discussion
The results from this study demonstrate that patients are not compliant with completing PRO surveys sent to them electronically, and there are several patient characteristics that are associated with even worse compliance. These include male sex, higher BMI, patients who do not frequently exercise (<3 times/wk), and patients who do not identify as athletes. Additionally, our study supports what many who participate in research have experienced, which is that there is a significant and steady decline in compliance with completing PRO surveys from baseline to 2 years after surgery. In contrast, other demographic variables, such as smoking status, alcohol use, employment status, insurance status, psychiatric diagnosis, primary language other than English, and concomitant procedures, were not correlated with compliance with completing PRO surveys. There were no other injury, operative, or postoperative variables found to be significantly associated with poorer compliance.
The logistical challenges of paper-based survey data entry and processing have resulted in considerable interest in the development of electronic survey administration.15,21,28 Electronic PRO surveys have several purported advantages, including ease of collection,7,27 more accurate and complete data,10,29 avoidance of collection errors,10 reduced administrative burden,7,8,12 increased patient compliance,8 smaller sample size requirements,10 and potential cost savings.14 In orthopaedics, studies have validated several PRO surveys for electronic administration.19 However, results from our study demonstrate that overall, patients are largely noncompliant with completing electronic PRO surveys.19 In this cohort of patients who underwent ACLR, only 7.4% of patients completed all surveys sent to them electronically, with 44.1% completing the preoperative survey and 24-month postoperative survey. Other studies have demonstrated decreased compliance with completing electronic PRO surveys. Palmen et al21 compared 3 methods of administration—traditional mail, telephone, and email—in 73 patients randomized to 1 of the 3 groups after hallux valgus surgery. The authors reported that email actually had the lowest completion rate at 33%, whereas traditional mail had the highest response rate at 88%. While 80% completed the PRO surveys via telephone, 34% completed the PRO surveys in the traditional mail group and a mere 24% in the email group. These completion rates are comparable with the one seen in the present study for the 6-month postoperative survey (44.1%), raising the question of whether electronic administration and contacting patients via email to complete PRO surveys are effective.
The administration of electronic PRO surveys via email poses a unique set of obstacles to effective administration and high patient compliance, such as a lack of internet access, outdated email addresses, use of spam/junk mail folders, ease of dismissal, impersonal nature, and risk of survey fatigue. Jenkins et al13 conducted an investigation in 2016 in which they surveyed 497 patients who presented to an orthopaedic outpatient clinic regarding their accessibility to the internet and preferred means for completing follow-up questionnaires. Interestingly, the authors found 72% of respondents to have internet access, and 40% indicated a preference for outcomes assessments to take place via email or the internet, while the remaining preferred a face-to-face evaluation or standard mail.13
While it is not surprising that face-to-face encounters with providers are preferred by patients, having these encounters >2 years out from routine surgery is not always indicated or feasible from a provider schedule and a patient finance/travel perspective. In addition, with the adoption of communication via email, physicians are contacting patients more frequently electronically to provide information to them as well as to request other details such as demographics, insurance status, and medical and social history. This issue is compounded by a fragmented health care system in which patients who have multiple medical comorbidities see multiple physicians, often in different health care systems, and are required to fill out surveys and questionnaires for each.
With the move toward outcome-based metrics for all aspects of health care economics, there will be a requirement for the data collection of outcomes beyond the academic environment. Our study suggests that patient compliance will be a major limitation in the move toward outcome-based health economics.
Whether administered in-office or electronically, few investigators have previously sought to evaluate patient characteristics associated with decreased completion rates of PRO surveys. Schamber et al26 reported lower completion rates of PRO surveys in patients over 75 years of age, of black or Hispanic ethnicity, with Medicaid or Medicare as their primary insurance, undergoing total knee arthroplasty compared with total hip arthroplasty, and undergoing revision joint surgery. Similarly, in a total joint arthroplasty registry study, Patel and colleagues23 observed lower completion rates of PRO surveys in older patients but also reported the presence of postoperative complications, medical comorbidities, discharge to a medical facility, and a high-volume surgeon as negatively associated with the completion of PRO surveys. However, these investigations were conducted in a much different demographic population than those who typically undergo ACLR. The results of the present study, in a comparatively younger, more active population, demonstrate that male sex, patients with a higher BMI, and those not identifying as athletes were all variables associated with decreased completion rates of PRO surveys. Contrary to our hypothesis, age was not correlated with poor compliance with completing electronic PRO surveys in this ACLR population. When taken in the context of the Schamber et al26 and Patel et al23 studies, this may suggest that age may only be a factor in compliance with completing PRO surveys for patients over a certain age. As our society continues to integrate electronic technologies into our lives, it is likely that this generational gap will decrease moving forward.
The significant findings of higher BMIs and lower activity levels in patients who do not complete PRO surveys may suggest a potential role for overall preoperative health status and health goals as variables affecting the reporting of PRO data. This may have an impact on the results of clinical outcome studies or on the outcome measures that are chosen for a given patient population. For example, if those who are more likely to complete PRO surveys at 2 years are more active, leaner patients, the outcomes may not be representative of the true population that underwent the procedure. It is difficult to know what the particular barrier(s) is for these patients who do not identify as athletes, have lower activity levels, or have higher BMIs, but it is likely multifactorial. Specifically, in the ACLR population, many of the PRO surveys are oriented to the athletic population, with many questions inquiring about high-level activities such as jumping or pivoting. Several questionnaires specifically inquire about the levels of sports activity in which the patient participates and specifically list sports such as basketball, soccer, skiing, tennis, and jogging. These PRO surveys may psychologically alienate those patients who do not participate in these activities regularly.
Further research is warranted to better understand the particular barriers present for specific patient populations in completing PRO surveys. Surgeons, however, should recognize that less active patients with higher BMIs are less likely to be compliant with completing their PRO surveys after surgery. Preoperative education about the importance of PRO surveys in this patient population may help improve compliance. Other methods of increasing compliance may include decreasing the number of surveys from 5 to just 1 or 2; however, this may not provide as well rounded a picture of how a patient feels that he or she is doing. Incentivizing patients, whether it be free parking at office visits or a small financial inducement, may be a way to improve compliance if a practice can afford to use this method.
The strengths of this study include the broad range of demographic, operative, and postoperative variables evaluated for effects on compliance with completing electronic PRO surveys. Furthermore, this is the first investigation to evaluate compliance with completing electronic PRO surveys for patients suffering from a single injury type, allowing for a more homogenous evaluation of injury and operative variables possibly influencing compliance with completing PRO surveys.
Limitations
Although the study patients were the sum of multiple different surgeons with different diversities of populations treated, they comprise a single-center study from an academic institution; thus, these results may not be generalizable to other regions of the country and populations and other diagnoses. The cohort of this study was a predominantly younger, healthier population, with relatively few patients having BMIs over 35 kg/m2. While this study is the first to evaluate the operative variables influencing the completion rates of postoperative PRO surveys for a single, specific sports medicine injury, the cohort size is significantly smaller than the 2 investigations performed in the arthroplasty literature.23,26 In addition, there was inconsistent reporting of patient education levels, occupational physical demands, and American Society of Anesthesiologists levels (surgeons practiced in multiple surgical centers with differing electronic health records, making the tracking of these data challenging if not in the operative notes), which may have resulted in variables associated with compliance with completing PRO surveys to not be analyzed. The timing of when the order set was initially assigned (between 3 and 7 days) may be a confounding factor for compliance with completing preoperative PRO surveys. More work needs to be done within sports medicine to identify variables that may be barriers to completing electronic PRO surveys and ways in which we can improve completion rates out to 2 years postoperatively and beyond.
Conclusion
This study found that overall compliance steadily declined the farther patients were out postoperatively, reaching woefully inadequate completion rates. Furthermore, patient-, injury-, operative-, and postoperative-specific variables associated with the completion of electronic PRO surveys in patients who underwent ACLR were identified. Specifically, male sex, not self-identifying as an athlete or participating in frequent recreational exercise, and higher BMI were all associated with lower completion rates of PRO surveys, demonstrating the need for new interventions to improve compliance and outcome reporting.
Footnotes
One or more of the authors has declared the following potential conflict of interest or source of funding: N.N.V. receives research support from Arthrex, Arthrosurface, Athletico, Conmed Linvatec, DJ Orthopedics, Miomed, Mitek, Ossur, and Smith & Nephew; receives publishing royalties from Arthroscopy and Vindico Medical–Orthopedics Hyperguide; has stock/stock options in CyMedica, Minivasive, and Omeros; is a paid consultant for Minivasive, OrthoSpace, and Smith & Nephew; and receives royalties from Smith & Nephew. B.J.C. receives research support from Aesculap/B. Braun, Arthrex, Medipost, and the National Institutes of Health (National Institute of Arthritis and Musculoskeletal and Skin Diseases and National Institute of Child Health and Human Development); receives royalties from Arthrex and DJ Orthopedics; is a paid consultant for Arthrex and Regentis; has stock/stock options in Carticept and Regentis; receives publishing royalties from Elsevier, Saunders/Mosby-Elsevier, and SLACK; and receives other financial support from Athletico, Ossur, Smith & Nephew, and Tornier.
Ethical approval for this study was obtained from Rush University Medical Center (No. 00000482).
References
- 1. American Journal of Sports Medicine. AJSM manuscript submission guidelines. Available at: http://www.sagepub.com/sites/default/files/upmbinaries/82010_AJSM426265_SubmissionGuidelines.pdf. Accessed May 22, 2017.
- 2. Black N, Burke L, Forrest CB, et al. Patient-reported outcomes: pathways to better health, better services, and better societies. Qual Life Res. 2016;25(5):1103–1112. [DOI] [PubMed] [Google Scholar]
- 3. Brodke DJ, Saltzman CL, Brodke DS. PROMIS for orthopaedic outcomes measurement. J Am Acad Orthop Surg. 2016;24(11):744–749. [DOI] [PubMed] [Google Scholar]
- 4. Budny J, Fox J, Rauh M, et al. Emerging trends in anterior cruciate ligament reconstruction. J Knee Surg. 2017;30(1):63–69. [DOI] [PubMed] [Google Scholar]
- 5. Buller LT, Best MJ, Baraga MG, et al. Trends in anterior cruciate ligament reconstruction in the United States. Orthop J Sports Med. 2014;3(1):23259 67114563664. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Cole BJ, Cotter EJ, Wang KC, et al. Patient understanding, expectations, outcomes, and satisfaction regarding anterior cruciate ligament injuries and surgical management. Arthroscopy. 2017;33(5):1092–1096. [DOI] [PubMed] [Google Scholar]
- 7. Coons SJ, Eremenco S, Lundy JJ, et al. Capturing patient-reported outcome (PRO) data electronically: the past, present, and promise of ePRO measurement in clinical trials. Patient. 2015;8(4):301–309. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Dale O, Hagen KB. Despite technical problems personal digital assistants outperform pen and paper when collecting patient diary data. J Clin Epidemiol. 2007;60(1):8–17. [DOI] [PubMed] [Google Scholar]
- 9. Garrett WE, Jr, Swiontkowski MF, Weinstein JN, et al. American Board of Orthopaedic Surgery Practice of the Orthopaedic Surgeon: Part-II, certification examination case mix. J Bone Joint Surg Am. 2006;88(3):660–667. [DOI] [PubMed] [Google Scholar]
- 10. Gasner AL, Raymond SA, Pearson JD. Data quality and power in clinical trials: a comparison of ePRO and paper in a randomized trial In: Byrom B, Tiplady B, eds. ePRO: Electronic Solutions for Patient-Reported Data. Farnham, UK: Gower; 2010:49–78. [Google Scholar]
- 11. Gilbert A, Sebag-Montefiore D, Davidson S, et al. Use of patient-reported outcomes to measure symptoms and health related quality of life in the clinic. Gynecol Oncol. 2015;136(3):429–439. [DOI] [PubMed] [Google Scholar]
- 12. Greenwood MC, Hakim AJ, Carson E, Doyle DV. Touch-screen computer systems in the rheumatology clinic offer a reliable and user-friendly means of collecting quality-of-life and outcome data from patients with rheumatoid arthritis. Rheumatology (Oxford). 2006;45(1):66–71. [DOI] [PubMed] [Google Scholar]
- 13. Jenkins PJ, Sng S, Brooksbank K, et al. Socioeconomic deprivation and age are barriers to the online collection of patient reported outcome measures in orthopaedic patients. Ann R Coll Surg Engl. 2016;98(1):40–44. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Jose N, Langel K. ePRO vs. paper. Available at: http://www.appliedclinicaltrialsonline.com/epro-vs-paper. Accessed October 23, 2017.
- 15. Kesterke N, Egeter J, Erhardt JB, et al. Patient-reported outcome assessment after total joint replacement: comparison of questionnaire completion times on paper and tablet computer. Arch Orthop Trauma Surg. 2015;135(7):935–941. [DOI] [PubMed] [Google Scholar]
- 16. Lavallee DC, Chenok KE, Love RM, et al. Incorporating patient-reported outcomes into health care to engage patients and enhance care. Health Aff (Millwood). 2016;35(4):575–582. [DOI] [PubMed] [Google Scholar]
- 17. Makhni EC, Padaki AS, Petridis PD, et al. High variability in outcome reporting patterns in high-impact ACL literature. J Bone Joint Surg Am. 2015;97(18):1529–1542. [DOI] [PubMed] [Google Scholar]
- 18. Malhotra K, Buraimoh O, Thornton J, et al. Electronic capture of patient-reported and clinician-reported outcome measures in an elective orthopaedic setting: a retrospective cohort analysis. BMJ Open. 2016;6(6):e01197 5-2016-011975. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Nguyen J, Marx R, Hidaka C, et al. Validation of electronic administration of knee surveys among ACL-injured patients. Knee Surg Sports Traumatol Arthrosc. 2017;25(10):3116–3122. [DOI] [PubMed] [Google Scholar]
- 20. Nwachukwu BU, Hamid KS, Bozic KJ. Measuring value in orthopaedic surgery. JBJS Rev. 2013;1(1):01874 474-201311000-00003. [DOI] [PubMed] [Google Scholar]
- 21. Palmen LN, Schrier JC, Scholten R, et al. Is it too early to move to full electronic PROM data collection? A randomized controlled trial comparing PROM’s after hallux valgus captured by e-mail, traditional mail and telephone. Foot Ankle Surg. 2016;22(1):46–49. [DOI] [PubMed] [Google Scholar]
- 22. Patel AA. Patient-reported outcome measures: the promise of PROMIS. J Am Acad Orthop Surg. 2016;24(11):743. [DOI] [PubMed] [Google Scholar]
- 23. Patel J, Lee JH, Li Z, et al. Predictors of low patient-reported outcomes response rates in the California Joint Replacement Registry. J Arthroplasty. 2015;30(12):2071–2075. [DOI] [PubMed] [Google Scholar]
- 24. Roberts N, Bradley B, Williams D. Use of SMS and tablet computer improves the electronic collection of elective orthopaedic patient reported outcome measures. Ann R Coll Surg Engl. 2014;96(5):348–351. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Sanders TL, Maradit Kremers H, Bryan AJ, et al. Incidence of anterior cruciate ligament tears and reconstruction: a 21-year population-based study. Am J Sports Med. 2016;44(6):1502–1507. [DOI] [PubMed] [Google Scholar]
- 26. Schamber EM, Takemoto SK, Chenok KE, et al. Barriers to completion of patient reported outcome measures. J Arthroplasty. 2013;28(9):1449–1453. [DOI] [PubMed] [Google Scholar]
- 27. Shiffman S. Two decades of change for PROs. Available at: http://www.appliedclinicaltrialsonline.com/two-decades-change-pros. Accessed October 23, 2017.
- 28. Smith MJ, Reiter MJ, Crist BD, et al. Improving patient satisfaction through computer-based questionnaires. Orthopedics. 2016;39(1):e31–e35. [DOI] [PubMed] [Google Scholar]
- 29. Stone AA, Shiffman S, Schwartz JE, et al. Patient non-compliance with paper diaries. BMJ. 2002;324(7347):1193–1194. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Tibor L, Chan PH, Funahashi TT, et al. Surgical technique trends in primary ACL reconstruction from 2007 to 2014. J Bone Joint Surg Am. 2016;98(13):1079–1089. [DOI] [PubMed] [Google Scholar]
- 31. Van Der Wees PJ, Nijhuis-Van Der Sanden MW, Ayanian JZ, et al. Integrating the use of patient-reported outcomes for both clinical practice and performance measurement: views of experts from 3 countries. Milbank Q. 2014;92(4):754–775. [DOI] [PMC free article] [PubMed] [Google Scholar]