Abstract
Background
In clinical research, minimizing patients lost to follow-up is essential for data validity. Researchers can employ better methodology to prevent patient loss. We examined how orthopedic surgery patients’ contact information changes over time to optimize data collection for long-term outcomes research.
Methods
Patients presenting to orthopedic outpatient clinics completed questionnaires regarding methods of contact: home phone, cell phone, mailing address, and e-mail address. They reported currently available methods of contact, if they changed in the past five and ten years, and when they changed. Differences in the rates of change amongst methods were assessed via Fisher exact tests. Whether participants changed any of their contact information in the past five and ten years was determined via multivariate modeling, controlling for demographic variables.
Results
Among 152 patients, 51% changed at least one form of contact information within 5 years, and 66% changed at least one form within 10 years. The rate of change for each contact method was similar over five (15-28%) and ten years (26-41%). One patient changed all 4 methods of contact within the past five years, and seven within the past ten years. Females and younger patients were more likely to change some type of contact information.
Conclusions
The type of contact information least likely to change over five to ten years is influenced by demographic factors such as sex and age, with females and younger participants more likely to change some aspect of their contact information. Collecting all contact methods appears necessary to minimize patients lost to follow-up, especially as technological norms evolve.
Keywords: Clinical research, follow-up, research methods, patient recruitment, longitudinal data, data collection
Introduction
To assess the effectiveness of orthopedic treatments surgeons require long-term outcome data. Minimizing the number of patients lost to follow-up is essential to preserve data validity; yet, this remains a challenge. Unlike clinical trials where patients must return to receive treatment, observational surgical trials are especially prone to data loss as treatment concludes prior to final data collection. Limiting dropout to less than 5% to prevent bias is advocated,1 while 80% retention is required for study classification as level 1 evidence.2, 3 Even with less than 20% dropout, studies’ results can be biased by non-random dropout.4 Furthermore, any dropout can negatively impact studies’ power.5 Data missing completely at random or at random may be accounted for via imputation, but proving that patients were truly lost randomly is nearly impossible and introduces critique to any study. Acceptable study retentions rates have proven to be difficult to achieve in some orthopedic scenarios, including that of perilunate fractures/dislocations, where long term follow-up only included one-third of eligible patients.6
Published methods of retaining patients include obtaining detailed demographic information, contacting patients quarterly, and excluding patients unlikely to comply with follow-up.7-12 Alternatively, proprietary services offer to locate patients lost to follow-up but incur costs up to $50,000.13, 14 Caution is recommended when considering some of these options,15 as they may compromise data validity or cross boundaries of ethical appropriateness.16
A simpler option to attempt to prevent missing data is to obtain exhaustive patient contact information. Yet, there is a balance required to collect sufficient data without introducing study fatigue to participants. Patients’ response rates to mail, e-mail, and phone solicitations varies providing little guidance,17 and in our own institution we historically only collected one phone number and address to be linked with our billing information. Additionally, the ideal, lasting contact point is unclear, and the current cultural shift away from traditional communication methods, such as having a home telephone, challenges traditional concepts. If we know at the time of enrollment what patient contact information is most and least likely to change during the duration of the study more accurate listings can be achieved, with increased efficiency and confidence, which can hopefully lead to increased retention rates.
The purpose of this study was to determine how orthopedic surgery patients’ contact information changed over time, and to determine what impact self-reported patient demographic information had on these changes. We sought to determine the stability of four specific types of contact information: home phone, cell phone, mailing address, and e-mail address. We tested the null hypothesis that there would be no difference over time in the rate of change for each mode of contact.
Materials and methods
We obtained institutional review board approval prior to conducting this cross-sectional study. Written informed consent was obtained from all patients. All patients over 18 years of age were eligible for the study. Patients unable to consent for study participation, lacking English proficiency, and active duty military personnel (and their families) were excluded from the study. Individuals in the military (n=8) were excluded based on their unique increased propensity for relocation for reasons beyond their control, which is not representative of the larger civilian patient population. As an exploratory analysis, a sample size of 150 patients was our goal. 152 patients were consecutively enrolled over a three-week period in four orthopedic surgeons’ outpatient clinics. No patients declined to participate.
After consent, one of three investigators read the standardized, 29-question survey to participants and recorded their answers. We queried each patient’s age, sex, race, and education level, and we obtained more detailed information with six questions for each type of contact information. These questions included “do you have [contact method x],” “has [contact method x] changed in the past ten years, and if so how many times,” “has [contact method x] changed in the past five years, and if so how many times,” and, “when is the last year [contact method x] changed.”
Data analysis
Descriptive statistics were produced to characterize the participants. Medians and ranges were used to report the distributions of numeric data. Percentages of the sample who had each type of contact information, and the percentage of participants that changed any and all contact information were determined. Fisher’s exact tests analyzed for differences in the frequency of contact information possession and changes in contact methods both between groups and among groups based on age, sex, race, and education level. Logistic regression models, both univariate and multivariate, were used to calculate odds ratios for differences in the frequency of changing a contact modality at five and ten years for each of the demographic groups. Statistical significance was dependent on p values <0.05 during exact testing or non-overlapping 95% confidence intervals for odds ratio determinations, with Bonferonni corrections performed for multiple comparisons.
Results
Table 1 provides descriptive data of the participants. Table 2 shows the percentage of the sample that had active forms of each type of contact information and statistical comparisons between the different modes of contact. Fifty one percent of participants changed at least one form of their contact information – home phone, cell phone, mailing address, or e-mail address – in the last five years. One participant (<1%) changed all of his/her contact information in the past five years. In the past ten years, 66% of participants changed at least one form of their contact information. Seven participants (5%) changed all forms of contact in the past ten years.
Table 1.
Demographic breakdown and descriptive statistics of the study participants.
| Variable | Number (%) |
|---|---|
| Male | 61 (40.1%) |
| Age (Mean ±SD) | 52.8 (±14.8) |
| 18-30 | 13 (8.5%) |
| 31-65 | 112 (73.7%) |
| >65 | 37 (17.8%) |
| Race | |
| White | 118 (77.6%) |
| Black/African-American/Other | 34 (22.4%) |
| Education Level | |
| High school graduate or less/GED | 40 (26.3%) |
| Some college/associate degree | 34 (22.4%) |
| College graduate | 45 (29.6%) |
| Professional degree | 33 (21.7%) |
| Expected Payor | |
| Self-pay | 1 (0.7%) |
| Medicare | 35 (23.0%) |
| Worker’s Compensation | 13 (8.55%) |
| Tricare | 13 (8.55%) |
| Private Insurance | 90 (59.2%) |
Table 2.
Number and percentage of sample with active forms of contact modalities.
| Contact Modality | N (%) |
|---|---|
| Home Phone | 108/152 (71%) |
| Cell Phone | 147/152 (97%)* |
| Mailing Address | 152/152 (100%)† |
| E-mail Address | 128/152 (84%)‡ |
: Statistically significantly greater proportion than home phone and e-mail address (p<0.05)
: Statistically significantly greater proportion than home phone (p<0.05)
: Statistical significance could not be assessed due to 100% proportion
The proportion of participants who changed their cell phone versus home phone versus e-mail address versus mailing address did not significantly differ from one another at either the five- or ten-year time point, respectively (Table 3). Whether a participant changed any of their contact information in the past five years was influenced by participant age and race. Each additional year in age reduced the odds of a participant changing some of their contact information by 2.2% (OR: 0.978, 95% CI: 0.967-0.989), and non-white participants were 2.40 times more likely to change some of their contact information (OR: 2.40, 95% CI: 1.07-5.35). However, when these two variables were factored in with sex and educational level, only age remained predictive (Table 4).
Table 3.
Rates of change for each method of patient contact.
| Contact Modality |
Rate of change at 5 years (95% CI) |
Rate of change at 10 years (95% CI) |
Median number of changes (range)* |
Median years since most recent change (range)* |
|---|---|---|---|---|
| Home phone |
14.8% (9.3-22.7%) | 25.9% (18.6-34.9%) | 1 (1-7) | 3 (0-8) |
| Cell phone |
16.9% (11.7-23.7%) | 28.4% (21.7-36.1%) | 1 (1-20) | 4 (0-8) |
| Mailing address |
28.3% (21.7-35.9%) | 41.4% (33.9-49.4%) | 1 (1-8) | 3 (0-8) |
| E-mail address |
24.2% (17.6-32.3%) | 33.6% (26.0-42.1%) | 1 (1-4) | 3.5 (0-10) |
Median over 10 years
Table 4.
Full multivariate model details on contact information changes at five and ten years.
| Variables | Odds Ratio | 95% Confidence Interval |
|---|---|---|
|
| ||
| Sex (female) | 1.67 | 0.83-3.32 |
| Age | 0.98 | 0.95-0.99 |
| Race (white) | 0.48 | 0.21-1.12 |
| Education (some college)* | 0.66 | 0.25-1.74 |
| Education (college degree)* | 0.65 | 0.23-1.78 |
| Education (graduate school)* | 0.64 | 0.24-1.67 |
| Variables | Odds Ratio | 95% Confidence Interval |
|---|---|---|
|
| ||
| Sex (female) | 2.55 | 1.22-5.32 |
| Age | 0.97 | 0.94-0.99 |
| Race (white) | 0.66 | 0.26-1.66 |
| Education (some college)* | 0.48 | 0.17-1.37 |
| Education (college degree)* | 0.76 | 0.24-2.37 |
| Education (graduate school)* | 0.53 | 0.18-1.54 |
: Compared to a high school or less education
Over the past ten years, female participants were more likely to change some aspect of their contact information compared to male participants (OR: 2.2, 95% CI: 1.1-4.4), and each additional year in age reduced the odds of a participant changing some of their contact information by 2.9% (OR: 0.971, 95% CI: 0.959-0.983). Independently, race and education level did not influence the frequency of change of some form of contact information at the ten year time point (p>0.05). Unlike the five year data, when all four demographic variables were assessed in a multivariate fashion, age and sex remained significant predictors of change with all other variables held equal (Table 4), with females being 2.55 times more likely to change (OR: 2.55, 95% CI: 1.22-5.32) and each year in age reducing the odds of change by 3.0% (OR: 0.97, 95% CI: 0.94-0.99). Education level and race were not significant predictors (both p>0.05).
Discussion
Clinical outcomes require assessment years after delivery of care to assess the effectiveness of orthopedic surgery treatments. To obtain such data researchers must be able to reach patients years after follow-up may have ceased. Our results demonstrate that the key in collecting contact information for clinical research is variety. If all four methods of contact information that we queried were collected, in this sample, only one patient would have been lost to follow-up at five years. While over a ten-year period the majority of patients will change some aspect of their contact information, only 5% of patients will change all their contact information.
Furthermore, there is not one type of contact information that is least likely to change for all participants, as the rates of change are influenced by sex and age, with females and younger participants more likely to change some aspect of their contact information. We hypothesize that females change their information more frequently secondary to name changes associated with marriage, and that younger participants change more frequently due to life changes that occur from 18-30 (becoming independent, new jobs, finishing school). Therefore, we believe it is imperative when conducting a long-term outcome study to collect a variety of patient contact information. While having all forms of contact information does not guarantee success in follow-up with a patient, having multiple methods to try will increase the odds of success.
Our study has several limitations. Only patients from orthopedic surgery clinics at a single, urban, academic medical center were surveyed, which could limit our ability to generalize our results. However, our clinics are representative of all patient types (i.e., traumatic conditions, elective conditions, all payor types), and we believe our patient mix is representative of patients enrolled in prospective, observational, patient-reported outcome studies. Second, since this was a cross-sectional survey, the potential for recall bias exists. Third, we are limited by the timeframe of our questions. We arbitrarily decided on the five- and ten-year time points, and thus cannot comment on how our results would differ for longer follow-up studies.
Nota et al. assessed whether a specific method of contact with patients would increase the likelihood of a response to a study survey.17 They found that phone calls provided higher response rates than e-mail or mail, and that younger patients were more likely to not respond. Yet, this was only assessed over 3 months. Our study builds on their data and demonstrates for study periods over five years, 24% of participants changed their phone number. Together, our studies demonstrate that collecting multiple forms of contact information is likely optimal as no single method ensures perfect responsiveness or stability over time.
A simple solution to finding patients would be a national medical identifier like those employed in other nations, but without such a resource studies have demonstrated how to improve data collection when patients are lost to follow-up -- with limitations. Sprague et al. described a variety of methods to limit their loss of participants at follow-up visits extending to one year after surgery.11 One method they employed was not including patients that they deemed unlikely to comply with follow-up instruction, which led to the exclusion of 144 patients, which was 15.8% of their patient population that met inclusion criteria. When this is combined with their actual dropout of 28 patients, for a total of 172 patients from a study group of 909, they approached the 20% dropout cut-off for being deemed a level 1 study. Additionally, by identifying a cohort of patients as being unlikely to comply with follow-up, investigators are introducing a non-random bias in their study population, which may affect the validity of the conclusions that the researchers reach.
Other authors discuss a variety of methods that researchers can employ in an attempt to locate patients that are lost to follow-up. Weinberg et al. discuss their method of locating clinical research participants up to 27 years after enrollment in a variety of studies.14 While this method proved to be quite successful, as they were able to find 84% of the 708 participants, there were a large number of costs – both in terms of time and dollars spent – to implement the protocol. They spent 603 hours and over $53,000 to achieve their results. Similarly, Louie et al. discuss a much cheaper (only $16 sans personnel costs) but also time intensive method (approximately 100 hours); these investigators were able to locate 74% of patients twelve years after surgery.13 In both instances, these methods add substantial time and cost to conducting a long-term follow-up study. Additionally, there can be ethical concerns with trying to retrospectively find patients’ contact information, and in some instances methods are limited by institutional review boards.16 Our data demonstrate that by being proactive in collecting contact information, researchers may be able to avoid such protocols.
Patterns of communication are changing and will continue to evolve, and it is important that researchers adapt. For example, the ability to send outcome surveys to patients securely via e-mail through database services such as REDCap provides an improved ability to communication with study participants.18 Yet, the uncertainty of what the future may hold is a challenge. Will research studies more commonly create Facebook, Twitter, or other social media pages to better connect with their participants?19-21 Moreover, what future technology that has yet to be discovered or widely implemented will completely change the research or communication process? These are all questions that researchers must consider when designing future studies.
Conclusions
One unexpected challenge in conducting a long-term follow-up study is the influence of disruptive technology and changes in habits of communication. This is highlighted by the propensity for younger patients to change their contact information. To overcome these cultural shifts our data support the collection of all contact information and vigilance by researchers to maintain high study retention rates in long-term follow-up studies.
Acknowledgments
Funding
This work was supported by a grant from the Doris Duke Charitable Foundation to Washington University in St. Louis to fund Doris Duke Clinical Research Fellow D.A.L.
This publication was supported by the Washington University in St. Louis Institute of Clinical and Translational Sciences grant UL1 TR000448, sub-award TL1 TR000449, from the National Center for Advancing Translational Sciences. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.
Neither funding source played any role in study design, data collection, or in manuscript preparation.
References
- 1.Schulz KF, Grimes DA. Sample size slippages in randomised trials: exclusions and the lost and wayward. Lancet. 2002;359:781–785. doi: 10.1016/S0140-6736(02)07882-0. [DOI] [PubMed] [Google Scholar]
- 2.Rudicel S, Esdaile J. The randomized clinical-trial in orthopedics -- obligation or option? J Bone Joint Surg Am. 1985;67A:1284–1293. [PubMed] [Google Scholar]
- 3.Wright JG. A practical guide to assigning levels of evidence. J Bone Joint Surg Am. 2007;89:1128–1130. doi: 10.2106/JBJS.F.01380. [DOI] [PubMed] [Google Scholar]
- 4.Overall JE, Shobaki G, Shivakumar C, et al. Adjusting sample size for anticipated dropouts in clinical trials. Psychopharmacol Bull. 1998;34:25–33. [PubMed] [Google Scholar]
- 5.SPRINT Investigators (Sample) size matters An examination of sample size from the SPRINT trial study to prospectively evaluate reamed intramedullary nails in patients with tibial fractures. J Orthop Trauma. 2013;27:183–188. doi: 10.1097/BOT.0b013e3182647e0e. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Forli A, Courvoisier A, Wimsey S, et al. Perilunate dislocations and transscaphoid perilunate fracture-dislocations: a retrospective study with minimum ten-year follow-up. J Hand Surg Am. 2010;35:62–68. doi: 10.1016/j.jhsa.2009.09.003. [DOI] [PubMed] [Google Scholar]
- 7.Losina E, Wright J, Katz JN. Clinical trials in orthopaedics research. Part III. Overcoming operational challenges in the design and conduct of randomized clinical trials in orthopaedic surgery. J Bone Joint Surg Am. 2012;94:e35. doi: 10.2106/JBJS.K.00009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Scharfstein DO, Hogan J, Herman A. On the prevention and analysis of missing data in randomized clinical trials: the state of the art. J Bone Joint Surg Am. 2012;94(Suppl 1):80–84. doi: 10.2106/JBJS.L.00273. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Sibai T, Carlisle H, Tornetta P., 3rd The darker side of randomized trials: recruitment challenges. J Bone Joint Surg Am. 2012;94(Suppl 1):49–55. doi: 10.2106/JBJS.L.00240. [DOI] [PubMed] [Google Scholar]
- 10.Slobogean GP, Sprague S, Bhandari M. The tactics of large randomized trials. J Bone Joint Surg Am. 2012;94(Suppl 1):19–23. doi: 10.2106/JBJS.L.00189. [DOI] [PubMed] [Google Scholar]
- 11.Sprague S, Leece P, Bhandari M, et al. Limiting loss to follow-up in a multicenter randomized trial in orthopedic surgery. Control Clin Trials. 2003;24:719–725. doi: 10.1016/j.cct.2003.08.012. [DOI] [PubMed] [Google Scholar]
- 12.Swiontkowski MF, Agel J. Lessons learned from benchmark orthopaedic trials. J Bone Joint Surg Am. 2012;94(Suppl 1):11–14. doi: 10.2106/JBJS.K.01622. [DOI] [PubMed] [Google Scholar]
- 13.Louie DL, Earp BE, Blazar PE. Finding orthopedic patients lost to follow-up for long-term outcomes research using the internet: an update for 2012. Orthopedics. 2012;35:595–599. doi: 10.3928/01477447-20120621-06. [DOI] [PubMed] [Google Scholar]
- 14.Weinberger M, Weinberger MH, Fineberg N, et al. Long-term follow-up of participants in clinical studies. J Clin Epidemiol. 2002;55:230–234. doi: 10.1016/s0895-4356(01)00437-1. [DOI] [PubMed] [Google Scholar]
- 15.Smith JS, Watts HG. Methods for locating missing patients for the purpose of long-term clinical studies. J Bone Joint Surg Am. 1998;80:431–438. doi: 10.2106/00004623-199803000-00018. [DOI] [PubMed] [Google Scholar]
- 16.Swiontkowski MF, Smith JS, Watts HG. Correspondence. J Bone Joint Surg Am. 1999;81:142. [PubMed] [Google Scholar]
- 17.Nota SFT, Strooker J, Ring D. Differences in response rates between mail, e-mail, and telephone follow-up in hand surgery research. Hand. 2014;9:504–510. doi: 10.1007/s11552-014-9618-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Harris PA, Taylor R, Thielke R, et al. Research electronic data capture (REDCap)-A metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42:377–381. doi: 10.1016/j.jbi.2008.08.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Khatri C, Chapman SJ, Glasbey J, et al. Social media and internet driven study recruitment: evaluating a new model for promoting collaborator engagement and participation. Plos One. 2015;10:e0118899. doi: 10.1371/journal.pone.0118899. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.O'Connor A, Jackson L, Goldsmith L, et al. Can I get a retweet please? Health research recruitment and the Twittersphere. J Adv Nurs. 2014;70:599–609. doi: 10.1111/jan.12222. [DOI] [PubMed] [Google Scholar]
- 21.Shere M, Zhao XY, Koren G. The role of social media in recruiting for clinical trials in pregnancy. Plos One. 2014;9:e92744. doi: 10.1371/journal.pone.0092744. [DOI] [PMC free article] [PubMed] [Google Scholar]