Abstract
Background
Patient-reported outcome measures (PROMs) have become increasingly important in assessing clinical outcomes. However, acquisition of data at routine time intervals can be challenging. The ability of e-mail reminders to save follow-up intervals when office visits are missed is unknown.
Materials and Methods
A retrospective review of a consecutive series of 186 shoulder surgical patients who underwent surgery between October 2, 2012, and July 2, 2013, was conducted. Simple Shoulder Test and 12-Item Short Form Health Survey scores were completed at preoperative visits using office-based tablet surveys. Patients were observed for completeness of PROMs at expected routine follow-up of 1 year and 2 years. When office visits were missed, e-mail reminders with links to online surveys were sent to patients without further incentives. Improvement in data acquisition achieved using e-mail reminders when patient follow-up was missed was assessed. The influence of the procedure performed was further analyzed to determine whether patients treated with different surgical procedures would be more compliant with PROM completion.
Results
Use of e-mail reminders significantly increased the number of patients for whom complete follow-up data were obtained. Compared with tablet surveys completed during office visits alone, the addition of e-mail reminders increased the collection of complete PROM data (both 1- and 2-year follow-up) by 25.8% (P < .001). Similar findings were observed for total shoulder arthroplasty and arthroscopic rotator cuff repair patients (increased by 25.7% and 34.4%, respectively; P < .001).
Conclusion
E-mail reminders serve as a mechanism to increase the completeness of follow-up data in the absence of in-office patient evaluation.
Keywords: Data collection, Patient-reported outcome measures, Tablet, Electronic surveys, Shoulder outcome assessments, E-mail surveys, E-mail reminder
Patient-reported outcome measures (PROMs) are becoming more important in health care in the assessment of clinical outcomes, quality, and value. Tracking meaningful patient outcomes is an essential method of evaluating the benefits of treatment.2, 4, 12 Whereas objective measures are important in terms of function, they do not necessarily correlate with the patient's satisfaction10; PROMs have helped to bridge this disconnect between subjective and objective findings. With the introduction of these PROMs, health care providers have been able to track changes over time to determine which interventions produce the best outcomes. With the advent of computer software created to track outcomes measurements, the use of electronically based surveys has become a more preferable questionnaire format than previous pen-and-paper surveys.5, 8, 14 Nevertheless, the ability to compile complete data at routine follow-up intervals can be difficult. Several factors, such as missed appointments, cost of care for well-visit checks, and ability of the patient to complete the PROMs during the office visit, all influence the data procurement.
An e-mail reminder with links to online-based surveys may minimize lost data. It allows procurement of data when appointments are missed, affords patients the ability to complete the surveys at their own pace from the comfort of their own home, and minimizes confounding influences from completing the surveys in the office setting. Most software platforms have options to allow e-mail reminders. However, no study to our knowledge has investigated the role of e-mail reminders in improving acquisition of complete PROM data sets.
The purpose of this study was to evaluate the ability of e-mail reminders to improve complete data sets for the Simple Shoulder Test (SST) and 12-Item Short Form Health Survey questionnaires after elective shoulder surgery. We hypothesize that the use of e-mail reminders will result in an increase in PROM complete data sets at expected time intervals.
Materials and methods
The institution's Shoulder and Elbow Outcome Repository was queried for a consecutive series of patients who underwent elective shoulder surgery during a 9-month period (October 2012 to July 2013). The Repository initiated the use of office-based tablets (iPad; Apple, Cupertino, CA, USA) beginning in October 2012. As a part of routine preoperative, 1-year, and 2-year visits, each patient completes the 12-Item Short Form Health Survey13 and SST7 using office-based tablets. Data are stored within the Repository using the CareSense database (MedTrak, Conshohocken, PA, USA). Only patients with preoperative survey data were included in this analysis. All patients were requested to return for routine office visits at both 1 year and 2 years after the index surgical procedure, with no further incentive for follow-up. At each office visit, tablets were used to collect PROM surveys.
The office-based tablet PROMs provide a user-friendly, touch screen, tablet-based interface that mandates full completion of each survey before the patient proceeds to the next set of questions. Any survey abandoned midway is deemed incomplete and not stored within the Repository. Included in the tablet survey is an option to be contacted by e-mail for future PROMs in the event that an office visit is missed. The patient maintains the right to not participate in e-mail reminders. When a patient has missed a routine office follow-up, an automated e-mail is generated with a secure link to complete the PROMs surveys using a Health Insurance Portability and Accountability Act–protected web-based portal.
Two groups were examined. The tablet-only group consisted of patients who completed tablet-based surveys in the office during the planned follow-up visit. Those patients who completed the survey online but subsequently scheduled their missed appointment were included in the tablet-only group. The e-mail reminder group consisted of patients who completed surveys using the online portal following an e-mail reminder after a missed office visit combined with the tablet-only group.
Improvement in survey completion rates through the use of e-mail reminders was evaluated at 1-year and 2-year follow-up to help define the role that e-mail reminders play in improving complete data sets when office visits are missed. The effectiveness of e-mail reminders was calculated from the difference in the survey completion rates between the e-mail reminder and tablet-only groups.
Completion rate analysis was further subdivided on the basis of the surgical procedure performed to examine the influence of type of procedure on compliance with PROM surveys. The 3 most common shoulder procedures performed during that time period were selected for analysis: 23472, total shoulder arthroplasty (TSA); 29827, arthroscopic rotator cuff repair; and 29828, arthroscopic biceps tenodesis.
Statistical analysis
Descriptive statistics were tabulated for all variables. χ2 tests were performed to evaluate the response rates of surveys completed by tablet device during a clinic visit alone in comparison to surveys completed by a tablet device together with surveys completed after e-mail reminders. Response rates were also examined after stratification by surgical procedure. Fisher exact tests were used to compare rates among small sample sizes, when appropriate. Data were analyzed using SPSS software version 23 (IBM, Armonk, NY, USA). All statistical tests were 2 tailed, and P ≤ .05 was considered significant.
Results
Among all patients, the use of e-mail reminders significantly increased the number of patients for whom follow-up data were obtained.
As noted in Table I, the e-mail reminder group was found to have a 25.8% increase in survey completion rate for a complete set of data (preoperative, 1-year postoperative, and 2-year postoperative follow-up). Complete 1- and 2-year postoperative data sets were available for 40.9% of the e-mail reminder group compared with only 15.1% of the tablet-only group (P < .001; Table I).
Table I.
Preoperative and postoperative survey completion rates
| Tablet-only group, n (%) | E-mail reminder group, n (%) | Increase in response with e-mail survey, % | P | |
|---|---|---|---|---|
| All procedures | ||||
| Preoperative | 186(100) | — | — | — |
| 1 year postoperative | 88(47.3) | 119(64.0) | 16.7 | .001* |
| 2 years postoperative | 37(19.9) | 91(48.9) | 29.0 | <.001* |
| 1 and 2 years postoperative | 28(15.1) | 76(40.9) | 25.8 | <.001* |
| Total shoulder arthroplasty | ||||
| Preoperative | 70(100) | — | — | — |
| 1 year postoperative | 45(64.3) | 55(78.6) | 14.3 | .061* |
| 2 years postoperative | 27(38.6) | 45(64.3) | 25.7 | .002* |
| 1 and 2 years postoperative | 24(34.3) | 42(60.0) | 25.7 | .002* |
| Shoulder arthroscopy with rotator cuff repair | ||||
| Preoperative | 61(100) | — | — | — |
| 1 year postoperative | 26(42.6) | 40(65.6) | 23.0 | .011* |
| 2 years postoperative | 6(9.8) | 28(45.9) | 36.1 | <.001* |
| 1 and 2 years postoperative | 2(3.3) | 23(37.7) | 34.4 | <.001† |
| Shoulder arthroscopy with biceps tenodesis | ||||
| Preoperative | 13(100) | — | — | — |
| 1 year postoperative | 7(53.8) | 10(76.9) | 23.1 | .411† |
| 2 years postoperative | 1(7.7) | 7(53.8) | 46.1 | .030† |
| 1 and 2 years postoperative | 0(0) | 6(46.2) | 46.2 | .015† |
Tablet-only group represents completed survey during office visits; e-mail reminder group includes all surveys in the tablet-only group together with those surveys completed using the online portal after e-mail reminders.
χ2 test.
Fisher exact test.
Using the 2-year data as the most recent follow-up data point, the e-mail reminder group increased the survey completion rate by 29% for all surgical procedures (P < .001; Table I).
Stratification by surgical procedure revealed that patients who underwent TSA had higher tablet-only completion rate (64.3%) than patients who underwent rotator cuff repair (42.6%) or arthroscopic biceps tenodesis (53.8%). For TSA patients, e-mail reminders increased the 1-year follow-up percentage by 14.3% and 2-year follow-up percentage by 25.7%. E-mail reminders increased the rate of complete data sets (both 1- and 2-year follow-up) from 34% in the tablet-only group to 60% in the e-mail reminder group (P = .002; Table I).
Among patients who had an arthroscopic rotator cuff repair, e-mail surveys significantly increased patient outcome response rates by 23% at 1 year postoperatively (P = .011) and 36% at 2 years postoperatively (P < .001). Complete 1- and 2-year postoperative data were obtained from 38% of patients in the e-mail reminder group compared with only 3% of the tablet-only group (P < .001; Table I).
Among patients who had an arthroscopic biceps tenodesis, e-mail surveys increased patient outcome response rates by 23% at 1 year postoperatively (P = .411) and 46% at 2 years postoperatively (P = .030). Complete 1- and 2-year postoperative data were obtained from 46% of patients in the e-mail reminder group compared with 0% from the tablet-only group (P = .015; Table I).
Discussion
The use of e-mail reminders with a link to web-based portals significantly increased the number of patients for whom follow-up data were obtained. Compared with office-based tablet surveys alone (tablet-only group), e-mail reminders significantly increased the overall collection of complete PROM data sets by 26%. These results suggest that the supplementation of e-mail reminders to an orthopedic practice may help minimize lost data. This may aid in providing a more complete and comprehensive data set for research and clinical purposes.
Electronic and web-based PROMs are appealing in the advancing technologic era. They offer several theoretical advantages, including a simple workflow for data collection,8, 14 a greater access to pertinent patient information, and the opportunity to incorporate that data into clinical care with real-time availability of data.6, 9, 11 Furthermore, it has been reported that electronically administered PROMs yield a 14-fold probability of obtaining a completed and thus a score-able survey.3 The use of iPad in data collection has been shown to detect 67% of omissions seen on the pen-and-paper counterparts.6
E-mail reminders are often used by outcomes database software platforms as a means of reminding patients to complete PROM surveys. However, before our study, the effectiveness of e-mail reminders at improving survey completion rates was largely unknown.
E-mail reminders allow patients the ability to complete the survey from the comfort of their own home and at their own pace, eliminating confounding influences associated with survey completion in a busy office setting. Reminders have the potential to capture outcome scores for patients who could not reschedule a missed appointment, live a great distance away, or have limiting medical comorbidities and for those who think that they are doing well enough not to justify the cost associated with a follow-up visit. Although this study was not designed to capture the rationale for office visit absence, the study clearly demonstrated the benefits of e-mail reminders in capturing follow-up survey data that would have otherwise been missed.
The role of incentives was not examined in this study. Patients who completed PROMs in this study did so on the basis of a commitment to research and understanding the impacts of the procedure on the larger population. Financial incentives, which are often used to encourage participation, were not used at any time point in recovery. In the future, linking PROM scores to alerts, which notify the patient of a need for further clinical evaluation, may help to increase compliance. For example, it has been previously shown that a 3-point drop in SST score from previous survey can reflect component dysfunction after anatomic TSA.1 Completion of PROMs in patients who cannot make follow-up visits can become clinically relevant for patients treated with anatomic TSA and may thus encourage compliance with follow-up surveys. Creating incentives based on the usefulness and accuracy of completing the survey may have greater utility once these alerts are established.
There are limitations to this study. The study represents the initial 9-month experience using office-based tablets at the institution. Patients more recently included in the Repository may have higher compliance with office-based tablets. In addition, based on the senior author's postoperative protocol, only shoulder arthroplasty patients were instructed to follow up after the 1-year postoperative follow-up. This may help explain why the 1-year postoperative difference was not significant for the TSA patients as the majority likely completed tablet surveys during their office visit, and it also can help explain why office-based tablet surveys were higher for TSA patients than for arthroscopic rotator cuff repair or biceps tenodesis patients.
Conclusion
E-mail reminders with web-based completion of PROMs serve as a mechanism to obtain improvement in data completeness in the absence of in-office patient evaluation. These reminders should be included in all outcome database software platforms.
Disclaimer
Jonathan C. Levy reports that he is a paid consultant for DJO Orthopaedics and receives royalties from DJO Orthopaedics and Innomed. All the other authors, their immediate families, and any research foundations with which they are affiliated have not received any financial payments or other benefits from any commercial entity related to the subject of this article.
Footnotes
The Western Institutional Review Board approved this study: WIRB Work Order 1-878473-1.
Location: All work was performed at the Holy Cross Orthopedic Institute and Holy Cross Hospital.
References
- 1.Buckingham B.P., Parsons I.M., Campbell B., Titelman R.M., Smith K.L., Matsen F.A. Patient functional self-assessment in late glenoid component failure at three to eleven years after total shoulder arthroplasty. J Shoulder Elbow Surg. 2005;14:368–374. doi: 10.1016/j.jse.2004.10.008. [DOI] [PubMed] [Google Scholar]
- 2.Calfee R.P., Adams A.A. Clinical research and patient-rated outcome measures in hand surgery. J Hand Surg Am. 2012;37:851–855. doi: 10.1016/j.jhsa.2012.01.043. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Dy C.J., Schmicker T., Tran Q., Chadwick B., Daluiski A. The use of a tablet computer to complete the DASH questionnaire. J Hand Surg Am. 2012;37:2589–2594. doi: 10.1016/j.jhsa.2012.09.010. [DOI] [PubMed] [Google Scholar]
- 4.Heckman J.D. Are validated questionnaires valid? J Bone Joint Surg Am. 2006;88:446. [PubMed] [Google Scholar]
- 5.Howell M., Hood A.J., Jayne D.G. Use of a patient completed iPad questionnaire to improve pre-operative assessment. J Clin Monit Comput. 2017;31:221–225. doi: 10.1007/s10877-015-9818-0. [DOI] [PubMed] [Google Scholar]
- 6.Kaka H., Ayearst R., Tran M., Touma Z., Bagovich M., Vinik O. Developing an iPad application for data collection in a rheumatology research clinic. Int J Technol Assess Health Care. 2015;31:99–102. doi: 10.1017/S0266462315000069. [DOI] [PubMed] [Google Scholar]
- 7.Lippitt S., Harryman D., Matsen F. A practical tool for evaluating function: the Simple Shoulder Test. In: Matsen FA, Fu FH, Hawkins RJ, editors. The shoulder: a balance of mobility and stability. American Academy of Orthopaedic Surgeons; Rosemont, IL: 1993. pp. 501–518. In: [Google Scholar]
- 8.Nitikman M., Mulpuri K., Reilly C.W. Internet-administered health-related quality of life questionnaires compared with pen and paper in an adolescent scoliosis population: a randomized crossover study. J Pediatr Orthop. 2017;37:e75–9. doi: 10.1097/BPO.0000000000000716. [DOI] [PubMed] [Google Scholar]
- 9.Schamber E.M., Takemoto S.K., Chenok K.E., Bozic K.J. Barriers to completion of patient reported outcome measures. J Arthroplasty. 2013;28:1449–1453. doi: 10.1016/j.arth.2013.06.025. [DOI] [PubMed] [Google Scholar]
- 10.Sernert N., Kartus J., Köhler K., Stener S., Larsson J., Eriksson B.I. Analysis of subjective, objective and functional examination tests after anterior cruciate ligament reconstruction. A follow-up of 527 patients. Knee Surg Sports Traumatol Arthrosc. 1999;7:160–165. doi: 10.1007/s001670050141. [DOI] [PubMed] [Google Scholar]
- 11.Shervin N., Dorrwachter J., Bragdon C.R., Shervin D., Zurakowski D., Malchau H. Comparison of paper and computer-based questionnaire modes for measuring health outcomes in patients undergoing total hip arthroplasty. J Bone Joint Surg Am. 2011;93:285–293. doi: 10.2106/JBJS.I.01104. [DOI] [PubMed] [Google Scholar]
- 12.Tyser A.R., Beckmann J., Weng C., O'Farrell A., Hung M. A randomized trial of the Disabilities of the Arm, Shoulder, and Hand administration: tablet computer versus paper and pencil. J Hand Surg Am. 2015;40:554–559. doi: 10.1016/j.jhsa.2014.10.046. [DOI] [PubMed] [Google Scholar]
- 13.Ware J., Kosinski M., Keller S.D. A 12-Item Short-Form Health Survey: construction of scales and preliminary tests of reliability and validity. Med Care. 1996;34:220–233. doi: 10.1097/00005650-199603000-00003. [DOI] [PubMed] [Google Scholar]
- 14.Yaffe M., Goyal N., Kokmeyer D., Merrell G.A. The use of an iPad to collect patient-reported functional outcome measures in hand surgery. Hand (N Y) 2015;10:522–528. doi: 10.1007/s11552-014-9731-x. [DOI] [PMC free article] [PubMed] [Google Scholar]