Abstract
Background: Patient-reported outcome (PRO) instruments allow patients to interpret their health and are integral in evaluating orthopedic treatments and outcomes. The purpose of this study was to define: (1) correlation between PROs collected by automated delivery of text messages on mobile phones compared with paper delivery; and (2) patient use characteristics of a technology platform utilizing automated delivery of text messages on mobile phones. Methods: Paper versions of the 12-Item Short Form Health Survey (SF-12) and the short form of the Disabilities of the Arm, Shoulder and Hand (QuickDASH) were completed by patients in orthopedic hand and upper extremity clinics. Over the next 48 hours, the same patients also completed the mobile phone portion of the study outside of the clinic which included text message delivery of the SF-12 and QuickDASH, assigned in a random order. Correlations between paper and text message delivery of the 2 PROs were assessed. Results: Among 72 patients, the intraclass correlation coefficient (ICC) between the written and mobile phone delivery of QuickDASH was 0.91 (95% confidence interval [CI], 0.85-0.95). The ICC between the paper and mobile phone delivery of the SF-12 physical health composite score was 0.88 (95% CI, 0.79-0.93) and 0.86 (95% CI, 0.75-0.92) for the SF-12 mental health composite score. Conclusions: We find that text message delivery using mobile phones permits valid assessment of SF-12 and QuickDASH scores. The findings suggest that software-driven automated delivery of text communication to patients via mobile phones may be a valid method to obtain other PRO scores in orthopedic patients.
Keywords: mobile phone, outcomes, QuickDASH, SF-12, text message
Introduction
Patient-reported outcome (PRO) instruments allow patients to interpret their health and are integral in evaluating orthopedic treatments and outcomes.8,15,16 The short form of the Disabilities of the Arm, Shoulder and Hand (QuickDASH) is a PRO instrument validated in the setting of hand and upper extremity pathology.3,4,10 The 12-Item Short Form Health Survey (SF-12) is a validated outcome score when assessing both physical (Physical Composite Score [PCS]) and mental function (Mental Composite Score [MCS]) of multiple patient populations.7,10,28 Previous work has considered verbal delivery and verbal telephone delivery of QuickDASH, finding equivalent results to paper administration.6,21 Other PRO instruments have been validated across multiple delivery platforms including verbal, telephone (verbal), and in-office computer touchscreen interfaces.5,11,17,19,27,29
Traditionally, evaluation of orthopedic patients has included in-office face-to-face visits. We recognize that mobile phone technology presents an opportunity to communicate with patients outside of the hospital or clinic.5 In developed countries, 85% of the population owns a mobile phone; 80% engages in text messaging, 56% accesses the Internet on their smartphone, and 43% download smartphone applications.9 Mobile phone text message reminders have been shown to improve outcomes across a variety of patient populations.18 The input of text into a mobile phone interface has become a common interaction utilized by text messaging, smartphone applications, and Internet browsing on smartphones. Software algorithms can be used to deliver health-related questions via mobile phones without the need for extensive human resource utilization.1,2 As our ability to communicate with and evaluate patients outside of the hospital continues to grow with technological advances, it will be important for surgeons and health care systems to understand the benefits and limitations of software and mobile phone technologies. We find no previous work that has attempted to validate text message or software-driven delivery of general health or orthopedic PRO instruments on mobile phones outside of a hospital setting. We also find no previous studies that characterize use patterns of automated mobile phone messaging technology when assessing orthopedic patients outside of the hospital. We hypothesized that there would be (1) a high correlation between PROs collected by automated delivery of text messages on mobile phones compared with paper delivery and (2) no difference in completion rate by patient demographics of PRO instruments collected by automated delivery of text messages on mobile phones.
Methods
Patients presenting to a hospital-based orthopedic hand and upper extremity clinic were enrolled in this study after formal consent was obtained. Inclusion criteria included any new or return patient who had access to a mobile phone with text messaging capabilities. Exclusion criteria included any patient who was undergoing a same-day office procedure (ie, injection) or any patient who underwent surgery within the previous 90 days.
Paper versions of the SF-12 and the QuickDASH were completed by patients in our orthopedic hand and upper extremity clinic on study day (SD) 0. Enrollment for the mobile phone portion of the study was performed by a research assistant through a secure Internet website on an in-office desktop computer and required input of the patient’s telephone number and time of day they wanted to receive delivery of the mobile phone portion of the study. Patients were not compensated for this study. Seven different mobile phone providers were represented in our 72-patient cohort. Patients with a single mobile phone provider (n = 19) were excluded from the SF-12 study group as the text messages required could not be appropriately formatted. Patients were told they would receive the text message protocol over the next 48 hours; no formal demonstration of the text message protocol was performed. Over the next 48 hours (SDs 1 and 2), the same patients completed the mobile phone portion of the study outside of the clinic which included text message delivery of the SF-12 and QuickDASH, assigned in a random order. Previous authors have utilized similar methods when validating other QuickDASH delivery methods.6,21 Internally developed software was used to automate the delivery of text messages to all enrolled patients. At the appropriate patient-requested time, the software sent the patient the first question of the study (Figure 1). After the patient responded to the question, the next question in the study protocol was sent until all questions had been answered (Figure 1). We used the standard prescribed scoring algorithm to score all paper and mobile phone QuickDASH administrations.23 Scoring software (Optum, Eden Prairie, Minnesota) was utilized to score all paper and mobile phone administrations of the SF-12 PCS and MCS.
Figure 1.
Screenshot from portion of SF-12 questionnaire as viewed on a mobile phone interface.
Note. SF-12 = 12-Item Short Form Health Survey; PRO = patient-reported outcome.
Characteristics of participants who completed both the paper and the mobile phone questionnaires were compared using Student’s t test for continuous variables and chi-square or Fisher’s exact test for categorical variables. The Wilcoxon rank-sum test was used to compare completers versus noncompleters in SF-12 MCS scores in the SF-12 group and to compare age in the QuickDASH group as these variables were not normally distributed based on Shapiro-Wilk test results.
The intraclass correlation coefficient (ICC) between the written and text message delivery of PROs administered to the same participant was assessed using a 1-way analysis of variance. Using previous methods, we considered excellent reproducibility to be an ICC of >0.75, an ICC of 0.4 to 0.75 to represent good reproducibility, and an ICC of <0.4 to represent poor reproducibility.25 Based on a sample size calculation (80% power, α = 0.05), 39 patients were required to detect an ICC value of 0.8 (excellent reproducibility) distinguishable from 0.6 (good reproducibility). Bland-Altman plots were also created for graphical representation of agreement between paper and mobile phone PRO scores. These plot the difference versus the average value and include intervals of agreement (mean difference, ±1.96 standard deviation of mean difference). SAS statistical software (version 9.4; SAS Institute, Inc, Cary, North Carolina) was utilized for analyses, and a P value of <.05 was considered statistically significant. This investigation was approved by the institutional review board and was deemed compliant with the Health Insurance Portability and Accountability Act (HIPAA).
Results
Seventy-two consecutive patients enrolled in this investigation. Seven eligible patients refused to participate in the study. Average age was 44 years (range, 18-67 years), 63% were female, 40% had a bachelor’s degree or higher, and 69% were working full-time/part-time (Table 1). Carpal tunnel syndrome (21%), fracture/trauma (11%), ganglion cyst (8.5%), osteoarthritis (8.5%), and unspecified numbness/paresthesia (8.5%) were the most common clinical presentations.
Table 1.
Demographics of Completers Versus Noncompleters of QuickDASH (mean ± SD, median [minimum-maximum], or n [%]).
| Variable | Completers (n = 54) | Noncompleters (n = 18) | P value for difference |
|---|---|---|---|
| Age | 44.3 ± 13.8 47.5 (18.0-67.0) |
41.5 ± 10.8 40.0 (23.0-55.0) |
.3861 |
| Diabetes (% yes) | 7 (13.0) | 0 (0.0) | .1808 |
| Education | |||
| <HS | 3 (5.56) | 2 (11.11) | .2744 |
| HS/GED | 11 (20.37) | 1 (5.56) | |
| Some college | 20 (37.04) | 6 (33.33) | |
| Bachelor’s degree | 12 (22.22) | 3 (16.67) | |
| Graduate degree | 8 (14.81) | 6 (33.33) | |
| Education | |||
| <HS | 3 (5.56) | 2 (11.11) | .2565 |
| HS | 11 (20.37) | 1 (5.56) | |
| >HS | 40 (74.07) | 15 (83.33) | |
| Employment | |||
| Not working | 12 (22.22) | 3 (16.67) | .7987 |
| Retired | 4 (7.41) | 0 (0.00) | |
| Part-time | 8 (14.81) | 2 (11.11) | |
| Homemaker | 2 (3.70) | 1 (5.56) | |
| Full-time | 28 (51.85) | 12 (66.67) | |
| Employment (% currently working including homemaker) | 38 (70.4) | 15 (83.3) | .3643 |
| Dominant hand | |||
| Right | 45 (83.33) | 14 (82.35) | .1848 |
| Light | 8 (14.81) | 1 (5.88) | |
| Ambidextrous | 1 (1.85) | 2 (11.76) | |
| QuickDASH-paper | 37.4 ± 25.8 36.4 (0.0-86.4) |
44.0 ± 23.5 42.0 (6.8-79.5) |
.3198 |
| Race | .0006 | ||
| Asian | 1 (1.85) | 1 (5.56) | |
| Black | 0 (0.00) | 4 (22.22) | |
| Hispanic | 1 (1.85) | 1 (5.56) | |
| White | 52 (96.30) | 12 (66.67) | |
| Race (% white) | 52 (96.3) | 12 (66.7) | .0024 |
| Sex (% women) | 34 (63.0) | 11 (61.1) | .8882 |
| Smoker (% yes) | 6 (11.1) | 7 (38.9) | .0080 |
| Randomization | |||
| Paper administration | .5862 | ||
| SF-12 first | 28 (51.85) | 8 (44.44) | |
| QuickDASH first | 26 (48.15) | 10 (55.56) | |
| Text administration | .3353 | ||
| SF-12 first | 31 (57.41) | 10 (55.56) | |
| QuickDASH first | 23 (42.59) | 8 (44.44) | |
Note. Bold and italicized text signifies statistical significance <.05. QuickDASH = short form of the Disabilities of the Arm, Shoulder and Hand; SD = standard deviation; HS = high school education; GED = general education development; SF-12 = 12-Item Short Form Health Survey.
Average QuickDASH scores on paper (37.4) and mobile phone (34.9) were not significantly different (P = .61). Average paper SF-12 PCS (41.5) and SF-12 MCS (49.9) scores were not significantly different from mobile phone delivery of the SF-12 PCS (41.7, P = .91) and SF-12 MCS (49.9, P = .99) (Table 2). The ICC between the paper and mobile phone delivery of QuickDASH was 0.91 (95% confidence interval [CI], 0.85-0.95). The ICC between the paper and mobile phone delivery of the SF-12 PCS was 0.88 (95% CI, 0.79-0.93) and 0.86 (95% CI, 0.75-0.92) for the SF-12 MCS. The Bland-Altman plots revealed that the majority of data points were within the 95% agreement limits (Figures 2-4).
Table 2.
Demographics of Completers Versus Noncompleters of SF-12 (mean ± SD, median [minimum-maximum], or n [%]).
| Variable | Completers (n = 39) | Noncompleters (n = 12) | P value for difference |
|---|---|---|---|
| Age | 41.7 ± 13.6 40.0 (18.0-67.0) |
45.1 ± 11.5 45.5 (27.0-66.0) |
.4491 |
| Diabetes (% yes) | 5 (12.8) | 0 (0.0) | .3231 |
| Education | .6817 | ||
| <HS | 2 (5.13) | 0 (0.0) | |
| HS/GED | 6 (15.38) | 3 (25.0) | |
| Some college | 12 (30.8) | 5 (41.7) | |
| Bachelor’s degree | 10 (25.6) | 1 (8.3) | |
| Graduate degree | 9 (23.1) | 3 (25.0) | |
| Education | .8078 | ||
| <HS | 2 (5.1) | 0 (0.0) | |
| HS | 6 (15.4) | 3 (25.0) | |
| >HS | 31 (79.5) | 9 (75.0) | |
| Employment | .4977 | ||
| Not working | 6 (15.38) | 2 (16.67) | |
| Retired | 2 (5.13) | 0 (0.00) | |
| Part-time | 7 (17.95) | 0 (0.00) | |
| Homemaker | 2 (5.13) | 1 (8.33) | |
| Full-time | 22 (56.41) | 9 (75.00) | |
| Employment (% currently working) | 31 (79.5) | 10 (83.3) | 1.0000 |
| Dominant hand | .5929 | ||
| Right | 32 (82.05) | 9 (81.82) | |
| Left | 6 (15.38) | 1 (9.09) | |
| Ambidextrous | 2 (2.56) | 1 (9.09) | |
| PCS-paper | 41.5 ± 10.0 40.9 (16.2-62.2) |
43.2 ± 11.4 40.5 (23.0-57.2) |
.6093 |
| MCS-paper | 49.9 ± 10.2 51.2 (31.8-63.8) |
50.9 ± 8.6 53.9 (34.8-59.3) |
.9384 |
| Race | .1574 | ||
| Asian | 1 (2.56) | 1 (8.33) | |
| Black | 2 (5.13) | 2 (16.67) | |
| Hispanic | 1 (2.56) | 1 (8.33) | |
| White | 35 (89.74) | 8 (66.67) | |
| Race (% white) | 35 (89.7) | 8 (66.7) | .0764 |
| Sex (% women) | 24 (61.5) | 7 (58.3) | .8424 |
| Smoker (% yes) | 5 (12.8) | 5 (41.7) | .0277 |
| Randomization | |||
| Paper administration | .1054 | ||
| SF-12 first | 21 (53.85) | 3 (25.00) | |
| QuickDASH first | 18 (46.15) | 9 (75.00) | |
| Text administration | .3353 | ||
| SF-12 first | 19 (48.72) | 8 (66.67) | |
| QuickDASH first | 20 (51.28) | 4 (33.33) | |
Note. Bold and italicized text signifies statistical significance <.05. SF-12 = 12-Item Short Form Health Survey; SD = standard deviation; HS = high school education; GED = general education development; PCS = Physical Composite Score; MCS = Mental Composite Score; QuickDASH = short form of the Disabilities of the Arm, Shoulder and Hand.
Figure 2.
QuickDASH Bland-Altman plot showing differences between paper and mobile phone delivery versus their mean values.
Note. Red lines represent upper and lower 95% agreement limits. QuickDASH = short form of the Disabilities of the Arm, Shoulder and Hand.
Figure 3.
SF-12 PCS Bland-Altman plot showing differences between paper and mobile phone delivery versus their mean values.
Note. Upper and lower red lines represent upper and lower 95% agreement limits. SF-12 = 12-Item Short Form Health Survey; PCS = Physical Composite Score.
Figure 4.
SF-12 MCS Bland-Altman plot showing differences between paper and mobile phone delivery versus their mean values.
Note. Upper and lower red lines represent upper and lower 95% agreement limits. SF-12 = 12-Item Short Form Health Survey; MCS = Mental Composite Score.
Seventy-five percent of patients fully completed the QuickDASH questionnaire compared with a 77% completion rate for the SF-12. Cigarette smokers were less likely to complete both the QuickDASH (P = .008, Table 1) and SF-12 (P = .028, Table 2). As compared with all other races combined, white patients were more likely to complete the QuickDASH (P = .002, Table 1). Patient age, sex, level of education, and employment status were not found to have an impact on completion rate of both the QuickDASH and SF-12 (Tables 1 and 2). For both QuickDASH and SF-12 PCS and MCS, there were no significant differences in the mobile phone completion rate depending on which PRO was administered first over mobile phone (Tables 1 and 2). Time to completion of QuickDASH via mobile phone was ≤3 minutes in 35% of patients, >3 and ≤6 minutes in 20% of patients, and >6 and ≤10 minutes in 19% of patients. Ten patients (19%) took 1- or >5-minute pause during administration of QuickDASH. Time to completion of the SF-12 via mobile phone was ≤3 minutes in 18% of patients, >3 and ≤6 minutes in 36% of patients, and >6 and ≤10 minutes in 13% of patients. Fourteen patients (36%) took 1 or more >5-minute pause during administration of the SF-12.
Discussion
Patient-reported outcome instruments are useful tools for tracking functional status and determining patient outcomes in orthopedic practice.8,15,16 Mobile phone and automated software technologies present the opportunity for surgeons and health care systems to interact with patients outside of the hospital giving providers the ability to communicate with patients at anytime and from anywhere.1,2
This study hypothesized that there would be a high correlation between PROs collected by automated delivery of text messages on mobile phones compared with paper delivery. We report an ICC of 0.91 between paper and mobile phone delivery of QuickDASH and an ICC of 0.88 and 0.86 between paper and mobile phone delivery of the SF-12 PCS and MCS. Previous authors reported ICCs of 0.92 and 0.91 when comparing paper QuickDASH with verbal delivery and verbal delivery over telephone, respectively.6,21 We find that automated software text message delivery via mobile phones of the SF-12 and QuickDASH demonstrates high correlation with paper formats and is a valid delivery format for these PRO instruments. These findings suggest automated software text message delivery via mobile phones may also be used to obtain other general health and orthopedic PRO data.
We found an overall completion rate of 77% for mobile phone delivery of SF-12 and 75% for QuickDASH. There were no significant differences in completion rates based on patient age, sex, level of education, or employment status. Cigarette smokers were less likely to complete the PROs by mobile phone in our study. Utilizing verbal administration over telephone of QuickDASH, previous authors reported a completion rate of 86%.6 The completion rates demonstrated by patients in our study approached previously reported rates, notably without requiring use of human resources to obtain PRO data. In addition, our software did not send reminder messages if a patient stopped answering the SF-12 and QuickDASH protocols prior to completion, and we speculate that reminder messages may have increased the completion rates in our study. When considering how to obtain orthopedic PRO data, health care systems can consider implementing automated mobile phone messaging software for patient populations above the age of 18 regardless of increasing age, work, or educational status. We feel automated mobile phone messaging might be best used as an initial communication tool in an effort to obtain data from a majority of patients without the need for human intervention over the phone or in-office visits. The majority of our cohort completed the mobile phone–delivered PROs in <6 minutes, and we find that automated mobile phone messaging in the administration of the SF-12 and QuickDASH allows for a timely patient response. Extended pauses were observed in 19% of patients in QuickDASH and 36% of patients in SF-12. These findings suggest that communication with automated messaging and mobile phones is valid for attaining patient data in noncaptive settings, when patients may be intermittently engaged with other activities. Given evolving human interaction patterns with mobile phone technology that are at times asynchronous, our observation that collection of PRO data in these settings is valid presents those obtaining PRO data with expanded opportunities when communicating with patients. We recommend further inquiry into other communication settings using automated software messaging and mobile phones in an effort to further understand appropriate use cases.
In an era where assessment of patient outcomes is paramount in determining quality of orthopedic care, cost-effective modalities to obtain patient data are key. Patients, health care systems, and society incur time and monetary costs when patients travel from their home or place of work for sometimes lengthy in-office clinical encounters. Previous work has considered relative costs and benefits to patients and providers when various traditional telehealth programs are utilized.12,22,24 Traditionally, patients have answered in-office PRO questionnaires on paper or through computer interfaces. Providing patients an opportunity to answer PRO instruments at their leisure in a setting of their choosing may be beneficial and reduce the need for further formal follow-up. Although our study did not attempt to distinguish location or activity performed while answering the PRO instruments in our study, our patients were at the very least not present in a hospital setting when the mobile phone portion of the study was administered. This may be significant when considering appropriate methods to obtain follow-up for various outpatient orthopedic surgical procedures where in-office long-term follow-up is not warranted. There are multiple factors that comprise the cost of mobile phone text messaging software applications including upfront costs for the design and build process, costs associated with Internet servers and web or app hosting, and security. These costs can vary depending on the complexity of the software application, and a formal analysis of these factors is outside the scope of this work. The actual delivery of messages through mobile phone applications is of minimal cost at $0.01 for every message sent and received in the present study.26 In addition, our technology platform eliminates the need for human intervention in the data gathering process. The cost of either partially or fully employing staff members to obtain PRO data may be immense; mobile phone messaging platforms can be built that alleviate this cost. From the patient’s perspective, the cost of receiving and sending mobile phone messages is relatively inexpensive, from no cost to minor cost per message (1-2 cents) or unlimited messaging plans coupled with voice service for a set monthly fee. With ever increasing focus on cost in our health care system and increasing scarcity of research funding,20 efforts should be made to minimize cost for both patients and health care systems. We demonstrate a technology platform that is valid when obtaining several PRO instruments and may provide cost savings to health care systems. Furthermore, our described communication methods may present an opportunity for surgeons and health care systems to track outcomes for meaningful use without the patient needing to return for an office visit. We recommend a more formal and exhaustive cost analysis of automated mobile phone messaging platforms.
Patient information privacy is important whether having discussions in person, by voice over the phone, through email, or through mobile phone messaging applications. The HIPAA Privacy Rule states that health care providers must inform accommodate patient requests to receive communication through alternative communication methods, including email and text messages.12,13 We recommend having an open and honest discussion with patients regarding the risks and benefits of different forms of digital communication. Surgeons should make an effort to communicate with patients in the individual patient-preferred method.
There were several limitations in this study. One mobile phone carrier did not appropriately display the SF-12 questions, and we subsequently excluded these patients from the final analysis. As this carrier issue was recognized during data collection, study enrollment was continued well beyond the goal number indicated by the a priori power analysis. Designing software communication platforms that work across the vast array of mobile phone providers, operating systems, and mobile phones is a challenging task. Prior to large scale deployment, health care systems should appropriately test software applications across the many possible phone and software setups available. In addition, we suggest that when building or rebuilding PROs, designers consider question presentation on the relatively small screens which are inherent in mobile phones. A vast array of mobile phone screen sizes exists, and more succinct communication will be important on these smaller user interfaces. We also did not collect more extensive PRO data utilizing other instruments. We utilized a general health PRO (SF-12) and an orthopedic-specific PRO (QuickDASH) and acknowledge that we would be able to draw stronger conclusions on the generalizability of our study if other patient populations or PRO instruments were considered. Future directions should also include validation of our described technology platform across other PRO instruments including adaptive PRO instruments including the Patient-Reported Outcomes Measurement Information System (PROMIS).15,16 Finally, we did not conduct a formal follow-up usability or patient satisfaction evaluation of our described mobile phone messaging platform and acknowledge that this information would be helpful in future assessments.
Text message delivery of the SF-12 and QuickDASH using mobile phones demonstrates high correlation with paper formats. We find that text message delivery using mobile phones permits valid assessment of SF-12 and QuickDASH scores. Patients utilizing automated text message delivery of PRO instruments demonstrated completion rates approaching prior studies, and we find no difference in completion rates based on age, sex, level of education, or employment status. The findings strongly suggest software-driven automated delivery of text communication to patients via mobile phones may be a valid method to obtain other PRO scores in orthopedic patients. The results suggest that appropriately designed software and mobile phone technology platforms may be utilized to communicate with patients outside of the hospital setting, and emphasize the need for further inquiry in this area.
Footnotes
Ethical Approval: Each author certifies that his or her institution approved or waived approval for the human protocol for this investigation and that all investigations were conducted in conformity with ethical principles of research.
Statement of Human and Animal Rights: All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975, as revised in 2008.
Statement of Informed Consent: Informed consent was obtained from all individual participants included in the study.
Declaration of Conflicting Interests: The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding: The author(s) received no financial support for the research, authorship, and/or publication of this article.
References
- 1. Anthony CA, Peterson AR. Utilization of a text-messaging robot to assess intraday variation in concussion symptom severity scores. Clin J Sport Med. 2015;25(2):149-152. [DOI] [PubMed] [Google Scholar]
- 2. Anthony CA, Polgreen LA, Chounramany J, et al. Outpatient blood pressure monitoring using bi-directional text messaging. J Am Soc Hypertens. 2015;9(5):375-381. [DOI] [PubMed] [Google Scholar]
- 3. Beaton DE, Katz JN, Fossel AH, et al. Measuring the whole or the parts? Validity, reliability, and responsiveness of the Disabilities of the Arm, Shoulder and Hand outcome measure in different regions of the upper extremity. J Hand Ther. 2001;14(2):128-146. [PubMed] [Google Scholar]
- 4. Beaton DE, Wright JG, Katz JN; Upper Extremity Collaborative Group. Development of the QuickDASH: comparison of three item-reduction approaches. J Bone Joint Surg Am. 2005;87(5):1038-1046. [DOI] [PubMed] [Google Scholar]
- 5. Bellamy N, Wilson C, Hendrikz J, et al. ; EDC Study Group. Osteoarthritis Index delivered by mobile phone (m-WOMAC) is valid, reliable, and responsive. J Clin Epidemiol. 2011;64(2):182-190. [DOI] [PubMed] [Google Scholar]
- 6. Bot AG, Becker SJ, Mol MF, et al. Validation of phone administration of short-form disability and psychology questionnaires. J Hand Surg Am. 2013;38(7):1383-1387. [DOI] [PubMed] [Google Scholar]
- 7. Busija L, Pausenberger E, Haines TP, et al. Adult measures of general health and health-related quality of life: Medical Outcomes Study Short Form 36-Item (SF-36) and Short Form 12-Item (SF-12) Health Surveys, Nottingham Health Profile (NHP), Sickness Impact Profile (SIP), Medical Outcomes Study Short Form 6D (SF-6D), Health Utilities Index Mark 3 (HUI3), Quality of Well-Being Scale (QWB), and Assessment of Quality of Life (AQoL). Arthritis Care Res (Hoboken). 2011;63(suppl 11): S383-S412. [DOI] [PubMed] [Google Scholar]
- 8. Carr AJ. Evidence-based orthopaedic surgery: what type of research will best improve clinical practice? J Bone Joint Surg Br. 2005;87(12):1593-1594. [DOI] [PubMed] [Google Scholar]
- 9. Duggan M, Rainie L. Cell Phone Activities 2012. Washington, DC: Pew Research Center’s Internet & American Life Project, Pew Research Center; 2012 [Google Scholar]
- 10. Fan ZJ, Smith CK, Silverstein BA. Assessing validity of the QuickDASH and SF-12 as surveillance tools among workers with neck or upper extremity musculoskeletal disorders. J Hand Ther. 2008;21(4):354-365. [DOI] [PubMed] [Google Scholar]
- 11. Frennered K, Hagg O, Wessberg P. Validity of a computer touch-screen questionnaire system in back patients. Spine (Phila Pa 1976). 2010;35(6):697-703. [DOI] [PubMed] [Google Scholar]
- 12. HIPAA Privacy Rule 45 CFR 164.522(b). [Google Scholar]
- 13. HIPAA Privacy Rule 78 FR 5634. [Google Scholar]
- 14. Ho YL, Yu JY, Lin YH, et al. Assessment of the cost-effectiveness and clinical outcomes of a fourth-generation synchronous telehealth program for the management of chronic cardiovascular disease. J Med Internet Res. 2014;16(6):e145. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Hung M, Baumhauer JF, Brodsky JW, et al. ; Orthopaedic Foot & Ankle Outcomes Research (OFAR) of the American Orthopaedic Foot & Ankle Society (AOFAS). Psychometric comparison of the PROMIS physical function CAT with the FAAM and FFI for measuring patient-reported outcomes. Foot Ankle Int. 2014;35(6):592-599. [DOI] [PubMed] [Google Scholar]
- 16. Hung M, Stuart AR, Higgins TF, et al. Computerized adaptive testing using the PROMIS physical function item bank reduces test burden with less ceiling effects compared with the short musculoskeletal function assessment in orthopaedic trauma patients. J Orthop Trauma. 2014;28(8):439-443. [DOI] [PubMed] [Google Scholar]
- 17. Johnson K. Use of telephone follow-up for post-cardiac surgery patients. Intensive Crit Care Nurs. 2000;16(3):144-150. [DOI] [PubMed] [Google Scholar]
- 18. Kannisto KA, Koivunen MH, Valimaki MA. Use of mobile phone text message reminders in health care services: a narrative literature review. J Med Internet Res. 2014;16(10):e222. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Kimman ML, Bloebaum MM, Dirksen CD, et al. Patient satisfaction with nurse-led telephone follow-up after curative treatment for breast cancer. BMC Cancer. 2010;10:174. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Larson RC, Ghaffarzadegan N, Diaz MG. Magnified effects of changes in NIH research funding levels. Serv Sci. 2012;4(4):382-395. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. London DA, Stepan JG, Boyer MI, et al. Performance characteristics of the verbal QuickDASH. J Hand Surg Am. 2014;39(1):100-107. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Ohinmaa A, Vuolio S, Haukipuro K, et al. A cost-minimization analysis of orthopaedic consultations using videoconferencing in comparison with conventional consulting. J Telemed Telecare. 2002;8(5):283-289. [DOI] [PubMed] [Google Scholar]
- 23. QuickDASH outcome measures. Toronto, Ontario, Canada: Institute for Work & Health; 2006. [Google Scholar]
- 24. Seto E. Cost comparison between telemonitoring and usual care of heart failure: a systematic review. Telemed J E Health. 2008;14(7):679-686. [DOI] [PubMed] [Google Scholar]
- 25. Shoukri M, Pause CA. Statistical Methods for Health Sciences. Boca Raton, FL: CRC Press; 1998. [Google Scholar]
- 26. Twilio. Pricing. Date unknown. https://www.twilio.com/. Accessed September 14, 2016.
- 27. Velikova G, Wright EP, Smith AB, et al. Automated collection of quality-of-life data: a comparison of paper and computer touch-screen questionnaires. J Clin Oncol. 1999;17(3):998-1007. [DOI] [PubMed] [Google Scholar]
- 28. Ware J, Jr, Kosinski M, Keller SD. A 12-Item Short-Form Health Survey: construction of scales and preliminary tests of reliability and validity. Med Care. 1996;34(3):220-233. [DOI] [PubMed] [Google Scholar]
- 29. Williams M, Amin A, Getgood A, et al. Telephone clinic follow-up following carpal tunnel decompression. J Hand Surg Eur Vol. 2008;33(5):641-644. [DOI] [PubMed] [Google Scholar]