Abstract
Background
Symptoms of depression and anxiety commonly coexist with chronic musculoskeletal pain, and when this occurs, standard orthopedic treatment is less effective. However, mental health intervention is not yet a routine part of standard orthopedic treatment, in part because of access-related barriers. Digital mental health intervention is a potential scalable resource that could be feasibly incorporated into orthopedic care.
Objective
This study’s primary purpose was to assess the feasibility of introducing a digital mental health intervention (Wysa) in an outpatient orthopedic setting to patients with coexisting symptoms of depression and/or anxiety. The secondary purpose was to perform a preliminary effectiveness analysis of the intervention.
Methods
In this single-arm, prospective cohort study, participants included adult patients (18 years and older) who presented to a nonsurgical orthopedic specialist at a single tertiary care academic center for evaluation of a musculoskeletal condition and who self-reported symptoms of depression and/or anxiety (Patient-Reported Outcomes Measurement Information System [PROMIS] Depression and/or Anxiety score ≥55). Face-to-face enrollment was performed by a research coordinator immediately after the participant’s encounter with an orthopedic clinician. Participants were provided 2 months of access to a mobile app called Wysa, which is an established, multicomponent digital mental health intervention that uses chatbot technology and text-based access to human counselors to deliver cognitive behavioral therapy, mindfulness training, and sleep tools, among other features. For this study, Wysa access also included novel, behavioral activation–based features specifically developed for users with chronic pain. Primary feasibility outcomes included the study recruitment rate, retention rate, and engagement rate with Wysa (defined as engagement with a therapeutic Wysa tool at least once during the study period). Secondary effectiveness outcomes were between-group differences in mean longitudinal PROMIS mental and physical health score changes at 2-month follow-up between high and low Wysa users, defined by a median split.
Results
The recruitment rate was 29.3% (61/208), retention rate was 84% (51/61), and engagement rate was 72% (44/61). Compared to low users, high users reported greater improvement in PROMIS Anxiety scores (between-group difference −4.2 points, 95% CI −8.1 to −0.2; P=.04) at the 2-month follow-up. Between-group differences in PROMIS Depression (−3.2 points, 95% CI −7.5 to 1.2; P=.15) and Pain Interference scores (−2.3 points, 95% CI −6.3 to 1.7; P=.26) favored high users but did not meet statistical significance. Improvements in PROMIS Physical Function scores were comparable between groups.
Conclusions
Delivery of a digital mental health intervention within the context of orthopedic care is feasible and has the potential to improve mental health and pain-related impairment to a clinically meaningful degree. Participants’ engagement rates exceeded industry standards, and additional opportunities to improve recruitment and retention were identified. Further pilot study followed by a definitive, randomized controlled trial is warranted.
Trial Registration
ClinicalTrials.gov NCT04640090; https://clinicaltrials.gov/ct2/show/NCT04640090
Keywords: digital health, mental health, depression, anxiety, chronic pain, musculoskeletal, orthopedic, pain management, health intervention, mobile phone
Introduction
Background
Of the over 70 million Americans who have chronic musculoskeletal pain, up to half of them have a comorbid diagnosis of depression and/or anxiety [1-4]. Chronic pain and psychological impairment such as depression and anxiety are driven by shared biological pathways and neurotransmitters, and when a person has both conditions simultaneously, the effectiveness of standard treatment for either condition in isolation is reduced [5-7]. For example, among people who seek care for chronic musculoskeletal pain, those with coexisting depression and/or anxiety report worse pain, greater physical impairment, increased postoperative opioid use, and a lower rate of return to work than those without depression and anxiety [8-11].
Despite strong evidence regarding the interconnection between physical pain and mental health, psychological assessment and intervention is not yet considered a routine part of orthopedic treatment. This is, in part, because of both provider- and patient-related barriers. Orthopedic clinicians do not feel equipped with the necessary time and referral resources to address mental health [12], and patients report barriers to seeking mental health treatment even outside an orthopedic setting. Among people who recognize their need for mental health treatment, 64% are resistant to reaching out to another person for assistance and 23% have financial, transportation, or time-related barriers to seeking care [13,14]. Even if all patients had the resources to seek appropriate care, unmet need would persist because of a national shortage of qualified mental health clinicians [15].
Digital mental health interventions address the accessibility-, cost-, convenience-, and stigma-related barriers to traditional in-person mental health treatment [16]. Furthermore, a growing body of evidence supports the effectiveness of digital mental health interventions in improving depression and anxiety symptoms, sometimes to a degree equivalent to that obtained from in-person treatment [17-19]. With the widespread, everyday use of smartphones, these emerging digital tools could function as efficient, widely available mental health resources for orthopedic clinicians to offer their patients [16,20-22].
Objectives
The primary purpose of this study was to assess the feasibility of introducing a digital mental health intervention (Wysa) in an outpatient orthopedic setting to patients with musculoskeletal pain and coexisting symptoms of depression and/or anxiety. The secondary purpose was to perform a preliminary effectiveness analysis of the intervention. We hypothesized that delivery of a digital mental health intervention would be feasible in an orthopedic clinic setting. Furthermore, compared to “low users” of the digital mental health intervention, “high users” would report greater improvements in mental and physical health symptoms at the 2-month follow-up.
Methods
Design
This was a single-arm, prospective cohort, pilot feasibility study performed at a tertiary care academic medical center in the United States. Institutional review board approval was obtained prior to participant recruitment (IRB #202005219), and the study was registered through ClinicalTrials.gov (NCT04640090). Participants were enrolled from December 8, 2020, through July 14, 2021, with some interruptions related to the COVID-19 pandemic.
Participants
Eligibility Criteria
Participants were recruited from among patients who presented to an orthopedic department for evaluation and treatment of a musculoskeletal condition. To be eligible, patients had to be adults (18 years or older) presenting to a nonsurgical orthopedic provider. Additionally, patients had to have symptoms of depression and/or anxiety as indicated by a score of 55 or higher on the Patient-Reported Outcomes Measurement Information System (PROMIS) Depression and Anxiety measures; this is collected as standard clinical care on check-in for each outpatient appointment at this orthopedic department. Patients who were actively planning to start in-person mental health treatment were excluded from participation. Patients without access to a mobile device were also not eligible.
Recruitment Process
We recruited participants from among patients who presented to 6 orthopedic clinicians, 5 of whom were board-certified physical medicine and rehabilitation physicians (physiatrists) with subspecialty training in spine and sports medicine and 1 who was a nurse practitioner with 11 years of nonsurgical orthopedics experience. Prior to the clinician’s encounter with each patient, the study coordinator notified the clinician of patients who met the age and PROMIS score criteria for the study. Then, at the end of a potentially eligible patient’s standard clinic encounter, the clinician briefly introduced the study. For those who expressed initial interest, the research coordinator approached the patient, provided further study details, and completed the final screening, consent, and enrollment process for those who were interested. When possible, participant enrollment was completed in person before the participant left the orthopedic clinic office. Patients who were interested in the study but did not have time to enroll immediately were contacted over the phone by the research coordinator to complete the enrollment process. No compensation besides access to the study intervention was provided to participants for their participation in the study.
Intervention
All study participants received a complimentary, 2-month subscription to Wysa, which is a commercially available digital mental health intervention. Wysa is a multicomponent intervention that uses artificial intelligence–based chatbot technology and human “coaches” (counselors) with master’s degrees in psychology to deliver therapeutic content such as cognitive behavioral therapy, dialectical behavioral therapy, motivational interviewing, mindfulness training, deep breathing techniques, and sleep meditations [23,24]. Commercially, basic chatbot features are freely available, and premium digital features and text-based sessions with a human coach are available for a monthly fee. In this study, participants received access to a novel version of Wysa that was specifically developed for users interested in tools to manage chronic pain (Figure 1). Using behavioral activation and pain acceptance principles, this pain-specific version of Wysa encourages users to “engage with things that bring joy, despite having pain” [25-28]. Added features include daily check-ins, weekly reports, the ability to unlock premium “reward” tool packs by engaging with the weekly reports, and a progress roadmap. The chatbot and human coach features are also still embedded within the pain version of Wysa. This version is not yet part of the Wysa commercial product.
Figure 1.
Screenshots of Wysa’s chronic pain version.
Variables
Data were obtained from 3 sources: electronic medical records, participants’ self-report, and usage reports from Wysa. Data collected as standard clinical care were obtained from the electronic medical record; this included basic sociodemographic variables, medical history, and pain medication use. Additionally, on the day of the orthopedic clinic visit and study enrollment, patients completed Adult PROMIS CAT Depression v1.0, Anxiety v1.0, Pain Interference v1.1, and Physical Function v2.0 measures as standard care prior to meeting with the clinician. These measures were completed on a tablet computer (iPad Mini; Apple Inc). Each PROMIS domain is normalized to the general United States population with a mean (SD) score of 50 (SD 10) [29]. Higher scores represent more of the domain; for instance, a score of 60 on PROMIS Anxiety indicates anxiety symptoms that are worse than those in the general population, but a score of 60 on PROMIS Physical Function indicates physical function that is better than that in the general population.
On study enrollment, participants completed additional self-reported measures on the tablet computers. These included participants’ pain duration, primary pain location, height and weight (to calculate BMI), history of tobacco use, exercise habits, Brief Resilience Scale [30], and Importance, Readiness, and Confidence to Change rulers [31].
At the 1- and 2-month follow-up, participants were emailed a questionnaire that included PROMIS Depression, Anxiety, Pain Interference, and Physical Function domains; participants received automated email reminders as needed 2 and 4 days later. For the 2-month follow-up, they also received up to 3 phone call reminders if they had not completed the questionnaire by the sixth day after the follow-up period. All electronic medical record and self-reported data were stored in a secure electronic Research Electronic Data Capture (REDCap; Vanderbilt University) database [32,33].
Wysa provided user-specific, timestamped usage data for each app interaction by participants. Individual users were identified with randomly generated codes, and only study personnel had access to the participant key. No personally identifiable data were shared with Wysa by the study team or vice versa.
Outcomes
Feasibility
The primary study outcomes were related to the feasibility of introducing a digital mental health intervention (Wysa) in an outpatient orthopedic setting and conducting a fully powered, randomized controlled effectiveness trial in this setting. Outcomes included the study recruitment rate (defined as the number of patients who signed the study informed consent document divided by the number of patients approached for participation), study retention rate (defined as the proportion of recruited participants who completed the 2-month follow-up PROMIS measures), and the rate of engagement with Wysa (defined as the proportion of recruited participants who interacted with Wysa at least once after onboarding to the app). Other engagement metrics were also evaluated, including the weekly engagement rate (defined as the proportion of recruited participants who completed at least 8 interactions with Wysa over the 8-week period, because this would approximately mirror the intensity of weekly sessions with an in-person therapist), the coach engagement rate (defined as the proportion of recruited participants who completed at least one session with a human coach), and the most frequently used features within Wysa identified in a descriptive analysis. An exploratory analysis was also conducted to assess for differences in patient demographics (ie, age, sex) between those who enrolled in this study and those who did not.
Effectiveness
A secondary purpose of the study was to perform a preliminary effectiveness analysis of the intervention. To account for clinical improvements that could be related to usual orthopedic care or natural variations in symptom severity over time, the effectiveness outcomes considered were the between-group differences (between high and low Wysa users) in 2-month changes on PROMIS Depression, Anxiety, Pain Interference, and Physical Function measures. High versus low Wysa usage was defined by a participant’s number of total Wysa interactions lying above versus below the median engagement level among participants. Minimum clinically meaningful effect sizes were a priori chosen in accordance with previously published literature involving patients with chronic musculoskeletal pain who were managed nonoperatively. Effect sizes also had to exceed the SE of measurement for each PROMIS computer adaptive test at the study institution to be considered meaningful. Therefore, clinically meaningful effect sizes were defined as at least 3.2 points for PROMIS Depression, 3.0 points for Anxiety, 2.0 points for Pain Interference, and 2.2 points for Physical Function [34-36].
Statistical Analysis
Univariate descriptive statistics were calculated for all study variables. Age and sex were compared between those who did and did not enroll using an unequal-variances t test and a two-sample chi-square test of proportions, respectively. A median split on the total number of Wysa interactions was chosen to categorize participants as high and low Wysa users because density plots of the distributions of Wysa total interactions, chatbot interactions, and coach sessions failed to identify any empirical cut points. For the effectiveness analyses, linear mixed-effects models were fit predicting each of the 4 PROMIS measures from Wysa usage status (high versus low) and month. Adjusting for age as a covariate in the models did not meaningfully influence the model results, so unadjusted models are reported. Missing data were omitted. Significance was set a priori at P<.05, and sample size for this feasibility study was determined by the availability of resources. Statistical analyses were performed using R (v4.0.2; R Core Team).
Results
Feasibility
The study recruitment rate was 29.3% (61/208; Figure 2). Among the 61 patients who enrolled in the study, the median age was 55 years (range 18-83 years) and 87% (53/61) were female (Tables 1 and 2). The most common reasons patients did not enroll were that they did not want to use the app (104/147, 70.7%) or did not want to complete the research surveys (26/147, 17.7%). No statistically significant differences were observed in the age or sex distribution of patients who enrolled versus those who did not enroll in the study (P=.21; Table 3).
Figure 2.
Participant flow, from initial study contact through final follow-up.
Table 1.
Baseline characteristics of patients who presented to an orthopedic clinic for a musculoskeletal condition, reported coexisting symptoms of depression and/or anxiety, and enrolled in a research study that provided access to a digital mental health intervention (Wysa) (N=61).
| Characteristic | Value | |
| Age (years), median (IQR) | 55 (42-64) | |
| Sex, n (%) | ||
|
|
Female | 53 (87) |
|
|
Male | 7 (12) |
|
|
Transgender | 1 (2) |
| Race, n (%) | ||
|
|
White/Caucasian | 55 (90) |
|
|
Black/African American | 6 (10) |
| Ethnicity, n (%) | ||
|
|
Hispanic | 2 (3) |
|
|
Not Hispanic | 59 (97) |
| Area Deprivation Indexa, n (%) | ||
|
|
Quartile 1 (least deprived) | 19 (31) |
|
|
Quartile 2 | 15 (25) |
|
|
Quartile 3 | 14 (23) |
|
|
Quartile 4 (most deprived) | 13 (21) |
| Pain duration (years), median (IQR) | 2 (0.9-5.9) | |
| Pain locationb | ||
|
|
Low back | 31 (51) |
|
|
Leg | 19 (31) |
|
|
Neck | 18 (30) |
|
|
Arm | 8 (13) |
|
|
Generalized pain | 3 (5) |
| BMI (kg/m2), mean (SD) | 29 (7) | |
| Tobacco use, n (%) | 4 (7) | |
| Moderate/strenuous exercise (minutes/week), median (IQR) | 40 (0-120) | |
| Exercise limited by pain, n (%) | 49 (85) | |
| Exercise limited by enjoyment, n (%) | 6 (11) | |
| Brief Resilience Scale scorec, median (IQR) | 3.2 (2.5-3.7) | |
| Potential for behavior changed, median (IQR) | ||
|
|
Importance of change | 76 (68-90) |
|
|
Readiness for change | 69 (50-82) |
|
|
Confidence in the ability to change | 68 (50-74) |
| Any pain medication use, n (%) | 46 (75) | |
| Specific pain medication usee, n (%) | ||
|
|
Opioid | 7 (15) |
|
|
Nonsteroidal anti-inflammatory drug | 15 (33) |
|
|
Neuropathic | 22 (48) |
|
|
Other | 20 (44) |
| Medical history, n (%) | ||
|
|
Hypertension | 18 (30) |
|
|
Hyperlipidemia | 17 (28) |
|
|
Cardiovascular disease | 7 (11) |
|
|
Diabetes | 8 (13) |
|
|
Sleep apnea | 9 (15) |
|
|
Depression | 24 (39) |
|
|
Anxiety | 23 (38) |
aThe national Area Deprivation Index is a community-level measure of social disadvantage based on a person’s 9-digit zip code [37,38].
bSome patients reported multiple pain locations.
cThe Brief Resilience Scale is scored from 1 to 5, with higher scores representing greater resilience [30].
dPotential for change measures are scored from 0 to 100, and higher scores are favorable [39].
eNumber of patients among the 46 patients who used any pain medication.
Table 2.
Summary of mental and physical health scores measured by the Patient-Reported Outcomes Measurement Information System over the 2-month follow-up in patients who presented to an orthopedic clinic for a musculoskeletal condition, reported coexisting symptoms of depression and/or anxiety, and enrolled in a research study that provided access to a digital mental health intervention (Wysa).
| PROMISa scores | Baseline (n=61), mean (SD) | 2-month follow-up (n=51), mean (SD) |
| Depression | 58.1 (7.3) | 54.7 (8.7) |
| Anxiety | 62.2 (5.9) | 58.0 (7.8) |
| Pain interference | 65.2 (6.5) | 62.1 (7) |
| Physical function | 35.9 (6.6) | 39.5 (6.7) |
aPROMIS: Patient-Reported Outcomes Measurement Information System.
Table 3.
Demographic comparison of patients who presented to an orthopedic clinic for a musculoskeletal condition, reported coexisting symptoms of depression and/or anxiety, and enrolled versus those who did not enroll in a research study that provided access to a digital mental health intervention (Wysa) (N=208).
| Demographic | Enrolled (n=61) | Did not enroll (n=147) | Between-group difference | |
|
|
|
|
Mean difference or % difference (95% CI) | P value |
| Age (years), mean (SD) | 53 (14) | 56 (18) | 3 (−2 to 8) | .21 |
| Female sex, n (%) | 53 (87) | 115 (78.2) | 9 (−3 to 21) | .21 |
The retention rate of participants who consented to participate in the study was 84% (51/61). More participants completed the 2-month follow-up measures (51/61, 84% using email and phone reminders) than the 1-month follow-up measures (36/61, 59% using email reminders only).
The overall rate of engagement with Wysa among participants who consented to participate in the study was 72% (44/61). The weekly engagement rate was 57% (35/61), and the coach engagement rate was 33% (20/61). The therapeutic mechanisms underlying the tools most commonly used by participants were related to mindfulness (122/351, 34.8%), cognitive behavioral therapy (76/351, 21.7%), and sleep (53/351, 15.1%; Multimedia Appendix 1). Of the rewards unlocked by participants, sleep-related rewards were the most common (16/48, 33%) Table 4).
Table 4.
Summary of engagement with a digital mental health intervention (Wysa) by patients who presented to an orthopedic clinic for a musculoskeletal condition and reported coexisting symptoms of depression and/or anxiety (N=61).
| Engagement metric | Median (IQR) | Range |
| Total interactions | 18 (0-64) | 0-544 |
| Interactions with chatbot | 9 (0-29) | 0-181 |
| Daily check-ins completed | 2 (0-16) | 0-81 |
| Weekly reports viewed | 0 (0-2) | 0-14 |
| Rewards unlocked | 0 (0) | 0-8 |
| Text-based sessions with a human coach | 0 (0-1) | 0-12 |
| Total messages to a human coach | 0 (0-27) | 0-382 |
Effectiveness
Compared to low Wysa users, high Wysa users reported greater improvement to a clinically meaningful degree in PROMIS Anxiety scores at the 2-month follow-up (between-group difference −4.2 points, 95% CI −8.1 to −0.2; P=.04; Table 5). Between-group differences in 2-month improvements in PROMIS Depression (−3.2 points, 95% CI −7.5 to 1.2; P=.15) and Pain Interference scores (−2.3 points, 95% CI −6.3 to 1.7; P=.26) favored high Wysa users and met clinically meaningful thresholds but did not meet statistical significance. In contrast, both groups reported clinically meaningful and statistically significant improvements in PROMIS Physical Function scores from baseline to 2 months (P=.99), but there was no between-group difference in this domain (Figures 3-6).
Table 5.
Summary of mental and physical health changes (measured by the Patient-Reported Outcomes Measurement Information System) across the 2-month follow-up in patients who presented to an orthopedic clinic for a musculoskeletal condition and reported coexisting symptoms of depression and anxiety subgrouped by frequency of use of a digital mental health intervention (Wysa).
| PROMISa domain | Baseline (n=61), mean (95% CI) | 2-month follow-up (n=51)b, mean (95% CI) | Within-group longitudinal change, mean (95% CI) | Between-group changec |
||
|
|
|
|
|
Mean (95% CI) | P value | |
| Depression | −3.2 (−7.5 to 1.2) | .15 | ||||
|
|
High usersd | 57.5 (54.5 to 60.4) | 52.8 (49.9 to 55.8) | −4.7 (−7.5 to −1.8) |
|
|
|
|
Low userse | 58.7 (55.8 to 61.7) | 57.3 (53.9 to 60.7) | −1.5 (−4.7 to 1.8) |
|
|
| Anxiety | −4.2 (−8.1 to −0.2) | .04 | ||||
|
|
High users | 62.1 (59.6 to 64.7) | 56.6 (54.0 to 59.1) | −5.6 (−8.2 to −3.0) |
|
|
|
|
Low users | 62.4 (59.8 to 64.9) | 61.0 (58.0 to 63.9) | −1.4 (−4.4 to 1.6) |
|
|
| Pain interference | −2.3 (−6.3 to 1.7) | .26 | ||||
|
|
High users | 64.5 (62.1 to 66.8) | 60.6 (58.2 to 63.0) | −3.9 (−6.5 to −1.2) |
|
|
|
|
Low users | 65.9 (63.5 to 68.3) | 64.3 (61.6 to 67.1) | −1.5 (−4.6 to 1.5) |
|
|
| Physical function | 0.0 (−3.4 to 3.5) | .99 | ||||
|
|
High users | 37.1 (34.6 to 39.5) | 40.5 (38.0 to 42.9) | 3.4 (1.2 to 5.7) |
|
|
|
|
Low users | 34.7 (32.3 to 37.1) | 38.0 (35.3 to 40.8) | 3.4 (0.8 to 6.0) |
|
|
aPROMIS: Patient-Reported Outcomes Measurement Information System.
bThe 2-month follow-up PROMIS measures were available for 30 high and 21 low Wysa users.
cClinically meaningful effect sizes are defined as at least 3.2 points for PROMIS Depression, 3.0 points for Anxiety, 2.0 points for Pain Interference, and 2.2 points for Physical Function [34-36].
dThere were 30 patients in the high Wysa use subgroup.
eThere were 31 patients in the low Wysa use subgroup.
Figure 3.
Mean longitudinal change in Patient-Reported Outcomes Measurement Information System (PROMIS) Depression scores over 2-month follow-up in patients who presented to an orthopedic clinic for a musculoskeletal condition, reported coexisting symptoms of depression and/or anxiety, and were high users (n=30, green circles) versus low users (n=31, blue triangles) of a digital mental health intervention (Wysa). Error bars represent standard errors.
Figure 6.
Mean longitudinal change in Patient-Reported Outcomes Measurement Information System (PROMIS) Physical Function scores over 2-month follow-up in patients who presented to an orthopedic clinic for a musculoskeletal condition, reported coexisting symptoms of depression and/or anxiety, and were high users (n=30, green circles) versus low users (n=31, blue triangles) of a digital mental health intervention (Wysa). Error bars represent standard errors..
Figure 4.
Mean longitudinal change in Patient-Reported Outcomes Measurement Information System (PROMIS) Anxiety scores over 2-month follow-up in patients who presented to an orthopedic clinic for a musculoskeletal condition, reported coexisting symptoms of depression and/or anxiety, and were high users (n=30, green circles) versus low users (n=31, blue triangles) of a digital mental health intervention (Wysa). Error bars represent standard errors. Asterisk represents a statistically significant between-group difference in longitudinal score changes from baseline to 2-month follow-up (P<.05).
Figure 5.
Mean longitudinal change in Patient-Reported Outcomes Measurement Information System (PROMIS) Pain Interference scores over 2-month follow-up in patients who presented to an orthopedic clinic for a musculoskeletal condition, reported coexisting symptoms of depression and/or anxiety, and were high users (n=30, green circles) versus low users (n=31, blue triangles) of a digital mental health intervention (Wysa). Error bars represent standard errors.
Discussion
Principal Results
Intervention for mental health, which is a comorbidity that so often accompanies musculoskeletal pain, can improve both mental health- and pain-related impairment in patients who present to an orthopedic clinic for a musculoskeletal condition. To address mental health in the orthopedic setting, an acceptable, scalable, effective intervention must be available. In this pilot study, we investigated the feasibility and potential for effectiveness of delivering the digital mental health intervention Wysa in an orthopedic setting to patients with coexisting symptoms of depression and/or anxiety. The recruitment rate was 29.3% (61/208) across 6 orthopedic providers and the retention rate was 84% (51/61) with respect to completion of the 2-month outcome measures. Of the enrolled participants, 72% (44/61) engaged with Wysa at least once during the study period. Compared to low Wysa users, high Wysa users reported greater improvements by 4.2 points in PROMIS Anxiety scores at the 2-month follow-up. This difference is statistically significant (P=.044) and meets the predetermined threshold for clinical meaningfulness. Between-group differences in 2-month improvement also favored high Wysa users for PROMIS Depression (by 3.2 points) and PROMIS Pain Interference (by 2.3 points); these differences met clinically meaningful thresholds, but they did not meet statistical significance in this pilot feasibility study.
Implications
These findings suggest that a fully powered, randomized controlled trial is feasible and warranted to assess the clinical benefit of offering a digital mental health intervention in the context of orthopedic care for patients with coexisting symptoms of depression and/or anxiety. Engagement in our study was notably higher than digital health industry standards, which measure engagement on the order of days instead of weeks or months [40]. The high engagement rates and positive clinical findings in this study are consistent with those in a study of 10,000 participants by Bailey et al [41], in which people with chronic low back and/or knee pain were given a 12-week digital health intervention sponsored by their employers [41]. In that study, 73% of participants continued engagement with the program into the third month. We hypothesize that introducing a digital health intervention through a trusted resource such as a physician or employer may contribute to these higher engagement rates. Furthermore, if an orthopedic provider referred a patient to a digital mental health intervention as part of standard care (as opposed to a research study), we hypothesize that rates of onboarding to the intervention would be even greater than the enrollment rates we observed in this study. Patients who present to an orthopedic clinic for a musculoskeletal condition have demonstrated a willingness to engage with digital (physical) health interventions [42], and incorporation of digital mental health support is a logical next step to improving clinical outcomes in orthopedic care.
This line of study may also have implications for ambulatory care settings other than orthopedic clinics. We believe that clinical encounters related to chronic pain are a particularly opportune setting to address mental health because when patients understand that their physical pain experience is directly linked to their mental health, their desire to reduce pain becomes a tangible (and potentially nonstigmatizing) motivator for addressing mental health. This approach of inserting a mental health access point and resources could be applied to other frequently used ambulatory care settings that address chronic pain (eg, primary care, rheumatology, gastroenterology, headache clinics) [43], and it would create opportunities to provide mental health screening and intervention for patients with symptoms of depression and/or anxiety who might not have otherwise sought mental health treatment. In our experience, the orthopedic setting poses some especially challenging barriers to the discussion of mental health with patients, which include patients’ expectations regarding appropriate content to discuss during the clinic encounter and providers’ lack of training and possible lack of inherent interest or budgeted time to discuss mental health with patients. Therefore, evidence of the feasibility of delivering mental health intervention within the orthopedic setting is certainly encouraging for potential success in other ambulatory care settings that address chronic pain. Furthermore, although no coordination was made with patients’ established mental health care providers in this study (for those who were already receiving some form of mental health treatment), better care coordination and communication between nontraditional mental health “access points” (eg, within orthopedics) and patients’ established mental health providers should be focused on in the future.
Lessons Learned
During this feasibility study, we gained insight into participant recruitment and retention that we anticipate can be used to further improve these rates in a subsequent fully powered randomized controlled trial. Regarding participant recruitment, although every orthopedic provider that we approached was willing to assist in recruitment, it was clear that their comfort in initiating a conversation about mental health with patients varied widely. Providers who were initially less comfortable were receptive to brief training with short scripts that could be used during patient-provider discussions, and their comfort level subjectively improved through the recruitment period. The research coordinator’s conversation with patients also evolved during the study as she learned the language and content that were most acceptable and compelling to patients (ie, “this app is designed to help with stress and well-being” as opposed to “this is a mental health app”). We are pursuing further pilot work to obtain a more comprehensive understanding of (surgical and nonsurgical) orthopedic provider and patients’ needs regarding discussion of mental health in an orthopedic setting, both in a clinical and research context. This additional work will be essential to facilitate widespread patient engagement with a mental health intervention that is introduced in an orthopedic context.
Regarding participant retention, integration of phone call reminders improved the follow-up survey response rate compared to the use of email reminders alone. However, many participants were still hesitant to answer phone calls and voicemails were left when possible. For future trials, we plan to integrate SMS text messaging–based reminders as well. We will also compensate participants not for use of the intervention but for completion of each follow-up survey. We anticipate these protocol changes to further improve our recruitment and retention rates in subsequent trials.
Strengths and Limitations
The primary strength of this feasibility study is that to our knowledge, it is the first investigation to incorporate a digital mental health intervention into an orthopedic care setting. The primary limitation is that there was no true control arm because of a lack of available resources. To account for longitudinal improvements that may have resulted from the usual orthopedic care that patients received during the study period (eg, physical therapy, injections, pain medications), we compared high versus low users of Wysa in our preliminary effectiveness analysis. However, healthy participant bias could still have been present; that is, high Wysa users may have been more ready than low Wysa users to address their mental health, and the same phenomenon could have occurred between patients who chose to participate and those who did not participate in the study. Nevertheless, successful completion of this feasibility study suggests that at least some orthopedic patients are interested in receiving mental health intervention as a component of their orthopedic care.
Generalizability
While the study findings offer insight into important next steps for this research, the generalizability of the results of this current single-center study is somewhat limited. Collection of mental health screening measures is becoming more commonplace in orthopedic clinics, but it is still not performed universally [44], and collection of these measures may have facilitated the recruitment process for our study. Comfort level and success in discussing mental health in an orthopedic setting are also likely provider-dependent [12]. The recruitment of participants from among patients who presented to 6 providers at our institution improves the generalizability of our findings, but all providers were nonsurgical specialists. Furthermore, we observed variations among providers with respect to the introduction of the study to the patients and the longitudinal refinement of each provider’s approach during the study. From an implementation standpoint, further work is also needed to understand how a digital mental health intervention could most effectively and efficiently be delivered to patients who present to an orthopedic clinic for a musculoskeletal condition when a research coordinator is not available to assist the patient with onboarding to the app. We are actively investigating these provider-level and implementation-related limitations to generalizability in further pilot work.
Conclusions
In summary, this pilot study explored the feasibility and potential for effectiveness of delivering the digital mental health intervention Wysa in an orthopedic setting to patients with coexisting symptoms of depression and/or anxiety. We demonstrated that it is possible for orthopedic providers to introduce a mental health intervention to these patients. Furthermore, high users of the intervention reported greater improvement in anxiety symptoms than low users at the 2-month follow-up. High users of the intervention may also achieve meaningfully greater 2-month improvements in depression and pain interference, but this study was not powered to detect these differences. It is valuable to continue this line of research because it offers potential to improve both mental health and pain-related impairment in people with chronic musculoskeletal pain and coexisting depression and/or anxiety in a manner that is feasible and scalable to deliver in an orthopedic setting. More pilot work is needed (and is ongoing) to optimize the ability of orthopedic providers to efficiently discuss mental health with these patients in a manner that is mutually acceptable. Ultimately, a fully powered randomized controlled trial is warranted, and this approach of inserting mental health access points and resources into non–mental health ambulatory care settings could also be considered in primary care and other specialty clinics that often address chronic pain.
Acknowledgments
This study was funded by a Washington University/BJC HealthCare Big Ideas innovation grant. Work on this study was also supported by the National Institutes of Health (K23AR074520) and the Doris Duke Charitable Foundation (to ALC). The study team purchased participant access codes for Wysa’s novel version tailored for people with chronic pain (which is not commercially available). Wysa provided participants’ deidentified app usage data to the study team free of charge.
Abbreviations
- PROMIS
Patient-Reported Outcomes Measurement Information System
- REDCap
Research Electronic Data Capture
Distribution of therapeutic mechanisms underlying the Wysa tools used by 61 patients who presented to an orthopedic clinic for a musculoskeletal condition and coexisting symptoms of depression and anxiety. CBT: cognitive behavioral therapy.
Footnotes
Authors' Contributions: ALC conceived the study idea, supervised the entire project, and contributed to writing the final version of the manuscript. DMH and JP Metzler contributed to participant recruitment, data collection, and editing of the manuscript. MAA performed all participant enrollment, contributed to follow-up data collection, and edited the final manuscript. AJL contributed to follow-up data collection and writing of the manuscript. PAA and JP Miller contributed to the statistical methods, data presentation, and editing of the final manuscript. MJS performed the statistical analysis and contributed to writing of the manuscript.
Conflicts of Interest: None declared.
References
- 1.Zelaya C, Dahlhamer J, Lucas J, Connor E. Chronic pain and high-impact chronic pain among U.S. adults, 2019. NCHS Data Brief. 2020 Nov;(390):1–8. [PubMed] [Google Scholar]
- 2.Bair MJ, Robinson RL, Katon W, Kroenke K. Depression and pain comorbidity: a literature review. Arch Intern Med. 2003 Nov 10;163(20):2433–45. doi: 10.1001/archinte.163.20.2433.163/20/2433 [DOI] [PubMed] [Google Scholar]
- 3.McWilliams LA, Cox BJ, Enns MW. Mood and anxiety disorders associated with chronic pain: an examination in a nationally representative sample. Pain. 2003 Nov;106(1-2):127–33. doi: 10.1016/s0304-3959(03)00301-4.S0304395903003014 [DOI] [PubMed] [Google Scholar]
- 4.Yelin E, Weinstein S, King T. An update on the burden of musculoskeletal diseases in the U.S. Semin Arthritis Rheum. 2019 Aug;49(1):1–2. doi: 10.1016/j.semarthrit.2019.04.010.S0049-0172(19)30175-1 [DOI] [PubMed] [Google Scholar]
- 5.Gambassi G. Pain and depression: the egg and the chicken story revisited. Arch Gerontol Geriatr. 2009;49 Suppl 1:103–12. doi: 10.1016/j.archger.2009.09.018.S0167-4943(09)00218-0 [DOI] [PubMed] [Google Scholar]
- 6.DeVeaugh-Geiss AM, West SL, Miller WC, Sleath B, Gaynes BN, Kroenke K. The adverse effects of comorbid pain on depression outcomes in primary care patients: results from the ARTIST trial. Pain Med. 2010 May;11(5):732–41. doi: 10.1111/j.1526-4637.2010.00830.x.PME830 [DOI] [PubMed] [Google Scholar]
- 7.Ang DC, Bair MJ, Damush TM, Wu J, Tu W, Kroenke K. Predictors of pain outcomes in patients with chronic musculoskeletal pain co-morbid with depression: results from a randomized controlled trial. Pain Med. 2010 Apr;11(4):482–91. doi: 10.1111/j.1526-4637.2009.00759.x.PME759 [DOI] [PubMed] [Google Scholar]
- 8.Cheng AL, Schwabe M, Doering MM, Colditz GA, Prather H. The effect of psychological impairment on outcomes in patients with prearthritic hip disorders: a systematic review and meta-analysis. Am J Sports Med. 2020 Aug;48(10):2563–2571. doi: 10.1177/0363546519883246. [DOI] [PubMed] [Google Scholar]
- 9.Edwards RR, Klick B, Buenaver L, Max MB, Haythornthwaite JA, Keller RB, Atlas SJ. Symptoms of distress as prospective predictors of pain-related sciatica treatment outcomes. Pain. 2007 Jul;130(1-2):47–55. doi: 10.1016/j.pain.2006.10.026.S0304-3959(06)00593-8 [DOI] [PubMed] [Google Scholar]
- 10.Lee H, Hübscher Markus, Moseley GL, Kamper SJ, Traeger AC, Mansell G, McAuley JH. How does pain lead to disability? A systematic review and meta-analysis of mediation studies in people with back and neck pain. Pain. 2015 Jun;156(6):988–997. doi: 10.1097/j.pain.0000000000000146.00006396-201506000-00004 [DOI] [PubMed] [Google Scholar]
- 11.Alattas SA, Smith T, Bhatti M, Wilson-Nunn D, Donell S. Greater pre-operative anxiety, pain and poorer function predict a worse outcome of a total knee arthroplasty. Knee Surg Sports Traumatol Arthrosc. 2017 Nov;25(11):3403–3410. doi: 10.1007/s00167-016-4314-8. http://europepmc.org/abstract/MED/27734110 .10.1007/s00167-016-4314-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Vranceanu AM, Beks RB, Guitton TG, Janssen SJ, Ring D. How do orthopaedic surgeons address psychological aspects of illness? Arch Bone Jt Surg. 2017 Jan;5(1):2–9. http://europepmc.org/abstract/MED/28271080 . [PMC free article] [PubMed] [Google Scholar]
- 13.Andrade LH, Alonso J, Mneimneh Z, Wells JE, Al-Hamzawi A, Borges G, Bromet E, Bruffaerts R, de Girolamo G, de Graaf R, Florescu S, Gureje O, Hinkov HR, Hu C, Huang Y, Hwang I, Jin R, Karam EG, Kovess-Masfety V, Levinson D, Matschinger H, O'Neill S, Posada-Villa J, Sagar R, Sampson NA, Sasu C, Stein DJ, Takeshima T, Viana MC, Xavier M, Kessler RC. Barriers to mental health treatment: results from the WHO World Mental Health surveys. Psychol Med. 2014 Apr;44(6):1303–17. doi: 10.1017/S0033291713001943. http://europepmc.org/abstract/MED/23931656 .S0033291713001943 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Kozloff N, Sommers BD. Insurance coverage and health outcomes in young adults with mental illness following the Affordable Care Act dependent coverage expansion. J Clin Psychiatry. 2017 Jul;78(7):e821–e827. doi: 10.4088/JCP.16m11357. [DOI] [PubMed] [Google Scholar]
- 15.Weil TP. Insufficient dollars and qualified personnel to meet United States mental health needs. J Nerv Ment Dis. 2015 Apr;203(4):233–40. doi: 10.1097/NMD.0000000000000271.00005053-201504000-00001 [DOI] [PubMed] [Google Scholar]
- 16.Kretzschmar K, Tyroll H, Pavarini G, Manzini A, Singh I, NeurOx Young People’s Advisory Group Can your phone be your therapist? Young people's ethical perspectives on the use of fully automated conversational agents (chatbots) in mental health support. Biomed Inform Insights. 2019;11:1178222619829083. doi: 10.1177/1178222619829083. https://journals.sagepub.com/doi/10.1177/1178222619829083?url_ver=Z39.88-2003&rfr_id=ori:rid:crossref.org&rfr_dat=cr_pub%3dpubmed .10.1177_1178222619829083 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Carlbring P, Andersson G, Cuijpers P, Riper H, Hedman-Lagerlöf Erik. Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: an updated systematic review and meta-analysis. Cogn Behav Ther. 2018 Jan;47(1):1–18. doi: 10.1080/16506073.2017.1401115. [DOI] [PubMed] [Google Scholar]
- 18.Firth J, Torous J, Nicholas J, Carney R, Rosenbaum S, Sarris J. Can smartphone mental health interventions reduce symptoms of anxiety? A meta-analysis of randomized controlled trials. J Affect Disord. 2017 Aug 15;218:15–22. doi: 10.1016/j.jad.2017.04.046. https://linkinghub.elsevier.com/retrieve/pii/S0165-0327(17)30015-0 .S0165-0327(17)30015-0 [DOI] [PubMed] [Google Scholar]
- 19.Linardon J, Cuijpers P, Carlbring P, Messer M, Fuller-Tyszkiewicz M. The efficacy of app-supported smartphone interventions for mental health problems: a meta-analysis of randomized controlled trials. World Psychiatry. 2019 Oct;18(3):325–336. doi: 10.1002/wps.20673. doi: 10.1002/wps.20673. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Mohr DC, Burns MN, Schueller SM, Clarke G, Klinkman M. Behavioral intervention technologies: evidence review and recommendations for future research in mental health. Gen Hosp Psychiatry. 2013;35(4):332–8. doi: 10.1016/j.genhosppsych.2013.03.008. https://linkinghub.elsevier.com/retrieve/pii/S0163-8343(13)00069-8 .S0163-8343(13)00069-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Abd-Alrazaq AA, Alajlani M, Ali N, Denecke K, Bewick BM, Househ M. Perceptions and opinions of patients about mental health chatbots: scoping review. J Med Internet Res. 2021 Jan 13;23(1):e17828. doi: 10.2196/17828. https://www.jmir.org/2021/1/e17828/ v23i1e17828 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Torous J, Bucci S, Bell IH, Kessing LV, Faurholt-Jepsen M, Whelan P, Carvalho AF, Keshavan M, Linardon J, Firth J. The growing field of digital psychiatry: current evidence and the future of apps, social media, chatbots, and virtual reality. World Psychiatry. 2021 Oct;20(3):318–335. doi: 10.1002/wps.20883. doi: 10.1002/wps.20883. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Wysa Mental Health Support, For Everyone. Wysa. [2021-03-10]. http://wysa.io .
- 24.Inkster B, Sarda S, Subramanian V. An empathy-driven, conversational artificial intelligence agent (Wysa) for digital mental well-being: real-world data evaluation mixed-methods study. JMIR Mhealth Uhealth. 2018 Nov 23;6(11):e12106. doi: 10.2196/12106. https://mhealth.jmir.org/2018/11/e12106/ v6i11e12106 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Burkhardt HA, Alexopoulos GS, Pullmann MD, Hull TD, Areán Patricia A, Cohen T. Behavioral activation and depression symptomatology: longitudinal assessment of linguistic indicators in text-based therapy sessions. J Med Internet Res. 2021 Jul 14;23(7):e28244. doi: 10.2196/28244. https://www.jmir.org/2021/7/e28244/ v23i7e28244 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Jensen MP, Ehde DM, Day MA. The behavioral activation and inhibition systems: implications for understanding and treating chronic pain. J Pain. 2016 May;17(5):529.e1–529.e18. doi: 10.1016/j.jpain.2016.02.001.S1526-5900(16)00505-8 [DOI] [PubMed] [Google Scholar]
- 27.Kratz AL, Davis MC, Zautra AJ. Pain acceptance moderates the relation between pain and negative affect in female osteoarthritis and fibromyalgia patients. Ann Behav Med. 2007 Jun;33(3):291–301. doi: 10.1007/BF02879911. http://europepmc.org/abstract/MED/17600456 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.McCracken LM. Learning to live with the pain: acceptance of pain predicts adjustment in persons with chronic pain. Pain. 1998 Jan;74(1):21–27. doi: 10.1016/S0304-3959(97)00146-2.00006396-199801010-00004 [DOI] [PubMed] [Google Scholar]
- 29.Cella D, Yount S, Rothrock N, Gershon R, Cook K, Reeve B, Ader D, Fries JF, Bruce B, Rose M, PROMIS Cooperative Group The Patient-Reported Outcomes Measurement Information System (PROMIS): progress of an NIH Roadmap cooperative group during its first two years. Med Care. 2007 May;45(5 Suppl 1):S3–S11. doi: 10.1097/01.mlr.0000258615.42478.55. http://europepmc.org/abstract/MED/17443116 .00005650-200705001-00002 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Smith BW, Dalen J, Wiggins K, Tooley E, Christopher P, Bernard J. The brief resilience scale: assessing the ability to bounce back. Int J Behav Med. 2008;15(3):194–200. doi: 10.1080/10705500802222972.901471909 [DOI] [PubMed] [Google Scholar]
- 31.Zimmerman GL, Olsen CG, Bosworth MF. A 'stages of change' approach to helping patients change behavior. Am Fam Physician. 2000 Mar 01;61(5):1409–16. https://www.aafp.org/link_out?pmid=10735346 . [PubMed] [Google Scholar]
- 32.Harris PA, Taylor R, Minor BL, Elliott V, Fernandez M, O'Neal L, McLeod L, Delacqua G, Delacqua F, Kirby J, Duda SN, REDCap Consortium The REDCap consortium: Building an international community of software platform partners. J Biomed Inform. 2019 Jul;95:103208. doi: 10.1016/j.jbi.2019.103208. https://linkinghub.elsevier.com/retrieve/pii/S1532-0464(19)30126-1 .S1532-0464(19)30126-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009 Apr;42(2):377–81. doi: 10.1016/j.jbi.2008.08.010. https://linkinghub.elsevier.com/retrieve/pii/S1532-0464(08)00122-6 .S1532-0464(08)00122-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Lee AC, Driban JB, Price LL, Harvey WF, Rodday AM, Wang C. Responsiveness and Minimally Important Differences for 4 Patient-Reported Outcomes Measurement Information System Short Forms: Physical Function, Pain Interference, Depression, and Anxiety in Knee Osteoarthritis. J Pain. 2017 Sep;18(9):1096–1110. doi: 10.1016/j.jpain.2017.05.001. http://europepmc.org/abstract/MED/28501708 .S1526-5900(17)30571-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Kroenke K, Yu Z, Wu J, Kean J, Monahan PO. Operating characteristics of PROMIS four-item depression and anxiety scales in primary care patients with chronic pain. Pain Med. 2014 Nov;15(11):1892–901. doi: 10.1111/pme.12537. http://europepmc.org/abstract/MED/25138978 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Chen CX, Kroenke K, Stump TE, Kean J, Carpenter JS, Krebs EE, Bair MJ, Damush TM, Monahan PO. Estimating minimally important differences for the PROMIS pain interference scales: results from 3 randomized clinical trials. Pain. 2018 Apr;159(4):775–782. doi: 10.1097/j.pain.0000000000001121. http://europepmc.org/abstract/MED/29200181 .00006396-201804000-00017 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Kind AJH, Jencks S, Brock J, Yu M, Bartels C, Ehlenbach W, Greenberg C, Smith M. Neighborhood socioeconomic disadvantage and 30-day rehospitalization: a retrospective cohort study. Ann Intern Med. 2014 Dec 02;161(11):765–74. doi: 10.7326/M13-2946. http://europepmc.org/abstract/MED/25437404 .1983380 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Singh GK. Area deprivation and widening inequalities in US mortality, 1969-1998. Am J Public Health. 2003 Jul;93(7):1137–43. doi: 10.2105/ajph.93.7.1137. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Bertholet N, Gaume J, Faouzi M, Gmel G, Daeppen J. Predictive value of readiness, importance, and confidence in ability to change drinking and smoking. BMC Public Health. 2012 Aug 29;12:708. doi: 10.1186/1471-2458-12-708. https://bmcpublichealth.biomedcentral.com/articles/10.1186/1471-2458-12-708 .1471-2458-12-708 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Baumel A, Muench F, Edan S, Kane JM. Objective user engagement with mental health apps: systematic search and panel-based usage analysis. J Med Internet Res. 2019 Sep 25;21(9):e14567. doi: 10.2196/14567. https://www.jmir.org/2019/9/e14567/ v21i9e14567 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Bailey JF, Agarwal V, Zheng P, Smuck M, Fredericson M, Kennedy DJ, Krauss J. Digital care for chronic musculoskeletal pain: 10,000 participant longitudinal cohort study. J Med Internet Res. 2020 May 11;22(5):e18250. doi: 10.2196/18250. https://www.jmir.org/2020/5/e18250/ v22i5e18250 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Hewitt S, Sephton R, Yeowell G. The effectiveness of digital health interventions in the management of musculoskeletal conditions: systematic literature review. J Med Internet Res. 2020 Jun 05;22(6):e15617. doi: 10.2196/15617. https://www.jmir.org/2020/6/e15617/ v22i6e15617 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Bartels SJ, Coakley EH, Zubritsky C, Ware JH, Miles KM, Areán Patricia A, Chen H, Oslin DW, Llorente MD, Costantino G, Quijano L, McIntyre JS, Linkins KW, Oxman TE, Maxwell J, Levkoff SE, PRISM-E Investigators Improving access to geriatric mental health services: a randomized trial comparing treatment engagement with integrated versus enhanced referral care for depression, anxiety, and at-risk alcohol use. Am J Psychiatry. 2004 Aug;161(8):1455–62. doi: 10.1176/appi.ajp.161.8.1455.161/8/1455 [DOI] [PubMed] [Google Scholar]
- 44.Ring D. Mental and social health are inseparable from physical health. J Bone Joint Surg Am. 2021 Jun 02;103(11):951–952. doi: 10.2106/JBJS.21.00121.00004623-202106020-00001 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Distribution of therapeutic mechanisms underlying the Wysa tools used by 61 patients who presented to an orthopedic clinic for a musculoskeletal condition and coexisting symptoms of depression and anxiety. CBT: cognitive behavioral therapy.