Summary Paragraph
While digital psychiatric interventions reduce treatment barriers, not all persons benefit from this type of treatment. Research is needed to preemptively identify who is likely to benefit from these digital treatments in order to redirect those people to a higher level of care. The current manuscript used an ensemble of machine learning methods to predict changes in major depressive and generalized anxiety disorder symptoms from pre to 9-month follow-up in a randomized controlled trial of a transdiagnostic digital intervention based on participants’ (N=632) pre-treatment data. The results suggested that baseline characteristics could accurately predict changes in depressive symptoms in both treatment groups (r=0.482, 95% CI[0.394, 0.561]; r=0.477, 95% CI[0.385, 0.560]) and anxiety symptoms in both treatment groups (r=0.569, 95% CI[0.491, 0.638]; r=0.548, 95% CI[0.464, 0.622]). These results suggest that machine learning models are capable of preemptively predicting a person’s responsiveness to digital treatments, which would enable personalized decision-making about which persons should be directed towards standalone digital interventions or towards blended stepped-care.
Keywords: digital therapeutics, digital intervention, personalized, artificial intelligence, machine learning, depression, anxiety
Anxiety and depressive disorders occur in one-fifth of the population (Steel et al., 2014). Traditional care systems are unable to meet the sizeable patient population—most persons with psychiatric diagnoses receive no specialized treatment and wait years before initiating care and then months before care begins (Wilhelm et al., 2019). Digital interventions may help to close this substantial treatment gap. Nevertheless, many persons may respond differently to digital interventions. Some persons may respond well to digital interventions, whereas others may be unresponsive to these digital interventions and require further traditional care. Consequently, research is needed to determine whether responses to these digital interventions can be predicted before digital treatment initiation to facilitate timely and efficient treatments. Three prior studies have examined predicting symptom changes in digital treatments using machine learning with early promising results in obsessive-compulsive disorder (N = 61) (Lenhard et al., 2018), depression (N = 283) (Pearson et al., 2019), and social anxiety (N = 26) (Månsson et al., 2015). Although promising, prior approaches to predicting responsiveness have examined a limited range of potential machine learning models and utilized smaller samples. The current trial examined a comprehensive ensemble of machine learning methods to predict changes in anxiety and depressive symptoms in a transdiagnostic digital interventions with a sample of 632 persons.
Method
Randomized Controlled Trial.
This study is based on a sample of 632 persons who received a digital intervention following discharge from an inpatient treatment for cardiologic, psychosomatic, and/or orthopedic rehabilitation surrounding work-related stress (Zwerenz et al., 2017). Participants were randomized to active intervention groups with one group receiving online supportive expressive therapy with feedback from a therapist (N = 303) and the other group receiving information related to stress management and coping (N = 329) (Zwerenz et al., 2017). The current study predicted changes in depressive and anxiety symptoms separately in each treatment group based on the Patient Health Questionnaire-9 and Generalized Anxiety Disorder-7 item questionnaire completed before the intervention and after a 9-month follow-up.1 Predictors in the machine learning models included responses to a variety of self-reported questionnaires occurring before the initiation of treatment (e.g. work capacity, perfectionism, calmness, need for occupational treatment, problem-solving, ambition, resignation, anxiety and depressive symptoms, social support, pension, education, sex, age, willingness to change, among others) (Zwerenz et al., 2017).
Data Analysis.
Data preprocessing was done in R and machine learning modeling was performed in python with sklearn. The current application was based on an ensemble of many machine learning models, which routinely demonstrate superiority and robustness compared to single-model solutions. The ensemble model consisted of two layers: (1) base models including a large variety of neural network models, ridge regression, random forests, general linear models, gaussian process, extreme gradient boosting, k-nearest neighbors, and support vector machine; and (2) an averaging layer that took the mean prediction for a given subject across models and validation folds. Notably, all models were trained on entirely out-of-sample nested cross-validation folds, which minimized overfitting (no hyperparameter tuning was done with subjects in the test fold).
Results
The ensemble predicted change in depressive symptoms in both treatment groups (r=0.482, 95% CI[0.394, 0.561]; r=0.477, 95% CI[0.385, 0.560]) and anxiety symptoms in both treatment groups (r=0.569, 95% CI[0.491, 0.638]; r=0.548, 95% CI[0.464, 0.622])with high accuracy (Figure 1). This accuracy was increased an average of 11.9% over a single base learner. Additionally, the ensemble accuracy was increased on average 6.00 % over a theoretically guided set of predictors (age, gender, medication status, and baseline severity) utilized in a GLM.
Figure 1.
Observed and predicted change in the Patient Health Questionnaire-9 and Generalized Anxiety Disorder-7 symptoms in both treatments.
Discussion
These results suggest that inferring treatment prognosis of changes in anxiety and depressive symptoms can be predicted using pre-treatment data alone. By delivering this information to clinicians, these machine learning models may be utilized to help guide future decision-making between low-resource digital interventions or higher levels of traditional care. The success of these predictions suggest that these models could guide recommendations about the appropriate level of digital or traditional care before any care commences, saving patients ‘limited time, energy, and resources while immediately triaging of patients to the appropriate levels of care. It may also save providers’ time by only seeing those patients who are unlikely to benefit from digital interventions.
Highlights.
Not all persons benefit equally to digital psychiatric interventions.
Effectiveness of digital treatments can be predicted using pre-treatment data.
Predictions could inform future treatment-decision making.
Acknowledgements:
The current manuscript utilizes public-use data made available from Zwerenz et al.
Funding: This research received no specific grant from any funding agency, commercial or not-for-profit sectors.
Footnotes
Declaration of Interests: Dr. Jacobson has no financial disclosures. However, Dr. Jacobson is the owner of a free application published on the Google Play Store entitled “Mood Triggers”. He does not receive any direct or indirect revenue from his ownership of the application (i.e. the application is free, there are no advertisements, and the data is only being used for research purposes).
Author Statement
NCJ conceptualized the study and wrote the first draft of the manuscript. NCJ and MDN both analyzed the data and revised the manuscript.
Because this is anonymized public use data, it is not considered human subject research and does not require institutional review board approval. All participants gave written consent to participate in the study (Zwerenz et al., 2017).
Conflicts of Interest The authors have no conflicts to declare.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- Lenhard F, Sauer S, Andersson E, Månsson KN, Mataix-Cols D, Rück C, Serlachius E, 2018. Prediction of outcome in internet-delivered cognitive behaviour therapy for paediatric obsessive-compulsive disorder: A machine learning approach. Int J Methods Psychiatr Res 27 10.1002/mpr.1576 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Månsson KNT, Frick A, Boraxbekk C-J, Marquand AF, Williams SCR, Carlbring P, Andersson G, Furmark T, 2015. Predicting long-term outcome of Internet-delivered cognitive behavior therapy for social anxiety disorder using fMRI and support vector machine learning. Transl Psychiatry 5, e530–e530. 10.1038/tp.2015.22 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pearson R, Pisner D, Meyer B, Shumake J, Beevers CG, 2019. A machine learning ensemble to predict treatment outcomes following an Internet intervention for depression. Psychol Med 49, 2330–2341. 10.1017/S003329171800315X [DOI] [PMC free article] [PubMed] [Google Scholar]
- Steel Z, Marnane C, Iranpour C, Chey T, Jackson JW, Patel V, Silove D, 2014. The global prevalence of common mental disorders: A systematic review and meta-analysis 1980–2013. International journal of epidemiology 43, 476–93. 10.1093/ije/dyu038 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wilhelm S, Weingarden H, Ladis I, Braddick V, Shin J, Jacobson NC, 2019. Cognitive-Behavioral Therapy in the Digital Age: Presidential Address. Behavior Therapy. 10.1016/j.beth.2019.08.001 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Zwerenz R, Becker J, Gerzymisch K, Siepmann M, Holme M, Kiwus U, Spörl-Dönch S, Beutel ME, 2017. Evaluation of a transdiagnostic psychodynamic online intervention to support return to work: A randomized controlled trial. PLoS ONE 12, e0176513. 10.1371/journal.pone.0176513 [DOI] [PMC free article] [PubMed] [Google Scholar]