Abstract
Background
The SARS-CoV-2 pandemic compromised the mental health of COVID-19 patients and their family members. Due to social distancing and lockdown measures, a remote, tele-psychotherapy program for former or current COVID-19 patients and their relatives was implemented.
Objective
The primary goal of this project was to evaluate intervention feasibility. The secondary aim was to assess the impact of the intervention by means of pre-post psychological changes.
Methods
After a phone-based eligibility screening and remote neuropsychological testing, participants completed online self-reports assessing baseline COVID-related psychopathology. Next, participants attended eight tele-psychotherapy sessions. After treatment, the online self-reports were completed again.
Results
Of 104 enrolled participants, 88 completed the intervention (84.6 % completion rate). Significant pre-post improvements were observed for generalized anxiety (d = 0.38), depression (d = 0.37), insomnia (d = 0.43), post-traumatic psychopathology (d = 0.54), and general malaise (d = 0.31). Baseline cluster analysis revealed a subgroup of 41 subjects (47.6 %) with no psychopathology, and a second subgroup of 45 subject (52.3 %) with moderate severity. Thirty-three percent of the second group reached full symptom remission, while 66 % remained symptomatic after treatment.
Conclusions
Remote brief tele-psychotherapy for COVID-19 patients and their first-degree relatives is feasible and preliminary efficacious at reducing COVID-related psychopathology in a subgroup of patients. Further research is needed to investigate distinct profiles of treatment response.
Keywords: Telepsychiatry, Telemedicine, COVID-19
1. Introduction
The Severe Acute Respiratory Syndrome coronavirus 2 (SARS-CoV-2) pandemic has been, for the past two years, an unprecedented threat to global public health. While two other coronaviruses have caused world-wide health issues in the past decades, namely the Middle East Respiratory Syndrome (MERS) and SARS, neither of them has spread as much as SARS-CoV-2 (Al-Hazmi, 2016). As SARS-CoV-2 spreads rapidly through respiratory particles, >500 million people all over the world to date have been infected by SARS-CoV-2 (https://covid19.who.int/). To counteract the rapid transmission of the coronavirus disease (COVID-19), most governments have adopted restrictive measures (e.g., lockdown) and social distancing during the various phases of the pandemic.
The clinical manifestations of the SARS-CoV-2 infection range from asymptomatic or paucisymptomatic forms (Gandhi et al., 2020), mostly characterized by common cold symptoms (e.g. cough, fever, gastrointestinal symptoms, anosmia), to more severe presentations such as pneumonia with respiratory failure, or multiorgan and systemic dysfunctions that can all lead to death (Zaim et al., 2020). Although COVID-19 was initially thought to selectively affect the respiratory system, it soon became clear that the disease could interest other organs, including the central nervous system (CNS). Several mechanisms have been proposed for the involvement of the CNS during COVID-19, including the cytokine storm, the breakdown of the blood-brain barrier, and the innate neuroimmune responses (Balcom et al., 2021). Interestingly, even when respiratory symptoms are mild, long-term cognitive impairments such as executive, attentional and memory deficits (Voruz et al., 2022), as well as a general state of malaise characterized by tiredness and fatigue (Kamal et al., 2021) have been reported.
Above and beyond cognitive symptoms, researchers and clinicians have attempted to characterize the heterogeneous cluster of psychopathological symptoms that typically emerge when patients are no longer positive to SARS-CoV-2 infection, and can persist for weeks if not months (Mazza et al., 2022; Raveendran et al., 2021). These symptoms include depression, anxiety, acute/post-traumatic stress, and sleep disturbances. While it was initially proposed that COVID-19-related psychopathology was the direct result of the involvement of the CNS during the infection, because these symptoms have been frequently observed in COVID-19 patients as well as in their first-degree relatives, across age ranges (Belz et al., 2022; Cai et al., 2020), it is now well documented that social and environmental factors associated with the SARS-CoV-2 pandemic, such as lockdown measures, social distancing, fear or experience of contagion, and fear or experience of losing loved ones, can trigger new psychopathological symptoms or exacerbate existing ones (Adams-Prassl et al., 2022; Cullen et al., 2020). COVID-19-related psychopathology maps onto well-established psychopathological macro-areas: 1) trauma and post-traumatic symptoms, mainly for those who have experienced hospitalization or loss of loved ones; 2) adaptation and functional reorientation due to physical complications or sequelae; 3) identity reorientation following the experience of illness/isolation/lockdown; and 4) exacerbation of premorbid psychopathological traits solicited by the COVID-19 experience. Notably, the severity of COVID-19-related psychopathology ranges from mild to more disabling conditions, and affects youth, adults, and elders irrespective of the severity of the acute COVID-19 illness (Cannito et al., 2020). In the vast majority of cases, it unequivocally affecting wellbeing, quality of life, and real-world functioning.
In response to the urgent need for treatments that could be offered safely without burdening an already strained national mental health system, and in compliance with the strict social distancing rules that were in place throughout the pandemic, a research-informed infrastructure was created with the goal of developing and studying the feasibility and efficacy of a remote treatment approach for COVID-19 patients and their first-degree relatives (Cantù et al., 2022). The harmonization of techniques and strategies deriving from several psychotherapeutic orientations (psychodynamic therapy, constructivist therapy and hermeneutic-phenomenological therapy) culminated in the development of the first national brief tele-psychotherapy program targeting COVID-19 related psychopathology (Biagianti et al., 2021). The program, designed to easily integrate with the workflow of the national mental health system, consists of 8 remote, 50-min, individual psychotherapy sessions that are offered weekly using secure video conferencing software. In this paper, we report findings from the single arm, naturalistic study that evaluated the feasibility, and preliminary efficacy of this brief tele-psychotherapy approach.
2. Methods
2.1. Study participants
This study was approved by the Local Ethical Committee (Comitato Etico Milano Area 2, approval #865_2020). Our population of interest included individuals with confirmed SARS-CoV-2 infection by positive nasopharyngeal swab or serology (patients group) and first-degree relatives of COVID-19 patients (relatives group) who met the inclusion and exclusion criteria outlined below (see Procedures section). Study participation was voluntary.
Participants were recruited as part of two ongoing studies: the first one was funded by Regione Lombardia, Cariplo e Fondazione Umberto Veronesi (Award Number: 2020-1366) and conducted by the University of Milan in collaboration with the Foundation IRCCS Ca’ Granda Ospedale Maggiore Policlinico and the Laboratory of Brain Injury and Therapeutic Strategies of the Istituto di Ricerche Farmacologiche Mario Negri. In this study, participants were recruited and/or referred from the Intensive, and Sub-Intensive Care Units, Pneumology, or Internal Medicine Departments of the Foundation IRCCS Ca' Granda Ospedale Maggiore Policlinico in Milan, Italy. Furthermore, participants were recruited from the three Psychosocial Centers associated with the Psychiatry Department of the same hospital. The second study was funded by Fondo Integrativo Speciale Ricerca (Award Number: FISR2020IP_05157) and conducted by the University of Verona. In this study, participants were patients requesting psychological/psychiatric support at the Section of Psychiatry of the Azienda Ospedaliera Universitaria Integrata of Verona. The two projects completely overlap in terms of inclusion and exclusion criteria, study procedures, experimental intervention, study outcomes, and delivery methods.
2.2. Procedures
All participants were initially contacted via phone by study coordinators or research assistants, who offered an initial description of study aims and procedures. If the prospective participant confirmed their interest, an initial telephonic screening was performed to assess eligibility based on a priori defined inclusion and exclusion criteria. Inclusion criteria were: (i) being between 18 and 80 years of age; (ii) having a positive SARS-CoV-2 test (nasopharyngeal swab or serology) for subjects in the patients group; (iii) having a relative currently hospitalized for COVID-19 for subjects in the relatives group, and no history of a positive SARS-CoV-2 test; (iv) having adequate sensory and motor abilities, without impairments in vision or hearing that would not allow to handle and use devices for video calls; (v) having access to Internet wireless technologies; and (vi) having good spoken and written Italian. Exclusion criteria were: (i) present or past medical history of schizophrenia, schizoaffective disorder, delusional disorder, bipolar disorder, current substance abuse, according to the diagnostic and statistical manual fifth edition (DSM-5); (ii) current diagnosis of cognitive impairment and/or dementia (e.g. Mild Cognitive Impairment, Alzheimer's disease, Parkinson's Disease); (iii) a severe medical condition which would interfere with participation; and (iv) being currently enrolled in other clinical trials assessing any psychological, or experimental pharmacological treatment.
If participants were deemed eligible, an informed consent was remotely obtained via digital signature. Next, a remote visit was scheduled with a trained neuropsychologist via Zoom to assess participants' verbal and visual intelligence quotient (IQ). The Test di Intelligenza Breve (Sartori et al., 1997) and the Standard Progressive Raven Matrices (Raven and Raven, 2003) were used, respectively. The Test di Intelligenza Breve outputs a Verbal IQ subscore (QIV), a Performance IQ subscore (QIP), and a Total IQ score (QIT). Four different neuropsychologists participated in a 2-h training procedure. This training procedure utilized a protocol similar to one used in previous interrater reliability studies. The four neuropsychologists independently learned the scoring procedures. The accuracy of each rater following the training procedure was also evaluated by the study PIs (BB, PB). Following the training procedure, the neuropsychologists were asked to score two videotaped administrations; one that was relatively straightforward to score, and another that had more ambiguous test performance. Excellent interrater agreement and accuracy were found on the quantitative scoring on both tapes.
Finally, the presence of suicidal ideation was investigated through the Columbia-Suicide Severity Rating Scale (C-SSRS, Posner et al., 2008). If participants had either a total IQ < 70 (definable as an intellectual disability; Harris and Greenspan, 2016) or a C-SSRS total score >0, they were excluded from the study. In cases of suicidal ideation, excluded participants were promptly referred to a site psychiatrist.
Once eligibility was fully confirmed with the neuropsychological assessment, participants were sent an email link to complete a battery of online self-report questionnaires. The battery included: the General Anxiety Disorder-7 questionnaire (GAD-7; Williams, 2014), which evaluates the severity of anxiety symptoms; the Patient Health Questionnaire (PHQ-9; Kroenke et al., 2001), which evaluates the severity of depressive symptoms;; the Insomnia Severity Index (ISI; Bastien, 2001), which assesses sleep disturbances; the Impact of Event Scale-Revised (IES-R; Christianson and Marren, 2012), which evaluated traumatic and post-traumatic symptoms; and the General Health Questionnaire (GHQ-12; Gureje and Obikoya, 1990), which evaluates mental wellbeing and distress.
After completing these questionnaires, participants were remotely assigned to a psychotherapist who scheduled eight tele-therapy sessions that were carried out through Zoom.
Four psychotherapists were involved in treatment delivery. Such delivery required technical and methodological adaptations of the psychotherapeutic process that emerged during the remote clinical work, and that were discussed via weekly remote group supervisions. Each case enrolled in the study was presented and discussed during the weekly group supervision for quality control and treatment fidelity purposes.
Regarding the use of face masks, recent literature reports that interactions with a mask-wearing individual modify the participant's perception of trustworthiness/untrustworthiness and subsequent behavior (Cannito et al., 2022). Therefore, since therapists and research personnel engaged in study procedures from safe, socially distanced environments, they did not use face masks. Similarly, study participants were encouraged to expose their face online as long as it was safe for them to do so. The choice was supported by evidence suggesting that facial expressions in the therapeutic dyad offer a framework to mutually collect information about personality aspects, processes that pertain to self-image and others-perception, as well as emotional, cognitive, and behavioral responses (Ramseyer and Tschacher, 2014). At the end of the 8-week cycle, participants were asked to repeat the battery of online self-report questionnaire, to assess changes in participants' psychopathological symptoms.
2.3. Intervention
The eight-session model is inspired by constructivist and hermeneutic-phenomenological theories, and has been specifically designed to i) support COVID-19 patients during all phases characteristic of the COVID-19 infection, hospitalization, remission and recovery; and ii) assist individuals that dealt with the COVID-19 infection of a family member, or experienced the loss of a family member because of COVID-19. The structure and rationale behind the topics covered in each session has been described elsewhere (Biagianti et al., 2021). Here, we briefly report the contents and goal of each session. In Session 1, the psychotherapist investigates the patient's current experience space and tries to identify the areas of suffering; this included the collection of information regarding premorbid health conditions, COVID-19 clinical severity, and type of hospitalization. In Session 2, the therapist attempts to define shared goals for the therapeutic process and to create an initial diagnostic framework. Session 3 is characterized by the validation of intrapsychic and interpersonal resources associated with a greater degree of adaptation to the stressful situation. Sessions 4–6 are focused on validation of traumatic and post-traumatic symptoms, regulation techniques and resource consolidation, as well as grief and triggered or exacerbated psychiatric symptoms. In Session 7, the therapist attempts to integrate the lived experience in the cohesive narrative of self, by recognizing the patient's emotions/behaviors experienced during the acute phase as their own. Finally, in Session 8 the therapist provides a summary of the therapeutic strategies that have been discussed during the past sessions.
2.4. Outcome measures
2.4.1. Primary outcome measures
The primary goal of the project was to evaluate the feasibility of a brief tele-psychotherapy model for COVID-19 patients and their relatives. This was assessed through the following primary outcome measures: (i) completion rate for the remote neuropsychological testing session; (ii) completion rate for online questionnaires; (iii) completion rate for tele-psychotherapy; (iv) reported side effects (raw score); and (v) overall program completion rate.
2.4.2. Secondary outcome measures
The secondary goal of this project was to evaluate the preliminary efficacy by means of well-established, standardized self-reports assessing COVID-19 related psychopathology. Secondary outcome measures were collected at baseline and immediately after the treatment for all participants. Participants were asked to complete a battery of online self-report questionnaires that included: the General Anxiety Disorder-7 (GAD-7; Williams, 2014), the Patient Health Questionnaire (PHQ-9; Kroenke et al., 2001), the Insomnia Severity Index (ISI; Bastien, 2001), the Impact of Event Scale-Revised (IES-R; Christianson and Marren, 2012) and the General Health Questionnaire (GHQ-12; Gureje and Obikoya, 1990).
2.5. Data analytic plan
Demographic and clinical data were inspected for normalcy using the Shapiro-Wilk test. Data for primary outcomes are reported as percentages. Differences in baseline characteristics between completers and dropouts, and between the patients and the relatives groups were assessed using independent-samples t-tests and chi square tests for categorical variables. Among completers, baseline to post-intervention changes on self-reported questionnaires scores were assessed using paired-samples t-tests. Within-group baseline to post-intervention effect size are reported.
Given the high heterogeneity in subject-specific changes in clinical variables after treatment, clustering analysis was conducted on baseline and post-intervention data separately, using the NbClust tool for R (Charrad et al., 2014) on scores from the five self-report questionnaires. Hubert and D indices were used to identify the best number of clusters. The Hubert index is a graphical method of determining the number of clusters. In the plot of Hubert index and in that of the D index, the significant peak in the second differences plot corresponds to a significant increase of the value of the measure. The majority rule was used to determine the best number of clusters. Next, a k-means algorithm, which allows to subdivide a set of subjects in k groups based on their features, was applied (Sinaga and Yang, 2020). Independent-samples t-tests were used to characterize clusters from baseline and post-intervention data separately. Finally, a confusion matrix was used to analyze the segregation of participants from baseline clusters to post-intervention clusters. p values below 0.05 were considered significant. All analyses were performed using IBM SPSS Statistics (Version 27, 2020), R (Version 4.2.1., 2022), and the Python Package Index (Version 3.6.3., 2022).
3. Results
As of June 30th, 2022, 114 individuals were referred to the study and underwent the preliminary telephone screening; out of these 114, 107 were eligible for enrollment following a formal assessment of inclusion and exclusion criteria. Of 107 participants, 104 (92 % of all referred) completed neuropsychological testing and were enrolled in the study. Completion rate of baseline questionnaires was >98 % of all enrolled participants, with only two subjects not completing the questionnaires. Of the remaining 102 participants, 10 (95 % of all eligible participants) dropped out, all of who completed at least one therapy session and baseline assessment. Out of the 92 participants who completed all eight tele-therapy sessions, 4 did not complete post-intervention questionnaires, and were thus lost to follow-up. This resulted in a final sample of completers of 88 participants (82 % of all eligible contacted participants, and 84.6 % of all enrolled participants). This information is summarized in the Consolidated Standards of Reporting Trials (CONSORT) Table below (Fig. 1 ).
3.1. Descriptive statistics
The sample of all screened participants who completed baseline questionnaires (N = 102), included 37 males (36 %) and 65 females (64 %), and 73 patients (71 %) and 29 relatives (29 %). Demographic and baseline characteristics (IQ and self-report questionnaire scores) for the whole sample and for patients and relatives separately are shown below (Table 1 ). With regards to the patients group (N = 73) vs relatives group (N = 29) comparison, a chi-square test of independence showed a significant association between gender and group (X2(1, N = 102) = 11.79, p < 0.01), with a significantly higher prevalence of females in the relatives group (26/29, 90 %). Independent samples t-tests identified statistically significant differences in education levels between patients and relative (t(44) = 2.38, p = 0.02), with patients having significantly higher levels of education compared to relatives.
Table 1.
N | Mean | SD | t | df | p | |
---|---|---|---|---|---|---|
Demographic characteristics | ||||||
Age (years) | ||||||
Whole sample | 100 | 49.1 | 16.0 | |||
Patients | 72 | 49.6 | 15.4 | 0.16 | 86 | 0.87 |
Relatives | 28 | 47.9 | 17.9 | |||
Education (years) | ||||||
Whole sample | 93 | 13.4 | 2.9 | |||
Patients | 68 | 13.8 | 2.9 | 2.15 | 80 | 0.04 |
Relatives | 25 | 12.2 | 2.9 | |||
Baseline IQ and self-report questionnaires scores | ||||||
Raven matrices (total score) | ||||||
Whole sample | 91 | 115.9 | 13.8 | |||
Patients | 64 | 115.1 | 14.8 | 0.45 | 77 | 0.66 |
Relatives | 27 | 118.1 | 10.9 | |||
QIT score | ||||||
Whole sample | 90 | 105.5 | 9.9 | |||
Patients | 63 | 105.9 | 10.7 | 0.63 | 76 | 0.53 |
Relatives | 27 | 104.7 | 8.2 | |||
QIV score | ||||||
Whole sample | 90 | 106.9 | 7.6 | |||
Patients | 63 | 107.8 | 7.8 | 1.67 | 76 | 0.10 |
Relatives | 27 | 104.9 | 6.7 | |||
QIP score | ||||||
Whole sample | 90 | 111.9 | 7.9 | |||
Patients | 63 | 112.6 | 8.7 | 1.41 | 76 | 0.14 |
Relatives | 27 | 110.4 | 5.4 | |||
GAD-7 score | ||||||
Whole sample | 98 | 10.0 | 4.9 | |||
Patients | 70 | 9.8 | 5.3 | 0.90 | 84 | 0.37 |
Relatives | 28 | 10.7 | 3.7 | |||
PHQ-9 score | ||||||
Whole sample | 91 | 10.2 | 5.6 | |||
Patients | 63 | 10.7 | 6.1 | 1.01 | 77 | 0.31 |
Relatives | 28 | 9.2 | 4.8 | |||
ISI score | ||||||
Whole sample | 98 | 9.9 | 6.3 | |||
Patients | 70 | 10.2 | 6.5 | 0.87 | 84 | 0.38 |
Relatives | 28 | 9.4 | 5.7 | |||
IESR score | ||||||
Whole sample | 97 | 4.44 | 2.5 | |||
Patients | 69 | 4.24 | 2.5 | 1.73 | 83 | 0.09 |
Relatives | 28 | 4.94 | 2.5 | |||
GHQ-12 score | ||||||
Whole sample | 98 | 20.0 | 5.5 | |||
Patients | 70 | 19.5 | 4.9 | 1.00 | 77 | 0.31 |
Relatives | 28 | 21.3 | 6.6 |
SD: Standard Deviation, df: degrees of freedom, QIT: Quoziente Intellettivo Totale (total IQ), QIV: Quoziente Intellettivo Verbale (Verbal IQ), QIP: Quoziente Intellettivo di Performance (Performance IQ), GAD-7: General Anxiety Disorder-7, PHQ-9: Patient Health Questionnaire-9, ISI: Insomnia Severity Index, IES-R: Impact of Event Scale-Revised, GHQ-12: General Health Questionnaire-12.
3.2. Comparison of completers vs dropouts
Differences between demographic and baseline characteristics were investigated between completers and dropouts (Suppl. Table 1). Compared to completers, participants who dropped out had significantly lower and QIV scores (t(88) = 2.06, p = 0.04) and QIT scores (t(88) = 2.62, p = 0.01). No significant differences in gender distribution or patients/relatives group belonging emerged between completers and dropouts (p > 0.05).
3.3. Preliminary efficacy of tele-psychotherapy
Whole-group paired-samples t-tests between baseline and post-intervention scores from completers (n = 88) detected a significant decrease of 21.1 % in generalized anxiety (GAD-7: t(85) = 3.51, p = 0.001, d = 0.38; Fig. 2A), of 21.2 % on depression (PHQ-9: t(78) = 3.30, p = 0.001, d = 0.37; Fig. 2B), of 20.8 % on insomnia (ISI: t(83) = 3.95, p < 0.0001, d = 0.43;, Fig. 2C), of 11 % on post-traumatic psychopathology (IES-R: t(82) = 4.99, p < 0.0001, d = 0.54; Fig. 2D). and of 35.7 % on general psychophysical distress (GHQ-12: t(83) = 5.03, p < 0.0001, d = 0.31; Fig. 2E). Effect sizes were in the medium range. Post-hoc power analyses setting α error at 0.05 indicated a Power (1 − β probability) of 0.936 for GAD-7, of 0.901 for PHQ-9, of 0.973 for ISI, of 0.998 for IES-R, and of 0.801 for GHQ-12.
Because patients and relatives significantly different at baseline for years of education and gender distribution, we investigated the effects of these two variables on the clinical outcomes to assess whether it was necessary to perform further sub-analyses. Years of education did not correlate with measures of psychopathological severity collected at baseline (all p-values > 0.5). No significant differences between males and females on baseline measures of psychopathological severity were detected, with the exception of baseline IES-R scores, which were significantly higher in females (t(95) = 2.59, p = 0.01). Nevertheless, a two-way repeated measures ANOVA with baseline vs post-intervention and gender as independent variables revealed that, while there were significant main effects of baseline vs post-intervention (F(1, 81) = 21.69, p < 0.0001) and of gender (F(1, 81) = 9.023, p = 0.004), there was no significant interaction between the two (F(1,81) = 0.18, p = 0.67; Suppl. Fig. 1).
4. Clinical clusters and treatment response
In light of the high heterogeneity in subject-specific changes in clinical variables after treatment, a comparison of co-occurrences was conducted. Clustering analysis was performed on baseline and post-intervention data separately, using scores from the five online self-report questionnaires. Based on Hubert and D indices, the best number of clusters was two, both at baseline and at post-intervention. At baseline, the k-means algorithm identified Cluster 1, with 41 subjects (47.6 %), and Cluster 2, with 45 (52.3 %). At post-intervention, the k-means algorithm identified Post-Cluster 1, with 48 subjects (55.9 %), and Post-Cluster 2, with 38 (44.1 %) (Fig. 3 ).
Subjects in Cluster 1 had significantly lower scores on all self-report questionnaires compared to subjects in Cluster 2 (Table 2 ). Similarly, subjects in Post-Cluster 1 had significantly lower scores on all self-report questionnaires compared to subjects in Post-Cluster 2 (Table 3 ).
Table 2.
N | Mean | SD | t | df | p | |
---|---|---|---|---|---|---|
Baseline self-report questionnaires scores | ||||||
GAD-7 score | ||||||
Cluster 1 | 40 | 6.3 | 3.5 | −9.56 | 82 | <0.001 |
Cluster 2 | 44 | 13.5 | 3.3 | |||
PHQ-9 score | ||||||
Cluster 1 | 37 | 6.3 | 3.9 | −7.41 | 75 | <0.001 |
Cluster 2 | 40 | 13.8 | 4.8 | |||
GHQ-12 score | ||||||
Cluster 1 | 40 | 16.5 | 4.3 | −7.02 | 82 | <0.001 |
Cluster 2 | 44 | 23.3 | 4.5 | |||
ISI score | ||||||
Cluster 1 | 40 | 5.0 | 3.7 | −10.05 | 82 | <0.001 |
Cluster 2 | 44 | 14.3 | 4.7 | |||
IESR score | ||||||
Cluster 1 | 39 | 2.6 | 1.5 | −9.06 | 81 | <0.001 |
Cluster 2 | 44 | 6.1 | 1.9 |
SD: Standard Deviation, df: degrees of freedom, GAD-7: General Anxiety Disorder-7, PHQ-9: Patient Health Questionnaire-9, GHQ-12: General Health Questionnaire-12, ISI: Insomnia Severity Index, IES-R: Impact of Event Scale-revised.
Table 3.
N | Mean | SD | t | df | p | |
---|---|---|---|---|---|---|
Post-intervention self-report questionnaires scores | ||||||
GAD-7 score | ||||||
Post-Cluster 1 | 48 | 4.4 | 3.5 | −10.25 | 52.4 | <0.001 |
Post-Cluster 2 | 38 | 12.8 | 3.3 | |||
PHQ-9 score | ||||||
Post-Cluster 1 | 44 | 4.1 | 3.9 | −9.12 | 50.87 | <0.001 |
Post-Cluster 2 | 35 | 13.1 | 4.8 | |||
GHQ-12 score | ||||||
Post-Cluster 1 | 48 | 13.9 | 4.3 | −6.95 | 84 | <0.001 |
Post-Cluster 2 | 38 | 19.8 | 4.5 | |||
ISI score | ||||||
Post-Cluster 1 | 48 | 4.2 | 3.7 | −9.45 | 84 | <0.001 |
Post-Cluster 2 | 38 | 12.6 | 4.7 | |||
IESR score | ||||||
Post-Cluster 1 | 48 | 2.1 | 1.5 | −6.95 | 84 | <0.001 |
Post-Cluster 2 | 38 | 4.9 | 1.9 |
SD: Standard Deviation, df: degrees of freedom, GAD-7: General Anxiety Disorder-7, PHQ-9: Patient Health Questionnaire-9, ISI: Insomnia Severity Index, IES-R: Impact of Event Scale-revised, GHQ-12: General Health Questionnaire-12.
With the goal of characterizing psychopathological severity, clusters' means were compared to literature-defined cut-off values. Proposed cut-offs for GAD-7 scores are >5 for mild, >10 for moderate, and >15 for severe anxiety (Williams, 2014). PHQ-9 scores of 5–9 reflect below threshold depression, 10–14 mild depression, 15–19 moderate depression, and ≥ 20 severe depression (MacArthur, and CT, 2009). Therefore, Cluster 1 presented with mild anxiety and below-threshold depression, while Cluster 2 presented with moderate anxiety and mild depression. This suggests that participants from Cluster 1 likely met diagnostic criteria for an anxiety disoders, and possibly for a depressive disorder. Post-Cluster 1 showed no anxiety and depression, while Post-Cluster 2 presented with moderate anxiety and mild depression. Insomnia is defined as severe when the ISI questionnaire score is 10 or greater (Bastien, 2001). The maximum mean score on each of the three IES-R subscales (Avoidance, Intrusion, Hyperarousal) is ‘4’, therefore the maximum ‘total mean’ IES-R score is 12. Lower scores are better. Therefore, Cluster 1 presented with no insomnia or post-traumatic symptoms, while Cluster 2 presented with severe insomnia and moderate post-traumatic symptoms. Post-Cluster 1 showed no insomnia or post-traumatic symptoms, while Post-Cluster 2 presented with severe insomnia and mild post-traumatic symptoms. Proposed cut-offs for GHQ-12 are 15–19 for mild distress, and 19–36 for clinical distress (Goldberg, 2011). Therefore, Cluster 1 and Post-Cluster 1 presented with mild distress, while Cluster 2 and Post-Cluster 2 with clinical distress. Based on these comparisons, participants belonging to Cluster 1 and Post-Cluster 1 can generally be considered asymptomatic, while participants belonging to Cluster 2 and Post-Cluster 2 had clinically significant psychopathology of moderate severity.
Finally, a confusion matrix mapping baseline vs post-intervention Clusters (Fig. 4 ) showed that, of those who were clinically symptomatic at baseline, 33 % (N = 15/45) showed symptom remission after the intervention, while 66 % (N = 30/45) had residual COVID-19 related psychopathology. Interestingly, of those who were asymptomatic at baseline, 80 % (N = 33/41) remained asymptomatic after treatment, while 20 % (N = 41) showed deterioration of clinical status during the course of treatment.
5. Discussion
5.1. Main findings
The SARS-CoV-2 pandemic has strained global and mental health systems, calling for novel approaches to tackle the psychological burden while ensuring safety measures such as social distancing and quarantine.
In this study, we show that a tele-psychotherapy intervention for COVID-19 patients and their relatives based on constructivist and hermeneutic-phenomenological theories is highly feasible, as indicated by an 82 % completion rate of all eligible contacted participants. This finding is in line with a recent meta-analytic study of digital interventions for mental health problems, which found a mean attrition rate of 24.1 % at short-term follow up (Linardon and Fuller-Tyszkiewicz, 2020).
Findings regarding preliminary efficacy indicate that, at the group level, remote brief tele-psychotherapy significantly reduced by at least 10 % anxiety, depression, post-traumatic symptoms, and sleep disturbances, as well as improved overall psychophysical wellbeing. Within-group effect sizes were in the medium range (0.31 < d < 0.54) and comparable to those recently reported by a meta-analysis (Fernandez et al., 2021) conducted on video-delivered psychotherapy (g = 0.54, adjusted for possible publication bias), and to those shown by a systematic review conducted on digital interventions based on the principles of cognitive behavioral therapy for mood disorders (Biagianti et al., 2022).
Nonetheless, clustering analyses revealed the presence of two subgroups of participants at baseline: one (48 %) for whom COVID-19 related psychopathology was absent or below the clinical threshold, and another (52 %) that had uniformly moderate-severe psychological symptoms prior to the intervention. This high degree of inter-individual clinical heterogeneity accounted for distinct profiles of treatment response: 33 % participants with significant COVID-19 related psychopathology achieved full symptom remission, while 66 % remained symptomatic after treatment. Expectedly, the vast majority of participants who did not have COVID-19 related psychopathology yet accessed the intervention could not show significant reduction in the self-reported questionnaires, largely due to floor effects.
Several hypotheses can be taken into consideration when interpreting these results. First, because of the inclusion and exclusion criteria operationalized in this study, almost half of the sample consisted of participants who accessed the free intervention in absence of significant COVID-19 related psychopathology. Possible reasons for seeking treatment include experiences of nonspecific psychological suffering that are less likely to be captured by the symptom-centered questionnaires that are typically used in psychotherapy trials. Arguably, selecting prospective patients based on stratification of COVID-19 related psychopathology would have resulted in greater efficacy. Second, 66 % of participants with severe COVID-19 related psychopathology at baseline showed residual symptoms after treatment. This high rate of non-responders speaks less about the lack of efficacy of the treatment approach and more about the unique features of COVID-19 related psychopathology. Consistently with studies recently published on the same topic (Eisma and Tamminga, 2020), we identified features in the narratives that are strictly related to the pandemic experience: the dynamics, conditions, and medical management of the COVID-19 infection, the sense of guilt about having infected a loved one, the patients' experience of isolation in intensive care units, the relatives' experience of isolation at home and inability to see and speak with the loved one, the lack of final farewell associated with the end of life of the loved one, and the absence or disruption of funerary rites (Hamid and Jahangir, 2022; Hernández-Fernández and Meneses-Falcón, 2022). Several authors have argued that these circumstances negatively affected the grief process for losses experiencing during the pandemic, leading to the development of enduring grief reactions or complicated bereavement (Diolaiuti et al., 2021; Nakajima, 2018) that likely require a longer and more targeted intervention model compared to the one employed in the current study.
5.2. Strengths
To our knowledge, this is the first national attempt to create a research-informed infrastructure to study the feasibility and efficacy of a remote tele-psychotherapy free service for COVID-19 patients and their first-degree relatives. Similar interventions have been developed implemented in India (Selvapandiyan, 2022) and China (Li et al., 2020), with equally promising models and results. These digital psychiatry models have all shown the ability allow to overcome barriers that are typical of traditional mental health care services and are not unequivocally bound to the COVID-19 pandemic. Importantly, such approaches greatly increase the number of users that can be reached. For example, it is likely that individuals living in rural and low-resource settings, members of ethnic minority groups, and of low-income groups will be more encouraged to access mental health services if this does not require traveling long distances to reach the nearest hospital or navigating the organizational intricacies of a mental health system. Therefore, the prompt availability of an eight-session tele-psychotherapy intervention able to reduce symptoms of depression, anxiety, post-traumatic symptoms, and sleep disturbances, at least in selected patients, has the potential to truly address the disparities in mental health care provision in underserved populations at a national level.
5.3. Limitations
These promising findings should still be interpreted with caution.
A major limitation of the current study is that it could not be investigated whether psychopathological symptoms were not already present in our sample prior to SARS-CoV-2 infection. This introduces both a spectrum bias and a detection bias. Additionally, the absence of a control group does not allow to infer causality with regards to treatment efficacy. Indeed, the reduction of psychopathology observed through the self-reported questionnaires from pre- to post-phases can be ascribed to generic factors, including recovery, reconnection with loved ones, time, etc. However, feedback from psychotherapists working on the project seems to qualitatively indicate that patients overall verbalized improvements in physical and psychological wellbeing upon exposure to treatment, indicating overall benefits.
Another limitation – one that seems very much tied to the nature of this clinical population – is the heterogeneity of participants both in terms of severity of COVID-19 symptoms and psychopathology. Because of illness severity and the reliance on self-report measures, this study is at risk for recall bias, arising because of differences in the accuracy or completeness of participant recollections of past events. This heterogeneity requires the study of larger samples, where clinical and psychological phenotypes can be identified, characterized, and assigned to personalized treatment approaches.
Regarding the feasibility and acceptability findings, we have not collected baseline information in our sample about previous exposure to psychotherapy or tele-psychotherapy, which could affect the experimental treatment under investigation. A final limitation is the lack of follow-up data, which limits the ability to draw conclusions about the potential durability of clinical improvements. Future studies should focus on the long-term benefits of tele-psychotherapy on mental health outcomes.
6. Conclusions
In sum, the brief tele-psychotherapy intervention studied in this trial has shown great innovation potential in that: (1) it offers readily available psychotherapeutic support to those who experience psychological suffering associated with COVID-19; (2) it helps psychotherapists to operate in acute and subacute settings, overcoming the barriers imposed by public health prevention measures; and (3) contributes to study and determine the procedures by which tele-psychotherapy can be best implemented.
As a results, its preliminary findings are in line with several open calls for telemedicine and monitoring system technologies adoption that have been recently issued by the WHO, the National Institute of Health, the Italian Ministry for Technological Innovation and Digitalization, the Italian Ministry of Health. Similar initiatives have been developed as an attempt to respond to this call for action, with promising treatment principles and results (Békés et al., 2022; Fleck et al., 2022; Robles et al., 2022).
While these digital approaches present the opportunity to enable remote engagement and delivery of care to underserved populations who are unable or unwilling to access mental health services, such as those who live in rural or under-resourced areas, members of ethnic minorities and individuals living in low-resource settings, citizens of low- and middle-income countries, and/or those who hesitate to approach traditional mental health settings, because of stigma, the preliminary nature of these initial studies require further research that should: i) examine how brief remote psychotherapy adjusts to emerging clinical difficulties; ii) characterize interfering occurrences associated with the virtual setting; and iii) investigate causality between these interferences and expected outcomes. Ultimately, above and beyond COVID-19-related psychopathology, these lines of research can provide precious information about the dissemination of remote tele-psychotherapy — a promising treatment model that can transform clinical practice, enhance cost-effectiveness, and lead to better wellbeing and quality of life for patients.
CRediT authorship contribution statement
Conceptualization, BB, IL, FC, EC, CB, MB, PB; Methodology, BB, IL, FC, AD, CB, MB, PB; Formal analysis, BB, IL, NT, AD, CB, MB, PB; Investigation, BB, IL, AD, FC, VG, SZ, CF, FG, VT, VD, GR, MC, CB, PB; Data curation, BB, IL, AD, GF, GR, MB, CB, PB; Writing—original draft preparation, BB, IL, AD, NT, GF, SZ, VG, CF, FG, FC, VT, VD, DR, EC, MB, CB, PB; Writing—review and editing, BB, IL, AD, NT, GF, SZ, VG, CF, FG, FC, VT, VD, DR, EC, MB, CB, PB; Project administration, BB, IL, AD, NT, GF, SZ, VG, CF, FG, FC, VT, VD, DR, EC, NS, EZ, MB, CB, PB; Funding acquisition, NS, EZ, MB, CB, PB. All authors have read and agreed to the published version of the manuscript.
Funding
This study was supported by Fondazione Cariplo under Award Number 2020-1366 (PI: Brambilla). Additionally, this study was supported by the Italian Ministry of Health (Ricerca Corrente 2023), and by the Fondo Integrativo Speciale Ricerca under Award Number 020IP_05157 (PI: Bellani). The content is solely the responsibility of the authors and does not necessarily represent the official views of the funding agencies.
Institutional Review Board statement
The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Ethics Committee, Milano Area 2, of Fondazione IRCCS Ca' Granda Ospedale Maggiore Policlinico (protocol code 962_2020, 03/11/2020).
Informed consent statement
All subjects gave their informed consent for inclusion before they participated in the study. The study was conducted in accordance with the Declaration of Helsinki, and the protocol was approved by the Ethics Committee of Fondazione IRCCS Ca' Granda Ospedale Maggiore Policlinico (#962_2020).
Conflict of interest
All authors report no conflict of interest.
Acknowledgments
None.
Footnotes
Supplementary data to this article can be found online at https://doi.org/10.1016/j.jad.2023.03.024.
Appendix A. Supplementary data
Data availability
Data will be made available upon request.
References
- Adams-Prassl A., Boneva T., Golin M., Rauh C. The impact of the coronavirus lockdown on mental health: evidence from the United States. Econ. Policy. 2022;37(109):139–155. doi: 10.1093/epolic/eiac002. [DOI] [Google Scholar]
- Al-Hazmi A. Challenges presented by MERS corona virus, and SARS corona virus to global health. Saudi J. Biol. Sci. 2016;23(4):507–511. doi: 10.1016/j.sjbs.2016.02.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Balcom E.F., Nath A., Power C. Acute and chronic neurological disorders in COVID-19: potential mechanisms of disease. Brain. 2021;144(12):3576–3588. doi: 10.1093/brain/awab302. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bastien C. Validation of the insomnia severity index as an outcome measure for insomnia research. Sleep Med. 2001;2(4):297–307. doi: 10.1016/S1389-9457(00)00065-4. [DOI] [PubMed] [Google Scholar]
- Békés V., Doorn K.A., Bőthe B. Assessing patients’ attitudes towards telepsychotherapy: the development of the unified theory of acceptance and use of technology-patient version. Clin. Psychol. Psychother. 2022;29(6):1918–1927. doi: 10.1002/cpp.2760. [DOI] [PubMed] [Google Scholar]
- Belz M., Hessmann P., Vogelgsang J., Schmidt U., Ruhleder M., Signerski-Krieger J., Radenbach K., Trost S., Schott B.H., Wiltfang J., Wolff-Menzler C., Bartels C. Evolution of psychosocial burden and psychiatric symptoms in patients with psychiatric disorders during the Covid-19 pandemic. Eur. Arch. Psychiatry Clin. Neurosci. 2022;272(1):29–40. doi: 10.1007/s00406-021-01268-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Biagianti B., Zito S., Fornoni C., Ginex V., Bellani M., Bressi C., Brambilla P. Developing a brief tele-psychotherapy model for COVID-19 patients and their family members. Front. Psychol. 2021;12 doi: 10.3389/fpsyg.2021.784685. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Biagianti B., Foti G., Di Liberto A., Bressi C., Brambilla P. CBT-informed psychological interventions for adult patients with anxiety and depression symptoms: a narrative review of digital treatment options. J. Affect. Disord. 2022;325:682–694. doi: 10.1016/j.jad.2023.01.057. [DOI] [PubMed] [Google Scholar]
- Cai X., Hu X., Ekumi I.O., Wang J., An Y., Li Z., Yuan B. Psychological distress and its correlates among COVID-19 survivors during early convalescence across age groups. Am. J. Geriatr. Psychiatry. 2020;28(10):1030–1039. doi: 10.1016/j.jagp.2020.07.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cannito L., Di Crosta A., Palumbo R., Ceccato I., Anzani S., La Malva P., Palumbo R., Di Domenico A. Health anxiety and attentional bias toward virus-related stimuli during the COVID-19 pandemic. Sci. Rep. 2020;10(1):16476. doi: 10.1038/s41598-020-73599-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cannito L., Anzani S., Bortolotti A., Di Domenico A., Palumbo R. Face mask reduces the effect of Proposer's (un)trustworthiness on intertemporal and risky choices. Front. Psychol. 2022;13 doi: 10.3389/fpsyg.2022.926520. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cantù F., Biagianti B., Lisi I., Zanier R. Psychotherapeutic and psychiatric intervention in patients with COVID-19 and their relatives: protocol for the DigiCOVID trial. JMIR Res. Protoc. 2022;11(11):e39080. doi: 10.2196/39080. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Charrad M., Ghazzali N., Boiteau V., Niknafs A. NbClust: an R package for determining the relevant number of clusters in a data set. J. Stat. Softw. 2014;61(6) doi: 10.18637/jss.v061.i06. [DOI] [Google Scholar]
- Christianson S., Marren J. The impact of event scale—revised (IES-R) Medsurg Nurs. 2012;21(5):321–323. [PubMed] [Google Scholar]
- Cullen W., Gulati G., Kelly B.D. Mental health in the COVID-19 pandemic. QJM. 2020;113(5):311–312. doi: 10.1093/qjmed/hcaa110. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Diolaiuti F., Marazziti D., Beatino M.F., Mucci F., Pozza A. Impact and consequences of COVID-19 pandemic on complicated grief and persistent complex bereavement disorder. Psychiatry Res. 2021;300 doi: 10.1016/j.psychres.2021.113916. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Eisma M.C., Tamminga A. Grief before and during the COVID-19 pandemic: multiple group comparisons. J. Pain Symptom Manag. 2020;60(6):e1–e4. doi: 10.1016/j.jpainsymman.2020.10.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fernandez E., Woldgabreal Y., Day A., Pham T., Gleich B., Aboujaoude E. Live psychotherapy by video versus in-person: a meta-analysis of efficacy and its relationship to types and targets of treatment. Clin. Psychol. Psychother. 2021;28(6):1535–1549. doi: 10.1002/cpp.2594. [DOI] [PubMed] [Google Scholar]
- Fleck M.P.A., Spanemberg L., Souza L.H., Salum G.A., Jr. Grief, interpersonal disputes and role transitions: the breadth of interpersonal telepsychotherapy as a strategy to reduce mental health suffering due to the COVID-19 pandemic in health professionals. Rev. Bras. Psiquiat. (Sao Paulo, Braz. 1999) 2022 doi: 10.47626/1516-4446-2022-2867. Advance online publication. doi:10.47626/1516-4446-2022-2867. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gandhi R.T., Lynch J.B., del Rio C. Mild or moderate Covid-19. N. Engl. J. Med. 2020;383(18):1757–1766. doi: 10.1056/NEJMcp2009249. [DOI] [PubMed] [Google Scholar]
- Goldberg D.P. General health Questionnaire-12 [data set] Am. Psychol. Assoc. 2011 doi: 10.1037/t00297-000. [DOI] [Google Scholar]
- Gureje O., Obikoya B. The GHQ-12 as a screening tool in a primary care setting. Soc. Psychiatry Psychiatr. Epidemiol. 1990;25(5):276–280. doi: 10.1007/BF00788650. [DOI] [PubMed] [Google Scholar]
- Hamid W., Jahangir M.S. Dying, death and mourning amid COVID-19 pandemic in Kashmir: a qualitative study. OMEGA J. Death Dying. 2022;85(3):690–715. doi: 10.1177/0030222820953708. [DOI] [PubMed] [Google Scholar]
- Harris J.C., Greenspan S. In: Handbook of Evidence-based Practices in Intellectual and Developmental Disabilities. Singh N.N., editor. Springer International Publishing; 2016. Definition and nature of intellectual disability; pp. 11–39. [DOI] [Google Scholar]
- Hernández-Fernández C., Meneses-Falcón C. I can't believe they are dead. Death and mourning in the absence of goodbyes during the COVID-19 pandemic. Health Soc. Care Commun. 2022;30(4) doi: 10.1111/hsc.13530. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kamal M., Abo Omirah M., Hussein A., Saeed H. Assessment and characterisation of post-COVID-19 manifestations. Int. J. Clin. Pract. 2021;75(3) doi: 10.1111/ijcp.13746. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kroenke K., Spitzer R.L., Williams J.B.W. The PHQ-9: validity of a brief depression severity measure. J. Gen. Intern. Med. 2001;16(9):606–613. doi: 10.1046/j.1525-1497.2001.016009606.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Li J., Li X., Jiang J., Xu X., Wu J., Xu Y., Lin X., Hall J., Xu H., Xu J., Xu X. The effect of cognitive behavioral therapy on depression, anxiety, and stress in patients with COVID-19: a randomized controlled trial. Front. Psychiatry. 2020;11 doi: 10.3389/fpsyt.2020.580827. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Linardon J., Fuller-Tyszkiewicz M. Attrition and adherence in smartphone-delivered interventions for mental health problems: a systematic and meta-analytic review. J. Consult. Clin. Psychol. 2020;88(1):1–13. doi: 10.1037/ccp0000459. [DOI] [PubMed] [Google Scholar]
- MacArthur J.D., CT, M. MacArthur foundation’s initiative on depression & primary care. Depress. Manag. Tool Kit. 2009;2009 [Google Scholar]
- Mazza M.G., Palladini M., Poletti S., Benedetti F. Post-COVID-19 depressive symptoms: epidemiology, pathophysiology, and pharmacological treatment. CNS Drugs. 2022;36(7):681–702. doi: 10.1007/s40263-022-00931-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nakajima S. Complicated grief: recent developments in diagnostic criteria and treatment. Philos. Trans. R. Soc. B Biol. Sci. 2018;373(1754):20170273. doi: 10.1098/rstb.2017.0273. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Posner K., Brent D., Lucas C., Gould M., Stanley B., Brown G., Fisher P., Zelazny J., Burke A., Oquendo M. Columbia University Medical Center; New York, NY: 2008. Columbia-Suicide Severity Rating Scale (C-SSRS) p. 10. [Google Scholar]
- Ramseyer F., Tschacher W. Nonverbal synchrony of head- and body-movement in psychotherapy: different signals have different associations with outcome. Front. Psychol. 2014;5:979. doi: 10.3389/fpsyg.2014.00979. Sep 5. PMID: 25249994; PMCID: PMC4155778. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Raveendran A.V., Jayadevan R., Sashidharan S. Long COVID: An overview. Diab. Metab. Syndr. Clin. Res. Rev. 2021;15(3):869–875. doi: 10.1016/j.dsx.2021.04.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Raven J., Raven J. In: Handbook of Nonverbal Assessment. McCallum R.S., editor. Springer US; 2003. Raven progressive matrices; pp. 223–237. [DOI] [Google Scholar]
- Robles R., Ascencio L., Díaz D., Ruiz S., Gálvez L., Sánchez M., Espinoza F., Hernández-Posadas A., Fresán A., Vega H., Morales-Chainé S. Implementation science of Telepsychotherapy for anxiety, depression, and somatization in health care workers dealing with COVID-19. Telemed. J. e-Health. 2022 doi: 10.1089/tmj.2022.0155. Advance online publication. [DOI] [PubMed] [Google Scholar]
- Sartori G., Colombo L., Vallar G., Rusconi M.L., Pinarello A. TIB: Test di Intelligenza Breve per la valutazione del quoziente intellettivo attuale e pre-morboso. Profess. Psicol. 1997;1:2–24. [Google Scholar]
- Selvapandiyan J. Adapting cognitive behaviour therapy for tele-psychotherapy services for COVID-19. Psychiatry Res. 2022;311 doi: 10.1016/j.psychres.2022.114483. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sinaga K.P., Yang M.-S. Unsupervised K-means clustering algorithm. IEEE Access. 2020;8:80716–80727. doi: 10.1109/ACCESS.2020.2988796. [DOI] [Google Scholar]
- Voruz P., Allali G., Benzakour L., Nuber-Champier A., Thomasson M., Jacot de Alcântara I., Pierce J., Lalive P.H., Lövblad K.-O., Braillard O., Coen M., Serratrice J., Pugin J., Ptak R., Guessous I., Landis B.N., Assal F., Péron J.A. Long COVID neuropsychological deficits after severe, moderate, or mild infection. Clin. Transl. Neurosci. 2022;6(2):9. doi: 10.3390/ctn6020009. [DOI] [Google Scholar]
- Williams N. The GAD-7 questionnaire. Occup. Med. 2014;64(3):224. doi: 10.1093/occmed/kqt161. [DOI] [Google Scholar]
- Zaim S., Chong J.H., Sankaranarayanan V., Harky A. COVID-19 and multiorgan response. Curr. Probl. Cardiol. 2020;45(8) doi: 10.1016/j.cpcardiol.2020.100618. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
Data will be made available upon request.