Skip to main content
European Heart Journal. Digital Health logoLink to European Heart Journal. Digital Health
. 2021 Apr 6;2(2):323–335. doi: 10.1093/ehjdh/ztab037

Internet-based treatment of anxiety and depression in patients with ischaemic heart disease attending cardiac rehabilitation: a feasibility study (eMindYourHeart)

Charlotte Helmark 1,2,, Robert Ahm 2, Christina M Andersen 2, Søren J Skovbakke 2, Robin Kok 2, Uffe K Wiil 3, Thomas Schmidt 3, Jacob Hjelmborg 4, Lisbeth Frostholm 5, Ditte H Frydendal 5, Tina B Hansen 1, Ann-Dorthe Zwisler 6,7, Susanne S Pedersen 2,6,8
PMCID: PMC9707909  PMID: 36712399

Abstract

Aims

Anxiety and depression are prevalent in 20% of patients with ischaemic heart disease (IHD); however, treatment of psychological conditions is not commonly integrated in cardiac rehabilitation (CR). Internet-based psychological treatment holds the potential to bridge this gap. To examine the feasibility of an eHealth intervention targeting anxiety and depression in patients with IHD attending CR.

Methods and results

We used a mixed-methods design, including quantitative methods to examine drop-out and change in anxiety and depression scores, and qualitative methods (thematic analysis) to evaluate patients’ and nurses’ experiences with the intervention. The therapist-guided intervention consisted of 12 modules provided via a web-based platform. The primary outcome was drop-out, with a drop-out rate <25% considered acceptable. Patients were considered as non-drop-out if they completed ≥5 modules. Out of 60 patients screened positive for anxiety and/or depression, 29 patients were included. The drop-out rate was 24% (7/29). Patients had a mean improvement in anxiety and depression scores of 5.5 and 4.6, respectively. On average, patients had 8.0 phone calls with their therapist and 19.7 written messages. The qualitative analysis of patients’ experiences identified four themes: treatment platform, intervention, communication with therapist, and personal experience. Patients were positive towards the intervention, although some found the assignments burdensome. From the nurses, we identified three themes: intervention, inclusion procedure, and collaboration with study team. The nurses were positive, however, due to limited time some struggled with the inclusion procedure.

Conclusion

Integrating an eHealth intervention in CR is feasible and the drop-out rate acceptable.

Keywords: Anxiety, Cardiac rehabilitation, Depression, E-health, Internet-based cognitive behavioural therapy, Ischaemic heart disease

Graphical Abstract

graphic file with name ztab037f4.jpg

Introduction

More than 83.5 million people in Europe live with cardiovascular disease, including ischaemic heart disease (IHD).1 Approximately 20% of patients with IHD have symptoms of anxiety or depression that warrant treatment.2 Psychological conditions such as anxiety and/or depression act as barriers to treatment adherence of cardiac rehabilitation (CR). This increases the risk of adverse outcomes in this vulnerable subset of patients,2 as evidence-based CR is shown to reduce cardiovascular mortality, rehospitalization,3 and improve psychological distress.4

Clinical guidelines recommend screening for psychological distress in patients with IHD combined with psychological treatment if patients screen positive.5 However, psychological treatment is not routinely offered as part of standard CR following screening.6 Cognitive behavioural therapy (CBT) is an effective treatment for anxiety and depression in cardiac patients,7 but lack of psychologists and challenged healthcare budgets hinder implementation of CBT in CR.6,8,9

Current technology has enabled the delivery of internet-based psychological treatment and internet-based CBT (ICBT) appear to be an equally effective treatment for anxiety and depression as face-to face therapy.10 ICBT interventions are typically delivered on a weekly basis and contain structured written material, audios, and videos, resembling face-to face treatment.11 ICBT is not geographically or timely confined, which enables patients with internet access to engage in the treatment from any place at any time of the day.11 Further advantages include reducing delay to psychological treatment, enabling quick access to treatment.11

The widespread use of internet-enabled devices ensures the possibility of delivering ICBT to a broad population, including all socio-economic backgrounds.12 However, implementing the service in routine care is challenging due to complex and fragmented healthcare systems, lack of funding, and limited stakeholder knowledge.12 Therefore, it can be challenging for patients with IHD to obtain treatment, since there—at least in Denmark—are no automatic referral channels from the CR centres for ICBT or other psychological therapies. This means that patients with IHD besides dealing with their somatic disease must contact their general practitioner for guidance and referral to psychological treatment or seek help elsewhere.2 For some patients, particularly those with depression, this may be an insurmountable task, with the risk that they never receive psychological treatment. In addition, psychological treatment might only be reimbursed under specific conditions and unaffordable for patients with e.g. low socio-economic status. Hence, integration of ICBT as part of CR has a considerable potential for the subset of patients with IHD suffering from anxiety and depression.

A systematic review on ICBT in populations with a chronic disease showed significant improvements in anxiety and depressive symptoms, but further research in cardiac populations is suggested due to limited evidence.13 Furthermore, this study points at exploring how to best integrate ICBT in routine care, as most studies have been conducted in research settings.13 Results on effect of ICBT targeting anxiety and depression in cardiac patients are mixed and very limited. Two studies found no effects on symptoms of anxiety and depression in randomized controlled trials (RCTs) of patients with myocardial infarction14 and heart failure,15 respectively, while two other RCTs show positive effects on anxiety and depression in cardiac populations,8,16 indicating a potential for ICBT in cardiac care. Thus, the high prevalence of anxiety and depression highlights the need to develop and evaluate the potential efficacy of ICBT models in CR.

The overall objective of the study was to examine the feasibility of an eHealth intervention targeting anxiety and depression in patients with IHD and integrated in standard CR as a precursor to an RCT.17 Focus areas of the study were acceptability from patients and CR nurses to assess how the intervention were received and implementation of the intervention to test logistics before the RCT.18 The specific aims were to evaluate (i) the dropout rate to assess acceptability, (ii) changes in anxiety and depression scores pre- and post-intervention to determine potential benefits and harms of the intervention, (iii) the extent of use of the treatment platform, (iv) the utility and experiences of the intervention from the patient’s perspective, and (v) the logistics of the intervention from the CR nurses perspective.

Apart from dropout rate, the evaluations did not include predefined outcome targets as these parameters were conducted as proof of concept before the RCT.

Methods

Design

This feasibility study is part of the eMindYourHeart study17 and was conducted using a prospective mixed-methods design to enhance the feasibility assessment of the intervention and the procedures.19 The use of parallel research methods was chosen to obtain a broader view of the context of the study and to draw on strengths and mitigate weaknesses from each individual method. Quantitative methods included descriptive analysis of dropout, changes in anxiety and depression scores post-intervention (i.e. at 3 months of follow-up) and the extent of use of the treatment platform. Qualitative methods included thematic text analysis of written evaluations from patients about their experiences with the intervention and from CR nurses’ their experiences with the inclusion procedures. The study is reported according to the CONSORT 2010 guideline (Consolidated Standards of Reporting Trials), using the checklist for feasibility trials.20

Participants

Participants in the intervention

We aimed to recruit 30 participants, which was a pragmatic choice and considered sufficient for the aims of the feasibility evaluation. Participants were Danish patients attending CR who screened positive on the Hospital Anxiety and Depression Scale (HADS)21 as part of routine care. Inclusion criteria were age ≥18 years, diagnosis of IHD, HADS score ≥8 on depression and/or anxiety,22 access to a computer or smartphone, ability to use computer or smartphone, proficient in the Danish language, and a signed informed consent. Exclusion criteria were severe psychiatric disorder (e.g. schizophrenia), severe cognitive difficulties (e.g. dementia), participation in other psychological intervention studies, seeing a psychologist or mental health professional for the treatment of depression and anxiety.

The participants were recruited consecutively from nine Danish CR centres. The centres were strategically chosen across all five Regions in Denmark and across hospitals and municipalities, to ensure variation with respect to logistics in the CR programmes and a variety in patient demographics. CR nurses recruited patients for the study when starting their CR programme and being routinely screened for anxiety and depression, as recommended by national23 and European guidelines.5

Participants in the evaluations

All patients who completed the intervention and all CR nurses involved in recruitment of patients were invited to participate in an evaluation of the intervention.

Intervention

The intervention was developed using a participatory design.24 This design was used to ensure the target patient group experienced the ICBT as relevant and useful. Psychologists and supervised graduate students of psychology (in their final year) with thorough training in mental health in cardiac patients guided the intervention as therapists. A CR nurse supervised the cardiac aspects when needed. The therapist contacted the patients within three workdays after completing the baseline questionnaire. The intervention began with a diagnostic telephone interview conducted by the therapist, using a brief purpose-designed interview protocol. The protocol was implemented to gain background information about patients’ life situation, experiences with heart disease, motivation to participate in the intervention, understanding of the concepts of the ICBT intervention, sleep problems, use of alcohol and recreational drugs, previous experiences with anxiety, depression and psychological treatment, suicidal ideation, and prior trauma as well as building a therapeutic alliance. The 12-week ICBT intervention consisted of nine mandatory and three voluntary modules, covering topics like behavioural activation and cognitive restructuring (Figure 1; Table 1). All mandatory modules contained written assignments. Patients could access the treatment platform at any time of the day using smartphone, tablet, or computer.

Figure 1.

Figure 1

Overview of the intervention.

Table 1.

Description of the modules in the internet-based cognitive behavioural treatment

Module Content Aim Therapeutic components
Extra Overview of treatment Introduction to treatment + information about IHD. Psychoeducation
1 Information and expectations about treatment To give the patients a better understanding of their own psychological reactions. Psychoeducation, motivation, and goals
2 How to handle difficult emotions To help patients understand and manage their own emotions in a healthier way. Psychoeducation, mentalization, self-compassion
3 Cognitive change (Introduction to cognitive diamond) To give patients a better understanding about their own psychological patterns and concrete tools on how to manage difficult thoughts. Psychoeducation, cognitive restructuring, cognitive diffusion
4 Attention training and control and avoidance and behaviour To help patients to less attention on bodily anxiety symptoms which can be interpreted as IHD. Moreover, to help them identify their own safety and avoidance behaviours. Psychoeducation, selective attention training
5 Exposure therapy To help patients face their fears and overcome them. Exposure therapy
6 Internal barriers, balancing own efforts in treatment and self-care Patients identify and work with potential treatment barriers. Also benefits of self-compassion and exercises promoting self-compassion. Cognitive restructuring, self-compassion
7 Behavioural activation and exercise in daily life To activate depressive patients and to promote physical exercises, to lower depression and anxiety. Behavioural activation
8 Acceptance of heart condition and finding life values To promote acceptance of current situation, and a shift in focus away from limitations and towards possibilities. Cognitive restructuring, life values
9 Relapse prevention To ensure patients know how to spot early signs of relapse and know how to handle these. Psychoeducation, cognitive restructuring
Extra Sleep Knowledge and tools on how to improve sleep. Psychoeducation
Extra Lifestyle changes Information about living with IHD, e.g. where to get help quitting smoking. Psychoeducation

Patients had a personal therapist assigned who supported them throughout the intervention, guiding them through the modules on a weekly basis by asynchronous written messages via the platform. Patients could use the written message function whenever they wanted, and the therapist would respond within two workdays. Furthermore, patients had four phone calls with the therapist during the intervention, including the diagnostic interview. In case patients had an additional need, further phone calls could be assigned.25 The dropout rate was tried diminished by therapists supporting all patients with individual approaches depending on the patient’s situation and the phone calls allowed for this individualization. Patients accessed the intervention platform by a GDPR-compliant website, using their NemID, a Danish national two-factor authentication solution.

Quantitative outcomes and statistics

The primary outcome was dropout rate where a rate <25% was considered a success, based on a systematic review of guided ICBT.26 Treatment completion was defined a priori as five or more mandatory modules completed, since this would ensure that the patient had engaged in both psycho-educative and cognitive and behaviour change content. Secondary outcomes were changes in HADS scores, extent of use of the platform and utility of intervention. Utility was evaluated with six generic questions from the Internet Evaluation and Utility Questionnaire,27 which were chosen as they fitted well with the context of the current intervention. Data on dropout and patient-reported outcomes were collected and managed electronically using the internationally recognized system for data management—Research Electronic Data Capture (REDCap)—via Odense Patient data Explorative Network (OPEN).28 Patient-reported data were collected at baseline and at 3-month follow-up. Extent of use of the platform included duration spent on the treatment platform and frequency and time of login. These data were captured via the treatment platform. Demographics were collected in the baseline questionnaire and clinical data through the Danish Cardiac Rehabilitation Database.29 Results were presented using descriptive statistics and reported with frequencies, percentages, means, range, and standard deviation where appropriate. All statistical analyses were conducted using Stata version 16.1.

Qualitative measurements

Data on patients’ experiences with the intervention were generated through a purpose-designed questionnaire at 3 months of follow-up, divided into categories regarding experiences with the intervention. Patients gave written feedback on the questionnaire, which did not call for alterations. Data on CR nurse’s perception of the intervention were likewise generated through a purpose-designed questionnaire divided into categories regarding logistics of the recruitment procedure. Two CR nurses gave written feedback on this, leading to minor adjustments. Both questionnaires were completed online using REDCap, and answered with open text, enabling the possibility of broad feedback. Both groups had the possibility to write issues beyond the predefined categories and give their evaluation by phone if preferred. The qualitative analysis processes were performed using thematic analysis, inspired by Braun and Clarke.30 For each evaluation, two authors (C.H. and S.J.S.) repeatedly read and coded the data independently and afterwards discussed the coding until consensus was reached. The codes were then sorted into categories and mapped into themes. To optimize credibility the themes were discussed, analysed, and revised with the inclusion of further two authors (C.M.A. and S.S.P.). The trustworthiness of the qualitative analysis was tried accomplished by involving the end-users in the design of the questionnaires to optimize reliability, by using investigator triangulation in the coding process to gain credibility and by describing the context to show transferability.

Hospital Anxiety and Depression Scale (HADS)

The HADS was chosen as the screening tool for anxiety and depression, since it has shown to be a valid measure in cardiac patients22,31 and is recommended in CR in the Danish national clinical guidelines.23 HADS is a 14-item questionnaire with 7 items each contributing to the subscales anxiety and depression. Each item is scored on a Likert scale from 0 to 3 leading to a score range from 0 to 21 for both subscales, with higher scores indicating more severe symptoms. The commonly used cut-off score of ≥8 was applied to identify symptoms of anxiety and depression.22

Ethics

Permission to conduct the study was obtained from the Danish Data Protection Agency at Odense University Hospital (17/41433 on 24 November 2017) via the umbrella permission of the Region of Southern Denmark. Ethical approval was obtained from the Regional Committees on Health Research Ethics for Southern Denmark (S-20180024). The study complies with the Helsinki Declaration with all patients providing written informed consent.32

Results

Participants and drop-out rate

The participants were recruited and treated between November 2019 and July 2020. Figure 2 outlines the flowchart of the study population. A total of 60 patients were identified with elevated HADS scores, of which six were excluded based on exclusion criteria, while 16 patients declined to participation and one died. After the baseline questionnaire and telephone interview, a further eight were excluded or declined. Thus, we included 29 participants in the feasibility study corresponding to 54% of eligible patients. Of the included patients, five dropped out and two did not complete the required five modules to be considered ‘completer’ of the intervention. This means we had a dropout rate of 24.1% (7/29), reaching the desired rate of <25%.

Figure 2.

Figure 2

Flowchart of the study population.

Out of 54 eligible patients, 25 (46%) were not included in the study. For these 25 patients, data are missing for five since they did not provide informed consent. Of the remaining 20 patients, four patients (20%) were female, and the mean age was 59.7 (SD: 10.1), with a range of 33–76 years. The characteristics of the 29 included patients are presented in Tables 2 and 3.

Table 2.

Characteristics of included patients (n = 29)

Completed participants (n = 22) Non-completed participants (n = 7)
Age, mean (SD) 64.6 (11.2) 53.0 (10.0)
 range 47–84 40–70
Female sex, n (%) 9 (41%) 2 (29%)
Living alone, n (%) 6 (27%) 4 (57%)
Employment status, n (%)
 Employed 9 (41%) 5 (71%)
 Retired 12 (55%) 1 (14%)
 Other 1 (4%) 1 (14%)
Highest completed education, n (%)
 Primary school/high school 3 (14%) 2 (29%)
 Short education (≤3 years) 7 (31%) 5 (71%)
 Bachelor’s degree (3–4 years) 9 (41%) 0
 Higher education (≥5 years) 3 (14%) 0
Body mass index, mean (SD) 28.1 (5.3) 30.2 (4.6)
 range 20.6–40.9 23.3–37.2
Currently smoking, n (%) 2 (9%) 3 (43%)
Previously diagnosed with psychiatric disorder, n (%) 2 (9%) 1 (14%)
Physically active ≥150 min per week, n (%) 7 (32%) 2 (29%)

Table 3.

Clinical data from the Danish Cardiac Rehabilitation Database (n = 18)a

Variable n (%) Missing n (%)
Primary indication for referral to cardiac rehabilitation
 ST-elevation myocardial infarction 5 (28) 2 (11)
 Non-ST-elevation myocardial infarction 7 (39)
 Stable angina pectoris 4 (22)
Primary cardiac procedure
 Percutaneous coronary intervention 11 (61) 0 (0)
 Medically managedb 6 (33)
 Heart valve replacement 1 (6)
Diabetes 1 (6) 0 (0)
Heart failure 1 (6) 0 (0)
Referred for exercise-based cardiac rehabilitation 16 (89) 1 (6)
Referred for group-based patient education 11 (6) 2 (11)

SD, standard deviation.

a

Out of the 22 completed participants, 18 were identified in the Danish Cardiac Rehabilitation Database.

b

These are classified as medically managed, as they are not registered as having undergone percutaneous coronary intervention or coronary artery bypass grafting.

Changes in anxiety and depression (measured with HADS)

We found a mean improvement for HADS anxiety scores at 5.5 points and a mean improvement for HADS depression scores at 4.6 points (Table 4). Among the 22 completed patients, three had missing HADS scores at follow-up. The minimal clinical important difference (MCID) reflects meaningful changes for the patient, and for HADS scores it is 1.8.33 We found that 16 out of 19 (84%) patients achieved the MCID for anxiety and 15 out of 19 patients (79%) achieved the MCID for depression. Three reported worse outcomes post-intervention compared to pre-intervention, where one of these were clinically relevant according to the MCID.

Table 4.

Changes in anxiety and depression (measured with HADS) (n = 19)a

HADS pre-scores HADS post-scores Change
HADS A
 Mean (SD) 11.7 (3.1) 6.1 (4.4) −5.5 (4.6)
 Range 6–18 0–15 −16 to +4
HADS D
 Mean (SD) 9.5 (3.2) 4.9 (3.8) −4.6 (5.1)
 Rangeb 5–17 0–12 −15 to +5

HADS, Hospital Anxiety and Depression Scale; SD, standard deviation.

a

Missing HADS data at follow-up, n = 3.

b

Reported worse outcomes post-intervention, n = 3.

Use of intervention

Patients who completed the intervention had a mean of 8 telephone calls and 19.7 written messages with their therapist during the intervention (Table 5). The diagnostic telephone interview is not included in these data. On average, patients spent 10.95 h on the treatment platform. Participants were automatically logged out after 5 min of inactivity, so the time consumption does not include patients who left the treatment platform unattended. For patterns of login, we found that patients logged on to the treatment platform a mean of 33.5 times, where 59.4% of the times took place during normal opening hours for CR centres, while 40.6% took place outside normal opening hours. The total time of the intervention from diagnostic interview to completion of the last module was on average 13.3 weeks.

Table 5.

Use of intervention for completed participants (n = 22)

Mean (SD) Range
Number of contacts with therapist
 Telephone calls 8.0 (3.9) 3–22
 Written messages on the treatment platforma 19.7 (22.6) 5–37
Time consumption on the treatment platform (h) 10.95 (6.5) 4.7–34.1
Number of log ons to the treatment platformb
Total (100%) 33.5 (13.4) 12–69
 Logins within normal opening hoursc (59.4%) 19.9 (8.3) 4–34
 Logins outside normal opening hourc (40.6%) 13.6 (7.3) 3–35
Total time of intervention including diagnostic interview (weeks) 13.3 (2.8) 10–20

SD, standard deviation.

a

Combined written messages on the treatment platform from participant and therapist.

b

Including training of participant, which included logging on twice.

c

Opening hours defined as Monday to Friday from 8 a.m. until 4 p.m., public holidays excluded.

Utility of the intervention

The same 19 patients who completed HADS also completed the utility questions post-intervention (Figure 3). The majority found the intervention easy to use, that the intervention kept their interest and attention, that the information was useful and easy to understand and were satisfied with the intervention. The majority were not worried about their privacy although one patient-reported being worried ‘mostly’.

Figure 3.

Figure 3

Utility of the intervention.

Patients’ evaluation of the intervention

Of the 22 patients who completed the intervention, 14 agreed to provide an evaluation. The text analysis led to the identification of four themes: treatment platform; intervention; communication with therapist; and personal experience (Table 6). Patients experienced that accessing and navigating on the treatment platform was easy, however few found the logon procedure burdensome. Regarding the intervention, the patients were highly satisfied with the concept, leaving them in control of time, place, and speed. Still, some found the written assignments inside some of the modules burdensome. All patients were highly satisfied with the communication with the therapist, appraising the competences of their personal therapist. The patients indicated that their personal motivation for participation was related to a need for help with worries, fear, anxiety, and loneliness, while some specified tools for coping. They experienced the intervention helped with better understanding and acceptance of difficult feelings, and the majority pointed at the cognitive diamond as particularly helpful. Some were positively surprised at the effectiveness of online intervention, and some found it a relief not having to engage in face-to-face treatment. The patients had individual patterns for when they engaged in the intervention based on energy levels, moods, convenience, and schedules.

Table 6.

Summary of patient’s experiences based on the written evaluations

Theme Findings Quotes
Treatment platform Pro:
  • Easy to access, navigate, and use

Con:
  • Burdensome logon procedure

  • Miss possibility to follow progress in the modules and to print out pages

‘Nice, easy and clear. Miss possibility to print out on paper’ (female, 73 years)
Intervention Pro:
  • In control of time and speed of the intervention

  • Very relevant content

  • Highly satisfied with the concept

  • Can participate from home

  • Fine balance between treatment elements

  • Length of intervention and amount of homework appropriate

Con:
  • Length of time to intervention too short

  • Amount of homework too much

‘With online (treatment)—I decide for myself when to engage in the treatment’ (female, 73 years) ‘It was nice that I did not have to leave home even though I needed help to live without anxiety’ (male, 60 years) ‘Written assignments were a bit burdensome, especially when you have limited energy’ (female, 52 years)
Communication with therapist Pro:
  • Highly satisfied with therapist

  • Therapist is competent

  • Sufficient time in phone calls

Con:
  • None

‘100% okay. It is a really good idea that you can text with your therapist during the intervention’ (female, 79 years)
Personal experience Pro:
  • Positively surprised of the concept

  • More relaxed and calm afterwards

  • Better acceptance of feelings

  • Better understanding of own reactions

  • Obtained various tools to cope with negative thoughts and feelings

  • No physical contact with therapist

  • Cognitive diamond helpful

Con:
  • Not enough time to go through the modules

  • Too many written assignments

‘I liked to see my psychologist in person, which I have tried before. But I am positively surprised that this (online treatment) can work’ (female, 64 years) ‘Actually this (online treatment) is an advantage when you are a bit of an “introvert” like me. I did not have to look anybody in the eyes’ (male, 56 years)

Nurses’ evaluation of the intervention

Among the involved CR nurses, 14 completed the evaluation, representing all nine centres. The text analysis led to three themes: intervention; inclusion procedures; and collaboration with study team (Table 7). Overall, the nurses were very positive towards the concept of ICBT, appreciating fast access to psychological treatment for their patients. The nurses experienced that patients do not want to or have the energy to contact their general practitioner seeking help for their anxiety and depression. They also experienced that some patients were not interested in online treatment, and for others it was offered to soon in the CR pathway. Regarding inclusion procedures, some were challenged by the additional task of including patients as part of clinical practice, but also that procedures gradually became less burdensome. Willingness to use clinical time on recruiting patients was in recognition of lack of psychological treatment within their CR centre. Sustainable support from the study team was highly appreciated, safeguarding motivation to participation in the study.

Table 7.

Summary of nurses’ experiences based on the written evaluations

Theme Findings Quotes
Intervention Pro:
  • Online treatment is a valuable and important concept

  • Most patients were interested and positive

  • Free of charge for the patients

  • Fast track to psychological treatment

Con:
  • Some patients had doubts or rejected the offer because treatment is online

  • Some patients might need the treatment later in the care pathway

‘The best part is to have an offer to our sad patients, and on top of that, that the patients can start the intervention so fast’
Inclusion procedures Pro:
  • Good introduction

  • Inclusion process became easier with routine

Con:
  • Challenging on top of usual assessment tasks at start of CR

  • Too much project information to both patients and the nurses

‘Good project, but we also have to register in other databases, leaving less time for the patient’
Collaboration with study team Pro:
  • Personal introduction to the study

  • High level of information, feedback and availability of the team

Con:
  • None

‘The best part is to feel that someone is passionate about this (psychological treatment)’

Discussion

This study examined the feasibility of integrating ICBT into routine CR in patients with IHD. We found a dropout rate from the ICBT intervention of 24%, which is below the a priori determined threshold. The study showed satisfactory improvements in anxiety and depression scores during the intervention. User patterns of the intervention showed a large variability. The evaluations revealed that patients were very positive towards the intervention, but some experienced the workload burdensome. The nurses were very positive about the concept of ICBT but stated that including patients was yet another task to manage in busy clinical practice.

Acceptability and adherence

Among 54 eligible patients with high HADS scores, 29 were included in the study. Of the 25 eligible non-included patients, six experienced minor mental problems, four were excluded at baseline with HADS scores ≤8, 1 died, leaving 14 out of 54 eligible patients (26%) with symptoms of anxiety and depression who declined participation. Three of these patients reported lack of energy and for four patients we do not have information about reason for declining. We can only speculate that for some patients the timing of the intervention may have been too soon after the cardiac event. Others may be unwilling to engage in a psychological intervention e.g. due to stigma or being uncomfortable with online treatment. To our knowledge, only one other study on ICBT within cardiology included patients in routine care, and they reported 1359 out of 1946 eligible patients (70%) declined participation.14 This indicates that this study has a satisfactory acceptability. To safeguard inclusion, additional nine CR centres are recruited for the RCT.

Only few studies on ICBT within cardiology are published. When comparing our findings with these, one RCT in patients with myocardial infarction found no effect of ICBT on symptoms of anxiety or depression.14 The study had a low treatment adherence with 38.4% completing only the introductory module and 15.4% completing additional modules, making it difficult to evaluate the effect of the intervention. Another RCT of ICBT in patients with cardiovascular disease found a significant effect on depression and had 59 patients (82%) completing at least four out of seven modules.8 A recent RCT in patients with acute coronary event also showed a positive effect of ICBT on both anxiety and depression where 92% completed at least four out of eight lessons.16 This suggests that poor adherence to treatment is a barrier to dissemination of potentially effective treatments, even though a linear relationship is still fully uncovered34 and the association was insignificant in the RCT by Lundgren et al.15 It has been proposed from previous ICBT studies that a factor attributing to poor adherence might be a lack of customization of the ICBT treatment to fit the target population.8,16,35 This is among the reasons why we decided using a participatory design for development of the intervention.24 There are substantial differences in recruitment procedures, length of interventions as well as numbers and content of the modules in the mentioned studies, making comparisons difficult. In the current study, patients were considered ‘completers’ if they had completed at least five of the mandatory modules. Two patients did not accomplish the required five modules due to personal life situations but still benefitted from the intervention based on the treatment evaluation of their therapist, suggesting that our choice of five modules might be somewhat arbitrary. On these grounds, and since there is no gold standard for what comprises a ‘completer’ in ICBT, we have decided to change our definition in the eMindYourHeart RCT. Patients will be considered completers regardless of number of completed modules, if they follow the treatment plan agreed upon with their therapist. A patient can drop out of the treatment in two ways: (i) active drop out by communicating this decision to their therapist, or (ii) passive drop out, i.e. the patient stops responding to their therapist attempts to contact them and the contact is not re-established within the 12 weeks of treatment. We have extended the treatment manual and developed a stricter, therapist independent rating system for assessing if a module is completed, and which modules are relevant for the individual patient. This to secure a more personalized approach depending on the patient’s individual situation and a high degree of interrater reliability.

Changes in anxiety and depression

Given the design of this uncontrolled study, we did not analyse the effectiveness of the intervention. With mean improvements in scores of 5.5 for anxiety and 4.9 for depression, the intervention indicated positive signs of obtaining the established MCID of 1.8.33 Although these results could be due to chance, they are encouraging. At an individual level, 18/22 (82%) of those patients who completed the intervention reached the cut-offs for absence of anxiety and depression (HADS scores < 8). Schneider et al.16 showed a range of recovery from anxiety and depression at 77%-85% that resembles our results. Regarding the three patients who had worsened HADS scores post-intervention, two had their treatment prolonged. Therefore, their follow-up HADS questionnaire was filled out before the intervention was completed, and in both cases, their therapist assessed clinical improvement at the end of the intervention. The third patient was severely impacted by COVID-19 and the associated social isolation. Out of the 22 completed patients, three stopped filling out the follow-up battery of questionnaires before they reached the HADS questionnaire, despite receiving two reminders. Since HADS will be the primary outcome of the eMindYourHeart RCT, HADS will be moved to the top of the follow-up questionnaire to safeguard this measure. In addition, the therapist will systematically encourage the patients to respond to this questionnaire at their last contact.

Use of intervention

Regarding use of intervention, Lundgren et al.15 found that number of logins was associated with improvements in depression in the ICBT group. However, data on numbers of logins and range are not reported making comparison difficult. Our results somewhat resemble the results of Schneider et al.16 in their RCT, as mean numbers of login were 33.5 (SD 13.4) vs. 26.4 (SD 16.6) and mean numbers of phone calls 8.0 (SD 3.9) vs. 4.8 (SD 3.1). The number of phone calls in this study was higher than expected which could be explained by the study design, testing, and adjusting procedures. Another explanation is the start of the COVID-19 pandemic in the trial period, which induced extra calls from patients. Our analysis showed that 40.6% of logins happened outside normal opening hours of CR services, which emphasizes that patients might prefer treatment at other timepoints than currently offered. One explanation could be that negative thoughts and emotions intrude at night-time when there is more time to think and ruminate. Concerning the duration of intervention, we found an average of 13.3 weeks that is longer than the stipulated 12 weeks. One reason is delay in time for some patients between the diagnostic interview and the start of first module, mostly caused by technical issues. Moreover, some patients requested extra time to complete the intervention due to personal situation. In the RCT, the length of the intervention will be monitored from start to end of the modules and the time kept to 12 weeks, making comparison easier.

Patient evaluation

A relatively high proportion of patients had a high educational level. We can only speculate if patients with low levels of education may be more reluctant to participate in ICBT or if CR staff are more reluctant to include patients with low levels of education, since this is seen in previous studies.16,36 Since all included patients with high levels of education completed the current intervention, it could also mean that our intervention might not be properly adjusted to participants with lower levels of education, despite efforts in the participatory design study.24 Some patients found the written assignments burdensome, signalling the need for personalized levels of the modules, which could potentially be related to level of education. Challenges with burden of assignments are also described previously.36,37 Wallin et al.36 argue for a more personalized ICBT approach, as some patients perceive text-based material strenuous and time-consuming. We have accordingly adjusted the one-size-fits-all approach to a more personalized model in the following RCT. Also consistent with the findings from Wallin et al.,36 some of our patients found the two-step logon procedure burdensome but given the GDPR rules, this is the best possible solution momentarily. Conversely, it might give other patients a feeling that the platform is secure and trustworthy, which is previously reported.37 Some patients expressed a preference to print out material from the platform. Previous studies have described this,12 and e.g. the Mindspot Clinic in Australia are sending out hard copies of the material upon request.38 Although this will not be possible in the RCT, since it will refrain us from collecting data on user activity on the platform, it should be considered if implementing into routine CR. Most patients found the intervention easily accessible and manageable and all were highly satisfied with their personal therapist and reported positive personal gains related to their mental condition. As one wrote: ‘I have become myself again, this is pure happiness’.

Nurses evaluation

The findings from the nurses’ evaluations revealed strong support for the concept of ICBT integrated into CR, as this gives patients fast access to psychological treatment. They appreciated the high level of availability of the eMindYourHeart study team, which might have a positive impact on patient recruitment. Still, it is evident that the CR nurses have a high workload and they struggled with inclusion procedures as it takes clinical time from their patients, which suggests that implementation is a complex process.39 It is therefore necessary to support the nurses to keep them engaged in recruiting patients, and in case of future implementation in routine care, a solid implementation plan is needed.

The European Society of Cardiology supports digital health as an innovative opportunity to improve the quality of care in secondary prevention.9 Especially in the light of the COVID-19 pandemic, ICBT might be an effective solution to treat anxiety and depression in patients with IHD as an integrated part of CR.40 Patients expressed high levels of gratefulness for the continuity of this intervention and the possibility to talk about their concerns related to COVID-19, while experiencing most elements of CR being cancelled. We managed to adapt the intervention to help also with concerns related to COVID-19, as it became obvious that this induced anxiety and worries among the patients.

Strengths and limitations

Strength of this study include the multicentre approach, testing of inclusion procedures in a large variety of CR models and centres across Denmark, and adaptations and optimisation of the intervention. Collaboration with the recruiting centres prior to the RCT also included a webinar for the CR staff (on their request) which is likely to be an advantage for successful recruitment in the RCT study. The study had following limitations: firstly, there were 16 patients declining to participate in the study, and since we only have sparse data on these patients, we are not able to examine if they differ from participating patients. Thus, there might be unidentified subgroups not willing to participate in ICBT. Declining patients are asked to fill out an informed consent to use of data in the RCT and due to the considerably higher volume in this study, these data might reveal important knowledge. Secondly, since the feasibility study were uncontrolled the positive changes in outcomes should be interpreted with great caution. In addition, clinical data from the Danish Cardiac Rehabilitation Database were rather incomplete, and other national registries will be considered for the RCT. We asked patients and nurses to evaluate their experiences with the intervention and procedures in writing, while oral interviews potentially could have led to deeper insights into barriers and facilitators of the intervention, as done by e.g. Wallin et al.36 However, an in-depth qualitative analysis was not the scope of this study and would have required thorough elaboration.

Conclusion

This study demonstrated the feasibility of the eMindYourHeart study, based on drop-out rate, mean improvements in HADS scores, use of intervention and experiences from patients and CR nurses. The study was an essential prerequisite for a larger interventional study. Thus, the intervention will be adjusted according to the reported results and used in the following eMindYourHeart RCT.

Lead author biography

graphic file with name ztab037ilf1.jpg

Charlotte Helmark is a PhD candidate employed at Zealand University Hospital and working in the crossfield between cardiology and psychology.

Acknowledgements

The authors would like to thank all participating patients and healthcare staff involved in the study. We would also like to thank Nicolaj Knudsen and Richard Jensen from Aarhus University Hospital for their help with adjustments and data collection on the platform used for the eMindYourHeart intervention. Thank you to the Danish Cardiac Rehabilitation Database (DHRD) under the Danish Clinical Quality Program for access to clinical data.

Funding

This trial was supported with a grant from University of Southern Denmark’s Strategic funding, the Danish Heart Foundation (18-R124-A8309-22106), Region of Southern Denmark and Region of Zealand joint research foundation (A343), Odense University Hospital Research Foundation (72-A3575), Region Zealand Palliation Fund and Sygesikring Denmark (2020-0041. None of the funding agencies has had any role in neither the design of the study nor in the collection, analysis, and interpretation of the data or writing of the manuscript.

Conflict of interest: none declared.

Data availability

The data from this article cannot be shared publicly due to the use of confidential health information.

References

  • 1. Timmis A, Townsend N, Gale C, Grobbee R, Maniadakis N, Flather M, Wilkins E, Wright L, Vos R, Bax J, Blum M, Pinto F, Vardas P.. European Society of Cardiology: cardiovascular disease statistics 2017. Eur Heart J 2018;39:508–579. [DOI] [PubMed] [Google Scholar]
  • 2. Pogosova N, Saner H, Pedersen SS, Cupples ME, McGee H, Höfer S, Doyle F, Schmid JP, von Käne R.. Psychosocial aspects in cardiac rehabilitation: From theory to practice. A position paper from the Cardiac Rehabilitation Section of the European Association of Cardiovascular Prevention and Rehabilitation of the European Society of Cardiology. Eur J Prev Cardiol 2015;22:1290–1306. [DOI] [PubMed] [Google Scholar]
  • 3. Salzwedel A, Jensen K, Rauch B, Doherty P, Metzendorf M-I, Hackbusch M, Völler H, Schmid JP, Davos C.. Effectiveness of comprehensive cardiac rehabilitation in coronary artery disease patients treated according to contemporary evidence based medicine: update of the Cardiac Rehabilitation Outcome Study (CROS-II). Eur J Prev Cardiol 2020;27:1756–1774. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4. Richards SH, Anderson L, Jenkinson CE, Whalley B, Rees K, Davies P, Bennett P, Liu Z, West R, Thompson DR, Taylor RS.. Psychological interventions for coronary heart disease: Cochrane systematic review and meta-analysis. Eur J Prev Cardiol 2018;25:247–259. [DOI] [PubMed] [Google Scholar]
  • 5. Ambrosetti M, Abreu A, Corrà U, Davos CH, Hansen D, Frederix I, Iliou MC, Pedretti RF, Schmid JP, Vigorito C, Voller H, Wilhelm M, Piepoli MF, Bjarnason-Wehrens B, Berger T, Cohen-Solal A, Cornelissen V, Dendale P, Doehner W, Gaita D, Gevaert AB, Kemps H, Kraenkel N, Laukkanen J, Mendes M, Niebauer J, Simonenko M, Zwisler AO.. Secondary prevention through comprehensive cardiovascular rehabilitation: from knowledge to implementation. 2020 update. A position paper from the Secondary Prevention and Rehabilitation Section of the European Association of Preventive Cardiology. Eur J Prev Cardiol 2020;doi: 10.1177/2047487320913379. [DOI] [PubMed] [Google Scholar]
  • 6. Pedersen SS, Andersen CM.. Minding the heart: why are we still not closer to treating depression and anxiety in clinical cardiology practice? Eur J Prev Cardiol 2018;25:244–246. [DOI] [PubMed] [Google Scholar]
  • 7. Reavell J, Hopkinson M, Clarkesmith D, Lane DA.. Effectiveness of cognitive behavioral therapy for depression and anxiety in patients with cardiovascular disease: a systematic review and meta-analysis. Psychosom Med 2018;80:742–753. [DOI] [PubMed] [Google Scholar]
  • 8. Johansson P, Westas M, Andersson G, Alehagen U, Broström A, Jaarsma T, Mourad G, Lundgren J.. An internet-based cognitive behavioral therapy program adapted to patients with cardiovascular disease and depression: randomized controlled trial. JMIR Ment Health 2019;6:e14648. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9. Frederix I, Caiani EG, Dendale P, Anker S, Bax J, Böhm A, Cowie M, Crawford J, de Groot N, Dilaveris P, Hansen T, Koehler F, Krstačić G, Lambrinou E, Lancellotti P, Meier P, Neubeck L, Parati G, Piotrowicz E, Tubaro M, van der Velde E.. ESC e-Cardiology Working Group Position Paper: overcoming challenges in digital health implementation in cardiovascular medicine. Eur J Prev Cardiol 2019;26:1166–1177. [DOI] [PubMed] [Google Scholar]
  • 10. Andersson G, Cuijpers P, Carlbring P, Riper H, Hedman E.. Guided Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: a systematic review and meta-analysis. World Psychiatry 2014;13:288–295. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11. Andersson G, Titov N, Dear BF, Rozental A, Carlbring P.. Internet-delivered psychological treatments: from innovation to implementation. World Psychiatry 2019;18:20–28. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12. Titov N, Hadjistavropoulos HD, Nielssen O, Mohr DC, Andersson G, Dear BF.. From research to practice: ten lessons in delivering digital mental health services. J Clin Med 2019;8:1239. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13. Mehta S, Peynenburg VA, Hadjistavropoulos HD.. Internet-delivered cognitive behaviour therapy for chronic health conditions: a systematic review and meta-analysis. J Behav Med 2019;42:169–187. [DOI] [PubMed] [Google Scholar]
  • 14. Norlund F, Wallin E, Olsson EMG, Wallert J, Burell G, von Essen L, Held C.. Internet-based cognitive behavioral therapy for symptoms of depression and anxiety among patients with a recent myocardial infarction: the U-CARE Heart Randomized Controlled Trial. J Med Internet Res 2018;20:e88. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15. Lundgren JG, Dahlström Ö, Andersson G, Jaarsma T, Köhler AK, Johansson P.. Effect of guided web-based cognitive behavioral therapy on patients with depressive symptoms and heart failure: a pilot randomized controlled trial. J Med Internet Res 2016;18:e194. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16. Schneider LH, Hadjistavropoulos HD, Dear BF, Titov N.. Efficacy of internet-delivered cognitive behavioural therapy following an acute coronary event: a randomized controlled trial. Internet Interv 2020;21:100324. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17. Pedersen SS, Andersen CM, Ahm R, Skovbakke SJ, Kok R, Helmark C, Wiil U, Schmidt T, Olsen KR, Hjelmborg J, Zwisler A-D, Frostholm L.. Efficacy and cost-effectiveness of a therapist-assisted web-based intervention for depression and anxiety in patients with ischemic heart disease attending cardiac rehabilitation [eMindYourHeart trial]: a randomised controlled trial protocol. BMC Cardiovasc Disord 2021;21:20. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18. Bowen DJ, Kreuter M, Spring B, Cofta-Woerpel L, Linnan L, Weiner D, Bakken S, Kaplan CP, Squiers L, Fabrizio C, Fernandez M.. How we design feasibility studies . Am J Prev Med 2009;36:452–457. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19. Hansen HP, Tjørnhøj-Thomsen T.. Meeting the challenges of intervention research in health science: an argument for a multimethod research approach. Patient 2016;9:193–200. [DOI] [PubMed] [Google Scholar]
  • 20. Eldridge SM, Chan CL, Campbell MJ, Bond CM, Hopewell S, Thabane L, Lancaster GA; PAFS consensus group. CONSORT 2010 statement: extension to randomised pilot and feasibility trials. Pilot Feasibility Stud 2016;2:64. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21. Zigmond AS, Snaith RP.. The hospital anxiety and depression scale. Acta Psychiatr Scand 1983;67:361–370. [DOI] [PubMed] [Google Scholar]
  • 22. Bjelland I, Dahl AA, Haug TT, Neckelmann D.. The validity of the Hospital Anxiety and Depression Scale. An updated literature review. J Psychosom Res 2002;52:69–77. [DOI] [PubMed] [Google Scholar]
  • 23.Sundhedsstyrelsen. National Klinisk Retningslinje for Hjerterehabilitering. 2015. https://www.sst.dk/da/udgivelser/2015/nkr-hjerterehabilitering (10 February 2021).
  • 24. Schmidt T, Kok R, Andersen CM, Skovbakke SJ, Ahm R, Wiil UK, Frostholm L, Pedersen SS.. Development of an internet-delivered program and platform for the treatment of depression and anxiety in patients with ischemic heart disease in eMindYourHeart. Inform Health Social Care 2021;doi:10.1080/17538157.2021.1878185. [DOI] [PubMed] [Google Scholar]
  • 25. Hadjistavropoulos HD, Schneider LH, Mehta S, Karin E, Dear BF, Titov N.. Preference trial of internet-delivered cognitive behaviour therapy comparing standard weekly versus optional weekly therapist support. J Anxiety Disord 2019;63:51–60. [DOI] [PubMed] [Google Scholar]
  • 26. Richards D, Richardson T.. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev 2012;32:329–342 [DOI] [PubMed] [Google Scholar]
  • 27. Ritterband LM, Ardalan K, Thorndike FP, Magee JC, Saylor DK, Cox DJ, Sutphen JL, Borowitz SM.. Real world use of an Internet intervention for pediatric encopresis. J Med Internet Res 2008;10:e16. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28. Harris PA, Taylor R, Minor BL, Elliott V, Fernandez M, O'Neal L, McLeod L, Delacqua G, Delacqua F, Kirby J, Duda SN; REDCap Consortium. The REDCap consortium: Building an international community of software platform partners. J Biomed Inform 2019;95:103208. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29. Zwisler A-D, Rossau HK, Nakano A, Foghmar S, Eichhorst R, Prescott E, Cerqueira C, Soja AMB, Gislason GH, Larsen ML, Andersen UO, Gustafsson I, Thomsen KK, Boye Hansen L, Hammer S, Viggers L, Christensen B, Kvist B, Lindström Egholm C, May O.. The Danish cardiac rehabilitation database. Clin Epidemiol 2016;8:451–456. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30. Braun V, Clarke V.. Using thematic analysis in psychology. Qual Res Psychol 2006;3:77–101. [Google Scholar]
  • 31. Christensen AV, Dixon JK, Juel K, Ekholm O, Rasmussen TB, Borregaard B, Mols RE, Thrysøe L, Thorup CB, Berg SK.. Psychometric properties of the Danish Hospital Anxiety and Depression Scale in patients with cardiac disease: results from the DenHeart survey. Health Qual Life Outcomes 2020;18:9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32. Rickham PP. Human experimentation. Code of ethics of the World Medical Association. Declaration of Helsinkr. Br Med J 1964;2:177. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33. Lemay KR, Tulloch HE, Pipe AL, Reed JL.. Establishing the minimal clinically important difference for the hospital anxiety and depression scale in patients with cardiovascular disease. J Cardiopulm Rehabil Prev 2019;39:E6–E11. [DOI] [PubMed] [Google Scholar]
  • 34. Enrique A, Palacios JE, Ryan H, Richards D.. Exploring the relationship between usage and outcomes of an internet-based intervention for individuals with depressive symptoms: Secondary analysis of data from a randomized controlled trial. J Med Internet Res 2019;21:e12775. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35. Lundgren J, Andersson G, Dahlström Ö, Jaarsma T, Köhler AK, Johansson P.. Internet-based cognitive behavior therapy for patients with heart failure and depressive symptoms: a proof of concept study. Patient Educ Couns 2015;98:935–942. [DOI] [PubMed] [Google Scholar]
  • 36. Wallin E, Norlund F, Olsson EMG, Burell G, Held C, Carlsson T.. Treatment activity, user satisfaction, and experienced usability of internet-based cognitive behavioral therapy for adults with depression and anxiety after a myocardial infarction: Mixed-methods study. J Med Internet Res 2018;20:e87. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37. Lundgren J, Johansson P, Jaarsma T, Andersson G, Kärner Köhler A.. Patient experiences of web-based cognitive behavioral therapy for heart failure and depression: qualitative study. J Med Internet Res 2018;20:e10302. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38. Titov N, Dear B, Nielssen O, Staples L, Hadjistavropoulos H, Nugent M, Adlam K, Nordgreen T, Bruvik KH, Hovland A, Repål A, Mathiasen K, Kraepelien M, Blom K, Svanborg C, Lindefors N, Kaldo V.. ICBT in routine care: a descriptive analysis of successful clinics in five countries. Internet Interv 2018;13:108–115. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39. Egholm CL, Helmark C, Doherty P, Nilsen P, Zwisler AD, Bunkenborg G.. “Struggling with practices”—a qualitative study of factors influencing the implementation of clinical quality registries for cardiac rehabilitation in England and Denmark. BMC Health Serv Res 2019;19: 102. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40. Neubeck L, Hansen T, Jaarsma T, Klompstra L, Gallagher R.. Delivering healthcare remotely to cardiovascular patients during COVID-19: a rapid review of the evidence. Eur J Cardiovasc Nurs 2020;19:486–494. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

The data from this article cannot be shared publicly due to the use of confidential health information.


Articles from European Heart Journal. Digital Health are provided here courtesy of Oxford University Press on behalf of the European Society of Cardiology

RESOURCES