Skip to main content
Journal of Medical Internet Research logoLink to Journal of Medical Internet Research
. 2021 Nov 11;23(11):e28874. doi: 10.2196/28874

Mobile Diary App Versus Paper-Based Diary Cards for Patients With Borderline Personality Disorder: Economic Evaluation

Sidsel Lund Laursen 1, Stig Helweg-Jørgensen 2, Astrid Langergaard 1,#, Jesper Søndergaard 1, Sabrina Storgaard Sørensen 1, Kim Mathiasen 2,3, Mia Beck Lichtenstein 2,3, Lars Holger Ehlers 1,
Editor: Rita Kukafka
Reviewed by: Iyubanit Rodriguez, Sara Landes
PMCID: PMC8663638  PMID: 34762057

Abstract

Background

The cost-effectiveness of using a mobile diary app as an adjunct in dialectical behavior therapy (DBT) in patients with borderline personality disorder is unknown.

Objective

This study aims to perform an economic evaluation of a mobile diary app compared with paper-based diary cards in DBT treatment for patients with borderline personality disorder in a psychiatric outpatient facility.

Methods

This study was conducted alongside a pragmatic, multicenter, randomized controlled trial. The participants were recruited at 5 Danish psychiatric outpatient facilities and were randomized to register the emotions, urges, and skills used in a mobile diary app or on paper-based diary cards. The participants in both groups received DBT delivered by the therapists. A cost-consequence analysis with a time horizon of 12 months was performed. Consequences included quality-adjusted life years (QALYs), depression severity, borderline severity, suicidal behavior, health care use, treatment compliance, and system usability. All relevant costs were included. Focus group interviews were conducted with patients, therapists, researchers, and industry representatives to discuss the potential advantages and disadvantages of using a mobile diary app.

Results

A total of 78 participants were included in the analysis. An insignificantly higher number of participants in the paper group dropped out before the start of treatment (P=.07). Of those starting treatment, participants in the app group had an average of 37.1 (SE 27.55) more days of treatment and recorded an average of 3.16 (SE 5.10) more skills per week than participants in the paper group. Participants in both groups had a QALY gain and a decrease in depression severity, borderline severity, and suicidal behavior. Significant differences were found in favor of the paper group for both QALY gain (adjusted difference −0.054; SE 0.03) and reduction in depression severity (adjusted difference −1.11; SE 1.57). The between-group difference in total costs ranged from US $107.37 to US $322.10 per participant during the 12 months. The use of services in the health care sector was similar across both time points and groups (difference: psychiatric hospitalization <5 and <5; general practice −1.32; SE 3.68 and 2.02; SE 3.19). Overall, the patients showed high acceptability and considered the app as being easy to use. Therapists worried about potential negative influences on the therapist-patient interaction from new work tasks accompanying the introduction of the new technology but pointed at innovation potential from digital database registrations.

Conclusions

This study suggests both positive and negative consequences of mobile diary apps as adjuncts to DBT compared with paper diary cards. More research is needed to draw conclusions regarding its cost-effectiveness.

Trial Registration

ClinicalTrials.gov NCT03191565; http://clinicaltrials.gov/ct2/show/NCT03191565

International Registered Report Identifier (IRRID)

RR2-10.2196/17737

Keywords: borderline personality disorder, dialectical behavior therapy, mobile app, psychotherapy, cost-consequence, mHealth, mobile phone

Introduction

Background

Borderline personality disorder (BPD) is estimated to affect between 0.7% and 4.8% of the general population in Scandinavia and accounts for approximately 15% of adult admissions in Danish psychiatric hospitals [1-3]. Patients diagnosed with BPD are often characterized as emotionally unstable, impulsive, and self-harming and have unstable relationships [4]. It has been shown that the mortality rate is 8 times higher in patients with BPD than in the general population [5]. Furthermore, BPD has large societal consequences, as one study has found that the total mean societal cost related to BPD is US $32,863.01 per patient per year [6].

One of the most well-researched and clinically effective psychosocial treatments for patients with BPD is dialectical behavior therapy (DBT) [7-12]. Studies have shown DBT to be effective in reducing nonsuicidal self-injury and increasing quality-adjusted life years (QALYs), when compared with treatment as usual or client-centered therapy [13-16]. The DBT includes elements from cognitive-behavioral therapy and is centered on learning a set of predefined skills, such as mindfulness, tolerance of distress, regulation of emotions, and navigation of interpersonal situations [17,18]. Learning the DBT skills is done through self-monitoring using weekly paper-based diary cards filled out by the patient [19,20]. The use of paper-based diary cards, however, has limitations for both patients and therapists. Working with paper diaries is burdensome and may reduce patient engagement in treatment [21,22]. Retrieving and reviewing paper diaries to evaluate the patient’s progress over time is a time-consuming process for the therapist, and it is not possible for the therapist to follow the patient alongside self-treatment at home [20-22].

New studies suggest that mobile diary apps can overcome some of these limitations and be more effective in reducing distress symptoms and suicidal and nonsuicidal self-harming behaviors compared with treatment with DBT alone [23-25]. Mobile apps have the potential to reduce barriers and increase engagement in treatment [21,22]. Mobile apps can automatically provide an overview of the patients' progress and enable therapists to monitor the present emotional stage of patients during treatment [21,24,26].

Objective

Economic evaluations of new health technology constitute important knowledge for decision-makers in health care as they estimate the value for money of the technology compared with alternative use of the scarce financial resources. To our knowledge, no economic evaluation has yet been conducted on the use of mobile apps as adjuncts to DBT in patients with BPD. Therefore, the aim of this economic evaluation is to investigate the costs and consequences of a mobile diary app compared with paper-based diary cards for patients with BPD, within the context of a psychiatric outpatient facility.

Methods

Overview

This economic evaluation was reported in line with the Consolidated Health Economic Evaluation Reporting Standards [27]. It was conducted alongside a pragmatic, multicenter, randomized controlled trial as part of the ENTER program [28]. The clinical study is described in detail in the study protocol [29] but will be summarized in the Methods section of this evaluation. Economic evaluation was carried out as a cost-consequence analysis, which is why costs and consequences are shown separately [30]. The cost-consequence analysis approach was chosen because of the possibility of investigating a broad range of consequences [30], such as intermediate outcomes and qualitative aspects. The time horizon was restricted to 12 months, and all relevant costs were included in the analysis. All costs were measured in Danish Krone (DKK) and subsequently converted to the US dollars (US $).

Participants

The participants were recruited from 5 Danish psychiatric outpatient facilities between June 2017 and December 2018. The participants were eligible for inclusion if they were ≥18 years of age, formally diagnosed by a psychiatric specialist with Emotionally Unstable Personality Disorder (F60.3 according to ICD-10 criteria) [31], and admitted for psychiatric outpatient treatment. To be included, participants also had to be either self-harming or to have had suicidal behavior within the last 3 years. Participants with comorbid disorders, such as depression, anxiety, and posttraumatic stress disorder, were allowed to participate. On the contrary, participants were excluded if they had no access to a working smartphone; an IQ <70; or a comorbid disorder such as substance abuse, bipolar disorder, or a disorder within the schizophrenic spectrum. Before entering the study, all participants signed an informed consent form.

Interventions

Participants were randomly assigned to either the mobile diary app group or to the paper-based diary card group using REDCap (Research Electronic Data Capture) [29]. In both groups, participants had to register entries about aspects such as emotional dysregulation, suicidal and self-harm thoughts, and skill use. The participants received the paper-based treatment registered daily on paper diary cards, whereas the participants using the mobile diary app were registered daily via the mDiary app (The Monsenso system). The mDiary app is a market solution containing psychoeducational material and visualizations of the participant’s data, which can be used in therapy sessions and for real time monitoring alongside treatment [32]. Participants in both groups received standard DBT (delivered as individual sessions, group sessions, and telephone coaching). The therapists instructed the participants on how to use the app or paper diary cards. Other treatments, such as medications, were allowed. Passive sensor data (eg, activity level and phone use) were collected from all participants through their smartphones. The treatment period was 40 weeks at 2 sites and 12 months at the other sites, and neither the participants nor the therapists were blinded to the group allocation.

Consequences

Health-Related Outcomes

Health-related quality of life was based on the EuroQol 5-Dimensions 5-Levels (EQ-5D-5L) questionnaire [33]; depression severity from the Patient Health Questionnaire-9 [34]; borderline severity was assessed using the Zanarini Rating Scale for BPD [35]; and suicide behavior from the Suicide Behaviors Questionnaire [36]. Data were collected at baseline and at the 12-month follow-up. The participants' utility scores were calculated using the Danish weights for the EQ-5D-5L questionnaire [37]. QALY gain was estimated by assuming linear interpolation between utility scores at baseline and a 12-month follow-up, after which the area under the curve was calculated [38]. For depression severity, borderline severity, and suicide behavior, the difference from baseline to follow-up was calculated and compared between the groups.

Health Care Sector

In Denmark, the civil registration number makes it possible to link information from several registries to an individual [39]. Information on psychiatric hospitalizations and consultations with general practice was obtained at an individual level from the Primary National Health Insurance Service Register and the Danish National Patient Registry [40,41]. The specific purpose of the consultations in general practice was not available, which is why all consultations were included. The number of psychiatric hospitalizations and consultations with general practice was estimated 30 days before and after baseline and 12 months before and after baseline, respectively.

Treatment Compliance and Skill Recordings

It was continuously assessed whether the participants were still receiving treatment and whether they were recording their skills use. The recording of the skills used per week was estimated at an individual level as the total number of skills for each participant divided by the number of days in treatment, which was subsequently calculated for a week. A participating therapist estimated the time spent on recording in the 2 groups based on experience from the trial.

The usability of the mobile diary app was assessed by patients and therapists using a system usability scale [42]. This assessment was conducted in focus group interviews alongside a feasibility study of the app.

Future innovation potential was assessed by app developers, researchers, and other personnel related to the study. This assessment was conducted during a workshop to discuss the advantages, disadvantages, and development potential.

Costs

All relevant costs were included in the analysis and adjusted to the price level in 2019 using the Danish net price index [43]. The conversion rate from kr to US $ was kr 634.0 per US $100 from September 23, 2021 [44]. The costs were not discounted because of a time horizon of 12 months [45].

The costs of health care services, patient costs, and municipality costs were used to adjust for baseline differences in the analysis. Health care services included primary care services, prescription medicine, inpatient and outpatient services, and emergency contacts to psychiatric or somatic hospitals. Patient costs included out-of-pocket medical expenses, including expenses for both medicine and primary care services. Municipality costs included nurse care, daily care, and domestic help at home. Information on each participant’s use of health care services and prices for public services was retrieved from the Danish Primary National Health Insurance Service Register, the Danish National Patient Registry, and the Danish National Prescription Registry [40,41,46]. Information regarding municipality costs was collected based on the Treatment Inventory of Costs in Psychiatric Patients questionnaire [47], which was administered to all participants at baseline. Baseline costs were calculated for resource use 12 months before the start of the study until the baseline date.

Information on intervention costs was provided by the key personnel at a psychiatric outpatient facility. The intervention costs of the app were estimated as the capital and operating costs. Capital costs included the one-time cost of starting and establishing the mobile diary app program and the education of therapists on how to use the program’s software. Operating costs included costs, such as program licenses. Startup establishment and program license costs were estimated as a range due to price instability of new devices, especially if the device is new in the market [48,49]. The pricing was estimated in collaboration with the software company and public purchasers, with an expected discount if the software were to be implemented in Denmark. The license fee was allocated to an estimated number of patients per therapist. The costs of educating the therapists were estimated using the time allocated for training and the average effective hourly wages for the therapists. The wages were based on information from Statistics Denmark on the national average gross wages corresponding to the therapists’ occupation [50]. Capital costs were annuitized over 3 years, with a discount rate of 4% per year [51,52]. Subsequently, the costs were allocated to an estimated number of patients per therapist per year and were included in the analysis as annual costs. The relevant intervention costs in the paper group included the price of paper and the printing of diaries for 12 months. The choice between the app and paper was assumed to not influence the overhead costs. See Multimedia Appendix 1 [43,44,48-53] for further information on cost estimation.

Statistical Analysis

To account for missing data, multiple imputation was performed, which was applicable because the missing data were assumed to be missing at random [54,55]. The consequences of borderline severity and suicide behavior are shown as nonimputed estimates due to a high number of missing values at both baseline (17 and 20 missing values, respectively) and follow-up (20 and 17 missing values, respectively). The analyses were conducted using an intention-to-treat analysis. See Multimedia Appendix 2 [54] for a summary of the missing data and a full description of the imputation model.

Data are reported either as the number of participants in each group and percentages, median, and IQR, or as means and SE. Incremental health-related outcomes were estimated as both unadjusted and adjusted outcomes based on the regression analysis. The health-related outcomes were adjusted for age, sex, relationship status, education level, baseline scores (utility score, depression score, suicide score, and borderline score), and baseline costs to control for baseline differences [38]. For consequences related to the health care sector and treatment compliance, P values were calculated using the Fisher exact test for categorical variables and Student t test (2-tailed) for continuous variables [56]. Statistical significance was set at a P value <.05. Statistical analyses were performed using the STATA (version 16.1, StataCorp).

Ethics and Consent to Participate

The trial was registered at ClinicalTrials.gov (identifier NCT03191565) and was performed in accordance with the Declaration of Helsinki and approved by the Danish Ethics Committee in the Region of Southern Denmark (Registration number S-20160085). Participants were recruited between June 2017 and December 2018. All participants signed an informed consent form before participating in the study.

Data Sharing Statements

The data sets generated and analyzed during this study are available from the corresponding author upon reasonable request.

Results

Overview

In total, 79 participants were enrolled in the trial, of whom 78 were eligible for inclusion in the analysis. Participants in both groups were predominantly women, with a mean age of 29 years (Table 1).

Table 1.

Characteristics of the participants at baseline.

Study population Mobile diary app (N=42) Paper diary cards (N=36)
Age (years), median (IQR) 26.4 (22.2-32.8) 27.2 (22.3-34.6)
Sex (female), n (%) >37 (≥88) >31 (≥86)
Cohabiting status, n (%)

Living with somebody 14 (33) 18 (50)

Living alone 28 (67) 18 (50)
Education, n (%)

Education <3 yearsa >37 (≥88) 27 (75)

Education ≥3 yearsb <5 (≤12) 9 (25)
Employment, n (%)

In employment 9 (21) 9 (25)

Not in employment 33 (79) 27 (75)
Costsc (US $), mean (SE)

Health care costs 17,200.35 (3351.31) 15,684.14 (2800.31)

Municipality costs 7785.36 (5680.32) 250.08 (140.15)

Patient costs 303.66 (49.47) 327.02 (48.09)

Total costs 25,289.37 (6918.34) 16,378.66 (2805.81)

aParticipants with highest education of public school, high school, further education <3 years and apprenticeship.

bParticipants with further education ≥3 years.

cAnnual cost the year before the study start date.

Consequences

Participants in both groups experienced an increase in utility scores from baseline to the 12-month follow-up (Table 2). The adjusted QALY gain difference between the app group and the paper group was −0.054 (SE 0.03; P<.001), indicating a smaller QALY gain in the app group. The unadjusted QALY gain difference was close to the adjusted estimate. Both groups showed improvements in the EQ-5D domains, except for domain 1 (mobility) in the app group and domain 2 (self-care) in the paper group. The improvement in domain 5 (anxiety and depression) was comparable between the 2 groups.

Table 2.

Quantitative consequences of the mobile diary app and paper-based diary cards.

Health-related outcomes Mobile diary app (N=42) Paper diary cards (N=36) Unadjusted between-group difference P value Adjusted between-group difference P value
EQ-5Da levels, utility and QALYb, mean (SE)c

Baseline d


Domain 1- Mobility 1.86 (0.15) 1.89 (0.17)





Domain 2—Self-care 1.52 (0.14) 1.31 (0.10)





Domain 3—Usual activities 2.86 (0.14) 2.67 (0.15)





Domain 4—Pain or discomfort 2.93 (0.16) 2.58 (0.21)





Domain 5—Anxiety or depression 3.17 (0.17) 3.11 (0.15)





Utility score 0.45 (0.05) 0.51 (0.05)




Follow-up


Domain 1-Mobility 2.2 (0.22) 1.57 (0.13)





Domain 2-Self-care 1.37 (0.13) 1.41 (0.14)





Domain 3-Usual activities 2.56 (0.19) 1.95 (0.17)





Domain 4-Pain or discomfort 2.66 (0.17) 2.30 (0.15)





Domain 5-Anxiety or depression 2.41 (0.17) 2.33 (0.18)





Utility score 0.6 (0.06) 0.74 (0.03)




QALY gain 0.078 (0.03) 0.115 (0.03) -0.037 (0.04) .38 -0.054 (0.03) <.001
Borderline severity, mean (SE)e

Baseline 18.6 (1.22) 19.5 (1.08)

Follow-up 8.82 (1.16) 8.83 (1.45)

Change within the group 9.85 (1.28) 10.67 (1.39) −0.81 (1.93) .67 −0.67 (1.73) .004
Depression severity, mean (SE)c

Baseline 17.52 (0.80) 16.11 (0.88)

Follow-up 12.34 (1.03) 10.47 (1.3)

Change within the group 5.18 (1.01) 5.64 (1.38) −0.46 (1.67) .78 −1.11 (1.57) .04
Suicidal behavior, mean (SE)e

Baseline 11.60 (0.75) 11.96 (0.88)

Follow-up 8.86 (0.58) 9.38 (0.62)

Change within the group 2.79 (0.59) 2.76 (0.72) 0.03 (0.93) .98 0.47 (0.73) <.001

aEQ-5D: EuroQol 5-Dimensions.

bQALY: quality-adjusted life year.

cImputed data set.

dEmpty cells concern descriptive measures not relevant for statistical testing.

eVariables with missing values.

Participants in both groups had decreased borderline severity, depressive severity, and suicidal behavior during the 12-month follow-up. The adjusted difference in the change within the group in depression severity between the app group and the paper group was −1.11 (SE 1.57; P=.04), indicating that participants in the app group had a smaller decrease in depression severity. The unadjusted differences showed a similar result. In addition, participants in the app group had a smaller decrease in borderline severity but a greater decrease in suicidal behavior. These results should be interpreted with reservations due to a high number of missing values at both baseline and at follow-up.

The use of health care sector services was similar across both the periods and the groups (difference: psychiatric hospitalization <5 and <5; general practice −1.32, SE 3.68 and 2.02, SE 3.19), indicating that registering in the app compared with on paper does not affect the resource use in the health care sector (Table 3).

Table 3.

Quantitative consequences of the mobile diary app and paper-based diary cards.

Health-related outcomes Mobile diary app (n=42) Paper diary cards (n=36) Between-group difference P value
Health care sector contacts
Participants hospitalized in a psychiatric hospital, n (%)

30 days before baseline <5 (≤12) <5 (≤14) ≤5 >.99

30 days after baseline <5 (≤12) <5 (≤14) ≤5 >.99
General practice contacts, mean (SE)

12 months before baseline 19.42 (2.4) 20.75 (2.83) −1.32 (3.68) .72

12 months after baseline 19.02 (2.18) 17 (2.32) 2.02 (3.19) .53
Treatment compliance

Participants, who never start treatment, n (%) <5 (≤12) 7 (19) ≤−2 .07

Number of treatment days for participants starting treatment, mean (SE) 265.93 (18.64) 228.83 (19.64) 37.1 (27.55) .18
Skill recordings, mean (SE)

All participants


Skills recorded per patient 647.93 (150.72) 409.89 (173.99) 238.04 (229) .30

Participants starting treatment


Skills recorded per patient 680.33 (156.56) 508.83 (212.54) 171.5 (257.9) .51


Per week 15.04 (3.04) 11.88 (4.29) 3.16 (5.10) .54
Time spent on recording

Time spent on completing the diary (minutes) 4 5 −1 a
System Usability Scale, mean (SD)

Patients (n=16) 81.2 (9.9)

Therapists (n=23) 61.6 (18.6)

aSystem usability scale is only reported for the mobile diary group.

The paper group had an insignificantly higher number of participants dropping out before the start of treatment compared with the app group (P=.07). Of those starting treatment, participants in the app group were more persistent in adhering to treatment and had an average of 37.1 (SE 27.55) more days of treatment than the paper group. Participants in the app group registered 3.16 (SE 5.10) more skills per week compared with participants in the paper group. Therapists and researchers estimated that the app was time saving for the patient, approximately 1 minute on each diary entry. They further pointed out a potential utility gain for patients from the possibility of choosing according to preferences between registering in an app or on paper diary cards.

The System Usability Score was 81.2 (SD 9.9) in patients, but significantly lower at 61.6 (SD 18.6) in therapists (for further information see Marceau et al [22]). In general, patients were satisfied with the mobile solution, whereas the therapists expressed concern regarding the potential for a negative influence on the patient-therapist interaction due to new tasks related to the implementation of the new technology [22]. In addition, reduced flexibility in data collection due to the standard structure of the app was described as a potential disadvantage. Finally, numerous implementation issues were highlighted in the focus groups, including concerns relating to a potential lack of technical skills of both the patient and therapist, and security and storage of the collected data.

Future potential for innovation was considered relevant as electronic data recording can be used to predict outcomes and benefits from changes in treatment strategy (Textbox 1). The mobile diary app could provide therapists with information on the patients alongside treatment, thereby making it possible to define cut-off values for intervening outside sessions. Program optimization and visualization of the entered data could further inform the patients about their progress, potentially increasing the patient’s self-insight and empowerment.

Innovation potential of the mobile diary app and paper-based diary cards.

Innovation potential

  • Real time recording makes it possible to track patients’ symptoms and possibly intervene outside sessions.

  • Electronic data from the app can potentially be used for predicting outcomes from treatment.

Costs

The between-group difference in total costs during the 12-months ranged from US $107.38 to US $322.13 per participant (Table 4).

Table 4.

Intervention costs (US $) per participant at 12-month follow-up.

Intervention cost Mobile diary app (US $) Paper diary cards (US $) Difference (US $)
Startup costa 9.41-28.44 0 9.41-28.24
License 97.96-293.88 0 97.96-293.88
Education 3.27 0 3.27
Paper and print 0 3.26 3.26
Total 110.64-325.39 3.26 107.38-322.13

aAnnuitized over 3 years with a 4% discount rate.

Discussion

Principal Findings

To our knowledge, this study is the first to evaluate the costs and consequences of a mobile diary app for patients with BPD. Our findings suggest that using a mobile app as an adjunct to DBT comes with a high acceptance of patients, and the potential for increased treatment adherence and future innovation from electronic database registrations. We also found, however, that the app was associated with a reduced improvement in patients’ depression and health-related quality of life scores within the first year of treatment. The app led to higher costs but was still a relatively inexpensive intervention.

The economic evaluation was conducted using a cost-consequence approach rather than the traditional cost-utility (ie, cost-per-QALY gained) framework. The cost-consequence approach has been criticized by health economists for not being a normative analysis, as the decision-maker has to choose from her or his own opinion concerning the relative importance of the list of consequences [30]. As the mobile diary app was still in early development, we believe there is a need to investigate a broad range of potential consequences, including intermediate consequences. Indeed, health professionals should be encouraged to engage in the debate about how to evaluate internet-based interventions in mental health. Health economists have concluded that the economic evaluation of devices in health care, in general, represents a number of methodological challenges; in particular, the use of QALY as an outcome may be too narrow to capture all relevant benefits to patients and health care organizations [47]. Specifically, within mental health, the use of QALY and EQ-5D has also been found to be challenging with regard to instrument sensitivity [57].

Participants in both the app and paper groups had a QALY gain, but the gain was significantly larger in the paper group. This result is difficult to explain as the difference in QALY gain between the groups was caused by a worsening in the mobility domain of the EQ-5D-5L among app users (Table 2). There is a risk that patients have interpreted the EQ-5D mobility domain differently, either as a question about physical restrictions or as limitations on mobility due to anxiety or depression. Nevertheless, the results suggest that using mobile apps as adjuncts to DBT might reduce the improvement in quality of life and depression scores within the first year.

There are several limitations to this study, including the small sample size, short follow-up period, and missing data. The missing data on depression severity and utility domains were handled by performing multiple imputations and were assessed as not having an impact on the results [54]. There was a considerable amount of missing data for both borderline severity and suicide behavior, making it infeasible to run the imputation model for these outcomes. Therefore, borderline severity and suicidal behavior were presented as nonimputed data, potentially indicating that these results could be biased [55]. Previously, a moderate correlation has been shown between the EQ-5D utility scores and measures for borderline severity, supporting our findings of a similar improvement in QALY and borderline severity [58].

Even though participants in the app group recorded more skills, this did not contribute to a larger QALY gain in the app group than in the paper group. One possible explanation is that a measurement of an effect like health-related quality of life might not capture the potential effects of the app [59], which is why including other effects is relevant. In this study, potential effects such as new and better information from the patients are highlighted, which is supported by the findings of Lauritsen [60], who concluded that the use of a mobile electronic diary can improve the quality and accuracy of data entered into the diary. A study by Derks et al [61] found that a mobile diary app together with a biosensor was effective in increasing self-awareness in patients with BPD. Similar results were found in this study, as key personnel stated that increased self-insight and empowerment might be a potential advantage linked to self-monitoring. Furthermore, both in this study and in prior studies [62,63], a mobile app has been proven to enhance treatment compliance. Thus, if this study had been conducted as a cost-utility, the aforementioned consequences would not have been included, meaning the conclusion might have been quite different.

In this study, concerns regarding the app were primarily from therapists who worried that the app would negatively affect their interaction with the patient. In contrast, findings by Austin et al [64] suggests that a mobile app integrated into DBT was useful for patients with BPD in building an alliance with their therapist and that it promoted collaboration between therapist and patient. However, it is important to state that participants who enrolled in this study and the study by Austin et al [64] might have been advocates for technology since they knew the new and alternative treatment would be a mobile app, which could explain why patients were pleased with the app and why it was primarily therapists who had concerns. Thus, the impact of a mobile diary app on the therapist-patient relationship should be explored in future research.

Conclusions

In conclusion, this study suggests that using a mobile diary app as an adjunct to DBT has positive as well as negative consequences for patients and, on average, will lead to higher costs than paper-based diary cards. The use of a mobile diary app in the treatment of patients with BPD is still a relatively new field of research, and further investigation in this area is essential.

Acknowledgments

The authors would like to thank all the participants, both patients and therapists, at 5 different locations. Furthermore, the authors would like to acknowledge the staff at the Center for Telepsychiatry in the Region of Southern Denmark, University of Southern Denmark, and Monsenso for contributing to the mDiary program.

SLL played a primary role in the economic evaluation. Contributions to the design of the trial and data collection were described in the protocol [29]. SLL, AL, JS, SSS, and LHE developed the analytic plan. SLL and AL analyzed the data. SLL, AL, JS, SSS, and LHE participated in data interpretation. SLL drafted the first version of the manuscript. All authors provided critical feedback, helped shape the analysis and writing of the manuscript, and approved the submission.

This study is part of the ENTER project funded by Innovation Fund Denmark (grant number 5159-00002B). The Innovation Fund Denmark has not been involved in the design of the study or the subsequent handling of data. More information regarding the Innovation Fund Denmark can be found at the website [65].

Abbreviations

BPD

borderline personality disorder

DBT

dialectical behavior therapy

EQ-5D-5L

EuroQol 5-Dimensions 5-Levels

QALY

quality-adjusted life year

Multimedia Appendix 1

Detailed information on cost estimates in the economic analysis.

Multimedia Appendix 2

A detailed description of the used approach for multiple imputation.

Footnotes

Conflicts of Interest: The mDiary program was developed and owned by the private sector company Monsenso. SHJ has been employed at a public sector psychiatric ward, and he has no affiliation with Monsenso, nor does any of the other researchers.

References

  • 1.Ekselius L, Tillfors M, Furmark T, Fredrikson M. Personality disorders in the general population: DSM-IV and ICD-10 defined prevalence as related to sociodemographic profile. Pers Individ Dif. 2001 Jan 19;30(2):311–20. doi: 10.1016/s0191-8869(00)00048-9. [DOI] [Google Scholar]
  • 2.Torgersen S, Kringlen E, Cramer V. The prevalence of personality disorders in a community sample. Arch Gen Psychiatry. 2001 Jun;58(6):590–6. doi: 10.1001/archpsyc.58.6.590.yoa20243 [DOI] [PubMed] [Google Scholar]
  • 3.Pedersen L, Simonsen E. Incidence and prevalence rates of personality disorders in Denmark-a register study. Nord J Psychiatry. 2014 Nov;68(8):543–8. doi: 10.3109/08039488.2014.884630. [DOI] [PubMed] [Google Scholar]
  • 4.National Collaborating Centre for Mental Health (UK) Borderline Personality Disorder: Treatment and Management. Leicester (UK): The British Psychological Society & The Royal College of Psychiatrists; 2009. [PubMed] [Google Scholar]
  • 5.Kjær JN, Biskin R, Vestergaard C, Munk-J Rgensen P. All-cause mortality of hospital-treated borderline personality disorder: a nationwide cohort study. J Pers Disord. 2020 Dec;34(6):723–35. doi: 10.1521/pedi_2018_32_403. [DOI] [PubMed] [Google Scholar]
  • 6.Wagner T, Fydrich T, Stiglmayr C, Marschall P, Salize H, Renneberg B, Fleßa S, Roepke S. Societal cost-of-illness in patients with borderline personality disorder one year before, during and after dialectical behavior therapy in routine outpatient care. Behav Res Ther. 2014 Oct;61:12–22. doi: 10.1016/j.brat.2014.07.004.S0005-7967(14)00107-7 [DOI] [PubMed] [Google Scholar]
  • 7.Stoffers JM, Völlm BA, Rücker G, Timmer A, Huband N, Lieb K. Psychological therapies for people with borderline personality disorder. Cochrane Database Syst Rev. 2012 Aug 15;(8):CD005652. doi: 10.1002/14651858.CD005652.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Rizvi SL, Steffel LM, Carson-Wong A. An overview of dialectical behavior therapy for professional psychologists. Prof Psychol Res Pr. 2013;44(2):73–80. doi: 10.1037/a0029808. [DOI] [Google Scholar]
  • 9.Linehan MM, Comtois KA, Murray AM, Brown MZ, Gallop RJ, Heard HL, Korslund KE, Tutek DA, Reynolds SK, Lindenboim N. Two-year randomized controlled trial and follow-up of dialectical behavior therapy vs therapy by experts for suicidal behaviors and borderline personality disorder. Arch Gen Psychiatry. 2006 Jul;63(7):757–66. doi: 10.1001/archpsyc.63.7.757.63/7/757 [DOI] [PubMed] [Google Scholar]
  • 10.Linehan MM, Dimeff LA, Reynolds SK, Comtois KA, Welch SS, Heagerty P, Kivlahan DR. Dialectical behavior therapy versus comprehensive validation therapy plus 12-step for the treatment of opioid dependent women meeting criteria for borderline personality disorder. Drug Alcohol Depend. 2002 Jun 01;67(1):13–26. doi: 10.1016/s0376-8716(02)00011-x.S037687160200011X [DOI] [PubMed] [Google Scholar]
  • 11.McMain SF, Links PS, Gnam WH, Guimond T, Cardish RJ, Korman L, Streiner DL. A randomized trial of dialectical behavior therapy versus general psychiatric management for borderline personality disorder. Am J Psychiatry. 2009 Dec;166(12):1365–74. doi: 10.1176/appi.ajp.2009.09010039.appi.ajp.2009.09010039 [DOI] [PubMed] [Google Scholar]
  • 12.Verheul R, Van Den Bosch LM, Koeter MW, De Ridder MA, Stijnen T, Van Den Brink W. Dialectical behaviour therapy for women with borderline personality disorder: 12-month, randomised clinical trial in the Netherlands. Br J Psychiatry. 2003 Feb;182:135–40. doi: 10.1192/bjp.182.2.135.S0007125000269778 [DOI] [PubMed] [Google Scholar]
  • 13.Brettschneider C, Riedel-Heller S, König HH. A systematic review of economic evaluations of treatments for borderline personality disorder. PLoS One. 2014 Sep 29;9(9):e107748. doi: 10.1371/journal.pone.0107748. https://dx.plos.org/10.1371/journal.pone.0107748 .PONE-D-14-00795 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Murphy A, Bourke J, Flynn D, Kells M, Joyce M. A cost-effectiveness analysis of dialectical behaviour therapy for treating individuals with borderline personality disorder in the community. Ir J Med Sci. 2020 May;189(2):415–23. doi: 10.1007/s11845-019-02091-8.10.1007/s11845-019-02091-8 [DOI] [PubMed] [Google Scholar]
  • 15.Priebe S, Bhatti N, Barnicot K, Bremner S, Gaglia A, Katsakou C, Molosankwe I, McCrone P, Zinkler M. Effectiveness and cost-effectiveness of dialectical behaviour therapy for self-harming patients with personality disorder: a pragmatic randomised controlled trial. Psychother Psychosom. 2012;81(6):356–65. doi: 10.1159/000338897. https://www.karger.com?DOI=10.1159/000338897 .000338897 [DOI] [PubMed] [Google Scholar]
  • 16.Swales M, Hibbs RA, Bryning L, Hastings RP. Health related quality of life for young people receiving dialectical behaviour therapy (DBT): a routine outcome-monitoring pilot. Springerplus. 2016 Jul 20;5(1):1137. doi: 10.1186/s40064-016-2826-9. http://europepmc.org/abstract/MED/27504235 .2826 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Chapman AL. Dialectical behavior therapy: current indications and unique elements. Psychiatry (Edgmont) 2006 Sep;3(9):62–8. http://europepmc.org/abstract/MED/20975829 . [PMC free article] [PubMed] [Google Scholar]
  • 18.O'Connell B, Dowling M. Dialectical behaviour therapy (DBT) in the treatment of borderline personality disorder. J Psychiatr Ment Health Nurs. 2014 Aug;21(6):518–25. doi: 10.1111/jpm.12116. [DOI] [PubMed] [Google Scholar]
  • 19.Lindenboim N, Comtois KA, Linehan MM. Skills practice in dialectical behavior therapy for suicidal women meeting criteria for borderline personality disorder. Cogn Behav Pract. 2007 May;14(2):147–56. doi: 10.1016/j.cbpra.2006.10.004. [DOI] [Google Scholar]
  • 20.Linehan MM. DBT Skills Training Manual, Second Edition. New York: Guilford Publications; 2015. [Google Scholar]
  • 21.Helweg-Joergensen S, Schmidt T, Lichtenstein MB, Pedersen SS. Using a mobile diary app in the treatment of borderline personality disorder: mixed methods feasibility study. JMIR Form Res. 2019 Sep 30;3(3):e12852. doi: 10.2196/12852. https://formative.jmir.org/2019/3/e12852/ v3i3e12852 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Marceau L, Link C, Smith L, Carolan S, Jamison R. In-clinic use of electronic pain diaries: barriers of implementation among pain physicians. J Pain Symptom Manage. 2010 Sep;40(3):391–404. doi: 10.1016/j.jpainsymman.2009.12.021. https://linkinghub.elsevier.com/retrieve/pii/S0885-3924(10)00361-1 .S0885-3924(10)00361-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Rizvi SL, Hughes CD, Thomas MC. The DBT coach mobile application as an adjunct to treatment for suicidal and self-injuring individuals with borderline personality disorder: a preliminary evaluation and challenges to client utilization. Psychol Serv. 2016 Nov;13(4):380–8. doi: 10.1037/ser0000100.2016-39463-001 [DOI] [PubMed] [Google Scholar]
  • 24.Rodante DE, Kaplan MI, Olivera Fedi R, Gagliesi P, Pascali A, Quintero PS, Compte EJ, Perez AI, Weinstein M, Chiapella LC, Daray FM. CALMA, a mobile health application, as an accessory to therapy for reduction of suicidal and non-suicidal self-injured behaviors: a pilot cluster randomized controlled trial. Arch Suicide Res. 2020 Oct 18;:1–18. doi: 10.1080/13811118.2020.1834476. [DOI] [PubMed] [Google Scholar]
  • 25.Schroeder J, Suh J, Wilks C, Czerwinski M, Munson S, Fogarty J, Althoff T. Data-driven implications for translating evidence-based psychotherapies into technology-delivered interventions. Int Conf Pervasive Comput Technol Healthc. 2020 May;2020:274–87. doi: 10.1145/3421937.3421975. http://europepmc.org/abstract/MED/33912357 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Lederer N, Grechenig T, Baranyi R. unCUT: bridging the gap from paper diary cards towards mobile electronic monitoring solutions in borderline and self-injury. Proceedings of the IEEE 3nd International Conference on Serious Games and Applications for Health (SeGAH); IEEE 3nd International Conference on Serious Games and Applications for Health (SeGAH); May 14-16, 2014; Rio de Janeiro, Brazil. IEEE; 2014. [DOI] [Google Scholar]
  • 27.Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D, Augustovski F, Briggs AH, Mauskopf J, Loder E, ISPOR Health Economic Evaluation Publication Guidelines-CHEERS Good Reporting Practices Task Force Consolidated health economic evaluation reporting standards (CHEERS) --explanation and elaboration: a report of the ISPOR health economic evaluation publication guidelines good reporting practices task force. Value Health. 2013;16(2):231–50. doi: 10.1016/j.jval.2013.02.002. https://linkinghub.elsevier.com/retrieve/pii/S1098-3015(13)00022-3 .S1098-3015(13)00022-3 [DOI] [PubMed] [Google Scholar]
  • 28.The mDIARY Study. ENTER. [2020-06-25]. http://www.ementalhealthresearch.dk/27-mdiary.html .
  • 29.Helweg-Jørgensen S, Lichtenstein MB, Fruzzetti AE, Møller Dahl C, Pedersen SS. Daily self-monitoring of symptoms and skills learning in patients with borderline personality disorder through a mobile phone app: protocol for a pragmatic randomized controlled trial. JMIR Res Protoc. 2020 May 25;9(5):e17737. doi: 10.2196/17737. https://www.researchprotocols.org/2020/5/e17737/ v9i5e17737 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Hartfiel N, Edwards R. Applied Health Economics for Public Health Practice and Research. Oxford, United Kingdom: Oxford University Press; 2019. Cost–consequence analysis of public health interventions. [Google Scholar]
  • 31.International statistical classification of diseases and related health problems 10th revision (ICD-10) World Health Organization. 2019. [2021-01-21]. https://icd.who.int/browse10/2019/en#/F60.3 .
  • 32.Monsenso. mDiary App. 2021. [2021-07-16]. https://www.monsenso.com/mhealth-mental-health/individuals/
  • 33.Janssen MF, Pickard AS, Golicki D, Gudex C, Niewada M, Scalone L, Swinburn P, Busschbach J. Measurement properties of the EQ-5D-5L compared to the EQ-5D-3L across eight patient groups: a multi-country study. Qual Life Res. 2013 Sep;22(7):1717–27. doi: 10.1007/s11136-012-0322-4. http://europepmc.org/abstract/MED/23184421 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001 Sep;16(9):606–13. doi: 10.1046/j.1525-1497.2001.016009606.x. https://onlinelibrary.wiley.com/resolve/openurl?genre=article&sid=nlm:pubmed&issn=0884-8734&date=2001&volume=16&issue=9&spage=606 .jgi01114 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Zanarini MC, Vujanovic AA, Parachini EA, Boulanger JL, Frankenburg FR, Hennen John. Zanarini rating scale for borderline personality disorder (ZAN-BPD): a continuous measure of DSM-IV borderline psychopathology. J Pers Disord. 2003 Jun;17(3):233–42. doi: 10.1521/pedi.17.3.233.22147. [DOI] [PubMed] [Google Scholar]
  • 36.Osman A, Bagge CL, Gutierrez PM, Konick LC, Kopper BA, Barrios FX. The suicidal behaviors questionnaire-revised (SBQ-R): validation with clinical and nonclinical samples. Assessment. 2001 Dec;8(4):443–54. doi: 10.1177/107319110100800409. [DOI] [PubMed] [Google Scholar]
  • 37.Jensen C, Sørensen SS, Gudex C, Jensen M, Pedersen K, Ehlers L. The Danish EQ-5D-5L value set: a hybrid model using cTTO and DCE data. Appl Health Econ Health Policy. 2021 Jul;19(4):579–91. doi: 10.1007/s40258-021-00639-3. http://europepmc.org/abstract/MED/33527304 .10.1007/s40258-021-00639-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Manca A, Hawkins N, Sculpher MJ. Estimating mean QALYs in trial-based cost-effectiveness analysis: the importance of controlling for baseline utility. Health Econ. 2005 May;14(5):487–96. doi: 10.1002/hec.944. [DOI] [PubMed] [Google Scholar]
  • 39.Schmidt M, Pedersen L, Sørensen HT. The Danish Civil Registration System as a tool in epidemiology. Eur J Epidemiol. 2014 Aug;29(8):541–9. doi: 10.1007/s10654-014-9930-3. [DOI] [PubMed] [Google Scholar]
  • 40.Andersen JS, Olivarius ND, Krasnik A. The Danish National Health Service Register. Scand J Public Health. 2011 Jul;39(7 Suppl):34–7. doi: 10.1177/1403494810394718.39/7_suppl/34 [DOI] [PubMed] [Google Scholar]
  • 41.Schmidt M, Schmidt SA, Sandegaard JL, Ehrenstein V, Pedersen L, Sørensen HT. The Danish National Patient Registry: a review of content, data quality, and research potential. Clin Epidemiol. 2015 Nov 17;7:449–90. doi: 10.2147/CLEP.S91125. doi: 10.2147/CLEP.S91125.clep-7-449 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Lewis JR. The System Usability Scale: past, present, and future. Int J Hum Comput Interact. 2018 Mar 30;34(7):577–90. doi: 10.1080/10447318.2018.1455307. [DOI] [Google Scholar]
  • 43.Net price index. Statistics Denmark. 2020. [2020-11-04]. https://www.dst.dk/en/Statistik/dokumentation/documentationofstatistics/net-price-index .
  • 44.Valutakurser [Exchange rates] EUROinvestor. 2019. [2020-06-25]. https://www.valutakurser.dk/
  • 45.Drummond M, Sculpher M, Claxton K, Stoddart G, Torrance G. Methods for the Economic Evaluation of Health Care Programmes. 4th Edition. Oxford: Oxford University Press; 2015. Sep, [Google Scholar]
  • 46.Kildemoes HW, Sørensen HT, Hallas J. The Danish National Prescription Registry. Scand J Public Health. 2011 Jul;39(7 Suppl):38–41. doi: 10.1177/1403494810394717.39/7_suppl/38 [DOI] [PubMed] [Google Scholar]
  • 47.Bouwmans C, De Jong K, Timman R, Zijlstra-Vlasveld M, Van der Feltz-Cornelis F, Tan SS, Hakkaart-van Roijen L. Feasibility, reliability and validity of a questionnaire on healthcare consumption and productivity loss in patients with a psychiatric disorder (TiC-P) BMC Health Serv Res. 2013 Jun 15;13:217. doi: 10.1186/1472-6963-13-217. http://bmchealthservres.biomedcentral.com/articles/10.1186/1472-6963-13-217 .1472-6963-13-217 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Kirisits A, Redekop WK. The economic evaluation of medical devices: challenges ahead. Appl Health Econ Health Policy. 2013 Feb;11(1):15–26. doi: 10.1007/s40258-012-0006-9. [DOI] [PubMed] [Google Scholar]
  • 49.Drummond M, Tarricone R, Torbica A. Economic evaluation of medical devices. Oxford Res Encycl Econ Financ. 2018 Mar 28;:A. doi: 10.1093/acrefore/9780190625979.013.105. [DOI] [Google Scholar]
  • 50.Statistikbanken - LONS20: Løn efter arbejdsfunktion, sektor, aflønningsform, lønmodtagergruppe, lønkomponenter og køn [Statistikbanken - LONS20: Wages estimated based on job function, sector, payment form, employment group, wage component and gender] Statistics Denmark. 2018. [2020-06-10]. https://www.statistikbanken.dk/statbank5a/default.asp?w=1536 .
  • 51.Levetider [Service lives] Agency for Modernisation - Ministry of Finance. [2020-06-18]. https://oes.dk/oekonomi/oeav/regnskabsregler/generelle-bogfoeringsbestemmelser/levetider/
  • 52.Den samfundsøkonomiske diskonteringsrente [The economic discount rate] Ministry of Finance. [2021-09-17]. https://fm.dk/media/10174/dokumentationsnotat-_-den-samfundsoekonomiske-diskonteringsrente.pdf .
  • 53.Ehlers LH, Vestergaard AS. Costing in Health Economic Evaluation. Aalborg: Aalborg University Press; 2019. pp. 1–88. [Google Scholar]
  • 54.Faria R, Gomes M, Epstein D, White IR. A guide to handling missing data in cost-effectiveness analysis conducted within randomised controlled trials. Pharmacoeconomics. 2014 Dec;32(12):1157–70. doi: 10.1007/s40273-014-0193-3. http://europepmc.org/abstract/MED/25069632 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55.Sterne JA, White IR, Carlin JB, Spratt M, Royston P, Kenward MG, Wood AM, Carpenter JR. Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls. Br Med J. 2009 Jun 29;338:b2393. doi: 10.1136/bmj.b2393. http://europepmc.org/abstract/MED/19564179 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56.Kirkwood B, Sterne J. Essential Medical Statistics. Malden, Mass: Wiley-Blackwell; 2003. [Google Scholar]
  • 57.Franklin M, Enrique A, Palacios J, Richards D. Psychometric assessment of EQ-5D-5L and ReQoL measures in patients with anxiety and depression: construct validity and responsiveness. Qual Life Res. 2021 Sep;30(9):2633–47. doi: 10.1007/s11136-021-02833-1. http://europepmc.org/abstract/MED/33835414 .10.1007/s11136-021-02833-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 58.van Asselt AD, Dirksen C, Arntz A, Giesen-Bloo J, Severens J. The EQ-5D: a useful quality of life measure in borderline personality disorder? Eur Psychiatry. 2009 Mar;24(2):79–85. doi: 10.1016/j.eurpsy.2008.11.001.S0924-9338(08)01654-4 [DOI] [PubMed] [Google Scholar]
  • 59.Bergmo TS. Using QALYs in telehealth evaluations: a systematic review of methodology and transparency. BMC Health Serv Res. 2014 Aug 03;14:332. doi: 10.1186/1472-6963-14-332. https://bmchealthservres.biomedcentral.com/articles/10.1186/1472-6963-14-332 .1472-6963-14-332 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 60.Lauritsen K, Innocenti AD, Hendel L, Praest J, Lytje MF, Clemmensen-Rotne K, Wiklund I. Symptom recording in a randomised clinical trial: paper diaries vs. electronic or telephone data capture. Control Clin Trials. 2004 Dec;25(6):585–97. doi: 10.1016/j.cct.2004.09.001.S0197-2456(04)00086-8 [DOI] [PubMed] [Google Scholar]
  • 61.Derks YP, Klaassen R, Westerhof GJ, Bohlmeijer ET, Noordzij ML. Development of an ambulatory biofeedback app to enhance emotional awareness in patients with borderline personality disorder: multicycle usability testing study. JMIR Mhealth Uhealth. 2019 Oct 15;7(10):e13479. doi: 10.2196/13479. https://mhealth.jmir.org/2019/10/e13479/ v7i10e13479 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 62.Carter MC, Burley VJ, Nykjaer C, Cade JE. Adherence to a smartphone application for weight loss compared to website and paper diary: pilot randomized controlled trial. J Med Internet Res. 2013 Apr 15;15(4):e32. doi: 10.2196/jmir.2283. http://www.jmir.org/2013/4/e32/ v15i4e32 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 63.Abu-El-Noor N, Aljeesh Y, Bottcher B, Abu-El-Noor M. Impact of a mobile phone app on adherence to treatment regimens among hypertensive patients: a randomised clinical trial study. Eur J Cardiovasc Nurs. 2021 Jun 29;20(5):428–35. doi: 10.1177/1474515120938235.6145550 [DOI] [PubMed] [Google Scholar]
  • 64.Austin SF, Jansen JE, Petersen CJ, Jensen R, Simonsen E. Mobile app integration into dialectical behavior therapy for persons with borderline personality disorder: qualitative and quantitative study. JMIR Ment Health. 2020 Jun 11;7(6):e14913. doi: 10.2196/14913. https://mental.jmir.org/2020/6/e14913/ v7i6e14913 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 65.Innovation Fund Denmark. [2021-10-19]. https://innovationsfonden.dk/en .

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Multimedia Appendix 1

Detailed information on cost estimates in the economic analysis.

Multimedia Appendix 2

A detailed description of the used approach for multiple imputation.


Articles from Journal of Medical Internet Research are provided here courtesy of JMIR Publications Inc.

RESOURCES