Skip to main content
JAMA Network logoLink to JAMA Network
. 2021 Oct 1;4(10):e2127622. doi: 10.1001/jamanetworkopen.2021.27622

Efficacy of Motivational Interviewing to Improve Utilization of Mental Health Services Among Youths With Chronic Medical Conditions

A Cluster Randomized Clinical Trial

Christina Reinauer 1,, Anna Lena Platzbecker 1, Rabea Viermann 1, Matthias Domhardt 2, Harald Baumeister 2, Katharina Foertsch 1, Hannah Linderskamp 1, Lisa Krassuski 1, Doris Staab 3, Kirsten Minden 4, Reinhold Kilian 5, Reinhard W Holl 6, Petra Warschburger 7, Thomas Meißner 1
PMCID: PMC8486984  PMID: 34596672

Key Points

Question

Does pediatrician training in motivational interviewing (MI) facilitate the use of mental health care services for youths with chronic medical conditions?

Findings

In this cluster randomized trial with 164 youths with chronic medical conditions and comorbid anxiety or depression, training physicians in MI did not significantly improve uptake rates of psychological counseling among youths compared with usual care. MI training was associated with longer patient-physician conversations and lower anxiety scores at 1-year rescreening.

Meaning

More research is needed to determine whether training pediatricians in MI and allotting more consultation time to discuss mental health could facilitate use of counseling services by youths with chronic medical conditions.


This cluster randomized clinical trial examined the efficacy of pediatricians using motivational interviewing (MI) vs treatment as usual in increasing youths’ utilization of mental health care.

Abstract

Importance

Despite the high prevalence of anxiety and depression in youths with chronic medical conditions (CMCs), physicians encounter substantial barriers in motivating these patients to access mental health care services.

Objective

To determine the efficacy of motivational interviewing (MI) training for pediatricians in increasing youths’ use of mental health care.

Design, Setting, and Participants

The COACH-MI (Chronic Conditions in Adolescents: Implementation and Evaluation of Patient-Centered Collaborative Healthcare—Motivational Interviewing) study was a single-center cluster randomized clinical trial at the University Children’s Hospital specialized outpatient clinics in Düsseldorf, Germany. Treating pediatricians were cluster randomized to a 2-day MI workshop or treatment as usual (TAU). Patient recruitment and MI conversations occurred between April 2018 and May 2020 with 6-month follow-up and 1-year rescreening. Participants were youths aged 12 to 20 years with CMCs and comorbid symptoms of anxiety and depression; they were advised by their MI-trained or untrained physicians to access psychological counseling services. Statistical analysis was performed from October 2020 to April 2021.

Interventions

MI physicians were trained through a 2-day, certified MI training course; they recommended use of mental health care services during routine clinical appointments.

Main Outcomes and Measures

The primary outcome of uptake of mental health care services within the 6-month follow-up was analyzed using a logistic mixed model, adjusted for the data’s cluster structure. Uptake of mental health services was defined as making at least 1 appointment by the 6-month follow-up.

Results

Among 164 youths with CMCs and conspicuous anxiety or depression screening, 97 (59%) were female, 94 (57%) had MI, and 70 (43%) had TAU; the mean (SD) age was 15.2 (1.9) years. Compared with patients receiving TAU, the difference in mental health care use at 6 months among patients whose physicians had undergone MI training was not statistically significant (odds ratio [OR], 1.96; 95% CI, 0.98-3.92; P = .06). The effect was moderated by the subjective burden of disease (F2,158 = 3.42; P = .04). Counseling with an MI-trained physician also led to lower anxiety symptom scores at 1-year rescreening (F1,130 = 4.11; P = .045). MI training was associated with longer conversations between patients and physicians (30.3 [16.7] minutes vs 16.8 [12.5] minutes; P < .001), and conversation length significantly influenced uptake rates across conditions (OR, 1.03; 95% CI, 1.01-1.06; P = .005).

Conclusions and Relevance

In this study, use of MI in specialized pediatric consultations did not increase the use of mental health care services among youths with CMCs but did lead to longer patient-physician conversations and lower anxiety scores at 1 year. Additional research is required to determine whether varying scope and duration of MI training for physicians could encourage youths with CMCs to seek counseling and thus improve integrated care models.

Trial Registration

German Trials Registry: DRKS00014043

Introduction

Youths with chronic medical conditions (CMCs) represent a high-risk group for comorbid anxiety, depression, or behavioral problems, with a reported prevalence of 10% to 42%.1,2,3,4,5,6 Concomitant psychological concerns compromise therapy for somatic disorders, health-related behavior, medication adherence, and increase long-term health risks into adulthood.7,8,9,10,11 Early detection of mental health problems and timely initiation of evidence-based treatment is important.12,13,14,15,16,17 However, physicians can encounter substantial individual barriers, and referring youths who lack intrinsic motivation to psychological counseling typically fails. Youths tend to underutilize services, fearing stigmatization and preferring autonomy,18,19,20,21 and only a fraction receives adequate treatment.22,23,24,25 Growing evidence exists for the effectiveness of integrated mental health care for pediatric in- and outpatient care. Identification of psychological disorders and referral to adequate treatment options are important factors for therapy uptake.26 Validated tools to detect comorbid mental health problems and brief psychological interventions, such as motivational interviewing (MI) are applicable.27 MI is an evidence-based, collaborative counseling technique, exploring intrinsic motivation and ambivalence.28,29 It has been shown to facilitate uptake of therapy in suicidal patients30 and the uptake of cognitive behavioral therapy or Internet-based interventions for depression in adolescents.31,32,33,34

Published data suggest that MI can be implemented into clinical routines to improve young patients’ mental health care services utilization.35,36 Our study aimed to determine the efficacy of MI training for pediatricians in increasing adolescents’ use of mental health care services.

Methods

The COACH-MI study was conducted within the research framework of the COACH consortium (Chronic Conditions in Adolescents: Implementation and Evaluation of Patient-Centered Collaborative Healthcare), which aimed to increase awareness of and access to mental health care for youths aged 12 to 20 years with CMCs. The study protocol was published previously.37 The study received ethical approval from the University of Düsseldorf institutional review board.

Design, Setting, and Participants

This single-center cluster randomized clinical trial was conducted at the University Children’s Hospital in Düsseldorf, Germany. Patient recruitment occurred between April 2018 and May 2020 with a 6-month follow-up and 1-year rescreening.

Clusters were constituted by physicians in specialized outpatient clinics who were randomized into either MI or treatment as usual (TAU) groups. Participating physicians (n = 51 randomized; n = 37 saw patients in the study setting; n = 18 [49%] MI; n = 19 [51%] TAU; n = 22 [60%] female, n = 19 [51%] senior physicians or fellows) had no previous training in psychiatry, psychotherapy, or MI, and they provided written informed consent. There were no exclusion criteria for physicians.

Youths aged 12 to 20 years with CMCs (condition duration more than 1 year and requiring continuous medical treatment) were invited to complete questionnaires on depression, anxiety, and medication adherence before their routine appointment in the specialized pediatric outpatient clinics (pulmonology, diabetes and endocrinology, metabolic diseases, cardiology, gastroenterology, rheumatology and immunology, and neurology). The study included 164 patients with conspicuous screening results. Written informed consent was obtained from patients and legal guardians (for patients aged younger than 18 years). Patient exclusion criteria included regular psychotherapy, psychosis, acute suicidal ideations, diagnosis of substance abuse, severe intellectual disability (IQ < 70), and language limitations. No patient was on psychotropic medication.

Assessments

Anxiety and depression symptoms were assessed using the Generalized Anxiety Disorder 7-item scale (GAD-7)38 and the Patient Health Questionnaire 9 (PHQ-9).39 The GAD-7 includes 7 prevalent symptoms of generalized anxiety disorder, while the PHQ-9 includes 9 criteria for depression, including 1 item about ideation of suicidal behavior or self-harm (item 9). Both questionnaires refer to the 2 weeks prior to assessment, and items score from 0 (not at all) to 3 (nearly every day). A positive screening result was defined as a score greater than or equal to 7 in either GAD-7 or PHQ-9 or a positive answer to item 9 of the PHQ-9 (ie, thoughts that you would be better off dead, or thoughts of hurting yourself in some way at least on several days [score ≥1]). A red flag, indicating severe symptoms, was displayed to physicians for patients who scored at least 15 of 21 points on the GAD-7, at least 20 of 27 points on the PHQ-9, or who gave a positive response to item 9. Consultations evaluated suicidality using the Columbia-Suicide Severity Rating Scale.40 Medication adherence was evaluated using the German version of the Medication Adherence Report Scale (MARS-D) questionnaire (maximum score 25).41 Medical records provided anthropometric data included age, sex, height, weight, body mass index (BMI; calculated by weight in kilograms divided by height in meters squared), and disease-related parameters. Patients reported subjective symptom burden, pain occurrence, and everyday-life restrictions (all Likert scale, 1 = not at all to 4 = nearly every day or extremely stressful), missed school or workdays within the last 6 months, and daily treatment duration. Data on race and ethnicity were not collected. The primary outcome was evaluated 6 months postintervention during a standardized follow-up phone call. Participants and/or their legal guardians were interviewed about mental health care service use. Rescreening with the same questionnaires was scheduled approximately 1 year postintervention.

Randomization

Pediatricians (n = 51) were cluster-randomized into the MI intervention (n = 25) or TAU (n = 26) groups by an independent biostatistician. Outpatient clinic specialization and expected cluster size (small or big) were considered using the minimization (dynamic allocation) method42 to neutralize imbalances between groups (Supplement 1). Patient allocation depended on the treating physician’s randomization into MI or TAU group; 94 patients saw an MI-trained pediatrician, and 70 received TAU consultation.

Intervention

We established MI training to enable physicians to deliver an MI-based ultrashort intervention aiming at patients’ motivation toward claiming mental health care. Pediatricians randomized into the MI training group received a 2-day training course conducted by a Motivational Interviewing Network of Trainers (MINT) certified trainer (Supplement 1). They signed a confidentiality agreement to prevent potential trial group contamination. One-day MI booster sessions were conducted 1 year after study initiation. All pediatricians engaged in counseling conversations with their patients on conspicuous screening results and recommended the use of mental health services. MI physicians offered a voluntary second visit to further discuss the topic with their patients, whereas TAU physicians were not advised about second appointments. Patients were blinded regarding a physician’s training, and all received the same standardized written feedback on assessments and contact information for mental health care appointments. If mutual consent was given, conversations were audio recorded to analyze treatment fidelity. Physicians self-reported conversation length and semiquantitative use of MI-consistent techniques after each consultation (6 basic MI techniques: open-ended questions, reflective listening, affirmations, advice with permission, creating collaboration, and emphasizing autonomy and control were graded from 0 = not used to 2 = often used, for a maximum score of 12 points).

Outcomes

The primary outcome measured was the use of mental health care services, defined as making at least 1 appointment (vs no appointment) by the 6-month follow-up. Patients on waiting lists were counted as positive outcomes. Secondary outcomes included baseline changes in GAD-7, PHQ-9, and MARS-D scores; the number of psychological sessions attended within the follow-up interval; sex-specific analyses; and severe adverse events (SAEs) as well as missed clinical visits evaluation.

Exploratory analysis included the influence of clinical parameters and conversation length on the primary outcome; reasons for not seeking psychological counseling; the physicians’ MI-consistency; satisfaction with the intervention and use of second MI/TAU appointments; and the type of mental health service utilized.

Statistical Analysis

The rate of mental health care uptake for the TAU group was estimated at 10%,43 and an increase to 30% for the MI group was expected. A sample size of n = 62 per trial group was calculated with NQuery Advanced version 8.1 (Statistical Solutions) using a 2-sided χ2 test with power at 80% and significance level at 5%. A cluster size of 5 patients per physician, estimated intracluster correlation coefficient of 2.5%, corrected by 10% for cluster effects, resulted in a sample size of n = 69 per group. Adjusting for 15% dropout, we aimed for n = 162 (n = 81 per group) patients to participate in the study (Supplement 1).37 Statistical analyses were performed with SPSS Statistics version 26 (IBM) from October 2020 to April 2021. MI efficacy was examined in an intention-to-treat approach. For imputation, a logistic regression model was used for the primary outcome models (including the variables sex, age, intervention, PHQ-9 and GAD-7 score, pain, burden of disease, and conversation length). The primary outcome was analyzed using a logistic mixed model, adjusting for the data’s cluster structure,44 age, and sex. Secondary and exploratory analyses used logistic mixed models, nonparametric tests, and mixed analysis of variance. For comparison of SAE rates, Fisher exact test was used. Means and SDs were reported for symmetrical data, whereas medians and IQRs were used for nonsymmetrical data. Two-sided P < .05 indicated statistical significance. We obtained only a few audio recordings, insufficient for a structured evaluation.

Results

Of 706 youths with CMCs, the study included 164 patients (97 [59%] were female youths, 94 [57%] had MI, and 70 [43%] had TAU); the mean (SD) age was 15.2 [1.9] years. Among these 164 patients, 62 had positive screening results for depression, 25 screened positive for anxiety symptoms, and 68 screened positive for both depression and anxiety, and 9 had a positive item 9 on the PHQ-9 (Figure 1). Patient variables for the MI and TAU pediatrician groups were equivalent at baseline (Table 1). Most patients had mild to moderately conspicuous mean (SD) screening results (GAD-7, 7.4 [0.3]; PHQ-9, 8.8 [0.3]). A total of 71 patients (MI: n = 43; TAU: n = 28) scored at least 10 points on either of the screening tests, indicating moderately severe depression or anxiety. Seven patients had red flags from screening scores. Altogether, 55 patients had a positive item 9.

Figure 1. CONSORT Flowchart of the Study.

Figure 1.

The patients’ allocations were based on their treating pediatricians’ randomization to MI or TAU groups. At follow-up 7 patients (4%) and at rescreening 32 patients (19.5%) were lost to follow-up. MI indicates motivational interviewing; PHQ-9, Patient Health Questionnaire 9; TAU, treatment as usual.

Table 1. Patient Characteristics of MI and TAU Counseling Groups.

Variable Mean (SD)
All (n = 164) MI (n = 94) TAU (n = 70)
Age 15.2 (1.9) 15.2 (2.0) 15.2 (1.9)
Sex, female, No. (%) 97 (59) 51 (54) 46 (65)
BMIa 0.3 (1.1) 0.2 (1.1) 0.3 (1.1)
Hemoglobin A1c, % (n = 42) 8.0 (1.5) 8.2 (1.7) 7.7 (1.3)
GAD-7 score 7.4 (3.6) 7.7 (3.6) 6.9 (3.5)
PHQ-9 score 8.8 (3.6) 8.7 (3.7) 8.9 (3.5)
MARS-D score 22.0 (3.1) 21.6 (3.4) 22.5 (2.6)
Subjective burden of disease 2.2 (0.8) 2.2 (0.9) 2.1 (0.7)
Painb 2.1 (0.9) 2.2 (0.8) 2.0 (0.9)
Everyday life restrictionsb 1.8 (0.9) 1.8 (0.9) 1.6 (0.9)
Absent school or workdays, median (IQR)c 10.0 (3.3-24.3) 12.5 (4-25.5) 7.0 (3-17)
Time requirement for therapy, median (IQR)d 5.0 (1-15) 5.0 (2-20) 5.0 (5-15)

Abbreviations: BMI, body mass index; GAD-7, Generalized Anxiety Disorder 7-item scale; MI, motivational interviewing; PHQ-9, Patient Health Questionnaire 9; TAU, treatment as usual.

a

BMI is calculated at weight in kilograms divided by height in meters squared.

b

Pain occurrence and everyday life restrictions (Likert scale, 1 = not at all to 4 = nearly every day or extremely stressful).

c

Absent school or workdays during last 6 months.

d

Time requirement for therapy related to chronic medical condition (minutes per day).

Primary Outcome

We found a difference between the 2 groups concerning mental health care access at follow-up but it did not meet the threshold for statistical significance (MI: 36 patients [40.0%] vs TAU: 18 patients [26.8%]; adjusted odds ratio [OR], 1.96 [95% CI, 0.98-3.92]; P = .06) (Table 2). Among positive outcomes, 6 patients were on waiting lists, and 4 had postponed appointments. For patients who scored at least 10 on either test, the adjusted OR for treatment initiation in the MI group was 2.08 (95% CI, 0.74-5.83; P = .16).

Table 2. Uptake of Mental Health Services After Conversations With MI or TAU Physicians at 6-Month Follow-up.

Characteristic No. (%)
MI (n = 94) TAU (n = 70) Total (n = 164)
Uptake of mental health services 36 (40.0) 18 (26.8) 54 (34.4)
No uptake 54 (60.0) 49 (73.2) 103 (65.6)
Lost to follow-up, No. 4 3 7

Abbreviations: MI, motivational interviewing; TAU, treatment as usual.

Secondary Outcomes

Anxiety and depression scores significantly improved upon rescreening in both groups independent of the intervention, leading to a high remission rate of 53.6% (70 of 132) (MI: 50% [38 of 76]; TAU: 57.1% [32 of 56]). Patients’ anxiety symptom scores further improved after physicians’ MI training (time point [baseline vs rescreening] × intervention effect; F1,130 = 4.11; P = .045), whereas depression symptoms did not (F1,130 = 0.54; P = .46; Figure 2). Mean (SD) adherence scores (MARS-D) were stable before and after intervention in both conditions (22.2 [2.7] vs 22.5 [2.7]; F1,130 = 1.75; P = .19; time point × intervention effect: F1,129 = 0.25; P = .62).

Figure 2. GAD-7 and PHQ-9 Scores at Baseline and Rescreening for MI and TAU Patients.

Figure 2.

Anxiety and depression significantly improved upon rescreening in both conditions for MI (GAD-7, 7.9 [3.6] vs 5.1 [3.7]; P < .001; PHQ-9, 8.3 [3.7] vs 5.4 [3.4]; P < .001) and TAU (GAD-7, 7.5 [3.3] vs 6.2 [4.6]; P = .036; PHQ-9, 8.8 [3.5] vs 6.5 [4.8]; P = .002). GAD-7 improved better in the MI group (time point [baseline vs rescreening] × intervention effect), compared with TAU (P = .045). GAD-7 indicates Generalized Anxiety Disorder 7-item scale; MI, motivational interviewing; PHQ-9, Patient Health Questionnaire 9; TAU, treatment as usual.

The total number of psychological counseling appointments completed at 6 months did not significantly differ between the MI and TAU groups; 6 MI patients (18%) and 5 TAU patients (28%) had at least one appointment; 12 MI patients (33%) and 5 TAU patients (28%) had 2 to 9 appointments; and 9 MI patients (25%) and 6 TAU patients (33%) had more than 10 appointments (U = 210; z = −0.499; P = .62). Reasons for not seeking psychological support are listed in Table 3. Missed clinical visits during follow-up were similar in both groups.

Table 3. Reasons for Not Seeking Psychological Support at Follow-up.

Reasons for not making an appointment (multiple options possible) No.
Total No. 103
No perceived need for further psychological support 77
A critical attitude toward psychologists (only for the mentally ill) 3
Did not have time to make an appointment 12
Forgot to make an appointment 5
Insecurity about mental health care (we did not know what to expect) 3
Psychological support will have no benefit 4
My child didn't want me to make an appointment (parents' reports, n = 58) 11
Other 5

Female patients did not have significantly higher mean (SD) screening scores (PHQ-9: 9.1 [3.8] vs 8.4 [3.3]; P = .21; GAD-7: 7.6 [3.5] vs 7.0 [3.7]; P = .25), reported symptom burden (2.3 [0.8] vs 2.0 [0.7]; P = .07), or rates of suicidal ideation (item 9: 35 female patients [36.1%] vs 20 male patients [29.9%]; P = .41), nor were female patients in general significantly more likely to make an appointment (OR, 1.19 [95% CI, 0.61-2.35]; P = .61).

SAEs were monitored throughout the study. Patients had 35 hospital admissions, primarily related to CMCs, and unrelated to the intervention. Four related SAEs were reported, regarding an inpatient crisis intervention within a pediatric or psychiatric department, with no significant differences between the treatment groups (MI, 3 patients; TAU, 1 patient; P = .64, Fisher exact test).

Exploratory Analysis

Individual effects on the outcome, uptake of mental health treatment, were observed for PHQ-9 scores (OR, 1.11; 95% CI, 1.01-1.21; P = .04) or GAD-7 scores (OR, 1.13; 95% CI, 1.02-1.24; P = .02), patient-reported burden of disease (OR, 1.64; 95% CI, 1.06-2.51; P = .03) and pain occurrence (OR, 1.59; 95% CI, 1.07-2.36; P = .02). Adding these covariates to the primary outcome analysis, the MI effect was not significant (OR, 1.82; 95% CI, 0.88-3.76; P = .10). In multivariate analysis the effect of MI was not moderated by most clinical characteristics (anxiety and depression symptoms, sex, age, standardized BMI, disease category, MARS-D, pain occurrence, daily treatment duration and everyday life restrictions), but only by the subjective burden of disease (intervention × burden of disease; F2,158 = 3.42; P = .04).

MI conversations had significantly longer mean (SD) times than TAU conversations (30.3 [16.7] minutes vs 16.8 [12.5] minutes; P < .001). As an individual parameter, conversation length was significantly associated with uptake rates (OR, 1.03; 95% CI, 1.01-1.06; P = .005). The influence of conversation length on the primary outcome model was more pronounced in the MI group (OR, 1.03; 95% CI, 1.00-1.06; P = .052) compared with TAU (OR, 1.03; 95% CI, 0.98-1.07; P = .25). Including an interaction term between conversation length and intervention (MI vs TAU) into our model showed no significant modifying effect of conversation length on the effect of MI on uptake.

When rating the statement “Conversation was helpful” on a Likert scale (0 = not at all to 4 = very helpful), satisfactory mean (SD) scores for both MI and TAU sessions were reported by patients (MI: 3.25 [0.99] vs TAU: 2.98 [1.19]; P = .17), parents (MI: 3.32 [0.96] vs TAU: 3.03 [1.05]; P = .21), and physicians (MI: 2.82 [0.91] vs TAU: 2.72 [0.82]; P = .55). MI physicians’ mean (SD) self-assessment scores were 9.8 (1.3) of 12 points for the use of MI-consistent conversation strategies. The optional second appointment with MI pediatricians was attended by 24 (25.5%) patients; no TAU patients attended an extra appointment. Most patients who attempted to obtain mental health care access were successful (81.5% [44 of 54]). First consultations were with a child and adolescent psychotherapist (n = 26), a clinical psychologist (n = 8), a child and adolescent psychiatrist (n = 8), or other psychosocial services (n = 2). Patients did not utilize online counseling, such as Internet-based cognitive behavioral therapy.

Discussion

This study incorporated MI training and annual anxiety and depression screening into specialized pediatric care for adolescents with CMCs in a collaborative care setting. The focus on mental health care led to higher-than-expected uptake rates, although the results were not statistically significant. Following an MI or TAU counseling session, 40.0% and 26.8% of adolescent patients with CMCs and anxiety or depression symptoms made appointments for psychological counseling, respectively. Discussions of anxiety and depression screening results between pediatricians and patients in outpatient care is novel. Conversation length (including TAU) was longer than expected, suggesting a relevant need to address and discuss mental health topics during routine care. Conversation length and the level of burden of disease had a relevant influence on mental health service-seeking behavior.

We deliberately selected a low cutoff of at least 7 points on the anxiety and depression screening questionnaires for high sensitivity and low-threshold referral to avoid chronic mental health impairments. This approach allowed us to avoid selecting a cohort with exclusively severe symptoms and to also include patients with fewer symptoms. Including mild anxiety or depression might lead to an underestimation of the benefits of MI training. Many patients reported no subjective need for psychological support, reflecting the issues of internal barriers, shame, and desired autonomy. Concurrently, some patients with mild symptoms might not yet require professional support. In the integrated care approach, the treating physicians can already provide valuable advice with appropriate training (eg, MI) for mild psychosocial, emotional, and behavioral disorders. To improve access rates, we recommend maintaining a low cutoff for offering mental health support for at-risk adolescents experiencing mild or subclinical depression and anxiety.

Our cohort showed high remission rates. This result is potentially due to mild cases, a simple intervention effect focusing on mental health, or spontaneous fluctuations over time. Depressive episodes in adolescents often remit spontaneously within a year.45,46 Whereas we observed significantly better improvements only in anxiety scores with the MI approach, other research found that physician MI interventions also alleviated depression symptoms.32,34 Short MI seminars have been successful in teaching physicians basic MI skills,36,47,48,49 but proficiency and long-term experience are associated with its success. MI delivered by a clinical psychologist with postgraduate MI-training improved treatment initiation and participation,31 but MI delivered by social workers was only partly effective in the recent STAT-ED intervention for adolescents with suicidal ideations.50 Further research is necessary to evaluate the optimal method and duration of successful physician MI training,51 such as implementation early in medical school, as well as the effectiveness of different dosages of MI for adolescents in the framework of clinical visits. Research has shown that single sessions or two 15-minute MI sessions enhance motivation for mental health treatment.52,53 MI can be implemented in a busy schedule as an add-on conversation technique (additional 15–20 minutes) during regular appointments, and can potentially improve treatment attendance31,54,55 and outcomes,56 encourage medication adherence,57,58 facilitate the transition to adult care,59 and reduce risk-taking behaviors.60 Researchers have combined MI with limited case management in a collaborative care setting with varying success (eg, incorporating follow-up phone calls by social workers).36,50 However, this approach requires additional resources and time from both caretakers and adolescents, a challenge reflected in the low rate of opting for a second appointment (25.5%).

Future Directions

Although routine screening has been established for certain pediatric diseases,4,61 we implemented anxiety and depression screening for all youths with CMCs and offered counseling and advice. Routinely discussing mental health in pediatric care targets prevention instead of intervention and is a cornerstone in enhancing risk perception and reducing prejudice. A resource-strengthening approach without a focus on psychopathology might be beneficial for patients with borderline screening results. No patient participated in e-Health services because they are not yet available in standard care. The rapidly growing availability of evidence-based and Internet-based interventions, as well as health apps, may increase mental health care accessibility for the young digital generation33,62,63,64 and reduce the risk of chronification due to prolonged waiting periods, but motivating adolescents to participate remains a challenge.65 This topic will be analyzed in the future research of the COACH consortium.66,67

Limitations

Although this study fulfilled high methodological standards, it has some limitations. For ethical reasons, a nonactive control group was not included. Physicians could not be blinded to the intervention condition, because blinding is impossible in trials on psychological interventions.68 A major limitation of our study is that comparing MI with TAU is only possible by providing additional time for MI conversations. Thus, the effects of MI and conversation length cannot be clearly separated in the evaluation. Although statistical significance was not obtained, with larger samples the effect of MI may become significant and may be of clinical relevance.

The power analysis included the estimated patient numbers per pediatrician (high or low). We anticipated equal sizes for MI and TAU groups, but differences occurred randomly because patient allocation was based on their pediatricians’ randomization. To avoid this issue, patients would have needed to change physicians after screening and would no longer have seen their regular pediatricians for counseling. Thus, group size difference was not due to systematic error, and we considered this factor unlikely to have distorted the outcomes. Additionally, we could not perform a structured fidelity assessment of audio recordings69 because of limited consent from patients and legal guardians for both MI and TAU consultations. We cannot rule out that TAU pediatricians intuitively used some patient-centered conversation techniques. As a proxy to coding for MI fidelity, we used physicians’ self-reports on the use of 6 core MI techniques after each MI conversation. Finally, previous studies have shown that 2-day training courses were sufficient to improve use of MI methods compared with TAU29,47,48,49,70 and we aimed at a realistic approach.

Conclusions

MI training of physicians and its application in the clinical routine of specialized pediatric consultations was not associated with increased access to mental health care services by adolescents with CMCs, but more research is needed. The MI approach may help break down barriers and stigma associated with psychological concerns. Involving pediatricians as facilitators of mental health treatment in a collaborative care setting is a potentially effective and sustainable approach to fill the current treatment gaps.

Supplement 1.

Trial Protocol

Supplement 2.

Data Sharing Statement

References

  • 1.Uhlenbusch N, Löwe B, Härter M, Schramm C, Weiler-Normann C, Depping MK. Depression and anxiety in patients with different rare chronic diseases: A cross-sectional study. PLoS One. 2019;14(2):e0211343. doi: 10.1371/journal.pone.0211343 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Pao M, Bosk A. Anxiety in medically ill children/adolescents. Depress Anxiety. 2011;28(1):40-49. doi: 10.1002/da.20727 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Pinquart M, Shen Y. Depressive symptoms in children and adolescents with chronic physical illness: an updated meta-analysis. J Pediatr Psychol. 2011;36(4):375-384. doi: 10.1093/jpepsy/jsq104 [DOI] [PubMed] [Google Scholar]
  • 4.Quittner AL, Saez-Flores E, Barton JD. The psychological burden of cystic fibrosis. Curr Opin Pulm Med. 2016;22(2):187-191. doi: 10.1097/MCP.0000000000000244 [DOI] [PubMed] [Google Scholar]
  • 5.Herzer M, Hood KK. Anxiety symptoms in adolescents with type 1 diabetes: association with blood glucose monitoring and glycemic control. J Pediatr Psychol. 2010;35(4):415-425. doi: 10.1093/jpepsy/jsp063 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Stapersma L, van den Brink G, Szigethy EM, Escher JC, Utens EMWJ. Systematic review with meta-analysis: anxiety and depression in children and adolescents with inflammatory bowel disease. Aliment Pharmacol Ther. 2018;48(5):496-506. doi: 10.1111/apt.14865 [DOI] [PubMed] [Google Scholar]
  • 7.Secinti E, Thompson EJ, Richards M, Gaysina D. Research review: childhood chronic physical illness and adult emotional health - a systematic review and meta-analysis. J Child Psychol Psychiatry. 2017;58(7):753-769. doi: 10.1111/jcpp.12727 [DOI] [PubMed] [Google Scholar]
  • 8.DiMatteo MR, Lepper HS, Croghan TW. Depression is a risk factor for noncompliance with medical treatment: meta-analysis of the effects of anxiety and depression on patient adherence. Arch Intern Med. 2000;160(14):2101-2107. doi: 10.1001/archinte.160.14.2101 [DOI] [PubMed] [Google Scholar]
  • 9.Smith BA, Shuchman M. Problem of nonadherence in chronically ill adolescents: strategies for assessment and intervention. Curr Opin Pediatr. 2005;17(5):613-618. doi: 10.1097/01.mop.0000176443.26872.6e [DOI] [PubMed] [Google Scholar]
  • 10.Kongkaew C, Jampachaisri K, Chaturongkul CA, Scholfield CN. Depression and adherence to treatment in diabetic children and adolescents: a systematic review and meta-analysis of observational studies. Eur J Pediatr. 2014;173(2):203-212. doi: 10.1007/s00431-013-2128-y [DOI] [PubMed] [Google Scholar]
  • 11.Galler A, Tittel SR, Baumeister H, et al. Worse glycemic control, higher rates of diabetic ketoacidosis, and more hospitalizations in children, adolescents, and young adults with type 1 diabetes and anxiety disorders. Pediatr Diabetes. 2021;22(3):519-528. doi: 10.1111/pedi.13177 [DOI] [PubMed] [Google Scholar]
  • 12.Weersing VR, Jeffreys M, Do MT, Schwartz KTG, Bolano C. Evidence base update of psychosocial treatments for child and adolescent depression. J Clin Child Adolesc Psychol. 2017;46(1):11-43. doi: 10.1080/15374416.2016.1220310 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Morey A, Loades ME. Review: how has cognitive behaviour therapy been adapted for adolescents with comorbid depression and chronic illness? a scoping review. Child Adolesc Ment Health. 2020;26(3):252-264. doi: 10.1111/camh.12421 [DOI] [PubMed] [Google Scholar]
  • 14.Moore DA, Nunns M, Shaw L, et al. Interventions to improve the mental health of children and young people with long-term physical conditions: linked evidence syntheses. Health Technol Assess. 2019;23(22):1-164. doi: 10.3310/hta23220 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Dolle K, Schulte-Körne G. The treatment of depressive disorders in children and adolescents. Dtsch Arztebl Int. 2013;110(50):854-860. doi: 10.3238/arztebl.2013.0854 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Stapersma L, van den Brink G, van der Ende J, et al. Effectiveness of disease-specific cognitive behavioral therapy on anxiety, depression, and quality of life in youth with inflammatory bowel disease: a randomized controlled trial. J Pediatr Psychol. 2018;43(9):967-980. doi: 10.1093/jpepsy/jsy029 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Thabrew H, Stasiak K, Hetrick SE, et al. Psychological therapies for anxiety and depression in children and adolescents with long-term physical conditions. Cochrane Database Syst Rev. 2018;12:CD012488. doi: 10.1002/14651858.CD012488.pub2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Schnyder N, Lawrence D, Panczak R, et al. Perceived need and barriers to adolescent mental health care: agreement between adolescents and their parents. Epidemiol Psychiatr Sci. 2019;29:e60. doi: 10.1017/S2045796019000568 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Black L, Panayiotou M, Humphrey N. The dimensionality and latent structure of mental health difficulties and wellbeing in early adolescence. PLoS One. 2019;14(2):e0213018. doi: 10.1371/journal.pone.0213018 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Corrigan P. How stigma interferes with mental health care. Am Psychol. 2004;59(7):614-625. doi: 10.1037/0003-066X.59.7.614 [DOI] [PubMed] [Google Scholar]
  • 21.Salaheddin K, Mason B. Identifying barriers to mental health help-seeking among young adults in the UK: a cross-sectional survey. Br J Gen Pract. 2016;66(651):e686-e692. doi: 10.3399/bjgp16X687313 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Hintzpeter B, Klasen F, Schön G, Voss C, Hölling H, Ravens-Sieberer U; BELLA study group . Mental health care use among children and adolescents in Germany: results of the longitudinal BELLA study. Eur Child Adolesc Psychiatry. 2015;24(6):705-713. doi: 10.1007/s00787-015-0676-6 [DOI] [PubMed] [Google Scholar]
  • 23.Jörg F, Visser E, Ormel J, Reijneveld SA, Hartman CA, Oldehinkel AJ. Mental health care use in adolescents with and without mental disorders. Eur Child Adolesc Psychiatry. 2016;25(5):501-508. doi: 10.1007/s00787-015-0754-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Lipson SK, Lattie EG, Eisenberg D. Increased rates of mental health service utilization by U.S. college students: 10-year population-level trends (2007-2017). Psychiatr Serv. 2019;70(1):60-63. doi: 10.1176/appi.ps.201800332 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Rickwood D, Deane FP, Wilson CJ, Ciarrochi J. Young people’s help-seeking for mental health problems. AeJAMH. 2005;4(3):218-251. doi: 10.5172/jamh.4.3.218. [DOI] [Google Scholar]
  • 26.Briggs-Gowan MJ, Horwitz SM, Schwab-Stone ME, Leventhal JM, Leaf PJ. Mental health in pediatric settings: distribution of disorders and factors related to service use. J Am Acad Child Adolesc Psychiatry. 2000;39(7):841-849. doi: 10.1097/00004583-200007000-00012 [DOI] [PubMed] [Google Scholar]
  • 27.Yonek J, Lee C-M, Harrison A, Mangurian C, Tolou-Shams M. Key components of effective pediatric integrated mental health care models: a systematic review. JAMA Pediatr. 2020;174(5):487-498. doi: 10.1001/jamapediatrics.2020.0023 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Westra HA, Aviram A. Core skills in motivational interviewing. Psychotherapy (Chic). 2013;50(3):273-278. doi: 10.1037/a0032409 [DOI] [PubMed] [Google Scholar]
  • 29.Miller WR RS. Motivational interviewing. Preparing people for change. 2nd ed.New York: The Guilford Press; 2002. [Google Scholar]
  • 30.Grupp-Phelan J, McGuire L, Husky MM, Olfson M. A randomized controlled trial to engage in care of adolescent emergency department patients with mental health problems that increase suicide risk. Pediatr Emerg Care. 2012;28(12):1263-1268. doi: 10.1097/PEC.0b013e3182767ac8 [DOI] [PubMed] [Google Scholar]
  • 31.Dean S, Britt E, Bell E, Stanley J, Collings S. Motivational interviewing to enhance adolescent mental health treatment engagement: a randomized clinical trial. Psychol Med. 2016;46(9):1961-1969. doi: 10.1017/S0033291716000568 [DOI] [PubMed] [Google Scholar]
  • 32.Hoek W, Marko M, Fogel J, et al. Randomized controlled trial of primary care physician motivational interviewing versus brief advice to engage adolescents with an Internet-based depression prevention intervention: 6-month outcomes and predictors of improvement. Transl Res. 2011;158(6):315-325. doi: 10.1016/j.trsl.2011.07.006 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Saulsberry A, Marko-Holguin M, Blomeke K, et al. Randomized clinical trial of a primary care internet-based intervention to prevent adolescent depression: one-year outcomes. J Can Acad Child Adolesc Psychiatry. 2013;22(2):106-117. [PMC free article] [PubMed] [Google Scholar]
  • 34.Van Voorhees BW, Fogel J, Pomper BE, et al. Adolescent dose and ratings of an internet-based depression prevention program: a randomized trial of primary care physician brief advice versus a motivational interview. J Cogn Behav Psychother. 2009;9(1):1-19. [PMC free article] [PubMed] [Google Scholar]
  • 35.Lundahl B, Moleni T, Burke BL, et al. Motivational interviewing in medical care settings: a systematic review and meta-analysis of randomized controlled trials. Patient Educ Couns. 2013;93(2):157-168. doi: 10.1016/j.pec.2013.07.012 [DOI] [PubMed] [Google Scholar]
  • 36.Mutschler C, Naccarato E, Rouse J, Davey C, McShane K. Realist-informed review of motivational interviewing for adolescent health behaviors. Syst Rev. 2018;7(1):109. doi: 10.1186/s13643-018-0767-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Reinauer C, Viermann R, Förtsch K, et al. ; COACH consortium . Motivational Interviewing as a tool to enhance access to mental health treatment in adolescents with chronic medical conditions and need for psychological support (COACH-MI): study protocol for a clusterrandomised controlled trial. Trials. 2018;19(1):629. doi: 10.1186/s13063-018-2997-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Spitzer RL, Kroenke K, Williams JBW, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006;166(10):1092-1097. doi: 10.1001/archinte.166.10.1092 [DOI] [PubMed] [Google Scholar]
  • 39.Kroenke K, Spitzer RL. The PHQ-9: a new depression diagnostic and severity measure. Psychiatr Ann. 2002;32(9):509-515. doi: 10.3928/0048-5713-20020901-06 [DOI] [Google Scholar]
  • 40.Posner K, Brown GK, Stanley B, et al. The Columbia-Suicide Severity Rating Scale: initial validity and internal consistency findings from three multisite studies with adolescents and adults. Am J Psychiatry. 2011;168(12):1266-1277. doi: 10.1176/appi.ajp.2011.10111704 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Mahler C, Hermann K, Horne R, et al. Assessing reported adherence to pharmacological treatment recommendations. translation and evaluation of the Medication Adherence Report Scale (MARS) in Germany. J Eval Clin Pract. 2010;16(3):574-579. doi: 10.1111/j.1365-2753.2009.01169.x [DOI] [PubMed] [Google Scholar]
  • 42.Pocock SJ, Simon R. Sequential treatment assignment with balancing for prognostic factors in the controlled clinical trial. Biometrics. 1975;31(1):103-115. doi: 10.2307/2529712 [DOI] [PubMed] [Google Scholar]
  • 43.Hintzpeter B, Metzner F, Pawils S, et al. Inanspruchnahme von ärztlichen und psychotherapeutischen Leistungen durch Kinder und Jugendliche mit psychischen Auffälligkeiten. Kindh Entwickl. 2014;23(4):229-238. doi: 10.1026/0942-5403/a000148 [DOI] [Google Scholar]
  • 44.Brown H, Prescott R. Applied mixed models in medicine. Chichester: John Wiley & Sons; 1999. [Google Scholar]
  • 45.Whiteford HA, Harris MG, McKeon G, et al. Estimating remission from untreated major depression: a systematic review and meta-analysis. Psychol Med. 2013;43(8):1569-1585. doi: 10.1017/S0033291712001717 [DOI] [PubMed] [Google Scholar]
  • 46.Mehler-Wex C, Kölch M. Depression in children and adolescents. Dtsch Arztebl Int. 2008;105(9):149-155. doi: 10.3238/arztebl.2008.0149 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Rubak S, Sandbaek A, Lauritzen T, Borch-Johnsen K, Christensen B. An education and training course in motivational interviewing influence: GPs’ professional behaviour--ADDITION Denmark. Br J Gen Pract. 2006;56(527):429-436. [PMC free article] [PubMed] [Google Scholar]
  • 48.Miller WR, Yahne CE, Moyers TB, Martinez J, Pirritano M. A randomized trial of methods to help clinicians learn motivational interviewing. J Consult Clin Psychol. 2004;72(6):1050-1062. doi: 10.1037/0022-006X.72.6.1050 [DOI] [PubMed] [Google Scholar]
  • 49.Broers S, Smets E, Bindels P, et al. Training general practitioners in behavior change counseling to improve asthma medication adherence. Patient Educ Couns. 2005;58(3):279-287. doi: 10.1016/j.pec.2005.06.001 [DOI] [PubMed] [Google Scholar]
  • 50.Grupp-Phelan J, Stevens J, Boyd S, et al. Effect of a motivational interviewing-based intervention on initiation of mental health treatment and mental health after an emergency department visit among suicidal adolescents: a randomized clinical trial. JAMA Netw Open. 2019;2(12):e1917941. doi: 10.1001/jamanetworkopen.2019.17941 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Vallabhan MK, Kong AS, Jimenez EY, Summers LC, DeBlieck CJ, Feldstein Ewing SW. Training primary care providers in the use of motivational interviewing for youth behavior change. Res Theory Nurs Pract. 2017;31(3):219-232. doi: 10.1891/1541-6577.31.3.219 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52.Schleider JL, Weisz JR. Little treatments, promising effects? meta-analysis of single-session interventions for youth psychiatric problems. J Am Acad Child Adolesc Psychiatry. 2017;56(2):107-115. doi: 10.1016/j.jaac.2016.11.007 [DOI] [PubMed] [Google Scholar]
  • 53.Forman DP, Moyers TB. With odds of a single session, motivational interviewing is a good bet. Psychotherapy (Chic). 2019;56(1):62-66. doi: 10.1037/pst0000199 [DOI] [PubMed] [Google Scholar]
  • 54.Chiappetta L, Stark S, Mahmoud KF, Bahnsen KR, Mitchell AM. Motivational interviewing to increase outpatient attendance for adolescent psychiatric patients. J Psychosoc Nurs Ment Health Serv. 2018;56(6):31-35. doi: 10.3928/02793695-20180212-04 [DOI] [PubMed] [Google Scholar]
  • 55.Lawrence P, Fulbrook P, Somerset S, Schulz P. Motivational interviewing to enhance treatment attendance in mental health settings: a systematic review and meta-analysis. J Psychiatr Ment Health Nurs. 2017;24(9-10):699-718. doi: 10.1111/jpm.12420 [DOI] [PubMed] [Google Scholar]
  • 56.Marker I, Norton PJ. The efficacy of incorporating motivational interviewing to cognitive behavior therapy for anxiety disorders: a review and meta-analysis. Clin Psychol Rev. 2018;62:1-10. doi: 10.1016/j.cpr.2018.04.004 [DOI] [PubMed] [Google Scholar]
  • 57.Hamrin V, Iennaco JD. Evaluation of motivational interviewing to improve psychotropic medication adherence in adolescents. J Child Adolesc Psychopharmacol. 2017;27(2):148-159. doi: 10.1089/cap.2015.0187 [DOI] [PubMed] [Google Scholar]
  • 58.Caccavale LJ, Corona R, LaRose JG, Mazzeo SE, Sova AR, Bean MK. Exploring the role of motivational interviewing in adolescent patient-provider communication about type 1 diabetes. Pediatr Diabetes. 2019;20(2):217-225. doi: 10.1111/pedi.12810 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 59.Powell PW, Hilliard ME, Anderson BJ. Motivational interviewing to promote adherence behaviors in pediatric type 1 diabetes. Curr Diab Rep. 2014;14(10):531. doi: 10.1007/s11892-014-0531-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 60.Sanci L, Chondros P, Sawyer S, et al. Responding to young people’s health risks in primary care: a cluster randomised trial of training clinicians in screening and motivational interviewing. PLoS One. 2015;10(9):e0137581. doi: 10.1371/journal.pone.0137581 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 61.Osier E, Wang AS, Tollefson MM, et al. Pediatric psoriasis comorbidity screening guidelines. JAMA Dermatol. 2017;153(7):698-704. doi: 10.1001/jamadermatol.2017.0499 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 62.Domhardt M, Steubl L, Baumeister H. Internet- and mobile-based interventions for mental and somatic conditions in children and adolescents. Z Kinder Jugendpsychiatr Psychother. 2020;48(1):33-46. doi: 10.1024/1422-4917/a000625 [DOI] [PubMed] [Google Scholar]
  • 63.Desai N. The role of motivational interviewing in children and adolescents in pediatric care. Pediatr Ann. 2019;48(9):e376-e379. doi: 10.3928/19382359-20190816-01 [DOI] [PubMed] [Google Scholar]
  • 64.Domhardt M, Schröder A, Geirhos A, Steubl L, Baumeister H. Efficacy of digital health interventions in youth with chronic medical conditions: a meta-analysis. Internet Interv. 2021;24:100373. doi: 10.1016/j.invent.2021.100373 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 65.Baumeister H, Nowoczin L, Lin J, et al. Impact of an acceptance facilitating intervention on diabetes patients’ acceptance of Internet-based interventions for depression: a randomized controlled trial. Diabetes Res Clin Pract. 2014;105(1):30-39. doi: 10.1016/j.diabres.2014.04.031 [DOI] [PubMed] [Google Scholar]
  • 66.Lunkenheimer F, Domhardt M, Geirhos A, et al. ; COACH consortium . Effectiveness and cost-effectiveness of guided Internet- and mobile-based CBT for adolescents and young adults with chronic somatic conditions and comorbid depression and anxiety symptoms (youthCOACHCD): study protocol for a multicentre randomized controlled trial. Trials. 2020;21(1):253. doi: 10.1186/s13063-019-4041-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 67.Geirhos A, Lunkenheimer F, Holl RW, et al. Involving patients’ perspective in the development of an internet- and mobile-based CBT intervention for adolescents with chronic medical conditions: findings from a qualitative study. Internet Interv. 2021;24:100383. doi: 10.1016/j.invent.2021.100383 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 68.Munder T, Barth J. Cochrane’s risk of bias tool in the context of psychotherapy outcome research. Psychother Res. 2018;28(3):347-355. doi: 10.1080/10503307.2017.1411628 [DOI] [PubMed] [Google Scholar]
  • 69.Hurlocker MC, Madson MB, Schumacher JA. Motivational interviewing quality assurance: a systematic review of assessment tools across research contexts. Clin Psychol Rev. 2020;82:101909. doi: 10.1016/j.cpr.2020.101909 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 70.Brug J, Spikmans F, Aartsen C, Breedveld B, Bes R, Fereira I. Training dietitians in basic motivational interviewing skills results in changes in their counseling style and in lower saturated fat intakes in their patients. J Nutr Educ Behav. 2007;39(1):8-12. doi: 10.1016/j.jneb.2006.08.010 [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplement 1.

Trial Protocol

Supplement 2.

Data Sharing Statement


Articles from JAMA Network Open are provided here courtesy of American Medical Association

RESOURCES