Abstract
Background
Few studies have examined the practical effectiveness and implementation potential of brief psychotherapies that integrate mental and physical health.
Objective
To determine whether an integrated brief cognitive behavioral therapy (bCBT), delivered by mental health providers in primary care, would improve depression, anxiety and quality of life for medically ill veterans.
Design
Pragmatic patient-randomized trial comparing bCBT to enhanced usual care (EUC).
Participants
A total of 302 participants with heart failure and/or chronic obstructive pulmonary disease (COPD) with elevated symptoms of depression and/or anxiety were enrolled from two Veterans Health Administration primary care clinics.
Intervention
bCBT was delivered to 180 participants by staff mental health providers (n = 19). bCBT addressed physical and emotional health using a modular, skill-based approach. bCBT was delivered in person or by telephone over 4 months. Participants randomized to EUC (n = 122) received a mental health assessment documented in their medical record.
Main Measures
Primary outcomes included depression (Patient Health Questionnaire) and anxiety (Beck Anxiety Inventory). Secondary outcomes included health-related quality of life. Assessments occurred at baseline, posttreatment (4 months), and 8- and 12-month follow-up.
Key Results
Participants received, on average, 3.9 bCBT sessions with 63.3% completing treatment (4+ sessions). bCBT improved symptoms of depression (p = 0.004; effect size, d = 0.33) and anxiety (p < 0.001; d = 0.37) relative to EUC at posttreatment, with effects maintained at 8 and 12 months. Health-related quality of life improved posttreatment for bCBT participants with COPD but not for heart failure. Health-related quality of life outcomes were not maintained at 12 months.
Conclusions
Integrated bCBT is acceptable to participants and providers, appears feasible for delivery in primary care settings and is effective for medically ill veterans with depression and anxiety. Improvements for both depression and anxiety were modest but persistent, and the impact on physical health outcomes was limited to shorter-term effects and COPD participants.
Clinical trials.Gov identifier: NCT01149772
KEY WORDS: clinical trial, cognitive behavioral therapy, primary care, veterans, heart disease, COPD
Introduction
Depression and anxiety are common in primary care,1 – 3 especially among medically ill patients.4 – 6 Despite the significant negative impact of depression and anxiety on patient health, few medically ill patients are recognized and appropriately treated for their mental health difficulties.3 , 4 Evidence-based psychotherapies (EBPs), including cognitive behavioral therapy (CBT), exist for depression and anxiety7 but are infrequently used in primary care clinics.8 , 9
EBPs, which typically involve 12 or more 1-h weekly therapy sessions and do not integrate physical health components, have proven challenging to implement in non-mental health settings such as primary care, and access to these services remains low.10 , 11 Data from the Veterans Health Administration (VHA) found that only one fourth of veterans with a new mental health diagnosis received at least one session of psychotherapy and only 10% received four or more sessions in the year following diagnosis.9
CBT has proven effective for medically ill patients, but studies have used high-intensity procedures (e.g., weekly sessions over 6 months) that are difficult to deliver in routine primary care settings.12 Other work, largely from efficacy trials, suggests that brief CBT may be an effective intervention for primary care,13 , 14 but bCBT has not addressed the potential integration of physical health concepts to improve patient engagement, treatment completion and outcomes.15 – 17 No known studies have tested an integrated physical and mental health treatment that can address the needs of medically ill patients while retaining feasibility for use primary care clinics.
This report details the primary outcomes of a pragmatic patient-randomized trial of bCBT delivered by mental health providers in primary care who agreed to deliver bCBT as part of routine care practices at two large VHA hospital primary care clinics. We hypothesized that integrated bCBT would be effective for depression and anxiety and secondarily for health-related quality of life (QOL). We also evaluated the implementation potential of bCBT as measured by provider intervention fidelity and patient-participant treatment initiation and completion.
Methods
The study used a parallel, patient-randomized clinical trial design to understand bCBT effectiveness and implementation potential.18 , 19 Data were collected at the Michael E. DeBakey (Houston) and Oklahoma City VA Medical Centers. The study was approved by local institutional review boards at both facilities. Participants were consented and compensated $110 for study assessments ($20 for baseline and $30 for 4-, 8- and 12-month follow-ups). Study enrollment began in February 2011 and ended in November 2013.
Participant Sample
The study used VHA databases and follow-up medical-record reviews to identify participants with a diagnosis of heart failure (HF; ICD-9 codes 398.91, 402.01, 402.11, 402.91, 404.01, 404.03, 404.11, 404.13, 404.91, 404.93, 428, 428.10, 428.9) or chronic obstructive pulmonary disease (COPD; ICD-9 codes 490, 491, 492, 493, 496, 508).20 , 21 Opt-out letters were mailed to all veterans with a chart diagnosis of HF and/or COPD who were actively enrolled with a primary care provider. Study staff contacted potential participants up to three times. Interested participants completed a telephone screening and were included if they (1) reported depression or anxiety symptoms using a five-item PRIME-MD screener (2 items for depression; 3 items for anxiety)22 and (2) screened positive for mild or greater functional impairment, as determined by the Medical Research Council dyspnea scale (score ≥3)23 , 24 and/or the New York Heart Association classification (class 2 or greater).25 Participants were also required to screen negative for cognitive impairment,26 current substance abuse and psychotic and bipolar disorder (Mini-International Neuropsychiatric Interview)27 and could not have active intent or a plan to commit suicide. Participants with suicidal ideation, without intent or plan, were eligible.
Final eligibility was determined during a baseline evaluation that required elevated symptoms of anxiety [Beck Anxiety Inventory (BAI) score of ≥16]28 and/or depression (Patient Health Questionnaire; PHQ-9 score of ≥10).29 Before randomization, a medical-record review confirmed that participants were not receiving any competing psychotherapy interventions.
Effectiveness Measures and Data Collection
Participants provided demographic information, including race/ethnicity, during the baseline evaluation. All randomized participants received an evaluation at 4, 8 and 12 months by independent evaluators masked to participant randomization group. The 4-month evaluation represented the immediate posttreatment assessment.
Primary Outcomes—Depression and Anxiety
Depression was assessed using the Patient Health Questionnaire (PHQ-9),30 a widely used and psychometrically strong nine-item measure. Scores range from 0 to 27, with scores ≥10 suggesting the presence of clinically significant depression. PHQ-9 treatment response was defined as a 50% reduction in total score or a total score of 9 or less indicating minimal or less symptoms.30
Anxiety was assessed using the Beck Anxiety Inventory (BAI),31 a reliable, valid, 21-item measure. Scores on the BAI range from 0 to 63, with scores between 16 and 25 indicating moderate anxiety and scores ≥26 indicating severe anxiety. Treatment response for the BAI was defined as either (1) a 50% reduction in total score or (2) mild or lower total score (e.g., score below 16).
Secondary Outcomes—Health-Related QOL
Participants with COPD completed the Chronic Respiratory Questionnaire (CRQ), a well-validated, 20-item self-report instrument with four subscales: dyspnea, fatigue, emotional functioning and mastery.32 Higher scores reflect better functioning, and an increase of 0.5 on any subscale score indicates a minimally clinically important difference.33 Participants with HF completed the Kansas City Cardiomyopathy Questionnaire (KCCQ34), a 23-item self-report inventory frequently used in cardiology trials to assess change in physical limitations, HF symptoms, self-efficacy, QOL and social limitations.35 The KCCQ overall summary score (OSS) was used, with higher scores reflecting improvement; an increase of 5 points was used to define response.36 Participants with both COPD and HF completed both instruments.
Controlling Variables
VHA patient administrative databases provided participants’ healthcare utilization information. Data were examined for three 4-month periods: baseline to 4 months, 4 to 8 months and 8 to 12 months.
Depression and Anxiety Medications
Changes in depression and/or anxiety medications were assessed at each time period for (1) the start of an anxiety or depression medication, (2) change in the dose of an existing medication or (3) a change in medication type. A dichotomous indicator variable (yes/no) was used to define active medication management.
Mental Health and Primary Care Visits
Current Procedural Terminology codes were used to identify mental health services (90801–90,911; 96,100–96,155). Primary care and specialty medical care visits were identified using VHA outpatient clinic stop codes. Visits were collected using count data (e.g., number of visits) during each of the three study time periods.
Intervention and Intervention Providers
Mental health providers and advanced trainees recruited from primary care settings delivered the bCBT. Providers were approached if psychotherapy was part of their scope of practice. Participating providers (n = 19) included six psychologists, two social workers, two physician assistants, six psychology fellows and three psychology interns. Providers received bCBT training and support including (1) access to an online bCBT training program, (2) session feedback and (3) facilitation to support bCBT use in primary care.37
bCBT was delivered in six weekly/biweekly sessions and two telephone “booster” sessions over 4 months. Participants attended the first session in person and could participate by telephone thereafter. Providers followed a manual. Patient-participants received a workbook to guide session content. Information about development and clinical content of the intervention can be found elsewhere.20 , 38 Intervention materials are available for public download through a VA website, including the provider manual39 and patient workbook.40
Providers and participants collaboratively determined the number and type (in-person or telephone) of sessions, depending on clinical need, provider availability and patient-participant preferences for care. bCBT content targeted management of physical and emotional symptoms. Integration of physical health occurred throughout the intervention. Providers reviewed life stressors, prior coping efforts, current strengths and resources to support improvements in both physical and emotional health. Providers also used structured goal setting and action-planning assignments to align physical and emotional health targets.
A menu of bCBT skill sessions was offered. Skills were selected to meet the individual needs of the patient-participant. Not all participants received all skill types. Skill sessions included (1) identification and modification of maladaptive thinking, (2) behavioral activation, (3) relaxation and (4) chronic disease self-management (CDSM), including diet/nutrition, exercise, managing medications, talking to your doctor and coping with exacerbations. Although CDSM skills addressed physical health concerns directly, other skills could address emotional or physical health concerns. For example, maladaptive thinking skills could address negative views related to coping with either physical or emotional health issues, behavioral activation could address physical activity (exercise) or pleasant events, and relaxation could address anxiety experiences or aid in coping with worry associated with physical health.
Intervention Fidelity
Providers delivered care to an average of 9.5 participants (median of 7; range 1–25). Sessions were audio recorded to assess fidelity and provide feedback to providers. Two experienced clinical psychologists (MS and NH) rated sessions for adherence and competence, using an established scale.41 Scores ranged from 1–8, with 4–5 considered “moderately” adherent/competent and 6, 7 and 8 classified as good, very good and excellent, respectively. The project collected 602 audio recordings and reviewed 137 (23%) for fidelity. Raters reviewed an average of 7.2 sessions for each provider. Mean adherence and competence ratings for all providers fell in the “good”-to-“very-good” range (adherence 6.7, competence 6.2).
Treatment Delivery and Implementation Potential
Data on participant treatment utilization of bCBT was collected from chart reviews for all bCBT participants. bCBT initiation was defined as attending one or more sessions and completion as receipt of four or more sessions. Implementation data were collected from bCBT veteran participants (posttreatment) and providers (after 6 months of bCBT delivery).
Participant Randomization
Once eligible, participants had been randomized to enhanced usual care (EUC) or bCBT, with 60% of participants going to bCBT and 40% to EUC. Unequal randomization was used to increase statistical power given the need to adjust for provider effects.42 Random number lists were created, using blocks of ten for three participant groups: COPD, HF and a group with both conditions. The study coordinator randomly assigned participants using sealed envelopes.
Participants in EUC received an assessment for depression and anxiety with a note placed in their electronic medical record encouraging providers to address these difficulties as part of their standard care practices.
Data Analyses
Primary analyses examined posttreatment differences between bCBT and EUC for depression (PHQ-9), anxiety (BAI) and health-related QOL (CRQ and KCCQ). The posttreatment outcome served as the dependent variable for the generalized linear models, with treatment group as the independent variable. Control variables included the baseline value for the outcome, the participant’s diagnosis of COPD and/or HF, baseline functional score (MRC or NYHA), depression and anxiety medication, and primary care and mental health visits for baseline to 4 months. Participants were nested within site.
Analyses were intention to treat (ITT), using multiple imputation procedures in SAS version 9.4 (SAS Institute, Inc., Cary, NC) to handle missing values. For missing data, the Markov Chain Monte Carlo method was used.43 We calculated the effect size, d, at 4 months, using the mean values from the estimated means and standard errors from linear model analyses of the imputed data sets. The 95% confidence interval for the effect size was calculated using formulas from Howell.44 Treatment response data were analyzed using “number needed to treat” (NNT) on the primary outcome measures.
Longitudinal analyses examined long-term outcomes, using a repeated measures linear mixed model. The outcome at 8 or 12 months served as the dependent variable. The model included terms for treatment group (bCBT or EUC), time (8 or 12 months) and the time-by-treatment interaction. Control variables included in the 4-month model were used for 8- and 12-month analyses. Participants were nested within site, and critical α for testing significance was set at p ≤ 0.05. All tests were two-sided.
Responder analyses focused on participants who met criteria for elevated baseline symptoms of depression and anxiety separately. For COPD and HF-related functioning, all participants completing the CRQ or KCCQ were included. Differences between treatment groups were tested by combining the chi-square analyses for the imputed data sets.45 , 46 We also conducted chi-square completer analyses using observed data with missing data excluded.
Implementation analyses were restricted to descriptive data for intervention fidelity, bCBT initiation and completion rates, and survey responses from providers and bCBT patient-participants.
Sample Size
The sample-size calculations examined differences in primary and secondary outcomes. To ensure 80% power at the 0.05 level, adjusting for repeated measures and intraclass correlations related to clustering of participants within providers, a sample size of 240 was needed. Given a potential 25% attrition, a final sample of 320 participants was estimated. Attrition was defined as being lost to follow-up (e.g., did not complete study assessments). The recruited sample of 302 participants was 94% of target.
Results
Sample Selection and Attrition
Of 8835 eligible participants, 1387 completed screening; 432 completed a baseline assessment (Fig. 1). Of these, 130 were excluded before randomization with the majority having subclinical scores on the BAI and PHQ-9 (n = 122). A total of 302 participants were randomized, 180 to bCBT and 122 to EUC.
Fig. 1.
Participant recruitment
The two groups did not differ significantly in proportion of participants completing the 4-month assessment [bCBT 73.3% (132/180), EUC 82.7% (101/122), p = 0.09]. Attrition for 8 and 12 months was comparable for bCBT and EUC. Total attrition at 12-month follow-up was 25.8% (n = 78). Reasons for dropout are shown in Fig. 1. Analyses indicated no significant differences between completers and noncompleters.
Participants did not differ significantly on baseline sociodemographic and clinical characteristics by intervention status (Table 1). Participant mean age was 65.5, and over 90% were male. Of those randomized, 62% had COPD, 25% had HF, and 13% had both. Over 85% of participants had elevated depression symptoms, 75.5% had elevated anxiety symptoms, and 62.9% had both. A total of 180 (59.6%) participants met criteria for a diagnosis of depression or anxiety according to the MINI, and 103 participants received an antianxiety or antidepressant medication or medication change during the 90 days before baseline (Fig. 2).
Table 1.
Baseline Sociodemographic and Clinical Characteristics by Intervention Status
| Overall N (n = 302) | bCBT (n = 180) | EUC (n = 122) | |
|---|---|---|---|
| Age (M, SD) | 65.5 (SD 8.6) | 64.9 (SD 8.8) | 66.5 (SD 8.3) |
| Education | |||
| High school or less | 123 (40.7%) | 73 (40.6%) | 50 (41.0%) |
| Some college | 134 (44.4%) | 80 (44.4%) | 54 (44.3%) |
| College graduate | 45 (14.9%) | 27 (15.0%) | 18 (14.8%) |
| Male sex | 285 (94.4%) | 169 (93.8%) | 116 (95.1%) |
| Race/ethnicity | |||
| Non-Hispanic white | 205 (67.8%) | 120 (66.7%) | 85 (69.7%) |
| African American | 69 (22.9%) | 42 (23.3%) | 27 (22.1%) |
| Hispanic | 8 (2.7%) | 3 (1.7%) | 5 (4.1%) |
| Other | 20 (6.6%) | 15 (8.3%) | 5 (4.1%) |
| Living status | |||
| Alone | 70 (23.2%) | 35 (19.4%) | 35 (28.7%) |
| Spouse | 169 (56.0%) | 108 (60.0%) | 61 (50.0%) |
| Family/other | 63 (20.8%) | 37 (20.6%) | 26 (21.3%) |
| Marital status | |||
| Married | 185 (61.3%) | 116 (64.4%) | 69 (56.6%) |
| Not married | 117 (38.7%) | 64 (35.6%) | 53 (43.4%) |
| Income | |||
| Less than $20 k | 104 (34.7%) | 67 (37.4%) | 37 (30.6%) |
| $20–$39 k | 107 (35.7%) | 61 (34.1%) | 46 (38.0%) |
| $40 + k | 89 (29.7%) | 51 (28.5%) | 38 (31.4%) |
| Chronic cardiopulmonary condition | |||
| COPD only | 187 (61.9%) | 112 (62.2%) | 75 (61.5%) |
| Heart failure only | 75 (24.8%) | 44 (24.4%) | 31 (25.4%) |
| Both | 40 (13.2%) | 24 (13.3%) | 16 (13.1%) |
| COPD/HF functional impairmenta | |||
| Mild | 66 (21.9%) | 43 (23.9%) | 23 (18.9%) |
| Moderate | 146 (48.3%) | 84 (46.7%) | 62 (50.8%) |
| Severe | 90 (29.8%) | 53 (29.4%) | 37 (30.3%) |
| PHQ-9 | |||
| Total score (M, SD) | 14.2 (SD 4.8) | 13.8 (SD 4.6) | 14.9 (SD 5.1) |
| N and % above cutoffb | 264 (87.4%) | 155 (86.1%) | 109 (89.3%) |
| BAI | |||
| Total score (M, SD) | 22.0 (SD 9.6) | 21.4 (SD 8.9) | 22.9 (SD |
| N and % above cutoff‡ | 228 (75.5%) | 138 (76.7%) | 10.7) 90 (73.8%) |
| N and % meeting both PHQ-9 and BAI criteria‡ | 190 (62.9%) | 113 (62.8%) | 77 (63.1%) |
| N and % with a depression or anxiety diagnosis | 180 (59.6%) | 108 (60.0%) | 72 (59.0%) |
| N and % with depression or anxiety medication | 103 (34.1%) | 58 (32.2%) | 45 (36.9%) |
aFunctional impairment defined by MRC and NYHA criteria. Mild = MRC scores of 3 or NYHA score of 2; moderate = MRC score of 4 or NYHA score of 3; severe = MRC score of 5 or NYHA score of 4
bCutoff score is 10 for PHQ-9 and 16 for BAI
cDiagnoses assessed using the MINI depression and anxiety subscales
dDepression and anxiety medication assessed for new medications or changes to existing medications during the 90 days prior to baseline
PHQ-9, Patient Health Questionnaire; BAI, Beck Anxiety Inventory; M, mean
SD, standard deviation; MRC, Medical Research Council; NYHA, New York Heart Association; bCBT = brief CBT; EUC = enhanced usual care
Fig. 2.
Flow of participants through each phase of the study
Intervention Data
Participants randomized to bCBT (n = 180) received, on average, 3.9 sessions (SD = 2.3), 152 (84.4%) received 1 or more sessions, 114 (63.3%) were classified as treatment completers (4 or more sessions), and 62 (34.4%) completed 6 sessions. Of participants who received 1 or more sessions, the mean was 4.6 (SD 1.6) and median was 5. Telephone delivery occurred in 60.3% of sessions.
Data on EUC mental health treatment were collected using VHA databases. At 4 months, 32 (26%) of EUC participants had received one or more mental health encounters, 6 (4.92%) had received one or more, and 3 (2.46%) had received four or more psychotherapy sessions. With regard to medications, additions or changes to antianxiety and antidepressant medication during the 4-month treatment period occurred in 61 (56.0%) of bCBT and 48 (44.0%) of EUC participants (chi-square p value = 0.33).
Posttreatment Outcomes (0–4 Months; Table 2)
Table 2.
Main Outcomes: Mean Observed Scores and Intention-to-Treat Analyses of Outcome Measures for Baseline to 4 Months for Participants Receiving CBT or Enhanced Usual Care
| Active treatment | ||||||||
|---|---|---|---|---|---|---|---|---|
| Baseline | 4 months | |||||||
| Measure | Mean | SD | Mean | SD | t | p value | d | 95% CI |
| BAI*,† | (n = 302) | (n = 233) | ||||||
| bCBT | 21.4 | 8.9 | 17.4 | 8.5 | t110 = −4.12 | <0.001 | 0.37 | (0.14, 0.60) |
| EUC | 22.9 | 10.7 | 22.7 | 11.2 | ||||
| PHQ-9† | (n = 302) | (n = 233) | ||||||
| bCBT | 13.8 | 4.6 | 10.3 | 5.1 | t24 = −3.16 | 0.004 | 0.33 | (0.10, 0.56) |
| EUC | 14.9 | 5.1 | 13.3 | 5.8 | ||||
| CRQ dyspnea‡ | (n = 223) | (n = 175) | ||||||
| bCBT | 3.0 | 1.0 | 3.4 | 1.1 | t88 = 2.15 | 0.03 | 0.21 | (−0.05, 0.48) |
| EUC | 2.9 | 1.0 | 2.9 | 1.0 | ||||
| CRQ emotion‡ | (n = 227) | (n = 175) | ||||||
| bCBT | 3.6 | 0.9 | 4.2 | 1.0 | t52 = 4.33 | <0.001 | 0.46 | (0.19, 0.73) |
| EUC | 3.2 | 1.0 | 3.3 | 1.1 | ||||
| CRQ mastery‡ | (n = 227) | (n = 175) | ||||||
| bCBT | 3.6 | 1.0 | 4.3 | 1.1 | t69 = 4.16 | <0.001 | 0.43 | (0.16, 0.70) |
| EUC | 3.5 | 1.2 | 3.5 | 1.2 | ||||
| CRQ fatigue‡ | (n = 227) | (n = 175) | ||||||
| bCBT | 2.6 | 0.9 | 3.2 | 1.1 | t64 = 3.40 | 0.001 | 0.35 | (0.08, 0.61) |
| EUC | 2.4 | 1.0 | 2.6 | 1.1 | ||||
| KCCQ OSS‡ | (n = 114) | (n = 86) | ||||||
| bCBT | 36.9 | 14.9 | 40.3 | 15.1 | t36 = 0.32 | 0.75 | 0.05 | (−0.32, 0.43) |
| EUC | 37.4 | 19.4 | 41.8 | 19.2 | ||||
Abbreviations: BAI, Beck Anxiety Inventory; CRQ, Chronic Respiratory Disease Questionnaire; EUC, enhanced usual care; KCCQ OSS, Kansas City Cardiomyopathy Questionnaire Overall Summary Score; PHQ-9, Patient Health Questionnaire; bCBT, brief cognitive behavioral therapy; EUC, enhanced usual care
Means are observed means
The 95% confidence interval for the effect size was calculated using the mean values from the estimated means and standard errors from the five imputations. Formulas were from Howell42
*Analysis was adjusted for unequal variances
†Lower scores indicate better functioning
‡Higher scores indicate better functioning
Table 2 contains the mean observed scores at baseline and 4 months. Relative to EUC, participants who received bCBT demonstrated greater improvements, except for HF QOL. bCBT participants realized a small-to-medium treatment effect for anxiety (BAI; d = 0.37) and depression (PHQ-9; d = 0.33). Subgroup analyses found no differences between COPD and HF groups in depression or anxiety outcomes. For COPD participants, improvements were seen for dyspnea (d = 0.21), emotion (d = 0.46), mastery (d = 0.43) and fatigue (d = 0.35). No significant treatment effects were noted for HF QOL (d = 0.05).
The number needed to treat (NNT) was calculated using responder data for depression and anxiety separately. The NNT for anxiety was 6.30 (medium effect) and for depression was 6.10 (medium effect).
Long-term Outcomes (4–12 Months; Table 3)
Table 3.
Mean Observed Scores and Intention-to-Treat Analyses of Outcome Measures for Follow-up at 8 to 12 Months for Participants Receiving CBT or Enhanced Usual Care
| Follow-up | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| 8 months | 12 months | Treatment effect | Time effect | Treatment × time effect‡ | ||||||
| Measure | Mean | SD | Mean | SD | t | p value | t | p value | t | p value |
| BAI*,† | (n = 227) | (n = 223) | ||||||||
| bCBT | 16.9 | 9.3 | 17.0 | 7.9 | t56 = −2.88 | 0.006 | t64 = 0.50 | 0.62 | t31 = −0.41 | 0.68 |
| EUC | 21.6 | 11.4 | 21.2 | 11.4 | ||||||
| PHQ-9*,† | (n = 228) | (n = 223) | ||||||||
| bCBT | 10.3 | 5.5 | 10.6 | 5.1 | t42 = −2.67 | 0.011 | t121 = 0.77 | 0.44 | t155 = −0.73 | 0.47 |
| EUC | 13.5 | 6.0 | 13.2 | 6.4 | ||||||
| CRQ dyspnea | (n = 172) | (n = 166) | ||||||||
| bCBT | 3.3 | 1.0 | 3.1 | 1.0 | t79 = .07 | 0.95 | t247 = −0.15 | 0.88 | t190 = 1.03 | 0.31 |
| EUC | 3.0 | 0.9 | 3.1 | 1.1 | ||||||
| CRQ emotion | (n = 172) | (n = 168) | ||||||||
| bCBT | 4.0 | 1.2 | 3.9 | 1.1 | t32 = 1.12 | 0.25 | t124 = −0.14 | 0.89 | t775 = 0.77 | 0.44 |
| EUC | 3.5 | 1.0 | 3.5 | 1.2 | ||||||
| CRQ mastery | (n = 172) | (n = 168) | ||||||||
| bCBT | 4.2 | 1.2 | 4.0 | 1.1 | t29 = 1.57 | 0.13 | t122 = −0.59 | 0.56 | t278 = 1.53 | 0.13 |
| EUC | 3.4 | 1.2 | 3.6 | 1.3 | ||||||
| CRQ fatigue | (n = 172) | (n = 168) | ||||||||
| bCBT | 3.1 | 1.1 | 3.0 | 1.1 | t29 = 1.25 | 0.22 | t125 = −0.37 | 0.71 | T277 = 0.67 | 0.51 |
| EUC | 2.7 | 1.0 | 2.8 | 1.1 | ||||||
| KCCQ OSS | (n = 84) | (n = 84) | ||||||||
| bCBT | 41.3 | 19.9 | 42.9 | 18.3 | t11 = 1.05 | 0.31 | t57 = −1.02 | 0.31 | t21 = 0.27 | 0.79 |
| EUC | 36.2 | 18.2 | 41.7 | 20.8 | ||||||
Abbreviations: BAI, Beck Anxiety Inventory; CRQ, Chronic Respiratory Disease Questionnaire; EUC, enhanced usual care; KCCQ OSS, Kansas City Cardiomyopathy Questionnaire Overall Summary Score; PHQ-9, Patient Health Questionnaire; bCBT, brief cognitive behavioral therapy
Means are observed means
*Analysis was adjusted for unequal variances
†Lower scores indicate better functioning
‡A significant treatment effect for the follow-up analysis indicates that covariate-adjusted post-treatment group difference at 4 months continued over the long-term for the outcome
Covariate-adjusted posttreatment differences between bCBT and EUC continued over the follow-up period for the BAI and PHQ-9 total score and indicated the initial treatment effect was maintained but not improved. Posttreatment effects were not maintained for the CRQ subscales, and no changes were identified for KCCQ over the 12-month follow-up period.
Treatment Response (Table 4)
Table 4.
Response Rate Among Participants with Clinically Elevated Depression or Anxiety
| ITT analyses | ||||||
|---|---|---|---|---|---|---|
| n Responders | % Responders | n Responders | % Responders | |||
| Measure | bCBT | EUC | ||||
| BAI* | (n = 138) | (n = 90) | F statistic | p value | ||
| 4 months | 47 | 34.1 | 17 | 18.9 | F(1, 16) = 4.68 | 0.05 |
| 12 months | 52 | 37.7 | 24 | 26.7 | F(1, 22) = 2.23 | 0.15 |
| PHQ-9 | (n = 155) | (n = 109) | ||||
| PHQ-9 <10 or 50% reduction† | ||||||
| 4 months | 67 | 43.2 | 29 | 26.6 | F(1, 23) = 5.71 | 0.03 |
| 12 months | 57 | 36.8 | 31 | 28.4 | F(1, 9) = 0.72 | 0.42 |
| BAI or PHQ-9‡ | (n = 180) | (n = 122) | ||||
| 4 months | 90 | 50.0 | 40 | 32.8 | F(1,56) = 7.91 | 0.007 |
| 12 months | 81 | 45.0 | 46 | 37.7 | F(1, 12) = 0.76 | 0.40 |
Abbreviations: BAI, Beck Anxiety Inventory; EUC, enhanced usual care; ITT, intention to treat; PHQ-9, Patient Health Questionnaire
The number of responders for the ITT analyses is the mean from the imputed data sets
F statistic is based on Rubin43 and Li et al.44 for combining results of chi-square tests on multiply imputed data sets. F is listed with the numerator and denominator degrees of freedom
*BAI treatment response is a BAI total score below 16 or a 50% reduction in the BAI total score from baseline
†PHQ-9 treatment response is a PHQ-9 total score below 10 or a 50% reduction in the PHQ-9 total score from baseline
‡BAI or PHQ-9 treatment response is a BAI total score below 16 or a 50% reduction in the BAI total score from baseline, a PHQ-9 total score below 10 or a 50% reduction in the PHQ-9 total score from baseline. This definition required clinically elevated BAI or clinically elevated PHQ-9 at baseline
ITT responder analyses indicated a change, relative to EUC, for anxiety (p = 0.05) and depression (p = 0.03) at 4 months with nonsignificant differences at 12 months (p = 0.15 and 0.42, respectively). Combining response for BAI and PHQ-9, we found that 50% of the bCBT group responded at 4 months (relative to 32.8% in EUC; p = 0.007) and 45% responded at 12 months (relative to 37.7% in EUC; p = 0.40).
Implementation Potential: Provider and Patient-participant Survey Data (Table 5)
Table 5.
Provider and Patient-Participant Post-bCBT Survey Data
| Provider items | Provider responses* | ||||
| Overall satisfaction with the bCBT intervention in the primary care setting | Poor N = 0 (0%) |
Fair N = 0 (0%) |
Good N = 3 (16.7%) |
Very good N = 9 (50%) |
Excellent N = 6 (33.3%) |
| The bCBT intervention and procedures are consistent with current evidence-based mental health treatments | Strongly disagree N = 0 (0%) |
Disagree N = 0 (0%) |
Neutral | Agree N = 9 (50%) |
Strongly agree N = 9 (50%) |
| The bCBT intervention is well suited for use within the VA’s Primary Care Mental Health Integration Program | Strongly disagree N = 0 (0%) |
Disagree N = 0 (0%) |
Neutral N = 1 (5.6%) |
Agree N = 8 (44.4%) |
Strongly agree N = 9 (50%) |
| Patient participant items | Patient participant responses* | ||||
| How satisfied are you with the amount of help you received as part of the bCBT program? | Quite dissatisfied N = 1 (0.9%) |
Mildly dissatisfied N = 6 (5.5%) |
Mostly satisfied N = 38 (34.6%) |
Very satisfied N = 65 (59.1%) |
|
| To what extent did the bCBT program meet your needs? | None of my needs have been met N = 3 (2.8%) |
Only a few of my needs have been met N = 20 (18.7%) |
Most of my needs have been met N = 57 (53.3%) |
Almost all of my needs have been met N = 27 (25.2%) |
|
| How important was it for you to have the option for telephone sessions? | Not at all N = 12 (11.0%) |
A little N = 6 (5.5%) |
Somewhat N = 13 (11.9%) |
Very N = 78 (71.6%) |
|
VA = Veterans Administration; bCBT = brief cognitive behavioral therapy
*Response categories vary depending on the survey item used
Providers reported high levels of satisfaction with bCBT (100% of respondents), found use of bCBT to be consistent with current evidence-based practices (100% agreement) and viewed the bCBT as being well suited for use within the primary care setting (94% agreement). bCBT patient-participants were mostly or very satisfied with the amount of assistance received (90% of respondents), found the program met most or all their needs (79% or respondents) and indicated that telephone sessions were somewhat or very important (83% or respondents).
Discussion
The current pragmatic trial documented the effectiveness and implementation potential of physical and emotional health integrated bCBT for medically ill veterans with depression and/or anxiety treated in the primary care setting. Implementation data suggested robust delivery of the intervention by providers, with high levels of participant engagement and adherence. Integrated bCBT resulted in significant and clinically important improvements for depression and anxiety, with short-term effects for physical health outcomes for participants with COPD.
The VHA and other healthcare organizations have dedicated significant resources to re-align primary care with specific efforts to integrate mental health services into a broader multi-professional approach to care.47 , 48 Unfortunately, significant practice gaps exist with regard to mental health interventions addressing physical health needs of patients while also retaining a feasible delivery model for dissemination.
In addition to its primary clinical effectiveness outcomes, our data show the potential for bCBT to improve access and engagement with psychotherapy in primary care. National data on mental health services and psychotherapy use suggest that access to evidence-based care remains limited. Wang et al.49 found that 14.3% of a nationally representative sample of mental health patients received evidence-based treatment. Among veterans with a newly diagnosed mental health condition, only 27% received one or more and 11% received four or more psychotherapy sessions.9 Although data from the current study must be interpreted with caution given the research approach used (e.g., participant vetting), delivery was robust with 84.4% of participants receiving one or more bCBT sessions and 63.3% completing treatment (4+ sessions).
Significant strengths of this study include the pragmatic trial design that used clinical providers for delivering treatment, a large and diverse participant sample from multiple sites, long-term follow-up and comprehensive publically available intervention training and materials.
Study outcomes should be interpreted with the following limitations. First, although this trial used limited exclusion criteria, participants completed screening procedures before randomization and may represent a more curated sample than found in frontline practice. Second, retention of bCBT participants at 4 months was somewhat lower than for EUC, likely because of the increased burden associated with treatment. Third, bCBT outcomes were maintained for depression and anxiety outcomes but generally not maintained for physical health QOL at 12 months. These outcomes suggest that delivery of a bCBT treatment, with an average of less than 4 sessions, may not be enough to sustain longer-term impact in chronic, progressive medical illnesses. Future efforts may require sustained interventions and/or community resources (e.g., peer support or other care management services) to maintain coping and self-management activities. Additional studies also need to better understand the differential impact of the intervention on COPD and HF QOL.
Acknowledgements
This material is the result of work supported by the Department of Veterans Affairs, Health Services Research & Development Grant no. 09-088, the VA South Central Mental Illness Research Education and Clinical Center, and the resources and facilities of the Houston VA HSR&D Center for Innovations in Quality, Effectiveness and Safety (CIN13-413). The funding organizations played no role in the design and conduct of the study; collection, management, analysis and interpretation of the data; preparation, review or approval of the manuscript; or decision to submit the manuscript for publication. The authors thank the providers and leadership of the Michael E. DeBakey VA Medical Center and the Oklahoma City VA Medical Center and also Joseph Mignogna, PhD; Lisa Kearney, PhD; Andrew Pomerantz, MD; and Joanne Kirchner, MD, for their consultation and collaborations. Jeffrey Cully had full access to the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Data analyses were completed by Shubhada Sansgiry, PhD, and Nancy Petersen, PhD. The opinions expressed are those of the authors and not necessarily those of the Department of Veterans Affairs, the US government or Baylor College of Medicine.
Contributions of the authors
Study conception or design—all authors
Acquisition, analysis, or interpretation of data—all authors
Drafting of manuscript—all authors
Critical revision of manuscript for important intellectual content—all authors
Compliance with ethical standards
Conflict of interest
The authors declare that they do not have a conflict of interest.
References
- 1.Olfson M, Shea S, Feder A, et al. Prevalence of anxiety, depression, and substance use disorders in an urban general medicine practice. Arch Fam Med. 2000;9(9):876–883. doi: 10.1001/archfami.9.9.876. [DOI] [PubMed] [Google Scholar]
- 2.Kessler RC, Berglund P, Demler O, et al. The epidemiology of major depressive disorder: Results from the National Comorbidity Survey Replication (NCS-R) JAMA. 2003;289(23):3095–3105. doi: 10.1001/jama.289.23.3095. [DOI] [PubMed] [Google Scholar]
- 3.Kroenke K, Spitzer RL, Williams JB, Monahan PO, Lowe B. Anxiety disorders in primary care: Prevalence, impairment, comorbidity, and detection. Ann Intern Med. 2007;146(5):317–325. doi: 10.7326/0003-4819-146-5-200703060-00004. [DOI] [PubMed] [Google Scholar]
- 4.Kunik ME, Roundy K, Veazey C, et al. Surprisingly high prevalence of anxiety and depression in chronic breathing disorders. Chest. 2005;127(4):1205–11. doi: 10.1378/chest.127.4.1205. [DOI] [PubMed] [Google Scholar]
- 5.Koenig HG. Depression in hospitalized older patients with congestive heart failure. Gen Hosp Psychiatry. 1998;20(1):29–43. doi: 10.1016/S0163-8343(98)80001-7. [DOI] [PubMed] [Google Scholar]
- 6.Roy-Byrne PP, Davidson KW, Kessler RC, et al. Anxiety disorders and comorbid medical illness. Gen Hosp Psychiatry. 2008;30(3):208–225. doi: 10.1016/j.genhosppsych.2007.12.006. [DOI] [PubMed] [Google Scholar]
- 7.Butler AC, Chapman JE, Forman EM, Beck AT. The empirical status of cognitive behavioral therapy: A review of meta-analyses. Clin Psychol Rev. 2006;26(1):17–31. doi: 10.1016/j.cpr.2005.07.003. [DOI] [PubMed] [Google Scholar]
- 8.Shafran R, Clark DM, Fairburn CG, et al. Mind the gap: Improving the dissemination of CBT. Behav Res Ther. 2009;47(11):902–909. doi: 10.1016/j.brat.2009.07.003. [DOI] [PubMed] [Google Scholar]
- 9.Mott JM, Hundt NE, Sansgiry S, Mignogna J, Cully JA. Changes in psychotherapy utilization among veterans with depression, anxiety and PTSD. Psychiatr Serv. 2014;65(1):106–112. doi: 10.1176/appi.ps.201300056. [DOI] [PubMed] [Google Scholar]
- 10.Funderburk JS, Sugarman DE, Labbe AK, Rodriques A, Maistro SA, Nelson B. Behavioral health interventions being implemented in a VA primary care setting. J Clin Psychol Med Settings. 2011;18(1):22–29. doi: 10.1007/s10880-011-9230-y. [DOI] [PubMed] [Google Scholar]
- 11.Mental Health. Cognitive Behavioral Therapy-Depression (CBT-D). US Department of Veterans Affairs. Available at http://www.mentalhealth.va.gov/depression/cbt-d.asp. Last accessed May 25, 2017.
- 12.Freedland KF, Carney RM, Rich MW, et al. Cognitive behavior therapy for depression and self-care in heart failure patients: A randomized clinical trial. JAMA Intern Med. 2015;175(11):1773–1782. doi: 10.1001/jamainternmed.2015.5220. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Stanley MA, Wilson N, Novy DM, et al. Cognitive behavior therapy for generalized anxiety disorder among older adults in primary care: A randomized clinical trial. JAMA. 2009;301(14):1460–1467. doi: 10.1001/jama.2009.458. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Nieuwsma JA, Trivedi RB, McDuffie J, Kronish I, Benjamin D, Williams JW. Brief psychotherapy for depression: A systematic review and meta-analysis. Int J Psychiatry Med. 2012;43:129–51. doi: 10.2190/PM.43.2.c. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Roy-Byrne P, Craske MG, Sullivan G, et al. Delivery of evidence-based treatment for multiple anxiety disorders in primary care. JAMA. 2010;303(19):1921–1928. doi: 10.1001/jama.2010.608. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Katon WJ, Von Korff M, Lin EH, et al. The Pathways Study: A randomized trial of collaborative care in patients with diabetes and depression. JAMA Psychiatry. 2004;61(10):1042–1049. doi: 10.1001/archpsyc.61.10.1042. [DOI] [PubMed] [Google Scholar]
- 17.Rollman BL, Belnap BH, LeMenager MS, et al. Telephone-delivered collaborative care for treating post-CABG depression: A randomized controlled trial. JAMA. 2009;302(19):2095–2103. doi: 10.1001/jama.2009.1670. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Cully JA, Armento MEA, Mott J, et al. Brief cognitive behavioral therapy in primary care: A hybrid type 2 patient randomized effectiveness-implementation design. Implement Sci. 2012;7(64). [DOI] [PMC free article] [PubMed]
- 19.Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: Combining elements of clinical effectiveness and implementation research to enhance public impact. Med Care. 2012;50(3):217–226. doi: 10.1097/MLR.0b013e3182408812. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Cully JA, Stanley MA, Deswal A, Hanania N, Phillips LL, Kunik ME. Cognitive-behavioral therapy for chronic cardiopulmonary conditions: Moving beyond mental health outcomes. Prim Care Companion J Clin Psychiatry. 2010;12(4):e1–e6. doi: 10.4088/PCC.09m00896blu. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Spitzer RL, Williams JBW, Kroenke K, et al. Utility of a new procedure for diagnosing mental disorders in primary care. The PRIME-MD 1000 study. JAMA. 1994;272(22):1749–1756. doi: 10.1001/jama.1994.03520220043029. [DOI] [PubMed] [Google Scholar]
- 22.Kunik ME, Azzam PN, Souchek J, et al. A practical screening tool for anxiety and depression in patients with chronic breathing disorders. Psychosomatics. 2007;48:16–21. doi: 10.1176/appi.psy.48.1.16. [DOI] [PubMed] [Google Scholar]
- 23.Fletcher GM. (Chairman). Standardised questionnaire on respiratory symptoms: a statement prepared and approved by the MRC Committee on the Aetiology of Chronic Bronchitis (MRC breathlessness score) BMJ. 1960;2:166529. [Google Scholar]
- 24.Bestall JC, Paul EA, Garrod R, Garnham R, Jones PW, Wedzicha JA. Usefulness of the Medical Research Council (MRC) dyspnoea scale as a measure of disability in patients with chronic obstructive pulmonary disease. Thorax. 1999;54(7):581–6. doi: 10.1136/thx.54.7.581. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Bennett JA, Riegel B, Bittner V, Nichols J. Validity and reliability of the NYHA classes for measuring research outcomes in patients with cardiac disease. Heart Lung. 2002;31(4):262–70. doi: 10.1067/mhl.2002.124554. [DOI] [PubMed] [Google Scholar]
- 26.Callahan CM, Unverzagt FW, Hui SL, Perkins AJ, Hendrie HC. Six-item screener to identify cognitive impairment among potential subjects for clinical research. Med Care. 2002;40(9):771–81. doi: 10.1097/00005650-200209000-00007. [DOI] [PubMed] [Google Scholar]
- 27.Sheehan DV, Lecrubier Y, Sheehan KH, et al. The Mini-International Neuropsychiatric Interview: The development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry. 1998;59(Suppl 20):22–33. [PubMed] [Google Scholar]
- 28.Beck AT, Epstein N, Brown G, Steer RA. An inventory for measuring clinical anxiety: Psychometric properties. J Consult Clin Psychol. 1988;56:893–897. doi: 10.1037/0022-006X.56.6.893. [DOI] [PubMed] [Google Scholar]
- 29.Spitzer RL, Kroenke K, Williams JB. Validation and utility of a self-report version of PRIME-MD: The PHQ Primary Care Study. JAMA. 1999;282(18):1737–1744. doi: 10.1001/jama.282.18.1737. [DOI] [PubMed] [Google Scholar]
- 30.Kroenke K, Spitzer RL, Williams JB. The PHQ-9: Validity of a brief depression severity measure. J Gen Intern Med. 2001;16(9):606–13. doi: 10.1046/j.1525-1497.2001.016009606.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Beck AT, Steer RA. Manual for the Beck Anxiety Inventory. San Antonio, TX: Psychological Corporation; 1990. [Google Scholar]
- 32.Guyatt GH, Berman LB, Townsend M, Pugsley SO, Chambers LW. A measure of quality of life for clinical trials in chronic lung disease. Thorax. 1987;42(10):773–8. doi: 10.1136/thx.42.10.773. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Jaeschke R, Singer J, Guyatt G. Measurement of health status: Ascertaining the minimally clinically important difference. Control Clin Trials. 1989;10(4):407–415. doi: 10.1016/0197-2456(89)90005-6. [DOI] [PubMed] [Google Scholar]
- 34.Green CP, Porter CB, Bresnahan DR, Spertus JA. Development and evaluation of the Kansas City Cardiomyopathy Questionnaire: A new health status measure for heart failure. J Am Coll Cardiol. 2000;35(5):1245–1255. doi: 10.1016/S0735-1097(00)00531-3. [DOI] [PubMed] [Google Scholar]
- 35.Krumholtz HM. Outcomes research: Generating evidence for best practices and policies. Circulation. 2008;118:309–318. doi: 10.1161/CIRCULATIONAHA.107.690917. [DOI] [PubMed] [Google Scholar]
- 36.Spertus J, Peterson E, Conard MW, et al. Monitoring clinical changes in patients with heart failure: A comparison of methods. Am Heart J. 2005;150(4):707–715. doi: 10.1016/j.ahj.2004.12.010. [DOI] [PubMed] [Google Scholar]
- 37.Mignogna J, Hundt N, Kauth MR, et al. Implementing brief cognitive behavioral therapy in primary care: A pilot study. Transl Behav Med. 2014;4(2):175–183. doi: 10.1007/s13142-013-0248-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Cully JA, Paukert A, Falco J, Stanley MA. Cognitive-behavioral therapy: Innovations for cardiopulmonary patients with depression and anxiety. Cogn Behav Pract. 2009;16:394–407. doi: 10.1016/j.cbpra.2009.04.004. [DOI] [Google Scholar]
- 39.Cully JA, Stanley MA, Kauth M, Naik A, Kunik ME. ACCESS: Adjusting to Chronic Conditions with Education, Support and Skills. Clinician Manual. Version 3.2. VA South Central Mental Illness Research, Education and Clinical Center, Houston, TX. Available at http://www.mirecc.va.gov/VISN16/clinicalEducationProducts.asp. Last accessed May 25, 2017.
- 40.Cully JA, Stanley MA, Kauth M, Naik A, Kunik ME. ACCESS: Adjusting to Chronic Conditions with Education, Support and Skills. Patient Workbook.. VA South Central Mental Illness Research, Education and Clinical Center, Houston, TX. Available at http://www.mirecc.va.gov/VISN16/clinicalEducationProducts.asp. Last accessed May 25, 2017.
- 41.Cully JA, Mignogna J, Stanley MA, et al. Development and pilot testing of a standardized training program for a patient-mentoring intervention to increase adherence to outpatient HIV care. AIDS Patient Care and STDS. 2012;26(3):165–72. doi: 10.1089/apc.2011.0248. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Schnurr PP, Friedman MJ, Engel CC, et al. Issues in the design of multisite clinical trials of psychotherapy: VA Cooperative Study No. 494 as an example. Contemp Clin Trials. 2006;26:626–36. doi: 10.1016/j.cct.2005.09.001. [DOI] [PubMed] [Google Scholar]
- 43.(eds). SAS (Stat 9.2) user’s guide, Second Edition.
- 44.Howell DC. Confidence intervals on effect size. University of Vermont. July 4, 2010. Available at http://www.uvm.edu/~dhowell/methods7/Supplements/Confidence%20Intervals%20on%20Effect%20Size.pdf. Last accessed May 25, 2017.
- 45.Rubin DB. Multiple Imputation for Nonresponse in Surveys. New York: John Wiley and Sons; 1987. [Google Scholar]
- 46.Li KH, Meng XL, Raghunathan TE, Rubin DB. Significance levels from repeated p-values with multiply-imputed data. Statistica Sinica. 1991;1:65–92. [Google Scholar]
- 47.Behavioral Health in Primary Care. Integrating Behavioral Health into Primary Care. SAMHSA-HRSA Center for Integrated Health Solutions. Available at http://www.integration.samhsa.gov/integrated-care-models/behavioral-health-in-primary-care. Last accessed May 25, 2017.
- 48.Zeiss AM, Karlin BE. Integrating mental health and primary care services in the Department of Veterans Affairs Health Care system. J Clin Psychol Med Settings. 2008;15(1):73–78. doi: 10.1007/s10880-008-9100-4. [DOI] [PubMed] [Google Scholar]
- 49.Wang PS, Berglund P, Kessler RC. Recent care of common mental disorders in the United States: Prevalence and conformance with evidence-based recommendations. J Gen Intern Med. 2000;15:284–292. doi: 10.1046/j.1525-1497.2000.9908044.x. [DOI] [PMC free article] [PubMed] [Google Scholar]