Skip to main content
Journal of General Internal Medicine logoLink to Journal of General Internal Medicine
. 2017 Sep 12;32(12):1330–1341. doi: 10.1007/s11606-017-4177-9

Behavioral Health Integration into Primary Care: a Microsimulation of Financial Implications for Practices

Sanjay Basu 1,2,, Bruce E Landon 2,3,4, John W Williams Jr 5, Asaf Bitton 2,4,6,7, Zirui Song 2,8,9, Russell S Phillips 2,3
PMCID: PMC5698230  PMID: 28900839

Abstract

Background

New payments from Medicare encourage behavioral health services to be integrated into primary care practice activities.

Objective

To evaluate the financial impact for primary care practices of integrating behavioral health services.

Design

Microsimulation model.

Participants

We simulated patients and providers at federally qualified health centers (FQHCs), non-FQHCs in urban and rural high-poverty areas, and practices outside of high-poverty areas surveyed by the National Association of Community Health Centers, National Ambulatory Medical Care Survey, National Health and Nutrition Examination Survey, and National Health Interview Survey.

Interventions

A collaborative care model (CoCM), involving telephone-based follow-up from a behaviorist care manager, or a primary care behaviorist model (PCBM), involving an in-clinic behaviorist.

Main Measures

Net revenue change per full-time physician.

Key Results

When behavioral health integration services were offered only to Medicare patients, net revenue was higher under CoCM (averaging $25,026 per MD in year 1 and $28,548/year in subsequent years) than PCBM (−$7052 in year 1 and -$3706/year in subsequent years). When behavioral health integration services were offered to all patients and were reimbursed by Medicare and private payers, only practices adopting the CoCM approach consistently gained net revenues. The outcomes of the model were sensitive to rates of patient referral acceptance, presentation, and therapy completion, but the CoCM approach remained consistently financially viable whereas PCBM would not be in the long-run across practice types.

Conclusions

New Medicare payments may offer financial viability for primary care practices to integrate behavioral health services, but this viability depends on the approach toward care integration.

Electronic supplementary material

The online version of this article (10.1007/s11606-017-4177-9) contains supplementary material, which is available to authorized users.

Keywords: primary care, mental health, behavioral health, costs, cost-benefit analysis

INTRODUCTION

Behavioral health conditions include mental illness, substance abuse, and unhealthy behaviors that affect at least 18% of all US adults1 and are associated with increased morbidity, mortality, and healthcare costs.2 4 Many common behavioral health conditions that do not manifest in severe mental illness can be addressed without the expertise of a psychiatrist.5 Behavioral health services offered through primary care practices are encouraged by recent provisions extending integrated behavioral health coverage benefits to Medicare recipients (see Table 1).30

Table 1.

Input data for the behavioral health integration model

Practice type FQHC Urban non-FQHC, high-poverty zone Rural non-FQHC, high-poverty zone Urban or rural, lower-poverty zone Sources
Practice/patient characteristics
 Volume (unique patients per FTE MD per year) 2040 (960, 3120) 1480 (860, 2100) 1760 (1060, 2460) 1620 (960, 2280) National Association of Community Health Centers, 2014–2015; Medical Group Management Association, 20146 8
 Payer distribution Medicare 11% (10%, 13%), Medicaid 57% (53%, 62%), Private 10% (8%, 11%), Uninsured 22% (20%, 25%) Medicare 27% (18%, 36%), Medicaid 20% (11%, 28%), Private 48% (38%, 58%), Uninsured 5% (1%, 10%) Medicare 20% (13%, 27%), Medicaid 18% (10%, 26%), Private 48% (34%, 62%), Uninsured 14% (2%, 26%) Medicare 22% (18%, 26%), Medicaid 13% (10%, 16%), Private 60% (55%, 64%), Uninsured 9% (6%, 13%) National Association of Community Health Centers, 2014; National Center for Health Statistics, 20156 , 9
 Prevalence of untreated/undertreated unhealthy alcohol use 9.7% (7.4%, 12.1%, Medicare), 23.3% (19.0%, 27.6%, Medicaid), 29.7% (27.9%, 31.6%, private), 30.4% (26.9%, 33.8%, self-pay) National Health and Nutrition Examination Survey, 2011-201410
 Prevalence of untreated/undertreated tobacco smoking 17.4% (14.3%, 20.4%, Medicare), 33.2% (28.4%, 37.9%, Medicaid), 13.3% (12.0%, 14.7%, private), 24.5% (21.7%, 28.2%, self-pay) National Health and Nutrition Examination Survey, 2011-201410
 Prevalence of untreated/undertreated major depressive disorder 14.8% (11.7%, 17.8%, Medicare), 23.8% (19.0%, 28.6%, Medicaid), 4.1% (3.2%, 5.0%, private), 9.8% (7.3%, 12.2%, self-pay) National Health and Nutrition Examination Survey, 2011-201410
 Prevalence of untreated/undertreated generalized anxiety disorder 22.9% (1.2%, 47.0%, Medicare), 26.9% (0.3%, 53.5%, Medicaid), 12.5% (6.2%, 18.9%, private), 7.7% (0.4%, 15.9%, self-pay) National Health Interview Survey, 201511
 Prevalence of co-morbidities requiring referral after screening and assessment 7.5% (5.0%, 10.0%) across payer types Epidemiological surveys of co-morbidities, including opiate abuse, bipolar disorder and schizophrenia, 2013-201512 15
Screening characteristics
 Sensitivity of screening questions PHQ-2 for depression, 76% (68%, 82%), GAD-2 for anxiety, 76% (55%, 89%); AUDIT-C for unhealthy alcohol, 81% (79%, 91%), self-reported tobacco, 88% (78%, 100%) Prior reviews, 1994-201616 19
 Specificity of screening questions PHQ-2 for depression, 87% (82%, 90%), GAD-2 for anxiety, 81% (60%, 92%), AUDIT-C for unhealthy alcohol, 83% (79%, 87%), self-reported tobacco, 89% (79%, 99%) Prior reviews, 1994-201616 19
Follow-up rates
 Proportion of screening positive patients showing up to behavioral therapy, rather than PCP management alone 50% in base case analysis, varied from 15% to 85% in sensitivity analyses SAMHSA-HRSA Center for Integrated Health Solutions, 201320
 Proportion accepting therapy showing up for phone call/appointment 66% in base case analysis, varied from 50% to 90% in sensitivity analyses Descriptive reports21
 Proportion who show up for first phone call/appointment and complete therapy 80% in base case analysis, varied from 60% to 95% in sensitivity analyses Descriptive reports21
Costs
 CoCM only
 Registry $800 ($0, $1600) 1st year, $550 ($0, $1100) subsequent years Prior review, 2004 and personal communication, 201722 , 23
 Care manager (per FTE per year) $90,773 ($74,888, $106,658) for RN + 30% for benefits alternatively: $72,257 ($61,262, $87,252) for Master’s-level + 30% for benefits IBM Kenexa Compensation Analysis, 201324
 Backroom space (per FTE per year) $419.9 ($77.1, $762.4) Medical Group Management Association, 20148
 Psychiatrist review (per year) 1 h/week for a panel of 120 patients, linearly scaled for smaller/larger panels, with hourly compensation calculated from annual $218,915 ($161,816, $323813) salary Prior cost-effectiveness analysis, 2001 and Medical Group Management Association, 20148
 Typical distribution of follow-up visit time required 20 (15, 35) min for alcohol, tobacco; 35 (30, 45) min for depression, anxiety × 4-12 visits/patient, plus additional 10 min for each no-show/rescheduled patient, and 1:10 probability of requiring in-person visit Prior demonstration projects, 200625 and 200726
 Additional documentation, population management, discussion, and administration tasks per patient per month, excluding call time 57 min (30, 80) for initial visits, 28 min (6, 45) for subsequent visits Prior demonstration project, 200726
 Support staff FTE costs per 1.0 behaviorist FTE $8589 ($2763, $11,452)/year Medical Group Management Association, 20148
PCBM only
 Behaviorist (per FTE per year) $91,754 ($53,196, $152,048) for PhD clinical psychologist + 30% for benefits alternatively: $76,986 ($63,513 $90,459) for LCSW, + 30% for benefits IBM Kenexa Compensation Analysis, 201324
 Clinic space (per FTE per year) $839.72 ($154.24, $1524.85) for each behaviorist Medical Group Management Association, 20148
 Typical distribution of follow-up visit time required 25 (20, 40) min for alcohol, tobacco; 45 (40, 55) min for depression, anxiety × 4-12 visits/patient, with payment for salary but no insurance payment to clinic for each no-show patient Prior review, 201227
 Additional documentation, population management, discussion, and administration tasks per patient per month 52 min (27, 73) for initial visits, 25 min (12, 41) for subsequent visits Ratio of documentation to direct clinic time from time use study, excluding administrative/other tasks exclusive to primary providers28
 Support staff FTE costs per 1.0 behaviorist FTE $8589 ($2763, $11,452)/year Medical Group Management Association, 20148
Both models
 Planning, coordination, informatics and workflow revision, and quality improvement during setup period $1411 per practice ($1287, $1554) Prior demonstration project, 2009,29 scaled per practice and updated for inflation
 Training for coordination (2 × 8 h, opportunity costs of lost visits for MD) $117/visit ($102, $122) × 20 visits/day × 2 days of training SAMHSA-HRSA Center for Integrated Health Solutions, 2013 and Medical Group Management Association, 20148 , 20
Medicare payments
 CoCM $140 for 70 min/patient for 1st month, $125 for 60 min/patient for subsequent months, $65 for additional 30 min/patient for all months (G0502 for an initial 30-min behaviorist session with 70 min per month of overall care management/staff effort, G0503 for each 26-min behaviorist follow-up with 60 min per month of care management, and G0504 for each additional 13 min of provider visit time with 30 min per month of care management) Prior review, 201730
 PCBM (non-CoCM payment rate) $48 for at least 20 min/patient/month (Medicare code G0507) Prior review, 201730

We hierarchically chose data sources to systematically prefer the highest-quality data by preferring to rely on national data for disease burden or practice-level statistics and individual participant meta-analytic data from randomized trials for treatment effect estimates when available, followed by individual randomized trials if meta-analyses were unavailable, followed by controlled demonstration projects, then uncontrolled studies, and finally descriptive studies. All cost and revenue figures are in 2016 US dollars. Persons with both Medicare and other forms of insurance were classified as Medicare. Ninety-five percent confidence intervals are in parentheses. Unhealthy alcohol use included heavy alcohol use or binge drinking. Tobacco smoking included current cigarette smoking. Major depression included a score of at least 10 on the PHQ-9 questionnaire with a 50% positive predictive value for the subsequent diagnosis.16 Generalized anxiety included feeling worried, nervous, or anxious on a daily basis while not taking medication for those feelings

Behavioral healthcare integration into primary care practices typically follows either a collaborative care model (CoCM) or a primary care behaviorist model (PCBM; Fig. 1, eTable 1).31 Both approaches incorporate brief screening for common behavioral health conditions including depression, anxiety, and alcohol or tobacco use. Behavioral health integration through a CoCM approach then involves care plan development often including pharmacotherapy from the primary care physician, with follow-up care by an RN or master’s-level behaviorist care manager by phone, who proactively uses a registry and validated scales to assess adherence, tolerability, and response.32 35 The CoCM approach requires periodic psychiatrist or psychiatric nurse practitioner/physician assistant review for payment through new Medicare codes.30 The three new Medicare Part B billing codes that apply to CoCM are G0502 for an initial 30-min behaviorist session with 70 min per month of overall care management/staff effort, G0503 for each 26-min behaviorist follow-up with 60 min per month of care management, and G0504 for each additional 13 min of provider visit time with 30 min per month of care management (Table 1).

Fig. 1.

Fig. 1

Behavioral health integration strategies

The PCBM approach, by contrast, involves in-person care by a primary care behaviorist (PhD psychologist or licensed clinical social worker) at the primary care site who provides brief, time-limited behavioral treatments.36 The PCBM approach can be paid through traditional billing mechanisms for psychologist or social worker visits, but can additionally qualify for a new general behavioral health services Medicare Part B billing code (G0507 for 15 min of behaviorist provider time with at least 20 min per month of care manager time from all staff; this code provides higher payments per period time than traditional codes such as those for routine psychotherapy, psychological testing, and health and behavioral assessment).37 , 38

Both behavioral health integration approaches require coordination between the primary care provider and behaviorist and referral of patients with severe mental health conditions or treatment-resistant disease to a psychiatrist. Medicare requires that for either model, the care manager would need to have formal education or training in behavioral health (ranging from social work to nursing to psychology), but Medicare did not specify a minimum education requirement. The care manager would also need to be available for face-to-face visits for CoCM (though such visits are not required).37 , 38

A key gap in knowledge before providing these services, however, is the economic viability of delivering such services by practices constrained by financial realities, which the new Medicare codes hope to address. To date, the clinical efficacy of CoCM has been demonstrated through numerous randomized trials, with consistent evidence of effectiveness in terms of clinical outcomes, and is typically cost-effective by common US thresholds of willingness to pay.32 , 34 , 39 In contrast, PCBM has been less rigorously tested but may provide greater accessibility and appears more commonly in current practice.40 Hence, the two approaches should not be viewed as equivalent evidence-based alternatives.33 , 38

While demonstration projects are effective in identifying logistical barriers to implementation of either approach to behavioral health integration, the American College of Physicians recently identified a key gap in knowledge that is difficult to address through such demonstrations: the cost and revenue implications of behavioral healthcare integration for primary care practices across the spectrum of typical US primary care practices.41 In this study, we estimated the cost and revenue implications to primary care practices of adopting either CoCM or PCBM approaches to integrate behavioral health services into primary care.

METHODS

Study Design

We estimated costs and revenues for both the CoCM and PCBM behavioral health integration approaches using a microsimulation model (Fig. 2). The model does not consider differences in clinical outcomes, as such outcomes are better assessed through randomized clinical trials. The focus of our approach on those outcomes is because a simulation model is more robust for predicting cost outcomes, which are more difficult to assess at a system level from randomized trials.

Fig. 2.

Fig. 2

Simulation model flow diagram

To simulate both integration approaches, we first estimated the prevalence and range of untreated or undertreated behavioral health conditions among adult patients seen at four different types of practices in the US: FQHCs, non-FQHC urban practices in high-poverty areas (≥20% of population in the ZIP code under the federal poverty threshold42), non-FQHC rural practices in high-poverty areas, and practices outside of high-poverty areas.

Next, we estimated the costs and revenues associated with behavioral health integration using either CoCM or PCBM, including transition costs of hiring staff, training them, and establishing electronic registries and in-clinic space, establishing psychiatric consultant review and referral arrangements, as well as routine screening, service delivery, and care coordination to provide behavioral health services (itemized for each model in Table 1).

Our revenue estimates incorporated three alternative scenarios: (1) if the services were delivered only to Medicare patients (using new Medicare payment mechanisms, Table 1); (2) if the services were extended from Medicare patients to all patients, but only Medicare paid the practice for these services (providing services to other patients without reimbursement, a theoretical scenario to minimize the transaction costs and difficulty of having distinct workflows for different groups of patients), and (3) if all payers (private and public, but not uninsured/self-pay patients) adopted the Medicare fee schedule for behavioral health integration.

Data Sources

Data sources and input data for the model are detailed in Table 1.

We obtained the prevalence and range of untreated or undertreated behavioral health conditions among adult patients in primary care using the most recent two waves of the National Health and Nutrition Examination Survey (NHANES, 2011-2014, N = 11,640), except for anxiety, which is not asked in NHANES and was instead estimated from the latest wave of the National Health Interview Survey (NHIS, 2015, N = 103,789).10 , 11 We specifically estimated the prevalence of untreated or undertreated excess alcohol use, tobacco smoking, depression, and anxiety (see Table 1).43 We subcategorized the prevalence estimates for each condition among participants by their type of insurance: Medicare, Medicaid, private, or self-pay/uninsured. We then obtained estimates of the proportion of patients covered by each type of insurance at each type of clinic from the National Association of Community Health Centers (for FQHCs, N = 1375 practices) and from the National Ambulatory Medical Care Survey (for all other practices; N = 1293 practices).6 , 9

We obtained estimates for transition costs including preparation, workflow revision, coordination across providers, hiring, training, information technology, space, and service delivery from the Medical Group Management Association and behavioral health integration experiences among primary care practices (Table 1).20 We also obtained estimates for Medicare reimbursement for specific CoCM and PCBM services from a recent review (also in Table 1).30

Cost and Revenue Estimates From Behavioral Health Integration

For each of the four types of practices (FQHC, non-FQHC urban high-poverty, non-FQHC rural high-poverty, and lower-poverty), we simulated a representative sample of 10,000 practices by repeatedly sampling with replacement from the normal probability distributions around the prevalence, cost, and revenue data points shown in Table 1.44 We did not simply use mean estimates of practice size, geography-specific costs/revenues, and patient characteristics among each of the four types. Rather, the microsimulation modeling technique involved simulating individual practices, such that the full national spectrum of practice characteristics was captured, and from this we present results focusing on these four types of practices. For each of the 10,000 practices of each type, we assigned a simulated population of patients based on the typical patient population served per full-time equivalent physician per year at each practice type, also shown in Table 1.7 , 9 Through Monte Carlo sampling,44 each patient was assigned an insurance type and an indicator variable (0/1) for having or not having each behavioral health condition individually or together, matching the overall distribution of insurance types by practice type, and the different prevalence of and correlation among conditions by insurance type (e.g., capturing the higher rate of conditions among Medicaid and uninsured patients and the common co-occurrence of tobacco use and depression).12

We estimated the number of true and false positives who would be detected if using the AUDIT-C tool for screening alcohol use, single-question screening for current tobacco smoking, PHQ-2 for depression, and GAD-2 for anxiety; sensitivity and specificity are shown in Table 1 from prior meta-analyses and epidemiologic surveys)12 19 , 45. We also estimated the additional prevalence of diagnoses that would be identified by the physician or BH specialist, requiring external referral after behavioral health services were initiated (such as bipolar disorder and opioid dependence), as shown in Table 1.

We added the ongoing and transition costs to perform behavioral health integration (space, IT infrastructure, and training, Table 1) to the costs of delivering behaviorist services. The transition costs included the training and time necessary for planning, coordination, informatics and workflow revision, and quality improvement, as well as the gradual process of increasing acceptance into the program from low levels to full capacity (Table 1). We computed the cost of behaviorist services by multiplying the proportion of patients accepting behavioral therapy upon screening positive; the costs of staff salary, benefits, and overhead needed in either model to provide behavioral healthcare services (increasing to the next higher 0.5 FTE to consider limitations of hiring flexibility, and computing costs during the beginning of the fiscal year to account for sunk costs); the additional support staff and associated overhead costs for supporting the behaviorist provider (sampling from the MGMA data per provider for receptionist and medical assistant time); the typical numbers of behavioral health visits per year for each type of diagnosis; time per visit to a behaviorist for each type of diagnosis for true positive screens; the time spent on documentation, population management, discussion, and administration tasks; and overhead (shown in Table 1).8 , 22 25 , 39 , 46 False-positive screens and diagnoses requiring external referral were simulated as resulting in one to three behaviorist visits for further diagnostic evaluation, referral coordination, and follow-up.

Our baseline simulation used a 50% rate of treatment acceptance among referred patients based on prior descriptive reports.20 Among the 50% who accept referral to the behaviorist, two types of time-consuming (i.e., costly) forms of no-shows for behavioral health integration were simulated: (1) failure to respond to phone calls (for CoCM) or show up for in-person visits (for PCBM) and (2) attrition from either program over time among those who show for the first phone call or appointment (costs in Table 1). Our base case analysis simulated 66% of patients accepting referral would show up for initial phone calls (for CoCM) or visits (for PCBM), and under either model 80% of patients going to the initial visit would make it to the final visit (with linear attrition between visits), based on prior descriptive studies.21 In sensitivity analyses (below), we varied these rates widely to identify critical thresholds for acceptance and show rates to ensure financial viability (i.e., net positive revenue from behavioral health services).

Primary and Secondary Outcome Metrics

We chose the primary outcome metric of net revenue change per full-time primary care (full-time equivalent, FTE) physician per year. We computed the main outcome metric as the total reimbursements for behavioral health integration services minus the total cost of service provision.

Our secondary outcome metrics included: (1) time costs for the care manager (for CoCM) or behaviorist (for PCBM) for delivering behavioral health services; (2) costs of behavioral health service integration; and (3) gross revenues for behavioral health service integration.

We computed the primary and secondary outcomes for both the first year of integration (including transition costs and linear panel growth based on typical patient volume screened per month) and per annum for the subsequent 9 years after integration (excluding transition costs and using stable panel sizes as newly screened patients entered the panel at the same rate as existing patients completed behaviorist visits). We used repeated Monte Carlo sampling with replacement to compute the mean and 95% confidence intervals around net revenue among each of the 10,000 simulated practices of each type.

Sensitivity Analyses

As detailed further in the Supplement, we varied the acceptance and show rates across the 95% confidence intervals shown in Table 1; simulated a potential productivity benefit to referring patients to a behaviorist for either CoCM or PCBM; recalculated the outcomes if the CoCM included a master’s level care coordinator rather than an RN and if the PCBM included a licensed clinical social worker rather than a PhD clinical psychologist; and varied the estimated prevalence of each behavioral health condition from half to twice its baseline value.

Simulations were performed in R (v. 3.3.2, The R Foundation for Statistical Computing, Vienna, Austria).

Role of the Funding Source

This work was supported by the Center of Innovation for Health Services Research in Primary Care (CIN 13-410) at the Durham VA Medical Center and by the Center for Primary Care, Harvard Medical School. The funders played no role in the study’s design, conduct, or reporting.

IRB Approval

Approved by the Stanford University Institution Review Board eProtocol #34359.

RESULTS

Outcomes from Behavioral Health Integration, When Treating Only Medicare Patients

When behavioral health integration services were offered only to Medicare patients in the simulated practices, the primary outcome of net revenue and the secondary outcome of gross revenue were higher when practices implemented CoCM than when practices implemented PCBM. The secondary outcomes of time and cost were lower for CoCM than for PCBM, although the confidence intervals for time and cost overlapped between the two models (Table 2). Behavioral health services for Medicare patients typically required fewer hours of service under CoCM (~1438 h/year of care manager time) than under PCBM (~1756 h/year of behaviorist time, Fig. 3) for a typical panel of ~46 Medicare behavioral health service patients per FTE PCP at any given time, with approximately three times as many patients over a given year.

Table 2.

Costs and revenues from behavioral health integration, per MD FTE per Year

Practice type Behavioral integration model Time for care manager/behaviorist (hours/years) Cost, year 1 (including training and transition), per MD FTE Cost, annual after year 1, per MD FTE Gross revenue, annual, per MD FTE Net revenue, year 1, per MD FTE Net revenue, subsequent years, per MD FTE
(A) Serving Medicare patients only
 FQHC
  CoCM 1205.6 (280.6, 3477.0) $20,639.0 ($13,633.2, $37,461.8) $16,823.1 ($9026.0, $35,781.5) $51,194.8 ($16,987.9, $100,309.9) $30,169.2 ($6481.1, $59,042.3) $33,756.6 ($10,932.4, $62,094.3)
  PCBM 1810.5 (520.0, 4148.3) $24,707.7 ($14,345.3, $45795) $21,467.3 ($9878.9, $44,703.0) $18,307.0 ($5992.4, $37,209.4) $-7068.4 ($-16,449.2, $3886.8) $-3744.5 ($-13,869.6, $5902.2)
 Urban non-FQHC, high poverty
  CoCM 1554.1 (519.8, 3359.9) $23,181.8 ($15,516.3, $36,928.2) $19,716.8 ($11,048.7, $35,046.3) $44,374.5 ($18,790.1, $78,983.5) $20,765.4 ($5393.7, $40,722.9) $24,216.9 ($9356.5, $43,281.4)
  PCBM 1691.6 (574.8, 3483.0) $23,719.4 ($14,936.5, $41,204.0) $20,349.5 ($10,457.0, $39,567.5) $17,073.1 ($7420.9, $31,030.9) $-7132.3 ($-17,371.5, $1575.6) $-3748.6 ($-14,511.0, $4754.9)
 Rural non-FQHC, high poverty
 CoCM 1494.7 (525.3, 3137.9) $22,665.4 ($15,349.9, $35,414.4) $19,107.3 ($10,863.9, $33,370.5) $48,196.2 ($21,049.4, $83,626.0) $25,701.6 ($8321.4, $46,706.8) $29,207.0 ($12,487.4, $49,597.7)
 PCBM 1786.1 (627.1, 3626.0) $24,378.1 ($15,128, $42,918.2) $21,024.4 ($10,732.8, $41,195.2) $18,228.7 ($8225.9, $31,140.6) $-6914.6 ($-16,848.8, $2174.1) $-3682.3 ($-14,791.8, $5354.7)
 Urban or rural, lower-poverty zone
  CoCM 1496.0 (551.8, 3060.5) $22,689.6 ($15,588.8, $35,191.6) $19,171.1 ($11,164.5, $33,141.6) $46,171.4 ($19,979.8, $79,356.6) $23,466.2 ($7134.1, $43,767.1) $27,009.6 ($11,201.4, $46,045.9)
  PCBM 1734.9 (620.3, 3465.9) $24,053.6 ($15,060.8, $40,475.6) $20,729.0 ($10,697.4, $38,431.0) $17,667.2 ($8002.8, $29,894.1) $-7092.3 ($-17,021.2, $1603.8) $-3646.3 ($-14,182.1, $4905.7)
(B) Serving all patients, but with payments for Medicare patients only
 FQHC
  CoCM 10,350.5 (3434.4, 22,549.8) $88,602.8 ($37,899.7, $182,663.5) $92,921.6 ($36,169.4, $198,435.2) $51,194.8 ($16,987.9, $100,309.9) $-37,407.9 ($-82,353.5, $-20,911.9) $-41,726.7 ($-98,125.3, $-19,181.5)
  PCBM 6746.2 (2006.6, 14,845.7) $58,769.4 ($23,813.1, $134,847.8) $60,212.1 ($21,108.9, $144,110.5) $18,307.0 ($5992.4, $37,209.4) $-40,462.4 ($-97,638.3, $-17,820.7) $-41,905.2 ($-106,901.1, $-15,116.5)
 Urban non-FQHC, high poverty
  CoCM 5925.0 (2226.6, 12,054.7) $56,388.7 ($28,958.9, $103,913.5) $56,879.1 ($25,930.6, $110897) $44,374.5 ($18,790.1, $78,983.5) $-12,014.2 ($-24,930.0, $-10,168.8) $-12,504.7 ($-31,913.6, $-7140.4)
  PCBM 3989.5 (1458.6, 8299.1) $39,717.4 ($19,920.8, $80,612.6) $38,520.8 ($16,410.9, $83,174.8) $17,073.1 ($7420.9, $31,030.9) $-22,644.3 ($-49,581.7, $-12,500.0) $-21,447.7 ($-52,144.0, $-8990.1)
 Rural non-FQHC, high poverty
  CoCM 7278.5 (2788.5, 14,915.4) $66,052.9 ($32,854.8, $126,604.5) $68,186.9 ($30,128.0, $135,572.5) $48,196.2 ($21,049.4, $83,626.0) $-17,856.6 ($-42,978.6, $-11,805.3) $-19,990.7 ($-51,946.5, $-9078.5)
  PCBM 4873.7 (1745.4, 10,403.0) $45,758.1 ($21,942.4, $96,298.3) $45,572.8 ($18,699.5, $101,715.4) $18,228.7 ($8225.9, $31,140.6) $-27,529.4 ($-65,157.8, $-13,716.5) $-27,344.1 ($-70,574.9, $-10,473.6)
 Urban or rural, lower-poverty zone
  CoCM 6642.6 (2588.1, 13,476.6) $61,419.9 ($31,543.3, $113,614.3) $62,759.6 ($28,769.3, $121,553.5) $46,171.4 ($19,979.8, $79,356.6) $-15,248.5 ($-34,257.7, $-11,563.5) $-16,588.2 ($-42,196.9, $-8789.6)
  PCBM 4494.0 (1585.9, 9275.6) $42,927.8 ($21,231.5, $86,843.7) $42,178.5 ($17,853.5, $90,955.3) $17,667.2 ($8002.8, $29,894.1) $-25,260.6 ($-56,949.6, $-13,228.7) $-24,511.3 ($-61,061.2, $-9850.7)
(C) Serving all patients, with payments from all public/private insurers
 FQHC
  CoCM 10,350.5 (3434.4, 22,549.8) $88,602.8 ($37,899.7, $182,663.5) $92,921.6 ($36,169.4, $198,435.2) $145,839.5 ($61,485.1, $269,646.3) $57,236.7 ($23,585.4, $86,982.8) $52,917.9 ($25,315.7, $71,211.1)
  PCBM 6746.2 (2006.6, 14,845.7) $58,769.4 ($23,813.1, $134,847.8) $60,212.1 ($21,108.9, $144,110.5) $69,809.6 ($30,139.7, $132,259.8) $11,040.2 ($6326.5, 15,753.9) $9597.4 ($9030.8, 10,164.7)
 Urban non-FQHC, high poverty
  CoCM 5925.0 (2226.6, 12,054.7) $56,388.7 ($28,958.9, $103,913.5) $56,879.1 ($25,930.6, $110,897.0) $89,692.5 ($42,010.4, $158,767.1) $33,303.8 ($13,051.5, $54,853.6) $32,813.4 ($16,079.9, $47,870.1)
  PCBM 3989.5 (1458.6, 8299.1) $39,717.4 ($19,920.8, $80,612.6) $38,520.8 ($16,410.9, $83,174.8) $42,718.6 ($19,938.1, $73,462.6) $3001.2 ($-7150.0, $13,152.2) $4197.8 ($-9712.2, $18,108.1)
 Rural non-FQHC, high poverty
  CoCM 7278.5 (2788.5, 14,915.4) $66,052.9 ($32,854.8, $126,604.5) $68,186.9 ($30,128.0, $135,572.5) $109,568.3 ($50,879.1, $192,318.7) $43,515.5 ($18,024.4, $65,714.2) $41,381.4 ($20,751.1, $56,746.2)
  PCBM 4873.7 (1745.4, 10,403) $45,758.1 ($21,942.4, $96,298.3) $45,572.8 ($18,699.5, $101,715.4) $51,923.6 ($25,475.2, $90,805.8) $6165.6 ($-5492.6, $17,825.7) $6350.8 ($-10,909.6, $23,612.7)
 Urban or rural, lower-poverty zone
  CoCM 6642.6 (2588.1, 13,476.6) $61,419.9 ($31,543.3, $113,614.3) $62,759.6 ($28,769.3, $121,553.5) $101,340.6 ($50,139.3, $177,342.6) $39,920.7 ($18,596, $63,728.3) $38,581 ($21,369.9, $55,789.1)
  PCBM 4494.0 (1585.9, 9275.6) $42,927.8 ($21,231.5, $86,843.7) $42,178.5 ($17,853.5, $90,955.3) $47,951.2 ($23,545.8, $84,212.7) $5023.4 ($-2630.9, $12,676.2) $5772.7 ($-6742.6, $18,289.3)

Computations include either a collaborative care (CoCM) or primary care behaviorist (PCBM) approach, serving (A) Medicare patients only, (B) all patients with only payment for Medicare patients, or (C) all patients with payment from all public/private insurers. Costs are in inflation-adjusted 2016 US dollars. Ninety-five percent confidence intervals are shown in parentheses, computed from 10,000 repeated Monte Carlo simulations sampling from the probability distributions of all input parameters in Table 1

Fig. 3.

Fig. 3

Time requirements for behavioral health integration in primary care practices. Plots display the distribution among 10,000 simulated practices of hours per year of patient contact required for (A) a care manager in CoCM or (B) an in-house behaviorist in PCBM, by condition, if services are only applied to Medicare patients. Alternatively, plots display the hours per year of patient contact required for (C) a care manager in CoCM and (D) an in-house behaviorist in PCBM if services are applied to all patients, regardless of insurance type. In the boxplots, the median is indicated by a bold horizontal line, interquartile range by the box, 1.5 times the interquartile range by the whiskers, and outliers by dots. FQHC, federally qualified health center; hi pov, high poverty area; lo pov, lower poverty area

The CoCM had lower space costs (due to telephone rather than in-person visits) and behaviorist costs (due to less time spent per visit), but higher costs from the registry as well as routine payment to a consulting psychiatrist. Behavioral health services were, notably, reimbursed at a disproportionately higher rate by Medicare under CoCM than under PCBM (Table 1), producing net revenue gains under CoCM and losses under PCBM (averaging $25,026 per MD FTE in year 1 and $28,548 in subsequent years under CoCM, versus -$7052 and -$3706, respectively, under PCBM across all practice types, Table 2). Offering behavioral health services to Medicare patients alone through CoCM produced net revenue gains for all of the simulated clinic types, in year 1 and in subsequent years, while 88% of simulated practices experienced net revenue losses through PCBM in year 1 and 73% in subsequent years.

Outcomes from Behavioral Health Integration, When Treating All Patients

When behavioral health integration services were offered to all patients in the simulated practices, but were only reimbursed by Medicare, none of the practices adopting either the CoCM or PCBM approach maintained positive net revenues in either the first or subsequent years (Table 2). Behavioral health integration services produced net revenues typically $33,890 lower when practices offered the services to all patients but were only reimbursed by Medicare, as opposed to when practices offered the services only to Medicare patients. Behavioral health integration services offered to all patients increased the time requirements for service by a factor of four (requiring, on average, ~7549 h/year from the care manager in CoCM versus ~5026 h/year from the behaviorist in PCBM per FTE PCP) with a typical panel of ~175 behavioral health service patients at any given time and approximately three times that amount over the course of a year. While more time was spent on depression/anxiety treatment than on tobacco/alcohol treatment when treating only Medicare patients (Fig. 3a–b), expansion of behavioral health services to all patients disproportionately increased the time spent on tobacco/alcohol treatment given the higher burden of tobacco/alcohol abuse among non-Medicare patients (Fig. 3c–d).

When behavioral health integration services were offered to all patients in the simulated practices, but were reimbursed at the same rate by all public and private insurers (remaining un-reimbursed for uninsured patients), net revenues averaged $43,495 per MD FTE in year 1 and $41,423 in subsequent years under CoCM versus $6308 and $6480, respectively, under PCBM, across all practice types (the net revenue changed to positive overall because of a higher demand helped to compensate for the costs of the behaviorist; Table 2). Behavioral health service under CoCM produced net revenue gains for all of the simulated clinic types, in year 1 and in subsequent years, such that no simulated practices experienced net revenue losses from offering behavioral health services to all patients if both public and private insurers matched current Medicare payment rates. Behavioral health service under PCBM, however, produced net revenue losses for 19% of the simulated clinic types in year 1 and 23% in subsequent years, even if both public and private insurers matched current Medicare payment rates.

Sensitivity Analyses

As detailed in the Supplement, when we varied our rates of treatment acceptance, show rate, and completion rate to their lower-bound ranges shown in Table 1 (15% acceptance, 50% initial show-up, and 60% completion), net revenues declined from the base case such that net revenues were often negative for most practices, but still higher for practices adopting the CoCM approach than the PCBM approach (eTable 2). Conversely, when we varied our rates of treatment acceptance, show rate, and completion rate to their higher-bound ranges shown in Table 1 (85% acceptance, 90% initial show-up, and 95% completion), net revenues increased from the base case such that net revenues were positive for all practices across all years for CoCM but some practices continued to lose net revenue for PCBM (eTable 3). We additionally found that a small productivity benefit of 5-min reduced visit length, if re-routed into additional visit time, would produce at least $3747 (95% CI: $1569, $6441) in additional revenue per full-time physician per year. Hiring lower-cost behaviorists would be expected to increase net revenues by at least $14,768 (95% CI: -$10,317, $61,589). However, varying the prevalence of each behavioral health condition did not significantly change net revenues, as the cost-to-revenue ratio per patient was not substantially changed (eTables 4 and 5).

DISCUSSION

We estimated that primary care practices adopting the CoCM behavioral health integration approach would tend to gain more net revenues than under the PCBM model using current rates of Medicare reimbursement, despite the costs of establishing a registry and consulting a psychiatrist to perform CoCM services. We additionally estimated that practices adopting the CoCM approach would likely maintain net positive revenues if they expand behavioral health services to all patients, but only if private insurers offered payments for such services. This result was driven by lower time costs of telephone-based follow-up and the higher reimbursements from Medicare for CoCM. CoCM would also be more financially viable than PCBM if all insurers paid Medicare rates. Notably, private insurer payments exceed Medicare payments for the same service, suggesting that participation from private insurers may well bring additional revenue to the practice.

Because financial barriers are considered a key obstacle to implementing behavioral health integration, our results imply that revenues of practices would be expected to vary widely by the type of behavioral health service integration strategy and payer support across their market, but less widely among types of practices or by prevalence of conditions, because of consistent cost-benefit ratios for the key personnel expenditures from integration. We also discovered that variations in patient demographics and practice type were less important than the prevalence of behavioral health conditions and the rate of no-shows among patients referred to behavioral health services in determining financial outcomes for practices. Yet even at low levels of behavioral service acceptance, show-up rates, and follow-up rates, the CoCM approach would remain financially viable.

Our results are necessarily limited by assumptions inherent to simulation modeling, which cannot incorporate all possible outcomes or complex realities. In our simulation, we did not account for additional grant funding that FQHCs receive, which is highly varied across sites and years, but has been observed to increase the probability of FQHCs providing on-site behavioral health services in previous years.46 Furthermore, we lacked sufficient rigorous data to expand our model to additional conditions, such as weight control, sleep disturbance, or narcotic drug use, for which the approaches simulated here could be applied. In addition, we limited our analysis to financial implications because economic outcomes are best assessed through simulations comprehensively incorporating costs and revenues; by contrast, simulations are less suited to identify other important factors for practices considering behavioral health integration, such as patient outcomes, or physician satisfaction. For Medicare Advantage beneficiaries, there is anticipated to be plan-by-plan variation in whether beneficiaries will be eligible for the new behavioral health integration codes—a source of variation we are not yet able to account for given the lack of systematic data on these variations. In the absence of robust data about how much of PCP panels are replaced by new patients in need of new behavioral health services, we did not make any assumptions about the change in healthiness of the population over time. The likely higher benefits of CoCM versus PCBM would go further toward strengthening our conclusion of CoCM’s greater financial viability, making our results from this model conservative. Finally, the proprietary nature of the MGMA data used here is a limitation for broad usage; the potential public use availability of other practice cost registries may eventually lead to the wider availability of financial data for practice planning.

Overall, our findings suggest that new Medicare payments offer financial viability for most primary care practices to integrate behavioral health services if implemented through a collaborative care approach. Given that most patients with behavioral health needs visit a primary care provider as their only source of care, and are more likely to turn to their primary care physician than a mental health professional for help,47 , 48 behavioral health integration may offer a critical opportunity to address major mental health and substance abuse issues in the US adult population.

Electronic supplementary material

ESM 1 (66.7KB, docx)

(DOCX 66 kb)

Acknowledgements

Contributors: N/A.

Funders: This work was supported by the Center of Innovation for Health Services Research in Primary Care (CIN 13-410) at the Durham VA Medical Center and by the Center for Primary Care, Harvard Medical School.

Prior presentations: None.

Compliance with Ethical Standards

Conflict of Interest

Asaf Bitton and John Williams serve part time as senior advisors to the Comprehensive Primary Care Plus initiative at the Center for Medicare and Medicaid Innovation (CMMI). The ideas expressed herein are solely the authors’ and do not represent any official position of the Center for Medicare and Medicaid Innovation. All other authors declare no conflicts of interest.

References

  • 1.Substance Abuse and Mental Health Administration . Substance Use and Mental Health Estimates from the 2013 National Survey on Drug Use and Health. Washington DC: Department of Health and Human Services; 2014. [Google Scholar]
  • 2.Davis M, Balasubramanian BA, Waller E, Miller BF, Green LA, Cohen DJ. Integrating behavioral and physical health care in the real world: early lessons from advancing care together. J Am Board Fam Med JABFM. 2013;26(5):588–602. doi: 10.3122/jabfm.2013.05.130028. [DOI] [PubMed] [Google Scholar]
  • 3.Institute for Healthcare Improvement . Integrating Behavioral Health and Primary Care: IHI 90-Day R&D Project Final Summary Report. Cambridge: IHI; 2014. [Google Scholar]
  • 4.American Psychiatry Association, Academy of Psychosomatic Medicine . Dissemination of Integrated Care within Adult Primary Care Settings. Arlington: APA; 2016. [Google Scholar]
  • 5.Krist AH, Phillips SM, Sabo RT, et al. Adoption, reach, implementation, and maintenance of a behavioral and mental health assessment in primary care. Ann Fam Med. 2014;12(6):525–533. doi: 10.1370/afm.1710. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.National Association for Community Health Centers . United States Health Center Fact Sheet. Arlngton: NACHC; 2014. [Google Scholar]
  • 7.National Association for Community Health Centers . Key Health Center Data by State. Bethesda: NACHC; 2015. [Google Scholar]
  • 8.Medical Group Management Association . DataDive. Washington DC: MGMA; 2014. [Google Scholar]
  • 9.National Center for Health Statistics . National Ambulatory Medical Care Survey. Atlanta: US Centers for Disease Control and Prevention; 2015. [Google Scholar]
  • 10.National Center for Health Statistics . National Health and Nutrition Examination Survey, 2011-2014. Atlanta: US Centers for Disease Control and Prevention; 2015. [Google Scholar]
  • 11.National Center for Health Statistics . National Health Interview Survey, 2015. Atlanta: US Centers for Disease Control and Prevention; 2016. [Google Scholar]
  • 12.Kessler R. National Comorbidity Survey: Reinterview (NCS-2), 2001-2002. Boston: Harvard Medical School; 2015. [Google Scholar]
  • 13.Whiteford HA, Degenhardt L, Rehm J, et al. Global burden of disease attributable to mental and substance use disorders: findings from the Global Burden of Disease Study 2010. The Lancet. 2013;382(9904):1575–1586. doi: 10.1016/S0140-6736(13)61611-6. [DOI] [PubMed] [Google Scholar]
  • 14.Han B, Compton WM, Jones CM, Cai R. Nonmedical prescription opioid use and use disorders among adults aged 18 through 64 years in the United States, 2003-2013. JAMA. 2015;314(14):1468–1478. doi: 10.1001/jama.2015.11859. [DOI] [PubMed] [Google Scholar]
  • 15.Cerimele JM, Chwastiak LA, Dodson S, Katon WJ. The prevalence of bipolar disorder in general primary care samples: a systematic review. Gen Hosp Psychiatry. 2014;36(1):19–25. doi: 10.1016/j.genhosppsych.2013.09.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Kroenke K, Spitzer RL, Williams JBW. The PHQ-9. J Gen Intern Med. 2001;16(9):606–613. doi: 10.1046/j.1525-1497.2001.016009606.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Manea L, Gilbody S, Hewitt C, et al. Identifying depression with the PHQ-2: A diagnostic meta-analysis. J Affect Disord. 2016;203:382–395. doi: 10.1016/j.jad.2016.06.003. [DOI] [PubMed] [Google Scholar]
  • 18.Kriston L. Meta-analysis: are 3 questions enough to detect unhealthy alcohol use? Ann Intern Med. 2008;149(12):879. doi: 10.7326/0003-4819-149-12-200812160-00007. [DOI] [PubMed] [Google Scholar]
  • 19.Patrick DL, Cheadle A, Thompson DC, Diehr P, Koepsell T, Kinne S. The validity of self-reported smoking: a review and meta-analysis. Am J Public Health. 1994;84(7):1086–1093. doi: 10.2105/AJPH.84.7.1086. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.SAMHSA-HRSA Center for Integrated Health Solutions . The Business Case for Behavioral Health Care. Washington DC: US Department of Health and Human Services; 2013. [Google Scholar]
  • 21.Truven Health Analytics. Financing of Behavioral Health Services within Federally Qualified Health Centers. Washington DC: Truven Health Analytics; 2012. http://www.integration.samhsa.gov/financing/Financing_BH_Services_at_FQHCs_Final_7_23-12.pdf. Last accessed 16 August 2017.
  • 22.First Consulting Group . Using Computerized Registries in Chronic Disease Care. Sacramento: California Healthcare Foundation; 2004. [Google Scholar]
  • 23.Unutzer J. Care management software/registry. 2017. [Google Scholar]
  • 24.IBM Kenexa . CompAnalyst Market Data. Armonk: IBM; 2013. [Google Scholar]
  • 25.Simon GE, Katon WJ, VonKorff M, et al. Cost-effectiveness of a collaborative care program for primary care patients with persistent depression. Am J Psychiatry. 2001;158(10):1638–1644. doi: 10.1176/appi.ajp.158.10.1638. [DOI] [PubMed] [Google Scholar]
  • 26.Wang PS, Lane M, Olfson M, Pincus HA, Wells KB, Kessler RC. Twelve-month use of mental health services in the united states: results from the national comorbidity survey replication. Arch Gen Psychiatry. 2005;62(6):629–640. doi: 10.1001/archpsyc.62.6.629. [DOI] [PubMed] [Google Scholar]
  • 27.Liu C-F, Fortney J, Vivell S, et al. Time allocation and caseload capacity in telephone depression care management. Am J Manag Care. 2007;13(12):652–660. [PubMed] [Google Scholar]
  • 28.The National Center on Addiction and Substance Abuse . Addiction Medicine: Closing the Gap between Science and Practice. New York: Columbia University; 2012. [Google Scholar]
  • 29.Liu C-F, Rubenstein LV, Kirchner JE, et al. Organizational cost of quality improvement for depression care. Health Serv Res. 2009;44(1):225–244. doi: 10.1111/j.1475-6773.2008.00911.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Press MJ, Howe R, Schoenbaum M, et al. Medicare payment for behavioral health integration. N Engl J Med. 2017;376(5):405–407. doi: 10.1056/NEJMp1614134. [DOI] [PubMed] [Google Scholar]
  • 31.Klein S, Hostetter M. In Focus: Integrating Behavioral Health and Primary Care. Qual Matters. 2014. http://www.commonwealthfund.org/publications/newsletters/quality-matters/2014/august-september/in-focus. Last accessed 19 April 2017.
  • 32.Woltmann E, Grogan-Kaylor A, Perron B, Georges H, Kilbourne AM, Bauer MS. Comparative effectiveness of collaborative chronic care models for mental health conditions across primary, specialty, and behavioral health care settings: systematic review and meta-analysis. Am J Psychiatry. 2012;169(8):790–804. doi: 10.1176/appi.ajp.2012.11111616. [DOI] [PubMed] [Google Scholar]
  • 33.Archer J, Bower P, Gilbody S, et al. Collaborative care for depression and anxiety problems. Cochrane Database Syst Rev. 2012;10:CD006525. doi: 10.1002/14651858.CD006525.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Katon WJ, Lin EHB, Von Korff M, et al. Collaborative care for patients with depression and chronic illnesses. N Engl J Med. 2010;363(27):2611–2620. doi: 10.1056/NEJMoa1003955. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Robinson PJ, Strosahl KD. Behavioral health consultation and primary care: lessons learned. J Clin Psychol Med Settings. 2009;16(1):58–71. doi: 10.1007/s10880-009-9145-z. [DOI] [PubMed] [Google Scholar]
  • 36.Szymanski BR, Bohnert KM, Zivin K, McCarthy JF. Integrated care: treatment initiation following positive depression screens. J Gen Intern Med. 2013;28(3):346–352. doi: 10.1007/s11606-012-2218-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Centers for Medicare and Medicaid Services . Fact Sheet: Behavioral Health Integration Services. Baltimore: CMS; 2017. [Google Scholar]
  • 38.Centers for Medicare and Medicaid Services . Frequently Asked Questions about Billing Medicare. Baltimore: CMS; 2017. [Google Scholar]
  • 39.Grochtdreis T, Brettschneider C, Wegener A, et al. Cost-effectiveness of collaborative care for the treatment of depressive disorders in primary care: a systematic review. PLoS ONE. 2015;10(5). doi:10.1371/journal.pone.0123078. [DOI] [PMC free article] [PubMed]
  • 40.Collins C, Hewson D, Munger R, Wade T. Models of Behavioral Health Integration. New York: Milbank Memorial Fund; 2010. [Google Scholar]
  • 41.Crowley RA, Kirschner N. The integration of care for mental health, substance abuse, and other behavioral health conditions into primary care: executive summary of an American college of physicians position paper. Ann Intern Med. 2015;163(4):298. doi: 10.7326/M15-0510. [DOI] [PubMed] [Google Scholar]
  • 42.Economics and Statistics Administration, US Department of Commerce . Poverty Areas. Washington DC: US Census Bureau; 1995. [Google Scholar]
  • 43.American Psychiatry Association . Diagnostic and Statistical Manual of Mental Disorders (DSM-5) Washington DC: American Psychiatric Publishing; 2013. [Google Scholar]
  • 44.Robert CP, Casella G. Introducing Monte Carlo Methods with R. 2010. New York: Springer Verlag; 2009. [Google Scholar]
  • 45.Plummer F, Manea L, Trepel D, McMillan D. Screening for anxiety disorders with the GAD-7 and GAD-2: a systematic review and diagnostic metaanalysis. Gen Hosp Psychiatry. 2016;39:24–31. doi: 10.1016/j.genhosppsych.2015.11.005. [DOI] [PubMed] [Google Scholar]
  • 46.Simon GE, Ludman EJ, Bauer MS, Unützer J, Operskalski B. Long-term effectiveness and cost of a systematic care program for bipolar disorder. Arch Gen Psychiatry. 2006;63(5):500–508. doi: 10.1001/archpsyc.63.5.500. [DOI] [PubMed] [Google Scholar]
  • 47.Molfenter T. Reducing appointment no-shows: going from theory to practice. Subst Use Misuse. 2013;48(9):743–749. doi: 10.3109/10826084.2013.787098. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Institute of Medicine (US) Committee on Crossing the Quality Chasm: Adaptation to Mental Health and Addictive Disorders . Improving the Quality of Health Care for Mental and Substance-Use Conditions: Quality Chasm Series. Washington (DC): National Academies Press (US); 2006. [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

ESM 1 (66.7KB, docx)

(DOCX 66 kb)


Articles from Journal of General Internal Medicine are provided here courtesy of Society of General Internal Medicine

RESOURCES