Skip to main content
Journal of General Internal Medicine logoLink to Journal of General Internal Medicine
. 2012 Oct 5;28(3):353–362. doi: 10.1007/s11606-012-2217-z

Determinants of Readiness for Primary Care-Mental Health Integration (PC-MHI) in the VA Health Care System

Evelyn T Chang 1,, Danielle E Rose 2, Elizabeth M Yano 2,3, Kenneth B Wells 3,4,5, Maureen E Metzger 6, Edward P Post 6,7, Martin L Lee 2,3, Lisa V Rubenstein 2,5
PMCID: PMC3579970  PMID: 23054917

ABSTRACT

BACKGROUND

Depression management can be challenging for primary care (PC) settings. While several evidence-based models exist for depression care, little is known about the relationships between PC practice characteristics, model characteristics, and the practice’s choices regarding model adoption.

OBJECTIVE

We examined three Veterans Affairs (VA)-endorsed depression care models and tested the relationships between theoretically-anchored measures of organizational readiness and implementation of the models in VA PC clinics.

DESIGN

1) Qualitative assessment of the three VA-endorsed depression care models, 2) Cross-sectional survey of leaders from 225 VA medium-to-large PC practices, both in 2007.

MAIN MEASURES

We assessed PC readiness factors related to resource adequacy, motivation for change, staff attributes, and organizational climate. As outcomes, we measured implementation of one of the VA-endorsed models: collocation, Translating Initiatives in Depression into Effective Solutions (TIDES), and Behavioral Health Lab (BHL). We performed bivariate and, when possible, multivariate analyses of readiness factors for each model.

KEY RESULTS

Collocation is a relatively simple arrangement with a mental health specialist physically located in PC. TIDES and BHL are more complex; they use standardized assessments and care management based on evidence-based collaborative care principles, but with different organizational requirements. By 2007, 107 (47.5 %) clinics had implemented collocation, 39 (17.3 %) TIDES, and 17 (7.6 %) BHL. Having established quality improvement processes (OR 2.30, [1.36, 3.87], p = 0.002) or a depression clinician champion (OR 2.36, [1.14, 4.88], p = 0.02) was associated with collocation. Being located in a VA regional network that endorsed TIDES (OR 8.42, [3.69, 19.26], p < 0.001) was associated with TIDES implementation. The presence of psychologists or psychiatrists on PC staff, greater financial sufficiency, or greater spatial sufficiency was associated with BHL implementation.

CONCLUSIONS

Both readiness factors and characteristics of depression care models influence model adoption. Greater model simplicity may make collocation attractive within local quality improvement efforts. Dissemination through regional networks may be effective for more complex models such as TIDES.

Electronic supplementary material

The online version of this article (doi:10.1007/s11606-012-2217-z) contains supplementary material, which is available to authorized users.

KEY WORDS: primary care, mental health, depression, collaborative care, implementation, readiness

INTRODUCTION

Depression affects 5–10 % of individuals1,2 and is projected to be the second leading cause of disability worldwide by 2020.3 Depression is also the largest single contributor to impaired mental health (MH) in primary care (PC) populations. Most patients with depression are cared for in PC rather than MH specialty settings, yet recognition and quality of care for depressive disorders in PC are lower than for other major chronic illnesses.46 Methods for improving depression care in PC have been extensively studied and shown to be effective714 and cost-effective.1519 Uptake of evidence-based models for improving depression care for PC populations, however, has been slow.2024 This paper aims to improve understanding of the uptake of depression care improvement models by investigating the determinants of adoption of three alternative, Veterans Affairs (VA) system-endorsed approaches to improving routine depression care.

Theories of innovation dissemination postulate that innovation characteristics affect adoption rates.25,26 However, the process of adopting an approach to improvement is affected not only by the characteristics of an innovation, but by the fit between the innovation and organizational characteristics. Accordingly, theories of organizational readiness suggest that characteristics of the adopting organization will also predict whether and when organizations will adopt a given innovation.27 Organizational readiness for change has been defined as “the extent to which organizational members are psychologically and behaviorally prepared to implement organizational change.”27 Within this framework, researchers have identified organizational characteristics that favor adoption of MH innovations.25,2729

This study takes advantage of the natural experiment created by VA’s endorsement of three substantively different depression care improvement models to meet system primary care-mental health integration (PC-MHI) goals. These models served as alternatives for PC practices to choose for screening and managing common MH disorders in the PC setting. We assess the proportion of practices that had adopted each of the three depression care models after official endorsement, and relate practice choice among the endorsed models to local readiness characteristics. These VA-endorsed models include collocation of mental health specialists (MHS) in PC settings, the Translating Initiatives in Depression (TIDES) model, and the Behavioral Health Laboratory (BHL) model. By the time these models were formally endorsed and incentivized in 2006, they had already been in development and/or spread to additional practices within the VA for over a decade. Adoption of at least one of these models was later mandated (in 2008, after the period of this study) through VA’s Uniform Services Package for Mental Health.30

The three models have varying levels of evidence support and requirements for organizational redesign. Collaborative care models, such as TIDES31,32 or BHL,3335 are highly evidence-based in terms of effectiveness. However, they require significant system redesign and training,3638 with or without the addition of collocation. Collocation alone, on the other hand, requires little system redesign. It has also been studied less often and, when studied, has not been found to improve patient outcomes over standard MH specialty care.3942

In this study, we used a qualitative assessment of the VA-endorsed models and a nationwide cross-sectional survey to measure the organizational readiness of 225 VA PC practices in relationship to which depression care improvement model the practices chose to adopt. We address the following questions:

  1. How do the three depression care improvement models differ by the organizational demands placed on PC practices for system redesign?

  2. One year after VA endorsement, what was the prevalence of the three depression care models?

  3. Do local organizational readiness factors predict PC practice choice regarding which model to adopt? If so, which readiness factors are associated with the decision to implement each model?

METHODS

Overview and Theoretical Models

To address organizational readiness factors in relationship to depression care improvement model adoption, we adapted the Organizational Readiness for Change (ORC) model29,43 as the framework for this study (Figure 1). Lehman, et al. developed the ORC to study how factors influence the adoption of innovation25,26,44 on an organizational level as used here, rather than on an individual or group level. We assessed which readiness characteristics predicted adoption of any model and of each model separately.

Figure 1.

Figure 1.

Conceptual model of independent variables derived from organizational readiness for change framework with hypothesized linkages to implementation of VA-endorsed depression care improvement models. + indicates positive relationship; - indicates negative relationship. † TIDES=Translating Initiatives for Depression into Effective Solutions.

To qualitatively explore depression care improvement model characteristics in relationship to our findings on model adoption, we used Rogers’ Diffusion of Innovation Theory.44 We assessed individual model characteristics for each VA-endorsed model, specifically focusing on model complexity in terms of the demands placed on practices for system redesign.

Study Sample

We used data from the 2007 VA Clinical Practice Organization Survey (CPOS) Primary Care Directors Module, a nationwide survey of PC directors on organization-level characteristics for their respective practices,45 to assess PC practice readiness.

The sample included all VA PC practices with ≥ 4,000 unique patients and ≥ 20,000 outpatient visits. The resulting survey sample comprised 250 VA PC practices, including 152 based in VA medical centers (VAMC) and 97 that were community-based outpatient clinics (CBOC). Survey packets were mailed as well as e-mailed; an electronic version was available for online completion. We achieved a 90 % response rate for a total sample of 225 out of 250 PC practices.

Measures

For dependent variables, we assessed which, if any, of the three endorsed models had been chosen for implementation by each PC practice based on a CPOS survey question asking about the degree of implementation of collocated care, TIDES, or BHL. We dichotomized responses (“fully” or “partially” implemented versus “planned but not yet implemented” or “not implemented”).

Some PC practices reported the implementation of more than one model. We assigned practices to a single model using a hierarchy such that they were first assigned to BHL; then if not BHL, to TIDES; then if not TIDES or BHL, to collocation. This reflected the specificity of the BHL model informatics, such that practices reporting BHL were likely to be using BHL software, even if they used additional TIDES training. Collocation is non-specific regarding program components and could co-exist with either TIDES or BHL programs.

For independent variables, we assessed practice readiness characteristics29,43 assessed in the CPOS 2007 survey. We developed scales using exploratory factor analysis guided by theory.46,47 Cronbach’s alpha coefficients48 were > 0.65 for each scale (Online Appendix Table 1).

We studied four components of adequacy of resources: financial insufficiency, clinical provider insufficiency, space sufficiency, and sufficiency for information technology (IT) support.

We studied two components of motivation for change: retreat for depression training and regional endorsement. The latter was based on whether or not the practice was located in one of three regional health system networks nationwide that had endorsed TIDES (no other models were regionally endorsed in 2007).

We studied two components of staff attributes: presence of a clinician champion for depression treatment (proxy for staff influence) and presence of psychiatrist, psychologist, and social worker on primary care staff (proxy for staff efficacy).

We studied seven components of organizational climate: orientation towards quality improvement (QI), competing demands and stress, communication and cooperation, teamwork, internal authority over the PC clinic, external authority over relationships with specialists, and resistance.

As descriptive characteristics, we assessed practice demographic characteristics. These included practice type as hospital-based vs. community-based (VAMC vs. CBOC) and practice size based on patient utilization data for fiscal year (FY) 2007 from the VA National Patient Care Database (Austin data); practice location in urban/rural settings based on the Area Resource File,49 and academic affiliation based on the VA Office of Academic Affiliation website.50 We used practice size as a covariate in regression analysis.

For the exploratory qualitative analysis of the three improvement models as they existed in 2007, we identified key model characteristics based on author (EC) review of the literature, review of VA intranet descriptions, and interviews with model developers.

STATISTICAL ANALYSIS

To explore associations between practice demographic characteristics and choice of depression care improvement model, we tested for significant differences between models using Pearson Chi-squared tests (and Fisher’s Exact tests51 for factors with cell counts < 5). To assess associations between readiness characteristics and each of the three improvement models, and between readiness and adopting none of the models, we used bivariate regression analysis. These analyses identified statistically important (p < 0.10) predictors of model choice. To further assess the predictors of model choice, we constructed separate multivariate logistic regression models, also controlling for practice size. BHL had an insufficient sample size for multivariable regression. All dependent variables used in predictive analyses had < 5 % missing data, obviating the need for imputation. All statistical analysis was performed with STATA 11/IC.

QUALITATIVE ANALYSIS

We used simplified cross-case analysis to generate hypotheses about the relationships between the characteristics of the three improvement models and the readiness characteristics that predicted them. We focused on key model characteristics, including goals, history, complexity, and elements of the Chronic Care Model used to improve depression care in the PC setting (i.e., self-management, clinical information systems, delivery system design).52,53

RESULTS

Table 1 shows the characteristics of the three VA-endorsed depression care improvement programs. The alternative collaborative care models, TIDES and BHL, were each associated with developed programs that included distinct VA-tested implementation methods, tools and training.31,32,34,39 These programs were open to any VA PC practice through national trainings, ongoing program support staff, and sharing of technologies. These two programs, however, had different histories and goals; TIDES was initially developed and tested through regional networks, whereas BHL was developed at a single site. As for collocated care, while the 2006 VA endorsement referenced a more complex collocated collaborative care model, no implementation program or tools were broadly available for the complex approach.

Table 1.

Qualitative Differences Between the BHL=Behavioral Health Laboratory, TIDES=Translating Initiatives for Depression into Effective Solutions, and Collocated Care Models

Depression Care Improvement Model Dimension Description of the Dimension by Each of the Three Depression Care Improvement Models
Goals • The TIDES initiative aimed at increasing detection and treatment completion among depression screen positive veterans in primary care through support from a depression care manager.
• BHL focused on serving as an organized mental health assessment and triage laboratory as a gateway to diverse services, including collaborative telephone care management and mental health specialty care.
• Collocated care focused on increasing convenient access to mental health specialty services for patients with mental illness in primary care. It served as a change agent to begin redirection of primary care processes around mental health care.
Program Development History • Between 2001 and 2005, TIDES engaged three VA multistate regional networks and six of their primary care practices in strategic plan-do-study-act cycles to develop specifications and training for the TIDES program in an evidence-based quality improvement approach. 31,32,61 In about 2004, the model began to spread spontaneously.
• BHL was developed in the late 1990s as a disease management program at a single site and was tested in randomized trials in a single site in one VA regional network.3335,39 BHL focused on laboratory-specific software that enabled sophisticated algorithm-based telephone assessment by a health technician, with referral of appropriate patients to a care manager. In about 2005, the model began to spread spontaneously.
• Collocated care began to spread beginning in the early 1990’s, but did not imply a specific program. The White River Junction collocated collaborative care model54,55 was developed over more than a decade during the late 1990’s and early 2000s within a single mental health/primary care practice setting in Vermont, but was not designed for spread.
Specific Focus on Depression Versus General Focus on Primary Care Mental Health Integration • TIDES focused specifically on enhancing outcomes for the population of primary care veterans screening positive for depression, using a stepped care model.10
• BHL and collocation focused on patients screening positive for depression or other common primary care disorders, including substance abuse, post-traumatic stress disorder, or anxiety.
Assessment and Triage of Referred Patients • TIDES and BHL used standardized assessment instruments and triage protocols for all patients covering major depression, dysthymia, suicidality, post-traumatic stress disorder, substance abuse, and anxiety. These models also assessed prior mental health history and knowledge. TIDES additionally assessed patient treatment preferences, education, and family involvement.
• BHL used health technicians or nurses or pharmacists for initial assessment; TIDES used Registered Nurse (RN) or Master of Social Work (MSW) care managers.
• Collocated care uses mental health specialist expertise to assess and triage patients.
Follow-Up of Depressed Patients • TIDES and BHL care managers followed patients using a standardized protocol and outcome assessment instruments. TIDES particularly emphasized support for management of depressed patients in primary care with referral to mental health specialty for patients with specific indications.
• Collocated care uses mental health specialist expertise to guide follow-up.
Patient Self-Management Support • TIDES and BHL guided patient self-management based on specific training and tools.
• Collocated care used mental health specialist expertise.
Telephone versus In-Person Support • TIDES and BHL were primarily carried out by telephone.
• Collocated care was primarily carried out through in-person visits.
Mental Health Specialty Supervision • TIDES and BHL care managers required mental health specialty supervision of care managers at least weekly. BHL integrated psychiatrists into ongoing supervision of laboratory activities on a more continuous basis.
Availability of Program Tools and Training • Both TIDES and BHL, but not collocated care, were associated with accessible tools, training, and program support for practices choosing to implement them.
Information Technology • TIDES used standard VA electronic medical record tools that were specifically redesigned to support depression care managers and their links to primary care clinicians. BHL used purpose-built software approved for use in VA but not linked to or supported by the VA information technology.
Model Complexity and Its Demand on PC Practices Less complex: Collocated care required space in primary care and availability of a mental health specialist, but no other major changes in how care was delivered.
Complex: BHL required organization of a laboratory including installation of software and creation of an organizational unit including mental health specialist(s), health technicians, and/or nurses or pharmacists. The software was maintained centrally by the original BHL developers and required little local information technology support.
Complex: TIDES required substantial engagement of local primary care clinicians and mental health specialists in supporting the care management approach. TIDES also required involvement of VA information technologists because it relied on adaptations based on standard VA electronic medical record tools that required local installation and updating.

As shown in Figure 2, in 2007, 124 of all 225 (55.1 %) PC practices had voluntarily implemented a VA-endorsed PC-MHI model. Of the 124 practices, 107 (86.3 %) had implemented collocated care, 39 (31.5 %) had implemented TIDES, and 17 (13.7 %) had implemented BHL. Only 17 of the 124 (14 %) implemented TIDES or BHL without also implementing collocation (Figure 2).

Figure 2.

Figure 2.

Number of primary care practices indicating implementation of each model. Numbers represent primary care practices adopting the model or combination of models. Models depicted are Collocated Care, TIDES=Translating Initiatives for Depression into Effective Solutions and BHL=Behavioral Health Laboratory. We assigned each primary care practice to only one of the three models based on the following rules: 1) Clinics that had implemented both TIDES and collocation were attributed to TIDES, due to the specificity of the TIDES approach; 2) Clinics that had implemented both collocation and BHL were attributed to BHL, due to the specificity of the BHL approach; 3) Clinics that had implemented both TIDES and BHL were attributed to BHL, due to the specificity of the BHL software program and the small sample size of BHL.

Although almost half (44.9 %) of the practices had not yet implemented any depression care improvement model, the majority were planning to do so, particularly collocated care. Of the 116 practices (missing = 2) that had not yet chosen to implement collocation, 63 (54.3 %) planned to implement it. Among the 171 practices (missing = 15) that had not yet implemented TIDES, 20 (11.7 %) planned to implement it. Among the 195 practices (missing = 13) that had not yet implemented BHL, 23 (11.8 %) planned to implement it.

Table 2 shows the PC practice organizational demographics in relationship to implementation of collocated care (Co), TIDES (Ti), BHL (BH), or no model (NM). Regarding MHS staffing on the PC staff, practices with a PC-based psychologist were more likely to have implemented an improvement model (41 % Co, 31 % Ti, 53 % BH, 20 % NM, p < 0.006). Practices with a psychiatrist on the PC staff were more likely to have implemented BHL (29 % Co, 29 % TI, 71 % BH, p < 0.001). There were no significant differences for social workers. The only other significant difference was that practices located in VA regional networks that had endorsed TIDES were more likely to have implemented TIDES (15 % Co, 49 % TI, 12 % BH, 6 % NM, p < 0.001). Descriptively, collocation had been in place in practices longer on average (6.2 years) compared to TIDES (2.7 years) or BHL (1.2 years).

Table 2.

Demographic Characteristics of Primary Care Practices in Relationship to Implementation of Each Alternative Depression Care Improvement Model*, n = 225

Primary Care Practice Setting Characteristics All VA Primary Care Practices (%) VA clinics with Collocated Care (%) VA clinics with TIDES† (%) VA clinics with BHL‡ (%) VA clinics with no Depression Care Improvement Model§ (%) Significance of Pearson Chi-Squared Test or Fisher’s Exact test║
(n = 225) (n = 72) (n = 35) (n = 17) (n = 101) (p)
Primary Care Practice Size
• Small (4,000–20,000 patients) 102 (45.3 %) 37 (51.4 %) 12 (34.3 %) 9 (52.9 %) 44 (43.6 %) 0.53
• Medium (20,001–40,000 patients) 65 (28.9 %) 19 (26.4 %) 12 (34.3 %) 6 (35.3 %) 28 (27.7 %)
• Large (40,001–108,715 patients) 58 (25.8 %) 16 (22.2 %) 11 (31.4 %) 2 (11.8 %) 29 (28.7 %)
Urban or Rural Location
• Rural, suburban 94 (41.8 %) 25 (35.7 %) 14 (40.0 %) 7 (41.2 %) 48 (49.5 %) 0.73
• Metropolitan area with population < 1 million 99 (44.0 %) 36 (51.4 %) 17 (48.6 %) 8 (47.1 %) 38 (39.2 %)
• Metropolitan area with population > 1 million 26 (11.6 %) 9 (12.9 %) 4 (11.4 %) 2 (11.8 %) 11 (11.3 %)
Located in a VA region that endorses TIDES 36 (16.0 %) 11 (15.3 %) 17 (48.6 %) 2 (11.8 %) 6 (5.9 %) <0.001
Academically Affiliated 123 (54.7 %) 39 (54.2 %) 22 (62.9 %) 6 (35.3 %) 56 (55.5 %) 0.31
Located within a VA Medical Center 137 (60.9 %) 41 (56.9 %) 25 (71.4 %) 9 (52.9 %) 62 (61.4 %) 0.46
Mental Health Specialists on the Primary Care Staff
• Psychiatrist 64 (28.4 %) 20 (29.0 %) 10 (28.6 %) 12 (70.6 %) 22 (22.0 %) 0.001
• Psychologist 69 (30.7 %) 29 (40.9 %) 11 (31.4 %) 9 (52.9 %) 20 (20.2 %) 0.006
• Social worker 170 (75.6 %) 53 (74.7 %) 30 (85.7 %) 13 (76.5 %) 74 (76.3 %) 0.64
Presence of Clinician Champion for Depression Treatment 39 (17.3 %) 19 (26.4 %) 4 (11.8 %) 2 (11.8 %) 14 (13.9 %) 0.14
Presence of Retreat for Depression Training 21 (9.3 %) 8 (12.3 %) 2 (6.9 %) 1 (8.3 %) 10 (12.7 %) 0.92
Duration model was in place in years (Mean ± SD; range) 6.19 ±7.12 2.74 ± 2.13 1.17 ± 0.75
0–38 1–8 0–2

*Bold numbers are significant, p < 0.05. The Biloxi/Gulfport VA facilities were excluded because of the circumstances following Hurricanes Katrina and Rita

TIDES=Translating Initiatives for Depression into Effective Solutions

BHL=Behavioral Health Laboratory

§ PC-MHI=Primary Care-Mental Health Integration

║ Fisher’s Exact test used when cell count < 5

As shown in Table 3, having more established quality improvement (QI) processes (OR 2.25, 95 % CI [1.36, 3.72], p = 0.002), a clinician champion for depression treatment (OR 2.37, 95 % CI [1.17, 4.78], p = 0.02), or a psychologist on the PC staff (OR 1.92, 95 % CI [1.06, 3.48], p = 0.03) were significantly associated with collocation. Being in a VA region that had endorsed TIDES (OR 8.5, 95 % CI [3.76, 19.20], p < 0.001) or having sufficient IT support (OR 1.61, 95 % CI [1.01, 2.57], p = 0.04) were significantly associated with TIDES. Having a psychologist (OR 2.80, 95 % CI [1.03, 7.64], p = 0.04) or psychiatrist on the PC staff (OR 7.57, 95 % CI [2.53, 22.61], p < 0.001), less financial insufficiency (OR 0.35, 95 % CI [0.17, 0.72], p = 0.004), or more sufficient space (OR 1.91, 95 % CI [1.07, 3.38], p = 0.03) were significantly linked with BHL. Having poorer communication among PC staff (OR 0.60, 95 % CI [0.38, 0.97], p = 0.04), fewer QI processes (OR 0.44, 95 % CI [0.28, 0.70], p < 0.001), insufficient financial resources (OR 1.43, 95 % CI [1.00, 2.03], p = 0.045), lacking a psychologist (OR 0.38, 95 % CI [0.21, 0.70], p = 0.002) or psychiatrist (OR 0.53, 95 % CI [0.29, 0.97], p = 0.04) on PC staff, and being in a VA region that did not endorse TIDES (OR 0.20, 95 % CI [0.08, 0.50], p = 0.001) were associated with having adopted none of the models.

Table 3.

Bivariate Analysis* of Organizational Readiness Factors and Primary Care Practice Setting Demographics Against Adoption of Collocation, TIDES†, BHL‡, or No Depression Care Improvement Model

Independent variables Practices with Collocation n = 72 / 223 OR [95 % CI] Practices with TIDES, n = 35 / 210 OR [95 % CI] Practices with BHL, N = 17 / 212 OR [95 % CI] Practices with No Depression Care Improvement Model, N = 101 / 225 OR [95 % CI]
Organizational Readiness for Change: Adequacy of resources
 Financial insufficiency 0.97 [0.67, 1.39] 0.91 [0.57, 1.46] 0.35 [0.17, 0.72] ** 1.43 [1.01, 2.03] *
 Sufficiency of space 0.85 [0.64, 1.14] 1.09 [0.75, 1.59] 1.91 [1.07, 3.38] * 0.93 [0.71, 1.22]
 Sufficiency for information technology (IT) support 0.85 [0.60, 1.20] 1.61 [1.01, 2.57] * 1.31 [0.70, 2.45] 0.83 [0.60, 1.15]
 Clinical provider insufficiency 1.01 [0.70, 1.47] 1.14 [0.71, 1.81] 1.09 [0.56, 2.10] 0.91 [0.64, 1.29]
Organizational Readiness for Change: Staff attributes
 Clinician champion in clinic for depression treatment 2.37 [1.17, 4.78] * 0.59 [0.20, 1.78] 0.65 [0.14, 2.98] 0.63 [0.31, 1.29]
 Psychiatrist on primary care staff 1.00 [0.54, 1.88] 0.98 [0.44, 2.17] 7.57 [2.53, 22.61] ** 0.53 [0.29, 0.97] *
 Psychologist on primary care staff 1.92 [1.06, 3.48] * 1.02 [0.47, 2.22] 2.80 [1.03, 7.64] * 0.38 [0.21, 0.70] **
 Social worker on primary care staff 0.81 [0.42, 1.56] 1.93 [0.71, 5.27] 0.98 [0.30, 3.16] 0.90 [0.48, 1.70]
Organizational Readiness for Change: Motivation for change
 Retreat for depression training 1.16 [0.45, 2.95] 0.53 [0.12, 2.42] 0.72 [0.09, 5.93] 1.25 [0.50, 3.11]
 Located in a VA regional network that endorsed TIDES 0.92 [0.43, 2.00] 8.50 [3.76, 19.20] ** 0.73 [0.16, 3.37] 0.20 [0.08, 0.50] **
Organizational Readiness for Change: Organizational climate
 Teamwork 1.09 [0.80, 1.48] 1.08 [0.73, 1.62] 1.23 [0.72, 2.09] 0.84 [0.63, 1.12]
 Communication and cooperation 1.02 [0.63, 1.67] 1.47 [0.78, 2.77] 2.56 [1.05, 6.24] * 0.60 [0.38, 0.97] *
 Orientation towards quality improvement 2.25 [1.36, 3.72] ** 0.98 [0.55, 1.74] 1.61 [0.70, 3.69] 0.44 [0.28, 0.70] **
 Competing demands and stress 0.89 [0.53, 1.50] 1.18 [0.60, 2.33] 0.58 [0.23, 1.44] 1.18 [0.72, 1.92]
 Internal authority over primary care clinic 1.00 [0.69, 1.46] 1.22 [0.75, 2.00] 0.71 [0.37, 1.37] 0.97 [0.69, 1.38]
 External authority over relationship with subspecialists 1.02 [0.69, 1.50] 1.20 [0.73, 1.96] 1.51 [0.75, 3.04] 0.79 [0.54, 1.14]
 Resistance 0.96 [0.66, 1.39] 0.89 [0.55, 1.43] 0.85 [0.46, 1.59] 1.16 [0.82, 1.65]
Primary Care Practice Setting Organizational Demographics
 Clinic size
 • Medium (20,001–40,000 patients) compared to small 0.73 [0.37, 1.42] 1.70 [0.71, 4.05] 1.05 [0.35, 3.10] 1.00 [0.53, 1.87]
 • Large (40,001–108,715 patients) compared to small 0.67 [0.33, 1.35] 1.76 [0.72, 4.28] 0.38 [0.08, 1.84] 1.32 [0.69, 2.52]
 Academic affiliation 0.97 [0.55, 1.70] 1.49 [0.71, 3.13] 0.43 [0.15, 1.21] 1.06 [0.62, 1.79]
 Located within VA Medical center (versus community) 0.79 [0.44, 1.39] 1.74 [0.79, 3.83] 0.69 [0.25, 1.86] 1.04 [0.61, 1.78]

* p < 0.05, bold numbers are significant, ** p < 0.01

TIDES=Translating Initiatives for Depression into Effective Solutions

BHL=Behavioral Health Laboratory

Table 4 shows results of multivariate regression for predicting collocation, TIDES, and no model implementation. Being a PC practice with strongly established processes for QI (OR 2.30, 95 % CI [1.36, 3.87], p = 0.002), or with a clinician champion for depression treatment (OR 2.36, 95 % CI [1.14, 4.88], p = 0.02), significantly predicted collocation. Being a PC practice located in a regional network that had endorsed TIDES (OR 8.42, 95 % CI [3.69, 19.26], p < 0.001) significantly predicted TIDES implementation. Overall, practices with fewer established QI processes (OR 0.47, 95 % CI [0.27, 0.85], p = 0.01), without a psychologist on staff (OR 0.37, 95 % CI [0.17, 0.80], p = 0.01), and located in regional networks that did not endorse TIDES (OR 0.22, 95 % CI [0.08. 0.57], p = 0.002) significantly predicted adopting none of the models. The direction, magnitude, and significance did not change when the models were adjusted for practice size (not shown).

Table 4.

Multivariate Regression‡ Results Predicting Adoption of Collocation or TIDES† or No Depression Care Improvement Model

Practice Implementation of Collocation Practice Implementation of TIDES Practice with No Implementation of Any Depression Care Improvement Model
n = 221 OR [95 % CI] n = 225 OR [95 % CI] N = 225 OR [95 % CI]
Clinician champion in clinic for depression treatment 2.36 [1.14, 4.88] *
Orientation towards quality improvement (QI) 2.30 [1.36, 3.87] ** 0.48 [0.27, 0.85] *
Sufficiency for information technology (IT) support 1.61 [0.97, 2.67]
Psychiatrist in primary care staff 1.04 [0.48, 2.26]
Psychologist in primary care staff 1.78 [0.96, 3.31] 0.37 [0.17, 0.80] *
Located in a VA regional network that endorsed TIDES 8.42 [3.69, 19.26] ** 0.22 [0.08, 0.57] **
Communication and cooperation 0.85 [0.47, 1.54]
Financial insufficiency 1.05 [0.70, 1.60]

* p < 0.05, bold numbers are significant. ** p < 0.01

TIDES=Translating Initiatives for Depression into Effective Solutions

‡ The multivariate regression models were constructed separately for collocation, TIDES, and no depression care model using measures that had bivariate associations of p < 0.10

DISCUSSION

This study shows that pre-existing demographic and readiness characteristics of PC practices are associated with whether the practice chooses to implement a depression care improvement model and with what type of model the practice chooses among alternatives. As such, the study validates the concept that both practice context and the differing characteristics of alternative innovation models shape model adoption. Our findings also extend the concept of organizational readiness to improve MH care toward a consideration of the specific readiness factors that may make implementation of alternative models more or less attractive. Our work thus provides a framework for additional investigation and a refined set of readiness factors for managers to consider in achieving improved depression care.

Of the three approaches, PC practices appear most ready to implement collocation. Collocation had been present the longest (average 6.2 years) in practices adopting it, and the majority of practices that had not adopted it planned to do so. These findings suggest that collocation is easier to adopt than other models, supporting our qualitative analysis showing few requirements for redesign to implement this model. Having a depression clinical champion or a QI-oriented culture independently predicted collocation. Consistent with theories of innovation diffusion, greater model simplicity may have made collocation particularly attractive within the context of local QI efforts.

In contrast to the local readiness factors associated with collocation, regional endorsement of the TIDES model independently predicted adoption of TIDES, as well as any depression care improvement model. The regional endorsement reflected VA administration and financing, which flows through regional administration.32 Given the spread approach for TIDES was regional, this finding suggests that model dissemination through regional networks may be effective, particularly for improvements that require substantial redesign.

There were insufficient BHL practices in 2007 to support multivariable analysis. Among the 17 practices reporting BHL implementation, the model had been in place an average of 1.2 years. Based on bivariate results, these early BHL practices had significantly more MHS, space, and financial resources, as well as a culture of better communication and collaboration. It may be that this model, which requires creation of a laboratory capable of assessing, triaging and arranging or providing treatment for any MH condition, is easiest to adopt when resources are more sufficient.

It is of some concern that collocation remained the most common approach in 2007. As originally envisioned by VA policymakers in 2006, collocated care was to be collocated collaborative care, based on the White River Junction VA literature.5457 This model offers open access to MH and has demonstrated improvement in depression care. Operational definitions of collocated collaborative care in the field, however, required only collocating one or more MHS (i.e., psychiatrists, psychologists, social workers or advanced practice nurses) in PC. There is no evidence that collocation alone improves depression outcomes.4042

There are limitations to our analyses. First, the analysis was cross-sectional, and causal relationships between organizational factors and implementation of various models cannot be proven. For instance, the implementation of collocation may increase the awareness of depression treatment among clinical leaders or increase the presence of psychologists in PC. However, organizational climate tend to remain stable despite short-term organizational change efforts, which suggests that some ORC measures may have a causal relationship with model adoption.5860 Second, assessment of organizational features relied on a single respondent, the PC clinic director, and may not reflect the practice as a whole. However, directors’ responses have been highly correlated with staff responses in prior work.29 Third, we were not able to analyze combinations of models or assess independent predictors of BHL adoption due to sample size limitations, despite a relatively large (for organizational surveys) sample size of 225 observations. Fourth, the results here may not generalize to PC clinics outside of the VA, although we expect findings to apply to other managed care systems. Fifth, we do not measure implementation fidelity to published models; the analyses reported here focus only on understanding the practice’s choice for implementing a PC-MHI model. Similarly, we cannot determine the superiority of one model over another. In addition, some of the collocated MHS may not be employed by PC, and are therefore unaccounted for in this PC-based survey. Finally, we expect that model adoption has changed substantially between 2007 and today.

In summary, this is one of the few studies to explore the relationships between organizational readiness factors and implementation of depression care improvement approaches. We found that several readiness factors should be considered by policymakers and system leaders, based on their significant relationships with model implementation. These include local PC practice orientation towards QI, presence of a clinician champion for depression, financial and space sufficiency, IT support sufficiency, and regional endorsement of the improvement model. Based on our study findings, better understanding of the determinants of model adoption will be essential for achieving more effective designs and dissemination strategies for improving care for depression and other MH conditions in PC practice populations.

Electronic supplementary material

ESM 1 (62.3KB, pdf)

(PDF 62 kb)

Acknowledgements

Contributors

Contributors include Steven Asch, MD MPH, Ann Chou, PhD, Johanna Klaus, PhD, Edmund Chaney, PhD, John McCarthy, PhD, Michael Mitchell, PhD, Susan Stockdale, PhD, and Brian Mittman, PhD.

The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs, or the US government, or other affiliated institutions.

Funders

Funding support provided by VA Office of Academic Affiliations, Health Services Research and Development through the Health Services Fellowship Training Program (TMP 65-020). Dr. Yano’s time was funded by the VA HSR&D Service through a Research Career Scientist Award (Project # RCS 05-195). The project was also supported by VA HSR&D Project #09-082 (Yano, PI).

Prior Presentations

Oral abstracts summarizing these findings were presented at the Society of General Internal Medicine 35th Annual Meeting on May 9-12, 2012 in Orlando, Florida, and the Academy Health Annual Research Meeting on June 25–27, 2012 in Orlando, Florida.

Conflict of Interest

The authors declare that they do not have a conflict of interest.

REFERENCES

  • 1.Kessler R, Chiu W, Demler O, Walters E. Prevalence, severity, and comorbidity of twelve-month DSM-IV disorders in the National Comorbidity Survey Replication (NCS-R) Arch Gen Psychiat. 2005;62(6):617–627. doi: 10.1001/archpsyc.62.6.617. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Singleton N, Bumpstead R, O’Brien M, Lee A, Meltzer H. Office of National Statistics: Psychiatric morbidity among adults living in private households, 2000. London: Her Majesty’s Stationery Office; 2001. [DOI] [PubMed] [Google Scholar]
  • 3.World Health Organization. Mental health: Depression. 2011; http://www.who.int/mental_health/management/depression/definition/en/. Accessed August 21, 2012.
  • 4.Simon G, Von Korff M. Recognition and management of depression in primary care. Arch Fam Med. 1995;4:99–105. doi: 10.1001/archfami.4.2.99. [DOI] [PubMed] [Google Scholar]
  • 5.Young A, Klap R, Sherbourne C, Wells K. The quality of care for depressive and anxiety disorders in the United States. Arch Gen Psychiat. 2001;58(1):55–61. doi: 10.1001/archpsyc.58.1.55. [DOI] [PubMed] [Google Scholar]
  • 6.Wang P, Lane M, Olfson M, Pincus H, Wells K, Kessler R. Twelve-month use of mental health services in the United States. Arch Gen Psychiat. 2005;62:629–640. doi: 10.1001/archpsyc.62.6.629. [DOI] [PubMed] [Google Scholar]
  • 7.Alexopoulos G, Reynolds C, Bruce M, et al. Reducing suicidal ideation and depression in older primary care patients: 24-month outcomes of the PROSPECT study. Am J Psychiat. 2009;166:882–890. doi: 10.1176/appi.ajp.2009.08121779. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Reiss-Brennan B, Briot P, Savitz L, Cannnon W, Staheli R. Cost and quality impact of Intermountain’s Mental Health Integration Program. J Healthc Manag. 2010;55(2):97–114. [PubMed] [Google Scholar]
  • 9.Unutzer J, Katon W, Callahan C, et al. Collaborative care management of late-life depression in the primary care setting: a randomized controlled trial. JAMA. 2002;288(22):2836–2845. doi: 10.1001/jama.288.22.2836. [DOI] [PubMed] [Google Scholar]
  • 10.Katon W, Von Korff M, Lin E, et al. Stepped collaborative care for primary care patients with persistent symptoms of depression: a randomized trial. Arch Gen Psychiat. 1999;56(12):1109–1115. doi: 10.1001/archpsyc.56.12.1109. [DOI] [PubMed] [Google Scholar]
  • 11.Katon W, Von Korff M, Lin E, et al. Collaborative management to achieve treatment guidelines: impact on depression in primary care. JAMA. 1995;273:1026–1031. doi: 10.1001/jama.1995.03520370068039. [DOI] [PubMed] [Google Scholar]
  • 12.Wells K, Sherbourne C, Schoenbaum M, et al. Five-year impact of quality improvement for depression. Arch Gen Psychiat. 2001;61:378–386. doi: 10.1001/archpsyc.61.4.378. [DOI] [PubMed] [Google Scholar]
  • 13.Gilbody S, Bower P, Fletcher J, Richards D, Sutton AJ. Collaborative care for depression. Arch Intern Med. 2006;166:2314–2321. doi: 10.1001/archinte.166.21.2314. [DOI] [PubMed] [Google Scholar]
  • 14.Bower P, Gilbody S, Richards D, Fletcher J, Sutton A. Collaborative care for depression in primary care. Br J Psychiat. 2006;189:484–493. doi: 10.1192/bjp.bp.106.023655. [DOI] [PubMed] [Google Scholar]
  • 15.Simon GE, Katon WJ, VonKorff M, et al. Cost-effectiveness of a collaborative care program for primary care patients with persistent depression. Am J Psychiat. 2001;158:1638–1644. doi: 10.1176/appi.ajp.158.10.1638. [DOI] [PubMed] [Google Scholar]
  • 16.Unutzer J, Katon WJ, Fan M-Y, et al. Long-term cost effects of collaborative care for late-life depression. Am J Manag Care. 2008;14:95–100. [PMC free article] [PubMed] [Google Scholar]
  • 17.Liu C-F, Hedrick SC, Chaney EF, et al. Cost-effectiveness of collaborative care for depression in a primary care veteran population. Psychiat Serv. 2003;54(5):698–704. doi: 10.1176/appi.ps.54.5.698. [DOI] [PubMed] [Google Scholar]
  • 18.Lave J, Frank R, Schulberg H, Kamlet M. Cost-effectiveness of treatments for major depression in primary care practice. Arch Gen Psychiat. 1998;55:645–651. doi: 10.1001/archpsyc.55.7.645. [DOI] [PubMed] [Google Scholar]
  • 19.Schoenbaum M, Unutzer J, Sherbourne C, et al. Cost-effectiveness of practice-initiated quality improvement for depression: results of a randomized controlled trial. JAMA. 2001;286(11):1325–1330. doi: 10.1001/jama.286.11.1325. [DOI] [PubMed] [Google Scholar]
  • 20.Nutting P, Rost K, Dickinson M, et al. Barriers to initiating depression treatment in primary care. J Gen Intern Med. 2002;17:103–111. doi: 10.1046/j.1525-1497.2002.10128.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Henke R, McGuire T, Zaslavsky A, Ford D, Meredity L, Arbelaez J. Clinician- and organization-level factors in the adoption of evidence-based care for depression in primary care. Health Care Manag Rev. 2008;33(4):289–299. doi: 10.1097/01.HCM.0000318766.29277.49. [DOI] [PubMed] [Google Scholar]
  • 22.Nutting P, Gallagher K, Riley K, et al. Care management for depression in primary care practice: findings from the RESPECT-depression trial. Ann Fam Med. 2008;6:30–37. doi: 10.1370/afm.742. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Kilbourne A, Schulberg H, Post E, Rollman B, Belnap B, Pincus H. Translating evidence-based depression management services to community-based primary care practices. Milbank Q. 2004;82(4):631–659. doi: 10.1111/j.0887-378X.2004.00326.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Belnap BH, Kuebler J, Upshur C, et al. Challenges of implementing depression care management in the primary care setting. Adm Policy Ment Health. 2006;33(1):65–75. doi: 10.1007/s10488-005-4237-z. [DOI] [PubMed] [Google Scholar]
  • 25.Scott WR. Innovation in medical care organizations: a synthetic review. Med Care Res Rev. 1990;47(165):165–192. doi: 10.1177/107755879004700203. [DOI] [PubMed] [Google Scholar]
  • 26.Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in Service Organizations: systematic review and recommendations. Milbank Q. 2004;82(2):581–629. doi: 10.1111/j.0887-378X.2004.00325.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Weiner B, Amick H, Lee SD. Review: conceptualization and measurement of organizational readiness for change: a review of the literature in health services research and other fields. Med Care Res Rev. 2008;65:379–436. doi: 10.1177/1077558708317802. [DOI] [PubMed] [Google Scholar]
  • 28.Hamilton AB, Cohen AN, Young AS. Organizational readiness in specialty mental health care. J Gen Intern Med. 2009;25(Suppl 1):27–31. doi: 10.1007/s11606-009-1133-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Lehman W, Greener J, Simpson D. Assessing organizational readiness for change. J Subst Abus Treat. 2002;22:197–209. doi: 10.1016/S0740-5472(02)00233-7. [DOI] [PubMed] [Google Scholar]
  • 30.VHA Handbook 1160.01 Uniform Mental Health Services in VA Medical Centers and Clinics. Washington, DC: Department of Veterans Affairs: Office of Patient Care Services; 2008.
  • 31.Rubenstein L, Meredith L, Parker L, et al. Impacts of evidence-based quality improvement on depression in primary care. J Gen Intern Med. 2006;21:1027–1035. doi: 10.1111/j.1525-1497.2006.00549.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Rubenstein L, Chaney E, Ober S, et al. Using evidence-based quality improvement methods for translating depression collaborative care research into practice. Fam Syst Health. 2010;28:91–113. doi: 10.1037/a0020302. [DOI] [PubMed] [Google Scholar]
  • 33.Zanjani F, Miller B, Turiano N, Ross J, Oslin D. Effectiveness of telephone-based referral care management, a brief intervention to improve psychiatric treatment engagement. Psychiatr Serv. 2008;59:776–781. doi: 10.1176/appi.ps.59.7.776. [DOI] [PubMed] [Google Scholar]
  • 34.Oslin D, Ross J, Sayers S, Murphy J, Kane V, Katz I. Screening, assessment, and management of depression in VA primary care clinics: The Behavioral Health Laboratory. J Gen Intern Med. 2006;21:46–50. doi: 10.1111/j.1525-1497.2005.0267.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Oslin D, Sayers S, Ross J, et al. Disease management for depression and at-risk drinking via telephone in an older population of veterans. Psychosom Med. 2003;65(6):931–937. doi: 10.1097/01.PSY.0000097335.35776.FB. [DOI] [PubMed] [Google Scholar]
  • 36.Liu C-F, Rubenstein LV, Kirchner JE, et al. Organizational cost of quality improvement for Depression Care. Heal Serv Res. 2009;44(1):225–244. doi: 10.1111/j.1475-6773.2008.00911.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Rubenstein LV, Jackson-Triche M, Unutzer J, et al. Evidence-based care for depression in managed primary care practices. Heal Aff. 1999;18(5):89–105. doi: 10.1377/hlthaff.18.5.89. [DOI] [PubMed] [Google Scholar]
  • 38.Post EP, Kilbourne AM, Bremer RW, Solano FX, Jr, Pincus HA, Reynolds CF, III. Organizational factors and depression management in community-based primary care settings. Implement Sci. 2009;4:84. [DOI] [PMC free article] [PubMed]
  • 39.Tew J, Klaus J, Oslin D. The behavioral health laboratory: building a stronger foundation for the Patient-centered medical home. Fam Syst Health. 2010;28(2):130–145. doi: 10.1037/a0020249. [DOI] [PubMed] [Google Scholar]
  • 40.Thielke S, Vannoy S, Unutzer J. Integrating mental health and primary care. Prim Care Clin Office Pract. 2007;34:571–592. doi: 10.1016/j.pop.2007.05.007. [DOI] [PubMed] [Google Scholar]
  • 41.Bower P, Knowles S, Coventry PA, Rowland N. Counselling for mental health and psychosocial problems in primary care (Review). Cochrane Database of Syst Rev. 2011(9). [DOI] [PMC free article] [PubMed]
  • 42.Krahn DD, Bartels SJ, Coakley E, et al. PRISM-E: comparison of integrated care and enhanced specialty referral models in depression outcomes. Psychiat Serv. 2006;57(7):946–953. doi: 10.1176/appi.ps.57.7.946. [DOI] [PubMed] [Google Scholar]
  • 43.Simpson D. A conceptual framework for transferring research to practice. J Subst Abus Treat. 2002;22:171–182. doi: 10.1016/S0740-5472(02)00231-3. [DOI] [PubMed] [Google Scholar]
  • 44.Rogers E. Diffusion of innovations. New York: The Free Press; 1995. [Google Scholar]
  • 45.Yano E, Fleming B, Canelo, et al. National Survey Results for the Primary Care Director Module of the VHA Clinical Practice Organizational Survey. Sepulveda, CA: VA HSR&D Center for the Study of Healthcare Provider Behavior; 2008.
  • 46.Chou A, Rose D, Farmer M, Canelo I, Rubenstein L, Yano E. Organizational Factors Affecting the Likelihood of CancerScreening among VA Patients. Paper presented at: 13th Annual Healthcare Organizational Research Association (HORA) ConferenceJune 2011; Seattle, WA.
  • 47.Kline R. Principles and practice of structural equation modeling. New York: The Guilford Press; 2005. [Google Scholar]
  • 48.Groves R, Fowler F, Jr, Couper M, Lepkowski J, Singer E, Tourangeau R. Survey methodology. 2. Hoboken: Wiley; 2009. [Google Scholar]
  • 49.Area Resource File (ARF). Rockville, MD: US Department of Health and Human Services, Health Resources and Services Administration, Bureau of Health Professions; 2008.
  • 50.Department of Veterans Affairs. Office of Academic Affiliations. http://www.va.gov/oaa/. Accessed Aug 21, 2012.
  • 51.Agresti A, Franklin C. Statistics: The art and science of learning from data. Upper Saddle River: Pearson Prentice Hall; 2009. [Google Scholar]
  • 52.Wagner E, Austin B, Davis C, Hindmarsh M, Schafer J, Bonomi A. Improving chronic illness care: translating evidence into action. Heal Aff. 2001;20(6):64–78. doi: 10.1377/hlthaff.20.6.64. [DOI] [PubMed] [Google Scholar]
  • 53.Improving Chronic Illness Care. 1996-2012; The MacColl Center. The Improving Chronic Illness Care program is supported by The Robert Wood Johnson Foundation, with direction and technical assistance provided by Group Health’s MacColl Center for Health Care Innovation. Available at: http://www.improvingchroniccare.org/. Accessed Aug 21, 2012.
  • 54.Pomerantz A, Shiner B, Watts B, et al. The White River Model of colocated collaborative care. Fam Syst Health. 2010;28(2):114–129. doi: 10.1037/a0020261. [DOI] [PubMed] [Google Scholar]
  • 55.Pomerantz AS, Cole BH, Watts BV, Weeks WB. Improving efficiency and access to mental health care: combining integrated care and advanced clinical access. Gen Hosp Psychiat. 2008;30:546–551. doi: 10.1016/j.genhosppsych.2008.09.004. [DOI] [PubMed] [Google Scholar]
  • 56.Pomerantz A, Sayers S. Primary care-mental health integration in healthcare in the Department of Veterans Affairs. Fam Syst Health. 2010;28(2):78–82. doi: 10.1037/a0020341. [DOI] [PubMed] [Google Scholar]
  • 57.Watts B, Shiner B, Pomerantz A, Stender P, Weeks W. Outcomes of a quality improvement project integrating mental health into primary care. Qual Saf Health Care. 2007;16:378–381. doi: 10.1136/qshc.2007.022418. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 58.Parker V, Wubbenhorst W, Young G, Desai KR, Charns MP. Implementing quality improvement in hospitals: the role of leadership and culture. Am J Med Qual. 1999;14:64–69. doi: 10.1177/106286069901400109. [DOI] [PubMed] [Google Scholar]
  • 59.Hemmelgarn A, Glisson C, James L. Organizational culture and climate: implications for services and interventions research. Clin Psychol Sci Pract. 2006;13(1):73–89. doi: 10.1111/j.1468-2850.2006.00008.x. [DOI] [Google Scholar]
  • 60.Shortell S, O’Brien J, Carman J, et al. Assessing the impact of continuous quality improvement/total quality management: concept vs implementation. Heal Serv Res. 1995;30(2):377–401. [PMC free article] [PubMed] [Google Scholar]
  • 61.Collaborative Care for Depression in the Primary Care Setting: a primer on VA’s Translating Initiatives for Depression into Effective Solutions (TIDES) Project. In: VA Health Services Research and Development Service, Office of Research and Development, Dept of Veterans Affairs, ed. Washington, DC 2008.

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

ESM 1 (62.3KB, pdf)

(PDF 62 kb)


Articles from Journal of General Internal Medicine are provided here courtesy of Society of General Internal Medicine

RESOURCES