Quality Improvement for Depression Enhances Long-term Treatment Knowledge for Primary Care Clinicians

Abstract

OBJECTIVE

We evaluated the effect of implementing quality improvement (QI) programs for depression, relative to usual care, on primary care clinicians' knowledge about treatment.

DESIGN AND METHODS

Matched primary care clinics (46) from seven managed care organizations were randomized to usual care (mailed written guidelines only) versus one of two QI interventions. Self-report surveys assessed clinicians' knowledge of depression treatments prior to full implementation (June 1996 to March 1997) and 18 months later. We used an intent-to-treat analysis to examine intervention effects on change in knowledge, controlling for clinician and practice characteristics, and the nested design.

PARTICIPANTS

One hundred eighty-one primary care clinicians.

INTERVENTIONS

The interventions included institutional commitment to QI, training local experts, clinician education, and training nurses for patient assessment and education. One intervention had resources for nurse follow-up on medication use (QI-meds) and the other had reduced copayment for therapy from trained, local therapists (QI-therapy).

RESULTS

Clinicians in the intervention group had greater increases compared with clinicians in the usual care group over 18 months in knowledge of psychotherapy (by 20% for QI-meds, P = .04 and by 33% for QI-therapy, P = .004), but there were no significant increases in medication knowledge. Significant increases in knowledge scores (P = .01) were demonstrated by QI-therapy clinicians but not clinicians in the QI-meds group. Clinicians were exposed to multiple intervention components.

CONCLUSIONS

Dissemination of QI programs for depression in managed, primary care practices improved clinicians' treatment knowledge over 18 months, but breadth of learning was somewhat greater for a program that also included active collaboration with local therapists.

Recent research findings and policy developments reinforce the importance of achieving broad-based improvement in quality of care for depression in primary care. Depression's high prevalence^1,2 and extensive social impact³ are well-documented. Furthermore, while depressed individuals often receive their only care in the primary care setting, rates of detection of depression and of appropriate treatment in such settings are moderate at best,⁴ eroding the cost-effectiveness of care.⁵ Previous research on improving care for depression in primary care settings shows that, as has been the case for other conditions,⁶ knowledge-based interventions, such as lectures or feedback of depression scores and management suggestions, have little, if any, effect on quality of care or health outcomes. Although studies have shown effects of educational interventions on short-term diagnostic or treatment knowledge,⁶ few studies have evaluated prolonged knowledge effects. Only one study demonstrated improved clinician knowledge of psychiatric diagnoses over nearly a year.⁷ The purpose of this study was to assess the effects of a combined educational and clinical care system quality improvement (QI) intervention on primary care clinician knowledge 18 months after the initiation of the intervention and 6 months after its completion.

Despite the lack of demonstrated efficacy of education alone for changing depression care, the successful collaborative care models^8,9 that include additional components such as case management have retained formal provider education as one element of the intervention program. In addition, these models may foster informal education (e.g., through increased contact between primary care providers and mental health specialists). These complex models have been successful in changing the process and outcomes of care. If increased primary care clinician knowledge about depression treatments contributes to the effectiveness of these interventions, this increased knowledge should be observable. Collaborative care experiments have demonstrated effects on short-term provider knowledge and attitudes but have shown that without more enduring changes to the delivery system structures, clinician practice may revert to old patterns and lower rates of appropriate care after terminating the active phase of a QI program.¹⁰ Demonstration of sustained gains in clinician treatment knowledge after participating in a successful QI program would support the importance of the clinician education component of the full collaborative care model for depression.

Improving clinician knowledge is virtually a universal goal of clinically based QI programs,¹¹ indicating that depression experts think that, while not sufficient, improved primary care provider knowledge is necessary for achieving improvement in the appropriateness of the treatments they order. However, there are many possible routes for improving clinician knowledge.^6,12–16 In addition to informal educational opportunities provided by the intervention program itself, participation in the program might motivate primary care providers to access other educational resources, such as continuing medical education (CME) conferences or pharmacy company detailing. This article also investigates these indirect intervention effects on provider education.

Data for this article come from Partners in Care (PIC),^17,18 a randomized trial of QI for depression that uses expert design with local implementation by study-trained leaders from the participating managed, primary care practices. PIC tested two interventions, each of which involved collaboration between mental health specialists and primary care clinicians, and each of which included identical primary care clinician education materials. Fourteen study-trained local primary care leaders and 14 local mental health specialist leaders taught the remaining primary care clinicians from their sites. One of the interventions provided improved resources for antidepressant medication management (QI-meds), while the other provided reduced copayments for use of practice psychotherapists trained in an efficacious form of psychotherapy (QI-therapy). In both arms, provider education emphasized choosing the clinically appropriate type of treatment, taking into account patient preferences for medication or psychotherapy, but one arm made access to psychotherapy easier, while the other made access to case management for antidepressant medications easier.

We hypothesized, based on the literature, that both study interventions would improve long-term clinician knowledge of depression treatment. The QI-meds intervention involved a psychiatrist as a local mental health leader for the study, while QI-therapy involved a psychologist. On the basis of social influence theory,¹⁹ we thought that the psychiatrist might have greater educational impact. Conversely, the QI-therapy intervention involved more ongoing interaction between primary care providers and the group of therapists treating study patients, which might enhance the educational impact of QI-therapy.²⁰ We expected both interventions to increase clinician exposure to formal study-related educational activities such as lectures and academic detailing by the study-trained local leaders, as well as to increase informal study-related education through nurse case managers and mental health specialists. We encouraged, but did not expect, additional professional activities that were not study-related (e.g., keeping up with the academic literature on depression treatment, having contact with representatives of pharmaceutical companies detailing them about antidepressant medications, or requesting preauthorization of depression treatments).

METHODS

Sample and Evaluation Design

We analyzed data from 181 primary care clinicians from 46 practices of 7 managed care organizations (MCOs) across the United States who participated in the PIC study. Within each MCO, we formed clinical units, which could be a single clinic, a cluster of small clinics, or a clinical care team within a large clinic. We refer to these clinical units as clinics. We then grouped these clinics (clinical units) into blocks of three within each MCO, matching primary care clinician specialty mix, patient demographics including ethnic mix, and type of relationship with behavioral health (whether on- or off-site, carve-in or carve-out). Within each block of matched clinics, we randomized one clinic each to two QI programs (QI-meds and QI-therapy) and usual care.

Data were collected using two self-administered surveys mailed to all 181 eligible primary care providers. The Clinician Background Questionnaire (CBQ) was mailed to clinicians prior to full implementation of the study interventions (June 1996) and the Clinician Follow-Up Questionnaire (CFQ) was mailed approximately 18 months following full intervention implementation (March 1997). The survey took approximately 20 minutes to complete. Nonresponding providers were called and encouraged to return the survey. Final response rates were 92% (N = 167) for the CBQ and 94% (N = 170) for the CFQ. The survey consisted of batteries that have been evaluated previously for reliability and validity, and batteries developed specifically for this study. The pre-post sample included 160 (88%) clinicians who completed both the CBQ and the CFQ. More information about the study design and the clinician measures is available on the PIC Web site: http://www.rand.org/organization/health/partners. care/portweb.

Interventions

The key objective of the PIC interventions was to increase the proportion of depressed patients who initiated and adhered to appropriate treatment within a feasible practice budget. A core program for QI was developed and implemented in both types of intervention (QI-meds and QI-therapy) practices.²¹ There were three core program components. Practices allocated in-kind resources to support half of the participation and intervention costs. Practices also identified a local multidisciplinary expert team consisting of clinical staff from primary care, mental health, and nursing who participated in study intervention training so that the program could be disseminated locally in their practices. Expert leaders received training on clinician education and team management, and were responsible for providing monthly or bimonthly lectures over a 6-month period. The research team provided teaching slides, copies of clinician manuals,²² and pocket reminder cards that summarized assessment and treatment of depression. Specific training for nurses included the use of study educational materials designed for patients (a brochure and videotape), patient assessment for symptoms and functioning, and instruction about making effective referrals.

The QI-meds and QI-therapy interventions each had unique resources beyond the core components. With the QI-meds intervention, practices had a dedicated depression nurse specialist to assist with patient monitoring for adherence to treatment and progress over 6 or 12 months (this varied by random assignment at the patient level). Practices in the QI-meds group were also encouraged to attend one to two additional clinician lectures about long-term follow-up given by local psychiatrists who also received training about consultation with monthly supervision from a national depression expert.

Practices in the QI-therapy group had a reduced copayment for their patients to receive study-specific cognitive behavioral therapy (CBT), and training of practice therapists in both individual and group, 12-session CBT (modified versions were developed for minor depression and comorbid anxiety disorder).^23,24 Therapist training involved reading the manual, participating in a 3-day workshop, and supervision by a national expert.

Regardless of the intervention condition, clinicians and patients retained full choice about their treatment (whether to use medication or psychotherapy), and use of study intervention resources was optional. Most patients had an initial contact with the depression nurse specialist (75%), 30% of the QI-meds patients received all recommended follow-up nurse contacts (mean 3.3 visits), and 40% of the QI-therapy patients received study CBT.²⁵

Depression Knowledge Measures

Relative to clinicians in the usual care condition, we expected the QI interventions to directly increase clinicians' knowledge. We examined four aggregate measures of knowledge about the treatment of depression based on a 12-item knowledge scale measuring endorsement of evidence-based statements from the Agency for Health Care Policy and Research (now the Agency for Healthcare Research and Quality) practice guidelines for depression and validated by a panel of clinicians as nonambiguous indicators of knowledge (see Table 2).²⁶ Statements were rated on a 5-point Likert scale ranging from “very false” to “very true.” We first reversed items 7, 10, and 11 (false statements) and counted items answered “definitely” or “mostly” correct to compute an overall knowledge scale (percent of all items answered correctly) and subscales for general knowledge (such as knowledge about phases of treatment), antidepressant medication knowledge, and psychotherapy knowledge. These questions were asked at baseline and 18 months after intervention implementation.

Table 2.

Unadjusted Knowledge about Depression Treatment: Percent Correct for Specific Questions and Aggregate Scores (N = 160)^*

Knowledge Measure	Baseline	Follow-up
Specific item†
To what extent do you believe each of the following statements is true or false? On the line next to each statement, circle one number (from 1 to 5) for the answer that is closest to your own.
1. The maintenance phase of treatment for major depression focuses on preventing recurrence.	90.0	92.7
2. If psychotherapy for major depression has no effect within 6 weeks of regular sessions, medication is recommended.	63.3	77.3*
3. An appropriate trial of antidepressant medication for major depressive disorder requires use of therapeutic dosages daily for at least 4 to 6 weeks.	91.5	92.8
4. Medication and psychotherapy are efficacious for depression in elderly adults as well as for the non-elderly.	85.8	89.7
5. Evidence suggests that primary care clinicians prescribe appropriate dosages of antidepressants to fewer than a third of patients with a current major depressive disorder.	70.3	69.7
6. Most tricyclic antidepressants have equivalent efficacy as SSRIs for depressed patients.	55.3	60.0
7. Dysthymic disorder is mild, brief depression.	60.0	62.5
8. The goal of cognitive therapy is to remove symptoms of depression by identifying and correcting patients' distorted, negatively biased thinking.	68.6	79.7*
9. In general, antidepressant medication should be discontinued after 4 to 9 months for patients with a single major depressive episode who no longer have symptoms of depression.	71.4	77.9*
10. Anxiolytics and sedatives (minor tranquilizers) have equivalent efficacy in major depression as antidepressant medications.	89.6	94.8*
11. Psychotherapy with a trained therapist is appropriate as the sole treatment formoderate major depression that is not chronic, psychotic, or melancholic.	52.6	61.0*
12. Tricyclic antidepressants and SSRIs have equivalent side-effect profiles.	88.2	94.7*
Aggregate score‡
Overall (all 12 items)	73.3	78.6*
General (items 1, 4, & 7)	77.6	80.9
Antidepressants (items 3, 5, 6, 9, 10, & 12)	77.1	81.1*
Psychotherapy (items 2, 8, & 11)	60.5	71.3*

^*P < .10 for pre- vs postdifference by t-test.

^†Rated on a 1 to 5 scale from “definitely true” to “definitely false” and rescored as percent of clinicians answering correctly (“definitely true” or “ mostly true”).

^‡Scored by first reversing all items except 7, 10, and 11 ( false statements), counting the number of items answered correctly, dividing the sum by the total number of items for percent correct, and multiplying by 100 for proportion correct.

SSRI indicates selective serotonin reuptake inhibitors.

Covariates

Although a randomized design was used in the PIC study, we adjusted for several important covariates to enhance precision and account for minor imbalances across intervention groups. The covariates used include the matched randomization blocks, patient mental functioning, physical functioning, the number of chronic medical conditions (all three measures aggregated to the clinician level as average measures of patient case mix for each clinician), and clinicians' demographics, clinical training, estimated proportion of visits involving patients with depression, and readiness to change.

Exposure to Partners in Care Intervention Activities

To explain the underlying circumstances through which the intervention might have affected clinicians, we asked clinicians at follow-up (18 months after the intervention implementation) about their exposure to specific aspects of the intervention programs. These included (1) whether clinicians were aware that their clinic made any changes or implemented any programs aimed at improving depression care for primary care patients, (2) the number of times clinicians participated in individual educational sessions provided by another clinician at the practice, (3) the number of times clinicians participated in group seminars or grand rounds at the practice, (4) whether clinicians knew how many of their patients were enrolled into the 2-year follow-up phase of PIC, and (5) whether clinicians used various practice resources for treating depression (clinical practice guidelines, feedback on standard patient outcomes, nursing assessment and follow-up of patients, patient education brochures and videos, clinician education or training sessions by local experts, and referral to mental health specialists for individual patients).

Exposure to Non-partners in Care Depression Practice Activities

We also examined a number of depression-related practice activities that PIC clinicians might have been exposed to independent of the study intervention programs. Clinicians were asked at follow-up about the number of hours in the past 3 years that they spent in CME for depression. They were also asked about the amount of time (in hours) spent in quality assurance for mental health care during the previous year, the number of articles read about major depression, the number of mental health consultations, and the number of times detailed about antidepressants by a pharmaceutical company representative. In addition, clinicians were asked about the number of preauthorization requests for a specialty referral or for antidepressant medications.

Analysis

We first used analysis of variance and χ²methods to examine the balance of the baseline clinician characteristics (demographics, background and practice characteristics, patient case mix, and readiness to change) across intervention conditions (QI-meds, QI-therapy, and usual care). Next, we used an intent-to-treat analysis to test whether clinicians exposed to the study interventions had improved knowledge relative to care-as-usual clinicians. We examined intervention effects on knowledge using change scores (postintervention minus preintervention outcome scores) and also present baseline values to provide a point of reference. To account for slight differences in some clinician characteristics at baseline, the baseline and change models controlled for the randomization blocks, clinician demographics (gender, age, ethnicity), clinical training (specialty and board certification), depression caseload (percentage of visits with depression patients), patient case mix aggregated to the clinician level, and readiness to change or improve the management of patients with depression. Change models also controlled for the baseline value of the knowledge score. In order to account for the multilevel data structure (clinicians nested within clinics), we used multilevel analysis to fit a 2-level “clinic effects” model^27–29 with the clinician as the level-1 unit and the clinic as the level-2 unit (both specified as random effects) using SAS PROC MIXED^30,31 for continuous measures of knowledge. We report the adjusted baseline values for each measure along with the amount of postintervention minus preintervention change. We present the overall effect of the QI interventions combined relative to usual care, then the pairwise comparisons for the three intervention groups. We used a 1-tailed, 5% significance level to test our a priori hypothesis of more improvement in knowledge about treating depression for intervention clinicians relative to usual care clinicians. Finally, we present unadjusted data on receptivity to change, and direct and indirect exposure to PIC measures for each intervention condition to explain the intervention effects on knowledge.

RESULTS

Clinician Characteristics

At baseline, primary care clinicians (N = 160 who completed both a CBQ and a CFQ) participating in the PIC study (Table 1)) averaged 44 years of age, 36% were women, and 70% were non-Hispanic white. Fifty-two percent were family or general practice physicians, 33% were internal medicine physicians, and 14% were nonphysicians (either nurse practitioners or physician assistants). On average, they completed training 12 years ago, and 78% were board-certified in their specialty. They spent on average 37 hours each week working in direct patient care and in primary care activities, and about 7% percent of their practices involved treating patients with depression. Intervention clinicians were significantly more likely to report that they definitely needed to change or improve the way they cared for depression (χ²₂) = 6.17, P = .03). Only 8.9% of care-as-usual clinicians were ready to change, compared with 27.3% of QI-meds clinicians and 29.4% of QI-therapy clinicians. This finding may be due, in part, to an early intervention effect (i.e., initial clinician orientation activity). On average, clinicians spent about 8 hours of their time in CME for depression during the past 3 years, and during the past year, less than 1 hour in quality assurance activities focused on mental health. During the past year, they read an average of 4.2 depression articles, had 6.1 consultations with mental health specialists, were detailed 14 times by a pharmaceutical representative about depression medication, and requested preauthorization for referrals or selective serotonin reuptake inhibitors 16.2 and 17.5 times, respectively. Providers did not differ significantly by intervention status (either for the 2-group comparison of combined interventions relative to usual care or for 3-group comparisons) on any of the demographic, background, or case mix characteristics, although some baseline differences are worth mentioning. Although not statistically significant, there were fewer nonphysicians in the QI-meds group relative to the QI-therapy and usual care groups, and moderate differences in gender, ethnicity, board certification, and readiness to change. Therefore, we adjusted our final multivariate models for age, gender, ethnicity, board certification, and readiness to change to balance groups prior to evaluating change in clinician knowledge. With two exceptions, no significant differences across intervention groups for any of our measures of indirect exposure to depression activities at baseline were demonstrated. Clinicians in the intervention groups (only for the pooled test, not for the 3-group comparison) read significantly more articles about depression (4.6 vs 4.2, F = 4.16, P < .001) and were detailed by pharmaceutical companies more often (14.5 times vs 14.0 times, F = 4.78, P < .001), but the magnitude of these differences was of little practical significance.

Table 1.

Baseline Primary Care Clinician Characteristics Overall and by Intervention Condition^*

Clinician Characteristic*	All Providers (N = 160)	QI-Meds (N = 49)	QI-Therapy (N = 58)	Usual Care (N = 53)	Meds + Therapy (N = 107)
Demographics
Age, y	43.7 (9.1)	44.0 (8.3)	43.1 (9.5)	44.0 (9.4)	43.5 (9.0)
Female, %	35.6	30.6	41.4	34.0	36.5
Non-Hispanic white, %	69.6	77.6	60.7	71.7	68.6
Provider type, %
Internal medicine physician	33.1	32.7	37.9	28.3	35.5
Family or general physician	53.1	61.2	43.1	56.6	51.4
Nonphysician	13.8	6.1	19.0	15.1	13.1
Time since completed training, y	11.7 (9.7)	12.0 (9.6)	12.0 (9.9)	11.1 (9.7)	12.0 (9.7)
Board certified, %	77.9	76.1	79.6	77.8	77.9
Hours per week in office-based direct primary care	37.1 (8.8)	36.7 (8.3)	36.6 (8.9)	38.1 (9.3)	36.7 (8.6)
Visits with depressed patients, % of total	6.8 (6.0)	6.4 (5.3)	7.1 (6.3)	7.0 (6.5)	6.8 (5.8)
Case mix
Mental health, 0–100†	50.1 (1.7)	50.2 (1.5)	50.1 (1.7)	50.2 (1.9)	50.1 (1.6)
Physical health, 0–100†	47.0 (3.0)	46.8 (3.1)	47.0 (3.3)	47.0 (2.8)	46.9 (3.2)
Number of chronic diseases†	1.5 (0.49)	1.5 (0.46)	1.6 (0.54)	1.5 (0.48)	1.5 (0.50)
Exposure to non-PIC depression practice activities
(Receptivity) Definitely need to change/improve management of patients with depression, %§	22.1	27.3	29.4	8.9	28.4
CME for depression, hours/past 3 years	7.7 (9.9)	7.3 (8.5)	7.8 (7.8)	7.8 (13.0)	7.6 (8.0)
QA for mental health care, hours/past year	0.69 (3.3)	0.93 (4.1)	0.98 (3.9)	0.17 (0.91)	.96 (4.0)
Read articles about major depression, no. times past/year‡	4.2 (4.4)	4.0 (4.1)	5.1 (5.7)	3.3 (2.5)	4.6 (5.0)
Consulted a mental health specialist about treating depression, no. times/past year	6.1 (5.9)	6.0 (6.0)	5.9 (5.8)	6.5 (5.9)	6.0 (5.9)
Detailed by a pharmaceutical company about depression medication, no. times/past year‡	14.0 (22.6)	16.6 (34.5)	12.7 (16.2)	12.8 (12.0)	14.5 (26.2)
Preauthorization requests for referral to mental health specialty, no. requests/past year	17.5 (23.9)	16.9 (25.4)	18.9 (23.6)	16.1 (23.3)	18.1 (24.2)
Preauthorization requests for SSRI antidepressants, no. requests/past year	16.2 (26.0)	16.1 (27.1)	16.1 (22.9)	16.4 (29.3)	16.1 (24.6)

^*Standard deviations in parentheses for continuous variables only.

^†Patient case mix data aggregated to clinician level.

^‡P < .001 for pooled interventions compared with usual care.

^§All comparisons except between QI-meds and QI-therapy were significant at P < .05. Otherwise, no other comparisons for other variables (3- or 2-group) were significant at P = .10 or below.

PIC indicates Partners in Care; CME, continuing medical education; QA, quality assurance; SSRI, selective serotonin reuptake inhibitor.

Effects on Clinician Knowledge

The interventions had positive effects on changing clinicians' knowledge about the treatment of depression. Table 2 shows the unadjusted knowledge measures preintervention and postintervention for all clinicians. At baseline, with the exception of one medication item (item 6), more clinicians gave correct answers for medication and general content questions than for psychotherapy questions. Although this pattern persisted at 18 months, when compared with medication or nonspecific depression care items, knowledge about psychotherapy had a greater increase. Between 8.4% and 14% more of the clinicians answered psychotherapy questions correctly compared with 0.6% to 6.5% more for medication questions and only 2.5% to 3.9% more for general depression questions. Aggregate scores for psychotherapy improved more, and the increases were significant for overall knowledge (7.2% increase in percentage correct), psychotherapy knowledge (17.8% increase), and for medication (5.2% increase). Clinicians appeared to learn the most about monitoring patients in psychotherapy (item 2), the goal of cognitive therapy (item 8), and the appropriate time to recommend psychotherapy as a stand-alone treatment (item 11). As for medication content, knowledge about clinician-prescribing patterns (item 5) decreased over time, although more clinicians knew about when to discontinue medications (item 9).

Table 3 shows the adjusted estimates for baseline and change in knowledge (post minus baseline) by intervention condition. We controlled for initial differences at baseline for the general treatment knowledge scale (with QI-therapy clinicians scoring significantly lower than usual care clinicians) in our change models. Compared with usual care clinicians, intervention clinicians had significantly greater increases in overall depression-related knowledge 18 months following program implementation. The overall knowledge measure differed significantly across groups with change scores near zero for usual care, 4.1 for QI-meds, and 10.5 for QI-therapy (F_2,42 = 3.37, P = .04 for the 3-group comparison and F_1,34 = 3.73, P = .06 for the 2-group test). The intervention effect sizes were 9.9% (P = .01) for QI-therapy and 3.5% (P = .32) for QI-meds. The amount of improvement relative to baseline was 14.5% for QI-therapy clinicians (equivalent to answering about 2 more questions correctly of 12 possible) and only 5.4% (less than 1 more question answered correctly) for QI-meds clinicians. Compared with usual care, clinicians in the intervention groups were more likely to correctly answer the questions about adding or switching to medication if psychotherapy is not effective in 6 weeks (item 2) and use of psychotherapy as the sole treatment for moderate depression (item 11).

Table 3.

Adjusted Clinicians' Knowledge by Intervention Condition: Baseline and Change^*

Knowledge Measure (% correct)	QI-Meds	QI-Therapy	Usual Care	All	M&U	P&U	M&P	I&U
Overall depression knowledge score (12 items)
Baseline value	75.7 (3.0)	73.5 (3.4)	75.3 (3.1)	.88	.92	.70	.63	.90
Post- minus prechange	4.1 (2.5)	10.5 (2.8)	.60 (2.6)	.04†	.32	.01†	.09†	.06†
General treatment score (3 items)
Baseline value	78.3 (4.2)	73.1 (4.7)	87.3 (4.4)	.09†	.13	.03†	.41	.04†
Post- minus prechange	−.35 (4.0)	9.1 (4.5)	1.7 (4.2)	.28	.72	.24	.12	.75
Antidepressant medication score (6 items)
Baseline value	81.3 (3.4)	80.2 (3.8)	78.0 (3.5)	.79	.50	.68	.84	.51
Post- minus prechange	2.6 (2.8)	7.1 (3.1)	.78 (2.9)	.33	.64	.14	.29	.30
Psychotherapy score (3 items)
Baseline value	62.2 (5.9)	59.3 (6.5)	56.2 (6.0)	.76	.46	.73	.73	.51
Post- minus prechange	12.7 (4.6)	19.5 (5.1)	−.99 (4.7)	.01†	.04†	.004†	.32	.006†

^*N = 109 for multivariate models. Clinic-level clustering accounted for in all estimates, standard errors (in parentheses), and tests. Two-sided P values for F-test comparison across all three intervention conditions shown in “All” column; 2-sided P values for pairwise comparisons shown in columns: M&U for QI-meds vs usual care; P&U for QI-therapy vs usual care; M&P for QI-meds vs QI-therapy; and I&U for interventions combined vs usual care. Adjusted for randomization blocks, demographics, training, depression caseload, patient case mix aggregated to the clinician level, and readiness to change.

^†P values < .10.

Examination of the knowledge subscales reveals large and statistically significant change scores for knowledge about psychotherapy but not for the medication or general treatment subscores. Clinicians in the QI-therapy group scored 19.5 points higher, clinicians in QI-meds group scored 12.6 points higher, and clinicians in usual care group scored 0.99 points lower (F_2,44 = 4.89, P = .01 for the 3-group test and F_1,34 = 8.75, P = .006 for the 2-group test). The effect of the intervention was a 13.6% greater increase for QI-meds and a 32.9% greater increase for QI-therapy compared with usual care. Across all knowledge scales, QI-therapy clinicians demonstrated more improvement relative to QI-meds and usual care clinicians. The overall knowledge measure was significant at P = .09.

Effects on Exposure to PIC Intervention Activities

Table 4 profiles the unadjusted effects of direct and indirect mechanisms that might explain differential change across intervention conditions. When clinicians received their baseline questionnaires, they were aware of being in intervention clinics and that their leaders had participated in training, although two thirds of them had not yet received intervention training. As noted above, while baseline knowledge did not differ much between experimental and control clinicians, clinician readiness to change did. Participation in clinician educational sessions differed significantly across intervention conditions. Nearly two thirds of the QI-therapy clinicians, compared with less than half of clinicians in the other groups, reported at least one individual session (χ²₂) = 9.70, P = .007) or group seminar (χ²₂) = 8.68, P = .03). About one third of clinicians in either the medication or psychotherapy conditions, compared with only 13.5% for care-as-usual clinicians, were aware that their clinic had made changes or implemented programs to improve depression care (χ²₂) = 10.40, P = .02) and knew the number of patients enrolled in the PIC study (χ²₂) = 9.49, P = .02).

Table 4.

Exposure to PIC Intervention Activities and Non-PIC Depression Practice Activities (at Follow-up) by Intervention Condition^*

Measure	N	QI-Meds	QI-Therapy	Usual Care	All	M&U	P&U	M&P	I&U
Exposure to PIC intervention activities
Any participation in clinician education about depression, % in past year
Individual sessions	158	35.4	57.9	30.2	.007†	.37	.003†	.04†	.02†
Group seminars or grand rounds	156	44.7	64.9	42.3	.03†	.85	.008†	.02†	.08†
Aware that clinic made changes or implemented programs to improve depression care, %	155	31.9	35.7	13.5	.02†	.01†	.001†	.54	.002†
Knew how many patients were enrolled in PIC, %	158	22.9	17.5	3.8	.02†	.002†	.019†	.34	.004†
Used practice resources for treating depression, % used “a little” or “a lot” in the last year
Clinical practice guidelines or manuals	159	69.4	66.7	54.7	.26	.25	.22	.98	.16
Feedback on patient outcomes	159	71.4	73.7	62.3	.40	.53	.30	.71	.32
Nurse assessment and follow-up	159	75.5	57.9	43.4	.004†	<.001†	.066†	.05†	.002†
Patient education brochures or videotapes	159	55.1	49.1	58.5	.61	.62	.43	.80	.45
Clinician education or training by local experts	159	63.3	70.2	56.6	.34	.49	.07†	.29	.13
Referral of depressed patients to mental health specialists	159	95.9	96.5	94.3	.61	.74	.59	.86	.60
Exposure to non-PIC depression practice activities
CME for depression, hours/past 3 years	156	5.5 (7.3)	4.7 (6.5)	4.1 (4.5)	.56	.86	1.00	1.00	.38
QA for mental health care, hours/past year	160	4.3 (14.3)	1.6 (5.0)	1.9 (6.6)	.26	.58	1.00	.37	.57
Read articles about major depression, no. of times past/year	157	5.7 (13.5)	3.5 (4.4)	4.2 (4.0)	.37	1.00	1.00	.50	.83
Consulted a mental health specialist about treating depression, no. of times/past year	157	6.8 (10.8)	7.6 (13.4)	4.3 (3.6)	.23	.70	.29	1.00	.09†
Detailed by a pharmaceutical company about depression medication, no. of times/past year	157	13.6 (13.4)	9.1 (10.5)	12.8 (16.4)	.18	1.00	.47	.26	.48
Preauthorization requests for referral to mental health specialty, no. of requests/past year	153	7.9 (16.5)	7.4 (10.8)	12.5 (21.3)	.24	.54	.35	1.00	.09†
Preauthorization requests for SSRI antidepressants, no. of requests/past year	153	8.8 (20.3)	12.3 (21.2)	16.2 (23.4)	.27	.30	1.00	1.00	.15

^*N = 109 for multivariate models. Standard errors in parentheses for continuous variables only. Two-sided P values for χ² test (for binary variables) or F-test (for continuous measures) comparing across all three intervention conditions shown in “All” column; 2-sided P values for pairwise comparisons shown in columns: M&U for QI-Meds vs usual care; P&U for QI-therapy vs usual care; M&P for QI-Meds vs QI-therapy; and I&U for interventions combined vs usual care.

^†P < .10.

PIC indicates Partners in Care; CME, continuing medication education; QA, quality assurance; SSRI, selective serotonin reuptake inhibitor.

There were significant differences by intervention condition in the percentage of clinicians reporting use of two key components of the multifaceted interventions. Clinicians in the intervention conditions reported significantly higher rates of using the nursing assessment and patient follow-up resources (χ²₂) = 13.11, P = .004), with only 43.4% of clinicians in usual care relative to 57.9% and 75.5% of clinicians in QI-therapy and QI-meds interventions, respectively. Although there were no overall differences by intervention condition, we did find significantly more clinicians in QI-therapy reporting participation in clinician education and training sessions by local experts (χ²₂) = 3.29, P = .07), with only 56.6% of usual care clinicians reporting use relative to 70.2% for QI-therapy and 63.3% for QI-meds. With two exceptions, we observed no differential use of clinical practice guidelines for depression, feedback on standard patient outcomes, patient educational materials, or referral to mental health specialists for individual patients.

Effects on Non-partners in Care Depression Practice Activities

We found no differences by intervention status in any of the indirect measures of professional practices, including CME, quality assurance activities, extent of participation in academic detailing by pharmacy companies for antidepressants, or requests for treatment preauthorization (Table 4), with two exceptions. Clinicians in the intervention groups (QI-meds and QI-therapy combined) were significantly more likely to consult mental health specialists for help in treating depression (P = .09) and less likely to request preauthorization for mental health specialty referrals (P = .09).

DISCUSSION

We found that clinicians exposed to multifaceted QI programs for depression in managed primary care practices gained knowledge about assessing and treating depression over 18 months following implementation. Furthermore, we found that the main activity accompanying this gain in knowledge was direct participation in the specific types of formal and informal educational activities that were part of the study protocol. Despite the availability of detailed clinician manuals, which were thoroughly reviewed by local leaders, control and intervention clinicians did not differ in their reported use of written guideline-based material. It is possible that the reinforcement of the materials through verbal presentations, consultations, and interactions with study-trained local leaders improved knowledge.

While change in overall knowledge of depression treatment improved modestly (3% to 9%), change in psychotherapy knowledge scores changed more substantively (13% to 30%) for the QI groups relative to usual care. Change of this magnitude (increases by 20% for medication and 33% for psychotherapy) is substantial, especially considering that the duration (18 months) was longer than previously demonstrated for QI programs on depression.⁷

Across intervention arms, knowledge about psychotherapy improved more than knowledge about medication management. This finding may reflect the fact that in medical training programs, psychotherapy is a content area not covered as well as medication management, leaving more room for improvement, and is consistent with previous findings of more significant improvements in knowledge for topics that physicians knew the least about initially.⁷ In all three arms of the study, baseline scores for medication knowledge were consistently higher than for psychotherapy knowledge. This differential mastery of basic knowledge in the two areas may have introduced a measurement effect. Clinicians in general were closer to the upper limit of our measure of basic knowledge (the “ceiling” of our knowledge score). We may have needed a measure reflecting a higher level of sophistication in medication use to fully detect our intervention effect for medications. In terms of fostering learning about psychotherapy, we found that contact with psychologists in the QI-therapy group had more effect than did contact with the psychiatrist local leader in the QI-meds group. This makes sense because the psychiatrist involvement focused more on assisting with medication management than with psychotherapy. Collaborative care activities may also have been greater in the QI-therapy group because more therapists were involved in delivering CBT compared to the more limited interaction between primary care physicians and the one psychiatrist local leader in the QI-meds group.

This finding is more consistent with a social network framework²⁰ rather than a social influence¹⁹ explanation as initially hypothesized. In addition, clinicians' experience with more patients sustaining improvement in the psychotherapy-resource intervention (C.D. Sherbourne, PhD., 1999, unpublished findings) could have reinforced greater attention to information about treatment. Furthermore, we think that after enrolling and learning of their intervention status, intervention clinicians were more attentive and receptive to change, as evidenced by the baseline difference in readiness for change, measured just about or shortly after clinician orientation sessions began. Consistent with stage models of change through awareness,³² the observed anticipation and attention to change early in the intervention conditions could have enhanced the learning process and facilitated the dissemination and full implementation of the QI programs throughout the intervention period.³³

Our study, one of a few longitudinal studies of provider change, is limited in several ways. The sample of primary care clinicians was moderate in size (N = 160), although typical for studies of this type, in which clinicians are enrolled as part of a larger evaluation involving primary data collection through mail questionnaires. Contamination across intervention arms in exposure to clinician education materials was possible, since randomization was at the clinic level within the same organizations. Implementation was up to practices, not the study investigators, and some patients could have crossed arms during the study. However, this would suggest that our estimates of intervention effects on clinician knowledge may be conservative. We are also uncertain about the extent to which the observed effects were due to engagement in the specific PIC intervention activities (clinician education about depression treatment and reinforcement from intervention staff) or merely due to general increased attention to improving depression care. However, our description of implementation suggests that the intervention groups differed specifically in exposure to intervention components. Furthermore, blinding in this study was partial. Clinicians did not know of their intervention assignment until after enrollment, but some completed their baseline survey after this point. Clinician outcomes (knowledge) were measured by self-report and thus were not blinded to intervention status. Clinicians in the intervention conditions were informed of their participating patients, while control clinicians were not (but patients in all conditions were allowed to inform their providers of study enrollment). Finally, differential readiness to change across groups may bias results; however, we adjusted for this in our multivariate analyses, and we suspect this finding represents an early intervention effect rather than a true prebaseline difference.

In conclusion, the multifaceted PIC intervention programs improved clinicians' knowledge about the treatment of depression, particularly knowledge of psychotherapy, relative to usual care over 18 months of follow-up in a diverse sample of managed care organizations. This finding is consistent with previous PIC findings of improved quality of care and health outcomes for patients over the first follow-up year.³⁴ Although insufficient to improve outcomes alone,^6,11,12 increasing clinician knowledge is one important part of changing clinical practice to achieve stable improvement in depression care. If clinicians are still more knowledgeable about depression treatment 18 months after initial training, then they may be able to implement, with some resource support, more sustainable changes in practice patterns—an issue for future studies to address, as the supplemental resources for care did not extend beyond 6 to 12 months in PIC. Future studies should also focus on clarifying the specific elements of such a multimodal program that improve clinician knowledge and have an impact on patient outcomes.