Teaching Motivational Interviewing to Primary Care Staff in the Veterans Health Administration

ABSTRACT

BACKGROUND

The Veterans Health Administration (VHA) is implementing the patient-centered medical home (PCMH) model of primary care which emphasizes patient-centered care and the promotion of healthy lifestyle changes. Motivational Interviewing (MI) is effective for promoting various health behaviors, thus a training protocol for primary care staff was implemented in a VHA health care setting.

OBJECTIVES

We examined the effect of the training protocol on MI knowledge, confidence in ability to use MI-related skills and apply them to written vignettes, perceived comfort level and skill in lifestyle counseling, and job-related burnout.

DESIGN

Training was provided by experts in MI. The training protocol consisted of three sessions—one half day in-person workshop followed by a 60-minute virtual training, followed by a second workshop. Each of the sessions were spaced two weeks apart and introduced trainees to the theory, principles, and skills of using MI in health care settings.

PARTICIPANTS

All primary care staff at the Veterans Affairs Palo Alto Health Care System were invited to participate.

MEASUREMENTS

Trainees completed a short set of questionnaires immediately before and immediately after the training.

RESULTS

We found support for our primary hypotheses related to knowledge, confidence, and written responses to the vignettes. Changes in perceived comfort level and skill in lifestyle counseling, and job-related burnout were not observed.

CONCLUSIONS

Training primary care staff in MI is likely to become increasingly common as health care systems transition to the PCMH model of care. Therefore, it is important for health care systems to have low-cost methods for evaluating the effectiveness of such trainings. This study is a first step in developing a brief written assessment with the potential of measuring change in a range of behaviors and skills consistent with MI.

Electronic supplementary material

The online version of this article (doi:10.1007/s11606-012-2016-6) contains supplementary material, which is available to authorized users.

INTRODUCTION

Within recent years the Veterans Health Administration (VHA) has begun transitioning primary care toward a team-based model of care known as the Patient-Centered Medical Home (PCMH; also known as patient aligned care teams within VHA). PCMH emphasizes enhanced coordination among staff, evidence-based tools for supporting patients in making healthy lifestyle changes, and patient-centeredness.^1,2

This initiative has historical significance in that it is the first time a health care system with over 1,000 care sites has attempted to implement this primary care model.² The VHA is interested in leveraging the many advantages of the PCMH model to help enhance the patient-centered nature of primary care services. VHA is aiming to complete the PCMH transition in 80% of clinics in 2012, with the adoption of this model in all VA primary care clinics by 2015.²

Prior studies on PCMH have included a smaller number of demonstration sites, ranging from a single health care site¹ to multiple sites across the U.S. ^3,4For example, researchers at Group Health in the state of Washington evaluated the PCMH model in one of their 20 primary care clinics.¹ Several reforms were made to the demonstration site, including collocating care team members, improving access to care, and collaborative care planning. Compared to the non-demonstration site, patients of the PCMH clinic reported significantly better experiences with their care in terms of doctor–patient interactions and increased access to care, while providers reported a significant reduction in work-related emotional exhaustion. Larger demonstrations have replicated these findings and found that the PCMH model is associated with improved care coordination and chronic disease management, as well as fewer hospitalizations and visits to the emergency department.³

Given the emphasis of the PCMH model on delivering patient-centered care and supporting patients in making healthy lifestyle changes,¹ it is critical to develop a clinical culture that supports these care principles. Thus, our facility chose to implement motivational interviewing (MI) to help support this transition. MI is a patient-centered clinical style designed to help patients identify and resolve discrepancies between their actual and desired behavior, and to help enhance their motivation to make healthy lifestyle changes.⁵ Research shows that MI is effective in supporting patients in changing various lifestyle behaviors including reducing substance use, increasing treatment engagement, and making improvements in dietary habits and chronic disease management.^6,7

Prior studies demonstrate the positive effects of teaching MI to medical students^8–12 and primary care staff. ^13–15 Studies with these samples show that brief training protocols in MI can lead to learning basic MI concepts and increased confidence in applying MI-related skills to various patient problems.^9,10,13 One recent study showed primary care providers receiving a brief, initial one and one-half day course in MI plus two half-day follow-up courses during the year were more likely to report using learned strategies in practice when compared to a no-training control group.¹³ Findings from the one-year follow-up showed that those in the MI training condition improved type 2 diabetic patients’ understanding of their illness and factors that may complicate and improve the disease process, and motivation to engage in diabetes management.¹⁶

The present study is, to our knowledge, the first evaluation of an MI training involving a wide range of primary care staff at a large VA hospital. We hypothesized that primary care staff participating in the training would demonstrate increased knowledge of MI concepts, increased use of and confidence in ability to engage in MI-related skills, and improved ability to apply MI-related skills to analogue written clinical vignettes at post-training. In addition, in light of findings showing that primary care providers report MI to be both a more practical counseling style than traditional advice giving and result in improvements in the doctor–patient relationship,¹³ we examined whether our training protocol might also improve trainees’ comfort level and perceived skill in lifestyle counseling, and decrease job-related burnout.

METHODS

Participants and Setting

Primary care staff employed at the Veterans Affairs Palo Alto Health Care System (VAPAHCS) were invited to participate. Participants included staff employed at the three inpatient facilities and seven outpatient clinics. The VAPAHCS serves more than 85,000 enrolled Veterans and operates nearly 900 beds. Participants included prescribing providers (e.g., physicians, nurse practitioners, physician’ assistants), nurses (e.g., registered nurses), mental health providers, and administrative/clerical staff. Staff not fitting into these categories (e.g., pharmacists) was labeled as “other.”

Training Protocol and Curriculum

The training protocol consisted of six sessions in two phases. Phase one consisted of three sessions—a half-day, in-person workshop, followed by a 60-minute virtual training, followed by a second, half-day, in-person training workshop. Each session was spaced two weeks apart and introduced trainees to the theory, principles, and skills of using MI in health care settings (Text Box 1). The present study is an evaluation of this initial phase of the training protocol. Training was provided by experts in MI and members of the Motivational Interviewing Network of Trainers.

Curriculum Evaluation

We evaluated phase one of the training protocol curriculum using a pretest-posttest design. Due to the scope of the training which consisted of offering training to all primary care staff, we were asked by clinic administrators to limit the time needed to complete paperwork. We asked trainees to complete a relatively short set of questionnaires immediately before and immediately after the training. We assessed: job-related characteristics; knowledge of MI and confidence in supporting lifestyle change; use of MI skills; application of MI skills to written clinical situations; perspectives on lifestyle counseling; and job-related burnout (Appendix is available online).

Job-Related Characteristics

Three questions assessed prior experience with MI (yes/no), job title, and length in current professional role at the VA.

MI Knowledge and Confidence

A three-item multiple-choice test was used to measure understanding of MI principles and strategies. One point was given for each item answered correctly. A single item assessed confidence in building patients’ inner motivation to engage in lifestyle change, using a scale of 0 (not confident at all) to 10 (complete confidence).

Use and Application of MI Skills

We developed a 12-item measure of various MI skills (e.g., resisting the desire to “fix” the patient by “telling them what to do”) and asked trainees to indicate on a scale of 0 (low) to 10 (high) the extent to which they engage in these behaviors. We followed the general approach described by Miller, Hedrick, & Orlofsky¹⁷ and developed seven written vignettes depicting various clinical situations (Text Box 2) to assess skill application.

Perspectives on Lifestyle Counseling

An eight-item assessment measured trainees’ perspectives on lifestyle counseling (adapted from Jallinoja et al.).¹⁸ Trainees indicated on a 5-point scale (totally disagree = 1 to totally agree = 5) the extent to which they agreed with statements expressing views about their role(s) in helping patients make important lifestyle changes (e.g., “My task is to give information on lifestyle-related risks”). Two subscales emerged from the measure—the responsibility and uneasiness scales—which have strong internal consistency, α = 0.83 and α = 0.87, respectively.

Job-Related Burnout

Two items from the Maslach Burnout Inventory¹⁹ were used to assess job-related burnout - emotional exhaustion (EE; “I feel burned out from my work”) and depersonalization (DP; “I have become more callous toward people since I took this job”). These items have shown high correlations with the full Maslach Burnout Inventory and are predictive of burnout in large samples of medical professionals.²⁰

Coding Protocol for Vignette Responses

Two coding systems were used to assess the quality of the written responses to the vignettes - The Helpful Responses Questionnaire¹⁷ (HRQ) and our own written evaluation tool. The HRQ is a brief assessment of empathy consisting of six written vignettes. Each response is scored using a 5-point ordinal scale measuring the quality of the reflection (1 = no reflection plus a comment that can interrupt the flow of conversation, 5 = a high quality reflection). We utilized the scoring system of the HRQ to evaluate the quality of reflections provided by trainees to the first two vignettes.

We developed a second coding system to evaluate written responses to the remaining clinical vignettes. The coding system was drawn from and guided by elements of the Motivational Interviewing Treatment Integrity²¹ (MITI) scale, which is widely utilized for evaluating the use of MI skills in the context of audio-taped sessions. However, collecting audiotapes from trainees in the present study was not feasible for a variety of reasons (e.g., desire for a more naturalistic training and evaluation model). The coding system largely mirrored the specific behavioral counts defined in the MITI and allowed the assignment of six codes: Giving Information (GI), MI Adherent (MiA), MI Non-Adherent (MiNa), Open Question (OQ), Closed Question (CQ), and Reflection (R).

Reliability of Coding System

The same three raters (blind to pre-/post-assessment) were used throughout the study. Two exercises were used to assess inter-rater reliability. First, we used a simple comparison method for comparing coding responses across the three raters. When discrepancies arose, vignettes were discussed until a consensus was reached, when needed additions were made to the coding system to clarify the guidelines for coding responses. In the second phase, raters were assigned 25% of the total sample of vignettes to establish reliability. An intra-class correlation of .93 was observed among the raters which showed a high degree of inter-rater reliability for the vignettes assessing reflections (vignettes 1 and 2). Kappas ranged from 0.61 to 0.96 for the MITI codes on the remaining vignettes, indicating substantial to almost perfect agreement among the three coders²² for the remaining vignettes. Once inter-rater reliability was established, the remaining portion of vignettes were coded.

RESULTS

Participant Characteristics

A total of 229 primary care personnel completed the pre-training assessment (Fig. 1). The majority of participants were prescribing providers and nurses (66%) and reported no prior experience with MI (Table 1). Of the 229 participants enrolled at baseline, 160 (70% of total sample) completed the post-training assessment. No significant differences between those who completed and did not complete the post-training assessment were observed on any baseline characteristics or pre-test assessments.

Figure 1.

Flowchart of primary care staff participating in the study.

Table 1.

Baseline Characteristics of Trainees

	Total Sample (n = 229)			Completers (n = 160)
		Min	Max		Min	Max
Participant Characteristics
Prior MI Experience, n, (%)	84 (37)			64 (40)
Job Title, n, (%)
Prescribing Provider	59 (26)			47 (29)
Nurse	91 (40)			69 (43)
Mental Health	30 (13)			19 (12)
Administrative/Clerical	27 (12)			17 (11)
Other	22 (10)			8 (5)
Years in Role, M (SD)	9.33 (8.91)	0	41	9.60 (9.00)	0	41
Dependent Variables M (SD)
MI Knowledge	1.51 (.86)	0	3	1.50 (0.82)	0	3
MI Confidence	5.57 (1.74)	0	10	5.61 (1.67)	2	9
MI Skill
Use	6.41 (1.48)	2.17	10	6.34 (1.49)	2.17	9.83
Vignettes
Reflections (quality)	1.49 (.81)	1	5	1.47 (0.81)	1	5
Open Questions (freq)	1.48 (1.01)	0	3	1.43 (0.98)	0	3
Behavior Counts (freq)
Giving Information	1.26 (1.14)	0	5	1.28 (1.13)	0	5
MI Adherent	0.42 (0.76)	0	4	0.44 (0.77)	0	4
MI Non-Adherent	0.83 (1.00)	0	5	0.84 (0.97)	0	5
Open Question	0.96 (1.13)	0	5	0.94 (1.08)	0	4
Closed Question	1.12 (1.21)	0	6	1.07 (1.15)	0	6
Reflection	0.04 (.21)	0	2	0.04 (0.22)	0	2
Lifestyle Counseling, M (SD)
Responsibility	3.82 (.87)	1	5	3.87 (0.82)	1	5
Uneasiness	2.27 (1.13)	1	5	2.24 (1.14)	1	5
Time Commitment	3.09 (1.27)	1	5	3.04 (1.18)	1	5
Job-related Burnout, M (SD)
Emotional Exhaustion	2.66 (1.70)	0	6	2.59 (1.70)	0	6
Depersonalization	1.50 (1.51)	0	6	1.37 (1.38)	0	6

* = p < 0.05, ** = p < 0.01, *** = p < 0.001

Dependent Variables

MI Knowledge and Confidence

A paired-samples t-test was used to examine pre-post change on two independent variables—MI knowledge and confidence to apply such skills. Our findings revealed a statistically significant increase on the three-item MI knowledge test, p < 0.001 (effect size, d = −77) and single item assessing confidence to build patients’ inner motivation to change, p < 0.001 (d = −0.64) from baseline to post-training (Table 2).

Table 2.

Baseline and Post-training Outcomes for Completers (N = 160)

	Assessment		t	df
	Baseline M, SD	Post-training M, SD
MI Knowledge	1.50 (0.83)	2.21 (0.75)	−9.55***	152
MI Confidence	5.61 (1.68)	6.59 (1.43)	−7.93***	156
MI Skills
Use	6.29 (1.51)	7.08 (1.47)	−5.63***	142
Vignettes
Reflections (quality)	1.48 (0.83)	2.98 (1.23)	−13.96***	139
Open Questions (freq)	1.46 (0.98)	1.88 (1.01)	−4.49***	128
Behavior Counts (freq)
Giving Information	1.29 (1.12)	0.85 (.89)	4.57***	153
MI Adherent	0.44 (.78)	1.22 (1.17)	−8.24***	153
MI Non-Adherent	0.84 (.97)	0.40 (0.74)	5.36***	153
Open Question	0.95 (1.09)	1.26 (1.23)	−2.65**	153
Closed Question	1.07 (1.16)	0.82 (1.19)	2.10*	153
Reflection	0.03 (.16)	0.21 (0.51)	−4.49***	153
Lifestyle Counseling
Responsibility	3.88 (.80)	3.91 (0.78)	-0.66	153
Uneasiness	2.25 (1.14)	2.26 (1.11)	-0.09	153
Time Commitment	3.04 (1.17)	3.10 (1.24)	-0.56	152
Job-related Burnout
Emotional Exhaustion	2.57 (1.70)	2.46 (1.66)	0.99	148
Depersonalization	1.38 (1.39)	1.17 (1.33)	1.71	142

* = p < 0.05, ** = p < 0.01, *** = p < 0.001

MI Skills - Use

As predicted, we observed statistically significant pre-post assessment increases in mean scores, p < 0.001 (d = −0.47) on a 12-item measure to assess self-reported use of MI skill.

MI Skills - Application (Vignettes)

Reflections (Vignettes 1–2) and Open Questions (Vignettes 3–4)

We observed statistically significant increases in the quality of reflections as measured by the HRQ provided for vignettes 1 and 2 from baseline to post-training, p < 0.001 (d = −1.22). We obtained similar results for the frequency of open questions on vignettes 3 and 4 from baseline to post-training, p < 0.001 (d = −0.40). These results suggest improvement in trainees’ ability to generate a higher quality reflection and open questions for written vignettes over the course of the first phase of training.

Behavior Counts (Vignettes 5–7)

As expected, the frequency of MI adherent responses, p < 0.001 (d = −0.69), open questions, p < 0.01 (d = −0.22), and reflections, p < 0.001 (d = −0.40) increased significantly from baseline to post-training. In contrast, the use of MI non-adherent responses, p < 0.001 (d = 0.44), closed questions, p < 0.05 (d = 0.17), and responses coded as giving information, p < 0.001 (d = 0.38) significantly decreased. The findings support our prediction that trainees’ would improve their ability to apply MI-related skills to appropriate clinical situations over the course of training (while decreasing their use of strategies inconsistent with the MI approach).

Perspectives on Lifestyle Counseling and Job-Related Burnout

The results of our analyses indicated no significant changes on the responsibility or uneasiness subscales (or time commitment item) nor on the measure of burnout. However, examination of the individual items of the former measure showed a small but significant baseline (M = 3.48, SD = 1.00) to post- training (M = 3.68, SD = 0.94) increase on a single item (“I have been able to help many of my patients to change their lifestyle to a healthier one”), t (153) = −2.85, p < 0.05, d = −0.23. Two additional items appeared to be trending toward significance (“My task is to give information on life-style related risks,” and “I have sufficient skills for lifestyle counseling”), p < 0.10, providing inconclusive evidence that the initial training protocol had any impact on trainees’ perspectives toward lifestyle counseling.

DISCUSSION

To our knowledge, this is the first study we are aware of that has evaluated a protocol to teach MI to an interdisciplinary group of VHA primary care staff. Our findings are consistent with prior studies showing that brief MI training protocols can lead to increased provider knowledge and confidence, and self-reported usage of MI skills in clinical practice.^9,10,13 This study also extends this literature by examining a novel approach to assessing the impact of brief MI training on a broad range of MI-consistent behaviors.

MI training studies with medical students often utilize patient simulations¹² or standardized patients¹¹ to measure behavior change. Both offer advantages in terms of measuring training outcomes, including a more realistic environment to demonstrate skill acquisition, and an assessment of skill from the patient’s perspective.⁸ However, these assessments can be relatively resource intensive requiring time needed for completing simulations and resources for audio taping/transcribing provider–patient encounters. Given the potential resource commitment of these measurement strategies, it may be beneficial for some health care systems to have a range of options for measuring change that are lower in cost and respondent burden.

The present study is a first step in developing such an assessment that has the potential of measuring change in a range of behaviors consistent with MI. Our findings show that providers completing the phase one of the training protocol improved their ability to generate MI-consistent responses to seven written vignettes. For example, we found improvements in the quality of reflective statements and frequency of open questions from pre- to post-training. Both strategies can improve providers’ ability to listen and may increase the efficiency and effectiveness of interactions with patients.⁵

We also observed reductions in written responses focused purely on providing information to patients. This suggests that providers may have improved their ability to educate patients in an MI-consistent manner by asking permission prior to providing information. Participants also demonstrated a reduction in closed questions which are, from an MI perspective, considered to be less efficient strategies for information gathering and engaging the Veteran.⁵ Our findings suggest that these vignettes may be sensitive to changes in trainees’ understanding of MI and their ability to apply related strategies.

A third objective of this study was to explore whether the MI training impacted outcomes that are thought to be promoted by the PCMH model, namely perceived ability and responsibility of staff to support lifestyle change, and a reduction in provider burnout.^1,2 In general, our training did not impact these outcomes. However, we found weak but inconclusive evidence suggesting a possible impact of training on an increased perceived ability and responsibility for supporting patients in making lifestyle change. These results may reflect the already high level of perceived responsibility and low level of uneasiness and burnout reported by staff at baseline, thus limiting the degree to which the training could impact these outcomes. It is also possible that the follow-up period used in the study was not long enough to capture changes in these variables.

This study has several limitations. First, a control group was not used. Therefore, we cannot determine whether the observed findings are the result of the training protocol or other factors, such as other initiatives to support the transition to PCMH. Second, the relatively short follow-up period does not allow us to determine whether any training gains will be maintained over time or whether such gains have an effect on patient outcomes such as satisfaction, use of care, and/or treatment adherence. Third, given time and resource constraints, we were unable to collect data on providers’ actual behavior over the course of the training. Thus, we are unable establish the validity of the written vignettes as an indicator of actual behavior change. This step is critical to determine if such a strategy is to be used as a proxy for measuring actual behavior change. Third, our assessment was brief, which resulted in the gathering of little information of constructs such as perspectives towards lifestyle counseling and job-related burnout. Future studies may wish to incorporate more comprehensive assessments of these constructs. Furthermore, detecting changes in these variables may require longer follow-up periods than used in the present study. Fourth, this evaluation included staff from a single VA health care system, which prohibits us from generalizing our findings to other VA hospitals. Larger, more representative samples of staff are needed to determine whether our training protocol may lead to such improvements for the larger population of VA primary care staff.

Conclusions

This study is unique in that it examined the impact of phase one of an MI training protocol on an interdisciplinary group of VHA primary care staff. We found considerable improvements in staff knowledge about MI, their confidence in applying MI to patient lifestyle issues, and ability to apply MI strategies to clinical vignettes. Our study also used written vignettes to assess improvements in a broad range of MI skill. Training primary care staff in MI is likely to become increasingly common as health care systems transition to the PCMH model of care. Therefore, it is important for health care systems to have low-cost methods for evaluating the effectiveness of such trainings. Before such measurement strategies are adopted, it is critical to determine whether written responses accurately reflect staff behavior.