Measuring Client Experiences of Motivational Interviewing during a Lifestyle Intervention

Abstract

The Client Evaluation of Motivational Interviewing was used to assess MI experiences in a predominantly female, African American sample from the Southeastern U.S. who received MI-based feedback during a multi-component lifestyle intervention. MI was experienced differently than a primarily White, male, Northeastern mental health sample.

The assessment of client experiences has been very valuable in evaluating counseling sessions (e.g., Soderlund, 2009). As such, several measures have been developed to assess various aspects of the counseling experience from the client perspective. For instance, measures have been developed to assess the working alliance between client and counselors (Horvath & Greenberg, 1989), the nature of the counselor-client relationship (Kelley, Gelso, Fuertes & Marmarosh, 2010) and counseling outcomes (Frey, Beesley, & Liang, 2009). Additionally, the assessment of client experiences in counseling has been related to outcomes (Bethea, Acosta, & Haller, 2008) and has facilitated clinician skill development (Soderlund, 2009). Likewise, within the context of motivational interviewing (MI), several qualitative studies have demonstrated the ability of clients to successfully evaluate the nuances associated with MI as a counseling approach (Angus & Kagan, 2009; Marcus, Westra, Angus, & Kertes 2011).

The Client Evaluation of Motivational Interviewing scale (CEMI) is a measure developed to provide the client’s perspective of his or her experience of MI (Madson, Bullock, Speed, & Hodges, 2009). MI is a counseling approach that is person centered, collaborative and focused on strengthening a client’s internal motivation and commitment to change (Miller & Rollnick, 2013). Miller and Rose (2009) suggested that MI facilitates change through a combination of relational and technical components. The relational component of MI encompasses an empathic, affirming, non-judgmental approach that is autonomy-supporting and intended to create a safe environment. Built on this relational foundation, the technical component of MI involves using strategies aimed at increasing clients’ talk about change and reducing their talk about not changing (Miller & Rose, 2009).

In the initial validation work, the factor structure for the CEMI was evaluated using exploratory and confirmatory factor analysis in a randomly split sample of 500 predominantly White (59%) male (53%) clients receiving acute inpatient psychiatric treatment in urban and suburban hospitals in the Northeastern United States (Madson et al., 2013). The two factor solution found in this sample suggested that clients were able to distinguish relational and technical aspects of MI, which is consistent with published descriptions of MI (Miller & Rose, 2009). Although initially developed to provide an assessment of clinician MI fidelity that did not rely on observational coding (see Madson & Campbell, 2006), this instrument also shows promise as a measure of the client experience of MI. The CEMI captures client rather than provider or expert coder impressions of the salient features of an MI session (e.g., collaboration and focus on change talk).

Originally developed as an alternative counseling approach to working with problem drinkers (Miller, 1983), the efficacy of MI has been extended to a wide array of behaviors ranging from abuse of other substances (Hettema, Steele, & Miller, 2005), eating disorders (Macdonald, Hibbs, Corfield, & Treasure, 2012), other mental health problems (Westra, Aviram, & Doell, 2011) and promoting positive health behaviors (Martins & McNeil, 2009). In addition to compelling and ever-increasing evidence for the broad applicability and efficacy of MI across settings and problem behaviors, there is also evidence to support the efficacy of MI for individuals from different racial and ethnic backgrounds (Hettema et al., 2005; Lundahl Tollefson, Kunz, Brownell, & Burke., 2010; Miller et al., 2008). Nonetheless, it remains unclear whether individuals from minority backgrounds, who receive MI for diverse problems and in diverse settings, have the same subjective experience of their MI sessions as individuals from the majority culture.

The negative experiences of African American clients in relation to interactions with health care workers have been well established (Broman, 2012; Peek, et al., 2013; Peek, Tang, Cargill, & Chin, 2011). For instance, Ratanawongsa, Zikmund-Fisher, Couper, Van Hoewyk, and Powe (2010) highlighted that African American individuals report less shared decision making in regard to their health status than White individuals. Peek and colleagues (2010) found that factors such as mistrust of providers, negative attitudes about health care and internalized racism influenced African American individuals’ involvement in communication and shared decision making. Specifically, these researchers found that African American individuals were less forthcoming about health information, acquiesced more to the provider, and were less likely to adhere to the prescribed treatment plan. In part, these differences may be related to traditional health care approaches that may neglect client goals, values, and cultural influences in relation to behavior change solutions. Thus, the emphasis in MI on collaborating and eliciting motivations and solutions from clients may improve African American individuals’ experiences with health care workers. Consistent with this speculation, Lundahl and colleagues (2010) suggested that individuals who have been marginalized may find MI appealing compared to traditional approaches.

The CEMI has to date only been examined in a mainly White non-Hispanic male population with co-occurring substance abuse and mental health problems there is a need for further evaluation of the CEMI and to better understand its functioning with diverse populations, Thus, the purpose of the current study was to estimate the factorial validity of the CEMI in a sample that differed from the validation sample on the dimensions of race, gender, treatment condition, and geographic location. In particular, the goal of this study was to better understand the psychometric functioning of the CEMI in a southern, primarily African American female sample, participating in a community based, MI enhanced, healthy lifestyle intervention focused on lowering blood pressure through diet and physical activity.

Method

Participants and Procedure

Participants (n=269) who completed the CEMI assessment were mainly female (85%) and African American (94.4%). The average age was 43.84 (SD = 12.13) years. All participants were non-mental health, community based individuals from a mid-sized city located in the Southeastern US who self-selected to participate in the lifestyle intervention. The CEMI data were collected as part of a larger, community based participatory research healthy lifestyle intervention, HUB City Steps (HCS). The methodology as well as secondary aims are further described elsewhere (Anderson-Lewis et al., 2012; Zoellner et al., 2011). The CEMI data reported here were obtained during the initial baseline assessment. After completing the assessment, participants participated in a motivational enhancement (ME) session where assessment feedback about health indicators (e.g., blood pressure) was discussed and a behavioral change plan was developed. The ME session was the only behavior change focused component at assessments. The CEMI items were a part of a series of questionnaires presented orally by trained research assistants immediately after the ME sessions. Measures were administered orally to account for any reading and comprehension concerns. This is different than the Madson et al., 2013 study in which measures were self-report as the CEMI is intended as a measure to be completed by clients. All procedures were approved by the University’s Institutional Review Board.

Motivational Enhancement Sessions

The ME sessions focused on building motivation to improve participants’ diet and physical activity behaviors. Participants received personalized feedback about various health factors such as weight, blood pressure, cholesterol, body mass index, and diet and were provided the opportunity to choose which health areas they wanted to discuss with a MI trained health coach. The goals of this interaction were to: build internal motivation among participants to improve diet and exercise behaviors and to develop an individualized change plan that they could implement. Sessions concluded with the health coach eliciting participant commitment to the change plan. Sessions were provided by five doctoral level psychology graduate students and three master’s level registered dietitians who received 24 hours of direct training and who were supervised by the first author, a member of the Motivational Interviewing Network of Trainers (Madson, Landry, Molaison, Schumacher, & Yadrick, in press). The average age of health coaches was 28.1 (SD = 4.7), seven were female, two were African American and six were White. There was an average of 2.1 (SD = 1.9) years of experience providing clinical services (Zoellner et al., 2011).

Client Evaluation of MI (CEMI)

The CEMI is a 16 item self-report measure aimed at assessing client perceptions of clinician use of MI across two factors. The CEMI also may provide a way to assess the quality of MI delivered to clients. Typically, the CEMI is a self-administered tool given to the client following an MI based counseling session and asks the client to rate the degree to which a counselor exhibited specific MI-related behaviors (Madson et al., 2009). Participants use a four point Likert-type scale (1 = not at all to 4 = a great deal) to rate the degree to which the MI counselor demonstrated each of 16 behaviors during their most recent session. Behaviors rated include, “focus on your weaknesses,” “help you talk about changing your behavior,” and “help you feel hopeful about changing your behavior.” Negative items are reverse scored and higher CEMI scores represent more MI consistent behavior.

As noted previously, the factor structure for the CEMI was established through an exploratory and confirmatory factor analysis with a split sample of 500 predominantly White (59%) male (53%) clients receiving acute inpatient psychiatric treatment in urban and suburban hospitals in the Northeastern United States (Madson et al., 2013). Results suggested that a two factor solution explained 51.1% of the cumulative variance. The two CEMI factors (relational and technical) were labeled on the basis of commonality among items after reviewing each item and its consistency with published descriptions of MI (Miller & Rose, 2009). Of note, items loading on the relationship factor tended to be the reverse scored items, such that a high score on the relationship factor reflected an absence of MI-inconsistent behaviors whereas items on the technical factor tended to reflect provider use of MI-consistent strategies to elicit and strengthen motivation for change. A weak inverse correlation was found between the two subscales with the split sample r = −.16 & −.17 respectively (Madson et al., 2013). The internal consistency ranged from .88 (relational) to .91 (technical). Thus, the CEMI yields a total score and two subscale scores (relational and technical) with higher scores indicating increased consistency with MI (Madson et al., 2013).

Results

The means from the 16 CEMI items in the confirmatory factor analysis by Madson et al. (2013) were compared using t-tests with means from the current study, n=225 and n=269 respectively. In order to better inform potential item revisions, we chose to compare mean differences for each item versus means for subscales or total scores; furthermore, this analysis was intended to help us better understand any differences in how these two samples reported experiencing the different aspects of MI assessed by the CEMI (i.e., whether certain aspects of the intervention may have been more salient to one sample versus the other). The alpha level was adjusted to p<.003 using a Bonferronni correction. Means, standards deviations, and differences are presented in Table 1. There were significant differences between the two samples on all items, with the current sample reporting higher mean endorsements than the validation sample. Item mean differences ranged from .17 (“Show you that she/he believes in your ability to change your behavior”) to 1.05 ("Push you forward when you became unwilling to talk about an issue further”).

Table 1.

Mean (SD) comparison of current and prior studies.

Item	Current	Prior	Current - Prior
1. Focused on your weakness.	2.87 (1.16)	2. 51 (1.15)	.36*
2. Help you talk about changing your behavior.	3.80 (0.50)	3.39 (0.81)	.41*
3. Act as a partner in your behavior change.	3.60 (0.70)	3.27 (0.86)	.33*
4. Help you discuss your need to change your behavior.	3.78 (0.49)	3.38 (0.81)	.40*
5. Make you feel distrustful of him/her.	3.93 (0.42)	3.21 (1.16)	.72*
6. Help you examine the pros and cons of changing your behavior.	3.73 (0.52)	3.31 (0.81)	.42*
7. Help you to feel hopeful about changing your behavior.	3.93 (0.31)	3.50 (0.70)	.43*
8. Argue with you to change your behavior.	3.99 (0.12)	3.15 (1.16)	.84*
9. Change the topic when you became upset about changing your behavior.	3.95 (0.34)	2.94 (1.18)	1.01*
10. Push you forward when you became unwilling to talk about an issue further.	3.45 (1.03)	2.40 (1.11)	1.05*
11. Act as an authority on your life.	3.71 (0.80)	2.95 (1.17)	.76*
12. Tell you what to do.	3.73 (0.76)	2.87 (1.19)	.86*
13. Argue with you about needing to be 100% ready to change your behavior.	3.94 (0.38)	3.09 (1.21)	.85*
14. Show you that she/he believes in your ability to change your behavior.	3.80 (0.49)	3.63 (0.59)	.17*
15. Help you feel confident in your ability to change your behavior.	3.90 (0.53)	3.56 (0.67)	.34*
16. Help you recognize the need to change your behavior.	3.73 (0.60)	3.38 (0.80)	.35*

Note:

^*=p<.001

A confirmatory factor analysis (CFA) was then conducted on the factor structure identified by Madson et al. (2013) using Mplus, version 5.21 (Muthén & Muthén, 2007). Data were first screened for multivariate outliers using Mahalanobis distance and two observations were removed because of extreme values. The model was estimated using the Satorra-Bentler chi-square (Satorra & Bentler, 1994) due to issues with normality, as fifteen of the items had skewness and kurtosis z-scores of over three. The model fit, χ2(103) = 230.88, p< .05, was poor based on the CFI = 0.69 and TLI = 0.63, and adequate based on the RMSEA = 0.068, 90% CI: .056 to .080. Examination of the factor loadings revealed an issue with item 9, “change the topic when you became unwilling to talk about an issue further,” as it had a significant negative loading for the Relationship factor (−0.37). Since the negative loading is not theoretically supported, the item was removed from subsequent analysis. The resulting model fit, χ2(89) = 212.34, p< .05, was still poor based on the CFI = 0.69 and TLI = 0.64 and adequate based on the RMSEA = 0.072, 90% CI: .060 to .085. Examination of the standardized residual covariance matrix indicated significant lack of fit between the proposed model and observed data for items 8 and 13 (z-score = 7.5) and items 14 and 15 (z-score = 5.6). After examining the content of the items, there were adequate similarities to justify correlating the items. Items 8 and 13 both shared the phrase “argue with you to change your behavior” and items 14 and 15 both contained phrases about readiness or ability “to change your behavior”.

A second CFA was conducted correlating the error terms of items 8 and 13, and also items 14 and 15. Model fit was adequate based on CFI = 0.94 and TLI = 0.92 and good based on RMSEA = 0.033, 90% CI: .010 to .050 (Byrne, 2010; Hu & Bentler, 1999), χ2(101) = 112.71, p< .05. The fit significantly improved based on the chi-square difference test, Δχ2(2) = −99.63, p< .05. The correlation between the two factors was −.05, p>.05, and loadings are presented in Table 2. Consistent with the lower factor loadings on the relational factor, internal consistency was .72 for the technical factor and .58 for the relationship factor. Means, standard deviations and item-total correlations for both factors are presented in Table 2. An alternative model with only one factor was also tested. The model fit was poor χ2(88) = 251.51, p< .05, CFI = 0.59, TLI = 0.51, RMSEA = 0.083, 90% CI: .071 to .096, and fit significantly worse than the proposed model based on the chi-square difference test, Δχ2(1) = 138.8, p< .05.

Table 2.

Factor Loadings, means, standard deviations and item-total correlations.

Item	Technical Factor	Relationship Factor	Mean	Std. Dev.	Item- Total
1. Focused on your weakness.		.25	2.87	1.16	.33
2. Help you talk about changing your behavior.	.51		3.80	.50	.40
3. Act as a partner in your behavior change.	.49		3.60	.70	.38
4. Help you discuss your need to change your behavior.	.50		3.78	.49	.44
5. Make you feel distrustful of him/her.		.15	3.93	.42	.25
6. Help you examine the pros and cons of changing your behavior.	.73		3.73	.52	.52
7. Help you to feel hopeful about changing your behavior.	.62		3.93	.31	.45
8. Argue with you to change your behavior.		.29	3.99	.12	.26
10. Push you forward when you became unwilling to talk about an issue further.		.40	3.45	1.03	.33
11. Act as an authority on your life.		.91	3.71	.80	.58
12. Tell you what to do.		.72	3.73	.76	.50
13. Argue with you about needing to be 100% ready to change your behavior.		.30	3.94	.37	.30
14. Show you that she/he believes in your ability to change your behavior.	.36		3.80	.49	.43
15. Help you feel confident in your ability to change your behavior.	.36		3.90	.35	.46
16. Help you recognize the need to change your behavior.	.34		3.73	.60	.38

Note: Boldface factor loadings are significant at p<.05. Item-total correlations are for each item with its respective factor.

Discussion

Given the increasing importance placed on the assessment of client perceptions of counseling (Kelly et al., 2010;), client feedback (Lambert, 2010; Soderlund, 2009) and how client perceptions (Marcus et al., 2011) relate to outcomes, the CEMI appears to represent an important step in evaluating MI from a client perspective. The addition of the CEMI may enhance training, supervision and evaluation of MI counselors and trainees. The aim of this study was to evaluate the factorial validity of the CEMI with a mainly female, African American sample that was participating in a healthy lifestyle intervention. The factor structure for the CEMI was established in a predominantly White male sample of patients receiving acute inpatient psychiatric treatment in urban and suburban hospitals in the Northeastern United States (Madson et al., 2013). Thus, the primary findings of the current study provide both opportunities to speculate about how the CEMI functions with individuals from different backgrounds and in different settings and to further revise the CEMI in ways that might make it more broadly relevant. In looking at CEMI scores in both samples, significant differences in the mean item endorsements for all items were found. Further, only modest support for the factor structure of the CEMI was found in this sample. The findings also point to potential opportunities for additional revisions and development of the CEMI.

Examination of item means and standard deviations across the two samples suggests that participants in the current study perceived their sessions as more MI-consistent and also evidenced less variability in their perceptions of the sessions than participants in the validation study. There are a number of factors that may account for these differences. First, it is possible that participants in the current study, who received MI because they voluntarily sought out an intervention to improve their health, may have had greater readiness to change and receptivity to counseling than participants in the validation study. Thus, providers and participants may have had more closely aligned goals at the beginning of counseling which might have enhanced CEMI scores. It is also possible that cultural or demographic differences between the two samples may have influenced responding with participants in the present study less likely to be critical of their counselor (Cohen, Nisbett, Bowdle, & Schwartz, 1996). Finally, it is possible that the MI delivered in the current study was of higher quality than the MI delivered in the prior study and thus the difference in participant perceptions reflects a real difference in the adherence of the intervention the two groups received. Compared to the validation study, the training and oversight appears to be more extensive in the current study (i.e., 24 hours of training and additional supervision vs. 6 hour workshop and monthly support sessions; Madson et al., in press; Madson et al., 2013).

Examination of the internal consistency reliability estimates for the CEMI subscales reveals lower estimates in the current sample than the validation sample. The internal consistency was good to excellent in the validation sample (relational = .88; technical =.91) and poor to acceptable in the current sample (relational = .58; technical = .72). Crocker and Algina (1986) suggested that several factors may influence item homogeneity (i.e., internal consistency). Participant responses may be biased and unrelated to content if items are poorly written or if technical flaws in the measure or administration existed. In this study, the CEMI was verbally administered by a research assistant vs. self-administered in the original validation study and differences in administration could have affected the results due to participant acquiescence, social desirability, or experimenter effects. Similarly, as discussed in more detail below, it appears based on anecdotal information from participants in this study that some items were difficult to understand as they were complexly worded. While these items were developed to represent MI concepts, they may have been too complex for participants to understand and thus they may have responded based on some other unrelated content. Another potential explanation for this finding may be that a conceptualization of MI that focuses on relational and technical components (Miller & Rose, 2009) may be more applicable to mental health counseling than healthy lifestyle counseling applications of MI or to pure MI versus ME - an adaptation of MI. Although MI is often discussed as a single intervention, the application of its principles and practices are actually quite unique in healthcare contexts (e.g., Rollnick, Miller, & Butler, 2008) and in adaptations such as ME approaches (Rollnick et al., 2002). This finding highlights the need to pay particular attention to the CEMI items and administration consistency in future studies.

Examination of the factor analysis results revealed that one difference between the CEMI administered in current study and the original study was emergence of two sets of items with highly correlated error terms. Item 8, “argue with you to change your behavior” and item 13, “argue with you about needing to be 100% ready to change your behavior” as well as item 14, “show you that she/he believes in your ability to change your behavior” and item 15 “help you feel confident in your ability to change your behavior” were highly correlated. Upon reviewing these two sets of items, it seems possible that differences between them are meaningful to MI practitioners and researchers, but may be less apparent to clients, particularly in certain contexts. Thus, one potential explanation for these differences may be that the individuals receiving the healthy lifestyle intervention were more “counseling naïve” than those receiving inpatient psychiatric treatments, and thus were less able to detect the subtle differences in session content necessary to produce distinct responses to the items in each of these item pairs. Although we do not have data on what proportion of individuals receiving the lifestyle intervention had a history of mental health counseling or psychotherapy, the National Survey on Drug Use and Health finds that fewer than 14% of Americans receive mental health treatment each year (Substance Abuse & Mental Health Services Administration, 2012), and there is evidence that African Americans are less likely to receive mental health services than Whites (Broman, 2012). The finding may also suggest that differences between the context and health behavior target addressed in the two samples may have influenced the way in which MI was practiced in the two samples such that the distinctions among these counselor behaviors were less clear (e.g., differences in readiness to change). Finally, it is possible that there are gender, racial, or regional differences in how these aspects of MI are experienced. Regardless of the source of the difference, removing item 13 and 14, the more complexly worded items, may improve performance of the measure as lack of understanding of the item can increase measurement error (Crocker & Algina, 1986).

Difficulties related to item 9, “change the topic when you became unwilling to talk about an issue further,” were also identified. Specifically, item 9 correlated negatively with relationship factor in this study; which was not found in the original study. Based on anecdotal information gathered during administration of the CEMI in the current study, we learned that item 9 was confusing and that participants could not differentiate between it and item 10 (push you forward when you became unwilling to talk about an issue further). Such anecdotal information was not available during previous work on the CEMI as it was self-administered versus researcher administered. Based on participant reports of confusion, combined with the results from this study, we suggest merging item 9 and 10 into one item and to reword the item aimed at better clarity for participants. The revised item could be “make you talk about something you didn’t want to discuss” as this captures the shared intent of item 9 and 10 but seems more direct and less confusing.

Another difference uncovered between the current study and the previous study of the CEMI was that item 1 (focus on your weaknesses) produced a lower factor loading. The factor loading was strong enough to suggest that rewording of the item would be more appropriate than removing the item. Thus, item 1 might benefit from being changed to “emphasize your strengths” as positively wording the item may better grasp the clinician’s use of this MI skill. Finally, item 5 (make you feel distrustful of him or her) did not significantly load on the relationship factor as it did in previous research. It is possible that this difference reflects differences between the two samples. For example, there may be much greater variance in client perceptions of the need for counseling and of counselors as trustworthy versus not depending on a variety of factors. These factors include the context of the interaction (Goodwin, 2003), gender of client and counselor (Bedi & Rickards, 2011), racial composition of the individuals (Townes, Chavez-Korell, & Cunningham, 2009) and focus of change (Tucker, Marsiske, Rice, Nielson, & Herman, 2011). Trust has recently been discussed as an outcome of MI consistent behaviors as opposed to a component of MI (Miller & Rollnick, 2013). Accordingly, we suggest removing this item and revising the others mentioned above as item 1 is more likely an outcome of MI whereas the other items with lower factor loadings have more theoretical relevance. Overall, the proposed revisions would result in a 12 item measure that potentially has broader relevance across patient populations and MI applications. Re-examination of Madson et al’s (2013) data with the three items (5, 13, 14) removed showed that the variance explained by the CEMI increased from 51.1 to 57.6 and items loaded on the same two factors. This suggests that these potential revisions can provide a more parsimonious assessment of client’s experiences receiving MI. However, more thorough examination of these changes is needed.

A final opportunity for revision of the CEMI identified based on the current study is the potential for limited variability in scores based on the response set offered. The results of this study show that most of the items have limited variability. In fact, only one item has a mean score below 3.0 and only two items have standard deviations above 1.0. This limited variability calls into question the discrimination of items and highlights a need to review the items or scoring. Given that these results differ from previous studies (Madson et al, 2009; 2013), it appears more viable to revise the response format perhaps changing the 4 point scale to a 5 point or 7 point response scale. Currently, the descriptor for the highest point on the response scale states “a great deal.” Adding more strongly worded end points, such as “never” or “always” may resolve this problem. Perhaps an additional explanation for this lack of variability is the actual experience of the participants was more appealing than their previous experiences with health care providers. Peek Tang, Cargill and Chin (2011) indicated that African American individuals in their research have demonstrated an increased desire for shared decision making yet often do not experience it. With MI’s client centered focus, appreciation of autonomy, and emphasis on eliciting client motivations and strategies for change, it may be the case that participants in this study were very pleased overall with their experience.

Limitations and Research Implications

These results should be interpreted within the limitations of the study. Most noteworthy is that the CEMI was orally administered in this study, which is different from previous studies (Madson et al., 2009; 2013). This change in procedure may have led to experimenter effects which need to be considered. For instance, the fact that a participant had to verbally provide answers to a researcher may have increased socially desirable responding and thus responses favoring high MI scores. Future investigation of CEMI should make attempts to adhere to original administration procedures. The sample was somewhat homogeneous (majority African American females self-selecting into a lifestyle intervention) which may limit the generalizability of our findings. Although as noted previously, contrasting findings from the current sample to those achieved in the primarily White, male, mental health sample utilized for initial validation of the measure has resulted in proposed changes to the measure that may help to make the measure more broadly relevant. Future studies of the CEMI with lifestyle interventions should seek more heterogeneous groups.

Further psychometric evaluation of the CEMI is warranted. First, it will be important to assess whether the internal consistency and factor structure are supported after the suggested revision. Another important step in the validation of the CEMI will involve exploring its convergent and predictive validity. For instance, it would be valuable to compare scores on the CEMI to scores on validated observational measures (Madson & Campbell, 2006) as a criterion and how it is associated with similar constructs such as those assessed by the Working Alliance Inventory (Hatcher, & Gillaspy, 2006). Also, the relationship between CEMI scores and client behavior change outcomes needs to be established before its use in exploring MI mechanisms of change. These steps are important in the process of establishing the CEMI as a more practical and cost-effective way of assessing MI fidelity and quality than existing observer-rated measures.

Implications for Counseling

After further revision and psychometric evaluation, the CEMI may be a valuable tool for use in practice, training and supervision utilizing MI. For counselors who use MI as part of their practice, the CEMI can be used to attain feedback from clients. Feedback such as this can help counselors learn from their clients how to improve the MI use (Miller & Rollnick, 2013). MI trainers and supervisors can use the CEMI as a tool for providing feedback to MI trainees. Traditionally, training and supervision involves session review and counselor self-report. However, Miller (2001) highlighted that trainee self-report is often inaccurate and diverse training evaluation tools are needed (Madson, Loignon & Lane, 2009; Söderlund, Madson, Rubak, & Nilsen, 2011). Thus, feedback from clients can provide another valuable source of information for trainee skill development. For example, feedback acquired from the CEMI could be compared with counselor self-report and supervisor observation and facilitate a discussion about the similarities and differences. Currently, the use of observational tools to evaluate MI is the “gold standard.” However, obtaining work samples to review in community settings taping is often difficult (Schumacher, Madson & Norquist, 2011). The CEMI, once further developed and evaluated, might become a valuable substitute to provide MI session information that may be otherwise unavailable. Finally, MI might be utilized as an approach to providing counseling supervision (Madson, Bullock, Speed, & Hodges, 2008). Thus, supervisors utilizing MI in their supervision approach may benefit from adapting the CEMI for evaluation of their supervision.

Uncovering the factors contributing to MI’s efficacy remains a priority. Client perceptions of MI are an underrepresented source of information and the CEMI shows promise as a measure for providing these data. However, additional refinement and evaluation is needed to develop the CEMI and determine how it might further contribute to MI research and training.