Development of an observational tool to assess health coaching fidelity

Abstract

Objective:

This study describes the development of the Health Coaching Index (HCI), an observational tool for assessing fidelity to implementing health coaching practical skills.

Methods:

Initial HCI items were developed, adapted following cognitive interviews, and refined during coding training. Participants (n = 42) were trainees who completed a National Board for Health and Wellness Coaching (NBHWC)-approved training program and coached a standardized patient. Interrater reliability for the HCI was determined by calculating interclass correlations from ten videos coded by three raters. Construct validity was evaluated from 42 recordings using Spearman’s Rho between HCI and Roter Interaction Analysis System (RIAS) codes.

Results:

The interclass correlation (ICC) for HCI total score was 0.81, considered an excellent level of inter-rater agreement. Some significant correlations between HCI and RIAS codes supported construct validity (e.g., patient activation: Rho = 0.32; empathy: Rho = 0.36).

Conclusion:

The HCI total score can reliably be used to assess fidelity to health coaching skills, and the HCI has construct validity similar to the RIAS as a measure of patient activation.

Practice Implications:

Adoption and further study of the HCI tool will allow for a more consistent implementation of health coaching skills, and may facilitate more robust training of health coaches for clinical practice and research.

1. Introduction

Chronic diseases are the leading cause of death in the United States and improving health behaviors (e.g., physical activity, tobacco use) is fundamental to preventing and controlling chronic diseases [1]. Thus, it is necessary to empower individuals to take an active role in managing their health behaviors on an ongoing basis to improve health outcomes [2,3]. Health coaching is emerging as a new profession that trains individuals as behavior change experts in multiple evidence-based practical skills for empowering self-management (e.g., motivational interviewing, eliciting accountability) to address the growing need for supporting an increase in healthy behaviors [4,5]. In addition, research supports the promise of health coaching as an effective intervention for managing chronic diseases [6–8].

Limitations of the current research include the high level of variability in use of the term health coaching [6]. A systematic review has identified similarities within interventions that use the term health coaching such that most include the following components: a patient-centered approach, encouragement of self-discovery, content education, goal setting determined by the patient, and mechanisms for developing accountability [9]. In addition, published interventions that utilize the term “health coach” in the context of promoting the self-management of medical conditions vary in regards to the educational background, level of training as a health coach [9], and additional skills incorporated (e.g., mindfulness practices) [10]. A common thread is that health coaching empowers the autonomy of patients in facilitating behavior change and thus is consistent with the guidelines of behavior change outlined by Self-Determination Theory [11–13].

Given the variability in health coaching definitions, training, and skills, there is significant variability in how coaching is currently performed. Health coach training programs need a tool to monitor the fidelity of delivering a robust health coaching model to facilitate standardization in both clinical practice and research [14]. In fact, the National Board for Health and Wellness Coaching (NBHWC) requires that examinees who wish to take the national certification exam pass a training program that has been approved by the NBHWC. For approval, programs must meet national standards for their faculty, curriculum and assessment of trainees’ practical skills [15,16]. While there are existing research tools validated for specific topics of behavior change (e.g., weight management [17]), there is no evidence-based observational tool that health coach training programs can use to assess that their graduates have adopted these practical skills (i.e., required behaviors, competence) [14]. It is important that this tool allow the topic of goal setting to be determined by the patient to capture the autonomy supportive intention of health coaching [15].

The study objectives were: (1) To develop a Health Coaching Index (HCI) observational tool for assessing fidelity of implementing practical health coaching skills that will generalize across clinical and research interventions; (2) to evaluate the interrater reliability of the HCI in standardized patients; and (3) to evaluate construct validity of the HCI as compared to the most widely used observational method for coding medical interactions, the Roter Interaction Analysis System (RIAS). We hypothesized that the HCI total index would correlate moderately in the positive direction with RIAS composite categories for patient-centeredness and patient activation. We also explored the relationship of subscales and individual items that assessed shared and distinct constructs.

2. Methods

This study followed recommended steps for measure development [18,19].

Clearly specify what is being measured.

The key components of health coaching were defined as the common elements elicited from a systematic review [9] and later classified by the Behavior Change Technique Taxonomy ([20]BCT in Table 1). The initial HCI was developed with a focus on the verbal and relational content of sessions and use for training purposes as modeled after a study that developed a similar tool for observing motivational interviewing [21]. We chose to develop an index primarily because we aimed to evaluate skill levels on multiple constructs that result in an overall assessment of proficiency rather than a scale that assesses items that share an underlying construct [18]. Therefore, the number of items stayed consistent in the development process to ensure each competency was assessed.

Table 1.

The Health Coaching Index Item Classification and Inter-rater Agreement.

	BCT Taxonomy	ICC Agreementa	Mean (SD)b
Long-term Vision, Designing Goals & Actions, and Self-monitoring (Possible 8 points)		0.56	1.02 (0.36)
Long-term Vision and Exploration of What Client Wants	1.3 Goal Setting (outcome)
	9.3 Comparative Imagining
Designing Long-term Goals, Short-Term Goals and/or Action Steps	1.1 Goal Setting (behavior)
	1.4 Action Planning
	1.7 Review Outcome Goal(s)
Accountability and Self-Monitoring	2.3 Self-monitoring of Behavior
Problem Solving	1.2 Problem Solving
	9.2 Pros and Cons
Use a Self-Discovery Process (Possible 8 points)		0.86	1.14 (0.36)
Values	13.3 Incompatible Beliefs
	13.4 Valued Self-identity
Explores Different Perspectives	13.2 Framing/Reframing
Elicits Learning and Insight	1.5 Review Behavior Goal(s)
Amplify Positive Resources	15.3 Focus on Past Success
Match Information to Client Interest and Needs (Possible 2 points)		0.48	0.98 (0.76)
Sharing Information with the Client	5.1 Information about Health Consequences (optional)
Have a Client-Centered/Mindful Approach (Possible 18 points)		0.83	1.20 (0.30)
Mindful Moment	11.2 Reduce Negative Emotions
Full Presence	3.3 Social Support (emotional)
Collaborative Partnership
Support Autonomy
Express Empathy
Invite Session Focus
Open-ended Questions
Active Listening and Reflections
Timing and Structure

Note. BCT = Behavior Change Technique Taxonomy v1 for categorizing intervention content.

^aThe bold numbers for subscale agreement were ICC values based on 10 videos assessed by all raters.

^bMean (SD) calculations are based on the full sample of coded videos.

For this initial study, we focused on the Vanderbilt Health Coaching Certificate Program (https://www.vumc.org/health-coaching/). This two-phase, 86-hour (currently 135-hour) health coaching course aims to teach the common components of health coaching with a foundation of mindfulness. The program included an emphasis on mindfulness because mindfulness in healthcare providers supports patient autonomy [4]. The first phase included instructional modules, two 3-day in-person intensives, weekly group teleconferences to practice the coaching skills and process, with ongoing faculty mentorship. The second phase of training is a practicum wherein trainees coach patients while under faculty mentorship. Overall, it takes trainees approximately 18 months to complete the program. Program applicants are required to be licensed in a healthcare field or practicing at the top of their field if licensure is not available (e.g., exercise physiologists, yoga therapists). The Vanderbilt Health Coaching Certificate Program is approved by the NBHWC.

Generate an item pool.

Eighteen initial items were generated from existing measures of relevant constructs and therapies including patient-centered communication [22], the therapeutic relationship [23], motivational interviewing [24], and documents describing health coaching (Health Coaching Performance Assessment [25], International Coach Federation Core Competencies [26], Health and Wellness Coaching Job Task Analysis [27], NBHWC Practical Skills Guidelines [15]). In addition, items identified from these measures were adapted to be consistently worded. An effort was made to write items that were worded simply [19].

Determine the format for measurement.

While initial items were rated on a 4-point scale from 0 (not at allcompetent), 1 (some-whatcompetent), 2 (competent) and 3 (highly competent), pilot applications of the instrument in the training context led to a 3-point scale with the aim of increasing consistency. The updated response options are: 0 (not yet competent), 1 (competent), 2 (proficient) and not applicable. This measurement approach allows for some variation in responses, while increasing the likelihood of inter-rater reliability.

Have experts review the initial item pool.

Subject matter experts in behavioral research and health coaching were consulted to review the original item pool and to improve the quality and content validity of the items. Cognitive interviews were conducted using a verbal probing debriefing technique [28–30] to refine the initial items compiled for the HCI. The Principal Investigator (SJS) conducted cognitive interviews using a semi-structured interview guide designed to elicit feedback regarding the meaning of items, information needed to respond, confidence in scoring the response, and any additional comments [28,31]. The interviews were conducted sequentially with two of the training faculty after using the HCI to evaluate video recordings of trainees who coached a simulated patient. Modifications were made based on feedback from each interview to maximize consistent interpretations of skill assessment. The Program Director for the Coaching Certificate Program was then consulted for any further modifications (RW). This expert review process resulted in a cognitively-informed HCI.

The HCI tool and manual were further iteratively refined through the process of training observational coders. As part of the training phase, the coding team and investigators (SJS, RQW) initially qualitatively evaluated reliability on practice sessions from a cohort of trainees not included in the analysis. The coding team were all NBHWC-certified heath coaches and faculty for academic health coach training programs. Four to six independent reviewers scored six coaching sessions that were transcribed verbatim. The team met and reviewed discrepancies to ensure consistency across coding categories and refine a scoring manual. This process required adding considerably more detail to the manual with tangible distinctions among scale responses. For example, to score a 2 (proficient) on Empathy, the manual specifies that the “Coach does 2–3 of the following behaviors throughout the session,” with specific behaviors listed. The team met and made changes to the manual until ratings became consistent. They did not meet to discuss coding after completing training, compiling the manual and starting to code study videos.

The resulting 18 modified final items listed in Table 1 are categorized into four domains consistent with the components identified in the above-mentioned systematic review [9]: (1) Long-term Vision, Designing Goals & Actions, and Self-monitoring (4 items); (2) Use a Self-Discovery Process (4 items); (3) Match Information to Client Interest and Needs (1 item); and (4) Have a Client-Centered/Mindful Approach (9 items).

Administer the items.

Participants were trainees who self-selected to complete the health coaching certificate program. Trainees coached a standardized patient (a professional actor thoroughly trained in a particular medical patient case) as a component of their course instruction and these sessions were video recorded for educational use (audio recordings used for RIAS evaluation). The coaching sessions occurred toward the end of the first training phase after practicing with peers for about 6 months, and typically before extensive experience with clients. The scenario set up a mock follow-up coaching session. Prior to the standardized patient session, trainees were provided written documentation of a first and second session with a client who was recently diagnosed as having pre-diabetes. The client was slightly over-weight and healthcare providers had recommended weight loss, exercise, and diet changes to help manage the pre-diabetes diagnosis. The scenario was the same across participants with slight variation by cohort reviewed (e.g., the first of three cohorts included an action step on mindful breathing that was removed for later cohorts). Trainees had up to 30 minutes to implement a coaching session. This research was approved by the local Institutional Review Board and consisted of additional observational coding using the HCI and the RIAS. This study also required that audio recordings of completed sessions be sent for evaluation by expert RIAS coders at RIASWORKS, a company that specialized in the application of RIAS.

Inclusion of validation items.

HCI scores were compared to the most widely used observational method for coding medical interactions, the RIAS. The RIAS has demonstrated consistent reliability and validity, and served as a check of the HCI’s construct validity [18,32]. Although the RIAS does not measure all of the constructs assessed by the HCI, we adopted this foundational system of analyzing interactions with healthcare providers to assess shared constructs and clarify important distinctions. In addition to individual codes, we used composite scores for patient-centeredness (a ratio of psychosocial and socio-emotional elements of exchange to medically-focused elements) [33,34], patient activation (asking for patient opinion, asking for patient understanding, restatement of patient disclosures) [35], and created two additional composite scores for all of the open-ended question codes and all of the closed ended question codes.

Evaluate the items.

The HCI scores are a proportion calculated as a sum of total raw points earned divided by the points possible to create a standardized scaled score since the number of items varied (some were excluded for categories deemed not applicable). To convert back on the rating scale of 0–2 to interpret the HCI, the standardized scaled score is multiplied times two. Interrater reliability was determined by calculating an interclass correlation with an emphasis on absolute value (ICC [A, k]) on the scaled scores using 10 randomly selected sessions that were each coded by three independent raters [36,37]. Interrater reliability is considered poor for ICC values < 0.40, fair when between 0.40–0.59, good when between 0.60–0.74, and excellent for values between 0.75–1.0 [37]. The remaining sessions were randomly assigned to be coded by at least one reviewer each. Percent agreement for ratings of each categorical item were based on whether raters agreed that the coach was not competent (ratings of 0) versus competent or proficient (ratings of 1 or 2) since Kappa scores may be affected when certain values are highly prevalent (e.g., most raters score a 2 for one item) [38]. These percent agreement values were used to inform improvements to the manual for rating each item in the future.

Construct validity was evaluated by conducting Spearman’s Rho correlations with single items and summary scores of the HCI and RIAS and assessing the strength of resulting coefficients. When more than one HCI coder scored a session, a mean score was used for validity analyses. Two RIASWORKS coders also rated 10 % of the same sessions to evaluate interrater reliability for the RIAS.

3. Results

The forty-two trainees who coached a standardized patient were primarily female (n = 41) health professionals (32 nurses, 2 social workers, 2 dieticians, 2 psychologists, 1 pharmacist, 1 occupational therapist, 1 yoga therapist, 1 physical therapist) with advanced degrees (n = 24 had graduate degrees [masters or doctoral]). There were forty-two unique videos (initially n = 43, one video was removed since it was of a study coder who participated in a prior training) for a total of 74 completed HCI ratings. The mean HCI total score was 1.14 (SD = 0.28), slightly higher than a competent rating. The ICC (A, k) for the HCI total score was 0.81 for the ten videos coded by all three raters, which is considered an excellent level of reliability [36,37]. The ICC values for the subscale totals of Long-term Vision, Designing Goals & Actions, & Self-monitoring were ICC = 0.56 (fair), Use of Self-discovery Process ICC = 0.86 (excellent), Match Information to Client Interest and Needs ICC = 0.48 (fair), and Have a Client-centered/Mindful Approach ICC = 0.83 (excellent) are also displayed in Table 1. We further identified percent agreement values for individual items that were rated by 1–3 coders, which ranged from 0.42 to 1.00 and identified opportunities for improving the coding manual (Table 2). In addition, Table 2 shows the distribution of ratings by item.

Table 2.

The Health Coaching Index Item Distributions and Item-Level Rater Agreement.

	N	Percent Rated	Rater Agreementa
Long-term Vision, Designing Goals & Actions, and Self-monitoring (Possible 8 points)
Long-term Vision and Exploration of What Client Wants			0.89
0	8	10.81
1	47	63.51
2	19	25.68
Designing Long-term Goals, Short-Term Goals and/or Action Steps			0.95
0	2	2.70
1	16	21.62
2	48	64.86
N/A	8	10.81
Accountability and Self-Monitoring			0.42
0	30	40.54
1	33	44.59
2	2	2.70
N/A	9	12.16
Problem Solving			0.64
0	27	36.49
1	38	51.35
2	5	6.76
N/A	4	5.41
Use a Self-Discovery Process (Possible 8 points)
Values			0.87
0	10	13.51
1	40	54.05
2	24	32.43
Explores Different Perspectives			0.73
0	20	27.03
1	38	51.35
2	16	21.62
Elicits Learning and Insight			0.73
0	23	31.08
1	49	66.22
2	2	2.70
Amplify Positive Resources			0.96
0	3	4.05
1	16	21.62
2	55	74.32
Match Information to Client Interest and Needs (Possible 2 points)
Sharing Information with the Client			0.61
0	16	21.62
1	23	31.08
2	15	20.27
N/A	20	27.03
Have a Client-Centered/Mindful Approach (Possible 18 points)
Mindful Moment			1.00
0	4	5.41
1	24	32.43
2	45	60.81
N/A	1	1.35
Full Presence			1.00
0	0	0.00
1	29	39.19
2	45	60.81
Collaborative Partnership			0.96
0	1	1.35
1	23	31.08
2	50	67.57
Support Autonomy			0.77
0	14	18.92
1	48	64.86
2	12	16.22
Express Empathy			1.00
0	0	0.00
1	14	18.92
2	60	81.08
Invite Session Focus			0.84
0	11	14.86
1	50	67.57
2	13	17.57
Open-ended Questions			0.83
0	32	43.24
1	34	45.95
2	8	10.81
Active Listening and Reflections			0.81
0	48	64.86
1	22	29.73
2	4	5.41
Timing and Structure			0.71
0	13	17.57
1	36	48.65
2	25	33.78

^aThe item-level numbers represent percent agreement values averaged across randomized rater combinations for all 42 videos.

Interrater reliability for coding the RIAS was also high for this study (r = 0.90; for categories that had a mean > 2.0). The results for hypothesized correlations of the HCI with RIAS variables are displayed in Table 3. All associations were small to moderate in the expected direction except for non-significant associations between the HCI and the RIAS scores for patient-centeredness, open-ended questions and dominance. Two relevant RIAS variables were not analyzed due to limited variance (i.e., partner statements, hurried/ rushed).

Table 3.

Correlations of the Health Coaching Index (HCI) with variables coded by the Roter Interaction Analysis System (RIAS).

		RIAS Variables
		PtCent	Activation	Empathy	Sympathetic	Engaged	Interest	Respectfulness	Interactivity	Dominance	Open-ended	Closed
Median (Min, Max)		9.82 (4.76, 34.75)	150.0 (69.00, 288.00)	4.50 (0, 11.00)	4.00 (3.00, 5.00)	4.00 (4.00, 5.00)	5.00 (4.00, 5.00)	4.00 (4.00, 5.00)	4.00 (4.00, 6.00)	4.00 (4.00, 5.00)	29.00 (10.00, 50.00)	9.00 (0, 36.00)
	Median (Min, Max)	Rho	Rho	Rho	Rho	Rho	Rho	Rho	Rho	Rho	Rho	Rho
HCI Total	0.59 (0.25, 0.79)	−0.12	0.32*
HCI Client-Centered Subscale	0.64 (0.33, 0.89)	−0.17	0.24
HCI Empathy	2.00 (1.00, 2.00)			0.36*	0.41**
HCI Full Presence Item	1.67 (1.00, 2.00)					0.21	0.30
HCI Encourage Autonomy Item	1.00 (0, 2.00)							0.26
HCI Collaborative Partnership	2.00 (0.67, 2.00)								0.15	0.13
HCI Open-ended Questions	0.50 (0, 2.00)										−0.14	−0.33*

^*p < 0.05

^**p < 0.01.

Note. PtCent = Patient-Centeredness. RIAS variables not analyzed due to limited variance: hurried/rushed (Median = 3.00 [3.00, 3.00] and partnership statements (Median = 0 [0, 8.00]).

4. Discussion and conclusion

4.1. Discussion

This study developed the HCI direct observation tool and demonstrated that the total score can be used to reliably rate fidelity to the practical skills used in a standalone health coaching session. When observing trainees from different cohorts, we noted that as the training became more structured over time it was easier to score the HCI. Therefore, training programs that choose to adopt the HCI may find it helpful to align curricula with scoring. This alignment of educational content, method of assessment and feedback provided will likely streamline the use of the HCI in a learning environment.

HCI ratings were generally distributed such that the most common rating was a 1 (competent). A few items deviated from this pattern. For items with the most common rating of 2 (proficient), this skill may have been taught especially well or was simpler to learn (e.g., long-term goals, amplify positive resources). Only one item (i.e., active listening) had the most common rating of 0 (not yet competent), which could have been a more difficult skill to master or this skill may not have been taught in a manner that was consistent with how it was rated. There was also some variability in agreement in ratings by subscale and item. Subscales (e.g., Match Information to Client Interest and Needs) and individual items (e.g., Accountability and Self-Monitoring) with lower agreement ratings should be applied with discretion. Item-level analyses were used to revise the coding manual for those that were less consistently rated. An example of an item with a lower level of agreement was the “accountability and self-monitoring item,” which was inconsistently considered not applicable. Thus, we clarified in the coding manual that this item should be rated as not applicable if there are no new or significantly revised action steps in the coaching session. We also reworded the coding guidance in the competent category to clarify inclusion of all scenarios.

Most HCI categories for the intervention content were aligned with at least one BCT code [20]. Many of the same BCT codes aligned with HCI items are supported as efficacious for facilitating health behavior change such as weight loss [39]. As the BCT becomes more widely adopted, it is likely there will be a high level of overlap with other efficacious behavior change interventions and health coaching techniques assessed by the HCI. The last HCI domain, Have a Client- centered/Mindful Approach with 8 items, was classified primarily by one BCT code for Social Support (emotional) since this code includes Motivational Interviewing. This HCI domain assesses a coaches approach and overall competence, rather than techniques. It is fundamental to the clarified definition of health coaching that health coaches implement the BCTs with an emphasis on supporting patients’ autonomy [9,10]. That is, coaches facilitate a visioning and goal-setting process that elicits patient-selected goals tied to personal values instead of directing the content of those goals. This autonomous process is more likely to result in sustainable behavior change [4]. It is innovative that the health coach training program evaluated in this study includes education and assessment of mindfulness as a skill that enhances coaches’ autonomy support [4].

Overall, correlations of the HCI total score with RIAS variables were in the expected directions supporting construct validity. Of particular importance was that the HCI was significantly positively associated with RIAS patient activation, which assessed if the provider asked for the patient’s opinion and understanding, and restated patient disclosures. One unexpected result was that the total score was not associated with patient-centeredness as operationalized with select RIAS variables. This lack of association was likely due to different definitions of the construct of patient- centeredness. The HCI defines patient-centeredness as the process of supporting patient autonomy and patient contributions to the discussion (more similar to RIAS patient-activation), whereas the definition of patient-centeredness coded with the RIAS is the ratio of the content of the conversation that was psychosocially or socio-emotionally-focused to medically-focused. It was also unexpected that open-ended questions as coded by the RIAS were not associated with HCI open-ended questions. Again, this appears due to different instructions with the two instruments. HCI raters counted all open-ended questions including those that were considered to encourage client insight and those that served to gather information. In the RIAS, the open-ended questions counted were limited to information-gathering categories (medical condition, therapeutic regimen, lifestyle information, psychosocial information, other information). As expected, the HCI rating for close-ended questions was significantly negatively related to the sum of the RIAS closed-ended variables.

To place these results within the broader context of the existing literature, it is relevant that a recent study published after we initiated our protocol reported using direct observation to validate fidelity to health coach training within clinical care [40]. Using a pragmatic approach to coding health coaching in medical and standalone patient-health coach visits, this recent study adopted categories that were consistent with the HCI, including the following: setting an agenda (HCI - invite session focus), gathering additional information (HCI - sharing information), providing information and education (HCI - sharing information), goal setting and action planning (multiple HCI categories), and providing emotional support (HCI - amplify positive resources). Categories of activities that differed from the HCI were: medication review and reconciliation, providing practical support, personal conversation, and bridging between patient and clinician. In addition, this prior study specified two additional categories unique to coaching during a medical visit (versus a standalone visit with a coach): identifying goals for the visit and reviewing clinician recommendations. Therefore the medical context of the interaction as compared to the HCI that was used outside the context of a medical visit, leads to differences in the HCI from other coding frameworks (e.g., the one used in this prior study, RIAS). The HCI was designed to assess health coaching as a standalone intervention and would need to be adapted to assess health coaching integrated within a medical visit.

In addition, an exemplar research-focused study validated a tool to assess weight management health coaching intervention fidelity [17]. Five factors were identified in this weight management health coaching checklist: set goals and monitor progress, assess and personalize self-regulatory content, session process, create a supportive and empathic climate, and stay on track. The content of these factors were generally consistent with the HCI with minor differences (e.g., did not assess mindfulness or values). As noted by the authors, identifying intervention components to this level of detail is important for understanding questions such as which intervention elements are related to study outcomes [17]. Whereas this research tool was validated specifically for the reported weight management intervention, the HCI was designed to be more broadly applicable to general health coach training.

Limitations and future directions

The HCI is intended to be broadly applicable to health coach training programs approved by the NBHWC, as it encompasses the practical skills that must be assessed per the NBHWC and items on the HCI are based on the broader academic literature on health coaching. Nonetheless, generalization of results of the current study may still be limited at this stage because coach trainees were all from one coach training program. Future directions include the following: assessing the performance of the HCI across a variety of health coach training programs; establishing the appropriate cutoff scores for the determination of adequate implementation of practical skills using the HCI; examining whether the HCI predicts coaching outcomes such as behavior change; and investigating which coaching skills contribute to changes in outcomes (e.g., does adding mindfulness strengthen outcomes). Another limitation of the HCI is that the coders may need to be extensively trained and use video-recorded coaching sessions to reliably apply the scoring manual. A future direction could be testing reliability of applying the HCI with audio recordings, novel users, or direct observations in real-time.

As evidence increasingly supports the effectiveness of health coaching [6–8], it is imperative that fidelity be an important implementation outcome not only for coach educational training programs, but also for clinical research. That is, intervention fidelity assesses whether a study intervention did not work or if it was not implemented as intended [41]. The study of behavioral interventions is also evolving to consider more cost-effective methods for evaluating fidelity in future studies beyond the gold standard of direct observation (e.g., chart-stimulated recall) [42]. Therefore, the current study may serve as the foundation of treatment fidelity assessment for the field of health coaching and future research could evaluate other more scalable methods for adapting this assessment.

4.2. Conclusion

In conclusion, these analyses demonstrate that it is possible to reliably use the HCI total score to assess fidelity to the delivery of practical health coaching skills and that the HCI shows construct validity with the widely adopted RIAS. It is important to consider the context of use for the HCI such that the variables included may differ in medical contexts or for other coach training programs. Adoption and further study of the HCI assessment tool may ultimately strengthen the methodological quality of health coach training, research and performance in health care.

4.3. Practice implications

Using the HCI could facilitate standardization of training programs and thus delivery of this intervention across clinical sites and studies. This standardization may lead to a more widespread recognition of a health coach’s skillset. Health coaches are trained in widely adopted behavior change techniques and could be ideal for implementing a variety of evidence-based interventions for promoting health behavior change and improving chronic disease outcomes.

As the healthcare landscape shifts to focus more on patient activation and self-management, those performing health coaching must demonstrate at least minimal competency in the related skills. In January 2020, the American Medical Association Current Procedural Terminology (CPT) panel launched newlevel 3 CPT© codes to assist the healthcare enterprise with data collection to demonstrate the rapidly growing dissemination of health coaching [43]. As use of these codes increases, it will become even more important to assess fidelity to delivery of practical skills in this unlicensed profession. Future iterations of the HCI may assist in this endeavor.