Abstract
Objectives
Assess fidelity of a motivational interviewing (MI) intervention focused on preventing early childhood caries in a cohort of American Indian mothers with newborns.
Methods
Four interventionists were trained to administer an oral health MI intervention. The MI sessions were audio recorded to enable scoring as part of fidelity studies to assess the interventionist’s degree of competence in using MI principles. Evaluation of the interventionists was completed using the Motivational Interviewing Treatment Integrity (MITI) 3.1.1. which assesses global ratings and behavior counts. Two reviewers evaluated and scored the MI sessions; a random sample of twenty percent of the total interviews was scored by a trained reviewer and fifty percent of the randomly selected files scored by an external expert. A total of 225 files were coded by the trained reviewer and 121 files by the external expert.
Results
Mean global scores for all interventionists combined were around 4.0 (expert competence). Scores for reflection to question ratios (around 0.6) were below beginner competence. Scores ranged from beginner to expert competence for open-ended questions (54-56%), complex reflections (38-43%), and MI-adherent statements (93-95%). There was variation in competence for the four interventionists when analyzed individually. Interrater reliability scores for the two reviewers ranged from fair (0.40-0.59) to good (0.60-0.74). The MI interventionist rated the best in the fidelity assessment had the worst study outcomes compared to other interventionists.
Conclusions
Individual interventionists’ scores for the MITI global ratings and behavior counts reflected variation in competence and ranged from below beginner to expert levels. A higher competence level of the interventionist as assessed by the fidelity study was not related to better study outcomes.
Introduction
American Indian and Alaska Native (AI/AN) children experience suboptimal oral health compared to non-ethnic and minority children.1 When comparing United States tribal groups, caries experience and the percentage of children with untreated decay is more prevalent on the Pine Ridge Reservation in South Dakota.2,3 The Pine Ridge Reservation of the Oglala Sioux Tribe is the second largest reservation in the United States, encompassing 4,353 square miles. Approximately 32,000 people reside on the reservation, often in small, isolated communities with limited access to health services.4 High prevalence of early childhood caries (ECC) and persistent barriers to obtaining dental care including insufficient numbers of dental providers on the Pine Ridge Reservation highlighted a need for effective oral health promotion efforts.4–6 Accordingly, an oral health intervention was developed using motivational interviewing (MI), a widely accepted guiding style for enhancing intrinsic motivation to change with world-wide dissemination and application in a variety of disciplines.7 While use of MI for oral health concerns is more nascent8–13 and minimally studied in indigenous groups, the technique is consistent with AI/AN cultural values of respect, autonomy, and honoring the individual’s wisdom.14–16
Increasing application of behavior-based interventions in clinical and research studies has led to greater emphasis on methodological assessment of reliability and validity through defined fidelity studies. The concept of fidelity was formally introduced in 1991 and advanced over time.17 An integral aspect of fidelity is assessment of treatment integrity to ensure the intervention was delivered as designed and outcomes were due to the intervention being tested. When fidelity is low and treatment outcomes are null, the erroneous conclusion is the intervention is ineffective and will not be disseminated, although not properly tested.17 Conversely, if fidelity is low and treatment outcomes are positive, the intervention may be disseminated. although not properly tested.17
In 1999 the National Institutes of Health (NIH) established the Behavior Change Consortium (BCC) yielding recommendations for strategies addressing treatment fidelity for health behavior change studies.18 The BCC established a comprehensive method for health behavior researchers to assess fidelity based on five key components: study design, training providers, delivery of treatment, receipt of treatment, and enactment of treatment skills. Treatment fidelity for receipt of treatment and enactment of treatment are focused on the participant and their response to the intervention.19 The BCC recommendations specifically focus on behavior change interventions and offer strategies deemed “best practices” in relation to key components.
Various instruments have been developed to quantify fidelity of MI interventions, with the Motivational Interviewing Treatment Integrity (MITI) scale most used. The MITI has consistently acceptable psychometric properties20 and codes global counseling skills and the interventionist’s behavioral articulation thereby providing structured feedback regarding competence. The MITI scale was introduced in 2005 and undergone several revisions20,21 with the MITI 4.2.1 as the current version and offered in multiple languages including Spanish, Danish, Dutch, German, and Norwegian.22 As a fidelity instrument, the MITI offers several advantages as a precise coding system measuring interventionists’ adherence to MI behaviors and empathy, a fundamental characteristic in MI and other psychosocial interventions.21, 22 The primary aim of this study was to assess MI fidelity for an oral health intervention in AI mother-child dyads.
Methods
Study Approval
A detailed description of the trial design and protocol has been published23 and only salient features are described here. The trial was approved by the Oglala Sioux Tribal Research Review Board, Aberdeen Area Indian Health Service Institutional Review Board, Colorado Multiple Institutional Review Board, and registered at ClinicalTrials.gov (NCT01116726).
Study Location and Design
The study was conducted on the Pine Ridge Reservation of the Oglala Sioux Tribe as a randomized controlled trial to determine whether an intervention using MI with AI mothers reduces ECC in children up to age 3 years compared to enhanced community services (ECS) alone. A total of 579 mothers with newborns were consented as dyads to participate in the intervention with the understanding and written consent of each participant provided in full accordance with ethical principles established by the World Medical Association Declaration of Helsinki.
Participants were randomized in a 1:1 ratio to the MI intervention and ECS or ECS alone between July 2011 and March 2014. The last follow-up visit was in January 2017. The MI intervention consisted of four visits with the first shortly after birth and subsequently at ages 6, 12, and 18 months. Data collection visits for outcomes and mediators or moderators occurred at baseline, 12, 24, and 36 months. Study outcomes included oral health knowledge (16 items) and behavior (13 items) scores reflecting maternal adherence with ECC prevention guidelines24 and scores reflecting children’s decayed, missing, and filled tooth surfaces (dmfs)23. Enhanced community services included public service announcements, billboards, recruitment fliers, and brochures focused on behavioral risk factors for ECC and oral health topics covered in MI sessions. Audio and visual materials were developed and approved in association with the Community Advisory Board.
Development of a Culturally Sensitive MI Intervention
A manual describing the MI intervention was developed in an iterative process using a Community Advisory Board and focus groups of mothers/caregivers to refine material. Topic areas included maternal oral hygiene, minimizing vertical transmission of caries, appropriate use of bottle feeding, cariogenic foods, regular cleaning of the child’s mouth, and use of preventive services.
The MI manual foundation was from MI materials developed for other AI/AN projects with modification for ECC prevention guidelines.15,16 The ECC topic areas were emphasized in association with five main principles governing MI sessions: 1) expression of empathy 2) development of discrepancy between what is and what is desired 3) avoidance of argumentation 4) rolling with resistance (now designated as sustain talk or wanting to stay the same) and 5) support of self-efficacy.
MI Interventionist Recruitment and Training
Preferred criteria for interventionists included having a college degree (preferably B.A. or B.S. in psychology, human services, health education, nursing, or social work), familiarity with the reservation environment, and residence on/or near the Pine Ridge reservation.
The interventionists initially attended a two-day training course provided by the external MI expert (Reviewer 2) affiliated with the University of New Mexico, Albuquerque and Motivational Interviewing Network of Trainers (MINT) and serving as an external trainer and reviewer. The internal MI leader (Reviewer 1) completed MI training with the external MI expert to obtain expertise as a trainer and reviewer. The internal MI leader attended: a week-long course with the external MI expert to learn the MITI coding system, all training sessions conducted by the external expert to train study interventionists, and an advanced MI training course.
Recruited interventionists completed training to learn about MI and demonstrate an understanding of the MI spirit and key components. Interventionists practiced fundamental MI skills focused on guiding recipients in strengthening motivation to improve oral health behavior related to ECC. Following training, each interventionist completed calls with reviewers to address additional questions. Each interventionist conducted a minimum of five audio-recorded practice sessions with individuals similar to study participants in age and with young children. Sessions were assessed by the reviewers and feedback given to recruited interventionists by reviewers prior to initiation of the MI intervention. Reviewers analyzed the first five interventions done by each interventionist, followed by three of the second five interventions, and provided immediate feedback to interventionists. During the study, interactive feedback sessions were conducted semimonthly with each interventionist for the first six months, then monthly for the study duration.
Delivery of the MI Sessions
Each MI session followed a general outline: 1) participant choice of specific topic areas, relating this to community educational and promotional materials 2) exploring pros/cons of change in specific areas 3) assessing importance of change and confidence of the participant in ability to make the change 4) enhancement of self-efficacy through identification of what the participant thought was possible to accomplish 5) eliciting commitment language to follow through on the decision reached in the session 6) developing a follow-up plan with specific action steps.
Audio Recording of MI Sessions for the MI Fidelity Study
All MI sessions were audio recorded using Audacity® software on encrypted laptop computers. The internal MI leader (Reviewer 1) reviewed and scored a random subset of 20% of the total session recordings using the MITI 3.1.1 criteria. For purposes of interrater reliability, the external MI expert (Reviewer 2) reviewed and scored a 50% random sample of the recordings reviewed by the internal MI leader (a target of 240 by the internal MI leader and 120 by the external MI expert). Raters independently scored each audio file for global ratings and behavior counts. After scoring, the reviewers discussed each audio file reviewed to establish agreement/disagreement in accordance with MITI 3.1.1 scoring criteria.
MITI Instrument for Fidelity Scoring
The MITI is a one-pass coding system designed to measure treatment fidelity for MI clinical trials and provides structured, formal feedback about improving practice in non-research applications including clinician training or quality monitoring in clinical trials. The MITI measures two components, global ratings and behavior counts (Figure 1). The fidelity methodology for this study was based on the MITI 3.1.1 as the current version at the time.21
Figure 1.
MITI 3.1.1 Coding Sheet
Global scores are assigned using a Likert scale from 1 (lowest) to 5 (highest) for five global dimensions (evocation, collaboration, autonomy/support, direction, empathy) assessing the relational aspects of MI practice.25 Global scores of 3.5 indicate beginner competence and 4.0 expert competence. Direction and empathy are rated for interventionist feedback, but not calculated in the global MI score.
Behavior counts relate to the interventionists’ ability to provide questions (closed and open-ended), reflections (simple or complex), and statements (MI adherent or MI non-adherent) during the MI session. Closed-ended questions require only a yes or no response, while open-ended allow for a range of responses. Simple reflections repeat the recipient’s statements to convey understanding, whereas complex reflections include interpretation of recipient’s statements. Statements inconsistent with the MI approach are non-adherent and may include interaction that is confrontational, advisory, or directive. Beginner competence requires approximately equal number of reflections and questions, at least 50% of questions being open-ended, at least 40% of reflections being complex, and at least 90% of statements being MI-adherent. Expert competence requires twice as many reflections as questions, 70% of questions being open-ended, 50% of reflections being complex, and 100% of statements being MI adherent.
Statistical Analysis
Mean values and 95% confidence intervals were calculated for the five global dimensions and a global average calculated from the combined dimensions of evocation, collaboration, and autonomy/support. Scales for the global dimensions range from 1 (lowest) to 5 (highest). The reflection to question ratio was calculated by dividing the total number of reflections by the total number of questions. The percent of open-ended questions was calculated as the number of open-ended questions divided by the total number of questions and multiplied by 100. The percent of complex reflections was calculated as the number of complex reflections divided by the total number of reflections multiplied by 100. The percent of MI-adherent statements was calculated by dividing the number of MI-adherent statements by the total number of statements multiplied by 100.
Intra-class correlations (ICCs) were used to evaluate the inter-rater reliability between the reviewers. ICCs below 0.40 are considered poor, 0.40-0.59 fair, 0.60-0.74 good, and 0.75 and above excellent.
Study outcomes (changes in the mother’s oral health knowledge and behavior scores and dmfs scores for children calculated as Year 3 value minus baseline value) were compared among interventionists using a one-factor analysis of variance (ANOVA). Because dmfs data were skewed to the right, median values were examined and a non-parametric Kruskal-Wallis test was also performed. A p-value of 0.05 was considered statistically significant.
Results
Data are presented separately for both reviewers in relation to the interventionists combined and individually (Table 1). The largest differences resulted for the percent complex reflections measure, where Reviewer 1 had on average percentages 4-8 points higher compared to Reviewer 2. In combining scores for all interventionists, average global scores were in the range of expert competence (around 4.0). Reflection: question ratios (around 0.6) were below beginner competence (1.0) indicating more questions than reflections. The percent of open-ended questions (54-56%), complex reflections (38-43%), and MI-adherent statements (93-95%) were between beginner competence (50%, 40%, and 90%, respectively), and expert competence (70%, 50%, and 100%, respectively). There was variation in competence levels of the interventionists when analyzed individually.
Table 1.
Global scores and behavior counts by Reviewer 1 and Reviewer 2 for interventionists combined (ALL) and individually (IND)
| MITI Measure | Beginner Competence | Expert Competence | ALL | IND 1 | IND 2 | IND 3 | IND 4 |
|---|---|---|---|---|---|---|---|
|
| |||||||
| Reviewer 1 | N = 225 | N = 66 | N = 24 | N = 72 | N = 63 | ||
| Mean Evocation | 4.0 (3.9-4.1) | 4.0 (3.9-4.1) | 3.3 (3.0-3.6) | 3.6 (3.5-3.8) | 4.8 (4.7-4.9) | ||
| Mean Collaboration | 3.9 (3.8-4.0) | 4.0 (3.9-4.2) | 3.0 (2.8-3.3) | 3.4 (3.2-3.5) | 4.7 (4.6-4.9) | ||
| Mean Autonomy | 4.2 (4.1-4.3) | 4.1 (4.0-4.2) | 3.6 (3.3-3.8) | 3.9 (3.7-4.0) | 4.8 (4.7-4.9) | ||
| Mean Direction | 4.1 (4.0-4.2) | 4.1 (4.0-4.2) | 3.8 (3.6-4.0) | 3.7 (3.5-3.8) | 4.8 (4.7-4.9) | ||
| Mean Empathy | 4.0 (3.9-4.1) | 4.0 (3.9-4.2) | 3.0 (2.7-3.2) | 3.6 (3.4-3.7) | 4.8 (4.7-4.9) | ||
| Mean Global Score | 3.5 | 4.0 | 4.0 (4.0-4.1) | 4.1 (3.9-4.2) | 3.3 (3.1-3.5) | 3.6 (3.5-3.7) | 4.8 (4.7-4.9) |
| Reflection: Question Ratio | 1.0 | 2.0 | 0.6 (0.6-0.7) | 0.6 (0.5-0.7) | 0.4 (0.3-0.4) | 0.7 (0.6-0.8) | 0.7 (0.6-0.7) |
| % Open-Ended Questions | 50 | 70 | 55.8 (52.7-58.9) | 46.3 (41.8-50.8) | 34.0 (28.3-39.6) | 47.6 (43.6-51.6) | 83.4 (80.4-86.5) |
| % Complex Reflections | 40 | 50 | 42.8 (39.4-46.1) | 52.7 (46.8-58.6) | 20.0 (12.1-27.9) | 29.6 (25.2-34.0) | 56.1 (50.5-61.7) |
| % MI-Adherent Statements | 90 | 100 | 93.3 (91.6-95.1) | 95.7 (92.2-99.1) | 85.5 (77.7-93.3) | 88.6 (85.5-91.6) | 99.3 (98.4-100.0) |
|
| |||||||
| Reviewer 2 | N = 121 | N = 36 | N = 13 | N = 36 | N = 36 | ||
|
| |||||||
| Mean Evocation | 4.0 (3.8-4.1) | 3.7 (3.5-3.9) | 3.3 (2.8-3.8) | 3.6 (3.3-3.8) | 4.9 (4.8-5.0) | ||
| Mean Collaboration | 4.0 (3.8-4.1) | 3.9 (3.7-4.1) | 3.0 (2.7-3.3) | 3.5 (3.3-3.8) | 4.8 (4.7-4.9) | ||
| Mean Autonomy | 4.1 (4.0-4.3) | 4.1 (3.9-4.2) | 3.5 (3.1-3.9) | 3.9 (3.7-4.0) | 4.7 (4.5-4.9) | ||
| Mean Direction | 4.2 (4.1-4.3) | 4.1 (3.9-4.2) | 4.2 (3.9-4.4) | 3.9 (3.7-4.1) | 4.8 (4.6-4.9) | ||
| Mean Empathy | 4.0 (3.8-4.1) | 3.9 (3.7-4.0) | 3.2 (2.7-3.6) | 3.7 (3.5-3.9) | 4.6 (4.4-4.7) | ||
| Mean Global Score | 3.5 | 4.0 | 4.0 (3.9-4.1) | 3.9 (3.8-4.0) | 3.3 (2.9-3.7) | 3.6 (3.5-3.8) | 4.8 (4.7-4.9) |
| Reflection: Question Ratio | 1.0 | 2.0 | 0.6 (0.5-0.7) | 0.5 (0.4-0.6) | 0.4 (0.3-0.5) | 0.6 (0.5-0.7) | 0.7 (0.6-0.8) |
| % Open-Ended Questions | 50 | 70 | 53.9 (49.6-58.2) | 42.0 (35.7-48.3) | 31.7 (25.6-37.8) | 48.3 (43.3-53.2 | 79.3 (73.3-85.4) |
| % Complex Reflections | 40 | 50 | 37.7 (33.5-41.9) | 48.1 (40.2-56.0) | 11.9 (6.1-17.7) | 26.1 (21.1-31.1) | 48.8 (41.9-55.7) |
| % MI-Adherent Statements | 90 | 100 | 94.6 (92.8-96.4) | 97.3 (95.4-99.1) | 87.2 (76.8-97.5) | 89.9 (86.2-93.6) | 99.5 (98.8-100.0) |
95% Confidence Intervals in Parentheses
Table 2 presents the intra-class correlations, which measured agreement between Reviewers 1 and 2. The ICCs were in the “fair” range (0.40-0.59) for seven measures, and in the “good” range (0.60-0.74) for three measures.
Table 2.
Interrater Reliability Between Reviewer 1 and Reviewer 2: Intra-class Correlations (ICCs) and 95% Confidence Intervals (CI)
| MITI Measure | ICC (95% CI) N = 121 |
|---|---|
| Evocation | 0.54 (0.44-0.64) |
| Collaboration | 0.63 (0.53-0.71) |
| Autonomy | 0.49 (0.37-0.60) |
| Direction | 0.59 (0.48-0.69) |
| Empathy | 0.51 (0.39-0.62) |
| Global Score | 0.62 (0.52-0.71) |
| Reflection: Question Ratio | 0.43 (0.30-0.56) |
| Percent Open-Ended Questions | 0.66 (0.56-0.74) |
| Percent Complex Reflections | 0.43 (0.31-0.56) |
| Percent MI-Adherent Statements | 0.40 (0.27-0.53) |
The interventionists’ educational and cultural background were examined in relation to MI competence. Interventionist 4 had an Associate degree in Nursing and B.A. degrees in Psychology and Spanish and was rated best in MI competence. Interventionist 1 had a B.A. degree in Criminal Justice and was rated second best. Interventionist 3 had a B.A. degree in Social Work with MI competence between interventionists 4 and 2. Interventionist 2 did not have a college degree and was rated worst. All but interventionist 3 were AI.
In the clinical trial, 289 mothers were randomized to the MI intervention: 91 received the intervention from the same interventionist, 140 from two interventionists, 51 from three interventionists, and 7 from four interventionists. To examine the interventionists’ effect on study outcomes, data were assessed from 79 participants with the same interventionist throughout the study and at least two MI visits, and 73 participants with two interventionists and three MI visits from the same interventionist (total sample = 152).
Table 3 presents outcome data (child dmfs at Year 3, and change in oral health knowledge and behavior scores of the mother from baseline to Year 3) by MI interventionist. For knowledge and behavior change scores, a positive value was an improvement and a negative value a decline. Outcome data were compared among interventionists 1, 3, and 4 as too few outcome data for interventionist 2 precluded analysis. Although interventionist 4 was rated best from MI fidelity data, associated dmfs data were worst among the three interventionists. Interventionist 1 was second-best from MI fidelity data with slightly better dmfs data, but associated with worse oral health behavior data compared to interventionist 3 (rated worst from MI fidelity data). Changes in oral health knowledge were not statistically significant among the three interventionists.
Table 3.
Changes in Study Outcomes by MI Interventionist
| Interventionist MI Competence | Study Outcome | N | Mean | SD | Median |
|---|---|---|---|---|---|
| 4 (Best) | Year 3 dmfs | 38 | 15.34 | 17.78 | 8 |
| Knowledge change score | 38 | 5.02 | 16.6 | 5.88 | |
| Behavior change score | 37 | −22.58 | 22.27 | −25 | |
| 1 (Middle) | Year 3 dmfs | 26 | 7.00 | 12.76 | 0 |
| Knowledge change score | 27 | 9.11 | 13.61 | 11.76 | |
| Behavior change score | 27 | −15.76 | 21.55 | −16.67 | |
| 3 (Worst) | Year 3 dmfs | 61 | 9.72 | 14.04 | 2 |
| Knowledge change score | 45 | 3.40 | 12.78 | 5.88 | |
| Behavior change score | 45 | −7.94 | 28.30 | −3.85 |
Year 3 dmfs: ANOVA p=0.07; Kruskal-Wallis p=0.01
Knowledge change score: ANOVA p=0.27; Kruskal-Wallis p=0.21
Behavior change score: ANOVA p=0.03; Kruskal-Wallis p=0.01
Discussion
Application of MI in oral health interventions is an increasing focus, yet MI fidelity assessment has not been addressed. To date a single study has reported on findings from fidelity assessment of an oral health MI intervention in an indigenous group of Aboriginal Australians.26 The current study adds to the fidelity knowledge base by examining fidelity of MI in an Americans Indian cohort as part of a larger global network of indigenous people.
On average, study interventionists performed in the expert competence range for the global ratings and at the level of beginner to expert competence for the behavior counts excluding the question to reflection ratio which was below beginner competence. Lower competence for the reflection to question ratio may have resulted from reflections being simple rather than complex. This difficulty was observed in the Australian Aboriginal study26 and cultural factors and related contextual nuances may have contributed.27 Further studies are merited to evaluate communication challenges in indigenous populations and influence from cultural factors on MI delivery to determine acceptable levels of fidelity apart from contextual factors in diverse populations.19,26
In comparing adherence of this study with National Institute of Health BCC recommendations for evaluating fidelity in behavior change interventions, recommended guidelines and strategies were implemented in relation to study design, training, and intervention delivery (Table 4). Excellent participation was observed with a high majority of participants completing at least three of the four MI visits. Training of interventionists was standardized by trainer and training materials to ensure skill acquisition met criteria. An interim workshop was provided by the study trainers to enhance interventionists competence in intervention delivery and address specific areas of lower competence with improvement measured by targeted scoring of audio files.
Table 4.
Comparison of National Institute of Health Behavior Change Consortium (BCC) Fidelity Recommendations and Implementation in Oral Health MI Intervention
| BCC Fidelity Domain | BCC Strategies | Oral Health MI Intervention |
|---|---|---|
| Study Design | Treatment protocol tests the hypothesis | Yes |
| Plan for implementation setbacks | Yes | |
|
| ||
| Training of Interventionists | Standardized training protocol | Yes |
| Skill acquisition measured, feedback, coaching, role-playing | Yes | |
|
| ||
| Intervention Delivery | MITI assessment | Yes |
| Processes to monitor and improve delivery | Yes | |
The interventionists’ educational background in health or human services appeared to influence MI competence yet was not associated with better study outcomes, while cultural background did not appear to strongly influence MI competence. The number of overall sessions delivered by an interventionist and maintaining the same interventionist for the MI sessions delivered to the participant were not associated with improved study outcomes. Findings suggest interventionist competence may not be the sole explanatory factor when analyzing study outcomes for MI oral health interventions involving caregiver participants. Study outcomes may be influenced by participant behaviors relative to intervention receipt and/or enactment of treatment skills, key BCC components for evaluating behavioral interventions, yet not part of the MITI 3.1.1. To partially address this limitation, the MITI 4.2.1 offers a more detailed assessment of interventionists’ persuasion in the behavior counts, and cultivation of change talk and softening of sustain talk in the global ratings.
Establishing a comprehensive method to maintain fidelity for MI interventions requires sufficient support due to fidelity assessment being a complex process. Being in a remote site made recruitment of well-prepared interventionists challenging. Due to rolling enrollment and varying needs in numbers of interventionists, some participants did not have the same interventionist for all MI sessions. Reviewers concluded ideal characteristics of an interventionist includes not only mastery of technical skills but ability to establish an interpersonal relationship reflecting a more complex exchange to engender collaboration and empathy, key elements of successful MI interventions.22
Study findings were evaluated in light of limitations in maintaining and assessing fidelity in MI studies. The MITI 3.1.1 used in this study has been revised as the MITI 4.2.1 version which addresses previous weaknesses involving inability to reliably measure empathy and direction in the global scores and assess interventionist’s use of additional factors including collaboration.22 Future fidelity studies will benefit from integration of the MITI 4.2.1 as the latest instrument. The reviewers’ interrater ICC reliability scores were in the fair to good range. Reviewers conducted regular calls to discuss coding, yet ICCs were not conducted until the study end. Higher frequency in reviewing coding may have led to additional MI coding training to increase reliability.
As a complex approach that is empirically supported, MI application in clinical research requires an intervention design that identifies active elements contributing to outcomes while providing adequate standardization for comprehensive assessment of integrity.28 As the first fidelity study for an oral health MI intervention in AI participants, findings confirmed fidelity studies provide useful feedback for interventionists and reviewers to ensure outcomes are valid and reliable while highlighting the complex relationship with study outcomes and participant influences.
Acknowledgments
Research reported in this publication was supported by the National Institute of Dental and Craniofacial Research of the National Institutes of Health (U54DE019259, U54DE019275, U54DE019285). The content is solely the responsibility of the authors and does not necessarily represent official views of the National Institutes of Health.
Footnotes
The authors declare no potential conflicts of interest with respect to authorship and/or publication of this article.
DR ANNE R WILSON (Orcid ID : 0000-0002-3304-8559)
References
- 1.Phipps KR, Ricks TL, Manz MC, Blahut P. Prevalence and severity of dental caries among American Indian and Alaska Native preschool children. J Public Health Dent. 2012;72:208–215. doi: 10.1111/j.1752-7325.2012.00331.x. [DOI] [PubMed] [Google Scholar]
- 2.Braun PA, Quissell DO, Henderson WG, Bryant LL, Gregorich SE, George C, et al. A Cluster-Randomized, Community-Based, Tribally Delivered Oral Health Promotion Trial in Navajo Head Start Children. J Dent Res. 2016;95:1237–1244. doi: 10.1177/0022034516658612. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Warren JJ, Blanchette D, Dawson DV, Marshall TA, Phipps KR, Starr D, et al. Factors associated with dental caries in a group of American Indian children at age 36 months. Community Dent Oral Epidemiol. 2016;44:154–161. doi: 10.1111/cdoe.12200. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Batliner T, Tiwari T, Wilson A, Brinton J, Daniels DM, Gallegos JR, et al. An assessment of oral health on the Pine Ridge Reservation. Fourth World J. 2013;12:5–17. [Google Scholar]
- 5.United States Department of Health and Human Services. Health Resources and Services Administration. https://datawarehouse.hrsa.gov/topics/shortageAreas.aspx. Accessed November 14, 2017. (Archived by WebCite® at: http://www.webcitation.org/6uyhlWfh5)
- 6.Wilson A, Brega AG, Batliner TS, Henderson W, Campagna EJ, Fehringer K, et al. Assessment of parental oral health knowledge and behaviors among American Indians of a Northern Plains tribe. J Public Health Dent. 2014;74:159–167. doi: 10.1111/jphd.12040. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Miller WR, Sorensen JL, Selzer JA, Brigham GS. Disseminating evidence-based practices in substance abuse treatment: a review with suggestions. J Subst Abuse Treat. 2006;31:25–39. doi: 10.1016/j.jsat.2006.03.005. [DOI] [PubMed] [Google Scholar]
- 8.Weinstein P, Harrison R, Benton T. Motivating parents to prevent caries in their young children: one-year findings. J Am Dent Assoc. 2004;135:731–738. doi: 10.14219/jada.archive.2004.0299. [DOI] [PubMed] [Google Scholar]
- 9.Harrison RL, Veronneau J, Leroux B. Effectiveness of maternal counseling in reducing caries in Cree children. J Dent Res. 2012;91:1032–1037. doi: 10.1177/0022034512459758. [DOI] [PubMed] [Google Scholar]
- 10.Wagner Y, Greiner S, Heinrich-Weltzien R. Evaluation of an oral health promotion program at the time of birth on dental caries in 5-year-old children in Vorarlberg, Austria. Community Dent Oral Epidemiol. 2014;42:160–169. doi: 10.1111/cdoe.12072. [DOI] [PubMed] [Google Scholar]
- 11.Plutzer K, Spencer AJ, Keirse MJ. Reassessment at 6–7 years of age of a randomized controlled trial initiated before birth to prevent early childhood caries. Community Dent Oral Epidemiol. 2012;40:116–124. doi: 10.1111/j.1600-0528.2011.00643.x. [DOI] [PubMed] [Google Scholar]
- 12.Gao X, Lo ECM, McGrath C, Ho SMY. Innovative interventions to promote positive dental health behaviors and prevent dental caries in preschool children: study protocol for a randomized controlled trial. Trials. 2013;14:118. doi: 10.1186/1468-6708-14-118. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Merrick J, Chong A, Parker E, Roberts-Thomson K, Misan G, Spencer J, et al. Reducing disease burden and health inequalities arising from chronic disease among Indigenous children: an early childhood caries intervention. BMC Public Health. 2012;12:323. doi: 10.1186/1471-2458-12-323. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Rollnick S, Miller WR. What is motivational interviewing? Behavioural and Cognitive Psychotherapy. 1995;23:325–334. doi: 10.1017/S1352465809005128. [DOI] [PubMed] [Google Scholar]
- 15.Tomlin K, Walker RD, Grover J, Arquette W, Stewart P. Motivational interviewing: enhancing motivation for change—a learner’s manual for the American Indian/Alaska Native counselor [Internet] Portland, OR: Oregon Health and Science University; 2014. [cited 2017 July 13]. http://www.oneskycenter.org/wp-content/uploads/2014/03/LearnersManualforMotivationalInterviewing.pdf. Accessed November 14, 2017. (Archived by Website® at: http://www.webcitation.org/6uyifyfB6) [Google Scholar]
- 16.Venner K, Feldstein S, Tafoya N. A manual for counselors in Native American communities. Center on Alcoholism, Substance Abuse, and Addictions; Department of Psychology; University of New Mexico: 2006. Native American Motivational Interviewing: Weaving Native American and Western Practices. https://www.integration.samhsa.gov/clinical-practice/Native_American_MI_Manual.pdf. Accessed November 14, 2017. [Google Scholar]
- 17.Moncher FJ, Prinz RJ. Treatment fidelity in outcome studies. Clin Psychol Review. 1991;11:247–266. [Google Scholar]
- 18.Borrelli B, Sepinwall D, Ernst D, Bellg AJ, Czajkowski S, Breger R, et al. A new tool to assess treatment fidelity and evaluation of treatment fidelity across 10 years of health behavior research. J Consult Clin Psychol. 2005;73:852–860. doi: 10.1037/0022-006X.73.5.852. [DOI] [PubMed] [Google Scholar]
- 19.Bellg AJ, Borrelli B, Resnick B, Hecht J, Minicucci DS, Ory M, et al. Enhancing treatment fidelity in health behavior change studies: best practices and recommendations from the NIH Behavior Change Consortium. Health Psychol. 2004;23:443–451. doi: 10.1037/0278-6133.23.5.443. [DOI] [PubMed] [Google Scholar]
- 20.Moyers TB, Martin T, Manuel JK, Hendrickson SM, Miller WR. Assessing competence in the use of motivational interviewing. J Subst Abuse Treat. 2005;28:19–26. doi: 10.1016/j.jsat.2004.11.001. [DOI] [PubMed] [Google Scholar]
- 21.Moyers T, Martin T, Manual J, Miller W, Ernst D. Revised Global Scales: Motivational Interviewing Treatment Integrity 3.1.1 (MITI 3.1.1) University of New Mexico: Center on Alcoholism, Substance Abuse and Addictions (CAASA); Revised January 22, 2010. https://www.integration.samhsa.gov/clinical-practice/Native_American_MI_Manual.pdf. Accessed November 14, 2017. [Google Scholar]
- 22.Moyers TB, Rowell LN, Manuel JK, Ernst D, Houck JM. The Motivational Interviewing Treatment Integrity Code (MITI 4): Rationale, Preliminary Reliability and Validity. J Subst Abuse Treat. 2016;65:36–42. doi: 10.1016/j.jsat.2016.01.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Batliner T, Fehringer KA, Tiwari T, Henderson WG, Wilson A, Brega AG, et al. Motivational interviewing with American Indian mothers to prevent early childhood caries: study design and methodology of a randomized control trial. Trials. 2014;15:125. doi: 10.1186/1745-6215-15-125. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Wilson AR, Brega AG, Campagna EJ, Braun PA, Henderson WG, Bryant LL, et al. Validation and impact of caregivers’ oral health knowledge and behavior on children’s oral health status. Pediatr Dent. 2016;38:47–54. [PMC free article] [PubMed] [Google Scholar]
- 25.Miller WR, Rose GS. Toward a theory of motivational interviewing. Am Psychol. 2009;64:527–537. doi: 10.1037/a0016830. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Jamieson L, Bradshaw J, Lawrence H, Broughton J, Venner K. Fidelity of Motivational Interviewing in an Early Childhood Caries Intervention Involving Indigenous Australian Mothers. J Health Care Poor Underserved. 2016;27:125–138. doi: 10.1353/hpu.2016.0036. [DOI] [PubMed] [Google Scholar]
- 27.Fixsen DL, Naoom SF, Blase KA, Friedman RM, Wallace F. Implementation research: A synthesis of the literature Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute. The National Implementation Research Network (FMHI Publication #231) 2005;11:247–266. [Google Scholar]
- 28.Hecht J, Borrelli B, Breger RK, Defrancesco C, Ernst D, Resnicow K. Motivational interviewing in community-based research: experiences from the field. Ann Behav Med. 2005;29:29–34. doi: 10.1207/s15324796abm2902s_6. [DOI] [PubMed] [Google Scholar]