“Allowing space for voice…all our voices:” Understanding Ho’ouna Pono implementation through educational leadership perspectives in rural Hawai’i schools

Abstract

Epidemiological research over the past two decades has highlighted substance use disparities that affect Native Hawaiian and Pacific Islander youth, and the lack of effective approaches to address such disparities (Okamoto et al., 2019). The Ho’ouna Pono curriculum is a culturally grounded, teacher-implemented, video-enhanced substance use prevention program that has demonstrated efficacy in rural Hawaiʻi in a large-scale trial (Okamoto et al., 2019). Despite its potential to ameliorate health disparities and address youth substance use, prevention programs such as Ho’ouna Pono have been poorly disseminated and implemented across Hawaiʻi, raising the question: Why are effective prevention programs not used in communities that most need them? The present study used concept mapping to understand previously identified implementation barriers and develop implementation strategies for Ho’ouna Pono. Seven Hawaiʻi Department of Education (HIDOE) educational leaders and administrators sorted Ho’ouna Pono implementation barriers (e.g., “There is a lack of HIDOE funding to support prevention curricula”), named concepts, and rated barriers’ perceived impact and difficulty. Multidimensional scaling and cluster analysis yielded a five-cluster solution: (1) Kumu (Hawaiian word for teacher) Controlled, (2) School Level Buy-in, (3) Curriculum, (4) Student Attitudes + Mindsets (Family + Community), and (5) Policy. Participant ratings identified eight high-impact and low-difficulty barriers. Discussion revealed important intersections among barriers indicating the need for coordinated and cross-level implementation strategies to support Ho’ouna Pono sustainment. Brainstormed implementation strategies using participants’ own language highlighted a need for participatory methods in school settings to bidirectionally share ways to best sustain substance use prevention programs.

Decades of research have indicated the need for substance prevention programs for Native Hawaiian and Pacific Islander (NHPI) youth, given their higher rates of drug use than non-NHPI youth (Mayeda et al., 2006; Nishimura et al., 2013; Wong et al., 2004). While few programs have been developed specifically for NHPI youth (Durand et al., 2015; Edwards et al., 2010) and for rural communities (Okamoto et al., 2014; Saka et al., 2014), even fewer are developed with community engaged and culturally grounded approaches (Okamoto et al., 2014). Culturally grounded prevention programs can reach and motivate youth by reflecting the cultural, community, and social context of the targeted population (Marsiglia & Kulis, 2009). One such program that has demonstrated efficacy is Ho’ouna Pono, a video-enhanced, teacher-delivered prevention intervention developed and tested with youth from rural Hawaiʻi Island. The intervention significantly improved youth substance use resistance strategies (e.g., avoid, explain, leave) in a 6-month randomized pilot study (Okamoto et al., 2016). In a subsequent fully powered, large-scale randomized controlled trial, receipt of the intervention was associated with a significantly slower rate of growth in cigarette and e-cigarette use (Okamoto et al., 2019).

Despite these promising advances in building prevention interventions for youth in their schools and communities, implementation has often been hindered by various contextual barriers, including shifting school priorities, teacher perceptions of the intervention, and changing societies and communities (Eisman et al., 2022; Okamura et al., 2022). Owens and colleagues (2014) noted that schools are uniquely challenging settings for implementation. They often differ from each other in their organizational culture, include professionals from a variety of disciplines (e.g., teachers, administrators, psychologists), and the school calendar may truncate innovation implementation efforts. Eisman et al. (2022) identified acceptability, intervention-context fit, and adaptability as implementation domains that posed challenges to teachers implementing a universal health prevention intervention with fidelity. Other school-based implementation research has found that organization characteristics like leadership and climate impact both individual attitudes toward the intervention and fidelity (Locke et al., 2016; Locke et al., 2020; Shoesmith et al., 2021; Zhang et al., 2022).

In our research about factors affecting Ho’ouna Pono implementation, teachers and other school staff identified competing priorities, lack of training, and external (i.e., school administration) pressure for other initiatives as implementation barriers (Okamoto et al., under review). Another important barrier identified was the emergence of new drug modalities and devices (e.g., electronic nicotine delivery systems, e-cigarettes, vapes) that were not addressed by the intervention. Moreover, Ho’ouna Pono has not been sustained in the schools that participated in the efficacy trial and there have not been efforts to promote widespread, systematic implementation of the intervention. Therefore, as we revise the program to address new drug modalities and devices, we are concurrently attempting to generate implementation strategies: such work could promote widespread implementation of an efficacious prevention intervention and shift substance use trajectories for students who engage in the intervention.

Implementation Frameworks

Implementation science employs a wide array of theories, frameworks, and measures to explore context and systematically study the process of translating science to practice (Eccles & Mittman, 2006; Wang et al., 2023). Determinant frameworks are used to identify the contextual factors that impede or facilitate implementation in a specific setting. One commonly used determinant framework is the Consolidated Framework for Implementation Research (CFIR), which was first developed by Damschroder and colleagues in 2009 and updated in 2022 based on user feedback (Damschroder et al., 2022). The current study launched in 2021 and therefore used the original version of the CFIR. Both versions of the CFIR organized implementation barriers and facilitators into five domains: (a) outer setting – the system where an organization implementing an innovation exists, (b) inner setting – the organization implementing the innovation, (c) characteristics of individuals – the people implementing the innovation, (d) characteristics of the innovation – the “thing” being implemented (e.g., Ho’ouna Pono), and (e) process – the stages by which the innovation is implemented.

The CFIR was used in a previous Ho’ouna Pono study to identify 50 barriers via focus groups with 24 Hawaiʻi State Department of Education (HIDOE) teachers and frontline staff (Okamoto et al., 2020). Focus group members were asked open-ended questions about Ho’ouna Pono implementation based on CFIR domains and revealed consistent themes, including the outer setting (e.g., “Marijuana use is socially acceptable on Hawai’i island, diminishing the need”), inner setting (e.g., “There is a lack of HIDOE funding to support prevention curricula”), and individual characteristics of the implementor (e.g., “Some HIDOE teachers are resistant to trying new curricula…because it feels like one more thing administrators want [us] to do in the classroom”).

A key benefit of using a determinant framework such as CFIR is that the identification of barriers and facilitators can be used to develop effective implementation strategies (Proctor et al., 2013). Implementation strategies are the actions that community members can take to implement an innovation in a specific setting. Powell and colleagues identified and defined 73 discrete implementation strategies that comprise the Expert Recommendations for Implementing Change (ERIC) strategy compilation (Powell et al., 2012; Powell et al., 2015; Waltz et al., 2015). Many websites, institutes, and technical assistance centers contain resources for using, identifying and selecting implementation strategies, which may be useful for school-based implementation efforts (e.g., Active Implementation Hub, https://implementation.fpg.unc.edu/; Penn Implementation Science Institute, https://mshp.med.upenn.edu/implementation-science-institute; University of Washington, https://impsciuw.org/). However, these resources take some expertise to navigate, which may pose a challenge for school-based staff less familiar with implementation science. Using existing implementation frameworks and taxonomies to create a shared understanding around implementation between researchers and school-based staff is a valuable first step for implementation of efficacious interventions such as Ho’ouna Pono.

Participatory Methods

Participatory methods create processes and change by promoting democratic collaboration with those affected by an issue through meaningful engagement (Vaughn & Jacquez, 2020). For Ho’ouna Pono, engagement with school-based staff and leadership in co-creating implementation and evaluation aligns with participatory method values (Ramanadhan et al., 2024). Concept mapping is one promising participatory approach that “integrates well-known group processes such as brainstorming and unstructured sorting with multivariate statistical methods” (Trochim & Kane, 2005, p. 187) to generate and validate community-derived meaning of constructs (Powell et al., 2017). The concept mapping steps include (a) preparation, (b) idea generation, (c) organization/sorting/rating, (d) visual representation, (e) interpretation, and (f) utilization. We used concept mapping to facilitate the co-creation of the implementation process and promote bidirectional learning between researchers and school-based staff (see Okamura et al., 2020 for more details). Concept mapping can also be used to identify and prioritize implementation barriers, which begins by selecting participants, defining the conceptual focus, generating a list of barriers, and gaining consensus on the most critical barriers. These steps were completed in a prior trial (Okamoto et al., under review), which obtained consensus around a list of 50 barriers to the intervention (described further in Methods).

The current study is built on this formative work and initial concept mapping steps to develop an implementation action plan for Ho’ouna Pono. The study progressed through two phases: ranking, sorting, and reflecting on the barriers using the CFIR determinant framework (Phase One) and refining the concept map and generating implementation strategies using the ERIC taxonomy (Phase Two). The goal of concept mapping was to understand Ho’ouna Pono implementation barriers from the perspective of educational leadership in Hawai’i Island and HIDOE, while promoting state-, district/complex-, and school-level investment and “buy-in” for the strategies used to implement the curriculum (Waltz et al., 2019). This participatory approach aligns with other community-engaged, team-based, and equitable methods to ensure equal voice in preparing, implementing, and sustaining an intervention (Vaugh & Jacquez, 2020). The study was exploratory with the goal of generating a shared understanding of Ho’ouna Pono implementation by eliciting concepts and strategies for further evaluation and testing.

Method

Participants

The first author and a HIDOE consultant/co-author on Hawaiʻi Island created a list of potential state (n = 13) and complex/district (n = 17) participants based on their leadership roles in promoting health education standards, health curriculum, and district- and school-level decision making. The research staff sent recruitment emails to the group and followed up via individual emails and phone calls as needed. Seven HIDOE state and complex leaders consented and participated in this study. It was important to recruit leaders who did not have experience implementing the curriculum to both provide insight into teacher and other school staff perspectives and to engage leadership to develop strategies that could be addressed with their positionality and role. The group included one Complex Area Superintendent, five State or District Education Specialists, and one School-based Behavioral Health Specialist. Participants self-identified as predominantly female (62.5%) and as Native Hawaiian (16.7%), Asian (25%), Portuguese (16.7%), White (25%), and Other Pacific Islander (8.3%). One participant did not identify their racial background. One third of participants held a master’s degree or higher.

Procedure

The objective, guiding framework, and key strategies used in each study phase are outlined in Table 1. The two study phases were: (a) in-person sorting, ranking, and discussion and (b) online naming, reorganizing, and strategy brainstorming. The research team conducted multidimensional scaling and cluster analysis (see Data Analytic Plan) between meetings. After the first meeting, two participants that could not attend the in-person sorting completed their sorting and rating asynchronously. One person opted out of the in-person ranking activity. Participants were compensated for each time point with a $50 donation to their school or complex area. Data integrity and security were maintained through standardized procedures aligned with university standards with only trained staff having data access. This study was deemed exempt by the University of Hawai’i at Mānoa, Hawai’i Pacific University, and Hawai’i Department of Education institutional review boards.

Table 1.

Research Design

	Phase
	One	Two
Goal/Purpose	Implementation barrier cluster sorting, rating, and reflecting	Cluster map refinement and implementation strategy generation
Guiding Implementation Framework	CFIR	ERIC
Methodology	Concept mapping, focus group discussion	Concept mapping, focus group discussion
Analysis	Multidimensional Scaling, Cluster Analysis, Go Zone, Thematic Analysis	Cluster Refinement, Thematic Analysis

Note. CFIR = Consolidated Framework for Implementation Research (Damschroder et al., 2009); ERIC = Expert Recommendations for Implementing Change (Waltz et al., 2014).

Phase One - Sorting, Rating, and Reflecting.

In the first phase, participants met for two hours (with lunch included) to complete informed consent, collect background demographic and training information, and sort and rank barriers. Consensus was attained on fifty barriers in a prior study via focus groups with 24 Hawaiʻi State Department of Education (HIDOE) teachers and frontline staff (Okamoto et al., 2020). In these focus groups, staff were asked open-ended questions about Ho’ouna Pono implementation based on CFIR domains. Participant feedback yielded a set of consistent barriers, spanning the outer setting (e.g., “Marijuana use is socially acceptable on Hawai’i island, diminishing the need”), inner setting (e.g., “There is a lack of HIDOE funding to support prevention curricula”), and individual characteristics of the implementor (e.g., “Some HIDOE teachers are resistant to trying new curricula…because it feels like one more thing administrators want [us] to do in the classroom”).

Participants were instructed to sort the 50 barriers based on their own version of how they are related. General guidance was given to participants that clarified that concept groupings tended to range from five to 20 concepts, avoided labels like “important” or “other,” and could be composed of a single statement if the statement did not relate to others. Participants also ranked barriers on a five-point scale regarding the extent to which they perceived a particular barrier as impacting implementation (1 = no impact, 2 = minimal impact, 3 = some impact, 4 = significant impact, and 5 = substantial impact) and the difficulty in overcoming each barrier (1 = very easy, 2 = easy, 3 = a little difficult, 4 = moderately difficult, and 5 = very difficult). A general discussion closed the first phase with participants answering open reflection questions such as “What are the most impactful barriers for you?” and “Describe implementation strategies to address these barriers.” The first author facilitated the discussion and provided summaries, prompts, and definitions for terms with the goal of producing insight and thoughtfulness for the next phase.

Phase Two - Refining of the Concept Map and Generation of Strategies.

In the second phase, concepts refined by multidimensional scaling and cluster analysis were shared with participants. Participants were shown various concept mapping visualizations (i.e., cluster map, go zone) to understand and reflect on their individual understanding relative to the group’s collective understanding of Ho’ouna Pono implementation barriers. The group reviewed individual barriers within clusters, reorganized clusters (i.e., moved individual barriers from one cluster to another), and renamed them to reflect shared understanding. A final discussion question followed each concept asking participants to reflect on “What implementation strategies could be used to address this concept/cluster of barriers?” Participants used the ERIC compilation (Powell et al., 2012; Powell et al., 2015; Waltz et al., 2015) as a resource to inspire potential strategies but they could also brainstorm strategies and ways of operationalizing those strategies using their own terms and language. There was no prioritization or selection of strategies beyond the brainstorming discussion. The research team told participants that these brainstormed strategies would inform the implementation action plan and future implementation research.

Data Analytic Plan

Demographic information, sort piles, and difficulty and impact ratings were entered and verified in GroupWisdom software (Concept Systems Incorporated, 2021). Multidimensional scaling was used to create a matrix of similarities between statements and HIDOE groupings from the sorting task. A two-dimensional solution served as the input for a cluster analysis using Ward’s algorithm, which uses minimal variance at each step to identify the optimal value of an objective function to define orthogonal clusters (Anderberg, 1973; Everitt, 1980). Cluster analysis adhered to the typical rule of at least five statements per cluster. Eleven solutions were examined based on point map and interpretability of the clusters (Johnsen et al., 2000; Trochim & McLinden, 2017). Difficulty and impact ratings were layered onto the cluster map to understand relative difficulty and impact via Go Zone graph. The concept mapping discussions were transcribed and reviewed to contextualize clusters and strategies.

Results

Phase One: Sorting, Rating, and Reflecting

The point map (stress value¹ = 0.19, iterations² = 18; not presented) visually represented multidimensional scaling and the extent to which each barrier was commonly grouped with other barriers. Cluster analysis revealed maps with four to 14 clusters, and bridging values ranging from zero to one were examined by item and cluster. Lower bridging values are typically considered anchoring items that represent homogeneity among items whereas higher bridging values indicate a higher relationship across various statements on the map. Layer values aggregate bridging values for the concept, with higher values indicating more connectedness with other concepts and items within the map. After reviewing the total eleven cluster maps, the research team reviewed the four-, five-, and six-cluster solution items for cohesive themes as identified by each label. The five-cluster solution was chosen based on stress values, iterations, bridging values, layer values, and similarities to CFIR domains. Table 2 presents the initial and final cluster solutions with individual items, and Figure 1 shows the final cluster map described in phase two of the study.

Table 2.

Ho’ouna Pono Concept Mapping Barriers and Brainstormed Implementation Strategies

I. Kumu Controlled (CFIR Individual Characteristics and Implementation Process)		Implementation Strategies
1	The frequent turnover of teachers in my school will disrupt the ongoing implementation of Ho’ouna Pono	Coordinated and cohort-based training Curriculum integrated into existing lesson plans to address all students Drug treatment counselor, school counselor, and school-based behavioral health specialist involvement
2	The delivery of the Ho’ouna Pono curriculum is assigned to a teacher who is resistant to using the curriculum
5	The responsibility for teaching health frequently shifts from one teacher to another teacher in my school
7	It is difficult for teachers to talk about drug and alcohol use in the classroom setting with their students
15	HIDOE teachers who prefer didactic teaching methods may not feel comfortable with the interactive approach of the Ho’ouna Pono curriculum
16	Senior HIDOE teachers are comfortable with their current drug and alcohol lessons, and are therefore not interested in the Ho’ouna Pono curriculum
23	Incorporating the Ho’ouna Pono curriculum into my curriculum map will take planning and effort
25	Drug prevention curricula like Ho’ouna Pono encourage the discussion of sensitive topics, which could lead to overstepping student-teacher interpersonal boundaries
28	Some HIDOE teachers believe their current drug and alcohol content is effective, so they don’t see any added benefit to the Ho’ouna Pono curriculum
29	Some HIDOE teacher believe that drug use is symptomatic of deeper issues (e.g., depression), which should instead be the focus of prevention
31	Some HIDOE teachers believe that drug prevention curricula like Ho’ouna Pono have limited impact in promoting internalization of youth’s drug-free attitudes
33	Some HIDOE teachers feel that prepackaged curricula like Ho’ouna Pono threaten their autonomy in the classroom
39	Lack of familiarity with Ho’ouna Pono makes me hesitant to use the curriculum
45	Teachers may not use the Ho’ouna Pono curriculum, because they are afraid that it could prompt their students to ask them about their personal drug use histories
48	Some HIDOE teachers are resistant to trying new curricula like Ho’ouna Pono, because it feels like “one more thing you [administrators] want me to do in the classroom”
II. School Level Buy-in (School level; CFIR Inner Setting)
13	The implementation and training related to the Ho’ouna Pono curriculum will be difficult to fit into teachers’ busy schedules	More required health education courses in middle school Create new incentives for continuing education for teachers Align educational policies with health and well-being outcomes
20	HIDOE job performance measures (e.g., CESA) do not explicitly measure growth on student health indicators
27	There may be a lack of enthusiasm for the Ho’ouna Pono curriculum because health education is not prioritized in the HIDOE
40	HIDOE senior teachers at the top of the salary scale (Class 7) do not need PDE credits associated with implementing the Ho’ouna Pono curriculum for reclassification
41	Substance use is not perceived to be a major problem in my school
42	The frequent turnover of principals in my school will disrupt the ongoing implementation of Ho’ouna Pono
49	HIDOE administrators are resistant to endorsing Ho’ouna Pono, due to fear of asking overburdened teachers to try something new in the classroom
50	Natural disasters on Hawaiʻi Island (e.g., lava, hurricanes) disrupt school-based prevention efforts, like Ho’ouna Pono
III. Curriculum (CFIR Innovation)
3	The Ho’ouna Pono curriculum introduces terms that might be difficult for ESL students to understand	Use teachers to develop the curriculum
8	The Ho’ouna Pono curriculum is structured primarily on skill-building (e.g., decision making) rather than on teaching standard health topics (e.g., facts about drugs and alcohol)
10	The Ho’ouna Pono curriculum would require additional work for teachers in immersion schools who need to translate the lessons into Hawaiian
*21	The Ho’ouna Pono curriculum does not extensively cover current or recent forms of substance use, such as vaping
*22	After 10 years, the Ho’ouna Pono curriculum may need updating, by changing youths’ language and jargon depicted in the videos
32	The 9-lesson Ho’ouna Pono curriculum has too much drug-specific content to fit within a semester-long health course
34	The Ho’ouna Pono curriculum introduces language and concepts that may be academically advanced for the 6th grade level
*38	The Ho’ouna Pono curriculum lacks a social media presence (e.g., Twitter, Instagram, Facebook)
47	Marijuana use is socially acceptable on Hawaiʻi Island, diminishing the need for Ho’ouna Pono
IV. Student Attitudes + Mindsets (Family + Community) (CFIR Outer Setting)
9	The structure of the Ho’ouna Pono curriculum can get repetitive for some students over the course of nine lessons, affecting their engagement	Use school community councils to elevate student voice
18	Some families live a “counter-culture” lifestyle, and might not want their children exposed to drug prevention
30	The relatability of the Ho’ouna Pono curriculum is limited primarily to 6th and 7th graders. Older youth may find it to be “silly” or too “immature”
V. Policy (CFIR Outer Setting)
4	The HIDOE OCISS (State-Level) Health Resource Teacher has not explicitly endorsed the Ho’ouna Pono curriculum for Hawaiʻi island schools	Talk with legislators about curriculum
6	Mainstream publications (Honolulu Magazine) do not sue health indicators in establishing their annual rank ordering of public schools
11	The HIDOE emphasis on “moving” school-wide standards-based test scores detracts from health education and prevention efforts like Ho’ouna Pono
12	The Ho’ouna Pono curriculum does not specifically address Common Core standards in writing and math
14	Health education is not included in the middle school promotion policy, therefore there is no urgency to implement drug prevention curricula like Ho’ouna Pono
17	Shifting National priorities (e.g., bullying) take the focus away from substance abuse prevention programs, like Ho’ouna Pono
19	The Hawaiʻi Board of Education may see Ho’ouna Pono as redundant to other drug prevention curricula in Hawaiʻi Island schools, like DARE
24	Qualified adult instructors outside of the HIDOE are not available to teach Ho’ouna Pono lessons to youth
26	There is a lack of HIDOE funding to support prevention curricula like Ho’ouna Pono
35	Restrictions on federal funding (e.g., Title 1) make it difficult to support curricula like Ho’ouna Pono
36	The HIDOE emphasis on college and career readiness diverts school resources away from drug prevention programs, like Ho’ouna Pono
37	The HIDOE does not specify the structure or content of substance use prevention in schools
43	The HIDOE emphasis on vocational training diverts school resources away from drug prevention programs, like Ho’ouna Pono
44	There may be a lack of enthusiasm for the Ho’ouna Pono curriculum because health education is not a standardized or measured content area
46	The turnover of state-level administrators will disrupt the ongoing implementation of Ho’ouna Pono

Note. Numbers correspond to numbering in the original Ho’ouna Pono Implementation, Adoption, and Sustainability survey (Okamoto et al., 2020); CFIR = Consolidated Framework for Implementation Research (Damschroder et al., 2009) mapped domains next to concepts in parentheses; “School Level Buy-In” was originally named “School level” shown in parentheses;

^*indicates move from “Student Attitudes + Mindsets (Family + Community)” to “Curriculum.”

Figure 1.

Finalized Ho’ouna Pono Implementation Barrier Concepts

Note. Participants (N = 7) represented Hawaiʻi State Department of Education state, complex, and district leadership; “kumu” means teacher in ʻōlelo Hawaiʻi (Hawaiian language); numbered items represent distinct barriers.

The Go Zone map (see Figure 2) integrated difficulty and impact ratings such that barriers identified fell into one of four quadrants: low difficulty and high impact (green); high difficulty and high impact (yellow), high difficulty and low impact (red), and low difficulty and low impact (blue). For example, item 32, “The 9-lesson Ho’ouna Pono curriculum has too much drug-specific content to fit within a semester-long health course,” had an average difficulty rating of approximately two and impact rating of approximately three (lower difficulty and higher impact) and is therefore in the green quadrant. Table 2 highlights eight items in the Go Zone map (see green quadrant in Figure 1) with high impact and low difficulty scores. These items potentially represent ideal barriers to prioritize in the implementation plan.

Figure 2.

Ho’ouna Pono Barrier Go Zone

Note. Participants (N = 6) represented Hawaiʻi State Department of Education state, complex, and district leadership; items represented in green indicate lower difficulty and higher impact whereas items represented in blue indicate lower difficulty and impact.

The phase one discussion reflecting on Ho’ouna Pono barriers and facilitators elucidated themes related to shared ownership and responsibility over the intervention. For example, one participant noted that characteristics of a successful implementation effort included ownership and normalization of the intervention, cutting across the “Kumu (Hawaiian word for teacher) Controlled” and “School Level Buy-In” domains. Other important “Kumu Controlled” strategies were training and creating teams (e.g., “make sure the teachers have a team”) to learn with a cohort of teachers and other staff. For example, one participant noted:

[Teachers have] taken ownership over it and created this, you know, kind of relationship. So, it prioritizes something that used to not be a normal part of – but now it’s more like normalized.

Also related to the “Kumu Controlled” domain, participants brought up another example of a successful implementation strategy (peer-to-peer learning) when teachers were required to pivot to online teaching during the COVID-19 pandemic.

They [teachers] convened these peer-to-peer sessions, and it was like come after school, drop in for an hour, learn from a friend and so, teachers started to just share [their] practice-and it became this little community, and all said and done, it attracted private school, public, charter, and home school, [and] even students joined.

Phase Two: Refining of the Concept Map and Generation of Strategies

In the first part of the phase two discussion, participants were asked to review the barriers within each cluster to look for similarities and any outliers that needed to be moved. Then, cluster names were reviewed and edited within the group. Cluster names included “Kumu Controlled,” “School Level Buy-in,” “Curriculum,” “Student Attitudes + Mindsets (Family + Community)” and “Policy.” Participant discussion changed one cluster name from “School Level” to “School Level Buy-In.” Additionally, three barriers (“The Ho’ouna Pono curriculum does not extensively cover current or recent forms of substance use, such as vaping,” “The Ho’ouna Pono curriculum lacks a social media presence (e.g., Twitter, Instagram, Facebook),” and “After 10 years, the Ho’ouna Pono curriculum may need updating, by changing youths’ language and jargon depicted in the videos”) were moved from the “Student Attitudes + Mindsets (Family + Community)” to “Curriculum” based on fit and proximity on the map.

In the second part of the phase two discussion, participants brainstormed potential implementation strategies within concepts while referencing the ERIC implementation strategies handout. For example, when presented the “Student Attitudes + Mindsets (Family + Community)” concept, one participant noted the potential effectiveness of local school-community councils to elevate student voice to promote Ho’ouna Pono awareness and implementation. Related to “Policy,” one participant suggested talking with legislators to promote visibility of the curriculum.

Are our legislators/state education leaders aware of the work you all have done? I am not really sure what it [will] take…but awareness might be the first step at that level.

Another participant affirmatively stated “I’m wondering if combining a grassroots effort with high-level decision makers might help. The BOE [Board of Education], local legislators, etc. could be involved too.” During the “Kumu Controlled” discussion, one participant noted about the concept mapping process that “You’re still allowing space for voice - all of our voices so, yeah, I think it’s just a matter of training.” Coordinated and cohort-based training, integration into existing curricula with lesson plans to address all students, and drug treatment counselor, school counselor and school-based behavioral health involvement were also identified as potential strategies to address the “Kumu Controlled” concept. For the “Curriculum” domain, one participant stated that using teachers to develop the curriculum might aid in wider applicability to all students and motivation to use the intervention. At the “School Level Buy-In”, more required health education courses in middle school, creating new incentives for continuing education for teachers (especially those who have reached their ceiling), and aligning educational policies with outcomes that address health and well-being were offered as potential strategies. A compilation of brainstormed strategies was given to participants as a starting point for future implementation discussions for tailored approaches.

Discussion

Implementing culturally grounded substance prevention programs for NHPI youth in schools is a public health priority. This study examined educational leaders’ perspectives on implementation barriers and used a community-engaged and participatory approach (concept mapping) to generate types of barriers and potential implementation strategies to address these concerns. The process highlighted how participatory methods can be used to engage and generate a robust list of potential strategies in a diverse group of HIDOE leadership. The concept mapping process also created bidirectional learning with HIDOE leadership and researchers and let HIDOE leadership learn from frontline teachers and staff about Ho’ouna Pono barriers. The process and outcome of this study provides a thoughtful pathway to Ho’ouna Pono implementation.

Concept mapping results revealed clusters that were aligned with CFIR domains as applied to schools. For example, “Kumu Controlled” barriers related to the CFIR Individual Characteristics and Implementation Process. Additionally, the “School Level Buy-in” cluster was similar to the CFIR Inner Setting and the “Curriculum” cluster was similar to the CFIR Innovation domain. There was an intersection between the “Student Attitudes + Mindset (Family + Community)” and “Policy” clusters that related to the CFIR Outer Setting domain. There is value added in having school staff members, in this case the HIDOE, name these constructs for better understanding and meaning. Indeed, emerging evidence suggests that team-based approaches to implementation science (Mathieu et al., 2019), including shared mental models (Mathieu et al., 2008), can create shared ownership and responsibility for the innovation.

Findings from this study revealed interconnected barrier concepts related to Ho’ouna Pono, especially for “School Level Buy-In,” “Curriculum,” and “Student Attitudes + Mindset (Family + Community).” Interestingly, concept map bridging values suggested that participants perceived the “Kumu Controlled” barriers as homogenous and did not tend to group these items with other barriers associated with other concepts like “Policy.” The Go Zone map revealed eight high-impact and low-difficulty barriers from the “Kumu Controlled,” “Curriculum,” and “Policy” concepts. Interestingly, the “Policy” high-impact and low-difficulty barriers related to state and complex standards, which may have been an artifact of the elevation at which participants perceived implementation in combination with their positionality. The “Kumu Controlled” and “Curriculum” high impact and low difficulty barriers related mostly to awareness around the curriculum, its contents, and the applicability over existing substance use prevention. It is interesting that HIDOE leadership did not perceive these concepts to be highly related as evidenced by the lower bridging values in the “Kumu Controlled” concept. This intersection of individual barriers and concept proximity is consistent with previous literature in school-based implementation that suggests that people vary in their understanding of their own and others’ roles (Owens et al., 2014) and may indicate a need for co-created implementation strategies that involve state-, complex-, and school-level staff.

It is important to situate these findings in the previous studies examining Ho’ouna Pono implementation using the CFIR with teachers and frontline HIDOE staff (Okamoto et al., 2019). Given the content in the concept mapping clusters (see Table 2), it appears that state and complex HIDOE leadership perceive Ho’ouna Pono barriers differently. Okamoto et al. (under review) found a four-factor structure (innovation barriers, state-level barriers, teacher-level barriers, and administrator-level barriers) using exploratory factor analysis on an identical barrier survey with 204 HIDOE teachers, administrators, and staff. These empirically derived groupings are less aligned with CFIR domains than the rationally derived concepts in the current study. Inner Setting barriers that map onto “School Level Buy-In” have impact on what teachers do; however, these barriers are largely outside of their control. Similarly, Individual Characteristics barriers that map onto the “Student Attitudes + Mindsets (Family + Community)” domain influence the extent to which teachers can deliver curricula related to substance prevention and health. While outside the scope of the current study, future studies could engage teachers and staff through concept mapping to create a shared understanding of barriers among teachers, school administrators, and HIDOE leadership.

Implications for School-Based Mental Health Promotion and Substance Use Prevention Program Implementation

The Go Zone Map identified multilevel barriers that would have a high impact on successful implementation, while being less difficult to overcome (see Table 2). HIDOE leadership could prioritize addressing these barriers, which may have broader implications for school-based prevention. Three barriers (39, 28, 7) specifically target teacher perceptions (i.e., “Kumu Controlled”), which may be addressed with strategies that employ cohort-based experiential training to build teacher efficacy in speaking with youth about drug and alcohol use, integration with current substance use curricula, and involving other school staff (e.g., counselors, school-based behavioral health specialists) to deliver the intervention. An additional three barriers (21, 22, 32) related to the “Curriculum” that are currently being addressed with updated video content to reflect modern language and current forms of drug use like vaping and could incorporate more teachers in the development of the content to bridge existing curriculum. The final two barriers (37 and 14) focus on HIDOE “Policy” and outer setting related concerns which may be addressed with legislative priorities (i.e., speaking with legislators) and internal HIDOE policies to mandate substance use-related health education.

Additional strategies brainstormed by HIDOE leadership included using school community councils to elevate student voice (for the “Student Attitudes + Mindsets (Family + Community)” concept), creating policies to require more health education courses and aligning educational outcomes that address health and well-being, and creating new incentives for continuing education (for the “School Level Buy-In” concept). These strategies align with a multilevel approach to promote Ho’ouna Pono across community-, student-, teacher-, and administrator-levels and the need for co-creation within these groups to sustain the intervention. Similar school-based prevention programs could focus on promoting teacher self-efficacy, ensuring updated intervention content, and addressing outer setting issues to support implementation (Cook et al., 2019; Kuriyan et al., 2021). Moreover, aligning implementation strategies that have been adapted for (Cook et al., 2019) and evaluated for feasibility and importance within school-based settings (Lyon et al., 2019) will be an important next step to expanding the generalizability of the current study’s findings.

Strengths and Limitations

Community-engaged and participatory approaches allow researchers to connect and partner with school staff members differently. Concept mapping is a participatory approach in which researchers and school staff members have equal input in creating shared meaning and ownership, in this case regarding implementation. The process elevates voices and redistributes power in the research process as one participant highlighted in the title of this manuscript. Too often, researchers engage with communities through advisory boards to address these concerns, but this method falls short because it creates an additional burden on school staff, and most studies do not have flexibility built in to change research designs to meet their needs. The current study’s approach also bridged school staff, teachers, and administrators with complex and state-level leadership to share and build off implementation barrier perspectives from those closest to the delivery of the intervention. Participatory methods have the potential not only to create generalizable findings that elevate voice but also to engage people from differing levels within school organizations to foster a shared understanding of an implementation problem.

The results of this study inform the co-planning and design of an implementation project to spread Ho’ouna Pono to a novel school district in Hawai’i. The present study provides direction for proactively addressing potential implementation barriers within the new school district by redistributing power to youth and families to plan implementation, while simultaneously supporting school-level engagement and “buy-in” for the implementation of the curriculum. Lessons learned from this study have the potential to inform how concept mapping can aid in co-creating implementation strategies and action plans (Powell et al., 2017).

The current study is not without limitations. It is unclear how representative concept mapping findings are of the larger HIDOE system and other districts given the small sample size. In a review of 104 concept mapping dissertations, sample size ranged in each phase – brainstorming from one to 555, sorting from five to 152, rating from zero to 152, and interpreting from one to 112 (Donnelly, 2017). The goal of the current study was to elicit sufficient feedback on barriers and strategies to develop a Ho’ouna Pono implementation plan. As such, participants were carefully selected through consultation with educational partners, community members, and school staff, and the 30 originally contacted represented State Education Specialist (n = 14), or District Education Specialists (n = 12), or Complex Area Superintendents (n = 5). The current sample is representative of the contacted participant pool in terms of their administrative and leadership HIDOE experience with the exception of the School-based Behavioral Health Specialist, who was included based on the suggestion of one Complex Area Superintendent. Nevertheless, we cannot conclude that the perceptions of study participants represent the other HIDOE leadership that did not participate in the study.

Moreover, the findings represent a snapshot in time that may not be representative of ever-changing priorities within the HIDOE and larger policy ecology, such as federal educational standards and increased mental health parity (Raghavan, 2008; Wortham et al., 2023). The implementation science field would benefit from more clearly articulating time within implementation phases and studies to elucidate the progression of implementation processes (Alley et al., 2023). Additionally, given that the CFIR was used to develop barriers, this may have limited the identified concepts. Finally, while beneficial to researchers, the ERIC implementation strategies were difficult for participants to understand. For example, one participant noted that the strategies felt too discrete and there was overlap between many of them. This is a challenge that has been identified by others, and there are ongoing efforts to further clarify the terminology used within the ERIC compilation (e.g., Yakovchenko et al., 2023). Future studies may wish to integrate tools that link strategies to barriers like the CFIR-ERIC matching tool (Waltz et al., 2019) and would ideally continue to involve youth and families in creating and testing implementation strategies in school settings. As the implementation science field continues to make strides in developing and specifying implementation strategies, a potentially vital role of researchers and implementation specialists will be to define and operationalize these strategies in local language for their given project and community. For example, the ERIC compilation has been translated into German, and Japanese and Spanish language translations are underway (Regauer et al., 2021; Van Pelt et al., 2023). This study could also have been strengthened by further contextualizing policy and outer setting recommendations with HIDOE leadership to create commitments to addressing these concepts and individual barriers to aid in future prevention implementation initiatives.

Conclusions

This study benefited from eliciting implementation barrier perspectives across different levels within the HIDOE. It also engaged both state- and complex-school staff and researchers in creating shared meaning and ownership over the implementation process. This study culminated in a set of potential implementation strategies that can be used to implement the intervention throughout schools in Hawai’i. The main barrier concepts were related to teachers, school context, the innovation, student, family, and community attitudes, and policies; and the potential strategies generated including cohort-based training, integration of the curriculum to existing lesson plans, using non-teachers to deliver the intervention, creating new requirements, outcomes, and incentives for health education, elevating student voice through school community councils, and talking with legislators. Implementation is an ongoing effort that requires years of relationship building, planning, and evaluation. Our findings suggest that, specifically for school-based implementation, determinants interact in unique ways, highlighting the need for multilevel implementation strategies. Specifically, a combination of strategies that address teachers’ relative priority (i.e., competing demands) should take in to account federal-, state-, and school-priorities for health education, teachers’ current continuing education credit level, and family and community culture related to substance use (including teachers’ personal experience and current attitudes). Future studies may examine time invested and the cost of those investments to inform future funding priorities.

Table 3.

Ho’ouna Pono High Impact, Low Difficulty Barriers

#	Item	Difficulty	Impact
Kumu Controlled
39	Lack of familiarity with Ho’ouna Pono makes me hesitant to use the curriculum	3.0	3.3
28	Some HIDOE teachers believe their current drug and alcohol content is effective, so they don’t see any added benefit to the Ho’ouna Pono curriculum	3.0	3.3
7	It is difficult for teachers to talk about drug and alcohol use in the classroom setting with their students	2.6	3.2
Curriculum
21	The Ho’ouna Pono curriculum does not extensively cover current or recent forms of substance use, such as vaping	2.6	3.3
22	After 10 years, the Ho’ouna Pono curriculum may need updating, by changing youths’ language and jargon depicted in the videos	2.6	3.3
32	The 9-lesson Ho’ouna Pono curriculum has too much drug-specific content to fit within a semester-long health course	2.4	3.2
Policy
37	The HIDOE does not specify the structure or content of substance use prevention in schools	3.0	3.5
14	Health education is not included in the middle school promotion policy, therefore there is no urgency to implement drug prevention curricula like Ho’ouna Pono	2.8	3.3

Note. Participants (N = 6) represented Hawaiʻi State Department of Education state, complex, and district leadership Impact ratings ranged from one (no impact) to five (substantial impact); Difficulty ratings ranged from one (very easy) to 5 (very difficult)