Concept Mapping as an Approach to Facilitate Participatory Intervention Building

Abstract

Background

A challenge to addressing community-defined need through community-based participatory intervention building is ensuring that all collaborators’ opinions are represented. Concept mapping integrates perspectives of individuals with differing experiences, interests, or expertise into a common visually depicted framework, and ranks composite views on importance and feasibility.

Objectives

To describe the use of concept mapping to facilitate participatory intervention building for a school-based, teacher-focused, positive youth development (PYD) promotion program for Latino, Hmong, and Somali youth. Particiants were teachers, administrators, youth, parents, youth workers, and community and university researchers on the projects’ community collaborative board. We incorporated previously collected qualitative data into the process.

Methods

In a mixed-methods process we 1) generated statements based on key informant interview and focus group data from youth workers, teachers, parents, and youth in multiple languages regarding ways teachers promote PYD for Somali, Latino and Hmong youth; 2) guided participants to individually sort statements into meaningful groupings and rate them by importance and feasibility; 3) mapped the statements based on their relation to each other using multivariate statistical analyses to identify concepts, and as a group identified labels for each concept; and 4) used labels and statement ratings to identify feasible and important concepts as priorities for intervention development.

Results

We identified 12 concepts related to PYD promotion in schools and prioritized 8 for intervention development.

Conclusions

Concept mapping facilitated participatory intervention building by formally representing all participants’ opinions, generating visual representation of group thinking, and supporting priority setting. Use of prior qualitative work increased the diversity of viewpoints represented.

Community-based participatory research (CBPR) intervention building relies on the input of collaborators with varied experiences and perspectives to develop interventions that appropriately address pressing community-identified needs.¹ One challenge to this process is ensuring that all perspectives are presented and given appropriate weight in identifying topics and prioritizing how they will be addressed. Although a consensus process is often used for such purposes in CBPR, limitations are that consensus idea generation and priority setting may be slow, particularly within new partnerships, and that even when implemented properly may not attend to all perspectives within a group.^2,3 Concept mapping, which is a structured method for organizing the ideas of multiple, diverse stakeholders into a common framework, offers a complementary approach.⁴

Concept mapping is a mixed methods approach that integrates qualitative perspectives of individuals with multivariate statistical methods to visually depict the composite thinking of the group.⁴ The approach also generates data that rank composite views on a given topic such as importance and feasibility to facilitate priority setting. Benefits of concept mapping include its visual end-products that can facilitate decision making,⁵ its use of stakeholder language,⁶ and the flexibility in how it is conducted.⁷ Concept mapping has been used in program planning and evaluation focused on health improvement initiatives,^8,9 community building,¹⁰ development of program theory,¹¹ and, rarely, participatory intervention development.^12,13

Concept mapping is particularly appropriate for CBPR intervention building because it is designed to integrate input from individuals with differing content expertise, interests, and experience, as would be true in many CBPR partnerships, and generates repeated opportunities for input in the processes of sharing opinions, interpreting results, and prioritizing next steps. Additionally, the visual representations of composite group thinking are conducive to the CBPR consensus making process, particularly for groups with differing levels of experience interpreting numerical data. A challenge for concept mapping within a CBPR partnership is balancing the need for multiple perspectives with the ideal group size for decision making within the intervention building process. One solution is the integration of data gathered from outside stakeholders into the process.

Our objective is to describe the utilization of concept mapping in a CBPR process to develop priorities and approaches for a school-based, teacher-focused intervention intended to promote PYD and increase school and educator connectedness for Somali, Latino, and Hmong (SLH) high school students. We describe our approach to concept mapping, the implementation of the process, and our lessons learned.

METHODS

Project Training for Resiliency in Urban Students and Teachers

Project Training for Resiliency in Urban Students and Teachers (TRUST) is a National Institutes of Health-funded CBPR study that aims to develop and pilot an intervention to support educators in engaging with SLH high school students in ways that build the PYD asset of student connectedness to their teachers and school, and increase classroom engagement.¹⁴ The study intends to address academic outcomes as key social determinants of health, and prevent health risk behaviors.

The Project TRUST CBPR core research team is composed of teachers, academic researchers, and community researchers from the organization Somali, Latino and Hmong Partnership for Health and Wellness (SoLaHmo). SoLaHmo is a multiethnic, community-driven program whose mission is to build on the unique cultural strengths of SLH communities to promote health and wellness through research, education, and policy. SoLaHmo’s vision is dedicated to the reality that SLH communities have the knowledge, skills, and power to participate as equal partners with researchers and health care professionals to maximize community health and wellness. All SoLaHmo members are trained in qualitative research processes using a training curriculum.¹⁵

Our project is further guided by a community collaborative board composed of the core research team, additional teachers, school administrators, youth, parents, and representatives of youth-serving community organizations. It is this larger group that engaged in the concept mapping process.

The study is approved by the University of Minnesota Institutional Review Board.

Concept Mapping overview

The concept mapping process consists of four general steps: 1) collecting data in response to questions or prompts that form statements; 2) sorting statements into meaningful associations and rating them according to a study-relevant scale; 3) mapping the statements on a 2-dimensional plane based on their relation to each other to develop the concept map and summarizing ratings; and 4) labeling concepts and comparing average ratings to guide planning or evaluation. These steps can be completed in different ways. For example, both open-ended survey questions¹⁶ and “brainstorming focus prompt” phrases⁶ have been used to generate statements. The latter has been used as an alternative to focus group interviews, because the short responses then serve as statements for sorting with little or no editing.^5,8,17 For steps 2 and 3, prior studies have varied in the timeframe in completion of steps, number of participants, and the method of sorting and analysis. For example, participants may sort and rate statements printed on paper individually during an in-person meeting, or online at their convenience. The advantage of live meetings is that they provide data immediately; however, obtaining digital data online can circumvent the data entry step.⁴ We used the Concept Systems Global (CSG) project software to facilitate the process.¹⁸

Collection of Qualitative Data Used to Generate statements

We chose to use previously collected qualitative data to develop statements for sorting to diversify the perspectives and opinions considered for inclusion in the intervention (Figure 1). This was important because our project includes communities with recent immigrants who speak multiple languages. We conducted key informant interviews with youth worker experts and focus groups with teachers, SLH youth, and parents of SLH youth from September 2010 to June 2013. We purposefully recruited 18 key informants identified by SoLaHmo researchers as experts working with SLH youth, from schools, afterschool programs, and social service agencies. Twenty-five teachers from middle and high schools with large SLH student populations were recruited to participate in four focus groups. We recruited 71 self-identified SLH youth through schools and after school programs who met inclusion criteria: attended public middle or high schools (grades 8–12), or graduated from, or dropped out of school within the most recent academic year to participate in nine focus groups (three from each ethnic community). Finally, six focus groups (two per ethnic community) were conducted with SLH parents of middle or senior high school students or recent graduates. A total of 35 parents participated.

Figure 1.

Trained bilingual SoLaHmo researchers conducted focus groups and interviews using a common semistructured guide. Guides addressed three domains: definition of resiliency for SLH youth, role of teachers in promoting resiliency for SLH youth, and areas for programming to help teachers promote resiliency for these youth. Language and framing was adapted for each group’s educational level and area of knowledge. All interviews were audio-recorded. Those in English were transcribed verbatim; those in Somali, Spanish, or Hmong were transcribed verbatim/translated by bilingual SoLaHmo researchers and reviewed by a second team member.

Preliminary analysis occurred by a version of the immersion-crystallization method¹⁹ adapted for participatory analysis.^20,21 Team members read transcripts, discussed emerging themes and categories, reached agreement on ideas contained in the key words and phrases, and formed initial codes; full team reached agreement on codes and a common coding structure was organized and implemented. Nvivo 10²² was used to capture codes and organize the data. To generate statements for the concept mapping process, each category under a theme was transformed to a statement with two associated exemplary quotes from focus group participants.

Sorting and Rating of Statements

Sorting and rating of statements occurred in person except for three participants unable to make the meetings who sorted online. For those who completed in person, the statements and quotes were printed onto index cards. Participants sorted the cards into piles with common themes in response to the focus prompt, “Think about these statements as they relate to a school-based program that promotes resiliency among Somali, Latino and Hmong youth.” Online participants sorted the statements into “piles” via drag-and-drop interface. All participants generated a label describing each pile. Participants also rated each statement on scales of 1 to 5 based on both how important (1 [not important] to 5 [very important]) and how feasible (1 [not feasible] to 5 [very feasible]) they thought it was to address the concept through a school-based intervention to promote resilience in SLH youth. In person piles, labels, and ratings were entered into the CSG software.

Generation of Concept Maps

CSG software uses multidimensional scaling (MDS) and hierarchical cluster analysis of the sorting data to generate point and cluster concept maps. MDS maps each statement as a point, and points that are closer to each other were generally sorted together more frequently by the community collaborative board. Hierarchical cluster analysis then draws boundaries around groups of points to delineate “clusters” of frequently associated statements/ideas. The CSG software identifies the top five labels for each cluster based on the five smallest Euclidean distances between the cluster and sort-pile centroids; with centroids computed as the average x and the average y value of the MDS coordinates for each point in the cluster and in each sort pile.¹⁶

Analysis and Interpretation of Ratings Data

Statement ratings for both feasibility and importance were first averaged across individual ratings then averaged across statements in each cluster. The average cluster feasibility and importance ratings were used to generate a relative pattern match graph, which displays the importance and feasibility rankings for each cluster. Ratings analysis was also used to generate a scatter plot of average importance and feasibility ratings for all statements and for statements within each cluster; quadrants of the scatter plots identify statements with both higher than average feasibility and importance ratings (Go-zone), lower than average feasibility and importance ratings (No-zone), or a combination of above and below average ratings (middle zones).

RESULTS

The community collaborative board that participated in the concept mapping exercise (n = 28) was made up largely of adults with a small cohort of youth (Table 1). The racial/ethnic distribution of participants was broad. Most participants (70%) were born in the United States.

Table 1.

Characteristics of Community Collaborative Board Participants in Concept Mapping Exercise (N = 28)

Characteristics	n	%
Role
Teacher/Administrator	7	23
Parent	3	10
Youth	3	10
Community Representative	2	7
University Researcher	8	27
SoLaHmo Researcher	5	17
Ethnicity
African	4	13
African and Caucasian	1	3
African American and Caucasian	1	3
African American and Hispanic/Latino	1	3
Asian American/Pacific Islander	6	20
Asian American and Caucasian	1	3
Caucasian	10	33
Hispanic/Latino	4	13
Years in the United States
<10	1	3
>10 but Not Born in the United States	6	20
Lifetime	21	70
Age (y)
Mean (SD)	38.2 (13.5)

The group gathered for four 2-hour meetings over 2 months. During the first meeting, we introduced the project and discussed the overall concept mapping process. During the second meeting, each participant reviewed 82 statements generated from the formative data, ranked each statement on importance and feasibility, sorted them into piles, and labeled each pile. The sorting process took from 1 to 1.5 hours to complete. At the third meeting, we returned results from the MDS and hierarchical cluster analysis to the group and discussed the visual output. We first discussed the cluster map (Figure 2) and reached agreement on the similarity of, but distinction between, clusters positioned close together on the map. We also used a formal consensus making process³ to choose a label for each cluster. We either chose one of the top five participant-generated labels or identified a new label for each cluster. Table 2 provides cluster/concept labels and exemplar statements from each cluster.

Figure 2.

Project TRUST Cluster Map With Labels

Table 2.

Labels for 12 Clusters and Exemplary Statements from Each Cluster from Project TRUST

Cluster, Total Number of Statements, and Exemplary Statements

Optimum teaching strategies to engage students (13 statements) Teachers should actively engage students so that students will feel more involved in the classroom Teachers should provide structure and consistency in their classrooms Teacher skill development (5 statements) Teachers should acknowledge that not all students are at the same educational level and should help each individual student to advance at their own pace Teachers should ask for student’s perspectives on lesson plans so that they can better engage students Teacher attributes that convey respect (6 statements) Teachers should be positive role models so that students will have someone to learn from on how to respect their teachers and peers Teachers should be genuine and sincere with their students Teachers show a genuine commitment to students (11 statements) Teachers should maintain high expectations and challenge students Teachers should engage students Build trust and create classroom community (9 statements) Teachers should establish a classroom environment where students feel safe asking questions and contributing to discussions Teachers should not act in a way that shows favoritism for one group of students or discriminates against a group Teacher–student relationship (9 statements) Teachers should actively listen to students and be interested in what they are saying Teachers should find common ground with students by sharing personal stories and showing vulnerability Parent–teacher involvement (5 statements) Language differences keep teachers and Somali/Latino/Hmong parents from connecting with each other Parents have a role in building a stronger relationship between teacher and student Bridging parent–teacher communication gap (5 statements) There are a number of ways that parents and teachers can communicate with each other, rather than relying only on parent–teacher conferences When teachers connect with parents, it shows how much they care for the student How to best support the entire student (4 statements) Somali, Latino, and Hmong teachers should act as role models for Somali, Latino and Hmong students Teachers should connect with each other to learn more about their students Teachers understanding students’ backgrounds (8 statements) Teachers should have cultural knowledge about students they work with so they can understand where the students are coming from Teachers should convey appreciation for students’ cultural and religious backgrounds Systemic issues in schools Large class sizes make it challenging for teachers to connect with every student There is not enough time to build quality relationships between teachers and students because the whole class period is devoted to curriculum School environment (6 statements) School programs should incorporate an awareness of cultural identity so that teachers can create learning environments that recognize the cultural contributions of students The school should create more opportunities for teachers and students to connect with each other

The fourth meeting focused on priority setting and so turned to the rankings of feasibility and importance for each statement, and across the clusters. We used the importance and feasibility scatter plots (Figure 3) to reflect on individual statements’ ranking on importance and feasibility. Statements with both high feasibility and high importance were identified as being in the ‘Go-zone’ for inclusion in the intervention. Statements not in the ‘Go-zone’ were brought to the group for discussion and group consensus regarding whether or not they would be included. For instance, statement 9, “Teachers should be fun and enthusiastic about the way they teach,” was not originally in the ‘Go-zone.’ After discussion that this statement was prioritized by youth in the qualitative data, it was moved to “Optimum teaching strategies to engage students” concept for inclusion.

Figure 3. Project TRUST Statement Importance by Feasibility Plot.

Statements in the right upper box have above-average ratings on importance and feasibility and so are in the “Go-zone.”

We then turned to rankings for clusters. Cluster average ratings ranged from 3.73 to 4.53 for importance and from 3.25 to 4.19 for feasibility. As presented visually in the importance and feasibility match plot (Figure 4), two clusters, systemic issues in schools and parent–teacher involvement, ranked lowest on both importance and feasibility. This rating information contributed to the group decision to address these two concept areas in future, broader programming. Given their similarly high importance and feasibility ratings, all other 10 concepts were included for intervention building.

Figure 4.

Project TRUST Importance and Feasibility Match Plot for 12 Concepts

After this process, the core research team integrated the identified priorities and program approaches with a theoretically defined intervention approach using PYD theory^23,24 and social cognitive theory²⁵ to build a conceptual model for the project. We used evidence-based practices that may be applicable to the priorities and conceptual model as we were able, adapted them as necessary, and built new components where needed. All teacher-focused components followed the evidence-based principles of professional development for educators.^26,27 Drafts of the curriculum were returned to the community collaborative board for input as they were developed. The resulting intervention, in the process of being pilot tested, is a nine-session professional development training focusing on teacher skill building regarding facilitating a sense of belonging in the classroom; connecting with students, families and communities; conveying respect for students as individuals; developing growth mindsets; and strengthening high expectations for learning.

DISCUSSION AND LESSONS LEARNED

We have identified concept mapping as a feasible, effective tool for soliciting broad input and identifying priorities as first steps in participatory intervention building. The exercise quickly engaged our newly formed community collaborative board in substantive decision making and ensured that all voices around the table were represented in the process.

Ease of Implementation

With brief instructions, all participants, including youth and participants for whom English was a second language, were able to complete the sorting exercises with minimal support. The visual presentation of the cluster maps and importance/feasibility graphs made the data easy to understand for participants and prompted valuable discussions regarding priorities. Because computer literacy was variable in our group, the in-person meetings worked well for the majority of participants. Working on tasks in the same room also built a sense of group identity for the project.

Intersection with CBPR Processes

Within a CBPR partnership, the approach to decision making may need to vary by the type of decision being made² and by the composition of the group. In our prior experience, decision making by consensus within a group marked by significant diversity in ages and language preferences can be challenging owing to the difficulty in eliciting opinions from all participants. Although there was no perceptible tension between community collaborative board members, we were aware that parents and youth were not as outspoken as other participants. We found concept mapping to be one tool to at least partially neutralize power differentials within the group owing to the multiple methods for soliciting input and reviewing group thinking. In our case, concept mapping included consensus building to come to decisions on labels for each concept, and priorities for the program being developed. This group decision making approach was augmented with the documentation of individual opinions through statement sorting, initial labeling, and ranking of importance and feasibility.

Formative Data Expanded the Perspectives Included in the Concept Mapping Process

We chose to use statements generated from outside focus groups and interviews rather than have the core collaborative team generate their own statements, as would typically happen with a concept mapping process. Because our project focused on three ethnic communities, composed largely of first- and second-generation immigrants to our region, using focus groups allowed us to gather data from non-English speakers and integrate their experiences into the planning process. Focus groups also brought in perspectives from individuals who were not willing or able to commit time to the community collaborative board. The downside is that gathering and analyzing these data took a significant amount of time and effort.

Overall, we found the qualitative analysis and concept mapping approaches to be complementary and worthwhile despite the added work. The university and SoLaHmo community researchers participated in both processes. Given our familiarity with the data, we noted that many of the themes generated within the qualitative analysis were similar to the labeled concepts, but that some of the statements were sorted differently. In the end, the group chose to move some of the statements to different clusters as part of the importance and feasibility consensus process. Our experience with the focus groups also reinforced the importance within the concept mapping process of having equal representation of groups whose opinions were foundational to the intervention being developed. For example, although we felt that the youth voice was imperative for framing student opinions and priorities, given that we were developing a teacher-focused PYD program, the community collaborative board was weighted toward teachers to be attentive to their unique time constraints, stresses, and needs. In the processes of reviewing the prioritization of statements, we recognized that concepts deemed important by youth in the focus groups were rated as less important by our majority adult community collaborative board. In the end, inclusion of the qualitative data allowed us to prioritize and include youth statements to amplify the voice of youth in the final outcome.

CONCLUSION

Concept mapping facilitates participatory intervention development by formally representing opinions of participants, generating visual representation of group thinking, and supporting priority setting. Our inclusion of key informant and focus group data as statements for the decision making group successfully added to the variety of opinions represented in the process including those of youth and non–English-speaking community members to be included as priorities were set. In the end the process led to the relatively rapid participatory identification and development of key components of an intervention. Given that concept mapping is helpful in addressing the common challenge within participatory intervention building of eliciting honest opinions across participants with varying perspectives, opinions, experiences, language abilities, and comfort speaking in a group setting we feel that it has broad applicability for participatory intervention building efforts.