Novel (Multi-Level) Focus Group Training for a Transdisciplinary Research Consortium

INTRODUCTION

Training of qualitative researchers is essential for building capacity and cohesion, and to address differences stemming from diverse academic disciplines. While many call for advanced skills training of research teams (Brehm, Rourke, & Cassell, 1999; Eisenhart, M., Jurow, 2011; Mazmanian, Coe, Evans, Longo, & Wright, 2014) and investment in qualitative research education, teams may often rush or overlook processes of coordinated training. High quality training is especially important when building research teams across geographically distinct areas of the country, with diverse disciplines, and varied perspectives and experience in qualitative research methods. There are few guidelines on how best to do this.

The Prevention of Lower Urinary Tract Symptoms (PLUS) Research Consortium is a seven-research center National Institutes of Health, National Institute of Diabetes and Digestive and Kidney Diseases effort, with one Scientific and Data Coordinating Center (SDCC), to expand research to develop the evidence that can inform practice, and policies focused on the prevention of lower urinary tract symptoms (LUTS) and the promotion of bladder health in women and adolescent females (Harlow et al., 2018). Foundational to PLUS research efforts is the Study of Habits, Attitudes, Realities and Experiences (SHARE), a qualitative consortium-wide study, which explores women and adolescent females’ experiences, perceptions, beliefs, knowledge, and behaviors related to bladder function and health across the life course (Low, et al, 2019).

We describe our approach to building an adaptive, sustainable, and flexible training model for transdisciplinary focus group moderators. As team scientists turn to qualitative research insights to provide expanded foundational knowledge, guidance for training options is essential, particularly those in multi-institution research consortia. Our hope is that our training approach and lessons learned can be a model to other large or small multi-center research efforts and help ensure that focus group data collection is systematic and consistent, while remaining flexible to each center’s unique context and population interests.

BACKGROUND

The organization of the SHARE study included five cores, including an Administrative Core, Training and Data Collection Core, Recruitment Core, Analysis Core, and Interpretation and Dissemination Core. The Training Core was responsible for focus group moderator training. Qualitative research does not represent a monolithic agreed-upon research paradigm, but is dynamic with many contradictions and diverse disciplinary and context-based perspectives (Brinkmann, Jacobsen, & Kristiansen, 2014; Denzin & Lincoln, 2011). SHARE investigators needed to identify their own unique transdisciplinary approach to qualitative research. Through individual and group reflection we reconciled natural epistemological and methodological differences among our researchers and across our consortium centers.

The Training Core had two primary objectives. First, we wanted to orient our focus group moderators to the PLUS consortium and to lower urinary tract health. Here, our goal was to equip moderators to facilitate a focused group conversation without making them experts on LUTS or bladder health, which could influence data collection. Second, because we designed the SHARE study to produce seminal bladder health insights, we expected to identify new bladder health constructs and needed a flexible training approach that would not obscure or deter the identification of these emerging constructs.

METHODS

Building the Training Protocol

It was important for the SHARE Moderator training protocol to reflect the PLUS transdisciplinary research approach. The PLUS research consortium operates by principles of inclusion, community engagement, attention to social ecology and life course, and an emphasis on health rather than disease (Harlow et al., 2018). Thus, our training model needed to be grounded, inclusive, and adaptive to best support the moderators and produce the highest quality data to answer our exploratory research questions.

Pedagogically, the training curriculum drew from best practices in adult education (Herman, & Mandell, 2005; Wilson & Hayes, (2000), action learning (Brockbank & McGill, 2003), and community engagement (Israel, Schulz, Parker & Becker 1998). We avoided exclusive didactic instruction in favor of collaborative learning, which emphasized engagement of all learners in the training process, including the developers of the training (SHARE investigators) as both experts in qualitative research and co-learners. This allowed us to employ best practices in focus group training (Krueger & Casey, 2015), moving from providing instruction to producing learning, emphasizing engagement, building on the existing expertise and knowledge of the trainees, and prioritizing experiential learning.

We developed a three-phase training model to leverage the experience and knowledge of the moderators and the transdisciplinary expertise of PLUS consortium members. The first training phase involved online training via three learning modules using Moodle, an interactive web-based educational interface. We used brief recorded instructional videos to introduce PLUS, the concept of bladder health, and qualitative research. Moderators completed the online training prior to the in-person training. We chose an online learning infrastructure to establish a flexible learning environment that could accommodate self-paced navigation of material and interaction among moderators and investigators (Green & Huntington, 2017). Specifically, the online training welcomed and oriented moderators to PLUS, our research goals, and the conceptual framework that guides all PLUS research, and with which the SHARE focus group guide is aligned (Brady et al., 2018). It also supported bringing all moderators to a comparable baseline knowledge/level about qualitative methodology and the SHARE protocol while allowing for efficient use of time at the in-person training.

The second training phase involved a two-day, in-person training comprised of four didactic and six experiential, action-learning components. This included a series of mock focus groups to build moderator familiarity with our qualitative research approach, the content area, and build skill in leading group discussions on bladder function and related experiences. The first mock focus groups were comprised of other members of the SHARE research team, referred to as internal mock focus groups. The second mock focus groups included community members and are referred to as external mock focus groups. Guidelines for best practices in probing and serving as a gatekeeper to the discussion were emphasized (Archer, 2007; Buetow, 2013). Particular consideration was given to the positionality of the research moderator role with respect to participants and expertise (Bourke, 2014; Raheim et al., 2016; Liamputtong, 2011) (i.e., reflection on how the researchers social position and perceived position of power may impact data collection) and how to best anticipate and troubleshoot specific focus group scenarios (Ulin, Robinson, & Tolley, 2005; Wuenschell, Dalrymple, & Shuler, 2007). We emphasized interaction during the focus groups as a mechanism to build the group narrative (Krueger & Casey, 2000; Morgan, 1998; Liamputtong, 2011). We also leveraged the diverse experiences of our focus group moderators (Dreachslin, 1998) particularly self-efficacy and reflection abilities (Hesse-Biber & Leavy, 2011; National Institutes of Health, 2005).

The third training phase included three supplemental trainings during the data collection phase. The first supplemental training, done via Moodle, used audio data produced during the in-person mock focus groups to refresh knowledge and skills. The second supplemental training was created in response to emergent needs captured from our ongoing SHARE evaluations (see Evaluation Approach) and involved three topics: “Refocusing on the SHARE Research Aims,” “Reviewing and Brushing up on Moderator Tips,” and “Introduction to the Distress Protocol” (i.e., guidelines for dealing with the unlikely event a participant experienced emotional distress during the focus group). The third supplemental training involved two moderator debrief calls and the development of Frequently Asked Questions (FAQ) to provide support for emergent needs. Table 1 summarizes the components of the training protocol. The timeline for all trainings and data collection is presented in Figure 1.

Table 1:

Training Protocol

Phase 1: Online Training (completed before In-Person training)
Mode	Topics
Videos, materials, discussion boards	Module 1: Welcome to PLUS
	Module 2: What is LUTS/Bladder Function/Normal Voiding and Healthy Bladder Concept (Webinar)
	Module 3: Qualitative Research and Focus Group (FG) Methodology
Phase 2: In-Person Training
Mode	Topics
Four in-person didactic and six in-person action learning sessions	Why Healthy Bladder (Didactic)
	Qualitative and FG Methodology (Didactic)
	SHARE Protocol (Didactic)
	Moderator Breakout 1: Unique Challenges and Strategies (Action Learning)
	Moderator Breakout 2: Protocol and Probes (Didactic)
	Internal Mock FG with SHARE team members (Action Learning)
	Debrief (Action Learning)
	External Mock FG with volunteers from the community(Action Learning)
	Debrief (Action Learning)
	Qualitative Evaluation (Action Learning)
Phase 3: Supplemental I - During data collection
Mode	Topics
Listen to mock focus group audio; engage in discussion board dialogue	LUTS & Care-Seeking – Diving Deeper
	Healthy Bladder – Diving Deeper
	Spanish language FG Probes
	Icebreakers
	Great Probes
	Knowledge Acquisition – Diving Deeper
	Terminology – Diving Deeper
Supplemental II - During data collection
Brief investigator videos, PI vision statements, recorded training	Module 1: Refocusing on the SHARE Research Goals - In Our Own Words
	Module 2: Reviewing and Brushing up on Moderator Techniques
	Module 3: Introducing the Use of the Distress Protocol

Figure 1.

Overview of Training, Data Collection, and Evaluation

Evaluation Approach

To evaluate moderator skill and efficacy, we developed a comprehensive multi-level mixed-method evaluation strategy that began during the first training activities and continued through the end of focus group data collection. This included six quantitative (QUANT) and/or qualitative (QUAL) assessments (See Table 2). The initial evaluative assessments included Moodle website hits, discussion board posts and module quizzes. Two formal evaluation surveys were conducted assessing perceived skills and knowledge acquisition and satisfaction: One at the mid-point of data collection and the second after the conclusion of data collection. Standardized field notes were completed by an assistant moderator or trained investigator after each focus group and included notes on moderator performance. The field notes were transcribed, coded and analyzed in Dedoose (“Dedoose,” n.d.) to assure attendance to issues of fidelity throughout the study.

TABLE 2:

Summary of Mixed Method Evaluative Components, Data Sources, and Rating of Perceived Usefulness

Data Source	Type	Mode	Who	Nature of Insight	Frequency	Rating
Moderator Evaluation Surveys	QUANT	Online	MOD	Moderator Experience, Satisfaction and Perceived Preparedness	After initial training and data collection	Δ Δ Δ Δ Δ
Training web-based Interface	QUANT	Counts	MOD	Training characteristics - utilization statistics (number of website hits and time spent)	Ongoing	Δ Δ
Discussion Board Posts	QUAL	Textual posts	MOD	Anticipated challenges, shared strategies, lessons learned	Ongoing	Δ Δ Δ
In-person Debrief/Performance Checklists	QUAL/QUANT	field notes; moderator performance checklist; observation	MOD IT SDCC	What worked? What didn’t? IT evaluation of MOD performance at In-person	One time	Δ Δ
FG Field Note Coding	QUAL	structured	IT MOD	MOD skill (e.g., keeps conversation on track, redirects tangents, manage transitions).	Every FG	Δ Δ Δ Δ Δ
Investigator Feedback	QUAL/QUANT	Online form; Coded transcripts, debriefs	RC IT Coding Team	After processing data evidence in field notes and transcripts of training issues.	Every FG	Δ Δ Δ Δ

FG=Focus Group

QUANT= Quantitative

QUAL = Qualitative

RC=Research Coordinator

MOD= Focus Group Moderators

IT= Investigator Team (Investigators + RCs)

SDCC= Scientific Data Coordinating Center

Rating (Δ=not useful; Δ Δ Δ Δ Δ=very useful)

Our trainings included multi-level, real-time, feedback mechanisms that identified instructional gaps and needs for supplemental training. Given our interest in developing a training infrastructure that could be sustained and replicated, we considered the value of all evaluation components and rated each component based on perceived usefulness. This was done by adapting a rating approach by assigning individual components to each investigator and having them weigh the value of the insight to our training goals on a five-point scale (Wolfenden et al., 2016). Each investigator then presented their rating to the research team, which determined a final score.

Table 2 depicts each evaluation component by source, research type, mode, the intended audience (who), the nature of insight, frequency of the assessment, and the rating of the usefulness of each component by the investigator team. The Moderator Evaluation Surveys and Focus Group Field Note Coding, in which field notes were coded by a subsection of our analysis codebook to evaluate moderator needs and assess data quality, proved to be most useful in the identification of Moderator needs to advance our research goals.

Characteristics of the SHARE Moderators

Fifteen moderators with qualitative research experience were identified across the seven PLUS Consortium centers (Table 3). The cohort was racially and ethnically diverse. Most had a masters (47%) or doctoral degree (33%). Most were also employed in a research related position. Moderators came from a variety of disciplinary backgrounds with content expertise including maternal health, adolescent and child health, community and environmental health, substance abuse and chronic disease. While all had qualitative research experience, some had very little formal qualitative research training. Four had limited experience with focus groups and four had no experience in research on women’s health issues. Most (11) said that they were “slightly” or “not” knowledgeable about lower urinary tract issues and bladder health. None of the 15 moderators had experience collecting qualitative data across all PLUS age groups (11–65+) years.

Table 3.

Moderator Characteristics (n=15)

Prompt	Answer	Count (%)
Racial/ethnic identity	White or Caucasian	8 (53)
	Hispanic or Latina	3 (19)
	Black or African-American	1 (7)
	Asian	1 (7)
	American Indian or Alaska Native	1 (7)
	Middle Eastern or North African	1 (7)
Age	26–44 years	7 (47)
	45–64 years	7 (47)
	65+ years	1 (6)
Highest level of education	Doctoral	5 (33)
	Masters	7 (47)
	Bachelors	3 (20)
Number of months of qualitative methods training	1–5 months	2 (13)
	6–11 months	1 (7)
	12–24 months	2 (13)
	2 or more years	10 (67)
Qualitative methods training	Yes, through training	7 (35)
	Yes, through educational classes	12 (60)
	None	1 (5)
Bladder Health Knowledge	Extremely or very knowledgeable	0 (0)
	Moderately knowledgeable	4 (27)
	Slightly knowledgeable	6 (40)
	Not knowledgeable at all	5 (33)

RESULTS: EVIDENCE OF SUCCESSES AND LESSONS LEARNED OF THE TRAINING

Satisfaction with the Training Curriculum

The training summoned the disciplinary expertise of the moderators to help guide our shared learning. For example, moderators led our community-engaged, external mock focus groups with volunteer women and adolescent females across the life course and provided guidance on fine tuning the focus group guide. Moderators were overwhelmingly satisfied with the use of mock focus group sessions with PLUS investigators at the in-person training. Ninety-one percent (91%) agreed that these sessions helped them use the focus group guide to process and facilitate discussion. The majority of moderators (82%) reported the mock focus group with community members was helpful (Moderator evaluation survey; See Table 4).

TABLE 4.

Moderator Evaluation Survey

After In-Person and Phase 1 Focus Group Data Collection	Count (%)
Satisfied with online training (pre in-person)	13/14 (93)
Online (pre in-person) prepared me for in-person training	11/11 (100)
In-person training purpose and objectives were clear	11/11 (100)
Internal (with IT) mock focus groups were helpful to my training	10/11 (91)
External mock focus groups (with community members) was helpful to my training	9/11 (82)
Breakout discussions were helpful to my training	11/11 (100)
Agree that all participants were given the opportunity to be involved in training	11/11 (100)
Satisfied with in-person training	11/11 (100)
Extremely confident to achieve PLUS goals	10/12 (83)
Knowledgeable about how to successfully execute a focus group	12/12 (100)
Reflections on First Focus Group
Satisfied with my first focus group	11/11 (100)
First focus group went well (recruitment, consenting, discussion)	12/12 (100)
First focus group produced rich data	12/12 (100)
Extremely confident on executing subsequent focus groups	11/12 (92)
Final Survey
Satisfied with post in-person online training	9/12 (75)
Extremely well informed in explaining PLUS	7/13 (54)
Extremely well informed on qualitative research	11/13 (85)
Extremely confident on focus group research	11/13 (85)
Extremely familiarity with SHARE protocol	8/13 (62)
Supplemental training met needs very or extremely well	12/13 (92)
Very satisfied with Distress Protocol training	6/13 (46)
Extremely satisfied with Supplemental II Training	8/13 (62)
Satisfied (Somewhat or Extremely) with moderator call	7/7 (100)
Used FAQ developed from moderator call	4/10 (40) (3 were not sure)
Agree (somewhat or strongly) that my center produced high quality data	10/10 (100)
Extremely satisfied with whole experience	9/10 (90)

One common challenge the moderators experienced in these sessions was how to welcome laughter, which occurred naturally and spontaneously in conversations about bladder experiences. Existing literature has shown that laughter can be used to break the ice and lighten the conversation (Browne, 2016; Robinson, 2009). Not allowing laughter can lead to a divisive or derisive situations (In-person training –group debriefing discussions; moderator performance checklist).

Ninety-two percent of moderators who attended the training were satisfied with the in-person training, and 71% were satisfied with online resources. The combination of the training and online resources, along with the ongoing interaction, allowed us to find a balance between the need to standardize moderator behavior to produce quality data, and capitalizing on individual disciplinary expertise and skill.

“One thing that resonated with me during this process was the transparency of the moderators engaging in dialogue about what worked and what didn’t work throughout this process. I personally believe this was a result of the foundational training all moderators/RC’s attended in Chicago” (Moderator evaluation survey).

The integration of moderators into the PLUS Research Consortium and orientation to our conceptual framework and research goals, seemed to enable moderators to identify with the Consortium’s vision and goals. We were also successful in finding a balance between orientating the moderators to the bladder and our emerging bladder health definitions (Lukacz et al., 2018) without making them experts or educators. This may have contributed to their enthusiasm when they shared that FG participants were eager for bladder health information and often asked moderators questions like “When are you coming back?”and “When can we set something up?” (Moderator call).

Value of Action Learning Principles

The mock focus group at the in-person training and the ongoing discussion boards were designed to enable the moderators to listen and learn from each other and share their own expertise, tips and tricks. These activities were described as particularly valuable by moderators to practice and then implement strategies to help produce high quality focus group data that emphasized health experiences and met the study goals (Moderator evaluation survey). For example, focus group field note coding revealed moderators employed several strategies presented in the training to foster dialogue on health rather than disease throughout the focus groups (Field notes).

Moderator Investment in PLUS

Engagement of moderators in ongoing, iterative training enhanced their experience and allowed them to have an investment in the quality of the data produced. As an outcome of the training, the moderators remained engaged in the phases of the interpretation process of the data to contribute to the trustworthiness of the findings. They also expressed a strong interest in making sure the focus group participants were included in dissemination of findings. Across the evaluation metrics, moderators described the experience as personally “rewarding” and feeling “honored” to participate. Overall they communicated a strong affinity for the training process. Many also expressed concern and empathy for the participants and pride in their involvement in this research.

I really loved this project! I love engaging women in conversations around health topics. The one sad thing is that I felt like some women who shared these intimate feelings for the first time ever in a group suddenly realized how “abnormal” their behavior is. (Moderator Survey)

Overall, moderators felt pride and ownership for their data. All the moderators reported that they felt they had produced rich data, both at the midpoint and at the end of data collection (Moderator evaluation surveys). Moderator pride and investment was also demonstrated in their interest in providing research participants with educational information (since focus group participants’ knowledge of bladder health was demonstrated to be very low) and overwhelming interest in volunteering to engage in the interpretation process of the data that they helped produce (Second moderator call).

Iterative Adaptation

The use of mixed methods to evaluate moderators across the course of the study (Table 2) allowed us to identify their ongoing needs and to attend to issues of fidelity and data integrity. For example, in the coding of field notes, we were able to code investigator impressions of emerging themes and steer the moderator to probe more toward the bladder health constructs of interest. The coding of field notes also provided insight on how well moderator guide questions were working, if the order of the questions was working, and/or if there were focus group mechanics that needed to be adjusted to best produce our data.

Coding the field notes revealed that moderators were dealing with common focus group challenges, such as how to handle fact-seeking participants who pressed moderators for answers to various questions and how to know when a topic was fully explored. Thus, the Training Core established online discussion forums to facilitate the exchange of tips and experiences from moderators who were able to solve these challenges, such as being able to divert and redirect fact seeking questions. Field notes also provided insight on less than optimal handling of situations by moderators, providing opportunities for feedback directly or via our online learning infrastructure.

In addition to the existing yet iterative evaluation metrics, approximately at mid data collection, investigators identified the need to facilitate a real-time conversation with all of our moderators. We hosted two moderator conference calls (Figure 1) that were informal discussions with the training team allowing for mutual exchange of feedback from investigators and moderators. Moderators valued the calls suggesting they particularly valued “having the opportunity to hear about the experiences of other moderators and the opportunity to briefly discuss the trade-offs involved in focus group facilitation (in terms of time management) and the chance to clarify specific things like should each question be explicitly asked when using a more naturalistic approach” (Moderator evaluation survey).

The need for the Moderator calls emerged because we had an effective communication infrastructure where moderators could reach out without judgment and feel comfortable asking questions and seeking help. In fact, by the conclusion of the study all the moderators were fully invested in the study as evidenced by reports that they felt confident in being able to convey SHARE and PLUS Research interests to others outside of the study setting. Some moderators suggested that the supplemental online materials, which included investigator videos, inspired them to be more connected to PLUS research goals (Moderator evaluation survey).

Lessons Learned

One of the key lessons learned was that the in-person training, particularly the experiential skills building (e.g., mock interviews) was key in preparing moderators. We also learned that better tracking of moderator changing roles would have allowed the Training Core more specificity and adaptability with respect to moderator training needs in these cases. For example, one moderator was unable to attend the in-person training and another joined after the in-person training. A more deliberate approach to explicitly address training gaps for moderators who could not attend the training in person or came on to the project later would have strengthened moderator preparedness and consistency.

Our moderator conference calls were valuable opportunities to check in. In retrospect, more virtual check-ins could have been a good investment. We could have provided more attention to moderator experience with focus groups across the life course, especially at the ends of our life course continuum and in our special population groups. This would involve more specific training for moderators facilitating discussions with our youngest group (11–14 years) who struggled with both shyness and over-talking thereby leaving the moderator out of the discussion (Field notes) and our oldest group (65+ years), who were inclined to focus on negative medical conditions. Additionally, more training may have been needed to address unique issues that emerged in Spanish language focus groups, which we had to troubleshoot real-time. With just over half of our moderators saying they were extremely confident in explaining PLUS, and only 62% and 46% respectively reported having familiarity with the SHARE and distress protocol, there was room for better orientation to the larger study and specific protocols.

Moderators suggested providing the opportunity for more mock focus groups and the opportunity to observe focus groups, as well as full verbatim transcripts from each center to review, would have been useful to prepare for their own focus groups (moderator evaluation survey). Also, while the audio recording of the mock focus groups was helpful, some found it cumbersome (moderator evaluation survey). A video recording might provide additional advantages to moderators.

DISCUSSION AND CONCLUSIONS

In SHARE, we built a successful adaptive, sustainable, and flexible training model for transdisciplinary focus group moderators for a national study on women and adolescent females’ experiences, perceptions, beliefs, knowledge, and behaviors related to LUTS and bladder health across the life course. Success was evident in our evaluative metrics on satisfaction, skill building, and research efficacy, as well as in the quality of the data produced to answer our exploratory research questions. Investment in moderator training led to successful completion of 44 focus groups with three to 12 participants in each group across the seven research centers.

A critical priority for our team was to ensure that the content and design of the moderator training protocol itself reflected our transdisciplinary commitment to inclusion, while aligning with the PLUS research approach, which attends to one’s lived experience within their social ecology and across the life course and emphasizes health rather than disease. Building a transdisciplinary training protocol was a challenge requiring reconciliation of disciplinary contributions from health behavior, sociology, psychology, nursing, public health, and behavioral medicine, and alignment with our transdisciplinary research principles.

We created a balance that leveraged the diversity of qualitative research perspectives while standardizing an approach to meet our sensitive exploratory research aims. Qualitative research can provide powerful, transformative insight. However, in order to produce high quality data we must invest in rigorous, yet, iterative and flexible training. Our training approach may be used by transdisciplinary research teams conducting multi-site research to assure focus group research credibility and trustworthiness.