Generating National Dental PBRN Research Ideas through the ToP Consensus Method Workshop

INTRODUCTION

The effect of oral health problems like tooth decay and gum disease on our communities is well known (1). The real challenge has been instituting research studies which quickly and efficiently address these problems in practical and impactful ways. Over the past decade there has been a renewed focus to narrow the divide between research and practice. One major approach attempts to include practitioners in the research process through various partnership arrangements. Community-based participatory research (CBPR) is “a collaborative approach that equitably involves community members, organizational representatives, and researchers in all aspects of the research process” (2, 3, and 4). Practice-based research networks (PBRNs) have been using CBPR principles for decades by actively involving practitioners in generating research ideas and conducting research in their practice settings (5, 6).

PBRNs have demonstrated they can make major and unique contributions in improving clinical practice (7, 8, and 9). These Networks are characterized by an organizational infrastructure that transcends a single practice or study. The growth in number and diversity of PBRNs is due to the advantages they offer both to research and quality improvement, their ability to bring practice-relevant topics onto the research agenda, and their ability to move scientific advances into routine practice quickly (10-17).

On April 12, 2012, the National Institute of Health and the National Institute of Dental and Craniofacial Research announced the establishment of a National Dental Practice-Based Research Network (PBRN) awarding a $66.8 million, 7-year grant to consolidate its initiatives. The National Dental PBRN comprises of six distinct regions throughout the United States. The University of Texas Health Science Center at San Antonio (UTHSCSA) Dental School is the administrative site for the Southwest Region that includes the states of Texas, Oklahoma, New Mexico, Arizona, and Kansas. Currently, the Southwest Region has over 1,000 of the 5,200 members enrolled nationwide. The goal of the National Network is to improve oral health through clinical research designed for and by dentists and dental hygienists (18). A key operating principle for the National Network is that most research questions or ideas originated originate from practitioners. The answers to these questions have the potential to immediately improve clinical practice (18). One of the primary goals of the Southwest Region of the National Dental PBRN is to gain input from the members within the five-state region on research studies. If the studies developed were not important and of interest to the practitioner, the likelihood of the dental practices actually participating would be low and the benefits of the research minimal.

To generate research ideas among members, the Southwest Region used the Technology of Participation (ToP) method. The ToP method brings a high level of participation to the decision-making process while providing the ability to design the process around the specific needs of the organization (19). This problem solving technique emerged from community building efforts in Chicago’s West Side in the mid-1960s. Supported through the Institute of Cultural Affairs, the methods were developed and further refined from the 1960s to the 1980s through thousands of community action campaigns in North and South America, Asia, Latin American and Europe (20, 21).

The roots of the consensus method workshop can be traced back to “brainstorming” from Alex Osborn in 1941 and the Delphi Process developed in 1950s from the Rand Corporation (22). The Institute of Cultural Affairs added a layer to these processes by using Jean Piaget’s work on Gestalt psychology, which states that images are understood in patterns as a whole (22). Through the ToP consensus method workshop, participants cluster topics and then name the clusters to make sense of their own brainstorming. Typical brainstorming often leaves this piece out and is usually processed at a later time by an individual as opposed to a group. Through the ToP consensus method workshop, the group not only develops the ideas but also determines the meaning of the ideas in relation to the focus question.

The ToP methods include: focused conversation, consensus method workshop, action planning and participatory strategic planning (19, 21, and 23). ToP values include: participation from everyone in the group, importance of teamwork and collaboration, encouragement of individual and group creativity, action and ownership of decisions made, and allowance of time for reflection and sharing (21). The goal of ToP is to reveal and encourage the group’s commitment to a plan of action, not necessarily to find a solution everyone agrees on.

The benefits of the ToP method have been realized in both non-profit and corporate settings ranging from remote villages in India to Fortune 500 companies. Organizations use the ToP method when groups are ready to move forward with action at the individual or group level. ToP is based on a model that naturally occurs within our human thought process known as the ORID – Objective level (data, facts), reflective level (feelings), interpretative level (meaning) and decisional level (action). Conversations and meetings are designed in a way that all levels are addressed and in the appropriate sequence. This is in contrast with the nominal group technique and Delphi process, which are both commonly used in idea generation and prioritization for CBPR projects and, typically end at the interpretive level. The ToP method leads the group to resolution and helps determine how the information will be acted upon. Unlike the nominal group technique and Delphi process, which focuses on organizational relationships, the ToP method encourages interaction among group members, and goes beyond clarification to allow participants to interject why something is or is not important (24). Facilitators provide time for discussions throughout the process in order for group consensus to be reached. In addition, the ToP method allows individuals and groups as a whole to reflect on decisions made, and provides time for sharing based on the group resolve. Together these activities build a sense of collective action.

The ToP method is not well suited when a decision is already made or a best solution already exists. A critical guideline for the ToP method is acknowledging that the group has the appropriate authority to make key decisions and recommendations about the topic focus. Leadership must also commit to providing feedback in co-designing the event and agreeing to follow through with the key outcomes of the event. For each facilitation event, a facilitator carefully crafts rational and experiential aims after meeting with the client and understanding the intention of the program, and creates a design that reflects the overall goals. The rational aim identifies the focus of the event and includes major goals for the workshop; whereas, the experiential aim seeks to understand the mood and the desired impact on the participants during the process. One of ToP’s limitations is the preparation work is needed ahead of time. At times the scripting may prevent proper response to a group’s needs as opposed to emergent facilitation such as Open Space Technology (25). One of the major benefits from the ToP method is that the structure allows conversations and decisions to be made with both large and small heterogeneous or homogeneous groups.

To our knowledge, this process has not been used previously in dental PBRN idea generation. The purpose of this article is to illustrate the research idea generation process employed by the Southwest Region of the National Dental Practice-Based Research Network (PBRN) based on the Technology of Participation (ToP) consensus method workshop.

METHODS

Staff of the Southwest Region of the National Dental PBRN set-up five independent workshops over 12-month period. The workshops were developed for practicing dentists, dental hygienists, dental academicians and researchers in the region to identify research ideas appropriate for examination through the PBRN mechanism. The perspectives offered by each group were valued as important in the process. One of the goals of these workshops was integrating the researchers with the practicing dentists and dental hygienists by encouraging a bi-directional conversation.

The following details will explain the major steps involved in planning and implementing the Southwest Region consensus method workshops.

Step 1- Creating the goals and objectives of the workshop with a focus question

The primary goal of the workshops was to identify research areas of interest that could be addressed in the dental offices and clinics. The experiential aims for the workshop were to increase confidence to develop research topics, renew energy about the potential to work together to improve patient care, and increase willingness to support the vision of the National Dental PBRN. After multiple planning meetings, the Southwest Region team decided the focus question for the five workshops would be: What research topics are important and feasible for dentists and dental hygienists to examine in the PBRN setting?

Step 2- Scheduling a large group of dental practitioners and researchers in the appropriate meeting space

Once the plans and goals for the workshops were in place, the Southwest Region team worked with a research administration lead from each site to schedule a one to two hour session with dentists, dental hygienists, and researchers to explain the purpose of the National Dental PBRN and to generate research ideas that could be potential topics for the members to explore. A minimum of an hour was allocated solely for the consensus method workshop at each site. Invitations were sent out to individuals affiliated with the sponsoring entity, including current members, faculty and alumnae. Continuing Dental Education Credits were provided for the workshops that were held at the UTHSCSA Dental Hygiene Continuing Education program, the University of Texas Health Science Center Houston (UTHSCH) and the First Annual Southwest Region Meeting. The credits were approved by the respective school or program.

To host such an interactive ToP consensus method workshop, finding a suitable space was critical. At each of the locations, a large room with chairs and tables were set-up in a horseshoe formation. A large flat wall space was needed to hang up the “sticky wall,” a large rectangular piece of adhesive parachute material, used to attach all of the research ideas from the participants. The space had to be large enough to encourage participants to break out into small groups during the session and not over-hear conversations from other groups. Each table was set up with markers, half sheets of paper, and table tents for names. Stress reliever items and multicolor pipe cleaners were provided to inspire creativity and encourage participants to remain engaged in the activities.

Step 3- Developing a detailed internal agenda and training staff as needed

The Southwest Region team created a detailed internal agenda for each workshop tailored to the setting (e.g., number of participants, room layout, dynamics of the leadership and support from the organization). The detailed internal agenda included a script for each of the sections of the consensus method workshop and the materials required. Based on the size of the group, the staff was trained to assist with different portions of the workshop. The staff was assigned roles ranging from facilitating a focused conversation, to being a scribe during a brainstorming session.

Step 4- Implementing the plan by hosting a consensus method workshop

Each session began with a review of the agenda and defining ground rules for the one to two hour session. It was emphasized that for the consensus method to work, everyone had to participate and listen to understand before disagreeing with someone. The focus question was strategically placed on the center of the “sticky wall” and referenced frequently throughout the session. A brief focused conversation or warm-up conversation occurred for the next 10-20 minutes to allow participants to think about research in daily practice. Focused conversation is a scripted discussion with a series of questions that increase in difficulty. Everyone answered the first question and then volunteers were asked to answer the remaining questions of increasing complexity or depth. The facilitator sat down during the conversation and listened to the responses from the room. The focused conversations included the following questions:

• As you think of the term “research”, what is the first word or image that jumps into your mind? We would like everyone to give an answer to this first question.

• What topics have dentists and dental hygienists explored in clinical research in the past? What worked? What did not work?

• How would research add value to your work as a busy dentist or dental hygienist?

• This leads us to our focus question for the rest of the session: What research topics are important and feasible for dentists and dental hygienists to examine in the PBRN setting?

Following the focused conversation, the facilitator briefly described the process for the consensus method workshop before beginning the session. It was emphasized the group would be progressing from individual thinking, to working with small groups, and then to a large group to answer the focus question.

The following steps involved in Consensus Building will include few examples from a consensus method workshop held at the First Annual Southwest Region Meeting.

Step 1- Brainstorming in layers

Brainstorming occurred in several phases; initially, silent individual brainstorming (five minutes), followed by personally selecting favorites, and sharing the favorites among a small group of no more than five participants identifying six to eight research ideas of interest (20-30 minutes). Each research idea was written on a half sheet of paper with a marker, answering the focus question with no more than four to seven words. The ideas are written large enough on the paper to be seen when posted on the “sticky wall” (Table 1). The small groups gradually shared their diverse research ideas with the whole group starting first by giving their two clearest research ideas to the facilitator. It was emphasized all cards would eventually be collected to assure every research idea would be honored. The cards were read aloud by the facilitator and placed randomly on the wall in no particular order for participants to see. Geometric symbols (e.g., circle, triangle, square) were placed on the top row of the “sticky wall” to aid the facilitator in locating cards the participants referenced during the activity. After each round of two research ideas, the facilitator asked if any cards needed clarification. If so, the group members were given the opportunity to explain the intent of their card.

Table 1. Consensus Building Concept.

Examples of research ideas brainstormed were:

1. Compliance and post-op instructions

2. In-office fluoride treatment necessary for dentally healthy adults?

3. Patient/dentist perceptions on prescribing analgesics

4. What is the proper recall schedule?

5. Effectiveness of homecare on implants.

Step 2- Clustering Similar Ideas

After the first round of cards, the facilitator requested two to three additional cards that were completely different than any other research idea already displayed. The entire group was then asked to identify similar research idea pairs. Those pairs were used to form larger idea clusters. The facilitator asked the group to “tag” the cluster with a one to two word name identifying the focus of the cluster. Once clusters were formed, the facilitator asked the participants to bring their group’s remaining cards and place them in the corresponding cluster (Table 1). If the participant’s ideas did not fit an existing cluster they were asked to put them off to an unnamed side cluster for further discussion by the group. The group would decide if the new cards were a completely separate idea or related to some of the other clusters.

Examples of research ideas clustered with tag names were:

Cluster 1: Tag name (Prevention):

In-office fluoride treatment necessary for dentally healthy adults?

What is the proper recall schedule?

Cluster 2: Tag name (Post-Op Care):

Compliance and post-op instructions

Patient/dentist perceptions on prescribing analgesics

Effectiveness of homecare on implants

Step 3- Naming the cluster

Once all the ideas were placed on the wall and clustered into clusters based on input from the participants, each cluster was carefully named. The facilitator chose the cluster with the most cards first and read each card aloud. Participants were asked a series of questions to develop a name for the cluster that evoked the intent of all of the cards within the cluster and answered the focused question. The naming process typically took five to ten minutes for the first cluster and then proceeded faster for the remaining clusters (Table 1). To facilitate consensus, we asked a volunteer to write out all of the suggestions for the cluster names from the group and read them aloud to inspire development of a meaningful and descriptive name. Questions asked during the naming process included:

• What is this cluster all about? What makes it different or special from all the other clusters?

• What are the key words or phrases on these cards?

• What overall research ideas do we want to see in place by this cluster?

• Does it describe what we have been talking about?

• Do any of these cards not belong here or need to be moved?

The focus question and the final names of each cluster were then read to summarize the group’s work.

Examples of research ideas named were:

Cluster 1 name: New approaches to prevention

In-office fluoride treatment necessary for dentally healthy adults?

What is the proper recall schedule?

Cluster 2 name: Improving compliance for post-op care

Compliance and post-op instructions

Patient/dentist perceptions on prescribing analgesics

Effectiveness of homecare on implants

Step 4- Closing reflection conversation

Following the consensus method workshop, the session ended with a brief five to ten minute closing conversation. Again, everyone answered the first question and participants were asked to give input on the other questions. The closing conversation included the following questions:

• In this brief time we have been together, what is one word or image that comes to mind?

• What surprises did you encounter during this session?

• What are you most pleased with? What concerns linger?

• How would you describe the process we just went through?

• What are key next steps to moving this forward?

Step 5- Sharing the results of the workshop with participants

A two-page report was generated headed by the focus question with a detailed table including the name and research ideas of each cluster. The report kept the exact wording from the participants intact and did not alter anything the group decided. The second page included group photos from the activity and a detailed list of the participants. The report was emailed to all group members within two weeks of the meeting with contact information and instructions on next steps. The results of the workshop were also disseminated through Southwest Region’s quarterly newsletter and post-annual meeting summaries.

Step 6- Following Through

The goals for this process included the pairing of specific study concepts and with individuals’ interests. Participants who were interested in the development of a particular research idea were identified and partnered with other practitioners and academic experts with shared interest in the study concept. These individual comprise a study protocol development group (“POD”) and establish a schedule of conference calls directed by our Southwest Regional team toward the development of the study concept. The Region Director and Deputy Director guide the POD discussions, and once the initial concept is discussed, the principal investigator (PI) transfers the group’s thoughts onto a study concept template, which provides a short overview of the study. The group provides feedback on the study concept with the intention of preparing a study concept for submission to the National Dental PBRN for consideration. Following initial review by a National Director, the Executive Committee (EC), comprised of full-time clinicians, reviews the study concept. The committee approves, suggests revisions or rejects the concept during one of its monthly meetings at which the Study PI presents the concept. If the EC approves the study concept, it is forwarded to the National Institute of Dental and Craniofacial Research (NIDCR) Clinical Studies Group where it is assessed the study for relevance, scientific rigor, impact and appropriateness to determine whether it overlaps with a study already funded by NIDCR, and appropriateness to the PBRN perspective.

This report represents the results from five ToP workshops to identify major themes of interest from its members to be addressed by the National Dental PBRN and subsequent POD development.

RESULTS

This was the first time the consensus method workshop was adopted within the National Dental PBRN setting. The Southwest Region team was able to successfully engage five independent groups with a total of 197 participants. Table 2 describes the composition of each group. Four workshops were conducted in affiliation with the academic institutions and hence the attendees comprised of academicians and practicing practitioners.

Table 2. Composition of Participants from Each Group.

Consensus Method Workshop	UTHSC San Antonio Dental School	UTHSC Houston School of Dentistry	1st Annual Southwest Region Meeting	Continuing Education Dental Hygiene Program	Texas A&M Baylor College of Dentistry
Non-Academic Dentist	-	-	31	1	-
Non-Academic Hygienist	-	-	21	42	-
PhD	-	-	1	0	-
*Faculty	45	11	3	6	45
Total Participants	45	11	53	43	45
Total Study Ideas Generated	25	36	76	36	32

^*Faculty: The faculty consisted of academic and private practitioners

The dialogue and interaction among researchers and practitioners provided a new insight beyond a traditional brainstorming exercise. Practitioners enthusiastically offered practical research ideas based on their daily clinical experience. Table 3 outlines the major ideas generated from each group. There was much variety of ideas ranging from effect of oral pain on cognitive function to improving quality and decreasing costs for oral health care.

Table 3. Research Ideas Generated – Cluster Names only, does not include the individual ideas.

Consensus Method Workshop
Research Ideas Generated	UTHSC San Antonio Dental School	UTHSC Houston School of Dentistry	1st Annual Southwest Region Meeting	UTHSC San Antonio Dental Hygiene	Texas A&M Baylor College of Dentistry
	Student success	Effect of oral pain on cognitive function	* New approaches to prevention	Contributing factors to periodontal disease	Management of incipient caries
	Patient screening and retention	Etiologic factors in periodontal disease	*Improving compliance for post- Op care	Parental attitudes impact on oral health	Management of periodontal and mucosal diseases
	Bonded restorations	Examining restorative outcomes	Practitioner preference for restorative treatment	Effects of product	Restoring function and aesthetics
	Structural deficits	Improving quality and decreasing costs	Correlations of poor nutrition to oral and systemic health	Valuing oral health	Behavioral modifications of skills for dentists and hygienists
	Implant success	Team approach to patient education	Is systemic disease related to oral disease?	Integrity of sealants	Managements of peri- implantitis

Top 5 clusters from each workshop are listed in above table

^*Research ideas generated in these two clusters are cited as examples in the methods section

The workshops were distinctive in terms of participants and settings, but produced similar ideas. Three of the most common research topics generated within the workshops were: 1) periodontal disease related issues, 2) systemic and oral disease associations and 3) use and integrity of sealants. Three of the five workshops selected periodontal disease related issues as the number one topic to explore based on importance and feasibility. Within the general topic, various unique research ideas emerged including: etiologic factors in periodontal disease, periodontal implications from dental care, relationship between systemic disease and oral disease, contributing factors to periodontal disease, management of periodontal and mucosal disease and management of perio-implantitis.

The research ideas that emerged from Southwest Region workshops and from other regions of the National Network seem to be in alignment with NIDCR’s 2014 strategic goal of exploring research questions related to clinical decision making and treatment for dental care. Research within this scope has produced evidence-based results and has the ability to impact the clinical care of large, diverse populations of dental patients in the United States. An evaluation of 2013 funding for R01 grants from NIDCR showed that approximately one-third (33%) of funded projects were clinical in scope. Of those, only 40% of the clinical projects or just over 10% of funded R01’s, focused on concepts consistent with ideas developed by practitioners in our system. It cannot be claimed that the research idea generation from dental practitioners differs significantly from academicians, even though these numbers suggest this, since the research question generation process itself is rarely published in the literature. From our experience with PBRNs, the research ideas generated are much more patient-centered and relative to day to day practice, addressing common issues that seek to bridge the gap between treatment recommended and actual care provided. Unfortunately, practice-centered questions are infrequently addressed by academics, which focus their attention and resources on discovering new treatments in more limited population samples.

There was no formal evaluation conducted for the research generation conducted at the multiple sites. After each consensus workshop, participants completed a one page survey rating the following: 1) processes were clear and focused; 2) facilitator was well prepared, knowledgeable and helpful; and 3) group discussions and interactions were stimulating. Over 95% of all participants rated each of the categories as valuable or very valuable. Comments from the survey included:

• “I was with experienced hygienists and together we could concretize our ideas into words.” Dentist in San Antonio

• “I enjoyed the methods used to generate “categories” for future studies. This was very informative.”

• My favorite part of the day was the brainstorming, idea generation and the exposure to new ideas.”

Currently, there are 13 PODs engaged in some level of concept development. Overall, this collaborative effort is viewed very successful has successfully bringing brought together practitioners and academicians through a shared research interest, capitalizing on respective expertise and experience (Table 4). The consensus method workshop effectively engaged dentists, dental hygienists, researchers and academicians in a process to identify research areas of interest, in a brief one to two hour time period. By having the actual clinical experts generate the topics, prioritize, and decide what is important to them, ownership was achieved. The Southwest Region team now has a clear understanding of their members’ interest from both the dentist and dental hygienist perspective.

Table 4. Ongoing PODs from ToPs Generated Research ideas.

Individual ToPs Generated Research ideas	PODS	Review Status
Effects of E-Cigarettes	1. Novel Tobacco Products	Approved by EC
Dry Mouth Study (Effective Treatment Options) Drug Induced Xerostomia and Caries Risk Correlation of Dry Mouth and Caries	2. Xerostomia and Dental Caries	Submitted to EC
Effectiveness of Pit and Fissure Sealants on Smooth Surfaces Retention of Sealants	3. Sealant Recommendation	Concept in development
Definitive Study on the Effectiveness of Fluorides Knowledge of Current Fluoride Usage	4. Fluoride Implications	Concept in development
Immediate Placement and Loading of Implants	5. Immediate Implant Placement	Concept in development
Primary Household Language-How Does it Affect Restorative Needs	6. Oral Health Literacy	Approved by EC
Mini Implants in Adolescent Population Congenitally Missing teeth	7. Craniofacial Anomaly	Concept in development
Treating Periodontal Disease with Lasers	8. Laser-Assisted Periodontal Therapy	Concept in development
Oral and Systemic Health Diabetes vs Periodontal Disease	9. Dental Office Diagnostic Testing	Concept undergoing revisions based on EC suggestions
What Percentage of Dentists Following the ADA Antibiotic Guideline Relationships of Antibiotics to Various Dental Diseases Such as Diabetes etc.	10. Prophylactic Use of Antibiotics in Dental Practice	Submitted to EC
Oral Appliances for Sleep Apnea Recognizing and Referring Patients with Possible Sleep Apnea	11. Sleep Apnea	Concept Undergoing revisions based on EC suggestions
Use of Cariogenic Beverages in Hispanic Children	12. Dietary Effects on Oral Health	Concept in development
Bisphosphonate Treatment and Referral	13. Bisphosphonate Osteonecrosis of the Jaw	Concept in development

IMPLICATIONS

The ToP method is likely to be useful in many settings in which structured dialogue between researchers and practitioners is deemed important. The diverse ideas generated in a face-to-face setting produced collaborative results that would not have been achieved with a simple survey requesting idea input from individual members. Using this ToP consensus method workshop, participants had time to process the material and interpret the data before moving forward. The consensus method workshop can be used to promote an equal partnership to better inform and engage practitioners in the research process.

Out of every group, champions have emerged who have taken on these research ideas and are currently developing them into study concepts and protocols. It is important to note consensus does not mean everyone is in total agreement, but that every idea is honored and everyone is willing to move forward in a common direction. Consensus “is not so much majority rule – winners and losers, as it is a convergence of the common sense of the total group” (21). One of the more subtle, yet critical, aspects of this process is the synthesis of perspectives from both practitioners and academician/researchers in the development of study concepts.

We recognize that other brainstorming methods exist (26-31) and could have been used. General brainstorming in a room with a small group could be briefer than the one to two hour consensus workshop method. However, general brainstorming often does not allow time for individual brainstorming, pairing and then large group discussion which considers different learning styles. It has also been our experience that through unstructured general brainstorming activity, that participants with the ability to express their ideas faster or are more vocal gain the general attention and the more introverted participants may find it more difficult to express their ideas. The Delphi method is another tool that can be used and assists with systematically ranking ideas based on criteria and can be done over the computer (24). We believe that the list of ideas generated through our sessions incorporated research interests of the participants which were achieved through the power of group discussion. The opportunity to give feedback and seek further clarification of ideas encouraged inquiry and also integration of ideas compared to other methods which are more likely to only draw interests from certain individuals.

CONCLUSIONS

The experiences presented in this article suggest the further use and expansion of the ToP consensus method workshop can help bridge the gap between research and practice. The opportunity to engage in dialogue with both researchers and practitioners demystifies the research process, allowing for concrete recommendations for new studies and the modification of existing scientific approaches. The flexibility inherent to this method promotes quick and effective ways of generating ideas that can be applied to other settings and target audiences as well.