How to talk to strangers: facilitating knowledge sharing within translational health teams with the Toolbox dialogue method

Abstract

Translational behavioral medicine involves experts from different disciplines and professions interacting to solve complex problems. Coordinating this expertise can be frustrated by the partially tacit nature of expertise and by the various ways in which it manifests in different communities. We describe a method—the Toolbox dialogue method—for addressing these challenges by means of a structured dialogue among team members concerning their respective approaches to complex problems. The Toolbox dialogue method consists of a philosophically grounded questionnaire—the “Toolbox”—deployed in workshops by collaborators from different disciplines and professions. The Health Science Toolbox was modified from an extensively utilized questionnaire designed for Science–Technology–Engineering–Mathematics (STEM) research and has been piloted with translational medicine teams. Eighty-five percent of participants in STEM workshops indicated a positive impact on awareness of the knowledge, opinions, or scientific approach of teammates. In the Health Science Toolbox, 35 % of questionnaire responses changed substantially from pre- to post-workshop, demonstrating impact of the workshops. The Toolbox dialogue method is a relatively brief workshop encounter that can have a positive impact on mutual understanding within translational medicine teams.

BACKGROUND

Addressing complex issues in translational medicine increasingly involves formation of cross-disciplinary teams. For example, development of strategies for smoking cessation may span cell/molecular-based research, epidemiology, and community implementation science. However, effective cross-disciplinary research confronts institutional, infrastructural, logistical, interpersonal, and conceptual challenges [1, 3, 5, 9, 10].

Often, conceptual challenges arise from differing and tacit classificatory schemes and value systems used by collaborators to approach research [6, 11]. If left unspoken, such differences can manifest as misunderstanding, disagreement, and ultimately, failure to achieve project objectives. We have developed the Toolbox method to identify and articulate these differences through structured dialogue about knowledge-generating aspects of research and practice [2]. A substantial body of research suggests that facilitated, constructive, open dialogue within teams can lead to positive outcomes [8, 13]. We provide evidence that the Toolbox approach has a positive impact on mutual understanding within collaborative research teams.

THE TOOLBOX DIALOGUE METHOD

The Toolbox method comprises three elements: first, the questionnaire, or “Toolbox”, provides the initial topics that structure dialogue about research and practice; second, the workshop centers on the team’s dialogue; and third, the analysis yields insights based on data collected from the participating team.

Questionnaire

The original Toolbox questionnaire, designed for collaborative teams in Science–Technology–Engineering–Mathematics (STEM) research [2], consists of six modules that comprise 34 targeted prompts grounded in established theory in the philosophy of science and interdisciplinary research. In collaboration with the Institute of Translational Health Sciences (ITHS) at the University of Washington, we revised the STEM Toolbox to yield a set of prompts more relevant to translational health science research. Guided by the structure of the STEM Toolbox, development of the Health Science Toolbox began with identification of barriers and challenges to this work through a review of translational health literature, dialogue with members of the ITHS Education Core, and a survey distributed to ITHS leadership. Issues that emerged included the complexity of clinical trials with human subjects, different standards of control, and the influence of social and behavioral factors on bench and clinical medicine.

Using methods of philosophical analysis [12], we identified general categories into which many of the conceptual challenges confronting translational medicine teams fall. Several of these overlap with modules in the STEM Toolbox and, where appropriate, prompts were retained. In other cases, new prompts were developed that directly target the concerns of translational health scientists. For instance, one new prompt concerns whether controls in clinical research should be held to the same standard as in basic research or whether the systems investigated in clinical research are too complex to be reduced to control and experimental variables. Draft modules were revised in consultation with ITHS, with comments solicited from health science professionals guiding decisions about module structure and statement language in several revision rounds.

In the end, six modules were included in the Health Science Toolbox (see Appendix). The issue addressed by each module is expressed in a “core question” (Table 1), and aspects of the issue are then highlighted by a set of “probing statements” that are associated with 5-point agree/disagree Likert-type response options, along with “don’t know” and “not applicable” options. The modular design allows users to customize the Toolbox questionnaire. Teams may use a single module or the full set, depending on the conceptual challenges that confront the team and the time available.

Table 1.

Focal themes and core questions in the Health Science Toolbox

Module themes	Core questions
Motivation	What is your primary motivation for conducting research?
Research approaches	What things need to be taken into account in identifying a research problem?
Methodology	What are the most important considerations in study design?
Confirmation	What types of evidentiary support are required for knowledge?
Values	Do values have a legitimate role in scientific research?
Reductionism	Can the world under investigation be reduced to independent elements for study?

Workshop

The Toolbox questionnaire is used in a facilitated workshop setting that lasts from 90–120 min. This approach is predicated on the assumption that articulation of one’s fundamental research approach can enhance both self-understanding and mutual understanding [8, 13]. While it is not necessary to discuss every statement, we recommend spending time on each of the modules selected for the workshop. In our experience, the dialogue is more relevant to team interests and is taken more seriously by the team when the facilitator maintains a hands-off approach, letting the team members work out their own views without much interruption (Looney et al., forthcoming). We have conducted two pilot workshops using the Health Science Toolbox with members of the ITHS community.

Analysis

A variety of analyses can be conducted with data gathered from a Toolbox workshop, such as a quantitative comparison of participants’ pre- and post-workshop responses to the probing statements and a qualitative analysis of the interaction among participants.

EVIDENCE OF IMPACT

The Toolbox dialogue method is a discovery method designed to encourage conversation in which collaborators work out individual and collective interpretations of fundamental scientific concepts. We examined both qualitative and quantitative indicators of impact, including responses by participants to post-workshop evaluations.

STEM workshops

To assess the immediate impact of the workshops, we analyzed post-workshop evaluations that posed open-ended questions about key personal discoveries, impact on professional development, and whether the workshop altered individual views on the philosophical basis of their own science. We obtained responses from 147 of 278 participants from 35 independent STEM workshops. In an iterative process we independently and then jointly developed a list of key themes that emerged from the open-ended responses [7]. Two team members (inter-rater reliability > 85 %) coded all surveys for the presence of themes. Analysis of results reveals that participants overwhelmingly find value in the Toolbox and that the outcomes are consistent with our goals for the dialogue (Table 2). It was common to hear comments such as “these were great items that I hadn’t fully considered before.” A solid majority gave unequivocally positive evaluations of the dialogue and said it would help their professional development, and only a small percentage of respondents expressed negative or skeptical comments.

Table 2.

STEM workshop participant assessments (n = 139) of the impact of the Toolbox workshop

Key themes from open-ended responses to post-workshop evaluations	Percent
Workshop had a positive impact on awareness of the knowledge, opinions, or scientific approach of teammates	84.9
Overall assessment was entirely positive	82.6
Statements about impact on professional development were entirely positive	77.4
Workshop helped participant become more aware of dimensions of cross-disciplinary research, including challenges associated with working across disciplines and awareness of other disciplinary perspectives	43.9
Workshop helped participant become more aware of dimensions of science or scientific research	41.7
Workshop had (or could have) a positive impact on research communication	33.1
Workshop had a positive impact on the social aspects of team-building	18.7
Made at least one skeptical or negative comment about some aspect of the Toolbox workshop	8.6

Health science pilot workshops

The Health Science Toolbox was pilot tested in two workshops at ITHS in 2011, with a total of 15 participants. Of the six modules discussed (see Table 1), the participants spent the most time discussing the Values and Reductionism modules. In addition, definition of ‘translational’ was a major topic of discussion. Thirty of the 38 statements were discussed in at least one of the workshops, with only one statement receiving consistent negative feedback. (That statement has been removed from the questionnaire in the Appendix.)

Of the 15 participants, ten completed both pre- and post-workshop Toolbox questionnaires. We examined the changes in scores from pre- to post-workshop (Table 3): “no change” indicates an identical response on a pre- and post-workshop questionnaire statement; “minor change” indicates a one-point difference within same side of the scale (e.g., from agree to strongly agree); and “substantive change” indicates a change from valenced to uncertain response, from uncertain to valenced response, or reversal of opinion from agreement to disagreement or vice versa [4]. With ten participants and 38 items, there are 380 possible comparisons, although item non-response resulted in 371 actual cases in Table 3. We found substantive changes in 30–40 % of responses. Of these, 41 % were cases where a participant initially had an opinion about an item but after the dialogue decided that s/he was no longer certain. Although reversals were less common, they did occur; consider the participant who initially strongly disagreed about one of the items concerning the importance of replication, but who wrote afterwards, “Aha! In the discussions I go to agree.” Another noted, “I realized as I filled this out—wow—my perspective has really changed […] It was kind of a very interesting activity for me to do—as sort of a self-reflection, a sort of mirror for me to look at.” Such findings suggest that the dialogue had its intended effect of making participants thoughtfully consider other points of view. Also of note, the occurrence of substantive changes was similar across all six modules, suggesting that there is value in retaining all six during the workshop.

Table 3.

Changes in views among participants in the pilot Health Science Toolbox dialogues

Module	No change (%)	Minor shift (%)	Substantive change (%)
Motivation	45	23	32
Research approach	48	10	42
Methods	47	20	33
Confirmation	44	19	37
Values	49	13	38
Reductionism	51	20	29

N = 371 response comparisons

DISCUSSION

The Toolbox dialogue method focuses on conceptual issues held in common by scientific collaborators who may nevertheless disagree on their interpretation. We demonstrated that the Toolbox method is effective in the context of STEM workshops, as the vast majority of participants indicated a positive impact on their views. Although our experience with the Health Science Toolbox is more limited, we anticipate it should be equally effective, given that we have modified the questionnaire systematically using translational health literature and expert guidance. Confidence in the Health Science Toolbox is supported by evidence of impact drawn out of the Likert-type data from the ITHS pilot workshops.

The Toolbox method is not designed to address institutional and logistical challenges for collaborative work or pragmatic issues such as budget allocation or authorship. Its central purpose is to enable individuals and teams to recognize and articulate their own interpretations of concepts central to their scientific practice. This process can be complex. For example, even if there is apparent agreement among workshop participants about the prompts, the basis for this agreement may be different in important ways. Conversely, large differences on some prompts may not pose problems for teams in practice; indeed, one of the principal motivations for conducting cross-disciplinary work is to take advantage of these differences through collaboration [9]. While the Toolbox is not intended to change anyone’s views on the topics discussed, we have demonstrated that explicit articulation of differences can benefit mutual understanding. Whether or not team members should alter some of their thinking on these issues is something the team must address; the role of the workshop is to put the team in a position to efficiently determine what (if anything) needs to be resolved.

Teams may incorporate the Toolbox into their development in different ways. Some teams may choose to incorporate regular Toolbox dialogues into their long-term development. Given the open-ended and modular nature of the Toolbox questionnaire, teams may use an initial dialogue to identify areas of concern and then create their own probing statements and/or modules that target issues that are specific to their needs. In addition to facilitating workshops, the Toolbox Project team [http://www.cals.uidaho.edu/toolbox/] can assist with the creation of such modules, suggest new tools for group development, or provide Train the Trainers workshops for teams interested in developing the Toolbox idea further (Looney et al., forthcoming).

CONCLUSION

Translational behavioral medicine brings to bear multiple disciplinary perspectives on behavioral aspects of illness and health, and the Toolbox dialogue method aims to enable the cognitive integration of these perspectives in the context of specific collaborative projects. The method involves use of structured dialogue to reveal how collaborators think about their collective work. While this is a relatively brief encounter in the life of a collaborative team, the dialogue concerns fundamental beliefs about the team’s common project and is a rare opportunity to build bridges collectively between different disciplinary and professional cultures.

APPENDIX

The Health Sciences Toolbox questionnaire