Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2019 May 1.
Published in final edited form as: Am Psychol. 2018 May-Jun;73(4):549–562. doi: 10.1037/amp0000281

Interdisciplinary Team Science and the Public: Steps Towards a Participatory Team Science

Jacob Kraemer Tebes 1, Nghi D Thai 2
PMCID: PMC5973546  NIHMSID: NIHMS946249  PMID: 29792467

Abstract

Interdisciplinary team science involves research collaboration among investigators from different disciplines who work interdependently to share leadership and responsibility. Although over the past several decades there has been an increase in knowledge produced by science teams, the public has not been meaningfully engaged in this process. We argue that contemporary changes in how science is understood and practiced offer an opportunity to reconsider engaging the public as active participants on teams, and coin the term participatory team science to describe public engagement in team science. We discuss how public engagement can enhance knowledge within the team to address complex problems, and suggest a different organizing framework for team science that aligns better with how teams operate and with participatory approaches to research. We also summarize work on public engagement in science, describe opportunities for various types of engagement, and provide an example of participatory team science carried out across research phases. We conclude by discussing implications of participatory team science for psychology, including changing the default when assembling an interdisciplinary science team by identifying meaningful roles for public engagement through participatory team science.

Interdisciplinary team science involves research collaboration among investigators from different disciplines who work interdependently to share leadership and responsibility (Bozeman & Boardman, 2014; NRC, 2015). Knowledge produced by such interdisciplinary research teams has increased steadily over the past several decades (Bozeman & Boardman, 2014; Wuchty, Jones, & Uzzi, 2007), including in psychology and the social sciences (Henricksen, 2016). Collaboration allows researchers to address complex problems unable to be addressed by individual investigators or a single discipline (Collins, Wilder, & Zerhouni, 2014). Also, interdisciplinary science teams can accelerate the translation of knowledge from laboratory to clinic to community (Collins et al., 2014; NIH, 2006a; Terry & Leshner, 2013).

The trend toward interdisciplinary team-based research is one outgrowth of the increasing specialization of disciplines, and the value scientists give to research collaborations that provide specialized expertise, resources, or affiliations (Leahey, 2016). We define research collaboration as social processes in which “human beings pool their human capital for the objective of producing knowledge” (Bozeman & Boardman, 2014, p. 3). A term from science and technology studies that is used to characterize the additional value (or “additionality”) from research collaboration is scientific and technical human capital, or STHC (Bozeman & Boardman, 2014). STHC refers to the knowledge, skills, and resources embodied by an individual that are essential to scientific productivity (Bozeman, Dietz, & Gaughan, 2001). It may include an individual’s specialized education, training, and expertise as well as the social networks and institutional resources available to that researcher (Bozeman & Boardman, 2014). STHC operates at multiple levels – the individual, the science team, the organization where individuals and teams are located, and the disciplines or fields in which researchers are trained, conduct research, publish, and participate in a scholarly community (Bozeman & Boardman, 2014).

The decades-long growth of interdisciplinary team science has mostly not included public stakeholders as research collaborators on science teams, despite increased public engagement in science over this period (Tebes, 2018). We use the term “public” to refer to individuals with a shared “interest in or who are impacted by the proposed research” (Burke et al., 2013, p. 494). Public stakeholders may include: adults, children, or youth; community members; employees, employers, or representatives of businesses, nonprofits, industry, or government; funders; policy makers; and researchers (Burke et al., 2013; Minkler & Wallerstein, 2011; PCORI, 2016).1

In this paper, we argue that contemporary changes in how science is understood and practiced offer a new opportunity to engage the public as participants on interdisciplinary science teams. We believe that doing so adds value because public stakeholders provide local, culturally-situated, and contextualized knowledge about complex psychological and social problems unknown or unavailable to a science team, but essential to its success (Tebes, 2018; Tebes, Thai, & Matlin, 2014). Engagement can also give the public a voice in prioritizing research to address local interests; guide how research is implemented, communicated, and disseminated; and enhance transparency between scientists and the public (Esmail et al., 2015; IOM, 2011; PCORI, 2013). We coin the term participatory team science to describe public engagement in team science.

This paper consists of five parts. First, we briefly summarize concepts in interdisciplinary team science critical to this discussion. Second, we describe contemporary changes in how science is understood and practiced, and offer a conceptual framework for organizing team science. Third, we discuss the nature of public engagement in science and summarize relevant taxonomies of engagement. Fourth, we provide an overview of participatory team science by drawing on the literature to identify opportunities for public engagement, illustrating its use in a hypothetical example, and discussing key challenges. And finally, we discuss implications of participatory team science for psychology.

Background

Types of Disciplinary Research Collaboration

Rosenfield (1992) was the first to define the distinctions among disciplinary research collaborations. In unidisciplinary research an investigator uses models and methods from a single discipline to study a problem, which is the traditional approach to science. Multidisciplinary research involves collaborations by investigators from two or more disciplines who focus on a common problem, but study it from the perspective of their own discipline. The research may be informed by multiple disciplines, but the work is done within each discipline. In interdisciplinary research there is collaboration among investigators from different disciplines who develop a shared mental model to guide the research (Porter & Rafols, 2009; Rosenfield, 1992). This differs from multidisciplinary research because collaborators blend models and methods from their own discipline to examine a problem in a new way. Finally, transdisciplinary research refers to an interdisciplinary research collaboration that eventually results in a new, hybrid discipline (Hall et al., 2012; Rosenfield, 1992). Women’s studies, bioengineering, sustainability science, neuroscience, and science and technology studies are examples of hybrid disciplines that grew out of interdisciplinary studies (Tebes et al., 2014).

Structures and Phases of Interdisciplinary Team Science

In their monograph Enhancing the Effectiveness of Team Science (NRC, 2015), the National Research Council identifies two primary structures for organizing team science: the science team and larger groups of teams. In the science team, the organizational structure for the research is a collaborating, interdependent team of researchers that typically numbers two to 10 individuals. Larger groups of teams involve groups requiring further differentiation of labor than is found in an individual team (NRC, 2015). A science team (or groups of teams) can exist within and across universities, academic medical centers (AMCs), or interdisciplinary research centers.2

Hall et al. (2012) identify four tasks that are carried out in phases on a science team: 1) development, 2) conceptualization, 3) implementation, and 4) translation. In the development phase, a prospective team assembles members who define a problem, develop a shared mental model to study it, and begin to establish a team identity. In the conceptualization phase, the new team identifies the specific research questions to be addressed, a design to do so, communication practices to conduct research, and team roles and responsibilities. In the implementation phase, the team carries out and coordinates the research, manages conflict, and integrates what is learned. In the final translation phase, the team applies its learning to address the real-world problem(s) that brought it together, which may include developing partnerships with industry, government, other investigators, and the public (Hall et al., 2012; NRC, 2015). As we show later, depending on the problem identified, the team task, the local context, and the team science phase, engaging public stakeholders can add value during one or more team phases.

Teamwork, Knowledge Integration, and Social Interactions on a Science Team

Teamwork occurs “when members interact interdependently and work together toward shared and valued goals” (Salas, Fiore, & Letsky, 2013, p. 41) to maximize team performance (Salas, Shuffler, Thayer, Bedwell, & Lazzara, 2015). This requires coordination, cooperation, and communication, as well as intersubjective understanding of team tasks and goals. Essential to team performance is a team’s integrative capacity (Salazar, Lant, Fiore, & Salas, 2012), that is, a team’s capability “to build effective communication practices, a shared identity, and a shared conceptualization of a problem” (Salazar et al., 2012, p. 528) to create new knowledge. A team’s integrative capacity is the foundation for knowledge integration, which requires managing the social interactions of the team to: 1) develop a shared mental model of the problem and how to address it scientifically; 2) identify team roles and tasks; and 3) establish communication structures and processes to carry out the work (Salas et al., 2013; Salazar et al., 2012). When these social processes are managed effectively, various cognitive, affective, and motivational states emerge within the team, such as trust, psychological safety, a collaborative spirit, openness to new ideas, and identification with the team that enable members to blend their competencies to address a scientific challenge in a new way (Salazar et al., 2012). Thus, it is knowledge integration that creates “additionality” essential to innovation in interdisciplinary team science.

The centrality of social interactions to knowledge integration in an interdisciplinary science team illustrates how team science is fundamentally a social enterprise, a perspective that aligns with Kuhn’s (1962) seminal work in the philosophy of science as well as with the views of contemporary philosophers of science (e.g., Giere, 2006; Knorr Cetina, 1999). Central to this perspective is that scientists’ social relationships and interpersonal transactions are essential to knowledge production (Tebes, 2005; Tebes et al., 2014; Tebes 2017), and that managing those transactions is critical to the success of science teams (NRC, 2015; Salazar et al., 2012).

The importance of managing social transactions among scientists in a science team led Bennett, Gadlin, and Levine-Finley (2010) to develop the “Field Guide” for collaboration and team science. The Guide recommends that collaborators schedule time to develop a shared vision for the research, identify common goals, communicate directly about the science, and discuss how recognition and credit will be shared. Bennett et al. (2010) and other scholars (Fiore, Carter, & Ascencio, 215; Salazar et al., 2012) also emphasize the need to build trust among team members, develop guidelines to manage conflict, and establish group norms for effective team management. Finally, teams that engage in reflective practice may be more intentional about using social processes to advance the work (Bennett et al., 2010; Salazar et al., 2012). Later we discuss how the Field Guide is also applicable to integrating the public on interdisciplinary science teams.

Contemporary Changes in How Science is Understood and Practiced

A Conceptual Framework for Interdisciplinary Team Science

As interdisciplinary team science has grown, it has mostly operated within a traditional normative, scientific framework (Tebes et al., 2014). That framework typically begins with an individual PI who seeks grant funding to study a problem defined from the perspective of a single discipline. The proposed study builds on prior research conducted in that discipline, and is evaluated for funding by disciplinary scholars for its innovativeness. Should the research result in a discovery, it is reviewed for publication in disciplinary journals. Knowledge about the problem advances, but usually only within a single discipline or one of its specialties. This “usual” narrative for doing science aligns with stories in the history of science of the lone scientist in pursuit of a great discovery (e.g., Archimedes, Newton, Einstein, Freud). That narrative has shaped the modern research laboratory, which is organized hierarchically with members accountable to a PI. Collaboration mostly takes place with junior colleagues, students, and staff, or perhaps in parallel with another PI’s lab through multidisciplinary research.

Interdisciplinary team science requires a different framework for science practice. Instead of organizing the research hierarchically in a single lab, a PI collaborates with one or more investigators from another discipline as a Co-PI on a common project. Some work may be done across labs or in a collaborative team, and various members of the team may be accountable to multiple investigators, just as Co-PIs are accountable to each other for the team’s success. Social processes that promote team integrative capacity are essential because knowledge integration depends on it. Thus, instead of organizing a lab or a team hierarchically to complete essential tasks, it may be more beneficial if the team is organized as a heterarchy; that is, as a complex adaptive system of interconnected, overlapping, and dynamic components that govern constituent interdependent and networked components (Tebes, 2012, Tebes et al., 2014). Such a system organizes itself flexibly in response to emerging environmental demands and is best understood holistically (Tebes et al., 2014; Cilliers, 2013; Miller & Page, 2007). Network science has shown that individuals function within various heterarchies (e.g., families, workplaces, communities) that consist of dynamic social networks that shape decision-making, health, and well-being (Christakis & Fowler, 2009). Other examples of heterarchies include Wikipedia, various biological signaling processes, and evolutionary systems (Tebes, 2012).

The concept of heterarchy offers an alternative organizing framework for science that captures the participatory, non-hierarchical structure of shared leadership and mutual accountability essential to team science. Lead investigators that operate within a heterarchy function interdependently depending on the task required. This does not preclude organizing some research tasks hierarchically when necessary, such as when efficiency in team functioning is a priority (Tebes et al., 2014). However, in a heterarchy the default organizational structure assumes mutuality, interdependence, and power sharing. Such a non-hierarchical, participatory structure is more common in large, scientific collaborations in which all parties benefit from the sharing of data, instrumentation, and expertise distributed across scientific teams (Shrum, Genuth, & Chompalov, 2007). Teams organized this way are increasingly common in the high-tech sector, where start-ups may function effectively with lateral accountability and competencies distributed across the company (Stark, 2009). Organizing science teams heterarchically encourages the development of shared leadership structures and processes with the expectation that team members contribute as both leaders and collaborators.3

Mode 1 and Mode 2 Science

The use of heterarchical structures to produce scientific knowledge reflects a transformation in science that is currently underway (Tebes et al., 2014). This transformation was first described in the book, The New Production of Knowledge, by Gibbons, Limoges, Nowotny, Schwartzman, Scott, and Trow (1994), and elaborated in a subsequent volume, Re-Thinking Science, by Nowotny, Scott, and Gibbons (2003a). Gibbons, Nowotny, and their colleagues have argued that for the past century, knowledge production in science has followed norms and practices that value research that is theoretically-driven; university- and institution-based; unidisciplinary; experimentally-focused; hierarchically-organized; investigator produced; prioritizing of investigator autonomy; and seeking universal knowledge (Gibbons et al., 1994; Nowotny et al., 2003a, 2003b). They refer to this as Mode 1 knowledge production in science. This contrasts with the emergence, in the latter half of the 20th century, of an alternative form of knowledge production, which they refer to as Mode 2, that values different norms and practices. Mode 2 knowledge production is application oriented; accountable not only to academic and scientific institutions but to government and industry as well as political, economic, and public stakeholders; is inter- or transdisciplinary; emphasizes use of multiple and mixed methods; is organized heterarchically; is co-produced with multiple stakeholders; is socially-distributed, collaborative, and transparent; and seeks knowledge that is embedded in local contexts and cultures (Nowotny et al., 2003a, 2003b).

Mode 1 science uses disciplinary norms, tools, and methods, whereas Mode 2 science is carried out in the “context of application” (Gibbons et al., p. 3), that is, to solve a problem that transcends solution by any single discipline. Mode 2 science is also consistent with the practices underlying knowledge integration in interdisciplinary team science because it accounts for the need to integrate multiple epistemic cultures (Knorr Cetina, 1999) essential to a science team. Thus, just as team science is fundamentally a social enterprise, Mode 2 science is also a public enterprise that values and prioritizes public engagement as integral to knowledge production (Nowotny et al., 2003a). For the Mode 2 scientist, engaging public stakeholders adds STHC to enhance knowledge integration to solve a problem, just as engaging interdisciplinary colleagues adds STHC to enhance knowledge integration in a science team. In both instances, adoption of a heterarchical structure for the work provides a framework for shared leadership and accountability that is socially-distributed and has as its goal the co-production of knowledge.

In our view, the boundaries between Mode 1 and Mode 2 science are more fluid than described by their authors. This is due, in part, to the now more widely recognized value of Mode 2 knowledge production in science and society (Boaz, Biri, & McKevitt, 2016; Thoren & Brejan, 2016). For example, social science researchers increasingly acknowledge the importance of transparency in science, the value of culturally-situated and indigenous epistemologies, the need to identify public benefits from taxpayer-funded research, the necessity of local accountability by scientists to communities, and the potential gains from application-oriented research (Tebes et al., 2014; Trickett et al., 2011). In the current climate, debating whether knowledge production results from Mode 1 or Mode 2 science, or their combination, is simply less relevant (Tebes et al., 2014).

Public Engagement in Science

The growth of Mode 2 science has occurred as calls for public engagement in science have increased (Boaz et al., 2016). Scholars from diverse fields support public engagement, including: the basic sciences (Kost et al., 2017), public health (Minkler & Wallerstein, 2011), global health (Lavery et al., 2010), implementation science (Damschroder et al., 2009), translational science (Selker & Wilkins, 2017), environmental sciences (Bonney et al., 2014), and science and technology studies (Stilgoe, Lock, & Wilsdon, 2014). Governments across the world also promote engagement. In the U.K., the national advisory group to foster public engagement in research, INVOLVE, publishes a Briefing Notes for Researchers (INVOLVE, 2012). In Canada, the agency responsible for the nation’s health, publishes the Policy Toolkit for Public Involvement in Decision Making (Health Canada, 2000). In the U.S., NIH funds community-engaged research through the Clinical and Translational Science Award (CTSA) program (NIH, 2006a), which is endorsed by the Institute of Medicine (IOM, 2011). Also, the emergence of comparative effectiveness research (CER) and patient-centered outcomes research (PCOR) led to the creation of the Patient-Centered Outcomes Research Institute (PCORI), a quasi-governmental body that promotes public engagement in research (PCORI, 2013). Finally, several international organizations advocate for public engagement (e.g., International Association for Public Participation; NESTA).

Progress toward public engagement in science is difficult to discern because it goes by various names: participatory action research (Fals-Borda & Rahman, 1991; Whyte, Greenwood, & Lazes, 1989), empowerment evaluation (Fetterman, Kaftarian, & Wandersman, 1996), citizen science (Irwin, 1995), patient and public involvement in research (Brett et al., 2012), stakeholder engagement in research (Concannon et al., 2014), and community-engaged or community-based participatory research (CBPR) (Israel, Schulz, Parker, & Becker, 1998; Minkler & Wallerstein, 2011).4 Although these various approaches resist unitary classification, they each: promote participation by stakeholders as collaborators in research, acknowledge a bi-directional partnership between researchers and stakeholders, seek to establish trust within the partnership to advance knowledge, respect differences in perspective among collaborators, and are committed to co-learning and the co-production of knowledge (Israel et al., 1998; Minkler & Wallerstein, 2010; PCORI, 2013; Stilgoe et al., 2014; Tebes et al., 2014).

A central argument for public engagement in science is that stakeholders are “experts” in local, culturally-situated, and contextualized knowledge relevant to the research question examined (Black et al., 2013; Trickett et al., 2011). Public engagement can inform the research-practice gap (Trickett et al., 2011), explain data anomalies (Tebes & Kraemer, 1991), and guide how research is implemented, communicated, or disseminated (IOM, 2011; PCORI, 2013). Culturally-situated knowledge is also often intersectional (Crenshaw, 1991), that is, it embodies the lived experience at the intersection of multiple social identities (e.g., black, gay, aged, Muslim) that involve experiences of oppression (e.g., racism, homophobia, ageism, religious bias). Such knowledge is essential to understanding social and psychological problems in multiple local contexts, and in the systematic study of solutions to those problems (Hall, Yip, & Zarate, 2016; Rosenthal, 2016). Incorporating more local, culturally-situated, and contextualized knowledge into science is also likely to enhance construct and external validity (Tebes, 2000, 2005; Leviton, 2017).

Taxonomies of Public Engagement

Public engagement in science involves research done with rather than to or for individuals who are participants rather than subjects (Bromley et al., 2015; Esmail et al., 2015). Since public engagement may differ across researchers and contexts, scholars have developed taxonomies of engagement and collaborative inquiry (Trickett & Espino, 2004). Arnstein’s (1969) ladder of citizen participation is an early influential taxonomy that applies to any form of public engagement. The ladder depicts eight rungs embedded in three levels of citizen power. The top rung with the most power is “citizen control”, followed in descending order, by “delegated power” and “partnership.” Arnstein considers these three forms of engagement to be actual participation, and the remaining five rungs to represent “tokenism” or “non-participation.” For Arnstein, “placation” (or advisement, as is common on research advisory boards), “consultation” and “informing” are successive descending rungs depicting forms of tokenism because participants have input into decision making but no real power to affect it. The final rungs, in descending order, are “therapy” and “manipulation”, in which citizens are delivered services or subject to policies mostly outside of their control.

Arnstein’s (1969) metaphorical ladder provides a comparative framework for other taxonomies of public engagement in science. One such taxonomy for CER/PCOR has six successive stages of engagement for health-related stakeholders (Concannon et al., 2012), with each stage specifying a translational objective: identifying research priorities (Stage 1), generating evidence through trials and observational studies (Stage 2), synthesizing evidence through reviews and meta-analyses (Stage 3), integrating evidence through various methods to reveal gaps in the literature (Stage 4), applying evidence through implementation and dissemination science, and policy (Stage 5), and evaluating CER/PCOR (Stage 6). In a review using this taxonomy, Concannon et al. (2014) show that CER/PCOR scholars generally engage patients, providers, and other stakeholders infrequently. Esmail et al. (2015), in another review, show that in CER/PCOR, researchers mostly examine impacts of engagement rather than processes or contexts. To address these limitations, Burke et al. (2013) propose a remedy: more collaboration between CBPR and CER/PCOR researchers. They argue that collaboration across these two scholarly traditions would build a stronger CER/PCOR translational evidence base but also include a wider array of stakeholders through CBPR. Participatory team science offers this opportunity.

Participatory Team Science

Participatory team science blends two growing developments in the practice of science, interdisciplinary team science with participatory research approaches. To varying degrees, each adopts heterarchy as an organizing framework and seeks to build integrative capacity within the research partnership to promote knowledge integration and the co-production of knowledge.

How best to assemble, manage, and sustain an interdisciplinary science team is just now becoming understood through studies in the science of team science (SciTS), an emerging area of research that examines the processes, outcomes, and impacts of team science (Falk-Krzesinski et al., 2010). Thus far, however, few SciTS studies have focused on public engagement on science teams, or on the added benefit (and cost) to knowledge integration of doing so (Tebes, 2018). An exception to this is work done through CTSAs (NIH, 2006a), which are required to engage community stakeholders as research partners. CTSAs have mostly been established in AMCs or university Clinical Translational Sciences Institutes (CTSIs), and thus far, meaningful engagement of stakeholders through CTSAs has been modest (Terry & Leshner, 2013). In part, this is because CTSAs operate within settings that prioritize Mode 1 science, which can devalue actual community participation (Freeman et al., 2014; Shepard et al., 2013). Kost et al. (2017) describe a CTSA with substantive public engagement in research and training as an exception.

Opportunities for Public Engagement in Participatory Team Science

In the following paragraphs, we offer examples of public engagement in participatory team science for each phase identified by Hall et al. (2012): development, conceptualization, implementation, and translation. These phases overlap with other taxonomies of public engagement, such as the Concannon et al. (2012) stages described earlier; the PCORI rubric for public engagement which emphasizes planning, conducting, and disseminating research as engagement activities (PCORI, 2016); and the well-established IOM translational framework (IOM, 2011). For each phase, we provide published examples of engagement consistent with participatory team science, although most were not originally conceptualized as exemplars of this approach.

In the development phase (Hall et al., 2012), a science team may partner with public stakeholders, such as a coalition or clinicians from a local clinic, to address a problem of mutual relevance. This might entail: a) deciding which stakeholders will participate on the team; b) developing a shared understanding of the problem; 3) identifying the mission and goals of the team; and 4) building trust, psychological safety, and team identity. Examples of stakeholder activities might include: engaging residents of public housing in problem identification for a smoking cessation intervention (Andrews et al., 2012); developing a youth partnership to design a survey of youth sexual health (Flicker et al., 2010); and conducting public forums to identify disparities in health care access to treat depression in primary care (Wells et al., 2013).

In the conceptualization phase (Hall et al., 2012), the newly assembled team identifies novel research questions to address, perhaps based on a theory of change developed by the team. In addition, the team develops a research design that accommodates the various interdisciplinary components of the research while considering stakeholder concerns. For some teams, an overall design may already have been developed for funding purposes, but significant details remain that may be specified collaboratively by the team. During this phase, the team establishes communication structures and practices, creates a shared language for interaction, and identifies roles and responsibilities within the team.5 Examples of engagement during this phase include: developing a team logic model to evaluate a neighborhood initiative (Tebes et al., 2014); obtaining feedback from community health workers, caregivers, and emergency department staff to design an asthma treatment trial (Martin et al., 2017); and partnering with an indigenous community to develop intervention modules to prevent alcohol use and suicide (Rasmus, Billy, & Mohatt, 2014).

In the implementation phase, the team carries out the research based on their roles and responsibilities. Since the research process is dynamic, member tasks, roles, and responsibilities require regular adjustment as implementation proceeds, and conflict management is required. Examples of activities during this phase might include: engaging promotoras’ (Latina community health educators) to help identify study sites, recruit participants, and develop intervention strategies and materials for cancer prevention (Larkey et al., 2009); supporting mental health consumers as they collect and analyze data about clinical services (Case et al., 2014); and sharing results with participants after a study is completed to determine how well the results align with participants’ “lived” experience (Tebes & Kraemer, 1991).

In the translation phase, the team seeks to translate findings to individuals and settings for real-world impact. Again, the team adapts to a dynamic research environment to adjust roles and responsibilities, and identify opportunities for translation and dissemination. Examples of activities during this phase might include: sponsoring a conference to build translational capacity for collaborating researchers and community members by sharing experiences from a community randomized trial (Khodyakov et al., 2014); publishing a participatory, team-authored, peer-reviewed article to inform academic and professional audiences about a suicide prevention mobilization initiative (Mohatt et al., 2013); and engaging multiple groups of patients, caregivers, and investigators connected to a large health care organization to develop practice guidelines for treating multiple chronic conditions (Bennett et al., 2017).

These examples draw on literature in CBPR, CER/PCOR, organizational and community psychology, and public health, to show potential opportunities for public engagement in participatory team science. They also show that teams can benefit from public engagement even when this occurs only during specific phases, which differs from CBPR. In our view, public engagement during all phases of team science is unrealistic, cost prohibitive, and unsustainable, given barriers to interdisciplinary team science and public engagement. As teams begin to engage public stakeholders, SciTS researchers can systematically study the benefits and costs of doing so, and how different types of engagement effect integrative capacity and knowledge integration.

An Integrated Example of Participatory Team Science

Next and in Figure 1 we provide a hypothetical example of how participatory team science might work. Our team seeks to address a complex problem: the combined impact of adverse childhood experiences, or ACEs, and neighborhood disadvantage on the health and well-being of children and families. ACEs are various childhood adversities (e.g., maltreatment, family violence, parental substance misuse, extreme poverty) that have been found to result in genetic, brain, and behavioral sequellae that have the potential for life-long health and developmental effects (Gilbert et al., 2015; Nugent et al., 2016). Neighborhood disadvantage (e.g., exposure to neighborhood violence, physical decay, and social and physical disorder) is a social determinant associated with diminished health and well-being (Marmot & Wilkinson, 2005; Sampson, Raudenbush, & Earls, 1997). Examining the impact of these adversities may lead to interventions and policies that mitigate or prevent their effects and promote health and well-being.

Figure 1.

Figure 1

Open in a new tab

Team Integrative Capacity across Four Phases in Participatory Team Science. Shown is a Hypothetical Science Team Studying the Impact of Adverse Childhood Experiences (ACEs) and Neighborhood Disadvantage on the Health and Well-Being of Children and Families

In our example, investigators from four disciplines – psychology, sociology, neuroscience, and economics – decide to address this issue from an interdisciplinary perspective. The psychologist is interested in understanding developmental processes and impacts within the school and family. The sociologist seeks to examine the interplay of neighborhood processes associated with neighborhood disadvantage on health and well-being. The neuroscientist wants to learn more about how relative exposure to these adversities alters specific reward mechanisms in the brain. The economist is interested in identifying causal social mechanisms for child and family outcomes using instrumental variables analysis with neighborhood census and geocoded crime data. As shown, our hypothetical team will also eventually include a coalition representative, a parent representative, a policymaker, a health care provider, and an urban planner. It is important to note that stakeholders may not occupy key roles during each phase, and may join the team for the duration of the research or during specific phases, depending on the nature of the collaboration.

Figure 1 depicts the progression of this participatory science team through the four phases described earlier (Hall et al., 2012). The development phase begins with the investigators’ initial discussions about a potential collaboration to study ACEs and neighborhood disadvantage. In these discussions, investigators begin to define the problem to be studied, construct a shared mental model to address it, and identify public stakeholders to inform the research. The newly-assembled interdisciplinary team then invites two community representatives -- one from a neighborhood coalition experienced in mobilizing residents and another from an active parent group in a public elementary school – to provide local, culturally-situated, and contextualized knowledge to inform the research. This expanded team then further refines the problem definition and contextualizes the mental model. Next, in the conceptualization phase, the team decides on the specific research questions to examine and the methods to employ. A local policymaker is invited to join the team, perhaps a representative from the Mayor’s office, who can help navigate key research implementation challenges, procure local and state resources, and align the research with various policy priorities. In the implementation phase, members identify specific sites for data collection, including local childcare settings, schools, after school programs, and health clinics. To assist with clinic implementation, the team invites a health care provider with ties to the local clinic network. In addition, the parent and coalition representatives and their respective networks become more active in framing messages to help recruit parents and community members. In the final translation phase, the team invites an urban planner with expertise in neighborhood redevelopment to join the team. The planner joins other team members to study results and disseminate them to public, professional, policy, and academic audiences. This maximizes the study’s potential for impact on diverse populations and settings, and on real-world challenges.

Figure 1 also shows that as the team progresses through these four phases, interactions among members are part of a recursive process that affects team integrative capacity. Building that capacity requires managing social interactions within the team to foster trust, psychological safety, and other qualities essential to task completion during each phase. Successful management of those interactions results in emotional, cognitive, and motivational states in the team that promote knowledge integration (Salas et al., 2013; Salazar et al., 2012). As noted earlier, it is knowledge integration that creates the additional value, or “additionality” emblematic of team science (Bozeman & Boardman, 2014). Although it may be challenging to manage interactions between investigators and public stakeholders on a science team, evidence from participatory research shows that it can be done effectively to generate new knowledge (Israel et al., 1998; Minkler & Wallerstein, 2011). What may be unique about participatory team science, however, is that social interactions are now managed within an interdisciplinary team of investigators who each operate interdependently to lead parts of the research, as is typical in a heterarchy. However, the strategies described earlier for establishing effective team science collaborations (Bennett et al., 2010) and research partnerships with the public (INVOLVE, 2012) provide guidance here as well.

Challenges to Participatory Team Science

We described how participatory team science might work, first by drawing on the participatory research literature and then by describing a hypothetical example. Our intent was to show that combining interdisciplinary team science and participatory research is possible. However, doing so is not without challenges, which we discuss next.

One challenge to participatory team science is that many policies and practices in universities and AMCs work against interdisciplinarity as well as team science (Klein 2010; Leahey, 2016; NRC, 2015). Examples include: aligning administrative and reward structures by department, and retaining promotion and tenure policies that do not support interdisciplinary or team-based research, whether in a science team or in CTSIs (Klein 2010; Marrero et al., 2013; McBee & Leahey, 2016). Gradually, however, changes are happening to address these challenges, in part due to enhanced incentives for team science (Collins et al., 2014; IOM, 2011). The creation of interdisciplinary research centers in academic settings has begun to shift the balance of power away from academic departments to create dynamic opportunities for interdisciplinary collaboration (NRC, 2015). Also, some academic settings are adopting promotion and tenure policies that value and reward team-based research (Hall & Vogel, 2014; NRC, 2015).

Another challenge to participatory team science is that disciplinary research receives the vast majority of funding (Bromham, Dinnage, & Hua, 2016; Ledford, 2015). Reductions in U.S. federal research support of all kinds over the past decade (NIH, 2016) have sustained this disparity despite calls for team science funding (Collins et al., 2014). Targeted support for team science through federal institutes and centers (Collins et al., 2014) as well as increases in federal research support for public engagement (PCORI, 2013) are positive trends in this regard. Although there is currently no funding for participatory team science, several potential sources of research support may be leveraged to provide funding (e.g., CTSAs, CTSIs, CER, PCORI).

Another challenge is that currently there are no generally accepted norms for providing compensation to public stakeholders engaged in research as consultants, community or content experts, or advisors (Black et al., 2013). This creates an equity issue when some members of the team are paid and others are not. Although this challenge is common to CBPR and other community-engaged research (Israel et al., 1998), it poses a significant problem for participatory team science because compensation inequities undermine the value ascribed to public stakeholders as leaders on science teams. Black et al. (2013) describe various barriers to compensation for community experts in CTSA-based research, and some innovations to address this.

Another challenge for participatory team science is determining its relevance to basic science. Most participatory research focuses on clinical, implementation, or translational research objectives, rather than on basic science. We have limited information on how best to engage the public on basic science, especially when an issue may be far removed from their expertise or experience. However, reports of engaging the public in basic science are increasing. DelNero and McGregor (2017) describe a program to promote dialogue between basic science cancer researchers like themselves and individuals living with cancer, and discuss how the program transformed their research. Kost et al. (2017) report on a successful initiative, funded through a CTSA, to engage community stakeholders in translational research that focuses on basic science mechanisms for public health. Several reports from the U.K. of patient and public involvement in basic science describe partnerships to study arthritis, dementia, diabetes, and sight-loss, and discuss how engagement can inform research priorities, improve designs, and sharpen how scientists communicate about their work (Dobbs & Whitaker, 2016; Petit-Zeman & Locock, 2013; Wilson et al., 2015). And a recent report on gene-drive modified organisms by U.S scientists calls for robust public engagement to aid in decision making and risk assessment (NASEM, 2016).

A final challenge for participatory team science is the dearth of education and training in team science for investigators (NRC, 2015) and the public (Parkes, Pyer, Wray, & Taylor, 2014). The National Research Council has called on universities to accelerate the development of training in team science for researchers and practitioners (NRC, 2015). Although evidence of program effectiveness is lacking, reports from a few training programs in AMCs, CTSIs, and research centers show promise (Concannon et al., 2012; Kost et al., 2017; Spring et al., 2012). Within psychology, only industrial/organizational psychology programs routinely provide training on team processes, and there is long-standing work in organizational behavior on self-directed work teams relevant to participatory team science (Orsburn, Moran, Muselwhite, & Zenger, 1990). Training in participatory approaches is not as well established in psychology, although it is a core competency in community psychology and widespread in public health. A need also exists to develop education and training programs for the public, and a few such programs are now available (Parkes et al., 2014; Horobin et al., 2017). In addition, CER/PCOR sites as well as AMCs and CTSIs provide fertile ground for blending participatory and team science approaches for training purposes, and to evaluate their effectiveness for researchers and the public. Finally, in addition to The Field Guide (Bennett et al., 2010), education and training resources relevant to participatory team science that can be adapted for researchers or the public are available online through PCORI (www.PCORI.org), INVOLVE (www.invo.org.uk), and the Team Science Toolkit (www.teamsciencetoolkit.cancer.gov).

Implications and Conclusions

More than 40 years ago in the pages of the American Psychologist, Walsh, Smith, and London (1975, p. 1067) called for adoption of “an interdisciplinary team approach to solving societal problems.” When compared to biomedicine, engineering, and the physical sciences, psychology has been late in answering their call. A recent electronic search of the 10 top-ranked general psychology journals, based on the Journal Impact Factor, found only 18 articles that even mentioned “team science” and only three that identified it as a substantive focus.6 And yet, psychologists have made seminal contributions to understanding how teams work in organizations (Salas, Rico, & Passmore, 2017), how research collaborations are fostered and sustained through cyber communication (Olson & Olson, 2014), how science teams are evaluated (Stokols et al., 2003), and how collaborative inquiry with communities leads to new knowledge (Trickett et al., 2011). Psychologists also have a long-standing tradition of multi- and interdisciplinary team-based practice in a variety of settings (McDaniel & DeGruy, 2014). As interdisciplinary scholarship has grown in psychology and other fields (Henriksen, 2016), the emerging field of interdisciplinary team science provides psychologists with new opportunities for collaboration as researchers, practitioners, evaluators, and consultants.

In this paper, we have argued that the current landscape of how science is understood and practiced is changing, moving not only toward interdisciplinary team science but also to participatory approaches that engage the public. However, science teams generally do not engage public stakeholders, and as a result, local, culturally-situated, and contextualized knowledge about a problem may not be incorporated into their work. Doing so may add scientific value because it can enhance a team’s integrative capacity and promote knowledge integration. We are not proposing that all science teams always engage the public, but ask simply that the potential benefits of public engagement be considered when assembling a team, that is, to change the default for how teams are assembled and problems examined in an interdisciplinary context. If interdisciplinary team science holds promise for addressing major real-world problems, so-called “grand challenges” (Efstathiou, 2016), such as addiction, cancer, global health, climate change, health disparities, and so on, we need to adopt an “all-hands-on-deck” approach to these challenges. We can no longer mostly ignore the public as key partners in this effort, and psychologists can lead the way through participatory team science.

Acknowledgments

The authors are grateful for support from the National Institute of Drug Abuse (T32DA019426), and acknowledge the contributions on their thinking from colleagues in the Interdisciplinary Research Consortium on Stress, Self-Control, and Addiction (UL1RR024925, RL5DA024858). The authors are also grateful for comments from three anonymous reviewers and the editors.

Footnotes

1

In health-related research, public stakeholders may also include: patients, clients, and consumers; family members; service providers and clinicians; purchasers (i.e., of health benefits for employees); payers (i.e., intermediaries in the health system); product makers and industry; hospitals and health systems; policymakers; training institutions (i.e., professional or trade); and researchers (Concannon et al., 2012; PCORI, 2016).

2

Science teams that involve research collaborations across geographic distances may also be established as “collaboratories” (Olsen & Olsen, 2014) when members use electronic tools to share data, resources, or expertise.

3

A few years ago, the National Institutes of Health (NIH) established the Co-Principal Investigator (Co-PI) structure for collaborating investigators who share equal leadership of a single research award (NIH, 2006b). This policy shift sought to ensure that each investigator receives credit for the research while also being held equally accountable. The Co-PI structure is heterarchical, but as noted later, university administrative practices and policies can undermine it.

4

Earlier calls for public engagement included Lewin (1946) who used the term “action research” to describe the collaboration between social scientists seeking to solve a social problem with those experiencing it, and Friere (1970) who partnered with oppressed peoples in collaborative inquiry. Wallerstein and Duran (2011) describe these scholars as representing the “Northern” and “Southern” roots, respectively, of the participatory research tradition.

5

Specifying team roles and responsibilities helps build transactive memory among team members. The team’s transactive memory, which is updated in later phases, identifies who in the team has specialized knowledge, how that knowledge is accessed and communicated, and if it is credible (Zajac et al., 2014).

6

We searched article texts for “team science” in the 10 top-ranked psychology journals, based on the 2016 Journal Impact Factor, excluding specialty journals. Journals included and the number of mentions of “team science” were: Psychological Science in the Public Interest (0); Annual Review of Psychology (0); Psychological Bulletin (0), Perspectives on Psychological Science (1); Psychological Review (0); Psychological Inquiry (2); Current Directions in Psychological Science (2); Psychological Science (2); American Psychologist (11); Psychological Methods (0).

Contributor Information

Jacob Kraemer Tebes, Yale University School of Medicine.

Nghi D. Thai, Central Connecticut State University

References

  1. Arnstein SR. A ladder of citizen participation. J Amer Institute Planners. 1969;35:216–224. [Google Scholar]
  2. Andrews JO, Tingen MS, Brunson J, … Magwood G. Application of a CBPR Framework to Inform a Multi-level Tobacco Cessation Intervention in Public Housing Neighborhoods. American Journal of Community Psychology. 2012;50(1–2):129–140. doi: 10.1007/s10464-011-9482-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Bennett LM, Gadlin H, Levine-Finley S. Collaboration & team science: A field guide. NIH Office of the Ombudsman; Rockville, MD: NIH Publication No. 10–7660; 2010. [Google Scholar]
  4. Bennett WL, Robbins CW, Bayliss EA, Wilson R, Tabano H, Mularski RA, … Boyd CM. Engaging stakeholders to inform clinical practice guidelines that address multiple chronic conditions. Journal of General Internal Medicine. 2017:1–8. doi: 10.1007/s11606-017-4039-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Black KZ, Hardy CY, De Marco M, Ammerman AS, Jean-Baptiste J. Beyond incentives for involvement to compensation for consultants: increasing equity in CBPR approaches. Progress in Community Health Partnerships. 2013;7(3):263. doi: 10.1353/cpr.2013.0040. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Boaz A, Biri D, McKevitt C. Rethinking the relationship between science and society. Health Expectations. 2016;19(3):592–601. doi: 10.1111/hex.12295. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Bonney R, Shirk JL, Phillips TB, Wiggins A, Ballard HL, Miller-Rushing AJ, Parrish JK. Next steps for citizen science. Science. 2014;343(6178):1436–1437. doi: 10.1126/science.1251554. [DOI] [PubMed] [Google Scholar]
  8. Bozeman B, Boardman C. Research collaboration and team science: A state-of-the-art review and agenda. New York: Springer; 2014. [Google Scholar]
  9. Bozeman B, Dietz J, Gaughan M. Scientific and technical human capital: An alternative model for research evaluation. Internat J of Tech Management. 2001;22:636–655. [Google Scholar]
  10. Brett J, Staniszewska S, Mockford C, Herron-Marx S, Hughes J, Tysall C, Suleman R. Mapping the impact of patient and public involvement on health and social care research: a systematic review. Health Expectations. 2012;17(5):637–650. doi: 10.1111/j.1369-7625.2012.00795.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Bromham L, Dinnage R, Hua X. Interdisciplinary research has consistently lower funding success. Nature. 2016;534(7609):684–687. doi: 10.1038/nature18315. [DOI] [PubMed] [Google Scholar]
  12. Bromley E, Mikesell L, Jones F, Khodyakov D. From Subject to Participant: Ethics and the evolving role of community in health research. Amer J Public Health. 2015;105:900–908. doi: 10.2105/AJPH.2014.302403. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Burke JG, Jones J, Yonas M, Elizabeth M. PCOR, CER, and CBPR: Alphabet soup or complementary fields of health research? Clinical Trans Science. 2013;6(6):493–496. doi: 10.1111/cts.12064. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Case AD, Byrd R, Kaufman J. Stakeholders’ Perspectives on Community-Based Participatory Research to Enhance Mental Health Services. American Journal of Community Psychology. 2014;54(3–4):397–408. doi: 10.1007/s10464-014-9677-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Christakis NA, Fowler JH. Connected: The surprising power of our networks and how they shape our lives. New York: Little, Brown, & Co; 2009. [Google Scholar]
  16. Cilliers P. Understanding complex systems. In: Sturmberg JP, Martin CM, editors. Handbook of Systems and Complexity in Health. New York: Springer; 2013. pp. 27–38. [Google Scholar]
  17. Collins FS, Wilder EL, Zerhouni E. NIH Roadmap/Common Fund at 10 Years. Science. 2014;345:274–276. doi: 10.1126/science.1255860. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Concannon TW, Meissner P, Grunbaum JA, McElwee N, Guise JM, Santa J, Leslie LK. A new taxonomy for stakeholder engagement in patient-centered outcomes research. Journal of General Internal Medicine. 2012;27:985–991. doi: 10.1007/s11606-012-2037-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Concannon TW, Fuster M, Saunders T, Patel K, Wong JB, Leslie LK, Lau J. A systematic review of stakeholder engagement in comparative effectiveness and patient-centered outcomes research. J of General Internal Medicine. 2014;29:1692–1701. doi: 10.1007/s11606-014-2878-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Crenshaw K. Mapping the margins: Intersectionality, identity politics, and violence against women of color. Stanford Law Review. 1991;43:1241–1299. [Google Scholar]
  21. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Sci. 2009;4:50. doi: 10.1186/1748-5908-4-50. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. DelNero P, McGregor A. From patients to partners. Science. 2017;358(6361):414. doi: 10.1126/science.358.6361.414. [DOI] [PubMed] [Google Scholar]
  23. Dobbs T, Whitaker I. Patient and public involvement in basic science research—are we doing enough? BMJ Opinion. 2016 Retrieved 11/5/17 blogs.bmj.com/bmj/2016/05/11/ppi-in-basic-science-research-are-we-doing-enough/
  24. Efstathiou S. Is it possible to give scientific solutions to Grand Challenges? On the idea of grand challenges for life science research. Studies in History and Philosophy of Science Part C: Studies History & Philosophy of Biological & Biomedical Sci. 2016;56:48–61. doi: 10.1016/j.shpsc.2015.10.009. [DOI] [PubMed] [Google Scholar]
  25. Esmail L, Moore E, Rein A. Evaluating patient and stakeholder engagement in research: moving from theory to practice. J Comparative Effectiveness Res. 2015;4:133–145. doi: 10.2217/cer.14.79. [DOI] [PubMed] [Google Scholar]
  26. Falk-Krzesinski HJ, Börner K, Contractor N, Fiore SM, Hall KL, Keyton J, Spring B, Stokols D, Trochim W, Uzzi B. Advancing the science of team science. Clinical and Translational Science. 2010;3(5):263–266. doi: 10.1111/j.1752-8062.2010.00223.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Fals-Borda O, Rahman MA, editors. Action and Knowledge: Breaking the Monopoly with Participatory Action Research. New York: Apex Press; 1991. [Google Scholar]
  28. Fetterman DM, Kaftarian SJ, Wandersman A, editors. Empowerment Evaluation: Knowledge and Tools for Self-Assessment and Accountability. Thousand Oaks, CA: Sage; 1996. [Google Scholar]
  29. Fiore SM, Carter DR, Asencio R. Conflict, trust, and cohesion: Examining affective and attitudinal factors in science teams. In: Salas E, Estrada AX, Vessey WB, editors. Team Cohesion: Advances in Psychological Theory, Methods, and Practices. Bingley, UK: Emerald Group Publishing; 2015. pp. 271–301. [Google Scholar]
  30. Flicker S, Guta A, Larkin J, Flynn S, Fridkin A, Travers R, Layne C. Survey design from the ground up: Collaboratively creating the Toronto Teen Survey. Health Promotion Practice. 2010;11(1):112–122. doi: 10.1177/1524839907309868. [DOI] [PubMed] [Google Scholar]
  31. Freeman E, Seifer SD, Stupak M, Martinez LS. Community Engagement in the CTSA Program. Clinical and Translational Science. 2014;7:191–195. doi: 10.1111/cts.12158. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Friere P. Pedagogy of the oppressed. New York: Seabury Press; 1970. [Google Scholar]
  33. Gibbons M, Limoges H, Nowotny H, Schwartzman S, Scott P, Trow M. The New Production of Knowledge. London: Sage; 1994. [Google Scholar]
  34. Giere RN. Scientific perspectivism. Chicago: University of Chicago Press; 2006. [Google Scholar]
  35. Gilbert LK, Breiding MJ, Merrick MT, Thompson WW, Ford DC, Dhingra SS, Parks SE. Childhood adversity and adult chronic disease: An update from ten states and the District of Columbia. American J of Preventive Medicine. 2015;48:345–349. doi: 10.1016/j.amepre.2014.09.006. [DOI] [PubMed] [Google Scholar]
  36. Hall GCN, Yip T, Zarate MA. On becoming multicultural in a monocultural research world: A conceptual approach to studying ethnocultural diversity. American Psychologist. 2016;71:40–51. doi: 10.1037/a0039734. [DOI] [PubMed] [Google Scholar]
  37. Hall KL, Vogel AL, Stipelman BA, Stokols D, Morgan G, Gehlert S. A four-phase model of transdisciplinary team-based research: Goals, team processes, and strategies. Translational Behavioral Medicine. 2012;2(4):415–430. doi: 10.1007/s13142-012-0167-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Hall KL, Vogel AL. Team-Based Science: Strategies for Success, Practical Tools, and Future Directions. National Cancer Inst, CBIIT Speaker Series; Washington, D.C: 2014. [Google Scholar]
  39. Health Canada. Health Canada Policy Toolkit for Public Involvement and Decision Making. Corporate Consultation Secretariat, Health Policy and Communications Branch, Minister of Public Works and Government Services; Ottawa, Ontario, Canada: 2000. [Google Scholar]
  40. Henriksen D. The rise in co-authorship in the social sciences (1980–2013) Scientometrics. 2016;107:455–476. [Google Scholar]
  41. Horobin A, Brown G, Higton F, Vanhegan S, Wragg A, Wray P, Walker DM. Co-producing public involvement training with members of the public and research organisations in the East Midlands. Research Involvement and Engagement. 2017;3(1):7. doi: 10.1186/s40900-017-0056-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Institute of Medicine. Learning What Works: Infrastructure Required for Comparative Effectiveness Research. Washington, DC: National Academies; 2011. doi.org/10.17226/12214. [PubMed] [Google Scholar]
  43. INVOLVE. Briefing notes for researchers: Involving the public in NHS, public health and social care research. INVOLVE; Eastleigh, United Kingdom: 2012. [Google Scholar]
  44. Irwin A. Citizen Science. London: Routledge; 1995. [Google Scholar]
  45. Israel BA, Schulz AJ, Parker EA, Becker AB. Review of community-based research. Annual Review of Public Health. 1998;19:173–202. doi: 10.1146/annurev.publhealth.19.1.173. [DOI] [PubMed] [Google Scholar]
  46. Khodyakov D, Pulido E, Ramos A, Dixon E. Community-partnered research conference model. Progress in community health partnerships. 2014;8(1):83–97. doi: 10.1353/cpr.2014.0008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Klein JT. Creating interdisciplinary campus cultures: A model for strength and sustainability. San Francisco: Jossey Bass; 2010. [Google Scholar]
  48. Kost RG, Leinberger-Jabari A, … Tobin JN. Helping basic scientists engage with community partners to enrich and accelerate translational research. Acad Med. 2017;92:374. doi: 10.1097/ACM.0000000000001200. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Knorr Cetina K. Epistemic Cultures: How the Sciences Make Knowledge. Cambridge, MA: Harvard University Press; 1999. [Google Scholar]
  50. Kuhn TS. The structure of scientific revolutions. Chicago: University of Chicago Press; 1962. [Google Scholar]
  51. Larkey LK, Gonzalez JA, Mar LE, Glantz N. Latina recruitment for cancer prevention education via community based participatory research strategies. Contemporary Clinical Trials. 2009;30(1):47–54. doi: 10.1016/j.cct.2008.08.003. [DOI] [PubMed] [Google Scholar]
  52. Lavery JV, Tinadana PO, Scott TW, Harrington LC, Ramsey JM, Ytuarte-Nuñez C, James AA. Towards a framework for community engagement in global health research. Trends in parasitology. 2010;26(6):279–283. doi: 10.1016/j.pt.2010.02.009. [DOI] [PubMed] [Google Scholar]
  53. Leahey E. From Sole Investigator to Team Scientist: Trends in the Practice and Study of Research Collaboration. Annual Review of Sociology. 2016;42:81–100. [Google Scholar]
  54. Ledford H. How to solve the world’s biggest problems. Nature. 2015;525:308–311. doi: 10.1038/525308a. [DOI] [PubMed] [Google Scholar]
  55. Leviton LC. Generalizing about Public Health Interventions: A Mixed-Methods Approach to External Validity. Annual Review of Public Health. 2017;38:371–391. doi: 10.1146/annurev-publhealth-031816-044509. [DOI] [PubMed] [Google Scholar]
  56. Lewin K. Action research and minority problems. Journal of Social Issues. 1946;2(4):34–46. [Google Scholar]
  57. Marmot MM, Wilkinson RG, editors. Social Determinants of Health. Oxford Press; 2005. [Google Scholar]
  58. Marrero DG, Hardwick EJ, Staten LK, … Saha C. Promotion and tenure for community-engaged research. Clinical Translational Science. 2013;6:204–208. doi: 10.1111/cts.12061. [DOI] [PMC free article] [PubMed] [Google Scholar]
  59. Martin MA, Press VG, Erwin K … CHICAGO Plan Investigators. 2017 Advance online publication https://doi.org/10.1080/02770903.2017.1342256.
  60. McBee D, Leahey E. New directions, new challenges: Trials and tribulations of interdisciplinary research. In: Frickel S, Albert M, Prainsack B, editors. Investigating Interdisciplinary Collaboration. New Brunswick, NJ: Rutgers Univ Press; 2016. pp. 35–63. [Google Scholar]
  61. McDaniel SH, deGruy FV., III An Introduction to Primary Care and Psychology. American Psychologist. 2014;69:325–331. doi: 10.1037/a0036222. [DOI] [PubMed] [Google Scholar]
  62. Miller JH, Page SE. Complex adaptive systems: Computational models for social life. Princeton, NJ: Princeton University Press; 2007. [Google Scholar]
  63. Minkler M, Wallerstein N, editors. Community-based Participatory Research for Health: From Process to Outcomes. San Francisco: Jossey-Bass; 2011. [Google Scholar]
  64. Mohatt NV, Singer JB, Evans AC, Matlin SL, Golden J, Harris C, … Tebes JK. A community’s response to suicide through public art: Stakeholder perspectives from the Finding the Light Within project. American Journal of Community Psychology. 2013;52:197–209. doi: 10.1007/s10464-013-9581-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  65. National Academies of Sciences, Engineering, & Medicine (NASEM) Gene Drives on the Horizon: Advancing Science, Navigating Uncertainty, and Aligning Research with Public Values. National Academies Press; 2016. [PubMed] [Google Scholar]
  66. National Institutes of Health (NIH) Institutional Clinical and Translational Science Award (U54) 2006a http://grants.nih.gov/grants/guide/rfa-files/RFA-TR-12-006.html.
  67. National Institutes of Health (NIH) Establishment of Multiple Principal Investigator Awards for the Support of Team Science Projects. 2006b NOT-OD-07-017. http://grants.nih.gov/grants/guide/notice-files/NOT-OD-07-017.html.
  68. National Institutes of Health. NIH Budget History: NIH Budget Mechanism Detail FY 2001–2015. 2016 Retrieved February 5, 2017 at: https://report.nih.gov/nihdatabook/index.aspx.
  69. National Research Council (NRC) Enhancing the effectiveness of team science. National Academies Press; 2015. [PubMed] [Google Scholar]
  70. Nowotny H, Scott P, Gibbons M. Re-Thinking Science: Knowledge and the Public in an Age of Uncertainty. Cambridge: Polity Press; 2003a. [Google Scholar]
  71. Nowotny H, Scott P, Gibbons M. “Mode 2” Revisited: The New Production of Knowledge. Minerva. 2003b;41:179–194. [Google Scholar]
  72. Nugent NR, Goldberg A, Uddin M. Topical review: The emerging field of epigenetics. Journal of Pediatric Psychology. 2016;41(1):55–64. doi: 10.1093/jpepsy/jsv018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  73. Olson JS, Olson GM. Working Together Apart: Collaboration over the Internet. San Rafael, CA: Morgan & Claypool; 2014. [Google Scholar]
  74. Orsburn JD, Moran L, Musselwhite E, Zenger JH. Self-Directed Work Teams: The New American Challenge. Homewood, IL: Business One Irwin; 1990. [Google Scholar]
  75. Parkes JH, Pyer M, Wray P, Taylor J. Partners in projects: preparing for public involvement in health and social care research. Health Policy. 2014;117(3):399–408. doi: 10.1016/j.healthpol.2014.04.014. [DOI] [PubMed] [Google Scholar]
  76. PCORI. Strategic Plan. Patient Centered Outcomes Research Institute; Washington, D.C: 2013. [Google Scholar]
  77. PCORI. [Accessed June 29, 2017];Engagement Rubric. 2016 www.pcori.org/sites/default/files/Engagement-Rubric.pdf. Published February 2014, updated June 6, 2016.
  78. Petit-Zeman S, Locock L. Bring on the evidence. Nature. 2013;501(12):160–161. doi: 10.1038/501160a. [DOI] [PubMed] [Google Scholar]
  79. Porter AL, Rafols I. Is science becoming more interdisciplinary? Measuring and mapping six research fields over time. Scientometrics. 2009;81:719–745. [Google Scholar]
  80. Rasmus SM, Charles B, Mohatt GV. Creating Qungasvik (a Yup’ik intervention “toolbox”): Case examples from a community-developed and culturally-driven intervention. American Journal of Community Psychology. 2014;54(1–2):140–152. doi: 10.1007/s10464-014-9651-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  81. Rosenfield PL. The potential of transdisciplinary research for sustaining and extending linkages between the health and social sciences. Social Science Medicine. 1992;35:1343–1357. doi: 10.1016/0277-9536(92)90038-r. [DOI] [PubMed] [Google Scholar]
  82. Rosenthal L. Incoporating intersectionality into psychology: An opportunity to promote social justice and equity. American Psychologist. 2016;71:474–485. doi: 10.1037/a0040323. [DOI] [PubMed] [Google Scholar]
  83. Salas E, Fiore SM, Letsky MP, Salas E, Fiore SM, Letsky MP, editors. Theories of team cognition: Cross-disciplinary perspectives. New York: Routledge; 2013. Why cross-disciplinary theories of team cognition? pp. 41–63. [Google Scholar]
  84. Salas E, Rico R, Passmore J, editors. The Wiley-Blackwell Handbook of the Psychology of Team Working and Collaborative Processes. Hoboken: Wiley-Blackwell; 2017. [Google Scholar]
  85. Salas E, Shuffler ML, Thayer AL, Bedwell WL, Lazzara EH. Understanding and Improving Teamwork in Organizations: A Scientifically Based Practical Guide. Human Resource Management. 2015;54:599–622. [Google Scholar]
  86. Salazar MR, Lant TK, Fiore SM, Salas E. Facilitating innovation in diverse science teams through integrative capacity. Small Group Research. 2012;43:527–558. [Google Scholar]
  87. Sampson RJ, Raudenbush SW, Earls F. Neighborhoods and violent crime: A multilevel study of collective efficacy. Science. 1997;277(5328):918–924. doi: 10.1126/science.277.5328.918. [DOI] [PubMed] [Google Scholar]
  88. Shepard PM, Idehen Al, Casado J, Freeman E, Horowitz C, Seifer S, Strelnick H. Amplifying the community voice in community-academic partnerships: A summary of and commentary on a thematic issue. Prog Comm Hlth Partshps. 2013;7:231–233. doi: 10.1353/cpr.2013.0036. [DOI] [PMC free article] [PubMed] [Google Scholar]
  89. Selker HP, Wilkins CH. From community engagement, to community-engaged research, to broadly engaged team science. J Clinical and Translational Science. 2017;1:5–6. doi: 10.1017/cts.2017.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  90. Shrum W, Genuth J, Chompalov I. Structures of Scientific Collaboration. Cambridge, MA: MIT Press; 2007. [Google Scholar]
  91. Spring B, Hall KL, Moller AC, Falk-Krzesinski HJ. An emerging science and praxis for research and practice teams. Translational Behavioral Medicine. 2012;2:411–414. doi: 10.1007/s13142-012-0180-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  92. Stilgoe J, Lock SJ, Wilsdon J. Why should we promote public engagement with science? Public Understanding of Science. 2014;23(1):4–15. doi: 10.1177/0963662513518154. [DOI] [PMC free article] [PubMed] [Google Scholar]
  93. Stark D. The Sense of Dissonance. Princeton, NJ: Princeton University Press; 2009. [Google Scholar]
  94. Stokols D, Fuqua J, Gress J, Trochim W. Evaluating transdisciplinary science. Nicotine & Tobacco Research. 2003;5(Supplement 1):S21–S39. doi: 10.1080/14622200310001625555. [DOI] [PubMed] [Google Scholar]
  95. Tebes JK, Kraemer DT. Quantitative and qualitative knowing in mutual support research: Some lessons from the recent history of scientific psychology. American Journal of Community Psychology. 1991;19:739–756. [Google Scholar]
  96. Tebes JK. External validity and scientific psychology. Amer Psychologist. 2000;55:1508–1509. [Google Scholar]
  97. Tebes JK. Community science, philosophy of science, and the practice of research. American Journal of Community Psychology. 2005;35:213–235. doi: 10.1007/s10464-005-3399-x. [DOI] [PubMed] [Google Scholar]
  98. Tebes JK. Philosophical foundations of mixed methods research: Implications for research practice. In: Jason L, Glenwick D, editors. Innovative methodological approaches to community-based research. Washington, DC: APA Books; 2012. pp. 13–31. [Google Scholar]
  99. Tebes JK, Thai ND, Matlin SL. Twenty-first century science as a relational process and a place for community psychology. Amer J Community Psychol. 2014;53:475–90. doi: 10.1007/s10464-014-9625-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  100. Tebes JK. Foundations for a philosophy of science of community psychology. In: Bond MA, Keys CB, Seranno-Garcia I, editors. Handbook of community psychology, Volume II: Methods of community psychology. Washington, DC: APA; 2017. pp. 21–40. [Google Scholar]
  101. Tebes JK. Team science, justice, and the co-production of knowledge. American Journal of Community Psychology. 2018 doi: 10.1002/ajcp.12252. in press. [DOI] [PubMed] [Google Scholar]
  102. Terry SF, Leshner AI. Assessing NIH’s big idea. Science Transl Medicine. 2013;5:1–2. doi: 10.1126/scitranslmed.3006851. [DOI] [PubMed] [Google Scholar]
  103. Thorén H, Breian L. Stepping stone or stumbling block? Mode 2 knowledge production in sustainability science. Studies in History & Philosophy of Science. 2016;56:71–81. doi: 10.1016/j.shpsc.2015.11.002. [DOI] [PubMed] [Google Scholar]
  104. Trickett EJ, Beehler S, Deutsch C, Trimble JE. Advancing the science of community-level interventions. American Journal of Public Health. 2011;101(8):1410–1419. doi: 10.2105/AJPH.2010.300113. [DOI] [PMC free article] [PubMed] [Google Scholar]
  105. Trickett EJ, Espino SL. Collaboration and social inquiry. American Journal of Community Psychology. 2004;34(1–2):1–69. doi: 10.1023/b:ajcp.0000040146.32749.7d. [DOI] [PubMed] [Google Scholar]
  106. Walsh WB, Smith GL, London M. Developing an interface between engineering and the social sciences: An interdisciplinary team approach to solving societal problems. American Psychologist. 1975;30(11):1067. [Google Scholar]
  107. Wallerstein N, Duran B. The theoretical, historical, and practice roots of CBPR. In: Minkler M, Wallerstein N, editors. Community-based Participatory Research for Health: From Process to Outcomes. San Francisco: Jossey Bass; 2011. pp. 25–45. [Google Scholar]
  108. Wells KB, Jones L, Chung B, Ramos E. Community-partnered cluster-randomized comparative effectiveness trial of community engagement and planning or resources for services to address depression disparities. J of Gen Int Med. 2013;28:1268–1278. doi: 10.1007/s11606-013-2484-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  109. Whyte WF, Greenwood DJ, Lazes P. Participatory action research: Through practice to science in social research. American Behavioral Scientist. 1989;32(5):513–551. [Google Scholar]
  110. Wilson P, Mathie E, Keenan J, McNeilly E, Goodman C, Howe A, … Peckham S. ReseArch with Patient and Public invOlvement: a RealisT evaluation – the RAPPORT study. Health Services Delivery Research. 2015;3(38):1–175. [PubMed] [Google Scholar]
  111. Wuchty S, Jones BF, Uzzi B. The increasing dominance of teams in production of knowledge. Science. 2007;316:1036–1038. doi: 10.1126/science.1136099. [DOI] [PubMed] [Google Scholar]
  112. Zajac S, Gregory ME, Bedwell WL, Kramer WS, Salas E. The cognitive underpinnings of adaptive team performance in ill-defined task situations: A closer look at team cognition. Organizational Psychology Review. 2014;4(1):49–73. [Google Scholar]

RESOURCES