A framework for coordination center responsibilities and performance in a multi-site, transdisciplinary public health research initiative

Abstract

Funding bodies in the USA and abroad are increasingly investing in transdisciplinary research, i.e. research conducted by investigators from different disciplines who work to create novel theoretical, methodological, and translational innovations to address a common problem. Transdisciplinary research presents additional logistical and administrative burdens, yet few models of successful coordination have been proposed or substantiated, nor have performance outcomes or indicators been established for transdisciplinary coordination. This work uses the NIH-funded Transdisciplinary Research on Energetics and Cancer (TREC) Centers Initiative as a case study to put forward a working framework of transdisciplinary research coordination center (CC) responsibilities and performance indicators. We developed the framework using a sequential mixed methods study design. TREC CC functions and performance indicators were identified through key-informant interviews with CC personnel and then refined through a survey of TREC research center and funding agency investigators and staff. The framework included 23 TREC CC responsibilities that comprised five functional areas: leadership and administration, data and bioinformatics, developmental projects, education and training, and integration and self-evaluation, 10 performance outcomes and 26 corresponding performance indicators for transdisciplinary CCs. Findings revealed high levels of agreement about CC responsibilities and performance metrics across CC members and constituents. The success of multi-site, transdisciplinary research depends on effective research coordination. The functions identified in this study help clarify essential responsibilities of transdisciplinary research CCs and indicators of success of those transdisciplinary CCs. Our framework adds new dimensions to the notion of identifying and assessing CC activities that may foster transdisciplinarity.

1. Introduction

Funding agencies are increasingly prioritizing collaborative research led by transdisciplinary teams to more holistically address complex societal problems (Corley, Boardman and Bozeman 2006; Croyle 2008; Jones, Wuchty and Uzzi 2008; Adler and Stewart 2010; Falk-Krzesinski et al. 2011). Transdisciplinary research approaches endeavor to mix concepts from two or more disciplines with the overall goal of creating new knowledge to solve problems relevant to society (Rosenfield 1992; Balsiger 2004; Klein 2013; Stokols, Hall and Vogel 2013). Transdisciplinary teams involve scientists with a range of disciplinary expertise who work across operational, disciplinary, and institutional boundaries to address multifactorial, multi-level societal problems (Rosenfield 1992; Balsiger 2004; Klein 2008). Worldwide, national agencies have funded transdisciplinary consortia to address multi-dimensional public health challenges such as tobacco use (Abrams et al. 2003), diet and physical activity (Patterson et al. 2013; Czajkowski et al. 2015), health disparities (Warnecke et al. 2008), and their relationships to chronic disease; transdisciplinary teams have also been mobilized to address social determinants of health, such as poverty (Pattanayak et al. 2006), housing (Lawrence 2004), sustainability (Pohl 2008), agriculture (Francis et al. 2008), climate change (Serrao-Neumann et al. 2015), and a variety of environmental issues (Pohl 2005; Di Iacovo et al. 2016).

Transdisciplinary research requires coordination and collaboration between individuals within teams, teams within research centers, and research centers within consortia. Decisions about transdisciplinary research coordination, that is, research coordination that spans traditional disciplinary and organizational boundaries, requires attention to institutional, administrative, operational, and individual challenges that can limit an initiative’s success (Balsiger 2004; Klein 2008; Rosenfield 1992). Collaborative research, generally, also requires administrative expertise to navigate organizational and institutional policies and procedures (Vogel et al. 2014), communicate across partnerships (Younglove-Webb et al. 1999; Gray 2008), and determine how to share project resources (Wardenaar 2015). Transdisciplinary research centers, specifically, require unique scientific expertise to promote integration of theories, methods, frameworks, and findings across both disciplines and projects (Stokols et al. 2003; Wickson, Carew and Russell 2006; Russell, Wickson and Carew 2008; Carew and Wickson 2010; Stokols, Hall and Vogel 2013; Vogel et al. 2014). Transdisciplinary researchers must engage with unfamiliar conceptual and scientific concepts (Corley et al. 2006; Boardman and Corley 2008; Vogel et al. 2014), understand how to cultivate a transdisciplinary approach, and communicate across different disciplinary languages and methods (Stokols et al. 2003, 2008; Carew and Wickson 2010; Hadorn, Pohl and Bammer 2010; Vogel et al. 2014). If not managed well, additional responsibilities and competing priorities shouldered by investigators in collaborative, transdisciplinary initiatives can contribute to role strain, hinder scientific productivity, and limit the success of collaborations (Bozeman and Boardman 2003; Boardman and Bozeman 2007).

Mobilizing third party entities, such as dedicated coordination centers (CCs), has become a common approach to shouldering challenges inherent to collaborative research, particularly in the health sciences (National Institutes of Health & Food and Drug Administration 2013; Patterson et al. 2013). The impetus for funding dedicated CCs in the context of transdisciplinary team science was based on the hypothesis that they would address some of the unique challenges identified in early evaluations of these initiatives. Specifically, the presence of a designated CC has been shown to increase communication, collaboration, social ties, and innovation (Cummings and Kiesler 2005; Corley et al. 2006; Boardman and Corley 2008; Hall et al. 2008a,b; Boardman, Gray and Rivers 2012; Gehlert et al. 2015). Investigations of research coordination have also found that certain center-level characteristics—such as affiliation with industry and number of disciplines represented—influence collaborative behavior (Cummings and Kiesler 2005; Hall et al. 2008a,b; Gehlert et al. 2015) and that affiliation with other institutions (i.e. research centers who share membership in a consortia) does not immediately translate to successful coordination across geographically dispersed investigators (Boardman and Corley 2008). Other work has indicated that multi-institutional research collaboration requires either significant epistemic development of involved disciplines (i.e. substantial comprehension of how knowledge is created in a given discipline) or a well-defined organizational structure (Corley et al. 2006), including standardized procedures and a clear system of oversight, roles, and responsibilities (Bangdiwala et al. 2003).

Prior research has largely focused on management and coordination within research initiatives without a dedicated CC tasked with promoting coordination. These studies have described collaborative processes but have not identified the performance outcomes or indicators of coordination success. Only one study to our knowledge has proposed a framework describing specific activities and interventions to promote collaboration by an independent scientific CC. In that study, Hessels developed a theoretical framework to characterize eight coordination processes hypothesized to increase performance of a Dutch chemical technology coordination task force, such as facilitating physical proximity of investigators and ongoing meetings, collaborating to propose research programs, and sharing manuscripts across collaborative partners prior to submission (Hessels 2013). When disciplinary norms differ among investigators, however, greater attention to organizational structures are likely needed to facilitate successful collaboration (Bozeman and Boardman 2003; Corley et al. 2006). CCs in the health sciences, generally, are responsible for developing communications infrastructure, harmonizing and managing data, and coordinating operations and administrative aspects (Blumenstein et al. 1995; Bangdiwala et al. 2003; Rolland, Smith and Potter 2011). Whether these functions are sufficient for coordinating transdisciplinary research remains an open question.

Interest in evaluating cross-disciplinary research has grown considerably in the last two decades. Prior work has identified outcomes that indicate the success of a transdisciplinary approach for addressing public health problems, which include increased collaboration and communication (Stokols et al. 2003; Hall et al. 2008a,b); development of new transdisciplinary models and methods (Stokols et al. 2003; Mâsse et al. 2008); translation of findings across levels of science and into public policy (Kobus and Mermelstein 2009; Stokols et al. 2010; Gehlert 2013; Czajkowski et al. 2016); training researchers in the transdisciplinary approach (Mitrany and Stokols, 2005; Stokols et al. 2010); and increased transdisciplinary scientific productivity (e.g. new transdisciplinary publications and transdisciplinary grants) (Hall et al. 2012). Despite the growth in cross-disciplinary research conduct and evaluation, few efforts have investigated the key functions or performance indicators of a dedicated CC tasked with promoting transdisciplinary coordination and collaboration. A conceptual framework of coordination modes in public funding systems proposes that common, socially agreed-upon processes and measures are critical to successful coordination (Lepori 2011). Yet, best practices for coordinating multi-site transdisciplinary research—a mediating factor in transdisciplinary research success—specifically, are unknown. The limited prior work this area presents a challenge for determining whether and how the movement toward funding dedicated CCs is advancing public investments in transdisciplinary science. A better understanding of overarching transdisciplinary CC functions and corresponding roles and responsibilities is needed, as are outcomes and indicators useful for evaluating CC performance. Such information could improve the organization and management of future transdisciplinary research initiatives, ultimately advancing important public health goals.

The purpose of this article is to present a framework of transdisciplinary research coordination using the Transdisciplinary Research on Energetics and Cancer (TREC) initiative—a transdisciplinary, multi-site, public health initiative in the United States—as a case study. The framework was developed using a sequential mixed methods study design wherein CC functions, roles and responsibilities, and performance outcomes were identified and operationalized through key-informant interviews with internal CC members and then refined using a web-based survey of CC constituents. We build on Hessels’ work by characterizing coordination activities for a transdisciplinary, public health initiative specifically, and add to the literature a set of CC performance outcomes agreed upon by the TREC membership. Finally, we propose a series of indicators to measure CC performance and help gauge the value added by this approached to coordinating multi-site, transdisciplinary research.

1.1. The TREC initiative

In 2005, the National Cancer Institute (NCI) established the first center grant mechanism in nutrition, energetics, and physical activity, referred to as the TREC initiative (Patterson et al. 2013; Gehlert et al. 2014). The TREC initiative mission was to accelerate progress toward reducing cancer incidence, morbidity, and mortality associated with obesity, low levels of physical activity, and poor diet through the work of transdisciplinary teams of scientists. Four research centers (Case Western Reserve University; Fred Hutchinson Cancer Research Center; University of Minnesota; University of Southern California) were funded during first iteration of TREC (2005–10). The Fred Hutch also served as the CC for TREC I. The TREC CC’s overall goal was to advance the TREC mission; coordinate training opportunities for new and established scientists to carry out integrative research on energetics, energy balance, and its consequences; and, with the NCI, develop an overall evaluation process for the TREC program.

During the second TREC grant cycle (2011–16), NCI funded four different TREC research centers (University of California, San Diego; Harvard University; University of Pennsylvania; and Washington University in St. Louis) along with a dedicated, geographically distinct CC (Fred Hutchinson Cancer Research Center). Based on the TREC I experience, the TREC II CC was designed to foster further transdisciplinary collaboration and success. The TREC II CC aimed to provide both scientific and organizational leadership to the TREC initiative, including facilitating scientific collaboration across TREC Centers. To this end, the CC was organized into five cores to reflect the need for further disciplinary integration and collaboration: (1) leadership and administration; (2) developmental projects; (3) data and bioinformatics; (4) education and training; and (5) integration and self-evaluation. Each core was led by a senior researcher and supported by project staff with various research and organizational expertise. Table 1 summarizes TREC CC core goals. The unique structure of this large public health initiative thus provides an opportunity to better understand the key functions of a CC in a multi-site transdisciplinary research initiative.

Table 1.

TREC Coordination Center (CC) cores and goals

CC core	CC core goal
Leadership and Administration	Facilitate transdisciplinary research through scientific leadership and organizational support with emphasis on efficient communication, coordination efforts, and expanded scientific collaboration across multiple research institutions; Create opportunities to disseminate results across multiple venues; Facilitate contacts between TREC awardees and NCI professional staff to allow for efficient interactions, consultations, and oversight functions
Developmental Projects	Coordinate the collaborative research proposed and conducted by TREC awardees during the 5-year funding cycle
Data and Bioinformatics	Create and manage relevant logistical infrastructure, including research data management and bioinformatics to support the TREC Research Centers
Education and Training	Create significant new opportunities for transdisciplinary training of scientists at every stage in their careers in the area of energetics and cancer.
Integration and Self-Evaluation	Facilitate integration and evaluation of TREC in collaboration with NCI and TREC Research Centers

2. Methods

A case study approach was chosen to allow an in-depth, multi-faceted exploration of the TREC CC (Crowe et al. 2011); specifically, a sequential, mixed-method case study approach was undertaken to first, gather perspectives of all TREC CC members about coordination of the multi-site transdisciplinary, initiative and second, to generalize the findings to a larger sample of TREC constituents. This design is appropriate when guiding frameworks are lacking (Creswell and Plano Clark 2011).

2.1. Qualitative phase

The research team developed a qualitative, semi-structured interview guide based on (1) a document review of the Request for Applications (RFA) and the TREC CC grant application; (2) a review of existing literature on transdisciplinary team science and research coordination and (3) prior qualitative work on the nature of transdisciplinarity conducted as part of internal TREC evaluations. One-on-one interviews allowed key informants to discuss how their specific roles and responsibilities contributed to the function of the TREC CC as well as their expectations of how the CC supported multi-site transdisciplinary research in the TREC initiative. Interview questions were designed to explore four overarching domains: (1) informant-specific roles within the TREC CC; (2) how the CC fostered cross-site transdisciplinary research; (3) perceived challenges of coordinating a multi-site, transdisciplinary initiative; and (4) performance outcomes and measures for CC success.

All TREC CC leadership and staff who had been working on the TREC project for >6 months were invited via email to participate in the study. Respondents received a briefing on the interview process and signed an informed consent document prior to beginning the interview. Interviews were conducted in a private office or by phone, were audio-recorded, and lasted approximately 60 minutes (range 32–78 minutes). Upon conclusion of the interview, each respondent received a $20 gift card for his/her time.

Interviews were transcribed professionally. Two members of the study team listened to audio recordings while reading transcripts to check transcript accuracy. Transcripts were then deidentified (i.e. names removed) and uploaded into Atlas.ti (Version 7, Berlin, Germany) a qualitative data analysis software program. Two members of the study team reviewed all transcripts and collaboratively developed a start list of codes and corresponding definitions representing CC member roles and responsibilities described by informants. They applied codes to transcripts and refined the code-book, adding, removing, or revising codes and definitions as needed. The coders applied a constant comparison analysis approach, in which concepts related to distinct CC roles were identified by iteratively comparing data from one interview to previous and subsequent interview data (Miles and Huberman 1994; Glaser and Strauss 2006; Bernard 2011). Emergent concepts were then clustered into thematic areas describing overarching functions of the TREC CC. These functions closely mapped to the original TREC CC Cores and were thus given names that corresponded to the cores. The study team regularly met to discuss the coding structure and reach consensus on emergent concepts and themes. Upon completion of thematic analysis, the study team presented findings to TREC CC Investigator and Operations teams for discussion of identified functions and the day-to-day roles and responsibilities they reflected. Consensus built during these debriefings was used to generate a working framework showing CC functional areas, CC roles and responsibilities, and CC performance outcomes contributing to TREC’s success at achieving transdisciplinary outcomes. Individual participants provided approval for their respective selected quotes to be used in reports and presentations.

2.2. Quantitative phase

Results from the qualitative phase informed development of a web-based survey designed to corroborate and further refine the framework. The survey questions asked CC constituents to rate how strongly they agreed with statements about CC roles and responsibilities, CC performance outcomes, and the CC’s contribution to facilitating transdisciplinary success on 5-point Likert scales from ‘strongly agree’ to ‘strongly disagree’. The survey also included questions about respondents’ role within TREC and basic demographics and provided free-text options for respondents to elaborate if needed. TREC CC leadership and staff who had participated in qualitative debriefing reviewed survey questions for clarity and completeness. The final survey was hosted on Survey Monkey. A recruitment letter containing a link to the final survey was emailed to TREC research center investigators and NCI program staff. Respondents provided informed consent and were offered the opportunity to be entered into a draw to receive one of three $20 gift cards upon survey completion.

Survey data was uploaded into SPSS, Version 25 for analysis. Individuals’ responses to questions about CC roles and responsibilities and CC performance outcomes were categorized as ‘agree’ (1 or 2), ‘disagree’ (4 or 5), or ‘neutral’ (3) and summarized across the sample. To assess possible variation in agreement between types of constituents, differences in survey responses between TREC research center investigators and NCI program staff were examined using Fisher’s exact tests and considered significant at P ≤ 0.05. Finally, the study team generated a list of measurable indicators intended to objectively measure CC success in coordinating transdisciplinary research. All study materials were reviewed and approved by the Fred Hutchinson Cancer Research Center Internal Review Board.

3. Results

All invited TREC CC staff and investigators agreed to participate in key informant interviews, which occurred between October and November 2014 (n = 11, 100% response rate). The working framework included 23 distinct TREC CC roles and responsibilities falling within five overarching functional areas. The domains and relationships in the working framework served as the basis for the quantitative survey. TREC research center investigators (n = 108) and TREC-affiliated NCI program staff (n = 6) were invited to complete the survey, of which 47 research center investigators (44%) and four NCI staff members (67%) responded, for an overall response rate of 45%. The final framework, depicted in Figure 1, illustrates the CC functional areas, roles and responsibilities, and performance outcomes important for facilitating TREC initiative success. Table 2 presents the proportion of survey respondents who agreed that specific CC roles and responsibilities were essential to the CC’s functioning in each of five overarching CC functional areas. Below, we present further qualitative and quantitative data to illustrate the framework presented in Figure 1.

Figure 1.

Framework of functional areas, responsibilities, and performance outcomes for a multi-site, transdisciplinary coordination center. Items listed in bold font indicate areas unique to CCs for a transdisciplinary initiative.

Table 2.

TREC constituents’ agreement with roles and responsibilities supporting CC functioning

CC functional areaa	Corresponding CC member roles and responsibilities	Agree n (%)
Leadership and administration	Provide infrastructure for between-site collaboration	54 (98.2)
	Plan scientific meetings	50 (90.9)
	Arrange conference calls	48 (87.3)
	Coordinate webinars	48 (87.3)
	Manage website	47 (87.0)
	Manage listserv	47 (85.5)
	Serve as a liaison with funders	41 (74.6)
	Respond to site needs and problem-solving	39 (70.9)
Data and bioinformatics	Assist with data management	41 (75.9)
	Assist with data analysis	27 (49.1)
Developmental projects	Coordinate pilot project RFAs	48 (87.3)
	Lead development of pilot project RFAs	42 (76.4)
Education and training	Offer training on how to integrate disciplinesa	43 (78.2)
	Provide guidance on integrated disciplinary approachesa	38 (69.1)
	Offer skills training	35 (63.6)
	Provide scientific expertise in initiative content areas	33 (60.0)
	Provide scientific mentorship	32 (59.3)
	Offer discipline-specific training	26 (48.2)
Integration and self-evaluation	Provide a broad perspective of initiative activities	44 (80.0)
	Assist sites with gauging transdisciplinary progressa	38 (69.1)
	Suggest midcourse corrections	38 (69.1)
	Develop transdisciplinary evaluation metricsa	36 (65.5)

Notes: Rows may not sum to 58 due to missing data. Responses do not differ between TREC investigators and NCI program staff based on Fisher’s exact tests.

^aFunctional areas correspond to TREC CC cores.

3.1. Leadership and administration

In qualitative interviews, TREC CC members indicated that a CC in a transdisciplinary initiative had a larger role in developing communications infrastructure, coordinating operations, and managing administrative aspects than in a traditional research initiative. They described the CC’s unique role in connecting investigators across the initiative, both in-person and through electronic communication, through activities such as face-to-face scientific meeting planning and cross-center webinars. One CC member summarized this role, saying:

‘I think the coordination center contributes that sense of what is transdisciplinary instead of what is multidisciplinary, or interdisciplinary, [which] can contribute to the scientists’ current understanding of what it means to have this type of approach’

Another interviewee described specific CC activities that responded to investigator needs while also promoting collaboration across the initiative:

‘It might be that people are not familiar with what other centers are doing, and I think that’s why we started with the webinars. So they can get in their mind, “Oh, well, so-and-so is doing a project that’s somewhat similar to what I’m doing. I wonder if we should collaborate.”’

The majority of survey participants agreed that CC roles and responsibilities with respect to leadership and administration functionality included providing infrastructure for between-site collaboration (98%); planning scientific meetings (91%); and coordinating webinars, managing the initiative website, and arranging conference calls (87%).

3.2. Data and bioinformatics

In qualitative interviews, TREC CC members asserted that assisting with data management and analysis were well-established CC roles and responsibilities. However, they noted that given the structure of TREC, in which individual TREC centers had developed study protocols prior to beginning the collaboration, the CC’s data management and harmonization capabilities were not fully utilized. This CC member explained:

‘One way that we thought of to maybe help foster collaboration was through data harmonization projects…[In] other multi-site research, the CC is specifically involved in a lot of data handling. […] There’s many systems involved with running those multi-site biomarker validation studies…But it was never said that for the [TREC] CC’, “There will be multi-site studies in TREC and they’re funded, and the CC will be the one coordinating and capturing and analyzing the data.”’

Although 76% of TREC investigators and NCI program staff agreed the CC should assist with data management, less than half of survey respondents agreed that assisting with data analysis was important to CC functioning.

3.3. Developmental projects

Each TREC Research Center and the TREC CC were required to allocate annual funding to support year-long developmental research projects that aimed to advance cross-site collaborative transdisciplinary research. One participant asserted, ‘The cross-center development projects are where we have the biggest potential impact on what’s happening in the science of the Centers’. Another participant emphasized that part of achieving that impact is realized through CC members who…

‘really try to get everyone’s perspective and try to make sure [TREC members] feel heard and that their input is taken into consideration. This year, for example, [development projects] are targeting early career investigators [for funding]. The TREC membership has prioritized fostering the next generation of transdisciplinary researchers’.

Over three quarters (87%) of survey respondents agreed that coordinating these RFAs was essential to CC functioning. Some survey respondents added in free-text responses that facilitating disciplinary integration throughout the developmental projects was an activity they perceived as unique to a transdisciplinary CC, specifically.

3.4. Education and training

The TREC developmental research projects represented one way the CC supported training and education. In qualitative interviews, some CC members saw their role as providing disciplinary-specific training, something with which less than half (48%) of survey respondents agreed was essential to CC functioning. CC members expressed that they had a unique opportunity to provide training in disciplinary integration, and, as this respondent explained, that the CC’s role was:

‘…to help understand the language and approaches of different disciplines. What does it mean when you hear terminology of a discipline and you have no idea what that means? How do you understand that and integrate it? How do you approach people whose research might sound interesting, and you’d like to incorporate it, but you don’t even know how to talk to them?’

CC members described the CC having a role in provide training for investigators at all stages of their careers, through workshops at TREC Scientific Meetings, working group calls, and webinars hosted by the CC throughout the year. A CC investigator illustrated this point:

‘Each TREC Center has different research questions and studies they’re doing with different scopes related to energy balance. [The CC] has a great opportunity to provide training and education that goes above and beyond the projects by providing more depth and breadth to the topic of energy balance in cancer to all TREC members’.

The largest proportion of TREC investigators and NCI program staff (78%) agreed that ‘offering training on how to integrate disciplines’ was essential to the CC’s education and training functions.

3.5. Integration and self-evaluation

Interviewees described that beyond measures of productivity, evaluating transdisciplinary processes and outcomes were CC roles that they perceived as unique to multi-site, transdisciplinary research. Members of the TREC CC Evaluation Core noted that in addition to providing a broad perspective of initiative activities, their role was to,

‘…try to understand what each site is doing related to transdisciplinary research, so over time we’ve developed some outcome metrics for what transdisciplinary research means. And we analyze those data and figure out how to give feedback in a constructive way to the individual sites’.

The TREC initiative also benefitted from a biostatistician at the CC who evaluated the basic measures of transdisciplinary research—the social ties between investigators within and across the center. Survey respondents largely agreed with the evaluation roles and responsibilities identified by TREC CC members as supporting CC functioning.

3.6. CC performance outcomes and indicators

Ten performance outcomes were identified based on qualitative data: (1) Facilitated collaboration between participating sites; (2) Decreased administrative burden to participating sites; (3) Facilitated transdisciplinarity (disciplinary integration) within sites’ research projects; (4) Used initiative resources effectively; (5) Decreased cost burden to participating sites; (6) Increased transdisciplinary scientific productivity at participating sites; (7) Increased efficiency of research conducted at participating sites; (8) Shaped transdisciplinary science at participating sites; (9) Facilitated implementation of research conducted at participating sites; and (10) Facilitated collaboration within participating sites. Table 3 gives the proportion of survey respondents who endorsed each CC performance outcome as relevant to gauge CC success in a multi-site transdisciplinary initiative. CC performance outcomes included items related to general CC administrative functioning as well as specific to coordinating transdisciplinary public health research. The three performance outcomes that the most survey respondents agreed reflected CC success were whether the CC: facilitated collaboration between participating sites (93%); decreased administrative burden to participating sites (85%); and facilitated transdisciplinarity within site’s research projects (80%). Of the ten performance outcomes identified, five were specific to collaborative transdisciplinary research coordination, which are denoted with an asterisk in Tables 3 and 4 (Facilitated collaboration within and between participating sites; facilitated transdisciplinary integration within site’s research projects; shaped transdisciplinary science at participating sites; increased transdisciplinary scientific productivity at participating sites).

Table 3.

TREC constituents’ agreement with CC performance outcomes (n = 47)

CC performance outcome	Agree n (%)
Facilitated collaboration between participating sitesa	38 (92.7)
Decreased administrative burden to participating sites	34 (85.0)
Facilitated transdisciplinarity within site’s research projectsa	32 (80.0)
Used initiative resources effectively	30 (76.9)
Decreased cost burden to participating sites	31 (75.6)
Increased transdisciplinary scientific productivity at participating sitesa	30 (73.2)
Increased efficiency of research conduct at participating sites	28 (68.3)
Shaped transdisciplinary science at participating sitesa	25 (62.5)
Facilitated implementation of research conducted at participating sites	24 (60.0)
Facilitated collaboration within participating sitea	24 (60.0)

Notes: Rows may not sum to 47 due to missing data. Responses do not differ between TREC investigators and NCI program staff based Fisher’s exact tests.

^aItems specific to collaborative transdisciplinary research coordination.

Table 4.

Performance indicators to evaluate CC facilitation of transdisciplinary success

Performance outcome	Performance indicator
Facilitated collaboration within and between participating sitesa	Number of infrastructure types for collaboration made available to initiative members (e.g. web-based, in-person, listserv, funding) Network ties demonstrated across initiative over time Proportion of initiative members who participated in CC-led training activities, webinars, scientific meetings, etc. Number of initiative research sites represented on author teams Number of initiative research sites represented on new grant proposals
Facilitated transdisciplinary integration within site’s research projectsa	Number of CC-led training opportunities that present and promote the science of collaboration Number of CC-led training opportunities to develop integrated theoretical frameworks Number of CC-led training opportunities to develop and/or test novel statistical models Number of CC-led training opportunities to develop and/or test novel intervention models Number of disciplines represented at each CC led- meeting, webinar, training Number and type of assessments conducted throughout the initiative; reports provided back to participating sites
Shaped transdisciplinary science at participating sitesa	Proportion of initiative of investigators that utilized training or expertise provided by the CC to develop or test novel intervention models in their transdisciplinary projects Proportion of initiative of investigators that applied new, disciplinarily-integrated theoretical frameworks to a transdisciplinary project Proportion of initiative members that utilized CC resources (e.g. funding) to participate in training outside of their primary discipline
Increased transdisciplinary scientific productivity at participating sites;a	Number of transdisciplinary publications listing CC members or CC grant number in acknowledgements or as co-authors Proportion of initiative investigators that used initiative data to apply for new grants
Facilitated implementation of research conducted at participating sites	Number of projects fully funded through CC-led RFA process for developmental pilot project awards Number of participating sites represented on initiative’s developmental pilot project awards Number of disciplines represented on new grant proposals (including initiative pilot awards) using initiative data
Decreased administrative burden;	Time between initiative member request and resolution Conference line(s) made available to initiative members
Decreased individual sites’ cost burden;	Central website created and utilized; number of website hits Infrastructure for data sharing developed
Used initiative resources effectively;	Proportion of initiative members that utilized CC data management and analysis expertise Proportion of initiative members that utilized communication infrastructure
Increased efficiency of participating sites	Initiative member satisfaction of listserv and webinar content and frequency

^aIndicates outcome specific to CCs for a multi-site transdisciplinary research initiative.

3.7. Performance indicators to measure CC contribution to achieving TREC mission

In qualitative interviews, respondents described the success of the CC would be determined by the success of the TREC initiative. Respondents underscored that the transdisciplinary initiative’s success as a whole was linked to CC performance, as summarized by one participant: ‘Coordination center success versus what the goal of the TREC centers are achieving—those are two different things, but they are related’. Correspondingly, participants said the CC performance outcomes should also reflect the larger goal of achieving the TREC mission. Table 4 presents 26 measurable indicators for the CC performance outcomes identified during qualitative interviews. The performance outcomes that are specific to CCs for a multi-site transdisciplinary initiative are listed first. Performance indicators to measure collaboration within and between research sites include: network ties demonstrated among investigators across the initiative over time, proportion of initiative members who participate in CC-led training activities, webinars, and scientific meetings, and number of initiative research sites represented on author teams. Performance indicators proposed to measure facilitation of transdisciplinary research within site’s research projects serve to record the number of CC-led training opportunities that promote the science of collaboration, develop integrated theoretical frameworks, and develop and test novel statistical and intervention models. In this table, because performance indicators could be used to gauge achievement of more than one outcome, some outcomes are combined.

3.8. Framework summary

The framework illustrated in Figure 1 identifies 23 CC roles and responsibilities in five functional areas that TREC members agreed are key to CC success (i.e. leadership and administration; developmental projects; data and bioinformatics; education and training; and integration and self-evaluation). It operationalizes CC success as eight specific performance outcomes, half of which are specific to transdisciplinary research coordination (i.e. facilitated collaboration, facilitated disciplinary integration, facilitated transdisciplinary science conduct, and facilitated transdisciplinary scientific productivity). Finally, the framework proposes that CC performance outcomes facilitate overall initiative success.

4. Discussion and conclusions

In this exploratory study, we aimed to develop a framework of transdisciplinary research coordination and performance in the TREC initiative based on the perspective of leadership and staff who implemented the TREC II CC’s day-to-day activities. We then confirmed these observations with investigators who utilized TREC II CC services. Qualitative results indicated that CC leadership and staff have distinct responsibilities which we categorized into five major CC functional areas that corresponded to the TREC CC cores. Based on qualitative data, we specified 10 performance outcomes TREC members agreed were appropriate for CCs of transdisciplinary initiatives. We then proposed a set of performance indicators to evaluate achievement of the performance outcomes. Quantitative results demonstrated overwhelming agreement with a framework of multi-site transdisciplinary research coordination grounded in the five essential functions (Figure 1). Two of these functional areas are common to most CCs (leadership and administration, data and bioinformatics) while three are critical and may be unique to multi-site, transdisciplinary research initiatives (developmental projects, education and training, integration and self-evaluation).

Both managing administrative aspects of an initiative and providing data management, harmonization, and statistical support are traditional roles associated with CCs in clinical, and epidemiologic research (Blumenstein et al. 1995; Stevens and Donald 2007; Rolland et al. 2011; Biswas et al. 2012). Not surprisingly, across all functional areas, items in the Leadership and administration functional area (i.e. providing infrastructure for communication and collaboration, planning meetings, responding to site needs) demonstrated the highest levels of agreement among survey respondents. In the TREC initiative, CC staff were responsible for planning semi-annual meetings and coordinating webinars, teleconferences, and the initiative website. These findings are consistent with prior work that emphasizes the a CC’s role in building communications infrastructure for collaboration (Rolland et al. 2011) and specifically, that a CC promote coordination through the development of communication lines and documentation (Bangdiwala et al. 2003). Of the eight coordination processes hypothesized to increase performance of a coordination task force in a chemical technology consortium described by Hessels, five included interventions similar to the roles and responsibilities of the TREC leadership and administration functional area. The interventions highlighted by Hessels included convening regular meetings, creating opportunities for consortium members to meet face-to-face, and coordinating the work of a consortium executive board (Hessels 2013).

Although a majority of TREC investigators agreed that data management was a function of CCs, less than half agreed that CCs should assist with data analysis. This finding may be unique to the design of TREC, in which research centers and the CC responded separately to RFAs without knowledge of with whom or to what degree they would be expected to collaborate. Other CCs for multicenter health initiatives are designed such that a central research question guides the initiative, and individual research centers are held to common data collection, laboratory, and analyses protocols, with a central goal of those CCs being to harmonize data across sites (Blumenstein et al. 1995; Biswas et al. 2012).

The major responsibilities of CCs in consortia are centered around administration and data management (Blumenstein et al. 1995; Rolland et al. 2011; Biswas et al. 2012; Hessels 2013). As a CC for a transdisciplinary public health research initiative, the TREC CC featured additional functions unique to this type of initiative, such as providing training, coordinating developmental projects, and conducting monitoring and evaluation of the initiative. Although there was wide agreement in most items under the Training functional area, less than half of survey respondents agreed that the function of a CC was to offer discipline-specific training. This is in contrast to non-transdisciplinary CCs described in the literature, wherein training is not mentioned as a CC role (Hessels 2013; Wardenaar, de Jong and Hessels 2014), or, if training is provided, its focus is on adherence to specific study protocols and requirements (Blumenstein et al. 1995; Bangdiwala et al. 2003), rather than conceptual goals of integrating disciplines and preparing a new cadre of investigators to conduct transdisciplinary research. To advance toward the TREC goal of disciplinary integration, the TREC CC facilitated training opportunities focused on the transdisciplinary approach. For example, during monthly webinars, investigators presented their ongoing transdisciplinary research; the CC Training Core offered targeted training workshops at face-to-face meetings tailored to the requests of TREC membership and provided funds for investigator exchange experiences. In addition to the training activities implemented by the TREC training core, coordinating developmental pilot projects was a unique role of the TREC CC that provided investigators an opportunity to work toward transdisciplinary success and, ultimately, the TREC mission of collaboration and disciplinary integration (Schmitz et al. 2016). Hessels and colleagues described two similar coordination process, ‘bundling research plans’ and ‘competitive project selection’ in which the consortium coordination task force prioritized funding research studies that promoted a common theme (Hessels 2013). This process was similar to the TREC CC’s leadership in RFA development and prioritization of pilot project funding. Additionally, while the Dutch coordination task force included a process in which a platform was created for collaboration between researchers and industry, the TREC initiative focused on creating collaboration across disciplines, institutions, and in several projects, collaboration within and between health systems.

Evaluation is a role tasked to most CCs. In epidemiologic and clinical consortia, CC evaluation centers around quality improvement for laboratory processes and protocol adherence (Bangdiwala et al. 2003) or evaluations to ensure training met intended goals (Blumenstein et al. 1995). However, multi-site, transdisciplinary collaborations require unique evaluation methods, as their breadth extends beyond individual disciplines and investigators (Stokols et al. 2003; Mâsse et al. 2008; Carew and Wickson 2010). Although many process measures for transdisciplinary evaluation exist, a comprehensive set of outcome measures has not yet been developed. Over two-thirds of survey respondents agreed that a CC in a multi-site, transdisciplinary initiative should develop transdisciplinary evaluation metrics, assist sites with gauging progress toward transdisciplinary research, and suggest midcourse corrections.

Successful coordination relies on clear measurement (Lepori 2011). The proposed set of 26 indicators to measure CC performance outcomes identified and prioritized by the TREC membership relies heavily on bibliometric analyses, an accepted measure to quantify outputs in research settings (Wardenaar et al. 2014). Previous work has proposed bibliometric analyses as an approach to measuring transdisciplinary success, wherein the quality and magnitude of integration of published work and new grants indicates success of a transdisciplinary approach (Mâsse et al. 2008). However, the indicators also rely on data to be collected from investigators in the initiative over time. In later evaluations of the TREC initiative, this approach facilitated evaluation of changes in the size and composition of social networks, an indicator of collaboration within and between participating sites (Gehlert et al. 2015; Sarah Gehlert et al. 2017).

The TREC CC is unique because its roles extended beyond the realms of administration, data management, and statistical analyses to include functional areas of training, developmental project coordination, and evaluation. Within those functional areas, they assumed responsibilities unique to a transdisciplinary CC, such as facilitating disciplinary integration throughout pilot projects, offering training on how to integrate disciplines, and develop transdisciplinary evaluation metrics. When the emphasis of an initiative is on transdisciplinary research, CCs are required to consider and act upon the unique aspects of transdisciplinary work, such as communication, coordination, disciplinary integration, working across and between the translational continuum, and developing new and innovative approaches and solutions to complex public health problems.

4.1. Limitations

Although this study is representative of all TREC CC staff and investigator perspectives. The survey administered to the larger TREC membership resulted in a low response rate (45%). The data presented here do not represent the experiences of all CCs in multi-site, transdisciplinary initiatives. However, the functions identified may serve as a framework for the design and evaluation of future CCs in transdisciplinary initiatives, particularly those that address public health problems. This study was an initial, exploratory assessment of the role a CC plays in transdisciplinary research and was not designed to evaluate causal relationships between research coordination design (e.g. roles and function) and transdisciplinary research outcomes (e.g. multi-level intervention models). Additional qualitative research is needed to better understand the mechanisms by which CC functions may contribute to transdisciplinary research outcomes, which can then be tested in future quantitative work. Moving beyond the single item questions used in this study to develop robust outcome measures will be essential to this work.

5. Conclusions

The success of multi-site, transdisciplinary research depends on communication and coordination (Gray 2008); furthermore, successful coordination coordination relies on shared understanding of what constitutes quality, as well as processes for measurement that are accepted by participating actors (Lepori 2011). This exploratory study represents a step toward identifying the functions and responsibilities required for CCs in multi-site initiative to facilitate transdisciplinary research success and the performance indicators to measure CC success. The CC responsibilities identified in this study could help improve effectiveness of future CCs, including planning for infrastructure and staffing needs. Furthermore, the results of this study can be used to inform markers for coordination success. A number of large projects in the health sciences as well others in climate change, technology, and environmental science now call for a transdisciplinary approach; some have mobilized designated CCs that are directed to help facilitate transdisciplinarity. The field of CCs and intermediary organizations in multi-actor research initiatives could benefit from further exploration of a causal relationship between CC performance and transdisciplinary success. Although the CCs may have some responsibilities that are oriented to facilitate transdisciplinarity, few of the activities have not been empirically linked to the transdisciplinary outcome and furthermore, have rarely been measured. Thus, this research adds new dimensions to the notion of identifying and assessing CC activities that may foster success of a multi-site, transdisciplinary initiative.