Skip to main content
PLOS ONE logoLink to PLOS ONE
. 2022 Nov 29;17(11):e0278043. doi: 10.1371/journal.pone.0278043

Interdisciplinary collaboration from diverse science teams can produce significant outcomes

Alison Specht 1,#, Kevin Crowston 2,*,#
Editor: Sergi Lozano3
PMCID: PMC9707800  PMID: 36445918

Abstract

Scientific teams are increasingly diverse in discipline, international scope and demographics. Diversity has been found to be a driver of innovation but also can be a source of interpersonal friction. Drawing on a mixed-method study of 22 scientific working groups, this paper presents evidence that team diversity has a positive impact on scientific output (i.e., the number of journal papers and citations) through the mediation of the interdisciplinarity of the collaborative process, as evidenced by publishing in and citing more diverse sources. Ironically these factors also seem to be related to lower team member satisfaction and perceived effectiveness, countered by the gender balance of the team. Qualitative data suggests additional factors that facilitate collaboration, such as trust and leadership. Our findings have implications for team design and management, as team diversity seems beneficial, but the process of integration can be difficult and needs management to lead to a productive and innovative process.

Introduction

Complex, global research challenges increasingly require the formation of teams that bring together individuals with diverse skills and perspectives who can work across disciplinary, organisational and national divides [15]. One form for these teams is a working group, a group of researchers from different institutions who are brought together and supported to work over a period of a few years on an (often self-identified) compelling research problem. The outcome of the team work may be to inform policy development, to produce new knowledge and understanding, or the creation of e-infrastructure platforms to support continued advances in science [6]. Such teams should not be mistaken for an established research team given a specific task within their work schedule as part of their employee duties.

Working groups can be assembled in a way that creates several kinds of group diversity. A kind of diversity that is increasingly important to science is disciplinary diversity [7]. In recent years interdisciplinary research has rapidly become the norm [8, 9], that is, research involving scholars from different disciplines collaborating to develop terminology, research approaches, methodologies, or theories that are integrated across multiple disciplines to address broader problems than a single disciplinary approach can address [8]. There is an accepted understanding that interdisciplinary research will foster important ideas beyond the boundaries of a single discipline, even creating new disciplines [7, 1012]. Further, there is some evidence that increasing the interdisciplinarity of the research team will increase the originality and creativity of the outcomes [7, 13, 14]. Certainly an interdisciplinary team is thought likely to be more creative and produce more novel results than conventional research teams due to the variety of disciplines being blended [1520].

In addition to disciplinary variety, other kinds of group diversity have impacts on group process and outcomes. Combinations of participants from different kinds of organisations can result in technology fusion, creating new opportunities for each participant [21, 22]. International collaboration in the environmental sciences has often been required due to the nation-boundary-blindness of organisms and processes. In addition, there are often compositional requirements, such as representative membership, as is the case in most European Union-funded projects, or to comply with societal expectations, such as gender and race representation.

While group diversity can provide the pathway for innovative insights, diverse teams face challenges to collaboration, for example friction, lack of communication and break points, such as those due to differing cultural norms around gender, power, and the preferred level of individual behaviour versus collectivism [23]. Group members in new combinations may find more questions than answers for a particular problem, defeating their purpose. Effective collaboration across disciplinary or national boundaries does not result from simply ‘putting people together in a room and shutting the door’. For diverse groups of people to make sense of the vast range of data and information increasingly available and generate an outcome that is innovative and creative, thoughtful group construction and support is required.

The goal of this paper is to identify key aspects of group composition conducive to productive work around multi-disciplinary, multi-layered problems. Is team diversity positively related to outcomes that combine the team’s diverse backgrounds and is the productivity of such a team enhanced by the level of diversity within it? If so, what aspects of diversity might be most important? Task-related diversity has been found to be positively related to performance, but bio-demographic diversity has not [24]. Does having a certain gender balance help? Does diversity itself create difficulties of communication, and impede cross-fertilisation across sectors and hence output? How might this work if more than one factor occurs at once?

Theory development

As a basis for our analysis, we develop a model of the factors that may predict the success of interdisciplinary science teams through providing the basis for the development of group collective intelligence [20, 25, 26] and the intellectual fusion or, as some would express it, integration, that is desirable to generate transformative outcomes [27, 28]. We are guided by the Input-Process-Output group framework (Fig 1). A version of this input-process-output model was used by Stokols et al. [29] in their examination of ‘transdisciplinary scientific collaboration’. In this section, we explain the model and the specific variables we chose for each aspect of the model.

Fig 1. Research model.

Fig 1

Inputs, mediators (process and emergent states) and outputs. Hypothesis numbers are shown.

In this model, inputs are the attributes that the team members bring to the teamwork. In our setting, we are focusing on the members themselves and their demographic backgrounds. A working group is composed in response to a problem: people are deliberately chosen with relevant skills and expertise to contribute to a solution, while keeping in mind other factors, such as organisation, country, and life situation. These choices create groups with different levels of diversity.

Outputs of a group are the measurable consequences of team function. For research groups the desired outcomes are new knowledge and understanding expressed usually through refereed publications, deposited data in open data repositories, and new, long-term collaborations across disciplinary, organisation and geographical boundaries. Lynch et al. [30], show a range of types of valued outputs from synthesis centers. As noted also by Hackman [31, p. 128], team outputs can extend beyond specific products to include participant satisfaction with their work and their perceptions of the effectiveness of the team.

Between inputs and outputs is the process that mediates the influence of inputs on outputs. The measurable and intended process in our study is that of collaboration across disciplinary boundaries. Interdisciplinary or trans-disciplinary group membership is often used to support cross-fertilisation to “co-design research, co-produce solution-oriented knowledge, and reintegrate the knowledge into strategies for problem-solving and the development of new scientific insights” [32]. The details of these interactions are subtle and difficult to capture in a comparable way across teams. Research has therefore usually adopted proxy measures. In this paper we assess interdisciplinary research output by the diversity of the authors and of the references they cite [33, 34].

Hypothesis development

Much of the research that has addressed the question of the effect of team composition on outputs has examined diversity directly without postulating an intervening process. Publication output, for example, has been positively related not only to disciplinary diversity but also low seniority of teams [35]. Others have found a positive effect of diversity of career stages and gender on publication output, but a negative effect of educational (disciplinary) diversity [36]. Gender heterogeneity in a group has been shown to enhance citation rates [37] and link positively with team performance [38]. The novelty of our approach is that rather than hypothesizing direct impacts of diversity on outputs, we expect outcomes to be moderated by the nature of the research process. We consider that different kinds of diversity may contribute in different ways and so develop a set of hypotheses for each of these relationships.

Inputs to process

We first consider how group diversity (the input) may have an impact on the group process, specifically the interdisciplinarity of the collaboration (the process).

Although men and women share common predictors for collaboration, women have been found to tend to engage in more interdisciplinary research collaborations [39], to have more collaborators than men [40] and they have characteristically more social sensitivity [25, 31]. Indeed, in studies of small groups (7–10 members) the collective intelligence of the group has been positively correlated with the percentage of women in a group [25], as has group effectiveness and innovation, particularly when task intensity is high [40, 41]. This result has been explained by improvements in the emotional intelligence of the group and a reduction in conflicts with the addition of a component of women [38, 42]. We therefore hypothesise that interdisciplinary collaboration will be enhanced by the proportion of female participants (Table 1, H1(a)).

Table 1. Hypotheses posed in this paper.
Hypothesis Proposed relationship
H1 There is a positive relationship between interdisciplinary collaboration and:
(a) proportion of female members
(b) the interdisciplinarity of the group
(c) There is a negative relationship between interdisciplinary collaboration and the diversity of international membership
H2 (a) There is a positive correlation between interdisciplinary collaboration in a group and number of publications.
(b) There is an inverted-U relationship between interdisciplinary collaboration in a group and impact as measured by the median number of citations.
(c) There is a positive correlation between interdisciplinary collaboration in a group and personal satisfaction
(d) There is a positive correlation between interdisciplinary collaboration in a group and perceived effectiveness

Without a range of disciplines present in a group—as assessed by a classification of the group members into their ‘home’ disciplines [2]—the meshing of disciplines could not occur. It has been found that the diversity of citations tends to grow with the diversity of disciplines of the authors [33]. We therefore hypothesise that interdisciplinary collaboration will be enhanced by the interdisciplinarity of the group (Table 1, H1(b)).

Finally, intra-country (regional, state) or international membership may also be required for project performance, depending on the problem being considered or the funding arrangements. In environmental and biodiversity science and management, global participation in some shape or form is essential not the least because organisms do not abide by human political boundaries, but also (i) to get a comprehensive grasp on a problem, (ii) to acquire the requisite data, and (iii) to access appropriate expertise and data. Indeed, international collaboration has become the norm rather than the exception in modern science [3, 34]. Group membership may be mandated or perceived as desirable for funding reasons (e.g. funding from the European Union or the Belmont Forum) along with a perception that the results will be enhanced by such inclusivity.

While there may be a clear need or desirability for international collaboration, there are clear challenges to successful collaboration. The physical dispersion of group members creates logistic problems due to travel approvals and cost, and if collaboration is to be remote, time-zone challenges can be prohibitive [1, 16]. Participants from different countries with different languages and ways of working can have communication and cultural challenges which take time to overcome, reducing the productivity of the group. This can result in less novel outcomes, and high transaction costs and communication barriers; for instance, data suggest that the more nations involved in a project, the fewer publications and the more conventional the research [3]. We therefore hypothesise that interdisciplinary collaboration will be negatively affected by the diversity of international membership (Table 1, H1(c)).

Process to outputs

We next consider how the process affects the outputs, in the first instance, the number of publications produced. Collaboration in general has been positively related to higher output [43, 44]. Researchers engaging in interdisciplinary research have been found to be less productive but have higher impact [45]. In contrast, authors who publish at moderate levels in disciplinarily diverse journals and with a moderate level of collaborator diversity have been found to publish more, while a moderate level of collaborator diversity is beneficial for authors who are more focussed on output [46]. We draw on these two streams of literature to propose that there is a positive correlation between interdisciplinary collaboration in a group and the number of publications produced (Table 1, H2(a)).

Our second output measure is the impact of the work as indicated by citation rates. Several articles have examined the effect of team diversity on the citations received (e.g., [4749]). We conceptualize these factors as having an impact via their effect on the interdisciplinarity of the team process. The effects of interdisciplinarity have mostly been studied at the level of individual articles rather than teams, generally finding a positive impact of citing more diversely (e.g., [5052]). In contrast, at the individual researcher and team level, the effect has been found to be curvilinear, with an inverted U-shape, meaning that impact is highest for intermediate levels of interdisciplinary [53, 54]. The effect has been attributed to the costs of learning to work across disciplines, where too much interdisciplinarity can become a distraction (Table 1, H2(b)).

In the previous two hypotheses, we have argued that teams with an interdisciplinary process will be more successful in publishing. We expect this success to be reflected in the evaluations of the team members of group function. For example, that there will be a positive relationship between the interdisciplinarity of the collaboration in a group and member satisfaction with the group (Table 1, H2(c)), and the perceived effectiveness of the group (Table 1, H2(d)).

Methods

The project follows a mixed methods approach using triangulation of quantitative results with qualitative survey information [55]. The quantitative study examines team demographics, processes and outcomes while the qualitative component examines the opinions proffered by team members about the reasons for group success or otherwise.

Before commencing the detailed description of the data acquisition for the reported project, the limitations related to the ethics statements made to the participants and accepted by the Institutional Review Board of the corresponding author’s university and complying sufficiently with the Australian ethics standards were as follows:

“Personal information that is collected will be used solely to enable network analysis of members within a working group and will not be used for any other purpose. Results of the research may at some future time be published. Although your responses may be identifiable to the researchers, responses will be kept confidential and no individual responses will be reported; only summarized findings will be reported.” and “Your identity will be held in confidence as an invitee to the survey associated with the relevant synthesis centre and group. Your identity will not be published.” This being so, we have anonymised all respondent identities, including identifiable links to their organisations and disciplines, and to the specific publications by each group (which would allow identification of the authors and hence the groups being studied). We have not published here or in the data repository for the paper the full demographic profiles of members.

Research setting

We test these hypotheses through studies of a selection of research working groups designed to have some degree of diversity, with members with a variety of disciplines, skillsets and origins. The research working groups we are focussing on are a facilitated version of the ‘self-assembly’ to which Twyman and Contractor [56] refer and which are well-discussed in the synthesis center literature [15, 30, 49, 57]. These groups gather, with logistic support, to analyse a specific complex problem, drawing on members from many institutions and countries for intermittent but concentrated periods of time apart from their normal workplace commitments. The staccato nature of the short but concentrated time the groups spend with each other in these ‘hot moments’ is often termed ‘island time’ (Eric Garnier pers. comm. in [15]) and has been found to be particularly conducive to creativity [26, 57]. The members of such groups are largely ‘volunteers’, driven by a common interest in the problem being addressed. They are supported to a limited extent in their endeavours, but not salaried to solve the problem at hand (Fig 2).

Fig 2. Working group workflow.

Fig 2

A depiction of the way the working groups work together in our scenarios, where groups of diverse people meet together intermittently over a period of time in a supported environment to examine difficult, multi-disciplinary, problems. The meetings might each last up to a week, and are often at six month to yearly intervals.

The groups we studied were drawn from a large cyberinfrastructure project (DataONE) and from two synthesis centers in the biodiversity and environmental fields. The DataONE project adopted the synthesis working group approach in its participatory engagement model, enabling it to tap into expertise in the wider community as it constructed its e-infrastructure [58].

Synthesis centers are purpose-built organisations designed to enable diverse groups of people (working groups) to synthesise disparate data and information on a particular topic to produce new understanding [17, 49, 59, 60]. They have been likened to the business ‘incubator’ [61, 62]. Synthesis centers support the use of existing information and apply, to greater or lesser extent, many of the elements of team science, open science and data-intensive science [5, 15, 30, 62]. They have been highly successful in facilitating collaborations and are highly productive [15, 17, 49, 60, 63].

The composition of synthesis center groups is largely determined by group leaders (Principal Investigators) within criteria set by the centers, such as a degree of inter-(or less often, trans-) disciplinarity, multi-national and multi-sectoral teams, career and gender balance ([51] and as described on the center web sites accessible through www.synthesis-consortium.org). This approach means that a large proportion of the group starts with agreement on a common goal. The working groups in our study were supported for a maximum of four years with multiple meetings over that time. For the synthesis centers and for DataONE, good outcomes are critical to continued funding and the organisations post the products of the groups they sponsor on their web sites for public access.

It is relevant to note that the work of these groups is collegiate, as far as communication and the development of trust will allow. Decisions are made as a group, for example, once the groups have examined the data and information to hand, they will determine their work plan, identifying the products (usually articles but some code and conference presentations) that would seem most achievable and valuable, and then separate the tasks among the group members according to the skill sets needed. Occasionally needed skills were not available within the group and additional experts were brought in to complete the specific task. As one survey respondent put it, “The group was very effective at dividing tasks, assigning roles, and then getting their parts done. Everyone wanted to contribute.” (C-1, R2). If a group member did not pull their weight they could be isolated. The group leader(s) keep track of these tasks and all group members will revisit their various tasks at the start of each meeting.

By confining our study to this cohort we were able to compare across three similarly-aged organisations all providing similar logistic and infrastructure support (an important factor influencing creativity of output according to [14]). The selection of the specific groups for study within each Center or project were determined by (a) advice of the director of the Center (JWP), (b) maturity of the group, and (c) response level to the qualitative survey (described below).

It should be noted that although our ambitions were to include another center and hence suite of groups in our study, the response from the center approached, although positive, was tardy (one year after initial enquiry) and did not fit our time-frame, especially given we already had results from 2014. The size of the sample we analyze in this paper restricts the analysis possibilities.

Data acquisition

We collected three types of data for the model:

  1. demographic profile of group members (inputs: Fig 1)

  2. collation and analysis of articles used and produced (process and outputs: Fig 1)

  3. group member perceptions (outputs: Fig 1)

For items 1 and 2 the data were available through the group websites. Item 3 was accomplished by a survey sent to the members as published on the group websites.

Working group demographics

The demographic profile of all members of each selected Working Group was collated. This included their country of origin and their gender (binary only; no members with non-binary identities were identified).

The primary scientific discipline of each group member was defined using their own statements to the contributing organisations, and where not available, their self-stated fields on sources such as ResearchGate and Google Scholar, and their pages on their organisation’s web sites. An initial list of forty-six primary disciplines was created using the Australian and New Zealand Fields of Research [64] to derive a smaller controlled vocabulary of 22 categories. This reduced vocabulary included biology and applied biology, chemistry, climate science, communication, computing, data science, earth sciences, ecology, education, engineering, environmental science, evolution studies, freshwater (and marine) studies, geography, health, hydrology, library and information studies, modelling, policy, sociology, and statistics. As mentioned, for confidentiality reasons these discipline fields are not listed to enable linkage with the participant or the group to which the member belongs.

Collation of articles used and produced

A key construct in our model is the interdisciplinarity of the group process. As we were unable to follow each group individually over the several years they were collaborating, we sought a measure that could be applied retrospectively. Interdisciplinarity can be assessed in many ways [5], of which bibliometric measures are the most developed and have valuable antecedent analyses [e.g. 6569]. We used journals as an indication of discipline. Journals are responsive to the communities they serve (they have to have an ‘audience’ to survive), and several disciplinary themes or ‘keywords’ are used to describe the journal field, thus encouraging relevant articles and providing appropriately qualified editors and reviewers to assess the submissions. In this way the selection of journals in which to publish is tailored to the subject matter. Of course, some journals are quite general and welcome a wide range of articles, which clouds the specificity of the domain description, although in our case these occur rarely in our study due to their high impact (e.g., Nature and Science). Other measures such as data deposition, grant success and conference presentations are also useful indicators of productivity as well as measurable for disciplinary variety, but they were inconsistently measured across our study groups, so we have confined ourselves to refereed journal articles produced by the groups.

We based our analysis on the groups’ publications. We obtained copies of all papers published by the groups up until August 2020, the vast majority of which were open access. The number of citations to each publication was acquired from CrossREF on 19 August 2020. Not all articles produced by the groups were in journals tracked fully by CrossREF, as CrossREF mainly scans English-language journals, but this is a relatively minor impediment to our analysis as English is the norm for the fields we are studying.

To assess the interdisciplinarity of the working groups’ collaborations, we analysed the literature on which the group drew to produce their published articles, the ‘inspiration’ measure of [9] and used by several authors in whole or in part as a measure of interdisciplinarity [33, 7072]. Specifically, we examined the disciplinarity diversity of the cited journals [76, 73]. We selected this measure based on the premise that groups working in an interdisciplinary way bring together diverse knowledge [2, 74, 75], which will be reflected in the diversity of the literature cited. To consider the inter-or trans-disciplinary nature of the outputs from a group rather than just their productivity, we also examined the degree to which the outputs appear in journals in a diversity of disciplines [49, 76].

To obtain these measures, we first extracted which journals were cited in the published papers, as well as the journals in which the groups published their own articles. A total of 1007 journals were identified in this process. Each journal was classified according to disciplinary category(ies) using SCOPUS, SCIMAGO and journal-stated discipline fields if not listed in those databases, being generous rather than reductionist in allocation. In this manner 128 specific journal disciplines were identified across all working groups, from agriculture to ecology to parasitology and water science. Guided by the Australia and New Zealand Fields of Research categorisation (ANZFoR [64]), these specific disciplinary categories were then clustered under 22 ANZFoR disciplinary divisions, with two non-ANZFoR categories created due to no suitable matches in the ANZFoR Divisions. These discipline categories for journals, although they are responsive to the communities they serve, are admittedly relatively general. The disciplinary divisions used were: agriculture, veterinary and food sciences, biological sciences, biomedical and clinical sciences, built environment and design, commerce, management and tourism services, earth sciences, economics, education, engineering, environmental sciences, health sciences, history, heritage and archaeology, human society, information and computing sciences, language, communication and culture, law and legal studies, mathematical sciences, philosophy and religious studies, physical sciences, and psychology. The two additional divisions were ‘bio water science’ which covered all freshwater and marine biological sciences, and ‘general science’ which enabled the classification of journals like Science, and Nature that cover multiple disciplines.

Group member perceptions

A third source of information came from a survey of Working Group members who were asked via electronic survey about their perceptions of group performance (satisfaction and perceived effectiveness on a 1–5 point scale, from not at all effective/satisfied to very effective/satisfied, with an invitation to make comments explaining their choice). The initial intent of these questions was to combine them with productivity as a multi-item measure of group success [31], but as the correlations were not high, we instead analyzed them individually. We also invited open-ended comments from subjects about the ‘primary factors that they felt contributed to their working group’s effectiveness or lack thereof’, for a total of five questions. These questions were part of a longer survey including questions not related to interdisciplinarity that we do not analyse in this paper. As there were many other components in the survey, we considered that having too many items for these scales would negatively impact the return rate. Links to the entire questionnaire were emailed first to the DataONE email distribution list during an All Hands meeting in October 2014 (including to members who were not in attendance at the meeting) at the end of Phase 1 of DataONE. The same survey was subsequently emailed to Working Group members of the two synthesis centers in late 2018. Only groups that had finished or were close to finishing their work were invited to participate and there was little change in group composition at the times of the surveys (2014 and 2018) in relation to when the groups started (and finished). In all cases, 2 weeks was given for return, with an extension of another week. We only included groups in our analysis from which we received survey responses from at least 20% of members.

Analysis

Operationalization of concepts

In this section we explain how we used the collected data to measure our research constructs. First, we computed measures of group diversity along the different demographics. For each group, we started with counts of members in different demographic categories (e.g., number of members from different countries). For country and discipline, we used the category counts to compute a measure of diversity (entropy) using the Shannon index [49, 7779]:

Diversityorentropy:H=Σ(pilnpi) (Eq 1)

where pi is the proportion of members in group i.

Since the data collected on gender were binary, we simply used the proportion of female members in the group. In our dataset the range of proportion female was 4–67%, meaning we had some nearly all-male groups but no all-female groups.

To assess the extent of the interdisciplinarity of the collaboration process, we measured the diversity of the disciplines of the journals in which the groups published and those they cited in their publications (computed using Eq 1 above). For this purpose, we combined the count of journals cited per discipline across all the working group’s publications. Diversity can, of course, be assessed in different ways, such as variety, balance and disparity [79]. In a similar manner to assessment of the variety of demographics, we used Shannon diversity as our measure for disciplinary diversity. Although this measures variety it is also affected by balance (lower balance leads to lower diversity). We also note that some studies of interdisciplinarity have assessed not just the variety of disciplines cited but also the atypicality of the combination (e.g., [80]), however, since our classification of journals is based on a different system than in Uzzi [80], we do not have data on typicality.

We measured the output of the group in four ways, first by the number of publications and the impact of the group’s work by computing the median number of citations the various publications attracted. We assessed satisfaction with group performance and perceived group effectiveness at an individual level through a survey of members. We used the mean of the individual scores as the group measure.

In addition to the variables in the model, we included two control variables: (i) the age of the group, with the assumption that groups that have been around longer have had more of an opportunity to publish and to have an impact (as measured by citations), and (ii) the Center (a three-level factor variable), to control for different expectations around publishing in the three settings.

Hypothesis testing

Hypotheses were tested using regression on the data emerging from the demographics and diversity measures. Data other than counts were standardized before regression. One problem arose in carrying out the regressions: as we had only 22 groups, using too many variables in the regressions led to overfitting. Unfortunately, the small number of data points also meant that we could not test the hypotheses simultaneously, e.g., with a structural equation model.

The problem of potential overfitting arose for the analyses of the demographic input variables. A regression that included all the demographic variables achieved nearly perfect R2, an indication that the model was overfit. To avoid this problem, we used a stepwise regression approach, adding variables that were most related to the outcome, but stopping with a small number of variables. We explored reducing the dimensionality of the input variables through factor analysis but did not find a satisfactory solution with a smaller number of factors. We also explored more modern techniques for variable selection such as lasso but did not have enough data to use them.

Where we hypothesised curvilinear relationships, we entered variables both as a linear and a squared term using the R poly function, which computes orthogonal polynomials to avoid multicollinearity. As the use of this function complicates interpretation of the regression coefficients, we present non-linear relationships graphically.

Qualitative analysis

The open-ended responses were subject to thematic analysis related to the research question (“In your opinion, what are the primary factors that contributed to your working group’s effectiveness or lack thereof?”) using an inductive semantic approach [81]. The themes thus identified alongside their associated (anonymised) comments were sent to the groups for validation of our interpretation and modified if required. In this process additional insight was often obtained.

Results

Group analysis

Demographic data

Demographic profiles were obtained for the groups that responded to the on-line surveys in sufficient numbers (>20% responses). The resulting population of 389 people came from 28 countries, the majority from the USA (62%), and the next highest from France (13%), the home of the third organisation in the study. The UK and Canada were also relatively well represented. The total population was predominantly male (68% male and 32% female) and 51% of members were from universities, 19% were from government organisations, and 15% from research organisations.

Ecology was by far the most common primary discipline type (23%), with an equal proportion of people in the computing, data science, statistics and modelling areas. These distributions reflect the focus of the sponsoring organisations and the emphasis of the working groups on working with data. Freshwater biology and ecology were clustered together into freshwater science, which contributed 4.6% of the total.

Group size was variable, ranging from 11 members up to a maximum of 28 (S1 Appendix). The demographics differed between groups, with some groups very international, and others exclusively from one country. The proportion of females in the groups ranged from 0% to 67%, an average of 33% ± 4%. While there are clear differences among groups, overall the mix of respondents seems representative of scientists who participate in working groups [18, 28, 42] and was consistent with the first author’s personal experience coordinating working groups. Groups that had at the time of the survey tended not to respond to the survey (nor to have output to analyse), and so are not included in the study. Nevertheless, we have a range of productivity and impact in the groups included.

Meeting attendance data (not that that is the only measure of fidelity to the group) was not available consistently for all groups, but for the data that were available, fidelity to the group was between 60% and 80% over 2–10 meetings. Fidelity was always greater than 80% for groups that had only two meetings.

Publication analysis

One hundred and fifty-seven journal articles were collectively produced by the groups at the time of our study (August 2020), and 6,749 articles were cited in these articles. The articles appeared in 83 different journals in 22 ANZFoR disciplinary divisions. The articles cited (the ‘inspiration’ of Gates et al. [9]) were drawn from journals in 18 ANZFoR disciplinary divisions.

Group A-1 had the lowest publication diversity of all the groups at 0.693 (S2 Appendix) with only two publications at the time of the study, one in the ANZFoR Division biological sciences and one in environmental sciences. The citation diversity was similarly low (0.287, S2 Appendix). The articles cited across these two papers were from forty-six journals, with thirty-two journals cited in one paper and seventeen cited in the other. The vast majority of the citations (91% and 93%) were to publications in the ANZFoR Division biological sciences.

In contrast, B-7, with eighteen publications at the time of the study, had the highest publication diversity at 1.874 (S2 Appendix) with publications across seven ANZFoR divisions (agriculture, veterinary and food sciences, biological sciences, earth sciences, environmental sciences, general science, human society, and information and computing sciences). To produce these articles, the group drew on papers from seventeen ANZFoR Divisions (agriculture, veterinary and food sciences, bio water science, biological sciences, biomedical and clinical sciences, built environment and design, commerce, management and tourism services, earth sciences, engineering, environmental sciences, general science, history, heritage and archaeology, human society, information and computing sciences, language, communication and culture, philosophy and religious studies, physical sciences, and psychology). The citation diversity (and its distribution amongst the papers), however, was not the highest of the groups at 1.301 (S2 Appendix). Group C-4 had the highest citation diversity with an index of 1.656 drawing on articles from ten ANZFoR Divisions (agriculture, veterinary and food sciences, bio water science, biological sciences, earth sciences, engineering, environmental sciences, general science, information and computing sciences, mathematical sciences, and psychology) across their three articles.

Citations per article ranged from a high of ninety-six different journals for paper C of group A-6, to no, or one, citation per article (e.g. B-6 paper C and B-3 paper F), while many groups cited forty or fifty articles per paper (e.g. C-1 paper A, A-9 paper F, B-1 paper B).

The average number of authors per article was 8.75 ± 0.51. This number alone is greater than the average number of authors for all journal discipline fields found in past studies of scientific collaboration. For example, a maximum of 5.9 authors per article for the environmental sciences was reported in Patience et al. [82] and 5.19 for synthesis centers by Hackett et al. [49]). This evidence suggests the working groups in our study were highly collaborative. The average number of authors per article was highest in group A-4 closely followed by C-1 and A-9, while the minimum average number of authors was 1.7 for group B-6 (S2 Appendix).

The bibliographic data are available in the repository of the Environmental Data Initiative [83].

Qualitative feedback

Response rates to the surveys per group ranged from 56% to a minimum of 20% (our cut-off for including the group in the study). Of the ninety-two respondents who provided feedback to the ranked questions about group effectiveness and satisfaction, only forty-eight provided open-ended responses about effectiveness, and twenty-two, about satisfaction. One hundred and twelve responses were received to the general request for comment across all three survey groups. Detailed commentary on the open-ended responses is included in the qualitative analysis.

Respondents’ perception of their group’s effectiveness and satisfaction was positive (all measures were above 2.5 on a 5-point scale, S3 Appendix)). However, some groups (A-7 and A-8, for example, and B-5) were as low as 3 on the 5-point scale, while others (A-4, B-6 and C-2) thought their groups were particularly effective and were correspondingly satisfied with the group’s function. There was not, however, a linear relationship between the two metrics, so we treated them as separate entities.

Hypothesis testing

In this section, we describe the results of testing the proposed hypotheses.

Prediction of interdisciplinary collaboration (H1)

To assess the level of interdisciplinary collaboration in the groups we used the diversity of the disciplinary fields of the references cited by the groups, and the diversity of journals chosen by the groups for their publications. We used stepwise regression to identify influential variables for each dependent variable separately.

The disciplinary diversity of the journals cited by the groups in the production of their articles (cited publication diversity) was positively related to the disciplinary diversity and the proportion of women in the group (p < 0.01; Table 2). The diversity of the journals in which the groups published was negatively related to the county diversity of the group (p < 0.05; Table 2) and positively related to discipline diversity (p < 0.05), but other factors were not selected by the regression.

Table 2. The effect of input variables on citation and publication diversity.
Diversity of publications cited Publication diversity
Intercept 0.000 (0.122) 0.000 (0.165)
Country diversity –0.403* (0.175)
Proportion of female members 0.644*** (0.130
Discipline diversity 0.391** (0.130) 0.445* (0.175)
Number of observations 22 22
adjusted R2 0.669 0.401

Mean, standard error (in parentheses) and significance.

+p < 0.1

* p < 0.05

** p < 0.01

*** p < 0.001

Prediction of output (H2)

As the outcome variable was a count, a Poisson regression was used to assess effectors of the number of publications produced. On the other hand, the median number of citations received was over-dispersed (the standard deviation was greater than the mean), with a few clear outliers, so a negative binomial regression was used instead.

The disciplinary diversity of the journals in which the groups published predicted the number of publications (p < 0.01, Table 3). The disciplinary diversity of the articles cited in producing those publications, our measure of interdisciplinary collaboration, also positively predicted the number of publications (p < 0.05, Table 3). As hypothesised, the median number of citations received had inverted-U relationships with both measures of interdisciplinary collaboration, as shown in Fig 3. Among the controls, there was a strong linear positive relationship between group age and the number of papers produced (p < 0.001; Table 3) and differences were noted among the centers. Specifically, the B groups had significantly fewer publications but more citations than the A groups, while the C groups were not significantly different (note that the choice of the A groups as the baseline to which B and C are compared is arbitrary).

Table 3. Factors predicting output variables.
Number of publications Median number of cites to
Intercept 2.105*** (0.160) 2.351*** (0.241)
Project age 0.818*** (0.156) 0.091 (0.220)
A groups
B groups ‒1.381*** (0.342) 1.854*** (0.473)
C groups 0.329 (0.287) 0.623 (0.451)
Publication discipline diversity 0.257** (0.099) 0.608 (0.767)
Publication discipline diversity squared ‒2.434*** (0.636)
Cited discipline diversity 0.372* (0.152) ‒1.788* (0.840)
Cited discipline diversity squared ‒1.627+ (0.831)
Number of observations 22 22
Nagelkerke’s pseudo R2 0.888 0.940

Regression weight, standard error (in parentheses) and significance of each variable is shown. “A” groups were used as the baseline for the group factor.

+ p < 0.1

* p < 0.05

** p < 0.01

*** p < 0.001

Fig 3. Influences on citations received.

Fig 3

The median number of citations received by a group’s publications against (a) publication discipline and (b) the diversity of publications cited. One group is an outlier to the plot in 4(a) and is not visible in the plot. The 95% confidence envelope around the trend is shown. ‘0’ on the axes is the grand mean for all groups for that variable, and the intervals shown are one standard deviation from the mean. The data points show the partial residuals for the groups, i.e., the residual after controlling for the other variables in the regression.

Perception of satisfaction and effectiveness

A model predicting self-reported satisfaction and perceived group effectiveness based only on process and emergent state variables (the hypothesised model) did not reveal any significant predictors. We therefore considered whether these outcomes might be affected also by group composition. To test this post-hoc hypothesis, we used stepwise regression to identify independent variables. We also added the number of publications (log transformed because of skew) as a predictor.

Satisfaction with the group was negatively related to the age of the group (p < 0.05; Table 4) and, as might be expected, positively related to the number of publications the group produced (p < 0.05; Table 4). Satisfaction was negatively related to the diversity of publications cited in the production of those papers (p < 0.05; Table 4), negatively related to the diversity of countries of the group members (p < 0.05; Table 4), and positively related to the proportion of female members in the group (p < 0.05; Table 4).

Table 4. Predictors of satisfaction and perceived group effectiveness.
Perceived satisfaction Perceived effectiveness
Intercept 0.000 (0.164) 0.017 (0.159)
Project age ‒0.572* (0.192)
ln (number of publications) 0.462* (0.209) 0.238 (0.168)
Publication discipline diversity ‒0.404 (0.255) 0.376 (0.213)
Cited discipline diversity ‒0.817* (0.313) –0.512* (0.199)
Country diversity ‒0.626* (0.242) ‒0.260 (0.176)
Proportion female 0.624* (0.289)
Disciplinary diversity 0.316 (0.247)
Number of Observations 22 22
Adjusted R2 0.422 0.169

Mean, standard error (in parentheses) and significance of each variable shown.

+ p < 0.1

*p < 0.05

**p < 0.01

***p<0.001

Predictors of perceptions of the effectiveness of the group were more limited, namely that the more diverse the publications sourced by the group the less effective the group was perceived to be (p < 0.001; Table 4), the higher disciplinary diversity of the group, the more effective respondents felt the group was (p < 0.05; Table 4), and there was a moderate positive influence of the proportion of female members (p<0.05; Table 4).

Summary

All hypotheses were supported except for Hypotheses 2(c) and 2(d) (Table 5). Country of origin negatively affected the process of publishing diversely and personal satisfaction with the group (Fig 4). Citing a diverse range of articles in producing output was correlated with lower satisfaction and perceived effectiveness, despite not actually affecting measurable output, which had a strong positive effect on personal satisfaction with the group (Fig 4). The proportion of female members had a positive effect on the diversity of articles cited and on personal satisfaction with the group (Fig 4). The diversity of disciplines represented in the groups had a strong positive effect on both metrics used for interdisciplinary collaboration and had a less positive effect on the perceived effectiveness of the group (Fig 4).

Table 5. Summary of the outcome of the hypothesis tests.
Hypothesis Proposed relationship Comment
H1 There is a positive relationship between interdisciplinary collaboration and:
(a) measures of gender diversity The more female members in the group, the greater the cited discipline diversity (Table 2).
(b) the interdisciplinarity of the group The more discipline diversity in a group, the greater the cited discipline and publication diversity (Table 2)
(c) There is a negative relationship between interdisciplinary collaboration and the diversity of international membership There is a negative relationship between publication diversity and the diversity of countries involved (Table 2)
H2 (a) There is a positive correlation between interdisciplinary collaboration in a group and number of publications. There is a positive relationship between the number of publications and their disciplinary diversity and the diversity of the articles cited (Table 3).
(b) There is an inverted-U relationship between interdisciplinary collaboration in a group and impact as measured by the median number of citations. Median number of citations received by a group’s publications is highest for intermediate values of publication and citation diversity (Table 3 and Fig 2)
(c) There is a positive correlation between interdisciplinary collaboration in a group and personal satisfaction Respondents’ satisfaction with the group was positively related to the number of publications the group produced and the proportion of female members in the group, but negatively related to the diversity of the cited publications and the diversity of countries (Table 4).
(d) There is a positive correlation between interdisciplinary collaboration in a group and perceived effectiveness Perceived group effectiveness was negatively related to the diversity of the cited publications. It was, however, positively related to the disciplinary diversity of the group (Table 4)
Fig 4. Model output.

Fig 4

Graphical summary of the quantitative results. *specifically median citations received by each article.

Qualitative analysis

The outcomes from the open-ended feedback to the survey of member’s satisfaction with and perceptions of the function of the group did not always compartmentalise neatly under our hypotheses nor did they directly complement the quantitative results. They added, however, more depth and some additional insights into perceptions of what made the group successful or not. It must be noted that any comments were entirely voluntary. These comments have been collated and can be viewed through the Environmental Data Initiative (link to be inserted on publication).

The quantitative analysis showed that diversity of country of origin, gender and disciplinary representation can affect group process and satisfaction with group function. There were many comments about the benefits and challenges because of disciplinary or skillset differences, while gender balance was never commented on.

Hypothesis 1b posited that there was a positive relationship between interdisciplinary collaboration and the interdisciplinarity of the group, and this was supported in the quantitative analysis. In the qualitative analysis, six respondents mentioned the positive benefits of having diverse disciplines in the groups, and although there were some difficulties, respondents mentioned that these difficulties were worth dealing with. As one commented, “Slightly different objectives and different viewpoints did not always make for the most effective discussions and decisions, but I think that is part of the process and is critical to interdisciplinary groups.” (C-3, R14). There was awareness of a need to deal with any challenges arising from disciplinary differences “everyone’s willingness to think actively about how to improve the merger between the disciplines” (C-5, R1). Generally, this diversity was regarded as stimulating “I learnt much by simply listening to so many different scientists with different and complementary skillsets” (A-7, R1), “we brought in a diverse range of disciplines to liven up the stew.” (C-2, R1). “The distance between disciplines was clearly sufficient to create some learning opportunities, but not thought to impede output” (A-7, R2). “I found the functioning of the group very effective despite people coming from different expertises” (A-3, R2).

Disciplinary differences were often spoken of interchangeably with skillset differences, “We also worked on group dynamics so different personality types and skillsets could work together and benefit from the multidisciplinarity (as opposed to becoming defensive in our corners).” (C-5, R2), “The diversity of skills represented…We were able to distribute the work effort in a way that made the most sense for moving the project forward.” (C-4, R1). A diversity of skillsets was inherent in every group in the study, and there were reasons for separation according to skillsets but it was felt that such separation should not exclude sub-groups from overall group discussions “Everyone got along just great, and we worked hard. I would gladly work with folks again. The group somewhat naturally by necessity split into two parts; those with the technical skills for the data wrangling and statistical modelling, and those who could not. The latter group thus was able to spend a lot of time conceptualizing, a good and important task, but it left out the technical folks from that part of the project. As one of the technical folks, I would like future projects to allow a more even mix of doing both.” (C-4, R5).

The value of diversity in general was, however, considered positive, “…I think the diversity of approaches and skills hugely contributed to the quality of the final paper. That paper was focused on how to effectively merge findings from fieldwork with the work of the modelers—that was very exciting and couldn’t have happened without the diverse membership of the group.” (C-5, R1). A member of another group, which comprised members from the non-research community, systems analysis, web developers, ecologists and those concerned with management and social engagement, noted that “having a diverse set of viewpoints to contribute to the work” (B-3, R1) was a factor that contributed to the group’s success.

Respondents identified several additional factors beyond those we measured in the quantitative analysis. These included the diversity of background (assumed to be organisation and country of origin) of team members, career stage representation, team fidelity, including having common goals and objectives, within-group trust and respect, and the nature of leadership.

Background was highlighted occasionally as evidenced by four responses: “To put it simply, the researchers and the management agency participants initially had difficulties communicating due to different perspectives and terminology barriers” (C-3, R6). However, this diversity was regarded by another in the group as “We had somewhat different perspectives and backgrounds but mostly saw eye-to-eye, had fun and got along well.” (C-3, R4). Rather interestingly, this group recognised the problem and turned it into a positive “I think the team did overcome this to some extent; indeed, one paper was largely devoted to terminology…” (C-3, R6). Another group observed the tension that can occur as “a lot of the collaboration depends on the diversity in the group, despite a temptation of some of the co-located members to conduct independent discussions and work without communicating with others.” (B-7, R5). This tendency for co-located members (especially if a common language was also involved) to have separate discussions excluding others in the group, was observed more than once by the authors. Ensuring this division is not disruptive is a particular role for the hosts of the groups.

The presence of early career researchers (including postdocs) was mentioned by three respondents as an important, positive, contributor to group function, “…some members of our group, young postdoctoral students who were invited by their supervisors … contributed a lot. Less established researchers can be a positive force in this type of working group.” (A-7, R1); “The importance of the meeting and postdoc support … to coordinate efforts and achieve plans jointly designed during meetings” (A-1, R2), “We had a good mix of career stage: senior scientists that served as excellent advisors, we had postdocs and PhD students that have the interest and ability to work and bring new ideas, and we had motivated early career scientists. We learned a lot and some excellent publications and collaborations came out of this working group.” (C-5, R3).

Sharing common goals and objectives was considered an important factor in group effectiveness, “Convergence of goals despite different backgrounds, skills and kinds of contributions” (A-5, R7); “Common interest and objectives…” (A-7, R2); “…consensus over the common goals.” (A-8, R2); “We had clear and common goals…” (C-1, R5); “…dedication to tasks; belief in goals” (B-27, R6); and “The group self-selected and each member has a vested interest in the successful outcomes and outputs” (B-7, R1).

The importance of good social skills, including within-group trust and respect was raised in several ways: “Enthusiasm, nice people, no intrigue” (A-7, R4); “…mutual trust…and friendly atmosphere” (A-4, R1); “We listened to each other and simply got along well!” (C-1, R5); “mutual respect” (B-1, R1); “compatibility, respect, and the skills and knowledge of the members” (A-2, R4); “Tremendous individual integrity, trust in teammates, empathy in reading others’ feeling, and supporting everyone on the team.” (C-2, R10); and “Collaborative and friendly attitude” (B-7, R6). “It is also a great group of people who were willing to listen to others, lead, let others lead, pull their weight, be responsive, be respectful, etc. Most of these people could have big egos, but they don’t. It was an outstanding professional and personal experience.” (C-2, R12).

Several comments were made about the quality of leadership. If leadership was not good, things could become difficult: “…We did not have very strong group leadership, which also contributed to not getting as much done as we could have.” (A-8, R4). This response incidentally was consistent with the quantitative ranking for effectiveness by group A-8 (section 4.1.3). “Effectiveness resulted from great leadership (encouraging openness to voices and ideas)…” (C-6, R2); “Good leadership” (B-4, R1); “Leadership. Co-chairs provided an encouraging working environment where each member of the WG can thrive.” (B-6, R2); “Strong leadership and vision from group PIs” (C-2, R13). As noted in the comments of R12, C-2 above, leadership need not rest with one person “Everybody’s willingness to take a leadership role when needed, take a step back and letting others lead when needed, and never dismissing any idea just because it did not align with an a priori opinion” (C-6, R4).

As a final mention, however, is that these groups, as with many such groups, have a core set of enthusiasts and the face-to-face meetings were pivotal. Many of the non-targeted comments were about participation (those fence-sitters) and of the importance of the face-to-face meetings for getting work done. “A smaller subset of the group did most of the work. Many folks in the group did little or nothing” (C-5, R4); “except for a couple of fellows” (A-2, R3); and “We had a few ‘doers,’ that is a few key folks kept things moving. A couple of the leaders were integral in keeping a vision up front” (C-2, R3). Getting people’s attention and commitment was easy within the meeting but between meetings (especially when there were a few) was harder: “most people are already overbooked for time so we had some difficulty finding people who wanted to lead projects.” (C-1, R3); “Once we went home after each meeting there was little contact and cooperation on projects. From my perspective, I think I could have made the outcome better if I had time to pursue collaborations after the face-to-face meetings. Unfortunately it was an extremely intense time in my ‘real’ job.” (A-2, R4); “The group was very effective during the week where we were together (at least the persons that assisted to the meetings). However, much of the momentum was lost after some months” (A-1, R1); and “Working sessions went generally well. But only three to four persons were really active between working sessions.” (A-3, R5). Basically “The number of meetings was super important and also helped people to have a sense of accountability. There is no substitute for being in the same room and working on projects together.” (A-4, R4). This conclusion has been argued cogently by Srivastava et al. [84].

Discussion

The goal of the work reported in this paper was to uncover the influences of group composition on the interdisciplinarity of teamwork, where the groups were attempting to find solutions to inter-disciplinary research problems. We posed a number of hypotheses about potential influences that may affect interdisciplinary practice resulting in articles, particularly articles that have impact. To examine the dynamics experienced by such groups, we used an input-mediator-output model (Fig 1) with a sample of working groups in the environmental sciences.

We found that key aspects of group composition had a largely positive effect on interdisciplinary team processes: the gender balance and the diversity of disciplines represented in the group all increased interdisciplinary collaboration (Table 5 and Fig 4), while the number of countries represented in a group offset this general trend. The number of countries is only a partial indicator of interpersonal, cultural challenges, but it is a reasonable measure which can imply a range of types of differences related to origin and background [85]. While it is a truism of research that correlation cannot prove causation, in this study the diversity of the team was set when the team was created, ruling out reverse causation, and making spurious correlation less plausible.

Our work has shown that the greater the level of interdisciplinary collaboration the greater the number of resulting publications. The impact of those publications (measured by the median level of citations received) was less straightforward, with greatest impact at moderate levels of interdisciplinary collaboration (Fig 3). It seems that being too interdisciplinary can mute impact, and too little may be ‘boring’. We acknowledge that our citation data are relatively short-term (1–10 years after publication). Even though papers are usually cited most heavily shortly after publication, it is possible that more interdisciplinary papers will go on to be more cited in the longer term, as suggested in the literature [50]. The causal direction of these results might be debated, though it is not clear why teams that publish more seem to choose to do so in an interdisciplinary way, and as citations follow the process, reverse causation at least seems ruled out.

Group-member perceptions enriched our understanding of the process. Not surprisingly personal satisfaction increased with the number of publications produced and, more unexpectedly perhaps, with the proportion of women (Fig 4). Most evident from the quantitative study was the negative effect of country and diverse work practice (measured by the level of diversity of articles sourced) on personal satisfaction and perceived effectiveness of the group. There were no comments received about difficulties specifically related to nationality. The opportunity of working with people from different disciplines and backgrounds was generally considered stimulating, and respondents recognised that this discomfort did not relate to the quality of the output.

The qualitative responses highlighted that different backgrounds were often difficult at first, but adjustments could be made in the time-frame available to these groups. Four factors were identified which promoted productivity and offset any difficult collaboration barriers: (i) having early career members in the group, (ii) sharing common goals and objectives, (iii) mutual trust and respect, and (iv) the quality and team-centered nature of group leadership (as per [86]). It must be remembered that these groups worked together in a formal manner (as in having supported collaboration) for up to four years with multiple meetings during that time, and they had ample time to reflect upon and resolve their differences to produce their outcomes.

Our study examined working groups, which are scientific teams, but with differences from managerially-assembled teams employed on a research project. For instance, it is easy for a member of a voluntary working group who is dissatisfied with the team process to discontinue participation. While we believe the hypothesized relationships between demographics and process and process and outputs should not be affected by the differences, the generalizability of the model should be tested with a broader range of team types.

In summary, it can be said that the number of (relevant) disciplines brought to the table and a ‘good’ gender balance in a group, an interdisciplinary process can be enhanced. Open-ended feedback suggested that this would be supported by having young members in a group, and ensuring good esprit de corps. A strong interdisciplinary process is positively related to having more publications, with the proviso that exploring and publishing too diversely is related to reduced impact. It seems that in the case of deliberately-formed scientific working groups, with sufficient time challenges can be successfully tackled as the same group of people learn to work together.

Conclusion

A fundamental question for our study was whether a diversity of participants in a working group does lead to diverse practices and outcomes. We think we have shown that, with some provisos and a degree of management, it does. We used an original conceptual model, namely positing the interdisciplinarity of collaboration as a moderating factor between team demographics and outcomes, and we think this has provided new insight into how groups function. The overall pattern of the findings is mostly consistent with our expectations, while the contradictory findings point to possible new perspectives on the function of research groups and a consequent management focus. We suggest that group diversity is not just a goal in itself, but is rather a support for building an interdisciplinary collaboration, the success of which has benefits for output (in moderation).

While our findings point to several factors important for working group success, they are limited by the fact that we had data from only 22 groups and 3 centers. As the survey response rates were not high, the conclusions about satisfaction and effectiveness could be more robust, and this deserves further exploration. We should also note that the groups studied were assembled by organisations that were already sensitive to some of the factors we have been discussing.

The qualitative data point to several factors important to the group participants that we were unable to capture with our quantitative data but were important to the successful function of the groups. Approaches such as the use of sociometers [87] and autoethnography [34] for example, could add greatly to the further understanding of the group dynamics that promote positive research outcomes.

Even with the limitations of our dataset, our findings reinforce the importance and apparent benefits of diversity of various types in research groups. They also indicate the need to pay attention to the interdisciplinarity of the collaboration itself and being more deliberate about the combination of new knowledge, working to ensure that the groups bring together their diverse perspectives. The negative impacts on satisfaction and effectiveness due to diversity due to (i) participant countries and (ii) the diversity of cited publications suggests a need to help participants with the knowledge integration process to make it more enjoyable and ensure fidelity within the group.

While our study is correlational rather than causal, our findings do suggest some guidelines for the management of cross-boundary groups to successfully engage in interdisciplinary work such as:

  1. ensure gender balance in a group, as this seems positively related to the interdisciplinarity of the work, the number of articles produced, and to overall group satisfaction;

  2. encourage publication (in conferences or journals) throughout the group process, as group satisfaction improves with the number of publications (in contrast to the delayed publication implied in a typical Working Group Workflow);

  3. be cautious if high impact articles are desired. Too much interdisciplinarity may be a distraction;

  4. allow the group time and give them support to work out ways to deal with personnel differences as well managed differences can be stimulating and rewarding;

  5. manage and support the discomfort that can occur when working across disciplinary boundaries, as some tension can result in novel outcomes; and

  6. support the group to achieve and maintain focus on the common goal, and ensure there is mutual respect and trust between members.

As stated at the beginning of this paper, global research challenges increasingly require the formation of teams that bring together individuals with diverse skills and perspectives who can work across disciplinary, organisational and national divides. Indeed, research has become so complex that individual scientists cannot achieve meaningful results without collaborating—the so-called collaboration imperative [88]. But while interdisciplinary research is needed, will assembling a diverse team necessarily result in publications that have impact? Our results show that interdisciplinary work is, in fact, positively related to publication rates, but if those publications are themselves spread across too wide a range of disciplinary domains (perhaps reflecting the work itself), their impact may be lessened, and contrary-wise, too narrow a view will diminish impact. At the same time some other aspects of team diversity can enhance or diminish this output. Paying attention to some simple factors in the design and management of such groups can improve the likelihood of a positive research outcome and enhance the satisfaction of group members.

Supporting information

S1 Appendix. Demographic profiles of the groups from the contributing organisations.

(DOCX)

S2 Appendix. Profile of publications produced and cited by each group.

(DOCX)

S3 Appendix. Responses to questions about perceived effectiveness and satisfaction with the group by respondents.

(DOCX)

Acknowledgments

Authors would like to acknowledge DataONE for facilitating our participation in the Usability and Assessment Working Group (inter alia) and for their consistent support for this project. We would also like to acknowledge the support and contribution of the synthesis center CESAB of the French Foundation for Research on Biodiversity and Jill Baron, co-Director of the John Wesley Powell synthesis center of the USGS. We are grateful for the open and transparent work practices of all three organisations which enabled this work to be conducted. We also would like to thank the several reviewers who provided invaluable advice on drafts of the article.

Data Availability

The data availability statement is aligned with the ethics requirements for the study. There were several limitations imposed by the Institutional Review Board of Syracuse University and accepted by the University of Queensland as suitable ethical conduct for the work proposed (Syracuse University #13-202, 16-203 and 18-230). These were as follows: “Personal information that is collected will be used solely to enable network analysis of members within a working group and will not be used for any other purpose. Results of the research may at some future time be published. Although your responses may be identifiable to the researchers, responses will be kept confidential and no individual responses will be reported; only summarized findings will be reported.” and “Your identity will be held in confidence as an invitee to the survey associated with the relevant synthesis centre and group. Your identity will not be published.” This being so, we have anonymised all respondent identities, including identifiable links to their organisations and disciplines, and to the specific publications by each group (which would allow identification of the authors and hence the groups being studied). We have not published here or in the data repository for the paper the full demographic profiles of members. With that consideration, the data are available as follows in the Environmental Data Initiative [83]. The data themselves will remain anonymised.

Funding Statement

The authors received no specific funding for this work.

References

  • 1.Baker B. The Science of Team Science. BioScience. 2015;65: 639–644. doi: 10.1093/biosci/biv077 [DOI] [Google Scholar]
  • 2.Cheruvelil KS, Soranno PA. Data-Intensive Ecological Research Is Catalyzed by Open Science and Team Science. BioScience. 2018;68: 813–822. doi: 10.1093/biosci/biy097 [DOI] [Google Scholar]
  • 3.Wagner CS, Whetsell TA, Mukherjee S. International research collaboration: Novelty, conventionality, and atypicality in knowledge recombination. Research Policy. 2019;48: 1260–1270. doi: 10.1016/j.respol.2019.01.002 [DOI] [Google Scholar]
  • 4.Xu J, Ding Y, Malic V. Author Credit for Transdisciplinary Collaboration. Glanzel W, editor. PLoS ONE. 2015;10: e0137968. doi: 10.1371/journal.pone.0137968 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Laursen BK, Motzer N, Anderson KJ. Pathway profiles: Learning from five main approaches to assessing interdisciplinarity. Research Evaluation. 2022; rvac036. doi: 10.1093/reseval/rvac036 [DOI] [Google Scholar]
  • 6.Specht A, Corrêa P, Belbin L, Loescher HW. Critical research infrastructure: The importance of synthesis centers. Elephant in the Lab. 2020. [cited 12 Jan 2022]. doi: 10.5281/zenodo.3660920 [DOI] [Google Scholar]
  • 7.Fontana M, Iori M, Montobbio F, Sinatra R. New and atypical combinations: An assessment of novelty and interdisciplinarity. Research Policy. 2020;49: 104063. doi: 10.1016/j.respol.2020.104063 [DOI] [Google Scholar]
  • 8.Roy ED, Morzillo AT, Seijo F, Reddy SMW, Rhemtulla JM, Milder JC, et al. The Elusive Pursuit of Interdisciplinarity at the Human—Environment Interface. BioScience. 2013;63: 745–753. doi: 10.1525/bio.2013.63.9.10 [DOI] [Google Scholar]
  • 9.Gates AJ, Ke Q, Varol O, Barabási A-L. Nature’s reach: narrow work has broad impact. Nature. 2019;575: 32–34. doi: 10.1038/d41586-019-03308-7 [DOI] [PubMed] [Google Scholar]
  • 10.Coccia M. The evolution of scientific disciplines in applied sciences: dynamics and empirical properties of experimental physics. Scientometrics. 2020;124: 451–487. doi: 10.1007/s11192-020-03464-y [DOI] [Google Scholar]
  • 11.Sharp PA, Langer R. Promoting Convergence in Biomedical Science. Science. 2011;333: 527–527. doi: 10.1126/science.1205008 [DOI] [PubMed] [Google Scholar]
  • 12.Pluchino A, Burgio G, Rapisarda A, Biondo AE, Pulvirenti A, Ferro A, et al. Exploring the role of interdisciplinarity in physics: Success, talent and luck. Hernandez Montoya AR, editor. PLoS ONE 2019;14: e0218793. doi: 10.1371/journal.pone.0218793 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Hall KL, Stokols D, Stipelman BA, Vogel AL, Feng A, Masimore B, et al. Assessing the Value of Team Science. American Journal of Preventive Medicine. 2012;42: 157–163. doi: 10.1016/j.amepre.2011.10.011 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Harvey S. Creative Synthesis: Exploring the Process of Extraordinary Group Creativity. AMR. 2014;39: 324–343. doi: 10.5465/amr.2012.02241 [DOI] [Google Scholar]
  • 15.Baron JS, Specht A, Garnier E, Bishop P, Campbell CA, Davis FW, et al. Synthesis Centers as Critical Research Infrastructure. BioScience. 2017;67: 750–759. doi: 10.1093/biosci/bix053 [DOI] [Google Scholar]
  • 16.Crowston K, Specht A, Hoover C, Chudoba KM, Watson-Manheim MB. Perceived discontinuities and continuities in transdisciplinary scientific working groups. Science of The Total Environment. 2015;534: 159–172. doi: 10.1016/j.scitotenv.2015.04.121 [DOI] [PubMed] [Google Scholar]
  • 17.Hampton SE, Parker JN. Collaboration and Productivity in Scientific Synthesis. BioScience. 2011;61: 900–910. doi: 10.1525/bio.2011.61.11.9 [DOI] [Google Scholar]
  • 18.Lee Y-N, Walsh JP, Wang J. Creativity in scientific teams: Unpacking novelty and impact. Research Policy. 2015;44: 684–697. doi: 10.1016/j.respol.2014.10.007 [DOI] [Google Scholar]
  • 19.Read EK O ’Rourke M, Hong GS, Hanson PC, Winslow LA, Crowley S, et al. Building the team for team science. Peters DPC, editor. Ecosphere. 2016;7. doi: 10.1002/ecs2.1291 [DOI] [Google Scholar]
  • 20.Woolley AW, Chabris CF, Pentland A, Hashmi N, Malone TW. Evidence for a Collective Intelligence Factor in the Performance of Human Groups. Science. 2010;330: 686–688. doi: 10.1126/science.1193147 [DOI] [PubMed] [Google Scholar]
  • 21.Bachmann R., Trust Power and Control in Trans-Organizational Relations. Organization Studies. 2001;22: 337–365. doi: 10.1177/0170840601222007 [DOI] [Google Scholar]
  • 22.Millar J, Demaid A, Quintas P. Trans-organizational innovation: a framework for research. Technology Analysis & Strategic Management. 1997;9: 399–418. doi: 10.1080/09537329708524294 [DOI] [Google Scholar]
  • 23.Verhoeven D, Cooper T, Flynn M, Shuffler ML. Transnational Team Effectiveness. 1st ed. In: Salas E, Rico R, Passmore J, editors. The Wiley Blackwell Handbook of the Psychology of Team Working and Collaborative Processes. 1st ed. Wiley; 2017. pp. 73–101. doi: 10.1002/9781118909997.ch4 [DOI] [Google Scholar]
  • 24.Horwitz SK, Horwitz IB. The Effects of Team Diversity on Team Outcomes: A Meta-Analytic Review of Team Demography. Journal of Management. 2007;33: 987–1015. doi: 10.1177/0149206307308587 [DOI] [Google Scholar]
  • 25.Woolley AW, Aggarwal I, Malone TW. Collective Intelligence and Group Performance. Curr Dir Psychol Sci. 2015;24: 420–424. doi: 10.1177/0963721415599543 [DOI] [Google Scholar]
  • 26.Bernstein E, Shore J, Lazer D. How intermittent breaks in interaction improve collective intelligence. Proc Natl Acad Sci USA. 2018;115: 8734–8739. doi: 10.1073/pnas.1802407115 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Hackett EJ, Parker JN. From Salomon’s House to Synthesis Centers. In: Heinze T, Münch R, editors. Innovation in Science and Organizational Renewal. New York: Palgrave Macmillan US; 2016. pp. 53–87. doi: 10.1057/978-1-137-59420-4_3 [DOI] [Google Scholar]
  • 28.O’Rourke M, Crowley S, Gonnerman C. On the nature of cross-disciplinary integration: A philosophical framework. Studies in History and Philosophy of Science Part C: Studies in History and Philosophy of Biological and Biomedical Sciences. 2016;56: 62–70. doi: 10.1016/j.shpsc.2015.10.003 [DOI] [PubMed] [Google Scholar]
  • 29.Stokols D, Fuqua J, Gress J, Harvey R, Phillips K, Baezconde-Garbanati L, et al. Evaluating transdisciplinary science. Nicotine & Tobacco Res. 2003;5: 21–39. doi: 10.1080/14622200310001625555 [DOI] [PubMed] [Google Scholar]
  • 30.Lynch AJJ, Thackway R, Specht A, Beggs PJ, Brisbane S, Burns EL, et al. Transdisciplinary synthesis for ecosystem science, policy and management: The Australian experience. Science of The Total Environment. 2015;534: 173–184. doi: 10.1016/j.scitotenv.2015.04.100 [DOI] [PubMed] [Google Scholar]
  • 31.Hackman J.R., 1977. Work design. In: Hackman JR, Suttle JL, editors. Improving life at work: behavioral science approaches to organizational change. Santa Monica, Calif: Goodyear Pub. Co; 1977. [Google Scholar]
  • 32.Bammer G O ’Rourke M, O’Connell D, Neuhauser L, Midgley G, Klein JT, et al. Expertise in research integration and implementation for tackling complex problems: when is it needed, where can it be found and how can it be strengthened? Palgrave Commun. 2020;6: 5. doi: 10.1057/s41599-019-0380-0 [DOI] [Google Scholar]
  • 33.Abramo G, D’Angelo CA, Zhang L. A comparison of two approaches for measuring interdisciplinary research output: The disciplinary diversity of authors vs the disciplinary diversity of the reference list. Journal of Informetrics. 2018;12: 1182–1193. doi: 10.1016/j.joi.2018.09.001 [DOI] [Google Scholar]
  • 34.Dusdal J, Powell JJW. Benefits, Motivations, and Challenges of International Collaborative Research: A Sociology of Science Case Study. Science and Public Policy. 2021;48: 235–245. doi: 10.1093/scipol/scab010 [DOI] [Google Scholar]
  • 35.Stvilia B, Hinnant CC, Schindler K, Worrall A, Burnett G, Burnett K, et al. Composition of scientific teams and publication productivity at a national science lab. J Am Soc Inf Sci. 2011;62: 270–283. doi: 10.1002/asi.21464 [DOI] [Google Scholar]
  • 36.De Saá-Pérez P, Díaz-Díaz NL, Aguiar-Díaz I, Ballesteros-Rodríguez JL. How diversity contributes to academic research teams performance: Diversity in academic research teams performance. R&D Management. 2017;47: 165–179. doi: 10.1111/radm.12139 [DOI] [Google Scholar]
  • 37.Campbell LG, Mehtani S, Dozier ME, Rinehart J. Gender-Heterogeneous Working Groups Produce Higher Quality Science. Larivière V, editor. PLoS ONE. 2013;8: e79147. doi: 10.1371/journal.pone.0079147 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Meslec N, Graff D. Being open matters: the antecedents and consequences of cross-understanding in teams. Team Performance Management. 2015;21: 6–18. doi: 10.1108/TPM-10-2014-0055 [DOI] [Google Scholar]
  • 39.van Rijnsoever FJ, Hessels LK. Factors associated with disciplinary and interdisciplinary research collaboration. Research Policy. 2011;40: 463–472. doi: 10.1016/j.respol.2010.11.001 [DOI] [Google Scholar]
  • 40.Bozeman B, Gaughan M. How do men and women differ in research collaborations? An analysis of the collaborative motives and strategies of academic researchers. Research Policy. 2011;40: 1393–1402. doi: 10.1016/j.respol.2011.07.002 [DOI] [Google Scholar]
  • 41.Curşeu PL, Pluut H, Boroş S, Meslec N. The magic of collective emotional intelligence in learning groups: No guys needed for the spell! Br J Psychol. 2015;106: 217–234. doi: 10.1111/bjop.12075 [DOI] [PubMed] [Google Scholar]
  • 42.Xie L, Zhou J, Zong Q, Lu Q. Gender diversity in R&D teams and innovation efficiency: Role of the innovation context. Research Policy. 2020;49: 103885. doi: 10.1016/j.respol.2019.103885 [DOI] [Google Scholar]
  • 43.Landry R, Traore N, Godin B. An econometric analysis of the effect of collaboration on academic research productivity. High Educ. 1996;32: 283–301. doi: 10.1007/BF00138868 [DOI] [Google Scholar]
  • 44.Fox MFrank Mohapatra Sushanta. Social-Organizational Characteristics of Work and Publication Productivity among Academic Scientists in Doctoral-Granting Departments. The Journal of Higher Education. 2007;78: 542–571. doi: 10.1353/jhe.2007.0032 [DOI] [Google Scholar]
  • 45.Leahey E, Beckman CM, Stanko TL. Prominent but Less Productive: The Impact of Interdisciplinarity on Scientists’ Research. Administrative Science Quarterly. 2017;62: 105–139. doi: 10.1177/0001839216665364 [DOI] [Google Scholar]
  • 46.Belkhouja M, Fattoum S, Yoon H David. Does greater diversification increase individual productivity? The moderating effect of attention allocation. Research Policy. 2021;50: 104256. doi: 10.1016/j.respol.2021.104256 [DOI] [Google Scholar]
  • 47.Barjak F, Robinson S. International collaboration, mobility and team diversity in the life sciences: impact on research performance. Soc Geogr. 2008;3: 23–36. doi: 10.5194/sg-3-23-2008 [DOI] [Google Scholar]
  • 48.Maddi A, Gingras Y. Gender diversity in research teams and citation impact in economics and management. Journal of Economic Surveys. 2021; joes.12420. doi: 10.1111/joes.12420 [DOI] [Google Scholar]
  • 49.Hackett EJ, Leahey E, Parker JN, Rafols I, Hampton SE, Corte U, et al. Do synthesis centers synthesize? A semantic analysis of topical diversity in research. Research Policy. 2021;50: 104069. doi: 10.1016/j.respol.2020.104069 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Chen S, Qiu J, Arsenault C, Larivière V. Exploring the interdisciplinarity patterns of highly cited papers. Journal of Informetrics. 2021;15: 101124. doi: 10.1016/j.joi.2020.101124 [DOI] [Google Scholar]
  • 51.Larivière V, Haustein S, Börner K. Long-Distance Interdisciplinarity Leads to Higher Scientific Impact. Glanzel W, editor. PLoS ONE. 2015;10: e0122565. doi: 10.1371/journal.pone.0122565 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52.Steele TW, Stier JC. The impact of interdisciplinary research in the environmental sciences: a forestry case study. Journal of the American Society for Information Science. 2000;51: 476–484. doi: [DOI] [Google Scholar]
  • 53.Belkhouja M, Yoon H David. How does openness influence the impact of a scholar’s research? An analysis of business scholars’ citations over their careers. Research Policy. 2018;47: 2037–2047. doi: 10.1016/j.respol.2018.07.012 [DOI] [Google Scholar]
  • 54.Whitfield J. Collaboration: Group theory. Nature. 2008;455: 720–723. doi: 10.1038/455720a [DOI] [PubMed] [Google Scholar]
  • 55.Schoonenboom J, Johnson RB. How to Construct a Mixed Methods Research Design. Köln Z Soziol. 2017;69: 107–131. doi: 10.1007/s11577-017-0454-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56.Twyman M, Contractor N. Team Assembly. In: Hall KL, Vogel AL, Croyle RT, editors. Strategies for Team Science Success: Handbook of Evidence-Based Principles for Cross-Disciplinary Science and Practical Lessons Learned from Health Researchers. Cham: Springer International Publishing; 2019. pp. 217–240. doi: 10.1007/978-3-030-20992-6_17 [DOI] [Google Scholar]
  • 57.Palmer MA, Kramer JG, Boyd J, Hawthorne D. Practices for facilitating interdisciplinary synthetic research: the National Socio-Environmental Synthesis Center (SESYNC). Current Opinion in Environmental Sustainability. 2016;19: 111–122. doi: 10.1016/j.cosust.2016.01.002 [DOI] [Google Scholar]
  • 58.Michener WK, Allard S, Budden A, Cook RB, Douglass K, Frame M, et al. Participatory design of DataONE—Enabling cyberinfrastructure for the biological and environmental sciences. Ecological Informatics. 2012;11: 5–15. doi: 10.1016/j.ecoinf.2011.08.007 [DOI] [Google Scholar]
  • 59.Halpern BS, Berlow E, Williams R, Borer ET, Davis FW, Dobson A, et al. Ecological Synthesis and Its Role in Advancing Knowledge. BioScience. 2020; biaa105. doi: 10.1093/biosci/biaa105 [DOI] [Google Scholar]
  • 60.Specht A., Synthesis centers: their relevance to and importance in the Anthropocene. In: Chabbi A, Loescher HW. Terrestrial ecosystem research infrastructures: challenges and opportunities. 2017. Available: http://www.crcnetbase.com/isbn/9781498751339 [Google Scholar]
  • 61.Bergek A, Norrman C. Incubator best practice: A framework. Technovation. 2008;28: 20–28. doi: 10.1016/j.technovation.2007.07.008 [DOI] [Google Scholar]
  • 62.Rodrigo A, Alberts S, Cranston K, Kingsolver J, Lapp H, McClain C, et al. Science Incubators: Synthesis Centers and Their Role in the Research Ecosystem. PLoS Biol. 2013;11: e1001468. doi: 10.1371/journal.pbio.1001468 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 63.Hackett E.J., Parker J.N., Conz D., Rhoten D., Parker A., Ecology transformed: NCEAS and changing patterns of ecological research. In: Olson G.M., Zimmerman A., Bos N. (Eds.), Scientific collaboration on the Internet, Acting with technology. MIT Press, Cambridge, Mass.; 2008. pp. 277–296. [Google Scholar]
  • 64.Australian and New Zealand Standard Research Classification (ANZSRC), 2020 | Australian Bureau of Statistics. 30 Jun 2020. [cited 5 Jul 2021]. Available: https://www.abs.gov.au/statistics/classifications/australian-and-new-zealand-standard-research-classification-anzsrc/latest-release (accessed 7.5.21). [Google Scholar]
  • 65.Wagner CS, Roessner JD, Bobb K, Klein JT, Boyack KW, Keyton J, et al. Approaches to understanding and measuring interdisciplinary scientific research (IDR): A review of the literature. Journal of Informetrics. 2011;5: 14–26. doi: 10.1016/j.joi.2010.06.004 [DOI] [Google Scholar]
  • 66.Leydesdorff L, Rafols I. Indicators of the interdisciplinarity of journals: Diversity, centrality, and citations. Journal of Informetrics. 2011;5: 87–100. doi: 10.1016/j.joi.2010.09.002 [DOI] [Google Scholar]
  • 67.Rafols I, Leydesdorff L, O’Hare A, Nightingale P, Stirling A. How journal rankings can suppress interdisciplinary research: A comparison between Innovation Studies and Business & Management. Research Policy. 2012;41: 1262–1282. doi: 10.1016/j.respol.2012.03.015 [DOI] [Google Scholar]
  • 68.Marx W, Bornmann L. Change of perspective: bibliometrics from the point of view of cited references—a literature overview on approaches to the evaluation of cited references in bibliometrics. Scientometrics. 2016;109: 1397–1415. doi: 10.1007/s11192-016-2111-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 69.Leydesdorff L, Wagner CS, Bornmann L. Betweenness and diversity in journal citation networks as measures of interdisciplinarity—A tribute to Eugene Garfield. Scientometrics. 2018;114: 567–592. doi: 10.1007/s11192-017-2528-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 70.Qin J, Lancaster FW, Allen B. Types and levels of collaboration in interdisciplinary research in the sciences. Journal of the American Society for Information Science. 1997;48: 893–916. [DOI] [Google Scholar]
  • 71.Huang L, Cai Y, Zhao E, Zhang S, Shu Y, Fan J. Measuring the interdisciplinarity of Information and Library Science interactions using citation analysis and semantic analysis. Scientometrics. 2022. [cited 26 Jul 2022]. doi: 10.1007/s11192-022-04401-x [DOI] [Google Scholar]
  • 72.Shu F, Dinneen JD, Chen S. Measuring the disparity among scientific disciplines using Library of Congress Subject Headings. Scientometrics. 2022;127: 3613–3628. doi: 10.1007/s11192-022-04387-6 [DOI] [Google Scholar]
  • 73.Craven D, Winter M, Hotzel K, Gaikwad J, Eisenhauer N, Hohmuth M, et al. Evolution of interdisciplinarity in biodiversity science. Ecol Evol. 2019;9: 6744–6755. doi: 10.1002/ece3.5244 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 74.Kerr NL, Tindale RS. Group Performance and Decision Making. Annu Rev Psychol. 2004;55: 623–655. doi: 10.1146/annurev.psych.55.090902.142009 [DOI] [PubMed] [Google Scholar]
  • 75.Stanley T, Matthews J, Davidson P. Insights from Stimulating Creative Behaviours in a Project-Based Organization Team. Technology Innovation Management Review. 2016;6: 26–33. doi: 10.22215/timreview/979 [DOI] [Google Scholar]
  • 76.Yegros-Yegros A, Rafols I, D’Este P. Does Interdisciplinary Research Lead to Higher Citation Impact? The Different Effect of Proximal and Distal Interdisciplinarity. Glanzel W, editor. PLoS ONE. 2015;10: e0135095. doi: 10.1371/journal.pone.0135095 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 77.Spellerberg IF, Fedor PJ. A tribute to Claude Shannon (1916–2001) and a plea for more rigorous use of species richness, species diversity and the ‘Shannon-Wiener’ Index: On species richness and diversity. Global Ecology and Biogeography. 2003;12: 177–179. doi: 10.1046/j.1466-822X.2003.00015.x [DOI] [Google Scholar]
  • 78.Aydinoglu AU, Allard S, Mitchell C. Measuring diversity in disciplinary collaboration in research teams: An ecological perspective. Research Evaluation. 2016;25: 18–36. doi: 10.1093/reseval/rvv028 [DOI] [Google Scholar]
  • 79.Wang J, Thijs B, Glänzel W. Interdisciplinarity and Impact: Distinct Effects of Variety, Balance, and Disparity. Smalheiser NR, editor. PLoS ONE. 2015;10: e0127298. doi: 10.1371/journal.pone.0127298 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 80.Uzzi B, Mukherjee S, Stringer M, Jones B. Atypical Combinations and Scientific Impact. Science. 2013;342: 468–472. doi: 10.1126/science.1240474 [DOI] [PubMed] [Google Scholar]
  • 81.Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Research in Psychology. 2006;3: 77–101. doi: 10.1191/1478088706qp063oa [DOI] [Google Scholar]
  • 82.Patience GS, Patience CA, Blais B, Bertrand F. Citation analysis of scientific categories. Heliyon. 2017;3: e00300. doi: 10.1016/j.heliyon.2017.e00300 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 83.Specht A, Crowston K. Analysis of papers produced and cited by synthesis center-type groups in the USA and France, 2010–2020, and open-source feedback from those groups. Environmental Data Initiative; 2022. doi: 10.6073/PASTA/C5033B64FC57E2B3DA45C8F8541BD98A [DOI] [Google Scholar]
  • 84.Srivastava DS, Winter M, Gross LJ, Metzger JP, Baron JS, Mouquet N, et al. Maintaining momentum for collaborative working groups in a post-pandemic world. Nat Ecol Evol. 2021; 1–2. doi: 10.1038/s41559-021-01521-0 [DOI] [PubMed] [Google Scholar]
  • 85.Salazar M, Salas E. Reflections of Cross-Cultural Collaboration Science: REFLECTIONS OF CROSS-CULTURAL COLLABORATION SCIENCE. J Organiz Behav. 2013;34: 910–917. doi: 10.1002/job.1881 [DOI] [Google Scholar]
  • 86.Kozlowski SWJ, Mak S, Chao GT. Team-Centric Leadership: An Integrative Review. Annu Rev Organ Psychol Organ Behav. 2016;3: 21–54. doi: 10.1146/annurev-orgpsych-041015-062429 [DOI] [Google Scholar]
  • 87.Parker JN, Cardenas E, Dorr AN, Hackett EJ. Using Sociometers to Advance Small Group Research. Sociological Methods & Research. 2020;49: 1064–1102. doi: 10.1177/0049124118769091 [DOI] [Google Scholar]
  • 88.Bozeman B, Boardman C. Research Collaboration and Team Science. Cham: Springer International Publishing; 2014. doi: 10.1007/978-3-319-06468-0 [DOI] [Google Scholar]

Decision Letter 0

Sergi Lozano

6 Apr 2022

PONE-D-22-04001Interdisciplinary collaboration from diverse science teams can produce significant outcomesPLOS ONE

Dear Dr. Specht,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. As you can see below, the reviewers focused their reports on complementary issues. Reviewer 1, raised a number of issues regarding both quantitative and qualitative analysis (please, notice PLOS ONE's publiucation criterion #3, https://journals.plos.org/plosone/s/criteria-for-publication#loc-3). Reviewer 2 is more concerned about interdisciplinarity's conceptualisation in the article. These are relevant issues that should be addressed in the revision of the paper.

Please submit your revised manuscript by May 21 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Sergi Lozano

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at 

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. In your Methods section, please provide additional information about the participant recruitment method and the demographic details of your participants. Please ensure you have provided sufficient details to replicate the analyses such as: a) the recruitment date range (month and year), b) a description of any inclusion/exclusion criteria that were applied to participant recruitment, c) a table of relevant demographic details, d) a statement as to whether your sample can be considered representative of a larger population, e) a description of how participants were recruited, and f) descriptions of where participants were recruited and where the research took place.

3. In your Data Availability statement, you have not specified where the minimal data set underlying the results described in your manuscript can be found. PLOS defines a study's minimal data set as the underlying data used to reach the conclusions drawn in the manuscript and any additional data required to replicate the reported study findings in their entirety. All PLOS journals require that the minimal data set be made fully available. For more information about our data policy, please see http://journals.plos.org/plosone/s/data-availability.

Upon re-submitting your revised manuscript, please upload your study’s minimal underlying data set as either Supporting Information files or to a stable, public repository and include the relevant URLs, DOIs, or accession numbers within your revised cover letter. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories. Any potentially identifying patient information must be fully anonymized.

Important: If there are ethical or legal restrictions to sharing your data publicly, please explain these restrictions in detail. Please see our guidelines for more information on what we consider unacceptable restrictions to publicly sharing data: http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. Note that it is not acceptable for the authors to be the sole named individuals responsible for ensuring data access.

We will update your Data Availability statement to reflect the information you provide in your cover letter.

4. Please note that in order to use the direct billing option the corresponding author must be affiliated with the chosen institute. Please either amend your manuscript to change the affiliation or corresponding author, or email us at plosone@plos.org with a request to remove this option.

5. Please include your full ethics statement in the ‘Methods’ section of your manuscript file. In your statement, please include the full name of the IRB or ethics committee who approved or waived your study, as well as whether or not you obtained informed written or verbal consent. If consent was waived for your study, please include this information in your statement as well.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: No

Reviewer #2: I Don't Know

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The study attempts to demonstrate how demographic diversity contributes to the interdisciplinary of collaborations and then contributes to perceptions of satisfaction and effectiveness of teams. A major contribution appears to be in its positioning as a mixed-method study. However, there are flaws with both the quantitative and qualitative aspects of the analysis. Additionally, the paper can be greatly strengthened with the inclusion of more theoretical development and analyses at the individual level. In general, the paper makes reasonable claims, but the argument needs to be strengthened and the analysis needs to be more convincing. The claims are significant, but the novelty is not particularly clear.

The paper does not effectively use literature to contextualize the research. There is little inclusion of logical mechanisms to explain the hypotheses, which reduces the potential contributions of the study. There is not much theoretical justification given to explain why these hypotheses will be present nor is there much discussion regarding the extent to which the literature provides evidence for alternative hypotheses. Specifically, there are limited references in all of Section 2.1, which diminishes the effectiveness of the hypotheses.

Sections 3 and 4 are the areas where much improvement needs to be made to help the paper. Firstly, the description of the research setting was helpful for establishing expectations and clarifying the collaborative environment. However, the variable operationalization has some opportunities for improvement. The gender balance variable does not distinguish whether there is a majority male or majority female team; either category can potentially have the same gender balance score. Another variable, such as the proportion of male or female in the group could be developed to make interpretation clearer. Currently, it seems as though information is being lost. Also, I appreciate using the median number of citations as a measure, and I would also recommend including the mean as well for a robustness check.

The explanation of the survey measures is insufficient. How many items comprise each construct? What are the items? How closely related are the items? Instead of only the mean value in the group, I would also recommend investigating the variance of responses as well. Much more information is needed to help the later interpretations of analysis given that these are group outcomes.

Once results were being presented, more analysis should be included to help support the claims. Overall, the authors mention the limitations that arise when performing statistical analyses on small sample sizes. However, I challenge the justification of the analysis on the self-reported data. There appears to be an opportunity for the researcher to conduct analysis at the individual respondent level, which would allow for the utilization of multilevel regressions where the individual responses can be nested within the group structure. Alternatively, since the group has been the level of analysis for the paper, modeling the group responses by using cluster robust standard errors would also be another approach to account for groups while analyzing the individual data. Therefore, supplementing the current analysis with analysis at the individual level would potentially improve the contributions from the paper. Additionally, providing two descriptive statistics tables would help the reader understand the group and individual data under study. To reiterate, the group level is the focus of analysis, but the lack of data makes it difficult to prove the claims, and I suggest investigating individual-level analysis to increase confidence in the analysis of self-reported data.

The Qualitative portion of the analysis was limited. It is not clear what how the inductive coding led to the emergence of themes since the presented results do not represent concepts that can be connected back to specific theoretical arguments. Additionally, there is no indication of the frequency or number of occurrences for the topics in the data. In how many teams did each of these types of findings occur? How many people included similar thoughts? Currently, the “categories” appear to be stable, and it is challenging for a reader deepen their understanding of the collaboration process.

In summary, there are positives with the paper, but there needs to be more theoretical argumentation from literature and the analysis should be supplemented to improve the evidence supporting the claims.

Reviewer #2: This is a good paper that adds to the literature on interdisciplinarity. The use of the input-process-output model to think about the role of diversity in interdisciplinary collaboration is helpful and illuminating.

There are a few limitations, though, that should be addressed in a revision before the paper is published. Principal among these is the way in which interdisciplinary collaboration is conceptualized in this study. Using the disciplinary diversity of publication venues and the disciplinary diversity of cited articles according to the journals they appear in are very blunt instruments for assessing the process of interdisciplinary collaboration, for a host of reasons. Given that interdisciplinary process is a central theme of this paper, much more time should be spent in convincing the reader that this is a valid way of conceptualizing it.

Also, I am not in a position to evaluate the statistical work in the paper, so some of my comments may be off-target because of my ignorance about this aspect of the paper.

My comments/concerns, by line(s):

l. 35. You take working groups to be a type of team. Can you say more about how a working group compares to the typical research team that is the focus of work in, say, the science of team science? These would be cohesive, interdependent groups of researchers typically with a leader or leaders and a common objective. Are you using the term ‘working group’ in the way it is used at, e.g., NCEAS? Or is there not a difference between a working group and a research team, on your way of thinking about them?

l. 87. Use of the input-process-output model is clarifying and helpful. Another discussion of interdisciplinarity where it occurs (and specifically, crossdisciplinary integration) is in O’Rourke et al. (2016): “On the nature of cross-disciplinary integration: A philosophical framework”.

ll. 95ff. In general, research teams are not composed in this way – many are pre-existing and find new problems, or are formed out of personal affinity before a problem is identified. (For a good discussion of the range, see Twyman & Contractor (2019): “Team assembly”.) Is this perhaps an answer to the question on l. 35, i.e., what are the differences between working groups and research teams? Is there a precedent for distinguishing these categories in this way?

l. 235. Focusing on the “collation and analysis of articles used and produced” seems like a very limited way of assessing the process of interdisciplinary collaboration. Can you explain and defend why you chose to do it this way?

l. 260. How was it determined whether a published paper was a paper published by a group? People may publish with others in a group and not have it be a group paper. Were these papers listed by the groups as group publications? What were the criteria here?

ll. 262ff. Given the limitations associated with CrossREF, why not use a broader and more inclusive platform, e.g., Google Scholar?

ll. 265ff. This is a potentially suggestive measure of the interdisciplinarity of an article, but why take it to be a measure of the interdisciplinarity of a collaboration? Surely an article could have a diversity of disciplines represented in its References section, but where that fact does not correspond to interdisciplinary process in the collaboration. For example, the co-authors could be responsible for introducing literature from their disciplines without there being any interdisciplinary collaboration involved. The same is true for the output measure – a group might decide to publish in a range of journals so that teammates can receive credit in their home departments by publishing in a journal the department recognizes, without that entailing any actual disciplinary integration in the production of those papers. Since your decision to center interdisciplinary process is really the crux of this paper, you need to develop and defend this choice in more detail, IMO. And you should also address this as a limitation in the discussion.

l. 295. Did you ask any questions about interdisciplinary process in the survey? Did you receive any open-ended responses that had a bearing on how you understand interdisciplinary process?

l. 309. What was the overall response rate to the survey?

ll. 465ff. Why did you choose the A groups as the control for the group factor? (And I guess I’m not sure what you mean by “group factor” here?)

ll. 519ff. For readability, I would encourage you to use a colon or comma to separate your quote introductions from the quotes themselves, e.g., ll. 535-7 where I was tempted by the lack of punctuation to read that as one long (and very awkward) sentence.

l. 636. Were there really no changes in the working groups over time that affected their diversity? If there were, how does that affect what you’re saying here? Does it have any impact on your analyses in this paper?

l. 655. Maybe supply a couple of examples of these suggestions?

l. 670. Might it not take more time for a working group that publishes widely to have impact? How can you distinguish between differences in impact vs. differences in rate of impact?

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2022 Nov 29;17(11):e0278043. doi: 10.1371/journal.pone.0278043.r002

Author response to Decision Letter 0


13 Sep 2022

I have uploaded a comprehensive (I hope) response to reviewers and editor's document. It is formatted as a table.

Response to reviewers

editor’s comments

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

We believe we did meet this and have further checked our formatting, moving Appendices to Supplementary material, among other things. Further specifics will be complied with if we have overlooked them.

2. In your Methods section, please provide additional information about the participant recruitment method and the demographic details of your participants. Please ensure you have provided sufficient details to replicate the analyses such as: a) the recruitment date range (month and year), b) a description of any inclusion/exclusion criteria that were applied to participant recruitment, c) a table of relevant demographic details, d) a statement as to whether your sample can be considered representative of a larger population, e) a description of how participants were recruited, and f) descriptions of where participants were recruited and where the research took place.

We have added the requested information in the methods and results section. The first Appendix (S1) has relevant demographic details.

3. In your Data Availability statement, you have not specified where the minimal data set underlying the results described in your manuscript can be found. PLOS defines a study's minimal data set as the underlying data used to reach the conclusions drawn in the manuscript and any additional data required to replicate the reported study findings in their entirety. All PLOS journals require that the minimal data set be made fully available. For more information about our data policy, please see http://journals.plos.org/plosone/s/data-availability

Upon re-submitting your revised manuscript, please upload your study’s minimal underlying data set as either Supporting Information files or to a stable, public repository and include the relevant URLs, DOIs, or accession numbers within your revised cover letter. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories. Any potentially identifying patient information must be fully anonymized.

Important: If there are ethical or legal restrictions to sharing your data publicly, please explain these restrictions in detail. Please see our guidelines for more information on what we consider unacceptable restrictions to publicly sharing data: http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. Note that it is not acceptable for the authors to be the sole named individuals responsible for ensuring data access.

We will update your Data Availability statement to reflect the information you provide in your cover letter.

We have made a statement in the body of the paper about the ethics conditions related to the data. This restricts us with respect to the full identity of the respondents, their journal publications and the centres with which they were associated.

We have uploaded the Appendices as Supporting Information (Appendices), and the data as mentioned.

4. Please note that in order to use the direct billing option the corresponding author must be affiliated with the chosen institute. Please either amend your manuscript to change the affiliation or corresponding author, or email us at plosone@plos.org with a request to remove this option. The corresponding author has been changed and is now Prof. Kevin Crowston of Syracuse University.

5. Please include your full ethics statement in the ‘Methods’ section of your manuscript file. In your statement, please include the full name of the IRB or ethics committee who approved or waived your study, as well as whether or not you obtained informed written or verbal consent. If consent was waived for your study, please include this information in your statement as well. We have included a section in the Methods as an introduction to the IRB conditions for the whole project.

reviewer comments

Reviewer 1

1.The study attempts to demonstrate how demographic diversity contributes to the interdisciplinary of collaborations and then contributes to perceptions of satisfaction and effectiveness of teams.

A major contribution appears to be in its positioning as a mixed-method study. However, there are flaws with both the quantitative and qualitative aspects of the analysis. Additionally, the paper can be greatly strengthened with the inclusion of more theoretical development and analyses at the individual level. In general, the paper makes reasonable claims, but the argument needs to be strengthened and the analysis needs to be more convincing. The claims are significant, but the novelty is not particularly clear. Yes, one aspect of the paper is to assess demographic diversity and its effect on interdisciplinary processes. Assuming disciplinary diversity of group members is a demographic metric, then yes.

We address the detailed suggestions below.

2.The paper does not effectively use literature to contextualize the research. There is little inclusion of logical mechanisms to explain the hypotheses, which reduces the potential contributions of the study. There is not much theoretical justification given to explain why these hypotheses will be present nor is there much discussion regarding the extent to which the literature provides evidence for alternative hypotheses. Specifically, there are limited references in all of Section 2.1, which diminishes the effectiveness of the hypotheses. We use the input-process-output model to structure our selection of factors and their relationships. We have added to the discussion of the hypotheses and do indicate the mechanism by which the chosen group composition factors are believed to affect the group process and for why the group process should affect the outcomes. Section 2.1 now includes approximately 30 citations to past studies supporting the hypotheses. There are of course additional studies that suggest alternative explanations, but we did not think it useful to include hypotheses that are beyond our focus on interdisciplinarity or for which we do not have data.

3. Sections 3 and 4 are the areas where much improvement needs to be made to help the paper. Firstly, the description of the research setting was helpful for establishing expectations and clarifying the collaborative environment.

However, the variable operationalization has some opportunities for improvement.

The gender balance variable does not distinguish whether there is a majority male or majority female team; either category can potentially have the same gender balance score. Another variable, such as the proportion of male or female in the group could be developed to make interpretation clearer. Currently, it seems as though information is being lost. Also, I appreciate using the median number of citations as a measure, and I would also recommend including the mean as well for a robustness check. As suggested, we now analyze the proportion of females in the group, which is a measure that has been used in other studies of group dynamics. The main results are unchanged.

As requested, we ran a regression to predict average citations instead of median. As average is a non-robust measure, the data have more outliers, which required a robust regression. The data were not over-dispersed, so we used a Poisson regression. In this regression, age was still a significant predictor (older groups are cited more on average, as would be expected) and there were significant differences among the centres in average citation. Publication and citation diversity still showed a curvilinear relationship (more citations for moderate levels of diversity). Since the results are qualitatively similar but require a more complex test, we feel it is preferable to report the results for median citations.

4. The explanation of the survey measures is insufficient. How many items comprise each construct? What are the items? How closely related are the items?

Instead of only the mean value in the group, I would also recommend investigating the variance of responses as well. Much more information is needed to help the later interpretations of analysis given that these are group outcomes. We only had a few survey measures to include in the analysis. We have added text to section 3.2.3 to explain the measures used. We had originally conceptualized perceived effectiveness and satisfaction as being sub-items of a measure of group performance, along with output, but when we discovered they were not well correlated we decided to treat them separately. However, we had only included a single item for each. This is because the questions were part of a longer survey (many of the survey questions were not on the topic of interdisciplinarity so were not analysed in this paper). Because the survey was quite long we considered that having too many items for scales would be problematic for return rate.

We note that Appendix 3 giving the variances (standard errors) of the measures was omitted from the paper, and it has been re-inserted.

5. Once results were being presented, more analysis should be included to help support the claims. Overall, the authors mention the limitations that arise when performing statistical analyses on small sample sizes. However, I challenge the justification of the analysis on the self-reported data. There appears to be an opportunity for the researcher to conduct analysis at the individual respondent level, which would allow for the utilization of multilevel regressions where the individual responses can be nested within the group structure.

Alternatively, since the group has been the level of analysis for the paper, modeling the group responses by using cluster robust standard errors would also be another approach to account for groups while analyzing the individual data. Therefore, supplementing the current analysis with analysis at the individual level would potentially improve the contributions from the paper. Additionally, providing two descriptive statistics tables would help the reader understand the group and individual data under study.

To reiterate, the group level is the focus of analysis, but the lack of data makes it difficult to prove the claims, and I suggest investigating individual-level analysis to increase confidence in the analysis of self-reported data. We re-examined the data we had on individuals with an eye towards carrying out an individual-level analysis. However, we note that the constructs in our model are all defined at the group level and do not seem meaningful at the individual level. For instance, it is not clear what it would mean for an individual to have an interdisciplinary group or process by him or herself, nor how to measure it in a way that does not end up as a group measure. We therefore have kept the focus in the paper at the group level.

6. The Qualitative portion of the analysis was limited. It is not clear what how the inductive coding led to the emergence of themes since the presented results do not represent concepts that can be connected back to specific theoretical arguments.

Additionally, there is no indication of the frequency or number of occurrences for the topics in the data. In how many teams did each of these types of findings occur? How many people included similar thoughts? Currently, the “categories” appear to be stable, and it is challenging for a reader [to] deepen their understanding of the collaboration process. We have revised the presentation of the qualitative results to make it clearer where the comments connect to concepts in our original theoretical model or to related concepts (e.g., other kinds of demographic diversity beyond what we measured). In addition, the comments revealed additional concerns that went beyond our model, e.g., the importance of leadership, which we report for completeness even though they are not related to diversity or interdisciplinarity.

We do not believe that turning qualitative data into quantitative numbers is appropriate. Since respondents were choosing what topics to mention, there’s no reason to believe that a theme being mentioned more often translates directly to it being more important, e.g., the lack of mention of the impact of gender, despite prior research and our results showing its impact. However, we chose quotations uniformly across the themes, meaning that there is a relation between how often a theme appeared in the data and how often we include a quotation for the theme.

7. In summary, there are positives with the paper, but there needs to be more theoretical argumentation from literature and the analysis should be supplemented to improve the evidence supporting the claims. We hope that the added theoretical argumentation and analysis addresses your concerns.

reviewer 2

1.This is a good paper that adds to the literature on interdisciplinarity. The use of the input-process-output model to think about the role of diversity in interdisciplinary collaboration is helpful and illuminating.

There are a few limitations, though, that should be addressed in a revision before the paper is published. Principal among these is the way in which interdisciplinary collaboration is conceptualized in this study. Using the disciplinary diversity of publication venues and the disciplinary diversity of cited articles according to the journals they appear in are very blunt instruments for assessing the process of interdisciplinary collaboration, for a host of reasons. Given that interdisciplinary process is a central theme of this paper, much more time should be spent in convincing the reader that this is a valid way of conceptualizing it. Thank you for your interest.

We have added text at the start of section 3.2.2 about the rationale for using the bibliometric method.

We have added a detailed analysis in 4.1.2 of two (three) groups that score high or low in the bibliographic diversity scale. We have added a comment about the per article citation rate.

2. Also, I am not in a position to evaluate the statistical work in the paper, so some of my comments may be off-target because of my ignorance about this aspect of the paper. We have kept your disclaimer in mind in responding to your suggestions.

l. 35. You take working groups to be a type of team. Can you say more about how a working group compares to the typical research team that is the focus of work in, say, the science of team science? These would be cohesive, interdependent groups of researchers typically with a leader or leaders and a common objective. Are you using the term ‘working group’ in the way it is used at, e.g., NCEAS? Or is there not a difference between a working group and a research team, on your way of thinking about them? We have revised the Introduction to address the characteristics of the working groups as scientific teams, in paragraph one, line 3 onwards.

We have added some text and a figure in section 3.1 Research Setting, to better illustrate the scenario and explain, hopefully better, the voluntary and part-time nature of the collaboration being studied.

l. 87. Use of the input-process-output model is clarifying and helpful. Another discussion of interdisciplinarity where it occurs (and specifically, cross disciplinary integration) is in O’Rourke et al. (2016): “On the nature of cross-disciplinary integration: A philosophical framework”. Thank you for highlighting this reference for us.

We have added it as a reference with respect to grand challenges. But this thoughtful paper has been added to by Bammer et al., 2020, which has also been added.

ll. 95ff. In general, research teams are not composed in this way – many are pre-existing and find new problems, or are formed out of personal affinity before a problem is identified. (For a good discussion of the range, see Twyman & Contractor (2019): “Team assembly”.) Is this perhaps an answer to the question on l. 35, i.e., what are the differences between working groups and research teams? Is there a precedent for distinguishing these categories in this way? This is a most helpful point.

There is a quote right at the start of Twyman and Contractor’s section “An outside authority is responsible for the performance of a staffed team, while self-assembled teams are responsible for their own success.” This is not quite the same for these synthesis groups, as the synthesis centre/dataone are keen to see outcome, but it is getting there. We have added a comment.

l. 235. Focusing on the “collation and analysis of articles used and produced” seems like a very limited way of assessing the process of interdisciplinary collaboration. Can you explain and defend why you chose to do it this way? We have added text arguing for this method at the start of 3.2.2.

l. 260. How was it determined whether a published paper was a paper published by a group? People may publish with others in a group and not have it be a group paper. Were these papers listed by the groups as group publications? What were the criteria here? Good question.

(a) the groups and the centre/DataONE management team kept a close eye on the publications (e.g., providing lists of papers by group), and (b) the teams may have occasionally brought other authors into a publication, but the initiative was the groups and the majority of authors were from the group.

ll. 262ff. Given the limitations associated with CrossREF, why not use a broader and more inclusive platform, e.g., Google Scholar? We could have used several sources and created a system of amalgamating them. However, it is, in our experience, unlikely to change the pattern of the results. We have queried several publishing house colleagues (e.g., DataCite) and CrossREF is accepted as pretty comprehensive for English-language articles. In other words, as long as the source of citation data is not systematically biased, then the limitations should not change the results. We have added an explanation to this paragraph.

ll. 265ff. This is a potentially suggestive measure of the interdisciplinarity of an article, but why take it to be a measure of the interdisciplinarity of a collaboration? Surely an article could have a diversity of disciplines represented in its References section, but where that fact does not correspond to interdisciplinary process in the collaboration. For example, the co-authors could be responsible for introducing literature from their disciplines without there being any interdisciplinary collaboration involved.

The same is true for the output measure – a group might decide to publish in a range of journals so that teammates can receive credit in their home departments by publishing in a journal the department recognizes, without that entailing any actual disciplinary integration in the production of those papers. Since your decision to center interdisciplinary process is really the crux of this paper, you need to develop and defend this choice in more detail, IMO. And you should also address this as a limitation in the discussion. We explain our choice of measure in the paper as follows.

“As we were unable to follow each group individually over the several years they were collaborating, we sought a measure that could be applied retrospectively. Interdisciplinarity can be assessed in many ways, of which bibliometric measures are the most developed and have valuable antecedent analyses (e.g. Wagner et al. [59], Leyesdorf and Rafols, 2011, Rafols et al., 2012, Marx and Bornemann, 2016, Leydesdorf et al., 2018).

To assess the interdisciplinarity of the working groups’ collaborations, we analysed the literature on which the group drew to produce their published articles, the ‘inspiration’ measure of [7] and used by several authors in whole or in part as a measure of interdisciplinarity [29], and [60], Huang et al., 2022, Shu et al, 2022).”

We think it unlikely that authors could draw on literature from different domains in a paper without offering any synthesis or integration, that is, to be multi-disciplinary in a single paper. The scenario suggested where members of the group publish in their home disciplines without any interdisciplinary integration would yield a high score for publication diversity but a low score for citation diversity, since the individual disciplinary papers would not need (or be able to) to cite outside the discipline, so that effect would be visible in our analyses.

l. 295. Did you ask any questions about interdisciplinary process in the survey? Did you receive any open-ended responses that had a bearing on how you understand interdisciplinary process? We asked about factors that helped or hindered effectiveness, satisfaction and general outcomes. The comments about disciplinary differences are listed in section 4.3

l. 309. What was the overall response rate to the survey? The response rate has been inserted.

ll. 465ff. Why did you choose the A groups as the control for the group factor? (And I guess I’m not sure what you mean by “group factor” here?) When running a regression using factor variables (those having discrete unordered levels), the typical approach is to represent the levels of the factors as dummy variables. One level of the factor is picked as the base and the regression identifies how the other levels differ from that base. The default in R is to use the first factor level as the base. Specifically, we are comparing data for working groups from three centres, so the variable representing the centre is a factor variable with 3 levels. The first level is arbitrarily selected as base value (i.e., the A groups), and the regression includes two dummy variables representing membership in the B or the C centre respectively. The regression weights for those dummy variables indicates how much on average a group in centre B or C differ from those in centre A, controlling for the other variables.

ll. 519ff. For readability, I would encourage you to use a colon or comma to separate your quote introductions from the quotes themselves, e.g., ll. 535-7 where I was tempted by the lack of punctuation to read that as one long (and very awkward) sentence. We have inserted semicolons regularly, and ‘ands’ as linking.

l. 636. Were there really no changes in the working groups over time that affected their diversity? If there were, how does that affect what you’re saying here? Does it have any impact on your analyses in this paper? There was little change in the group composition at the times of the surveys (2014 and 2018) in relation to the start of the groups. This could only be ascertained, however, from records of meeting attendance and some unsolicited comments. Meeting attendance data was not available consistently for all groups, and for the data that were available, fidelity to the group as evidenced by meeting attendance was between 60% and 80% over 2-10 meetings. Fidelity was always greater than 80% for groups which had only two meetings. Some of this text has been entered in the paper.

Feedback has been included in section 4.3 that mentions the classical ‘committed’ and ‘less committed’ members of groups.

l. 655. Maybe supply a couple of examples of these suggestions? These are already mixed in the qualitative section (4.3), and were mainly exhortations to exercise patience. This sentence was removed from the main body of the paper.

l. 670. Might it not take more time for a working group that publishes widely to have impact? How can you distinguish between differences in impact vs. differences in rate of impact? Some of our working groups were quite old (e.g., created in 2010) while others were young, so we believe the data are sufficient to capture impact. We control for age of the working groups to enable comparisons, as we expect groups to be more productive (have more papers out) and have more citations with age.

Analyzing the rate of impact retrospectively would be quite challenging as it would require identifying the timing of each individual citation. As we are already controlling for age, this additional analysis was not pursued.

Attachment

Submitted filename: Specht_Crowston_Response to reviewers_220830.docx

Decision Letter 1

Sergi Lozano

18 Oct 2022

PONE-D-22-04001R1Interdisciplinary collaboration from diverse science teams can produce significant outcomesPLOS ONE

Dear Dr. Crowston,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

As you can see below, both reviewers are satisfied with the new version of the manuscript. I am fine with that. However, Reviewer 2 pointed out to some minor issues that should be fixed before proceeding towards publication.

Moreover, Reviewer 2 also disagrees with some of the statements made in the text about interdisciplinarity and how to measure it. I would like to ask you to answer his comments (considering the relevance of the issue) and, eventually, referring to this debate in the manuscript (if you find it interesting enough).

Please submit your revised manuscript by Dec 02 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Sergi Lozano

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: (No Response)

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: (No Response)

Reviewer #2: I Don't Know

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: (No Response)

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: (No Response)

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: I greatly appreciate the authors' decisions in how they chose to address my comments, questions, and concerns. Overall, the paper is more effective in its goal and the main document does not have the same issues. I have no issues with this paper being accepted due to the enhanced literature review and additional analyses that were conducted at my recommendation.

Reviewer #2: This is a good paper, and one worth publishing. I appreciate the work the authors did to improve the paper in light of the previous round of comments. I think the results are especially interesting and helpful. I suspect I will cite this paper in the future.

FWIW, I disagree with both parts of this reply to my previous comment on ll. 265ff:

"We think it unlikely that authors could draw on literature from different domains in a paper without offering any synthesis or integration, that is, to be multi-disciplinary in a single paper. The scenario suggested where members of the group publish in their home disciplines without any interdisciplinary integration would yield a high score for publication diversity but a low score for citation diversity, since the individual disciplinary papers would not need (or be able to) to cite outside the discipline, so that effect would be visible in our analyses."

I have been in more than one crossdisciplinary collaboration that generated papers which would appear interdisciplinary on the authors assumptions but were decidedly multidisciplinary in their construction: different authors were given responsibility for different sections, and they brought their references along with them. And also in my experience, if a group publishes papers in the home disciplines of their members while keeping those members as co-authors (e.g., if you have the same set of co-authors but different first authors), the publications will often share a multidisciplinary set of references in common. (Perhaps there are some disciplines whose journals discourage citation outside the discipline, but I have not encountered that.)

That said, the authors are correct that the bibliometric methods they use as proxies for interdisciplinary collaboration are standard in the literature. I think they are weak measures of interdisciplinary collaboration, and as such represent a substantial limitation of this paper, but I don’t think it is fair to hold that against the authors since they are cleaving to standard practice here.

One general comment: I find the numbering helpful in keeping straight how you are nesting the sections and subsections. Perhaps this is inconsistent with PLOS One style, though? At this point, I have difficulty keeping track of which are the main sections, which are the subsections, and which are the sub-subsections.

I do have a few small comments/suggestions, none of which are deal-breakers. By line number:

l. 85 Reference [5] is repeated in here as reference [29].

ll. 112-3 This still seems too fast to me. You can have a diverse team with very little integration – e.g., a multidisciplinary team that has one person (say, the PI) who is especially committed to broad citation. “[T]he diversity of authors and the references they cite” give you very little information about whether any of the processes mentioned in the quote from [33] actually took place in the team. IMO, of course. (Not that you are in a position to do better than this, given that you weren’t part of these groups – see the comment above.) It would make me a bit happier to see the authors note the gap between the processes mentioned in the quote and the proxy variables mentioned in the last sentence.

l. 185 Should be ‘have’, not ‘has’.

ll. 205-8 I have difficulty parsing the clause that includes two instances of "accepted by" – is there something missing between "university" and the second "accepted"?

l. 372 Space missing between ‘members’ and ‘who’.

ll. 479-81 On what is this claim based?

l. 543 If you make reference to the section numbering, you should number your sections.

ll. 669-70 The two "but" clauses in this sentence makes it difficult to interpret. Rewrite?

ll. 682-3 Not sure how to read this last clause. Is it supposed to be another example of positive consideration of the value of diversity in general? Maybe set its context up a bit more?

ll. 789-90 Would it be fair to say that your results only show muted short-term impact?

ll. 800-3 Perhaps this is not surprising, given that you only draw from groups that were productive and productivity is positively correlated with satisfaction (ll. 481-3).

ll. 868-70 Not sure what you mean by "staged output"? What findings suggest this? I am not sure I can locate what in your Results section supports this guideline.

l. 874 You don't need the commas in (4) – they make it hard to parse.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Michael O'Rourke

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Decision Letter 2

Sergi Lozano

9 Nov 2022

Interdisciplinary collaboration from diverse science teams can produce significant outcomes

PONE-D-22-04001R2

Dear Dr. Crowston,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Sergi Lozano

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Acceptance letter

Sergi Lozano

17 Nov 2022

PONE-D-22-04001R2

Interdisciplinary collaboration from diverse science teams can produce significant outcomes

Dear Dr. Crowston:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Sergi Lozano

Academic Editor

PLOS ONE

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Supplementary Materials

    S1 Appendix. Demographic profiles of the groups from the contributing organisations.

    (DOCX)

    S2 Appendix. Profile of publications produced and cited by each group.

    (DOCX)

    S3 Appendix. Responses to questions about perceived effectiveness and satisfaction with the group by respondents.

    (DOCX)

    Attachment

    Submitted filename: Specht_Crowston_Response to reviewers_220830.docx

    Attachment

    Submitted filename: PLOS R1 reviews_221101.docx

    Data Availability Statement

    The data availability statement is aligned with the ethics requirements for the study. There were several limitations imposed by the Institutional Review Board of Syracuse University and accepted by the University of Queensland as suitable ethical conduct for the work proposed (Syracuse University #13-202, 16-203 and 18-230). These were as follows: “Personal information that is collected will be used solely to enable network analysis of members within a working group and will not be used for any other purpose. Results of the research may at some future time be published. Although your responses may be identifiable to the researchers, responses will be kept confidential and no individual responses will be reported; only summarized findings will be reported.” and “Your identity will be held in confidence as an invitee to the survey associated with the relevant synthesis centre and group. Your identity will not be published.” This being so, we have anonymised all respondent identities, including identifiable links to their organisations and disciplines, and to the specific publications by each group (which would allow identification of the authors and hence the groups being studied). We have not published here or in the data repository for the paper the full demographic profiles of members. With that consideration, the data are available as follows in the Environmental Data Initiative [83]. The data themselves will remain anonymised.


    Articles from PLOS ONE are provided here courtesy of PLOS

    RESOURCES