Abstract
Introduction:
The National Academies of Sciences (NAS) emphasize the need for interdisciplinary team science (TS) training, but few training resources are available. COALESCE, an open-access tool developed with National Institutes of Health support and located at teamscience.net, is considered a gold standard resource but has not previously been evaluated. COALESCE launched four learning modules in 2011. The Science of TS (SciTS) module, an interactive encyclopedia, introduces foundational concepts. Three scenario-based modules simulate TS challenges in behavioral, clinical, and basic biomedical sciences. This study examined user characteristics, usage patterns, and effects of completing the four modules on TS knowledge, attitudes, and skills.
Methods:
Repeated measures ANOVA tested for pre-post changes in performance and compared learning by users with biomedical versus other disciplinary backgrounds.
Results:
From 2011 through 2017, the site attracted 16,280 new users who engaged in 6461 sessions that lasted more than 1 min. The modal registrant identified as working in a biomedical field (47%), in an academic institution (72%), and expressed greater interest in the practice than the SciTS (67%). Those completing pre- and post-tests (n = 989) showed significant improvement in knowledge, attitudes, and skills after taking all scenario-based modules (p < 0.005); knowledge and attitudes were unchanged after the SciTS encyclopedia. Biomedical and other health professionals improved comparably.
Conclusion:
Evaluation of the TS training tool at teamscience.net indicates broad dissemination and positive TS-related outcomes. Site upgrades implemented between 2018 and 2020, including adding five new modules, are expected to increase the robustness and accessibility of the COALESCE training resource.
Keywords: Team Science, cross-disciplinary research, online training, faculty development, translational research
Introduction
Contemporary science relies on diverse forms of expertise and specialized methodologies to solve complex health challenges. 1 – 8 Collaboration has become essential, necessitating partnerships among scientists trained in different fields. 6 , 9 – 14 Reports issued over the past 14 years by the National Academies of Sciences (NAS) 13 , 15 – 17 emphasize the need to remove barriers to successful team science (TS), defined as the study and practical application of principles about how to assemble, optimize, and lead groups of researchers who collaborate on a shared endeavor to achieve more than additive results.
NAS called for “interdisciplinary team-based approaches in training, education, and research” and “innovative education and training models and methodologies, which include a focus on team science.” 16 Yet, although a need to improve the effectiveness of science teams is widely recognized, available training opportunities remain scarce. The paucity of resources to train science teams contrasts with the abundance of both in-person and online resources available to train clinical practice teams. 18 – 20 In contrast, only a few academic institutions offer in-person training workshops or a course in team science to train research teams. 21 – 23 Several federal funding agencies offer occasional creatively facilitated week-long residential IDEAS labs (also called Sandpits or Innovation Labs) that assemble geographically distributed researchers to explore new interdisciplinary science. These closely mentored, in-person training formats carry a high cost and participant burden, however. Digital dissemination of online, self-administered training materials warrants consideration as an alternative or complementary training platform. By eliminating needed costs for travel, coordination, and real-time faculty salaries, digital delivery holds potential to make TS training more broadly accessible.
In 2011 with NIH support, we developed COALESCE (CTSA Online Assistance for Leveraging the Science of Collaborative Effort) as a learning resource for scientists planning to undertake cross-disciplinary, collaborative health science research. Although several printable guides 9 and online repositories of TS resources 24 are available, COALESCE remains the only e-learning training resource. A free, open-access online educational tool located at teamscience.net, COALESCE hosts a series of online learning modules shown in Table 1: the four original modules created in 2011, four additional modules added in 2018, and one module currently under development. All use an interactive format and aim to improve users’ knowledge, attitudes, and self-efficacy for team science skills. 25 These particular learning objectives are based on Bloom’s taxonomy, 26 which has guided the development of traditional curricula and web-based training in many disciplines, 27 including medicine. 28 The current evaluation focuses on the first four modules developed for the site: the Science of Team Science (SciTS) module and three scenario-based modules (basic, biomedical, and clinical team science) because these have been accessible on COALESCE long enough to accumulate users for analyses. The SciTS module takes the form of an interactive encyclopedia: its short videos and animations introduce considerations involved in assembling, managing, and evaluating a team. Users move freely throughout the SciTS module, choose questions of interest, and watch experts answer based on the team science evidence base. A more guided structure characterizes the three multimedia scenario-based modules that address the praxis (i.e., practical application) of team science knowledge. Here users assume the role of a senior scientist, an early-stage scientist, or a research development officer, while navigating research challenges in behavioral, basic biomedical, and clinical medical sciences, respectively. They make decisions about choosing and managing a team, while receiving feedback that lets them learn from mistakes in a safe environment. 29
Table 1.
COALESCE Module descriptions
Module Title | Brief Description | Analyzed in Present Study? |
---|---|---|
The Science of Team Science | The module introduces the science of team science via a collection of expert videos, animations, and activities. | Yes |
Behavioral Team Science Wing | Assuming the role of an experienced investigator, the learner navigates through a series of scenarios to form a team de novo, identify and secure funding, successfully launch the team project, manage conflict, and evaluate performance. | Yes |
Biomedical Team Science Wing | The learner takes on the role of a research development officer to navigate through a series of scenarios involved in initiating a large cross-disciplinary, inter-institutional project. The learner develops a proposal and budget, develops a relationship with the funder, and promotes collaboration. | Yes |
Clinical Team Science Wing | In this interactive course, the learner assumes the role of an early career investigator to form a team for a 5 year R01 grant from the NIH. The learner manages conflict with more senior investigators, sustains team engagement, and evaluates success. | Yes |
Stakeholder Dialogue about Evidence-Based Practice (EBP) | The learner starts a dialogue between various stakeholders to understand differing perspectives about research and EBP. | No |
Shared Decision Making with Individual Clients | The learner balances the three circles of EBP when applying shared decision making with individual clients. | No |
Collaborative Decision Making with Communities | The learner balances the three circles of EBP when applying collaborative decision making with a community. | No |
Implementation of EBPs | The learner uses two real-world case examples to understand and practice implementing EBPs. | No |
Engaging Community Partners in Team Science | The learner navigates three real-world research examples to become familiar with the challenges encountered during the phases of team assembly, launch, and maturation. | No |
The goal of this study was to characterize COALESCE (teamscience.net) usage and users, and to evaluate whether completing the modules improves team science knowledge, attitudes, and self-efficacy about team science skills.
Materials and Methods
Because the study involved research on the effectiveness of an instructional strategy, the Northwestern University Institutional Review Board (IRB) judged the study to be exempt research. User data from October 20, 2011, through December 4, 2017, were analyzed (preceding changes made to the site after this date). Website utilization was tracked via Google Analytics data on page views, acquisition channels, and new versus returning visitors. To access the interactive learning modules, visitors needed to register with a valid e-mail address and answer several questions about gender, discipline, profession, and primary interest in the practice versus the science of TS (SciTS). Registrant characteristics and survey responses of those who took module pre- or post-tests were tracked through a password-protected administrative portal hosted on Northwestern University servers.
Upon entering a COALESCE module, the user was invited to take a voluntary online pretest. The pretest was not re-offered if the user declined the invitation and entered the module (so that pre-assessment responses would not be contaminated by exposure to educational material in the module). After completing the module, the user was invited to take the same assessment as a posttest. Each of the four modules had unique test questions developed by the research team to match its training content, but all modules assessed knowledge about team science, attitudes toward team science, and self-efficacy about team science skills. The knowledge assessment for all modules contained 10 multiple-choice questions, asking about information conveyed in that particular module. Attitudes assessments used three to five questions on a five-point Likert scale 30 on which users expressed the favorability of their opinions about team science. Five-point Likert scales were also used for the skills assessments that included between three and seven questions: users rated their confidence (self-efficacy) to act effectively in team roles associated with that module. Once having begun the pre- or posttest for a module, users could not skip questions; options were to complete or exit the test. After completing the post-assessment for any module, users were invited to take a satisfaction survey consisting of four questions about the anticipated impact of the modules on their future research, favorite and least favorite sections of the tool, and suggested new modules.
Statistical Analysis
An anonymized dataset containing descriptive characteristics and pre- and posttest scores of registered users was downloaded from the COALESCE (teamscience.net) server. Data were analyzed using SPSS software. Demographic characteristics of those included in the analyses because they took paired pre- and posttests, versus those excluded because they took only pre- or posttests, were compared via Chi-Square tests for categorical variables and t-tests for continuous variables. Main study outcomes were team science knowledge, attitudes, and self-efficacy about TS skills for each of the four modules. Data were analyzed using repeated measures ANOVA testing the effect of time (pre- vs. posttest) on the outcomes for the modules. Because the modules’ effects on learning were of primary interest, cases were included in analyses only if they had both pre- and posttest scores for a given module. Interactions between time and disciplinary group (biomedical vs. other field) also were examined to test whether users with a biomedical background learned more than users from non-biomedical fields.
Results
Site Traffic and Navigation
Between October 20, 2011, and December 4, 2017, the COALESCE website was visited by 16,280 new unique users and had 31,294 page views (not including pages viewed within the portal that housed the learning modules) accomplished in 21,716 sessions. A majority of visits were brief: 65.4% of sessions involved visiting a single page and 67.3% involved spending 10 s or less on the site. On the other hand, 6461 sessions (29.8%) involved spending more than 1 min on the site. As is shown in Fig. 1a, the number of new users of the site grew steadily from 2011 to 2015, when recruitment of new first-time users peaked. As Fig. 1a also shows, the number of new users visibly declined between 2015 and 2017, but the number of page views continued to increase, prompting us to upgrade the site in the ways discussed below. As Fig. 1b indicates, a majority of users (50%) accessed COALESCE (teamscience.net) directly by typing the URL into their browser, clicking on a bookmark, or following a link in an e-mail or other electronic documents. Another group of users (29%) was referred via external links from another website, such as sites hosted by the NIH and universities featuring teamscience.net as a resource. The remaining users (20%) accessed the website by way of search engines. Despite some variation over time, direct access remained the most common way for users to access COALESCE (teamscience.net).
Fig. 1.
Traffic and navigation through COALESCE. The amount of traffic COALESCE (teamscience.net) received between its launch on October 20, 2011, and December 4, 2017, as evaluated using Google Analytics. A. Visits and Page Views, 2011-2017. New Users are those navigating to the site for the first time in a two-year period via a specific device and specific browser. Page View is a view of a page on the site that is being tracked by the Analytics tracking code. B. Ways Users Accessed COALESCE, by Year (2011-2017). Direct: visitors who reached the site by entering “teamscience.net” into the web address bar of their browser, clicking on a bookmark, or following a link in an email or other electronic document. Referral: visitors who clicked on a link from a referring site to teamscience.net. Search: visitors who reached the website via search engines like Google, Bing, and Yahoo.
Study Sample
Figure 2 shows the flow of participants through the study. Of 16,280 unique visitors to teamscience.net during the study period, 16% (n=2686) chose to register and provide demographic information. A majority of registered users identified as female (66%), reported working in academia or education (72%), and expressed primary interest in the practice (rather than the science) of TS (67%). Approximately half (47%) of registered users identified as being in a biomedical field.
Fig. 2.
Participant flow through COALESCE (teamscience.net) evaluation study. Diagram shows the progression of COALESCE (teamscience.net) users from discovery of the site, through registration, through completion of a pre- or posttest, through completion of both pre- and posttest assessments of knowledge, skills, and attitudes for each of four modules.
Among COALESCE registrants, 37% (n=989) voluntarily completed at least one pre- or posttest (Fig. 2). Of those completing a pre- or posttest, 22% (n=219) completed both a pre- and posttest for at least one module, allowing their change in performance to be examined. There were no differences in gender, educational attainment, employment sector, or primary team science interest between participants included in the analyses and those excluded for lacking a paired pre- and posttest. Those included in the analyses were more likely than those excluded to have been trained in a biomedical field versus a different discipline (X 2 (1) = 5.11, p < 0.05). They were also somewhat younger (m = 38.10, SD = 11.47 vs. m = 41.39, SD = 12.70 years, t (899) = 3.30, p < 0.005) and had several fewer years of professional experience (m = 9.04, SD = 7.69, vs. m = 12.14, SD = 10.41 years, t (921) = 3.95, p < 0.001). The number of users completing each COALESCE module is shown in Table 2, together with the average change in team science skills, attitudes, and knowledge that users displayed after completing each module. All four modules increased users’ self-confidence about being able to perform team science skills, and all effect sizes were large. Attitudes toward team science grew more positive after users completed the behavioral, basic biomedical, and clinical modules, showing small to medium effect sizes, but attitudes did not improve significantly after taking the SciTS module. Likewise, knowledge scores improved after completing the behavioral, basic biomedical, and clinical modules, and effect sizes were large, but knowledge did not change significantly after taking the SciTS module (Table 2). As Table 3 indicates, no significant interactions were found between discipline (biomedical vs. other) and change in skills and attitudes for any module. After taking the clinical module, users from a biomedical background appeared to show significantly greater improvement in knowledge than those from other disciplines (F(1, 75) = 7.72, p=0.0069), but this effect failed to meet the criterion for statistical significance (p=0.0042) after Bonferroni correction for multiple comparisons.
Table 2.
Change in team science skills, attitudes, and knowledge after completing COALESCE (teamscience.net) modules *
Module | Assessment | |||||
---|---|---|---|---|---|---|
Attribute | Pre- | Post- | Sig | Cohen’s d | N | |
Science of Team Science | Skills | 3.1(0.8) | 3.9(0.8) | p < 0.001 | 0.87 | n = 117 |
Behavioral Team Science | Skills | 2.7(0.7) | 3.8(0.8) | p < 0.001 | 1.29 | n = 75 |
Basic Biomedical TScience | Skills | 2.2(1.0) | 3.8(0.8) | p < 0.001 | 1.65 | n = 71 |
Clinical Team Science | Skills | 2.9(0.8) | 3.8(0.8) | p < 0.001 | 1.33 | n = 78 |
Science of Team Science | Attitudes | 3.9(0.7) | 4.0(0.7) | p = 0.085 | 0.16 | n = 115 |
Behavioral Team Science | Attitudes | 3.7(0.6) | 3.9(0.6) | p < 0.005 | 0.39 | n = 75 |
Basic Biomedical TScience | Attitudes | 3.7(0.6) | 4.0(0.6) | p < 0.001 | 0.60 | n = 71 |
Clinical Team Science | Attitudes | 3.6(0.7) | 3.8(0.6) | p < 0.005 | 0.37 | n = 78 |
Science of Team Science | Knowledge | 53.4% | 54.0% | p = 0.678 | 0.04 | n = 110 |
Behavioral Team Science | Knowledge | 53.9% | 72.4% | p < 0.001 | 0.98 | n = 74 |
Basic Biomedical TScience | Knowledge | 52.8% | 79.3% | p < 0.001 | 1.22 | n = 72 |
Clinical Team Science | Knowledge | 43.6% | 71.3% | p < 0.001 | 1.54 | n = 77 |
Data shown for COALESCE users who completed both pre- and posttest for any module. Skills and attitudes questions, scored on a 1–5 Likert scale, are shown as mean (SD). Knowledge scores are shown as mean percentage of correctly answered questions. p-values derived from repeated measures ANOVA show significance of change from pre- to posttest. TScience: team science
Table 3.
Comparison of change from pre- to post-assessment score for all modules by users from biomedical versus other disciplinary backgrounds *
Module Assessment | BioMed Pre- | BioMed Post- | Other Pre- | Other Post- | BioMed vs. Other Pre-Post | BioMed N | Other N | |
---|---|---|---|---|---|---|---|---|
Team Science | Skills | 2.9(0.9) | 3.9(0.8) | 3.2(0.8) | 3.9(0.8) | p=0.185 | n=59 | n=58 |
Biomedical | Skills | 2.7(0.8) | 3.9(0.7) | 2.8(0.6) | 3.6(0.8) | p=0.063 | n=48 | n=27 |
Behavioral | Skills | 2.1(1.0) | 3.8(0.7) | 2.4(0.9) | 3.8(0.9) | p=0.144 | n=48 | n=23 |
Clinical | Skills | 2.9(0.8) | 3.9(0.6) | 2.8(0.8) | 3.5(1.0) | p=0.153 | n=54 | n=24 |
Team Science | Attitudes | 4.0(0.6) | 4.1(0.6) | 3.8(0.7) | 3.9(0.7) | p=0.676 | n=58 | n=57 |
Biomedical | Attitudes | 3.8(0.6) | 4.0(0.6) | 3.5(0.7) | 3.6(0.7) | p=0.469 | n=48 | n=27 |
Behavioral | Attitudes | 3.7(0.7) | 4.0(0.6) | 3.8(0.6) | 4.1(0.6) | p=0.716 | n=48 | n=23 |
Clinical | Attitudes | 3.6(0.6) | 3.8(0.6) | 3.5(0.8) | 3.8(0.7) | p=0.302 | n=54 | n=24 |
Team Science | Knowledge | 53.4% | 55.2% | 53.3% | 52.8% | p=0.430 | n=56 | n=54 |
Biomedical | Knowledge | 56.0% | 76.7% | 50.0% | 64.6% | p=0.192 | n=48 | n=26 |
Behavioral | Knowledge | 52.3% | 80.2% | 53.8% | 77.5% | p=0.449 | n=48 | n=24 |
Clinical | Knowledge | 43.6% | 74.9% | 43.8% | 63.3% | p<0.01 | n=53 | n=24 |
Skills and attitudes rated on 1–5 Likert-type scale and shown as mean (SD). Knowledge scores reflect mean percentage of correctly answered multiple-choice questions. p-values based on repeated measures ANOVA reflect interaction between discipline and pre- to posttest performance change. Team Science: Science of Team Science.
Of those who completed a paired pre- and posttest, 77 also completed a satisfaction survey. Across all modules, a majority of respondents (84%) reported that completing COALESCE training was very likely to impact their future research (Fig. 3). Only a small number of participants (5%) reported that completing the modules was unlikely to impact their future research. Although all of the modules received positive reviews, the SciTS module accumulated the greatest number of suggestions for improvement. Recommendations focused primarily on how to improve the module’s flow and usability.
Fig. 3.
Satisfaction Survey Response. After completing each COALESCE module, users were asked, “How likely is this module to impact your future research?” Response options were: very likely, to some extent, and unlikely. The percentage of respondents endorsing each response option for each module is shown earlier. TS, team science.
Discussion
COALESCE (teamscience.net) remains the first and only open-access, online training for team science in the health and medical professions, and has been considered a gold standard team science training resource. 31 , 32 However, the tool’s effect on team science learning, attitudes, and skills had not previously been evaluated. From its launch in 2011–2017, COALESCE (teamscience.net) attracted 16,280 unique new visitors who engaged in 6461 sessions lasting more than 1 min, indicating broad dissemination. Users showed significant improvement in knowledge, attitudes, and skills after taking any of the three scenario-based modules in basic biomedical, behavioral, or clinical science, indicating that these modules successfully meet the learning objectives for which they were designed.
Learners’ self-confidence about their ability to perform team science skills also increased after using the SciTS interactive encyclopedia module, but knowledge and attitudes were unchanged. For the most part, biomedical and other health professionals improved comparably after taking the modules. The one exception involving differential knowledge change after taking the clinical module was no longer significant after correction for multiple comparisons, suggesting that the finding may be spurious. Finally, most learners believed that the experience of taking the modules would positively impact their future research. In aggregate, these findings support the premise that an online e-learning tool such as COALESCE (teamscience.net) offers an acceptable, effective, low-cost, readily disseminable resource to improve team science outcomes.
Across the website’s lifespan, most users have accessed COALESCE directly, by typing “teamscience.net” into a browser or following a link in an e-mail. There also has been a gradual increase in the proportion of visits directed to the website through search engines and external referrals, such as the National Cancer Institute’s Team Science Toolkit, NIH Intramural Training and Education, and university CTSA webpages. The site’s sustained usage is likely attributable to continuing public interest in team science, since no marketing campaign was undertaken for COALESCE.
Consistent with the developers’ intended audience, the modal COALESCE user comes from a biomedical discipline and is employed in an academic/educational setting. Despite identifying largely as academic researchers, very few users expressed interest in learning about the scientific evidence base that grounds team science. Instead, most say they came to the site in order to learn the practical aspects of how to do research in scientific teams.
Importantly, taking any of the four modules increased learners’ confidence that they had the skills needed to engage effectively in team science. Moreover, learners with backgrounds in biomedicine or a different discipline showed generally comparable improvements in team science knowledge and attitudes after taking the scenario-based modules. In aggregate, the current findings support the appraisal of COALESCE (teamscience.net) as an effective resource for both biomedical and non-biomedical scientists who seek learning opportunities in cross-disciplinary, collaborative research.
SciTS was the only COALESCE (teamscience.net) module not to improve team science knowledge or attitudes; it also received the most suggestions for improvement. Designed differently than the other more directed, scenario-based modules, SciTS is an interactive encyclopedia (i.e., a buffet-style repository of information wherein users select questions that interest them from banks of videos, animations, and activities). User feedback suggests that some felt overwhelmed by the number of choices available to them and the lack of a clear structure guiding progression through the materials. Therefore, in 2018, this module was modified by adding a table of contents and a tracking system to help users navigate through the information. No structural changes were made to any of the three scenario-based modules because knowledge, attitudes, and skills improved after taking any of them.
Nevertheless, several factors prompted an upgrade to the teamscience.net site in 2018, in addition to the improvements to the SciTS module described earlier. One factor involved changes in site traffic and navigation patterns that we observed between 2015 and 2017. We attributed the decline in new users at least partially to the increasing use of mobile technologies to access the Internet. Because COALESCE (teamscience.net) had been programmed in Flash, rather than HTML, mobile device users could not reliably access its video content. Meanwhile, the increased number of page views during the same period suggested a need for greater server capacity. Other technical upgrades performed in 2018 met user requests for disability access, enhanced privacy, improved user experience, and Shareable Content Object Reference Model (SCORM) compliance – a set of standards for e-learning technology.
Also, given growing concern with ensuring the real-world translation and uptake of research findings 33 and specific recommendations from the CTSA Team Science Domain Task Force (DTF) (predecessor to the current CTSA Collaboration and Engagement DTF), the scientific scope of the COALESCE (teamscience.net) tool was extended to highlight the application of team science principles to community-engaged research. As advised by the DTF, four modules created with public health experts and community advocates for the NIH-sponsored evidence-based behavioral practice project (www.ebbp.org) were added to COALESCE (teamscience.net). 34 – 36 The added modules address shared clinical decision making, collaborative community decision making, stakeholder perspectives, and implementation science. Neither these four modules nor the technical changes made in 2018 are included in the present evaluation because they are too new to have accumulated sufficient data for analysis. Finally, a new module addressing how academic investigators can identify and engage community stakeholders in research collaborations is also now under development for addition to the COALESCE (teamscience.net) suite of modules. This module, funded by NCATS, has been requested repeatedly since the First Annual International SciTS Conference in 2010, when COALESCE was introduced. Development of the new module and addition of modules on implementation science and community collaboration also is consistent with the expanded scope of CTSA DTF team science efforts to engage community as well as academic partners.
Limitations
There are limitations to the present study that stem primarily from sample selection and testing. The all-volunteer sample was comprised of those who found the site of their own accord, not in response to active recruitment or incentivization. Hence, they are likely to have been more favorably disposed, or at least more curious about team science, than the average member of the population. Also in terms of sampling, only 37% of those who registered on the teamscience.net website completed a module pre- or posttest, and only 22% of these completed both the pre- and posttest for any module, albeit demographic differences between these two subgroups were modest. Hence, the present results are drawn from a relatively small, highly motivated, selected sample and may not generalize to the average person. On the other hand, whereas the threat to validity from selective sampling would be expected to bias test performance in a positive direction, the threat from repeated testing would be expected to work in the opposite direction. That is, a highly selected and motivated sample would be expected to achieve high pretest scores, introducing ceiling effects that made posttest improvement difficult to detect. Nevertheless, improvements were observed on all measured team science outcomes after taking any scenario-based module. Another limitation is that although most learners reported that the experience of completing the modules would influence their future research, it was not possible to assess long-term objective effects on respondents’ performance in interdisciplinary teams, a limitation that partly reflects lack of consensus about optimal performance metrics. 37
Despite these limitations, there remains a clear and sustained need for team science training resources in the health and medical fields. Given the shortage of in-person training options, combined with the expense and logistical challenges involved in creating them, COALESCE (teamscience.net) provides a much-needed, accessible e-learning opportunity to introduce team science training to interested researchers. Given evidence that taking any of the scenario-based modules enhances team science knowledge, attitudes, and skills, our recommendation is for learners to begin using COALESCE (teamscience.net) by taking any of the clinical, behavioral, or basic biomedical research modules. Because a change in knowledge was not observed after taking the SciTS module and because many users found the experience overwhelming, rather than attempting to navigate the entire module, we advise using it as an interactive encyclopedia to be consulted when questions arise about the evidence base for team science.
Conclusions
COALESCE (teamscience.net) remains the first and only open-access, online training in TS for health researchers. Initial evaluation of the training modules indicates widespread dissemination and positive outcomes. The COALESCE modules were updated in late 2017-early 2018 based on these results, user feedback, and CTSA DTF recommendations. COALESCE is well positioned to play a valuable role in the future of team science training. Updates to its accessibility, content, and evaluation systems can be expected to further bolster its utility in supporting the practice and science of team science across the translational research spectrum.
Acknowledgements
The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. Research reported in this publication was supported by the National Institutes of Health’s National Center for Advancing Translational Sciences, Grant Numbers 3UL1RR025741 and UL1TR001422 and Office of Behavioral and Social Sciences Research, Contract N01-LM-6-3512, Online Tool to Support CTSA Research Workforce Development in Behavioral Team Science. No financial disclosures were reported by the authors of this paper.
Footnotes
Financial Disclosure Information
Bonnie Spring has no financial disclosures. Ekaterina A. Klyachko has no financial disclosures. Phillip W. Rak has no financial disclosures. H. Gene McFadden has no financial disclosures. Donald Hedeker has no financial disclosures. Juned Siddique has no financial disclosures. Leland R. Bardsley has no financial disclosures. Angela Fidler Pfammatter has no financial disclosures.
References
- 1. Gray B. Collaborating: Finding Common Ground for Multiparty Problems. San Francisco: Jossey-Bass, 1989. [Google Scholar]
- 2. Cummings JN, Kiesler S. Coordination costs and project outcomes in multi-university collaborations. Research Policy 2007; 36(10): 1620–1634. [Google Scholar]
- 3. Guimera R, et al. Team assembly mechanisms determine collaboration network structure and team performance. Science 2005; 308(5722): 697–702. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Hall KL, et al. Assessing the value of team science: a study comparing center- and investigator-initiated grants. American Journal of Preventive Medicine 2012; 42(2): 157–163. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Rhoten D, Parker A. Risks and rewards of an interdisciplinary research path. Science 2004; 306(5704): 2046. [DOI] [PubMed] [Google Scholar]
- 6. Wuchty S, Jones BF, Uzzi B. The increasing dominance of teams in production of knowledge. Science 2007; 316(5827): 1036–1039. [DOI] [PubMed] [Google Scholar]
- 7. Borner K, et al. A multi-level systems perspective for the science of team science. Science Translational Medicine 2010; 2(49): 49cm24. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Falk-Krzesinski HJ, et al. Advancing the science of team science. Clinical and Translational Science 2010; 3(5): 263–266. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Bennett LM, Gadlin H, Levine-Finley S. Collaboration & Team Science: A Field Guide. National Institutes of Health Publication No 10-7660; Bethesda, MD, 2010. [Google Scholar]
- 10. Bennett LM, Gadlin H. Collaboration and team science: from theory to practice. Journal of Investigative Medicine 2012; 60(5): 768–775. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Croyle RT. The National Cancer Institute’s transdisciplinary centers initiatives and the need for building a science of team science. American Journal of Preventive Medicine 2008; 35(2 Suppl): S90–93. [DOI] [PubMed] [Google Scholar]
- 12. Klepper A, Pendleton M. Building failure into successful team science: highlights of the second SINAInnovations event. Disease Models & Mechanisms 2014; 7(3): 311–312. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. National Research Council (U.S.). Committee on the Science of Team Science, Cooke NJ, Hilton ML, National Research Council (U.S.). Board on Behavioral Cognitive and Sensory Sciences, National Research Council (U.S.). Division of Behavioral and Social Sciences and Education. Enhancing the Effectiveness of Team Science. Washington, DC: The National Academies Press, 2015. [PubMed] [Google Scholar]
- 14. Buscemi J, Steglitz J, Spring B. The impact of team science collaborations in health care: a synopsis and comment on “Interprofessional collaboration: effects of practice-based interventions on professional practice and healthcare outcomes.” Translational Behavioral Medicine 2012; 2(4): 378–379. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. National Academy of Sciences, National Academy of Engineering, Institute of Medicine. Facilitating interdisciplinary research. Washington, DC: The National Academies Press, 2005. [Google Scholar]
- 16. IOM (Institute of Medicine). The CTSA Program at NIH: Opportunities for Advancing Clinical and Translational Research. Washington, DC: The National Academies Press, 2013. [PubMed] [Google Scholar]
- 17. National Research Council. Convergence: Facilitating Transdisciplinary Integration of Life Sciences, Physical Sciences, Engineering, and Beyond. Washington, DC: The National Academies Press, 2014. [PubMed] [Google Scholar]
- 18. Agency for Healthcare Research and Quality. TeamSTEPPS™ Guide to Action: Creating a Safety Net for Your Healthcare Organization. AHRQ Publication No. 06-0020-4, Rockville, MD, 2006. [Google Scholar]
- 19. Marlow SL, et al. A systematic review of team training in health care: ten questions. The Joint Commission Journal on Quality and Patient Safety 2017; 43(4): 197–204. [DOI] [PubMed] [Google Scholar]
- 20. Weaver SJ, Dy SM, Rosen MA. Team-training in healthcare: a narrative synthesis of the literature. BMJ Quality & Safety 2014; 23(5): 359–372. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Grewal D. Perspective: building a team science training program. Science; 2011. https://www.sciencemag.org/careers/2011/06/perspective-building-team-science-training-program. Accessed January 17, 2019.
- 22. Ameredes BT, et al. The Multidisciplinary Translational Team (MTT) model for training and development of translational research investigators. Clinical and Translational Science 2015; 8(5): 533–541. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Read EK, et al. Building the team for team science. Ecosphere 2016; 7(3): e01291. [Google Scholar]
- 24. Vogel AL, et al. The team science toolkit: enhancing research collaboration through online knowledge sharing. American Journal of Preventive Medicine 2013; 45(6): 787–789. [DOI] [PubMed] [Google Scholar]
- 25. Cairncross S, Mannion M. Interactive multimedia and learning: realizing the benefits. Innovations in Education and Teaching International 2001; 38(2): 156–164. [Google Scholar]
- 26. Bloom BS. Taxonomy of Educational Objectives: The Classification of Educational Goals. 1st ed New York: Longmans, Green, 1956. [Google Scholar]
- 27. Felder RM. Teaching engineering at a research university: problems and possibilities. Educ Quimica 2004; 15(1): 40–42. [Google Scholar]
- 28. Bloom BS. Effects of continuing medical education on improving physician clinical care and patient health: A review of systematic reviews. International Journal of Technology Assessment in Health Care 2005; 21(3): 380–385. [DOI] [PubMed] [Google Scholar]
- 29. Schank RC, et al. The design of goal-based scenarios. The Journal of the Learning Sciences 1993; 3(4):305–345. [Google Scholar]
- 30. Likert R. A technique for the measurement of attitudes. Archives of Psychology 1932; 22(140): 55. [Google Scholar]
- 31. Hesse BW. Coalesce (CTSA online assistance for leveraging the science of collaborative effort). JAMA 2011; 306(17): 1925–1926. [Google Scholar]
- 32. Aronoff DM, Bartkowiak BA. A review of the website TeamScience.net. Clinical Medicine & Research 2012; 10(1): 38–39. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33. National Academies of Sciences, Engineering, and Medicine. Communities in Action: Pathways to Health Equity. Washington, DC: The National Academies Press, 2017. [PubMed] [Google Scholar]
- 34. Spring B. Evidence-based practice in clinical psychology: what it is, why it matters; what you need to know. Journal of Clinical Psychology 2007; 63(7): 611–631. [DOI] [PubMed] [Google Scholar]
- 35. Satterfield JM, et al. Toward a transdisciplinary model of evidence-based practice. The Milbank Quarterly 2009; 87(2): 368–390. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36. Spring BF. Resources for training in evidence based behavioral practice (EBBP). Newsletter of the International Society for Evidence-Based Health Care 2010; 1: 24–25. [Google Scholar]
- 37. Falk-Krzesinski HJ, et al. Mapping a research agenda for the science of team science. Research Evaluation 2011; 20(2): 145–158. [DOI] [PMC free article] [PubMed] [Google Scholar]