The Multidisciplinary Translational Team (MTT) Model for Training and Development of Translational Research Investigators

Abstract

Multi-institutional research collaborations now form the most rapid and productive project execution structures in the health sciences. Effective adoption of a multidisciplinary team research approach is widely accepted as one mechanism enabling rapid translation of new discoveries into interventions in human health. Although the impact of successful team-based approaches facilitating innovation has been well-documented, its utility for training a new generation of scientists has not been thoroughly investigated. We describe the characteristics of how multidisciplinary translational teams (MTTs) promote career development of translational research scholars through competency building, inter-professional integration, and team-based mentoring approaches. Exploratory longitudinal and outcome assessments from our experience show that MTT membership had a positive effect on the development of translational research competencies, as determined by a self-report survey of 32 scholars. We also observed that all trainees produced a large number of collaborative publications that appeared to be associated with their CTSA association and participation with MTTs. We conclude that the MTT model provides a unique training environment for translational and team-based learning activities, for investigators at early stages of career development.

INTRODUCTION

In the 21^st century, health science research has evolved into an increasingly complex and sub-specialized set of disciplines. This evolution, coupled with increasing emphasis of National Institutes of Health (NIH) and other funding agencies to solve complex human health problems, requires effective collaboration of diverse research professionals. As a result, over the past 25 years, multi-institutional collaboration has grown rapidly (1), leading to greater impacts of papers and patents (2, 3). Originating from the NIH roadmap initiative, the Clinical Translational Science Award (CTSA) has more recently challenged over 60 institutions to implement transformative approaches to the conduct of translational research (4, 5), referred to as research which can be translated to the treatment of diseases, and ultimately produce meaningful health outcomes (6).

Following the Institute of Medicine review of the CTSA program in 2013 (5), it was recognized that there is an urgent need for CTSAs to develop innovative approaches to train the translational research workforce of the future, and that there is now a need for the career development of a “qualitatively different” kind of investigator comprising the future workforce (7). However, the clinical and translational environment is changing rapidly (8, 9); therefore, deliberate planning, integrated execution, and adaptability are required to address the changing face of clinical and translational research (10). For example, it has been argued that clinical and translational research scholars should be developed within a program that stresses pre-established clinical and translational research competency requirements that are flexible and tailored to the specific needs of individual scholars (11).

With these priorities and emphases, is clear that the focus of high-impact clinical and translational science, now and into the future, will be on interdisciplinary team science (2, 6, 12), and the training of scientists in this mode. Accordingly, this directly implies that there is an established and continuing need for new approaches in the training of team-based clinical and translational scientists of the future, who will develop new methods of disease treatment and prevention (13, 14). However, it has been reported that only about one-half of all CTSA-supported institutions offer team-based training (15), which also implies that the numbers of scholars being trained nationwide in this fashion is relatively low, providing both a cause for some concern, and a rationale for the approach that we pursued.

Although interdisciplinary research teams have been well-studied in other contexts, their application to translational biomedical research is relatively nascent. Interdisciplinary research includes multidisciplinary approaches, where investigators work collaboratively from within specific disciplines, and transdisciplinary approaches, where investigators develop a shared conceptual model, drawing from distinct fields, to solve a common problem (6). Previously, we have defined Multidisciplinary Translational Teams (MTTs) as distinct academic-industry models with defined structures, processes, and goals to generate a product (3). Initially, these teams work within a multidisciplinary framework (3), but over time, successful teams can evolve into transdisciplinary teams, developing a shared conceptual framework to extend the boundaries of their particular fields (6), into areas that overlap with others. Along those lines, we have recently reported on how our MTTs evolved and were able to foster translational innovations, through a variety of interventions designed to promote team unity and function, as well as opportunities for team leadership (16). Furthermore, a novel feature of the MTT is the focus on continuing training of investigators in team-relevant skills, which directly addresses the consensus statement calling for an approach and curricula that are designed to promote team work and interdisciplinary training of translational scientists, in order to promote translational innovation (7). Since translational scientists traditionally have not been trained in team building, few proven models are currently available to inform successful strategies. Large deficiencies exist in the understanding and implementation of the best-practices for training researchers in team behaviors and research competencies. Therefore, there exists an opportunity to identify new approaches to train future translational scientists.

We postulated that placement of scholar/trainees within MTTs would enhance their collaborative research productivity, and promote their incorporation of translational and team-based competencies. Accordingly, here we report our descriptive observations of an initial exploratory assessment, which includes: 1) the characteristics of the MTTs that make them potentially effective as environments for clinical and translational research training, 2) descriptive productivity metrics of trainees who utilized CTSA resources, 3) self-reported assessments of the translational competencies development within the MTTs, and 4) conclusions regarding the potential impact of the MTT training model, in training future translational research scientists.

CHARACTERISTICS OF THE MTT AS AN ENVIRONMENT FOR CLINICAL AND TRANSLATIONAL RESEARCH TRAINING

The MTT is composed of a strategic core of investigators who can interact with institutional and extra-institutional scientists (3) with a membership structure depicted in Figure 1. The primary focus of the MTT is on translational innovation; this is a unique, experiential environment that promotes research capacity building, inter-professional integrative skills, cognitive support and complementary mentorship models that extend the conventional one-on-one mentoring approach.

Figure 1.

Schematic of member groups within the Multidisciplinary Translational Team (MTT), with emphasis on early-career investigators (Assistant Professors), shown in the gray filled ellipse. Other MTT members and member groups are shown in additional ellipses around the circumference of the main MTT circle. The designation as Project Manager for some early-career investigators allows development of team leadership and administrative skills, as previously presented (3, 29).

Research Capacity Building

Academic researchers are typically life-long learners who keep up with the latest developments in their respective areas. However, this approach can also lead to increasing sub-specialization, resulting in only small incremental solutions to complex disease problems (17). In contrast, due to membership diversity and regular MTT member interactions, the MTT structure provides an opportunity for transformational learning that enhances research capability (18), with repeated trainee exposure to multiple viewpoints, experiences, and expertise of established investigators. We refer to the enhanced capability of teams as rapid adaptation to change, rapid generation of new knowledge, and improved research performance, as evidenced by successful collaborative translational grants and publications (19). These characteristics allow the MTT to be responsive to the challenges of translational research, while simultaneously providing a rich environment for learning, particularly to the early-career members. Regular exposure to alternative scientific approaches toward translational projects enhances the trainees’ ability to adapt and apply new approaches to future research problems that are not otherwise attainable without the team structure.

As alluded to above, to ensure cross-collaboration and exchange of ideas, MTTs are built by inclusion of members from multiple departments within an academic health center (3). MTTs are typically lead by one senior faculty leader with demonstrated expertise in the disease field or mechanistic problems being focused on by the MTT, and to insure multi-disciplinarity, involve at least 2–3 senior faculty members of departments and/or divisions other than that of the MTT leader. Other faculty members and health area experts are added based on content or expertise necessary to address the research problems undertaken by the team. It is also required that at least one to several early-career investigators be embedded within each MTT (Figure 1). We believe that this basic structure of a team with a cross-collaborative and multidisciplinary focus promotes translational innovation and research capacity development, from which the embedded trainees can benefit. While this example is specific to a CTSA-supported institution, we note that the principles illustrated here should be applicable for any institutional setting with committed faculty and interactive cores or centers of expertise.

Inter-professional integration skills

To achieve the goals of a translational research team, integration and alignment of group members’ scientific knowledge and research experience are necessary to effectively delineate the team direction of the team moving forward. Integration and alignment occurs through the process of social negotiation, a feature of sociocultural learning theory (20) . Social negotiation occurs when individuals communicate about shared problems and develop mutual understanding about problems and their solutions (20). This negotiation phase is critical in defining the team direction, and the ultimate achievement of its goals; it is a process which all teams must experience, and is essential to development of high performing teams (21). Accordingly, the MTT provides real-world experience in inter-professional integration that cannot be taught in a classroom.

Inter-professional communication and support

MTTs regularly interact in face to face meetings. An important aspect of these meetings is the democratic nature of participation by all MTT members. This is particularly important for early-career investigators, in that their contributions may not be valued in a more traditional hierarchical group structure. Presentation by other group members helps to hone the skills of the early-career investigators to present their ideas in larger groups, without the fear of skepticism and rejection, based on their relatively junior stature. This egalitarian approach can promote the development of competencies in experimental design and interprofessional communication, which are important for developing the careers of translational research trainees. In addition, early-career investigators receive cognitive support when they interact with experienced individuals or other peers (e.g., junior faculty, research specialists, biostatisticians) during practice-based research work This process, referred to as scaffolding (22, 23), can assist trainees in attaining a higher level of performance than they would otherwise reach working independently, under a singular primary mentor. Scaffolding also applies to similar level peers, who may be experts in different disciplines, but can help each other learn about their respective areas of specialized knowledge. As such, scaffolding is another potentially important factor in the ultimate development of individual team members, who can then more productively interact with other team members, to contribute to the success of the team and promote its translational research initiatives.

Complementary Mentorship Models

Effective mentoring enhances professional identity and personal competence, resulting in greater research productivity (24). The MTT model provides the framework for team mentoring that extends beyond a conventional one-on-one mentoring arrangement. For example, mid-level and senior investigators have the opportunity to co-mentor novices through regular MTT meetings and discussions. However, we emphasize here that the MTT mentoring model is not meant to replace the more traditional one-on-one mentorship relationships, but rather supplement and enhance it. The closest example that parallels the MTT model would be semi-annual thesis or mentorship meetings; however, the MTT meets more frequently, typically 1–2 times per month, thus increasing the interactions of trainees within a multi-mentor setting. The MTT promotes also promotes the formation of peer networks and peer mentoring. Peer mentoring promotes adaptation to MTT environment and retention within the group.

As mentioned above, an additional advantage of the MTT construct is the regularity of meetings. These frequent interactions keep the mentees on track, and keep the mentors abreast of both positive developments, such as successful experiments and emerging discoveries, as well as negative events, avoiding lost time and effort. In this way, regular, structured MTT interactions promote more rapid skill development. This point is substantiated by education training data from the National Center on Educational Statistics (NCES), as cited by Green and others (25), indicating that “nearly 90% of people working with mentors once/week reported considerable skill improvements, whereas similar improvements were reported in only 36% of people working with mentors several times/year.” Our data, presented below, suggest that this same effect of repeated mentoring can be achieved within the MTT, for the development of early-career translational research investigators.

Team-based Competencies

To better evaluate the impact of the MTT model on training, we identified specific translational team-based research competencies, from a larger set of 99 translational competencies, developed earlier by the CTSA consortium (26). These 9 team-based competencies might be expected to be facilitated by participation in the MTT team-based mentoring and experiment processing model. These include leading multidisciplinary teams, facilitating innovation, mentoring, and fostering creativity (See Table 1). Specific examples of these competencies are presented, which could form the basis for a rubric for skill-based assessment, in the future. Thus, in the sections that follow, we present our initial findings on the impact of the MTT on early-career investigator development, productivity, and skill acquisition associated with attainment of translational research competencies.

Table 1.

Competency Categories and Targeted Team-Based Research Competencies Items

Competency Category	Specific Team-Based Competency Example
Research questions	Identify basic and preclinical studies that are potential testable clinical research hypotheses.
Study design	Formulate a translational research question for study in vivo models and propose study designs for a research question
Sources of error	Assess data sources and data quality to answer research questions.
Biomedical informatics	Develop protocols utilizing management of information
Responsible conduct of research	Assure the need for privacy protection throughout all phases of a study.
Scientific communication	Communicate clinical and translational research findings to different groups of individuals, including colleagues, students, the lay public, and the media.
Translational teamwork	Build and manage an interdisciplinary / intradisciplinary / multidisciplinary team that matches the objectives of the research problem. Manage an interdisciplinary team of scientists. Advocate for multiple points of view. Clarify language differences across disciplines. Demonstrate group decision-making techniques. Manage conflict. Manage a clinical and/or translational research study.
Leadership	Work as a leader of a multidisciplinary research team. Manage a multidisciplinary team across its fiscal, personnel, regulatory compliance and problem solving requirements. Maintain skills as mentor and mentee. Validate others as a mentor. Foster innovation and creativity.
Cross-disciplinary training	Apply principles of adult learning and competency-based instruction to educational activities. Provide clinical and translational science instruction to beginning scientists. Incorporate adult learning principles and mentoring strategies into interactions with beginning scientists Develop strategies for overcoming the unique curricular challenges associated with merging scholars from diverse backgrounds.

RESEARCH AND PRODUCTIVITY OUTCOMES OF CTSA-ASSOCIATED TRAINEES

Through a university-wide effort lead by the CTSA, UTMB currently has a portfolio of 15 active MTTs that include 273 investigators across the four UTMB Graduate Schools. This initiative has had a significant transformative impact on research culture, collectively moving the university from one that valued the individual R01 laboratory, to one that now embraces team-based models for addressing translational science (27, 28). Over the past 5 ½ years, our MTTs have produced over 350 publications, 75 invention disclosures and 45 issued patents, and attracted $288 M in extramural support. Of these MTT publications, 151 are jointly authored publications (957 citations). Our analysis of publication patterns has concluded that as a result of CTSA collaboration, the impact factors of MTT publications have increased, and as a measure of innovation, successful MTTs have begun to publish in new domains (29).

We examined the broad impact of this MTT training environment on CTSA-associated trainee productivity. This descriptive examination was based on: 1) a longitudinal assessment, wherein we tracked trainees associated with the MTTs from July 2009 to September 2013 (representing 4.25 years of CTSA funding), and 2) an outcomes assessment, focused on the number of grants and publications by the trainees. In this case, a CTSA–associated trainee was a broad category, defined as one who was either: 1) a member of an MTT, 2) associated collaboratively with an MTT, or 3) had interaction and/or participation with one or several of our CTSA key resources. These trainees included M.D.- and Ph.D.-early career faculty (assistant professors or instructors), M.D.-fellows, Ph.D.-post doctoral associates, and pre-doctoral graduate students.

Longitudinal Assessments

A total of 222 trainees institution-wide were identified as being associated with the CTSA. Trainees were identified either by administrative reports from the MTTs, or from on-line registration of CTSA trainees, though we acknowledge that some trainees with short-term experiences with MTTs (e.g., less than 1 year) may not have been captured by these collection methods. These omissions were minimal, but would include rotating graduate or medical students with laboratory rotations of only 4–8 weeks.

Of these 222 trainees associated with the CTSA across the institution, 110 (~50%) were directly associated with MTTs (either as members or collaborators), with 29 also accessing other CTSA education and training activities through our Education, Training and Career Development Key Resource. These education and training activities included: 1) CTSA-sponsored clinical and translational seminars, 2) attendance of Scholars group mentoring meetings, 3) participation in CTSA-sponsored leadership workshops, 4) completion of a CTSA-associated certificate training program, and 5) participation in mentoring workshops. We also found that over that time period, 48 trainees (22% of the 222 CTSA-associated trainees, 9 of whom were MTT members), benefitted from utilization of the Clinical Research Center (CRC), a site for patient recruitment and patient-centric research. In this case, having benefitted was defined as: 1) having conducted their own study in the CRC, 2) having conducted a collaborative study in the CRC within another investigator, 3) having conducted a study in the CRC as an MTT project, or 4) having consulted with the CRC staff for guidance regarding a potential future study.

Grants and Publications Productivity

In our exploratory assessment of research productivity data, we found that between 2012–2014, a total of 15 CTSA-associated trainees obtained federal, or other (e.g., foundation and/or pharmaceutical) funding, and 12 trainees had grants under review. Furthermore, 80 CTSA-associated trainees had co-authorships in 164 CTSA publications, institution-wide. This represents an average of 2 papers per trainee over the period assessed, but the emphasis here is that these were collaborative co-authored papers, rather than single-author or first author/senior author papers. We ascribed this outcome as being due to the collaborative nature of the MTTs with which some of the trainees were involved, such that these collaborative team-based papers likely would not have been published by the trainees in the absence of MTTs at the institution. These data suggest that our trainees benefitted in several important ways from CTSA-associated training, and potentially with involvement with MTTs. Further, since publications typically appear several years after completion of a study, we would expect publication numbers by the trainees to increase over time with further work, which is something that we will also assess further, in the future.

TRANSLATIONAL COMPETENCIES ASSOCIATED WITH MTT MEMBERSHIP

Approach

In order to assess the possible influence of the MTT as a potentially impactful model for translational research training, we collected data on self-assessment of confidence in competency development, in a specific sub-set of trainees, who were our translational scholars. Accordingly, in the 2011–12 academic year (about halfway through our CTSA funding cycle, with most of our MTT’s having been in place for ~3 years), we administered an exploratory survey of our institutional Translational Research Scholars Program (TRSP) Scholars (of which our CTSA-supported KL2 Scholars are a subset), to determine all scholars’ perceived confidence in CTSA-recommended competencies development (26). A total of 99 items across 15 competencies were rated within the survey. A subset of the scholars were members of MTTs focused on a range of health problems, such as aging and sarcopenia, pediatric respiratory infections, women’s reproductive health, burn injury, and vaccine development. The objective was to summarize the survey responses and compare the scholar groups with respect to the 15 translational competencies, and also to determine whether membership in an MTT had any effect on self-perceived confidence in competency attainment.

Scholars rated their perceived competence at performing 15 specific translational competencies across categories, that included: 1) Clinical and Translational Research Questions, 2) Literature Critique, 3) Study Design, 4) Research Implementation, 5) Sources of Error, 6) Statistical Approaches, 7) Biomedical informatics, 8) Regulatory Support and Knowledge, 9) Responsible Conduct of Research, 10) Scientific Communication, 11) Cultural Diversity, 12) Translational Teamwork, 13) Leadership, 14) Cross-Disciplinary Training, and 15) Community Engagement. The specific competencies under each category were adapted into survey items, for which scholars rated their confidence regarding the attainment of a particular competency. The subsequent selection of the 9 team-based competencies, from within the list of 15 presented above (Table 1), was not based on pre-testing or validation, but rather by discussion and agreement by CTSA leadership, senior investigators, and institutional education experts, as to what constituted team-based competencies.

We investigated whether the items “hung together” within each competency by using a principle components analysis, utilizing a basis of eigenvalues. If a single eigenvalue explains most of the variance using the items of the competency, this means the competency items are based on an underlying construct or latent variable. Items were consistent within the competency group, based on having a minimum eigenvalue of one. If more than one factor was identified, the item loadings were required to be at least 0.5. For loadings less than 0.5, the items were assessed on Factor 2. For simplicity, if only a few items were on Factor 2, the competency was considered measured by Factor 1, with the low-loading items deleted.

Groups of interest included Scholar level (Associate, Early, and Advanced), and membership on at least one MTT. A comparison of the scholar levels and MTT membership factors was also analyzed by Fisher’s Exact Test. For the competencies analysis, we constructed scales based on a mean of the items in the competency subject to the above limitations. We used means because of the variable number of items for the competencies (from 4 to 10). For the mean scales, we used a two-factor Analysis of Variance (ANOVA) for MTT membership and scholar level. Statistical significance was set at alpha = 0.05. Computations were done with SAS 9.2.

Results

There were a total of 32 scholars surveyed, of which 16 were members of an MTT, while 5 of the total were Associate Scholars, 16 were Early Scholars (2 with NIH funding), and 11 were Advanced Scholars (all with NIH Funding; see Table 2). Membership in an MTT was determined from a report requested from the MTT leaders after the survey was collected from the scholars. Eighteen scholars who participated in the survey were women, and 14 were men, demonstrating a good representation of both sexes in the survey sample cohort; however, the data was not analyzed based on gender. Age and race data collection was not included in the originally-administered survey. Thirty-two scholars completed the survey; a total of 31 respondents had sufficient data for inclusion in our analysis.

Table 2.

Scholar Level by MTT (Count, Row Percentage, Column Percentage)

		MTT Member?
Scholar Level		Yes	No	Total
1: Associate Scholar		1	4	5
Row Percent		20.0	80.0	100.0
Column Percent		6.3	25.0
2: Scholar		9	7	16
Row Percent		56.3	43.8	100.0
Column Percent		56.3	43.8
3: Advanced Scholar		6	5	11
Row Percent		54.6	45.5	100.0
Column Percent		37.5	31.3
Total		16	16	32
		50.0	50.0	100.0
		100.0	100.0

Fisher's Exact test p = 0.5007 (not significant)

The factors of Scholar Level and MTT membership were shown to be not related by the Fisher’s Exact test (Table 2; P=0.5007), suggesting that no selection bias existed, as to the survey-reported scholar level and membership in an MTT. In other words, the scholar levels were randomly distributed between MTT members and non-MTT members, and thus, there was no bias toward having more experienced Advanced Scholars in MTTs.

For the principal components analysis, we found the first principle component , i.e., Factor 1, explained between 45 and 77 percent of the total variation (Table 3, Column 2), suggesting an underlying construct supporting the observed variation. Minimum loadings on Factor 1 were less than 0.45 for the Study Design (III), Biomedical Informatics (VII), Regulatory (VIIIA), and Responsible Conduct (VIIIB) competencies (Column 6). We found that we could simplify the structure of these scales by dropping a single item from each scale (III9, VII7, VIIIA1, and VIIIB6), which resulted in remaining items having loadings greater than 0.5.

Table 3.

Competency principle components analysis

		Largest Eigenvalue	Percent of Total Variance Explained by				Minimum Item Loading Factor 1	Excluded Item
	Competency		Factor 1	Items	Factor 2	Item(s)
	Column	1	2	3	4	5	6	7
I	Clinical & Translational Research Questions	3.15	0.45	3–7	0.17	1–2	0.64
II	Literature Critique	3.57	0.51	1–7	0.15	5	0.63
III	Study Design	4.29	0.48	1–8	0.16	9	0.16	9
IV	Research Implementation	1.99	0.50	1–3	0.30	4	0.59
V	Sources of Error	3.96	0.57	2–7	0.15	1	0.61
VI	Statistical Approaches	6.07	0.61	1–6,8–9	0.13	7,10	0.56
VII	Biomedical Informatics	6.12	0.68	1–6,8–9	0.11	7	0.22	7
VIIIA	Regulatory Support and Knowledge	2.71	0.45	3–6	0.20	1	0.33	1
VIIIB	Responsible Conduct of Research	3.90	0.56	1–5,7	0.20	6	0.44	6
IX	Scientific Communication	3.38	0.68	1–5	*		0.69
X	Cultural Diversity	3.72	0.74	1–5	*		0.66
XI	Translational Teamwork	4.09	0.58	1–7	0.16	5,7	0.67
XII	Leadership	3.27	0.65	1–5	*		0.74
XIII	Cross Disciplinary Training	3.03	0.75	1–4	*		0.81
XIV	Community Engagement	3.86	0.77	1–5	*		0.83

Bold indicates team-based competencies per Table 1.

^*Second eigenvalue less than 1

The sample sizes, means across scholars, standard deviations, minimums and maximums are shown in Table 4. All competencies (with the exception of Research Implementation and Community Engagement) had the maximum mean of 5, indicating presence of high perceived confidence by some respondents for 13 of the 15 translational competencies. Conversely, Biomedical Informatics, Cross Disciplinary Training, and Community Engagement had minimums of zero, indicating presence of low perceived confidence by some respondents, for those 3 translational competencies. Biomedical Informatics and Literature Critique had the lowest and highest means, respectively, across scholars, and Biomedical Informatics was also the most variable (SD =1.24).

Table 4.

Summary statistics for competencies: respondent n, mean, standard deviation, minimum and maximum.

Variable	Label	N	Mean	Std Dev	Minimum	Maximum
ctrq	Critical and Translational Questions	31	4.12	0.50	2.71	5
lc	Literature Critique	31	4.37	0.53	2.71	5
sd	Study Design	31	3.93	0.61	2.57	5
ri	Research Implementation	31	3.31	0.67	2	4.5
se	Sources of Error	30	3.72	0.72	2	5
sa	Statistical Approaches	31	3.85	0.79	1.6	5
bi	Biomedical Informatics	30	2.89	1.24	0	5
rsk	Regulatory support and knowledge	29	3.74	0.79	2	5
rcr	Responsible conduct of research	30	3.96	0.88	1.5	5
sc	Scientific Communication	31	3.47	1.01	1.6	5
cd	Cultural Diversity	29	3.57	1.14	0.8	5
tt	Translational Teamwork	30	3.74	0.80	1.29	5
l	Leadership	31	3.81	0.73	2	5
cdt	Cross Disciplinary Training	31	3.37	1.04	0	5
ce	Community Engagement	27	2.91	1.06	0	4.2

Bold indicates team-based competencies per Table 1.

We computed ANOVA for each competency scale across the groups; the means for the groups are shown in columns 1, 2, 4–6, of Table 5. The associated variance (MSE) is in column 8. We found there was a statistically significant effect of MTT membership for the competencies of Study Design (III, P=0.0011), Research Implementation (IV, P= 0.0493), and Statistical Approaches (P=0.0161). We also noted that Leadership (XII; P=0.0541) came very close to statistical significance, as a function of MTT membership. Scholar Level was also observed to be associated with Study Design (P=0.0463). There were no statistically significant differences observed for the remaining competency scales.

Table 5.

Means, p-values, and mean square errors by MTT Membership and Scholar level.

		MTT Membership Means			Scholar Level Means				MSE	n
		Yes	No	p-value	Associate	Scholar	Advanced	p-value
	Competency	1	2	3	4	5	6	7	8	9
I	Clinical & Translational Research Questions	4.28	3.96	0.1411	3.89	4.18	4.15	0.7746	0.2397	31
II	Literature Critique	4.51	4.22	0.0993	4.43	4.31	4.41	0.6613	0.2828	31
III	Study Design	4.27	3.58	0.0011*	3.43	3.88	4.23	0.0463*	0.2157	31
IV	Research Implementation	3.56	3.05	0.0493*	3.10	3.31	3.42	0.8493	0.4211	31
V	Sources of Error	3.96	3.48	0.1029	3.40	3.72	3.87	0.6591	0.4941	30
VI	Statistical Approaches	4.16	3.53	0.0161*	3.82	3.68	4.11	0.2428	0.5171	31
VII	Biomedical Informatics	3.17	2.61	0.1914	2.93	2.57	3.27	0.3340	1.5001	30
VIII A	Regulatory support and knowledge	3.63	3.84	0.5030	3.76	3.64	3.86	0.8351	0.6793	29
VIII B	Responsible Conduct of Research	3.94	3.97	0.9770	4.10	3.83	4.05	0.7738	0.8419	30
IX	Scientific Communication	3.69	3.24	0.3521	2.96	3.48	3.69	0.5620	1.0252	31
X	Cultural Diversity	3.64	3.49	0.7749	3.32	3.52	3.78	0.7858	1.4283	29
XI	Translational Teamwork	3.98	3.50	0.1419	3.39	3.70	3.97	0.5085	0.6164	30
XII	Leadership	4.08	3.52	0.0541	3.44	3.81	3.98	0.6169	0.4783	31
XIII	Cross Disciplinary Training	3.63	3.09	0.1841	3.10	3.23	3.67	0.5120	1.0646	31
XIV	Community Engagement	3.08	2.75	0.5462	2.65	3.04	2.84	0.8736	1.2214	27

P-values based on 2-way ANOVA Type III Sum of Square.

Bold indicates team-based competencies per Table 1.

Multiple comparison grouping for Scholar level only.

In general, the MTT members reported higher numerical means for all competencies, except Regulatory Support (VIIIA) and Responsible Conduct of Research (VIIIB), the latter means of which differed by only three one-hundreths of a score point, leading to P values approaching one, and thus a potential similarity, or possibly no change due to MTT membership. There is no clear overall pattern of mean changes across scholar levels, although for Study Design, the means were positively associated with increasing scholar level, suggesting that the advanced scholars had higher perceived confidence in that translational competency (mean score=4.23), and possibly that early scholars supported by the CTSA also had higher confidence (mean score=3.88), than associate scholars not directly supported by the CTSA (mean score=3.43).

DISCUSSION

From survey analyses, we concluded that according to the scholars’ perceptions, the development of a number of translational research competencies, some of which were team-based competencies, was significantly associated with MTT membership. Specifically, MTT membership was associated with scholars’ confidence in the translational competency categories of Study Design, Research Implementation, and Statistical Approaches. For most of the competencies the variability was modest. The exception was Biomedical Informatics, with an MSE of 1.5001 (Table 5), which suggests that special training in Biomedical Informatics should be undertaken for all scholars.

A positive impact of MTT membership on translational training in this initial study suggests that it may be worthwhile re-administering the survey in a subsequent year, with some revision, refinement, and validation. We acknowledge that competency assessment is a complex and challenging process, because being competent goes beyond just the recall of knowledge; it also includes one’s ability to apply knowledge to everyday problems, or activities, with success, in specific contexts or situations (30–32).

Within the nine targeted translational team-based research competencies we analyzed, we found that the mean score value for the Translational Teamwork competency associated with MTT membership increased by 0.48, but did not achieve statistical significance (P= 0.1419, Table 5). We acknowledge that the survey was given at a time that was early to mid-stage in the MTT lifespan of many of our MTTs, and that we have documented more recently, an evolution in their progress, focus, and success (16). Thus, it is possible that the interval of time that we studied for MTT membership was not sufficient for scholars to feel they had achieved this particular competency. We suspect that with an increased scholar sample size, and continued MTT participation over time, a measurable impact would conceivably be measurable. Thus, Translational Teamwork remains a team-based competency category of significant interest, for which we believe further research is warranted.

We believe that the MTT provides a strong mentoring construct for all scholars and trainees who are members. However, it is important to point out that the published evidence to support the perception that mentoring is an important part of academic medicine training is not strong (33), and may require studies using more rigorous methods and innovative approaches. For example, in a review of the literature, Sambunjack et al., (33) reported that in some fields fewer than 20% of faculty members had a mentor. Clearly, for our early-career KL2 Scholars, our approach of assigning a primary mentor, the requirement of MTT membership, and the subsequent accessibility to career mentors in both the MTT and TRSP, addresses this shortcoming, and appears to have had a measurable effect on the development of some self-reported translational competencies.

There has been recent support for the fact that a variety of approaches to team-based training may be necessary, with a survey of CTSA institutions indicating that the vast majority of education leaders consider team-based interdisciplinary training as important, but that only about one-half of CTSA institutions offer such training (34). With our MTT-based approach, we sought to embed our KL2-supported and TRSP Scholars within our teams conducting translational health science projects, and also support them with translational science and career development through the TRSP, to immerse them in translational science activities and concepts. Our survey and longitudinal data suggest that this approach has early indications of success in both trainee collaborative productivity, and confidence building with respect to translational competencies.

LIMITATIONS OF STUDY

The wording of the items for the scoring scales was taken directly from the consensus-driven wording of the CTSA competencies (26). It is likely that these competency items need to be reviewed and rewritten in order to minimize ambiguity, particularly in the context of team-based competencies. Furthermore, it is acknowledged that the original 99 CTSA competencies were written for a Master’s level of competency achievement (26), which, again, should likely be revisited in light of the emphasis on KL2 scholars within CTSAs, who are typically at an assistant professor level, and potentially far beyond the Master’s level. Alternatively, the questions could be replaced by other methods to assess the competency, rather than relying on self-report. We intend to explore such modifications in the future, to better assess competency achievement in early-stage scholars, as it relates to MTT membership.

We acknowledge that our scholar sample size in this report is modest, and that the competencies were assessed by self-report, and included team-based competencies that were pre-selected by agreement. The scholars were equally involved in the meetings and mentor discussions; therefore, all TRSP individuals’ had the same pre-survey coaching, which minimizes bias. However, it is possible that these factors may have impacted the power of the study, and possibly the likelihood of finding associations between more competency categories. Also, we did not consider the data we collected on whether the scholars felt they needed improvement, because the responses were unscored handwritten answers, not amenable to simple statistical assessment. Future surveys should likely attempt to assess scored criteria related to this topic area. We also did not control for variables such as scholar prior knowledge or mentor, which could influence outcomes. Future studies should likely take these factors into consideration.

Finally, we note that true competency development, including enduring and recognized expertise, is a longitudinal process that requires hours of practice and multiple forms of study and repetition, to reach proficiency, regardless of discipline (35). Thus, while the results of our competency self-reporting survey are encouraging, we recognize that important measures of successful translational competency attainment for our scholars will be their success in future translational productivity, team leadership, and academic promotion. As such, a more comprehensive longitudinal competency assessment should be done through multiple methods, such as observational rubrics, in assessing team-based outcomes associated with MTT membership and participation.

CONCLUSIONS

It has been pointed out by Larson, et al., (36) that although universities may be prepared to conduct interdisciplinary research, well-developed programs for faculty development and trainee preparation are lacking. To address this deficiency, we pursued an approach in which early-career investigators were placed within a translational team-based research environment within an MTT, in which they were a member of a large team, with ample opportunity for repeated interactions with senior-level faculty in the conduct of translational research projects. Our results suggest that early-career investigator participation within the MTT model can facilitate the attainment of important translational competencies, and that this could be a model of choice for institutions attempting to train the next generation workforce of translational team-based scientists.

Abbreviations

CTSA

clinical translational science award

ITS

Institute for Translational Sciences

MTT

multidisciplinary translational team

TRSP

translational research scholars program

UTMB

University of Texas Medical Branch