Core informatics competencies for clinical and translational scientists: what do our customers and collaborators need to know?

Abstract

Since the inception of the Clinical and Translational Science Award (CTSA) program in 2006, leaders in education across CTSA sites have been developing and updating core competencies for Clinical and Translational Science (CTS) trainees. By 2009, 14 competency domains, including biomedical informatics, had been identified and published. Since that time, the evolution of the CTSA program, changes in the practice of CTS, the rapid adoption of electronic health records (EHRs), the growth of biomedical informatics, the explosion of big data, and the realization that some of the competencies had proven to be difficult to apply in practice have made it clear that the competencies should be updated. This paper describes the process undertaken and puts forth a new set of competencies that has been recently endorsed by the Clinical Research Informatics Workgroup of AMIA. In addition to providing context and background for the current version of the competencies, we hope this will serve as a model for revision of competencies over time.

INTRODUCTION

Over the past decade, the training and evaluation of informatics professionals has become increasingly competency-based.^1–7 As the field continues to evolve, competencies must be adjusted accordingly. Most efforts to define informatics competencies have been regarding those to be demonstrated by informatics professionals. Relatively little work has focused on the skills and knowledge needed by people who collaborate with informaticians but are not themselves informatics professionals. By 2009 the competencies developed by the National Institutes of Health (NIH)-funded Clinical and Translational Science Award consortium included 14 informatics competencies for researchers collaborating with informaticians.⁸ This paper describes the process undertaken to revise those competencies and provides context for the resulting set of competencies, which have been endorsed by the Clinical Research Informatics Workgroup (CRI-WG) of AMIA. In addition to providing context and background for the current version of the competencies, we hope the deliberative process described will serve as a model for revision and updates of competencies over time.

BACKGROUND AND REVISION PROCESS

Many CTSA sites operate master’s degree programs that target clinicians who are interested in clinical research or translational research as a career. The intent of the competencies developed was not to make CTS trainees experts in any of the competency areas. Rather, the objective was to enable CTS trainees to function as part of “multidisciplinary teams of clinician-scientists,”⁸ to further their ability to effectively interact and collaborate with domain experts in biomedical informatics. From the beginning, many of the CTSA programs have included formal training in informatics, typically a one-semester class (Figure 1). These competencies were to address both the factual knowledge (information that one can recite or discuss) and the skills (ability to accomplish a task) needed by clinical investigators. A fuller discussion of knowledge versus skills is available in Epstein et al.⁹

Figure 1:

Survey of Education committee participants regarding which scenario best describes the training available at their institution for C & T scientists (multiple answers allowed). Percentages based on 31 responding sites. Duplicate answers from the same site were omitted.

Although the 2009 version of the competencies was widely adopted across the CTSA consortium, it became clear over time that both content and pedagogical issues not addressed by them had emerged, including the rapid adoption of EHRs, the growth of biomedical informatics, and the realization that some of the competencies had proven to be difficult to measure in practice. Consequently, the workgroup tasked to revise the competencies needed to incorporate both domain expertise from the Informatics committee and training expertise from the Education committee. This conscious decision to include both perspectives proved important not only for developing measurable competencies, but also for targeting the competencies at the right level of difficulty considering the target audience: these competencies are focused on informatics competencies for master’s-level CTS trainees rather than for informatics trainees. This distinction was particularly relevant because many CTSA sites also had graduate informatics degree programs (Figure 1). While narrow in target audience, the competencies needed to cover the entire translational research continuum, commonly referred to as T1–T4,¹⁰ addressing both what CTS trainees should know (knowledge) and what they should be able to do (skills).¹¹

The process of competency revision was iterative, building consensus once all members initially contributed their individual responses to the framing questions. Table 1 summarizes the competency revision timeline. (More detail is provided in the digital supplement.) Table 2 summarizes the evolution of the individual competencies showing the original competencies, the January 2014 version, and the CRI-WG recommended changes. Initially, workgroup members evaluated each existing competency, evaluating it for relevancy both to the state of the art and to the entire T1–T4 continuum. When problems were identified, potential remedies included refinement, major revision, or deletion. Finally, missing competencies were added. Once the list of competencies was developed, they were rewritten to reflect measurable knowledge and skills for CTS trainees at the master’s level, based on Bloom's Revised Taxonomy of Educational Objectives.¹² The competencies were tailored to reflect educational objectives at the “Remember, Understand, and Apply” levels of Bloom’s taxonomy, because the goal was to create CTS researchers who understood the important factors governing informatics in CTS research and could collaborate effectively with informaticians, rather than to create fully trained informaticians. In contrast, it was assumed that informatics professionals would function at the “Analyze, Evaluate, and Create” levels of the taxonomy.

Table 1:

Timeline and major milestones in competency revision process

Period	Activity
February 2013	General discussions of interest in revision by CTSA Education and Informatics Key Function Committees (KFCs)
April 2013	Solicitation of joint workgroup members
June 2013	Biomedical Informatics Training Workgroup (BITWG) formally launched to oversee revisions
Summer–Fall 2013	Series of teleconferences involving evaluation of existing competencies, proposed alternatives, and iterative revisions
November 2013	Completion of first draft of revised competencies by BITWG
January 2014	Presentation to Education and Informatics KFCs, minor revisions made
February 2014	Sunset of CTSA KFCs
Spring 2014	Presentation of competencies to Education Workgroup of the Association for Clinical and Translational Science and Clinical Research Informatics Workgroup (CRI-WG) of AMIA
Summer 2014	Comments solicited from AMIA CRI-WG general membership
Fall 2014	Series of teleconferences with AMIA CRI-WG leadership, with iterative refinements
November 2014	Endorsement of competencies by AMIA CRI-WG leadership
Spring 2014	AMIA CRI-WG submits competencies for formal AMIA endorsement
September 2015	AMIA Executive Committee endorses CRI-WG recommendation and final version13

Table 2:

This summarizes the evolution of the CTS informatics competencies showing the original competencies, the January 2014 version, and the November 2014 final version, as well as the rationale for changes at each stage

Orig. No.	Original Competency (2009)	Issues Identified in BITWG Discussions	Revised Competency (January 2014)	Issues Identified in CRI-WG Discussions	Final No.	Final Competency (November 2014)
1	Describe trends and best practices in informatics for the organization of biomedical and health information	Overly broad; not focused on CTS	Describe and make use of best practices for managing, protecting, and analyzing biomedical and health information in clinical and translational research	Overly broad; subsumes several more specific competencies below; removed and converted to Goal and Scope introduction to competencies document
2	Develop protocols utilizing management of information using computer technology	Overly broad; overly ambitious for CTS researchers; considered more appropriate for informatician	Utilize informatics-based tools in translational research including: understand which informatics tools are relevant; be able to locate those tools; and use those tools in research for managing and analyzing biomedical and health information		1	Utilize informatics-based tools in translational research including: locate relevant informatics tools; select appropriate informatics tools; and use those tools in research for managing and analyzing biomedical and health information
3	Describe the effects of technology on medical research, education, and patient care	Overly broad	Dropped
4	Describe the essential functions of the electronic health record (EHR) and barriers to its use	Overly clinical; refocused to emphasize research	Describe the essential functions of the EHR and its feeder systems (eg, radiology, lab) and the challenges to using these data in clinical and translational research	Broadened to cover clinical systems in general; global change: “translational research” replaced by “clinical and translational science (CTS)” where appropriate	2	Describe the essential functions of the major clinical systems (eg, the EHR and its feeder systems, such as radiology and lab) that are relevant to CTS and the challenges to using these data for research
New				Needed parallel competency to new competency #2 to cover research computer systems	3	Describe the essential functions of major research computer systems (eg, clinical trials management systems, biospecimen management systems, research grant and finance management systems, and research services tracking systems) that are relevant to CTS
New				Gap identified: no discussion of role of organizational factors in informatics	4	Compare and contrast the organizational roles and principal responsibilities essential for access, management, and governance of data related to CTS
5	Explain the role that health information technology standards have on the interoperability of clinical systems, including health IT messaging	Overly broad; overly clinical; focus on messaging overly narrow; refocused to broaden the types of systems, but narrowed to emphasize research	Explain the role that health information technology standards have on the interoperability of research, clinical, and administrative information systems and on secondary use of data for translational research	See global change statement above	5	Explain the role of health information technology standards in the interoperability of research, clinical, and administrative information systems and on secondary use of data for CTS
6	Access patient information using quality checks via EHR systems	Too specific	Dropped
7	Retrieve medical knowledge through literature searches using advanced electronic techniques	Too basic: need to be able to do more than simply retrieve; add examples for clarity	Access and critique biomedical and health science knowledge through literature searches using advanced search techniques (PubMed, Google Scholar, etc.)	Added MeSH to list; increased specificity of competence (identify, retrieve, and manage); changed examples to “such as”	6	Identify, retrieve, and manage biomedical and health science knowledge through literature searches using advanced search techniques (MeSH, PubMed, Google Scholar, etc.)
8	Discuss the role of bioinformatics in the study design and analyses of high dimensional data in areas such as genotypic and phenotypic genomics	Too narrow; focus broadened beyond bioinformatics	Describe the essential information management, analysis, and visualization methods that apply to translational research data: genomic, proteomic, and other omic data; τhe EHR and its data; and population-level data	Bullet list changed to “such as” to make clear list is not exhaustive; added to examples: clinical data; imaging data; consumer and patient-reported data; and environmental exposure data	7	Describe the essential information generation, management, analysis, transformation, summarization, and visualization methods that apply to CTS data, such as: genomic, proteomic and other “-omics” data; clinical data; imaging data; consumer and patient-reported data; and population-level and environmental exposure data

Orig. No.	Original Competency (2009)	Issues Identified in BITWG Discussions	Revised Competency (January 2014)	Issues Identified in CRI-WG Discussions	Final No.	Final Competency (November 2014)
9	Collaborate with bioinformatics specialists in the design, development, and implementation of research projects	Too advanced for master’s curriculum; refocused on specific knowledge and skills that will enable this collaboration	Illustrate the nature of the contributions in consulting and/or collaborating with biomedical and health informaticians throughout the life cycle of individual clinical translational research projects: understand the terminology and principles of biomedical informatics in sufficient depth to interact effectively with informaticians; understand the different subdomains of biomedical informatics in order to identify the appropriate informatics consultant or collaborator; and enumerate the roles of biomedical informatics specialists in the design, development, and implementation of translational research projects	No change	8	Illustrate the nature of the contributions in consulting and/or collaborating with biomedical and health informaticians throughout the life cycle of individual clinical and translational research projects:use the terminology and principles of biomedical informatics to interact effectively with informaticians; discriminate among the different subdomains of biomedical informatics in order to identify the appropriate informatics consultant or collaborator; and enumerate the roles of biomedical informatics specialists in the design, development, and implementation of translational research projects
New		Gap identified: previous version developed before current privacy and security concerns	Discuss the ethical, legal, privacy, and security implications of biomedical and health informatics	Social implications added	9	Discuss the fundamental principles and practices that address the ethical, legal, social, privacy, and security implications of biomedical and health informatics
New		Gap identified: structuring of data in information not adequately addressed	Identify how structure and organization can impact researchers’ translational research foci through fundamental data categories and elements, terminologies and taxonomies, and ontologies	Corrected ambiguity by adding “of information in a domain”	10	Identify how structure and organization of information in a domain can impact researchers’ translational research foci through fundamental data categories and elements, terminologies and taxonomies, and ontologies
New		Gap identified: inadequate attention to newer analytic techniques, such as visualization	Describe commonly utilized data visualization methods for performing research and communicating research to specific audiences	Visualization handled differently from other data handling methods; removed and subsumed under 7 above, and addressed in 11 below
New				Gap identified: need to address literacy and cultural issues broadly in CTS	11	Describe and make use of best practices for developing research instruments and protocols and for communicating results of research that are appropriate to specific audiences and special populations, addressing literacy and numeracy, cultural attitudes, and special terminologies

DISCUSSION

Over the years, multiple initiatives have defined biomedical informatics competencies for various target groups. This work began primarily in the late 1980s¹ and has continued.^2,3 Much of this work has focused on competencies for those who identified themselves as representing different specialties in the field, including biomedical informaticians,^2,4 bioinformaticians,⁵ clinical informaticians,⁶ and nurse informaticians.⁷ Other work has focused on informatics competencies for other practitioners, including medical students,¹⁴ pharmacists,¹ practicing nurses,^15,16 and public health workers.¹⁷ As expected, the informatics competencies for non-informatician practitioners require a significantly lower level of mastery than would be expected of professionals who identify themselves primarily as informaticians. In developing these CTS informatics competencies, the workgroup was acutely aware of this distinction. The goal was explicitly not to primarily develop competencies for translational informaticians or clinical research informaticians. Rather, the goal was to develop competencies for translational researchers who will be the “customers” and collaborators of full-time informaticians. In addition, the competency development was grounded in the observation that many CTSA sites offer a CTS master’s degree that has a one-semester informatics class. In selecting Bloom’s taxonomy terms,¹² the workgroup attempted to describe the level of mastery that would enable researchers to be successful informatics “customers.” It was assumed that these CTS researchers would not be doing significant informatics projects on their own. Instead, they would be collaborating with informaticians trained at the level of the doctoral degree.

In retrospect, it is clear that an important factor in the success of this project was the creation of a core writing group that could iterate rapidly to incorporate input from the larger BITWG. Equally important was the fact that this core contained members from each of the major stakeholder groups. As compared with the original 2009 version, the new competencies were written to reflect potential curriculum goals and objectives. The new competencies are expressed at a much higher level of specificity, which simplifies their translation to course syllabi and content. Another change from the original competencies was the reordering of the topics into a more intuitive sequence, with specific examples in “such as” lists within the competencies. Finally, the competencies were rewritten, moving from a discipline focus (bioinformatics in original competency 8; medical/clinical informatics in original competencies 4–6) to one that reflects application of a method (eg, visualization in new competency 7) that can be applied to a variety of data types derived from the disciplines, from molecular/omic data to population data. The changes made by the AMIA CRI-WG demonstrated the value of including input from individuals outside of the CTSA consortium. We believe that the resulting competencies¹³ will be valuable to a variety of trainees beyond the CTSA programs who need to know about informatics, but who are not training to become informaticians themselves. Anecdotally, the Northwestern University Big Data Steering Committee has adopted these competencies as guidance for a new course, Data Science for Clinical, Translational, and Population Researchers, targeted at students in a number of health sciences graduate programs.

Future work in this area should include an extension beyond the master’s-level trainee, to the level of PhD and the independent researcher without the benefit of either a master’s or PhD degree in CTS. What should they know? What should they be able to do? Similarly, CTS informatics is emerging as a discrete discipline, which would benefit from defined competencies for full-time CTS informaticians. As noted above, the workgroup initially attempted to extend this work to address explicitly those deeper levels of mastery that would be expected of CTS doctoral students or students in CTS informatics programs. While the group was able to reach consensus for the CTS trainee competencies presented here, developing consensus for all these different levels of skill acquisition and mastery proved an intractable task.

The authors hope that these competencies can contribute to the development of shared curriculum among CTSA sites, which would assist institutions with limited access to graduate education in informatics to build CRI competency among their CTS trainees.

CONTRIBUTORS

A.V., E.M., and J.S. led the Biomedical Informatics Training Workgroup and developed the draft competencies with input from the larger workgroups listed below. A.V. created the original draft and led the selection of Bloom’s taxonomy. J.S. and U.T. managed interactions with the Informatics Key Function Committee. A.V. and E.M. managed interactions with the Education KFC. U.T. managed the interactions with the AMIA Clinical Research Informatics Workgroup. All authors contributed to the editing and revision of the document. A.V. and J.S. conducted revisions based on reviewer comments. All authors approved the final manuscript.

FUNDING

This work was supported in part by the National Center for Advancing Translational Sciences, National Institutes of Health, through its CTSA grant award UL1TR000050 at the University of Illinois at Chicago, UL1TR000003 at the University of Pennsylvania, UL1TR000427 at the University of Wisconsin – Madison, and UL1TR000150 at Northwestern University. The work is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

COMPETING iNTERESTS

The authors have no competing interests.

SUPPLEMENTARY MATERIAL

Supplementary material is available online at http://jamia.oxfordjournals.org/.