Development of a core competency framework for clinical informatics

Abstract

Objectives

Until this point there was no national core competency framework for clinical informatics in the UK. We report on the final two iterations of work carried out in the formation of a national core competency framework. This follows an initial systematic literature review of existing skills and competencies and a job listing analysis.

Methods

An iterative approach was applied to framework development. Using a mixed-methods design we carried out semi-structured interviews with participants involved in informatics (n=15). The framework was updated based on the interview findings and was subsequently distributed as part of a bespoke online digital survey for wider participation (n=87). The final version of the framework is based on the findings of the survey.

Results

Over 102 people reviewed the framework as part of the interview or survey process. This led to a final core competency framework containing 6 primary domains with 36 subdomains containing 111 individual competencies.

Conclusions

An iterative mixed-methods approach for competency development involving the target community was appropriate for development of the competency framework. There is some contention around the depth of technical competencies required. Care is also needed to avoid professional burnout, as clinicians and healthcare practitioners already have clinical competencies to maintain. Therefore, how the framework is applied in practice and how practitioners meet the competencies requires careful consideration.

Introduction

The healthcare sector in many countries is facing increasing demand as people live longer and healthier lives.¹ The public’s expectation of healthcare is also increasing and is tempered by various financial constraints. The healthcare sector has lagged behind other sectors regarding its adoption and use of digital technology. In the UK, the Topol review was carried out to assess how the healthcare workforce can be prepared for the digital future. The review makes many recommendations on the use of genomic technology, robotics, artificial intelligence (AI) and digital medicine, including the training and education of healthcare professionals in such areas.¹ At the cutting edge of this digital upskilling of the workforce are informaticians from clinical, health and social care disciplines.

The American Medical Informatics Association (AMIA) defines clinical informatics as ‘the application of informatics and information technology to deliver healthcare services’.² The UK Faculty of Clinical Informatics (FCI) defines a clinical informatician as: ‘A clinical informatician uses their clinical knowledge and experience of informatics concepts, methods and tools to promote patient and population care that is person-centred, ethical, safe, effective, efficient, timely, and equitable’.³ (statement 3, P22)

As yet there are no UK-based overarching competency frameworks aimed at multiple informatics disciplines. Instead existing frameworks tend to focus on specific domains such as nursing or bioinformatics.⁴ The UK FCI was created to provide support for clinical informaticians, including those with clinical roles in the health and social care domains applying informatics in practice. It is the intention of the FCI to provide and accredit competencies for informaticians. This includes accreditation of the UK’s National Health Service (NHS) Digital Academy programme which aims to create digital leaders for the digital transformation of the NHS. The present study forms part of a programme commissioned by the FCI to create a national competency framework for clinical informaticians in the UK.

Competency describes the behaviours, characteristics, skills, attitudes and knowledge application used to successfully achieve something. Competence therefore is the achievement of a single competency or multiple competencies. A core competency framework describes the essential set of competencies required to achieve competence in a specific area. There currently exist many competency frameworks aimed at various clinical informatics disciplines, such as the ELIXIR (the European life-sciences Infrastructure for biological Information) and TIGER (Technology Informatics Guiding Education Reform) frameworks for bioinformaticians and nurses. This paper reports on the methods used to generate and refine the UK FCI’s Core Competency Framework (CCF), which covers the core competencies required to develop clinical informaticians’ professional competencies, and to provide a process for the FCI to provide accreditation for training and education programmes and individual clinical informaticians.⁵

Background

The Core Competencies Project spans two primary phases. The first part of the work explored the definition of the professional attributes of a clinical informatician. The FCI carried out phase 1 of the work (figure 1) which consisted of three reports based on (1) discovery,⁶ (2) validation⁷ and (3) consultation around the output competences of a clinical informatician.³ This included defining clinical informatics, clinical informaticians, inclusivity, professional boundaries and the functional domains (scope) of clinical informatics practice. This work was carried out using qualitative methods (eg, interviews) with the faculty’s membership. The second phase (reported in this paper) involved an iterative process consisting of three separate iterations. The first iteration involved the combination of findings from both academia and industry in the form of an analysis of over 50 informatics job postings and a systematic literature review which explored the commonality of informatics competencies across different clinical informatics domains (such as medicine, nursing, pharmacy).⁴ Following synthesis of this information, an initial draft competency framework was generated which was then presented to 15 informatics experts in one-to-one semi-structured interviews and adapted following feedback. The amended version was presented for wider evaluation through a digital survey with 87 participants. In light of the survey results, the final version of the competency framework⁵ was updated and disseminated publicly on the FCI’s website.

Figure 1.

Overview of the core competencies project. The focus of this paper is the reporting of the final two iterations and dissemination from phase 2, as seen in the darker highlighted section.

Figure 1 provides an overview of the iterative steps followed to create the final core competency framework. This paper reports on the interviews and survey (iterations 2 and 3 of phase 2). The principle research questions were:

• RQ1: Was the mixed-methods approach chosen appropriate for generating a core competency framework?

• RQ2: How do participants feel the framework should be applied and to whom?

• RQ3: What competencies were considered ‘core’ and how do they fit in with an evolving profession?

Iteration 1: interviews

Methods

Semi-structured interviews were carried out with participants (n=15) involved in various aspects of informatics. Participants were sent a copy of the interview schedule (see online supplemental appendix box 2) and the framework prior to the agreed interview date. Due to the coronavirus lockdown, the interviews were carried out online using Zoom and audio recorded. Transcribed interviews were analysed using framework analysis.⁸ Framework analysis involves five stages consisting of familiarisation with the data, identification of the thematic framework, indexing, charting and mapping/interpretation. We followed the Standards for Reporting Qualitative Research (SRQR) guidelines.⁹ AD stratified the interview quotes into themes. A second coder (JM) independently matched the quotes to the primary themes. The percentage agreement between both coders was 85.6% with an unweighted Cohen’s kappa showing substantial agreement (κ=0.67).

The sample

A purposive sample was obtained through the FCI who sought to contact a wide variety of its members and associates with different professional backgrounds. Participants were from a broad set of backgrounds (table 1). Participants were involved in informatics at various levels from being the sole informatician in an organisation through to Chief Clinical Information Officers (CCIOs) and digital leads. We omit the current roles of participants to maintain anonymity. Sectors of employment included the NHS, adult social care and the prison system. Age and other demographic details were not requested. The clinical backgrounds were also varied and included general practice, urological surgery, pathology and oncology. Nurses represented multiple clinical areas across their careers in both the NHS and private sector (including nursing homes).

Table 1.

Demographics and background summary showing the number and percentage of interview participants

Background	n (%)
Biomedical scientist	1 (6.6)
Physiotherapist	1 (6.6)
Paramedic	1 (6.6)
Software engineer	1 (6.6)
Pharmacist	2 (13.3)
Senior lecturer (bioinformatics, social care)	2 (13.3)
Nursing	3 (20)
Medical (three general practitioners, one consultant)	4 (26.6)

Researcher characteristics and reflexivity

Following the SRQR guidelines,⁹ details of the researchers' characteristics and personal attributes that may influence the research are detailed. AD has a background in Computer Science (PhD) and Nursing Science (BSc). AD works as a senior lecturer in Health Informatics and Health Data Sciences and was previously a software engineer/data scientist in industry and a former cardiac nurse. AD did not personally know any of the interviewees that took part in the study with the exception of one lecturer whom he knows in a professional capacity. AD’s research paradigm is situated in post-positivism. JM’s background is in Health Psychology (MSc, PhD). JM formerly worked as a lecturer in healthcare sciences teaching a course unit on Digital Public Health, and currently works as a Research Associate in behavioural weight management. JM did not personally know any of the interviewees. JM’s research paradigm is situated in constructivism.

Results

A total of 8 hours and 25 min (M=33.66, SD=12.28) of interview data were generated. Given the nature and topic of the interviews the results are reported in two main ways. The first set of results pertains to the required changes to the composition of the framework and to specific areas such as a specified competency. This does not require any interpretive themes and is reported descriptively. The second set of results uses framework analysis to identify themes around the potential application of the framework, who it is for and how the participants see the informatics profession changing in the future.

Requested changes to the framework

Table 2 provides an overview of the suggested changes to the framework broken down by topic and subtopic. This shows how many participants mentioned this topic and how many times.

Table 2.

Main topics and subtopics of suggested change with number of participants requesting change and number of times mentioned by participants

Topic/subtopic	Participants	References
Framework composition
Language and terminology	12	55
Missing competency	12	35
Representative of core competency	9	31
Target level/granularity	13	38
Specific changes
Change to competency	9	51
Change to diagram or preamble	10	20
Change to structure, order or grouping	7	18

Framework composition

Language and terminology

It was mentioned (n=2) that the wording should make specific reference to ‘social care’ rather than just ‘healthcare’:

you just get people looking at it, and thinking, that doesn’t apply to me. And then they never revisit it. So, it’s that first impression of appearing as welcoming as they actually are, and showing from within the competencies, that they recognise that it’s not about a job title, it’s about the roles you are doing. – P2

Social workers, the social care sector tends to notice these things, if they feel it’s too contextualised within healthcare than social care. […] And you have some healthcare professionals who really lean more towards working in the community for instance. So their roles overlap much more with their social care, social work side of the system. And so it would be helpful for that to be reflected as well. – P8

Missing competency

Table 3 summarises what participants viewed as missing competencies.

Table 3.

Summary of missing competencies identified by participants

Participant	Missing competency
P1	1. Choosing between types of hardware and technology.
	2. How data is transmitted between systems.
	3. Accessing ethical and legislative requirements.
P4	1. How to access and use evidence bases (ie, collating evidence for a start-up).
	2. Communication as part of change management (eg, workshops, comms teams, weekly meetings and teaching sessions).
	3. Ethics and clinical safety.
P5	1. More on governance and regulatory frameworks.
	2. Validation and verification of software.
	3. Audit cycles.
	4. Presenting data visually to patients (ie, infographics).
	5. Communicating with stakeholders and getting management buy in.
P6	1. Agile project management.
P7	1. Postgraduate teaching and supervising.
P8	1. Understanding integration between systems (ie, clinical and social care systems).
	2. Ethical decision-making.
P11	1. Understanding more about others roles.
P12	1. Decision-making should have its own domain.
	2. Action behavioural change.
P14	1. Leadership resilience.
P15	1. Clinical improvement as its own section (research and evaluation of what we are doing).

Three participants made reference to the fact that artificial intelligence (AI) and machine learning only receive a nominal mention:

[…] there’s a very nominal mention of, sort of, AI in there. I think, machine learning could be stronger, in the frameworks, I think, it’s something that people might be expected to start to look at in the clinical informatics, kind of, role. – P5

The target level

As the framework was intended to list the core competencies grouped by domain only, none of the versions of the framework presented were stratified into different levels of competence. Thirteen participants discussed the level or granularity of the competencies. Participants were mostly concerned that although they thought some competencies were ‘core’, they were unsure about them being ‘entry level’ competencies:

I wasn’t sure whether 1.4 [health administration and services] a, b or c, at an entry level needed to be. I think they do need to be as someone progresses as a clinical informatician but they struck me as the sort of things that you wouldn’t say examine a nurse who was newly qualified or even a newly qualified doctor or a biomedical scientist, for that matter. – P1

Three participants specifically identified the ‘leadership and management’ competency as one that was particularly affected by level. One participant pointed out that clinical informaticians may need to cultivate leadership and management skills at an earlier career point, due to the nature and novelty of the role.

[…] some of the informaticians that we’ve worked with have actually said […] that those sort of leadership skills actually are, you know, needed more rapidly than anticipated. Because, they’re thrown into quite, sometimes quite high-level meetings […]…and from that sense they can also have quite a bit of influence, maybe, with some quite big decisions sometimes. – P5

Representation of a ‘core’ competency

Whether or not participants thought a competency was core or not seemed to depend on their role and experience showing the variety of work and experiences of informaticians.

If you look at all of them they’re not for every clinician; some clinicians have more inclination towards using some of them than others. So the current seven [domains], if you look at it in detail, they go way beyond what would be called for all of us. […] – P7

Seven participants agreed that all or most of the competencies presented were ‘core’ and that it was not necessary to be an expert in them all, but rather to have at least an understanding or awareness of them.

It could be unfair for us to expect people to be experts across all of those domains […] so I think the core competencies have to have an understanding that, yes, you should have a basic understanding across all of these domains but we don’t expect you to be an expert in all of them. – P12

Application of the framework and impact on the profession

For this section we applied framework analysis to examine themes surrounding the intended application of the framework, impact on the profession and barriers. Several participants indicated an interest in how the framework might be applied and how people might meet the competencies:

Participant 12: So there needs to be some line that we need to meet and the core competencies should be that, but we just need to be sensitive about how we’re ensuring our members are meeting the competencies.

Interviewer: It sounds like you’re saying a light touch is preferable.

Participant 12: Yeah. We don’t want to beat them with an exam stick, you know. These people are already professionals, so we need to, I don’t know, acknowledge the fact that they are already professionals within their own right and experts in their own right and this is an additional bit and not scare them off by putting them through hours and hours of examinations or whatever.

Another mentioned how this could be used to provide flexible portfolio training routes for clinicians and other healthcare practitioners.

Now what’s interesting is that within that portfolio route there’s no underpinning framework, or nothing anywhere near as detailed as this. And, I think, this would be a really useful competency framework that might underpin flexible portfolio training routes for physicians. Nurses are also starting to come through and people that want to take time out of their regular day job to also diversify into, sort of, informatics type roles – P5

Participants pointed out that some practitioners work more or less in a single competency domain or domains:

I mean some people; their job description will purely be one domain because that’s where they will fit. I was trying to think of a CCIO. There’s probably a CIO in an organisation, would this description apply to someone’s job and I think to be fair, it’s a fairly good description of what you would expect someone to have to have to do that role. – P3

Some barriers were also identified; one from the perspective of those involved in social care work, which seemed to work in two directions:

Participant 2: I think the barriers, certainly from a social care perspective, are the fact that you have to be a registered clinician, and there is no registration process for most people working within social care. […] the vast majority of people that work in social care are professionals without a professional registration.

Interviewer: It also sounded to me, like there was actually a barrier coming from the other direction as well, with people from social care actually doing these roles, but not identifying as such?

Participant 2: Hundred per cent, yeah, 100 per cent. There are people in social care, who, if they were in an NHS trust, would have CCIO, CNIO responsibilities, but they do not have them in social care, because the sector does not recognise that as a role yet. So, it’s a two-way barrier, it’s not a one-way barrier at all.

It was generally felt that the framework should be open to as many professional groups as possible, the caveat being that they should also have a clinical role. It was felt that there was a greater distinction between bioinformatics and the other types of health informatics. Some participants suggested that, in future, pathways to becoming clinical informaticians may change. Currently, those with medical/clinical training acquire informatics skills subsequently, but future pathways may involve information technology (IT) and chief information officers acquiring clinical knowledge:

[…] people who are getting up to be heads of IT, chief information officers as opposed to chief clinical information officers could probably start to pick up the levels of clinical knowledge to be a clinical informatician so long as it’s not about the clinical interpretation… […] – P1

Iteration 2: survey

Following incorporation of interviewees’ feedback into the framework, the amended version was presented for wider evaluation through a digital survey.

Methods

A bespoke digital survey data collection tool (figure 2) was created using Python Flask to allow participants to view the survey side-by-side with the competency framework for easy comparison. The online survey was piloted with the research team and internally reviewed. Most of the questions were based on those asked in the interviews with the addition of rating scales (Likert items) for generating summary data. Participants were identified by the FCI and participants were asked to forward the survey to other relevant professionals.

Figure 2.

Screenshots of the survey data collection tool. The framework can be viewed and scrolled on the right of the screen during the primary survey so that participants can reference the framework easily as they complete the survey.

The survey was hosted by the University’s Research IT team. Survey data were analysed using descriptive statistics (means, percentages), and by using sentiment analysis using the Bing lexicon.⁸ Sentiment analysis is a computational method for assessing opinions in textual data and whether or not the writer expresses a positive, negative or neutral sentiment.

The sample

A total of 87 respondents completed the survey with between 0 and 49 years of experience in informatics (M=12.8, SD=10). The median age of respondents was 50 years (range: 29–73 years). Table 4 shows the breakdown of respondents by informatics discipline. The ‘other’ informatics backgrounds identified can be seen in box 1.

Table 4.

Demographics and background summary showing the number and percentage of survey respondents

Informatics discipline	n (%)
Clinical informatics	60 (62.5)
Public health informatics	12 (12.5)
Other informatics	9 (9.4)
Pharmacy informatics	8 (8.3)
Nursing informatics	7 (7.3)
Gender
Male	57 (65.5)
Female	29 (33.3)
Prefer not to say	1 (1.1)

Box 1. Other informatics backgrounds entered by respondents.

• Research Informatics.

• Imaging Informatics.

• Clinical Coding/Healthcare Evaluation Data (HED)/SNOMED CT (Clinical Terms).

• Engineering.

• Knowledge management.

• Allied Health Professional (AHP) in NHS working on digital innovation.

• Urgent Care Informatics.

• Little experience in informatics.

• Dental Informatics.

Results

We present the results of the survey including the Likert item questions followed by the open-ended questions. The questions asked in the survey can be seen in the online supplemental appendix box 3. Figure 3 shows the responses for self-rated informatics expertise. This was rated on a scale from 1 (novice) to 5 (expert). The percentage of responses can be seen on the y-axis with the actual number of respondents in the bars.

Figure 3.

How would you rate your informatics expertise? (1=novice, 5=expert): Note the number of respondents is displayed on the bars.

A summary of the questions asked for the Likert items can be seen in table 5. Figure 4 shows the results for each of these questions per informatics background.

Table 5.

Likert item questions with score ranges

#	Question	Score 1–5
1	How well do you think these competencies represent those required by professionals to work in the field of Clinical Informatics in the UK?	1=not well, 5=very well
2	These competencies represent core competencies.	1=strongly disagree, 5=strongly agree
3	The framework strikes the right balance between clinical and technical areas of Clinical Informatics.	1=strongly disagree, 5=strongly agree
4	How easy to understand do you think the competencies in the framework are?	1=very difficult, 5=very easy
5	How well do you think the competencies in the framework are presented?	1=very badly, 5=very well
6	How many of these competencies do you currently cover?	1=none of them, 5=all of them

Figure 4.

Likert question responses per question by informatics background.

The remaining questions were open-ended questions, an overview can be seen in table 6.

Table 6.

Free text open-ended questions

#	Question
7	If there are any competencies that you think are missing from the framework, please briefly describe them.
8	If there are any competencies in the framework you think are not 'core' competencies, please indicate which ones by their number(s) followed by a brief reason.
9	Would you suggest any changes to the terminology used in the framework? If so, please briefly describe.
10	Do you have any comments on the overall structure or grouping of the framework competencies?
11	Do you have any comments about any individual competencies in the framework? If so please state the number of the competency and then make your comment.
12	Is there anything else that you would like to mention?

Competencies missing from the framework

Thirteen respondent’s (32.5%) stated that they could not identify any missing competencies. Of those identified as missing, competencies suggested included aspects of diplomacy and negotiation, clinical safety of systems, medical device regulations and knowledge of research governance. ‘Cyber security’ was mentioned by several respondents (n=3).

Odd that there is no mention of cyber security at all, and little on confidentiality and information governance. Burying them under Data security and governance (2.6) is insufficient.

Some element of historical lessons learnt in informatics was also suggested (n=2).

I would add a historical lessons section on previous successful and not successful implementations of health informatics (eg, NPfIT, Choose & Book, eRS, PACS/DICOM, Wachter review, Topol review) which require to learn [from] past success and failures.

One respondent mentioned that an awareness of programming/software development is not sufficient and that practitioners should actually cultivate skills to create custom-built solutions.

If I think about some of the health informaticians/clinical bioinformaticians I have had contact with over the last few years—many of them discuss actually doing software development roles—for example, making Apps/interactive interfaces and also requiring programming skills to do so. I think this is not strong enough in the framework—I don't think it’s sufficient to have an awareness of these things/or just be able to evaluate them I think they actually need the knowledge and skills to build solutions for themselves (particularly if you consider a financially strained NHS where potentially external solutions will not be affordable). Some bio-informaticians have also had to undertake roles such as server migration (in genomics) and implementing cloud solutions—again I don't think these are alluded to anywhere—[cloud] solutions should definitely be mentioned.

Are the competencies 'core' competencies?

Although some participants recommended adding software development skills when asked what competencies might be missing from the framework, others (n=2) suggested this may not be a core competency:

I think there are many current active Clinical Informaticians who struggle to fulfil all of these competencies currently. One good example would be the fields of advanced coding/visualisation/modelling and AI. I think as an initial set of 'core' competencies, only basic information on these may be appropriate. More advance training may allow some parties to specialise in that field. (Analogous with for example, A Basic Surgical Training then specialisation say as an Orthopaedic surgeon, as opposed to an Ophthalmologist).

Another participant pointed out that the requirement should be to have an awareness of—rather than being able to practice—these skills:

While the range is pretty comprehensive, it aligns with competencies in other countries and the requirement is often at the 'awareness' rather than the practitioner level—so I think these are OK.

Terminology used in the framework

Of the 37 participants that answered this question, 18 (49%) accepted the terminology presented and did not request any changes. Of those that did request changes, the majority were minor. The changes suggested were very varied with little to no agreement and focused on different aspects, for example:

I would like to see the concept of clinical assurance and clinical risk management in Domain 4 Also more emphasis on governance—risk ownership, project assurance, project delivery, product ownership and maintenance in Domain 6.

I think all terms are accessible to those working in the field, but might be difficult for those who are thinking of changing into a career in informatics, particularly those with limited experience.

Comments on the overall structure or grouping of the framework competencies

Of the 46 respondents answering this question, 28 (61%) had no comment on the structure or thought the structure was good and did not require any changes. The majority of responses indicated minor changes to the organisation of the competencies with no significant homogeneity of opinion.

I really liked it. I think AHPs [Allied Health Professionals] would find it helpful in understanding their competencies in this field of practice and would support a career pathway for them into the professional cohort. The language is familiar and readable.

Another respondent considered the utility of identifying how one would progress from the essential core competencies through to expert level.

I think it would be useful to identify a progression from 'core' to 'expert'. This would support the identification of a learning programme to support a career path in clinical informatics and support the progression from Associate to Fellow within the Faculty

Comments about individual competencies in the framework

The specific changes to competencies suggested by respondents can be seen in table 7.

Table 7.

Specific changes proposed to competencies (NB: competencies from second iteration of the framework)

Competency	Suggested change
2.1 (a) Demonstrates knowledge of key information technology components including hardware and software, and how they can be used in health and social care settings.	2.1 (a) Should be at the level of ‘Analysing’ rather than ‘Knowledge’ for core competencies.
5.2 (a) Demonstrates understanding of models for effective knowledge acquisition and storage, including strengths and limitations.	5.2 (a) Needs to include something on dissemination.
5.2 (b) Understands the transformation of knowledge (from generation to modelling) into clinical decision tools.	5.2 (b) Do you want to talk about computable knowledge?
4.1 (a) Applies quality improvement and process engineering to facilitate business and clinical transformation, measuring and analysing appropriate outcomes.	4.1.(a) Please emphasise that Clinical Safety covers the deployment and use of health information technology systems, not just their manufacture.
4.3 (a) Understands the organisational and human factor challenges to effective use of health information systems and technologies and can apply appropriate methods to address these and ensure maximum user engagement and widespread adoption.	4.3. (a) One bullet point for behavioural change is not really enough!
3.1 Methodologies and applications.	Domain 1 and competencies therein, need to include a global perspective as a second bullet point. 3.1 needs to mention databases, data access/storage options. The database (or other) also fundamentally affects the ability to create the analyses. Only mentioning R/Python is akin to saying you need a steering wheel to drive without mentioning the car’s engine.
2.2 Working with project stakeholders.	2.2 …I think the inclusion of human factors and user-centred design at this stage might be useful (It is referred to in the change section 4).
2.1 (d) Can identify appropriate technology to resolve healthcare problems and contribute to the development of good practice to do this.	2.1 (d) Can identify appropriate technology to resolve healthcare problems and contribute to the development of good practice to do this—what do we mean by this and particularly resolving healthcare problems?
2.4 (a) Demonstrates knowledge of the range of technology for transmitting information (eg, messaging between systems) and clinical standards (eg, standards for structuring clinical information) for information needed to support the creation of interoperable systems, and promotes their importance to drive an integrated delivery care model.	2.4 (a) ‘Clinical standards’ should read ‘clinical information standards’ as without context clinical standards could mean something different.
3.1 (a) Is able to explain the methodological concepts of basic descriptive statistics, probability, predictive modelling and artificial intelligence (AI) and know when and how to use them to solve health and social care, and clinical practice and research problems. 3.1 (b) Is able to demonstrate how data quality effects analysis, and resulting clinical and healthcare insights, and how important it is to improve to derive maximum potential from its utilisation. 3.1 (c) Understands how technologies (eg, R, Python, Github) facilitate the analysis, display of results and reproducibility of analyses.	3.1 (a–c) I would be very thoughtful about exactly how much you would expect basic, core clinical informaticians to be able to explain about methodological concepts of modelling and AI. Or indeed expect a robust understanding of R, Python, Github.

Another respondent suggested defining broad skills in an ‘assumptions’ section as one could consider skills like project management as not being specific to clinical informatics:

Suggest defining some of the broad skills in an assumptions section that is, ‘it is assumed that each individual has the following competency areas not specific to CI’ for example, project management skills, quality improvement skills, governance skills—and only including those skills specific to CI.

The final CCF

Overall, participants were generally positive about the framework. This is quantified by sentiment analysis of the survey responses using the Bing lexicon¹⁰ which showed a net sentiment score of 60.33% (negative=70, positive=283). The final domains and subdomains included in the framework can be seen in figure 5. The final version of the framework can be viewed on the FCI’s website (https://facultyofclinicalinformatics.org.uk/core-competency-framework).

Figure 5.

Domains and subdomains in the final competency framework.

Competencies are defined hierarchically by complexity using Bloom’s original Taxonomy¹¹ progressing from level 1 to 6 consisting of remembering (knowledge), understanding (comprehension), applying, analysing, synthesising and creating and finally evaluation. Such competencies can be graded against the taxonomy.¹² The CCF has four principle areas: health, data and technologies, leadership and management and people. There are six domains comprising of health and well-being in practice, IT and systems, working with data and analytical models, enabling human and organisational change, decision making and leading informatics teams and project. Each domain contains a number of sub-domains (n=36) which contain the individual competencies (n=111).

The rationale for action on a suggestion to make a change to the framework was based on several factors including the number of people expressing a similar desire for change, whether the detail provided was sufficient to understand the suggested change and the relevance to clinical informatics in general. Following collation of data (systematic review, interviews and survey) the authors determined the final competencies at each stage through internal consensus based on the synthesised data.

Discussion

The use of an iterative approach for competency definition is advocated by Greenhalgh and Macfarlane.¹³ Our approach was similar to that of Jidkov et al which used an iterative method to develop 20 universal health informatics competencies for postgraduate medical education. The approach featured a literature review, content analysis and expert review.¹⁴ As opposed to a purely topic-based approach, competency approaches have been gaining more attention in both nursing and medical education.¹² This is not without some issues however, such as problems related to identifying priorities as well as defining and measuring how competencies are met.¹⁵ Therefore one should aim to strike a balance between covering the most salient areas and keeping the number of competencies to the smallest number required. Another threat to competency models is that of clinician burnout in postgraduate education.¹⁶ As pointed out in the responses, there is a potential additional burden on clinical informaticians as they already have professional membership as clinicians and allied health professionals, with codes of conduct and clinical competencies to maintain. Consideration is required in terms of how clinical informaticians will be required to meet the competencies defined in the framework. Several participants indicated that a light touch is preferable to more intense examinations. It should also be recognised that participants may work in one (or several) domains deeply and not have experience spanning all the domains depending on their role.

The mixed-methods approach chosen appears to be appropriate for generating a core competency framework, as it allowed us to take a systematic and structured approach to co-designing the framework with no major issues. The combination of qualitative and quantitative methods contributed to the robustness of the final output. There already exist specific frameworks for certain informatics subdomains, such as TIGER for nurses¹² and ELIXIR¹⁷ for bio-informaticians. In addition there are international frameworks and recommendations for informatics education such as¹⁸ and.¹⁹ Although there is some overlap between the CCF and competencies identified in such international frameworks, the CCF adopts a UK-specific perspective, taking into consideration the unique characteristics of UK health and social care. Considering this local perspective is crucial to the development of competency frameworks.^{4 12 20} The Topol review specifically identifies the FCI as ideally positioned to recruit, retain and credentialise NHS data science specialists.¹

Participants felt that the framework should be open to as many professional groups as possible. It was also highlighted that practitioners may work in depth in a few domains and may not span all the domains. However, it was considered that at entry level, an awareness of the other domains would be sufficient and that practice-level skills may not be required in all the domains. The participants generally felt that all the competencies were ‘core’ competencies depending on the level that a practitioner was expected to engage with them. There was some contention in terms of the more technical aspects (specifically around domain 3, ‘working with data and analytical methods’) where some felt that informaticians should have more extensive skills (ie, being able to build software), whereas others put forward that clinicians may struggle to meet this requirement. As it stands the framework only stipulates an awareness of such issues rather than practical experience.

The evolving technical requirements of informatics and the overlap with the profession of data science is a key consideration. Douglas Fridsma, the former president and COE of the AMIA, discussed how such similarities may be operationally defined via the development of core competencies.²¹ There may be some overlap in areas of analytical and computational skills in healthcare data science as seen in a recent content analysis of healthcare data scientists job postings²² and informatics. Although data science may involve a deeper dive into analytical methods and processes, informatics spans a wider remit and can be involved with decision-making, communication and implementation of data analysis and health systems. The boundaries between such professions may be blurred at times, but as Fridsma points out, core competencies can help to distinguish professions as well as define how they overlap.

Another point raised was the importance of topics such as AI and the subdomain of machine learning (ML). Both AI and ML are mentioned prominently in the Topol review²³ and are receiving serious consideration and funding for their application to healthcare. AI is not a new concept and has been around since the 1950s.²⁴ Many algorithms are referred to as a black box because the exact workings of certain algorithms (eg, neural networks) can be complex and unpredictable.²⁴ Despite their current popularity, it is unclear whether AI and ML will continue to hold the same prominent position in future. The framework therefore incorporates AI and ML into the broader category of ‘Analytical Methodologies and Applications’ rather than a section in its own right. Many statistical methods are still widely used in medical contexts, and ‘statistical learning’, which is focused on prediction as well as supervised and unsupervised modelling, spans the domains of machine learning and statistics.²⁵

One of the challenges going forward will be the implementation of the framework. This will include tasks such as developing operational definitions for the competencies concerning their related knowledge, skills, attitudes and behaviours as well as defining how such competence will be measured for individuals.¹⁵ The CCF provides the initial foundation for defining the requirements to be a clinical informatician in the UK and provides a starting point for professional development in this fast-moving area of healthcare.

As the interviews highlight, careful consideration is required when linking the framework to developmental stages and/or academic levels. Various stage models of professional development exist^26–28 and mostly originate from the field of cognitive psychology. These focus on moving through various developmental stages (eg, novice, advanced beginner, competent, proficient and expert²⁹). Stage models are often based on attributes including attitudes, skills and underpinning knowledge which are supplemented with additional skills and knowledge from the workplace.³⁰

Criticism of such models suggest that some aspects of professional skill development, such as the skill being developed in terms of understanding and practice, may be concealed as the primary focus is on the developmental stages themselves.³⁰

More recent work considers models that account for multiple dimensions such as progression of skills over time and embodied understanding of practice.³⁰ This reflects the difference between skill progression over time that is possible without such understanding of practice which differentiates experienced and expert practitioners. Other models consider extra dimensions, such as functional and foundational competency domains and stages of professional development.³¹

It can also be difficult to ascertain exactly which skills, knowledge and attitudes differentiate between being an expert or not. Given that there is a connection between the individual and their characteristics and their professional practice,³⁰ applying a staged model can be complex, especially when considering applying this to the various clinical groups that form the clinical informatics community with their underlying cultures and contexts. This may have to be applied in slightly different ways to account for these variations in context.

The FCI have developed a working group that seeks to address such issues, including the mapping of the framework to academic courses, accreditation using the framework and other details around its validation and implementation. Progress of the working group is reported publicly online via the FCI website giving the option for more end users and stakeholders to comment and feedback on the implementation as well as future revisions.

Limitations

Interviews were conducted at the start of the coronavirus pandemic which may have impacted on recruitment levels. It is possible that not all perspectives of all professional groups relevant to clinical informatics were represented. We were however able to engage with a range of participants that represented different roles, professional backgrounds and experience levels. A further limitation is that the work is based in the UK and therefore may not be fully generalisable outside of the UK without adaptions for other contexts. Finally, the survey only received 87 responses, which given the size of the informatics community in the UK would not indicate a large take-up. This was partly due to the project’s limited time frame as well as the start of the COVID-19 pandemic, which meant many potential respondents prioritised pandemic-related work.

Conclusion

We used a mixed-methods iterative approach, using both qualitative and quantitative methods to develop a core competency framework for clinical informatics, consisting of 111 competencies spread over 6 principle domains with 36 subdomains. The domain around working with data and analytical methods seems to be the most contentious, due to variations in the extent to which practitioners are involved with the practical aspects of producing systems and coding. The overlap between fields like healthcare data science and informatics blurs at this point. Competency frameworks can be used to define these overlaps and also differences. Finally, consideration and care is needed regarding the application of such frameworks for professionals that already have a primary clinical competency set to meet to avoid burnout and to ensure all groups can meet these competency requirements, even if their main area of work spans only one or a few of the primary domains. We recommend an annual review process for the framework moving forward to ensure its continuing relevance.

Data availability statement

No data are available. To keep participants anonymous, original interview transcripts/recordings are not publicly available.

Ethics statements

Patient consent for publication

Not required.

Ethics approval

The University of Manchester Research Ethics Committee reference 8985 and 9591.