Summary
Background: While health informatics recommendations on competencies and education serve as highly desirable corridors for designing curricula and courses, they cannot show how the content should be situated in a specific and local context. Therefore, global and local perspectives need to be reconciled in a common framework.
Objectives: The primary aim of this study is therefore to empirically define and validate a framework of globally accepted core competency areas in health informatics and to enrich this framework with exemplar information derived from local educational settings.
Methods: To this end, (i) a survey was deployed and yielded insights from 43 nursing experts from 21 countries worldwide to measure the relevance of the core competency areas, (ii) a workshop at the International Nursing Informatics Conference (NI2016) held in June 2016 to provide information about the validation and clustering of these areas and (iii) exemplar case studies were compiled to match these findings with the practice. The survey was designed based on a comprehensive compilation of competencies from the international literature in medical and health informatics.
Results: The resulting recommendation framework consists of 24 core competency areas in health informatics defined for five major nursing roles. These areas were clustered in the domains “data, information, knowledge”, “information exchange and information sharing”, “ethical and legal issues”, “systems life cycle management”, “management” and “biostatistics and medical technology”, all of which showed high reliability values. The core competency areas were ranked by relevance and validated by a different group of experts. Exemplar case studies from Brazil, Germany, New Zealand, Taiwan/China, United Kingdom (Scotland) and the United States of America expanded on the competencies described in the core competency areas.
Conclusions: This international recommendation framework for competencies in health informatics directed at nurses provides a grid of knowledge for teachers and learner alike that is instantiated with knowledge about informatics competencies, professional roles, priorities and practical, local experience. It also provides a methodology for developing frameworks for other professions/disciplines. Finally, this framework lays the foundation of cross-country learning in health informatics education for nurses and other health professionals.
Keywords: Health informatics competencies, nursing, inter-professional care, education, curriculum, workforce development, recommendations, methodology
1. Introduction
1.1 Health Informatics versus Nursing Informatics?
Identifying informatics competencies in nursing has had a long history and pertinent activities have increased over the last 15 years, with peaks between 2006 and 2009, staying at a high level afterwards [ 1 ]. This continued interest in education correlates with the advancement of information technologies, their implementation to support healthcare processes and the potential to make a difference in patient outcomes, and the health IT knowledge and skills gap in nursing [ 2 ]. Over the years, the focus on competency recommendations has shifted to reflect changing technical and other priorities in healthcare. For example, in 2001, educators distinguished between computer and informatics skills [ 3 ] and focused on technologies such as relational databases [ 4 ]. Although these recommendations have not become obsolete, the two examples highlight the need for continuously monitoring and updating informatics competencies in order to keep pace with on-going technical developments and their use.
As observed in other clinical informatics sub-specialties, informatics competencies in nursing belong to a cohesive field of its own known as “nursing informatics”. However, the notion has grown that there has to be a common foundation with input from a variety of healthcare professions [ 5 ]. In this sense, the term “health informatics” comprises informatics for different disciplines and professions and recognizes inter-professionalism as an intrinsic feature of informatics [ 6 ]. This view matches activities to reshape healthcare from a silo dominated field to a process and patient oriented service, which requires health information technology (IT) to support inter-professional care across the continuum, based on proper process management and inclusive of quality management [ 7 , 8 ].
Informatics , which focuses on data, information, knowledge, the applications and the users themselves, is to be distinguished from Information Systems which focuses on the organizational use of health IT and from Information Technology , which primarily addresses systems development and system life cycle management issues [ 9 ]. Due to its focus on data, information and knowledge, health informatics needs to balance requirements that are specific to professions or roles and those that are generic and applicable across the spectrum of healthcare professionals. “What health informatics competencies are relevant for a special group of health professionals (e.g. nurses) and for different professional roles?” is thus a valid question and needs to be answered for each group separately.
1.2 The Benefit of International Recommendations in Education
As health IT industry is increasingly working in a global arena, international recommendations in medical and health informatics education [ 10 , 11 ] have become more desirable. Other trends also speak in favor of a global perspective. While traditional education is always situated in a local context, new modes of teaching and learning for example, such as Massive Open Online Courses (MOOCs) [ 12 ], are emerging that fulfill a niche for hundreds of thousands of learners from diverging backgrounds, different settings and countries. Recommendations taking a global perspective therefore match the idea of MOOCs. Research and education form a strong alliance at all levels of the translational process to improve health practice. Research, which is defined by internationally valid standards, strongly depends on the collection, analysis and publication of electronic data, also from nursing [ 13 ] and from sites in different countries [ 14 ]. Thus, informatics competencies play an essential role in the education of health and nursing researchers in order to work successfully in an international environment.
Furthermore, international recommendations can foster harmonization of education and may increase workforce mobility across countries [ 15 ]. In this light, the European Union (EU) and the United States of America (USA), signed a Memorandum of Understanding (MoU) focused on health IT in 2010. The MoU agreed to further develop the workforce’s health IT skills to fully utilize the potential of new and emerging technologies. This work took place from 2013–2015 under the direction of the EU-US Workforce Development Workgroup (WDW), which compiled the results of an analysis that mapped competencies in the form of the HITCOMP (Health IT COMPetencies) Tool and Repository [ 15 , 16 ].
1.3 Learning and Teaching: the Competency Based Approach
Competency based approaches have increasingly gained attention in the context of medical and nursing education over the last decades [ 17 , 18 , 19 ] to overcome the shortcomings of the pure topic related approach. In psychology, competencies are understood as “[…] a roughly specialized system of abilities, proficiencies, or skills that are necessary to reach a specific goal.” [ 20 ]. Competencies as such embrace a set of behaviors and intent and cover cognitive, social and emotional aspects of performance [ 21 ]. Due to the close relationship between “demands, tasks, problems, and goals” [ 20 ] and respective competencies, competency based learning is always associated with a specific role, scenario or task domain. For example, there are specific competencies for the nurse leader role [ 22 ] or for inter-professional cooperation [ 23 ]. In this context, we will refer primarily to cognitive competencies, which encompass the acquisition and understanding of new knowledge through iterative mental processes of sensing, experiencing, thinking and reasoning/interpreting [ 21 ]. We will focus particularly on those cognitive competencies that contribute to the successful accomplishment of nursing roles within an organization. In accordance with management theory [ 24 ], which refers to entire organizations, we call them “core competencies” of the professional role.
There are various frameworks that breakdown and describe competencies. According to some frameworks, competencies consist of both knowledge and skills [e.g. 25, 26]. These frameworks either refer to different (academic) levels [ 10 , 27 ] or at a specific level, e.g. graduate programs [ 25 , 28 ]. Competencies can be graded along Bloom’s modified six cognitive process dimension terms (i.e. remember, understand, apply, analyze, evaluate and created) referring to “factual knowledge”, “conceptual knowledge”, “procedural knowledge” and “metacognitive knowledge” [ 29 ] or as “prerequisite knowledge and skills”, “fundamental knowledge” and “procedural knowledge and skills” [ 25 ].
Although health informatics knowledge is global and requires international recommendations, education is typically local and the competencies addressed in the study programs and courses have to be embedded into the environment where they will be used. These two controversial requirements have to be reconciled in order to assist teachers and learners effectively in their efforts. The primary aim of this study is to empirically define and validate a framework of globally accepted core competencies in informatics and to enrich this framework with exemplar information derived from local educational settings. The methodology for developing a framework of this type should be applicable for any kind of target group. Within this study, the framework was developed for nurses with a focus on the professional roles they typically hold. The study was performed within the TIGER ( Technology Informatics Guiding Education Reform ) Initiative that includes an international network of nursing experts and reaches out to the entire community of healthcare professionals.
2. Methods
2.1 The TIGER Approach
In response to the initial formation of the Office of the National Coordinator for Health Information Technology (ONC) in the United States of America in 2004, the TIGER Initiative was officially launched in 2006 when key nursing stakeholders convened to ensure nursing was included in the mandate for health informatics education and application.
TIGER is dedicated to enabling clinicians, particularly nurses, to better exploit the use of informatics tools, their principles and underlying theories to improve patient care [ 30 ]. In 2012, the TIGER International Committee was established to widen the reach of TIGER activities in the international scientific community. When TIGER transitioned from a standalone initiative to the Healthcare Information and Management Systems Society (HIMSS) in 2014, the scope was extended to include other clinical disciplines and to enter into an inter-professional space even beyond clinicians. Today, TIGER seeks to empower educators as change agents and learners to prepare the inter-professional, inter-disciplinary clinical workforce for the digital age [ 31 ]. Based on these goals, TIGER initiated the international informatics competency synthesis project, which seeks defining and validating a recommendation framework.
2.2 Overview of Methodological Steps
In order to achieve this aim, a mixed method approach was pursued that combined quantitative and qualitative methods and iteratively validated the core competency areas while enriching the framework.
Figure 1 provides an overview of the three steps used to launch and finalize this project inclusive of the methods applied and the perspective from which the information was captured.
Figure 1.
Methodological approach.
2.3 Survey
In order to identify globally relevant core competency areas to be represented in the framework, a survey was conducted. This survey made use of an existing online questionnaire [ 32 ] that had been designed drawing upon international literature, that had been further refined in two workshops and tested in three countries [ 32 ]. The questionnaire was comprised of 24 core competencies areas in health informatics, which had been extracted from the international literature [ 10 , 11 , 18 , 25 , 33 , 34 , 35 ] and embraced 10 technological items (e.g. eHealth, telematics and telehealth), two legal and ethical items (e.g. data protection, ethics and IT), 11 IT related management items (e.g. project management, change management) and finally, biostatistics/statistics. Five professional roles of nurses were targeted in accordance with the methodology established prior to this study [ 32 ]: 1) clinical nursing (e.g. care planning), 2) quality management (e.g. organizational development), 3) coordination of inter-professional care (e.g. case management), 4) nursing management (e.g. ward or hospital management) and 5) IT management in nursing (e.g. introduction of new IT systems). A role-based approach was chosen because role specific training for different groups of health professionals had been identified as a highly desirable action in order to increase the health IT skills of the healthcare workforce [ 36 ]. The meaning of these roles was explained by examples (see ►Online Appendix A). Survey participants were asked to rate the relevance of the 24 core competency areas within the five roles on a scale from 0 to 100. In order to help survey participants to get a more detailed understanding of what these core competency areas meant, examples subsumed under these core competency listings were given (►Online Appendix A). We did not distinguish between the different cognitive process dimensions “factual knowledge”, “conceptual knowledge”, “procedural knowledge” and “metacognitive knowledge” or any other gradating categories because this specification would have required assumptions on the (academic) level of the learning and teaching activities. It is important to note that these level differences were out of the scope of this survey. The core competency areas were presented in alphabetical order.
A personalized link to the questionnaire was sent to 72 people from 24 countries in the Americas (6 countries), Europe (10 countries), Asia (6 countries), and Australia/Pacific (2 countries). Participants were either a member of or associated with the International TIGER Committee. As TIGER comprises primarily experts with an interest in education, the majority (46%) came from universities, 26% came from health IT industry, 14% from hospitals (including university hospitals), 13% from professional associations and 1% did not give any information about their affiliation. All survey participants are regarded as high profile experts in their countries. The survey was open from November 23, 2015 to January 18, 2016. Out of the 72 experts invited, we yielded 43 responders (response rate approximately 60%). They came from 21 countries: Americas (4 countries), Europe (10 countries), Asia (5 countries) and Australia/Pacific (2 countries) and had the following affiliation: 37% from universities, 21% from health IT industry, 35% from hospitals (including university hospitals) and 7% from professional associations.
2.4 Workshop
In order to validate the survey findings within a global expert community, a workshop was held at the 13 th International Congress on Nursing Informatics (NI2016) in Geneva on June 28, 2016 [ 37 ]. The aim was to find out if core competency areas were missing and how to cluster them. Twenty-eight experts from 13 countries on four continents attended the workshop (six of the experts had participated in the survey). Workshop attendees reported the following types of affiliation: 39% universities, 25% hospitals (including university hospitals), 18% health IT industry, 11% professional associations and 7% gave no information. Following an introduction to the aims and visions of the TIGER Initiative, the results of the international survey on core informatics competency areas were presented and made available to the audience as a handout. The workshop then focused on discussing informatics competency issues with workshop attendees. The discussion was stimulated with the following questions: (1) “What are the top 5 informatics competencies for nurses according to your experience?” and (2) “What is your opinion about the core competency areas just presented?”. The discussion was recorded and all contributions were summarized utilizing the inductive categorization method. This work included five persons from the team of authors (UH, TS, MB, HM, PC) who additionally clustered the individual competencies making use of the greater domains also mentioned by the experts at the workshop. These domains were mapped to the 24 core competency areas by the same team members plus one other author (OS). Cronbach’s alpha was computed to test the internal consistency of these domains across the five nursing roles. An alpha value of 0.70 and above was interpreted as “satisfactory” in accordance with the literature [ 38 ].
2.5 Case Studies
Local exemplar case studies, which were compiled for this study, should illustrate the use of the core competency areas selected to represent leading institutions of education in the field of nursing and health informatics. One case study per country was chosen covering Brazil (BR), Germany (GER), New Zealand (NZ), Taiwan/China (TW-CHN), the United Kingdom (UK) and the United States of America (USA). These countries were selected to mirror a variety of different (professional) cultures. While conducting this study, no case study from an African country was available to be included into this project. These case studies should provide background information on educational institutions, varying types of students, the academic level, teaching strategy and the educational format. They also should describe if and in which way informatics core competency areas were addressed by these courses and programs. Thus, they were meant to serve as exemplars of teaching and learning individual competencies included in the core competency areas and potentially beyond.
2.6 Ethical Considerations
The methodology of this study was presented to the Ethical Committee for Research at Hochschule Osnabrück Germany, which decided that there was not need to submit a full proposal to the Committee.
3. Results
3.1 Overview
The three-step methodology resulted in a recommendation framework that consists of globally accepted core competency areas:
sorted by relevance within five different professional roles in nursing based on the empirical findings of the survey,
clustered by coherent overarching domains identified in the workshop and verified by a reliability analysis and
illustrated and validated by educational programs and courses in various countries by case studies.
The framework provides recommendations for high priority core competency areas to be addressed when developing curricula for the five professional roles in nursing. It also gives advice regarding the cohesion of core competency areas. Finally, it allows insight into how these areas are broken down into individual competencies in different local settings and for different purposes. This approach resulted in very concrete recommendations but also demonstrates the feasibility of the methodological approach. Each study step is described in-depth in the following sections.
3.2 Global Relevance of Core Competency Areas for Professional Roles in Nursing
► Table 1 shows the priority list and outlines the core competencies areas per professional role. Each role was characterized by one to three lead core competency areas with (nearly) the same mean percentage of relevance (in italics ► Table 1 ) and by a specific profile of core competency areas. In this sense, the leading core competency area for clinical nursing and nursing management was nursing documentation . For the quality management role, it was quality management and for coordination of inter-professional care, it was data protection and security and information management and knowledge management in patient care . For IT management in nursing, there were three (nearly) equally important core competency areas: information systems and communication systems , principles of nursing informatics and data protection and security . There was a mixture of genuine IT competency areas (e.g. information systems and communication systems), IT related management areas (e.g. strategic management and leadership), and legal and ethical issues. Competencies in the areas of nursing documentation and information management and knowledge management in patient care could be found among the top 10 in all five roles. The same held true for competencies in the area of principles of nursing informatics . Data protection and security , process management as well as quality management were rated as highly relevant in four out of the five roles among the top 10 areas of competencies.
Table 1.
Top 10 core competency areas in the five roles and related mean relevance (REL) (0…100).
| Clinical Nursing (Direct Patient Care) | ||
|---|---|---|
| Core competency area | REL ± SD, n = 41 | |
| 1 | Nursing documentation (including terminologies) | 94.4 ± 16.7 |
| 2 | Information and knowledge management | 82.2 ± 23.5 |
| 3 | Principles of nursing informatics | 80.5 ± 23.1 |
| 4 | Data protection and security | 80.0 ± 23.2 |
| 5 | Ethics and IT | 79.5 ± 21.6 |
| 6 | Information and communication systems (including interoperability) | 75.1 ± 24.4 |
| 7 | Quality management | 72.0 ± 22.3 |
| 8 | Decision support by IT | 70.2 ± 28.5 |
| 9 | eHealth, telematics and telehealth (including interoperability) | 69.5 ± 25.0 |
| 10 | Assistive technology for ageing people | 69.0 ± 25.5 |
| Quality Management | ||
| Core competency area | REL ± SD, n = 41 | |
| 1 | Quality management | 96.1 ± 13.2 |
| 2 | Process management | 86.8 ± 17.4 |
| 3 | Nursing documentation (including terminologies) | 84.4 ± 22.5 |
| 4 | Information and knowledge management | 83.2 ± 20.3 |
| 5 | Information and communication systems (including interoperability) | 82.0 ± 21.0 |
| 6 | Principles of nursing informatics | 80.2 ± 22.0 |
| 7 | Data protection and security | 79.5 ± 23.3 |
| 8 | Project management | 78.5 ± 21.0 |
| 9 | Principles of management | 78.5 ± 20.8 |
| 10 | Change management and stakeholder management | 77.6 ± 25.5 |
| Coordination of Inter-professional Care | ||
| Core competency area | REL ± SD, n = 41 | |
| 1 | Data protection and security | 85.9 ± 20.2 |
| 2 | Information and knowledge management | 85.4 ± 20.1 |
| 3 | Nursing documentation (including terminologies) | 83.4 ± 21.4 |
| 4 | Process management | 83.2 ± 20.8 |
| 5 | Information and communication systems (including interoperability) | 81.5 ± 23.0 |
| 6 | Ethics and IT | 78.8 ± 23.7 |
| 7 | eHealth, telematics and telehealth (including interoperability) | 77.6 ± 22.8 |
| 8 | Quality management | 77.1 ± 22.6 |
| 9 | Principles of nursing informatics | 74.6 ± 23.4 |
| 10 | Principles of management | 74.6 ± 23.5 |
| Nursing Management | ||
| Core competency area | REL ± SD, n = 43 | |
| 1 | Nursing documentation (including terminologies) | 92.1 ± 13.9 |
| 2 | Principles of management | 87.9 ± 18.6 |
| 3 | Strategic management and leadership | 86.7 ± 19.9 |
| 4 | Quality management | 85.1 ± 20.3 |
| 5 | Human resource management | 84.4 ± 18.8 |
| 6 | Change management and stakeholder management | 84.2 ± 19.8 |
| 7 | Information and knowledge management | 84.0 ± 22.1 |
| 8 | Principles of nursing informatics | 82.3 ± 20.1 |
| 9 | Process management | 81.2 ± 20.4 |
| 10 | Ethics and IT | 80.5 ± 26.0 |
| IT Management in Nursing | ||
| Core competency area | REL ± SD, n = 41 | |
| 1 | Information and communication systems (including interoperability) | 89.5 ± 15.3 |
| 2 | Principles of nursing informatics | 89.5 ± 19.2 |
| 3 | Data protection and security | 89.0 ± 17.3 |
| 4 | IT risk management | 86.8 ± 19.3 |
| 5 | Project management | 86.8 ± 17.8 |
| 6 | Process management | 86.1 ± 16.2 |
| 7 | Information and knowledge management | 86.1 ± 22.7 |
| 8 | Decision support by IT | 85.4 ± 19.8 |
| 9 | Applied computer science/informatics | 83.4 ± 19.7 |
| 10 | Nursing documentation (including terminologies) | 83.4 ± 22.2 |
Other core competency areas were rather distinctive, such as strategic management and leadership , which appeared among the top 10 only in nursing management, and assistive technology for ageing people , which belonged to the top 10 only in clinical nursing.
Besides these more role specific considerations, the survey results also revealed the importance and relevance of many health informatics core competency areas. Mean relevance values of core competency areas ranking at position 10 still ranged between 69.0 and 83.4 across all roles ( Table 1 ).
►Online Appendix B provides an overview of the ratings for all 24 core competency areas within the five roles.
3.3 Validation and Clustering of the Core Competency Areas
When asked about the highest priority of core competencies, workshop attendees mentioned a variety of different items and competencies including greater core competency areas. These different statements were analyzed with the aim of identifying similarities and differences with regard to the 24 core competency areas (validation) and with the aim of clustering these areas. Based on the consensus of five experts from the team of authors, four domains emerged from these statements. ► Table 2 (left column) shows the domains, the corresponding items and competencies resulting from summarizing the workshop findings and finally the associated core competency areas from the survey (right column). All statements could be matched with core competency areas.
Table 2. Four domains of competency areas mentioned by the workshop attendees (left) and corresponding core competency areas from the survey (right).
| Domains of competency areas | Associated core competency areas (main similarities) | |
|---|---|---|
| Data, information and knowledge (DIK) domain | 1) know how to use data/information not only how to enter data |
|
| 2) perform care planning and use of data |
|
|
| 3) make use of indicators (information) for decision making |
|
|
| 4) analyze what data are needed and are useful, link to data/information science |
|
|
| 5) nurses as knowledge workers: access and use evidence based & structured information |
|
|
| 6) use data for research and development |
|
|
| 7) information governance |
|
|
| Information exchange and information sharing (IEIS) domain | 1) continuity of care |
|
| 2) sharing of information with the patient, work in partnership, learn to listen |
|
|
| 3) provide information map of caring for the citizens |
|
|
| 4) health information exchange |
|
|
| 5) interoperability |
|
|
| Ethics and legal issues (EL) domain | 1) ethics |
|
| 2) security and privacy |
|
|
| 3) use of social media and ethical use of data |
|
|
| Systems life cycle management (SLCM) domain | 1) address requirements |
|
| 2) communicate with engineers |
|
|
| 3) design thinking |
|
|
| 4) process design |
|
|
The four domains derived from the expert statements were used as an initial structure to cluster the 24 core competency areas. As there were some areas that could not be assigned to one of the four domains, two more clusters were introduced by the five experts from the team of authors, which then resulted in six final domains. Two core competency areas were assigned to two domains, i.e. assistive technology for ageing people and information and communication systems. Table 3 shows the clustering and the internal consistency values of these domains computed by Cronbach’s alpha.
Table 3. Cronbach’s alpha for the six final domains in each professional role with the related core competency areas within the domains.
| Domains | Clinical nursing (n = 41) | Quality management (n = 41) | Coordination of inter-professional care (n = 41) | Nursing management (n = 43) | IT management in nursing (n = 41) |
|---|---|---|---|---|---|
| Data, information and knowledge (DIK) | 0.87 | 0.89 | 0.90 | 0.88 | 0.87 |
|
|||||
| Information exchange and information sharing (IEIS) | 0.78 | 0.79 | 0.76 | 0.87 | 0.76 |
|
|||||
| Ethics and legal issues (EL) | 0.87 | 0.85 | 0.84 | 0.67 | 0.76 |
|
|||||
| Systems life cycle management (SLCM) | 0.84 | 0.78 | 0.91 | 0.84 | 0.91 |
|
|||||
| Management in informatics (MAN) | 0.94 | 0.87 | 0.96 | 0.90 | 0.93 |
|
|||||
| Biostatistics and medical technology (STAT&TECH) | 0.77 | 0.81 | 0.77 | 0.90 | 0.87 |
|
All Cronbach’s alpha values were 0.70 or larger (“satisfactory”) [ 37 ] with the only exception of ethics and legal issues (EL) for nursing management, which reached a value of 0.67. In summary, the expert statements from the workshop attendees could be mapped to the 24 core competency areas of the survey (► Table 2 ) and provided a consistent structure for clustering the core competency areas (► Table 3 ).
3.4 Illustration of Core Competency Areas by Cases Studies and the Final Recommendation Framework
The selected exemplar case studies covered educational activities for nurses in six countries, including two case studies describing courses at the undergraduate level (BR, UK-SCO), two at the postgraduate level (NZ, USA) and two at the certificate level in continuing education (GER, TW-CHN). There were exemplar case studies that embraced single courses (BR, GER, NZ, TW-CHN, USA) and one that described a sequence of learning units embedded in a nursing program (UK-SCO). Similarly, four of the case studies focused on nurses (BR, GER, TW-CHN, UK-SCO), one primarily on nurses but also on informatics students (USA) and the other was entirely inter-professional (NZ). In all six cases, the courses were or are designed drawing on national recommendations [e.g. 32, 39, 40, 41] or developments (BR, GER, NZ, TW-CHN, UK-SCO, USA. The case studies are described in full length in ►Online Appendix C. Competencies tied to the core competency area in each exemplar case study are represented as building blocks. These building blocks contain competency descriptions in each area (► Figure 2 ).
Figure 2.
Building blocks with competency examples from the respective exemplar case studies and their relation to a core competency area.
Table 4 constitutes the recommendation framework consisting of the core competency areas, their mean relevance values, the associated domains and the case study building blocks referring to the core competency areas. It hereby shows the distribution of competency building blocks from individual exemplar case studies. It mirrors the breadth and depth of the courses described and the competencies associated.
Table 4.
Recommendation framework of health informatics for nurses. Legend: DIK = data, information, knowledge; EL = ethical and legal issues; IEIS = information exchange and information sharing; SLCM = system life cycle management; MAN = management in informatics; STAT&TECH = biostatistics and medical technology; REL = mean relevance from 0 … 100.
| Roles | Clinical nursing (direct patient care) | Quality management | Coordination of inter-professional care | Nursing management | IT management in nursing | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Core competency area | Domains | REL n = 41 | Covered by case studies from | REL n = 41 | Covered by case studies from | REL n = 41 | Covered by case studies from | REL n = 43 | Covered by case studies from | REL n = 41 | Covered by case studies from |
| Principles of nursing informatics | DIK | 80.5 |
|
80.2 |
|
74.6 |
|
82.3 |
|
89.5 |
|
| Information and knowledge management in patient care | DIK | 82.2 |
|
83.2 | 85.4 |
|
84.0 |
|
86.1 |
|
|
| Nursing documentation (including terminologies) | DIK | 80.5 |
|
84.4 |
|
83.4 |
|
92.1 |
|
83.4 | |
| Decision support by IT | DIK | 70.2 |
|
72.7 | 70.0 |
|
74.7 |
|
85.4 |
|
|
| Information management in research | DIK | 51.0 |
|
72.4 |
|
60.5 | 63.3 |
|
71.5 |
|
|
| Information management in teaching, training and education | DIK | 61.7 | 67.1 | 66.1 | 70.0 | 74.4 | |||||
| Resource planning and logistics | DIK | 56.6 |
|
65.4 | 71.7 | 76.0 | 71.7 | ||||
| Data protection and security | EL | 80.0 |
|
79.5 |
|
85.9 |
|
80.2 |
|
89.0 |
|
| Ethics and IT | EL | 79.5 |
|
75.9 |
|
78.8 |
|
80.5 |
|
83.4 |
|
| eHealth. telematics and telehealth | IEIS | 69.5 |
|
69.8 |
|
77.6 | 66.3 |
|
80.0 |
|
|
| Assistive technology for ageing people | IEIS/ STAT&TECH | 69.0 |
|
54.9 |
|
70.2 |
|
63.3 |
|
70.2 |
|
| Information and communication systems | IEIS/SLCM | 75.1 |
|
82.0 |
|
81.5 |
|
75.1 |
|
89.5 |
|
| Applied computer science/informatics | SLCM | 53.7 |
|
63.7 |
|
64.9 |
|
57.4 |
|
83.4 |
|
The case studies from Brazil and UK-Scotland were similar with regard to addressing undergraduate students in their role as clinical nurses, i.e. direct patient care. As these were beginner courses, they focused primarily on the informatics essentials of healthcare professionals. They differed with regard to the number of teaching hours. The Brazilian course, which was a single course, included 30 hours, whereas the Scottish course covers a set of learning units that is integrated into pre-registration nurse training. These competencies matched with the ones for clinical nursing in the top 10 priority list ( Table 1 ) closely.
The US course addressed core competency areas for graduate nurses working in different roles. As a self-paced online course, no specific information on its duration was given. It emphasized competencies associated with the selection, use and evaluation of different types of systems and the exploitation of data, information and knowledge for patient care and research. The New Zealand course covered a large range of informatics competencies for various roles, which corresponded with its design as an inter-professional series of four or eight courses towards a certificate or diploma in health informatics respectively (with 150 teaching and learning hours per course). The German continuing education course was highly specialized in nursing management and analytics while being restricted in its duration (30 hours). It embraced all core competency areas of the top 10 priority list (► Table 1 ) for the role of nursing management and included statistics. It also assumed that certain competencies were already available. The Taiwan-China course was an example how developments in one country (Taiwan) were transferred to another country (China). It spanned a large range of core competency areas that were manifested in a 7-day hands-on workshop focused on learning advanced spreadsheet management and programming.
These exemplar case studies also revealed areas only partly covered by the 24 core competency areas. The case study from the USA emphasized competencies that aim to enable health professionals to teach patient/citizen digital health literacy. This approach is closely associated with consumer health informatics. Furthermore, the Scottish case study considered health service literacy, which encompasses knowledge about the national healthcare system, its professionals, roles and processes, as informatics is embedded within this wider context which is critical to understand.
4. Discussion
4.1 Recommendation Framework
We propose a framework consisting of 24 core competency areas in health informatics and five major roles of nurses. The core competency areas were ranked by international experts in a survey, validated by a different group of experts in an international workshop and clustered into six domains. Four domains are equivalent to the domains derived from the workshop expert discussion, i.e. data, information, knowledge; ethical and legal issues; information exchange and information sharing and system life cycle management. The fifth domain summarizes the core competency area that focuses on management activities in the context of informatics. Finally, the sixth domain subsumes biomedical imaging, assistive technology for ageing people and signal processing and biostatistics/statistics . Nearly all domains have at least “satisfactory” or better values for internal consistency across all roles and can be therefore regarded as reliable. The constituents of these domains, the core competency areas, had been extracted from a comprehensive analysis of the international literature and had been validated in two workshops [ 32 ] prior to this study. In this study, the core competency areas were again validated in a workshop and also via the cases studies. From this multi-layered process, it can be concluded that these areas are valid.
The cells of the matrix framework contain the relevance ratings of international experts. From a global perspective, we recommend areas of core competencies with high relevance ratings in their respective role ( Table 1 ), the globally accepted core competency areas in health informatics for nurses. The interpretation of what is a “high” or a “low” relevance is not given by this framework intentionally to leave the meaning of these values open to the user. The resulting globally accepted areas are meant to function as a compass to show the direction towards areas of high relevance and/or towards domains of high internal consistency. They should help learners and teachers find their route through the large realm of information technology and its application in healthcare. These recommendations are complementary to the TIGER recommendations [ 35 ] focused on basic IT skills.
From a specific perspective, we recommend learning from the examples provided within the different exemplar case studies. They are detailed enough with regard to (academic) levels, cognitive process dimensions and competencies at the layer of cognitive activities [ 29 ]. They are also situated in a specific context defined by the national healthcare system, the educational system of nurses and other clinicians, status of health IT adoption and culture. The case studies, which refer to the core competency areas, demonstrate how these competencies are combined to form an entire curriculum or program. We recommend that readers construct their personalized set of objectives for learning and teaching from the core competency areas and case studies. The framework is instantiated by the case studies, which indicate the core competencies that are primarily addressed.
The exemplar case studies illustrate the need for diversity in learning and teaching. This may lead to the inclusion of some competencies outside the globally accepted informatics core competency areas, e.g. statistics in the German case study. Due to the fact that the globally accepted areas consist of many core competencies, which cannot be considered in one single course, a selection has to be made that reflects local or cultural differences. Also, the teaching and learning styles may differ. The Taiwan-Chinese course allows the nurses to dive deeply into technology; the USA course incorporates a clinical case, which acts as a red thread to exemplify the use of informatics; the Scottish program accompanies the baccalaureate nursing program in the sense of a spiral learning approach; the New Zealand course pursues a fully fledged inter-professional approach; and finally, the Brazilian course is strongly tied to the establishment of nursing education as such.
The mixed methods approach combined a survey with case studies to warrant the integration of different perspectives to build the recommendation framework.
4.2 Limitations and Strengths
We did not include the roles of nursing teachers and scientists because both roles are very much dominated by the roles they refer to when teaching nurses and clinicians and when conducting research. Thus, a nursing teacher working with students in quality management should dispose of similar competencies as quality managers themselves. Likewise, nursing scientists conducting research in clinical nursing should be familiar with competencies of nurses working at the point of care. Furthermore, information management in research is a core competency area of its own that directly addresses the needs of researchers. The role-based approach as such was neither questioned in the survey nor at the workshop. Clinical nursing [ 14 , 42 , 43 ], quality management [ 44 ], care coordination [ 45 ], nursing management, i.e. leadership [ 46 ] and IT management in nursing as informatics nurse specialist [ 47 ] or as chief nursing information officer [ 48 ] are roles with a high informatics affinity that are well accepted by the current literature and are generic enough to cover inpatient and outpatient settings. There are certainly more roles nurses can take on, which are not explicitly considered in this framework, for which the framework, however, could offer a rough orientation.
Although we regard communication as a highly desirable competence, particularly in the context of change management and inter-professional care, we argue that communication aligns better with social and horizontal competencies and is therefore out of the scope of the framework at this stage. Moreover, it also appears as a sub-competence in many core competency areas, particularly those that focus on innovation and change.
Our findings are corroborated by other studies. Competencies of nurse leaders [ 21 ] match – despite a different clustering – to a large extent with the core competency areas for nursing managers within our study. The same applies to other global compilations of competencies in nursing informatics [ 49 ] and clinical informatics [ 50 ].
This study does not provide a picture of competencies that distinguish health informatics in nursing from health informatics for other healthcare professionals because this study did not compile information about these other healthcare professions. Due to the role based approach it could be hypothesized that roles better define the need for a certain competency pattern than professions, e.g. the role “coordination of inter-professional care”. However, a profession is constituted by its body of knowledge and methods thus the details within the domain “data, information and knowledge” should vary to some extent for different professions despite a potential similarity at the formal level.
It is very likely that new competency areas will emerge as the framework is constantly instantiated with more exemplar case studies (inclusive of African countries) and is used in real life. When the framework needs updating, formalized methodologies for the following tasks can assist with this process: a) identifying and updating core competency areas, b) measuring their relevance throughout the international community and c) adding international exemplar case studies that illustrate these core competency areas and provide insight into the competencies and local curricula.
4.3 Outlook
The procedure of conducting a survey, obtaining feedback in a workshop and including exemplar case studies provided knowledge about the validity of core competency areas and about new competencies to be included. In particular, we propose adding consumer health informatics as a core competency area due to the high importance of patient-centered healthcare and the requirement that healthcare professionals share data, information and knowledge with patients and citizens [ 51 ]. This addition would bundle competencies that are already included in the framework such as referred to by “telehealth” and “assistive technologies for ageing people” and would highlight this area. As not all core competency areas are yet covered by case studies, there is a need to integrate additional ones in the future.
5. Conclusions
This international recommendation framework for core competency areas in health informatics for nurses aims at providing a grid to embrace knowledge about competencies, professional roles, priorities and practical experience. The framework refers to the term health informatics in nursing to demonstrate its rooting in nursing informatics [ 45 , 52 ] and its openness towards other healthcare professionals and their interaction with nurses. We contend that learning and teaching on the individual level are active processes of constructing the educational space. Therefore, our recommendations should work as a framework that guides and stimulates learners and teachers alike. It should leave enough room for individual creativity, aspiration for innovation and personal fulfillment. Due to technology being a moving target, this recommendation framework should be revised and updated regularly. We propose a five-year period of validity.
Acknowledgment
We wish to thank all international experts who contributed to the survey and the workshop. We also thank Beth Elias, PhD, Term Associate Professor at Virginia Commonwealth University, Richmond, VA, USA, for her assistance in preparing this manuscript.
A preliminary version of this paper was presented at the multi-conference HEC2016 in August 2016 in Munich/Germany.
Funding Statement
Funding : This work was supported by the European Commission within Horizon 2020 grant number (Grant Agreement 727552 EUUSEHEALTHWORK), HIMSS North America, the HIMSS Foundation and the Federal Ministry of Education and Research (bmbf) Germany grant number (16OH21026).
Footnotes
Supplementary material published on our website https://doi.org/10.3414/ME17-01-0155
Online Supplementary Material (PDF)
Online Zusatzmaterial (PDF)
Online Supplementary Material (PDF)
Online Zusatzmaterial (PDF)
Online Supplementary Material (PDF)
Online Zusatzmaterial (PDF)
References
- 1.Kokol P, Blazun H, Vosner J, Saranto K. Nursing informatics competencies: bibliometric analysis. Stud Health Technol Inf. 2014;201:342–348. [PubMed] [Google Scholar]
- 2.Brenner SK, Kaushal R, Grinspan Z, Joyce C, Kim I, Allard RJ et al. Effects of health information technology on patient outcomes: a systematic review. J Am Med Inform Assoc. 2016;23(05):1016–1036. doi: 10.1093/jamia/ocv138. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Staggers N, Gassert CA, Curran C. Informatics competencies for nurses at four levels of practice. J Nurs Educ. 2001;40(07):303–316. doi: 10.3928/0148-4834-20011001-05. [DOI] [PubMed] [Google Scholar]
- 4.Hoffmann S, Ash J.A survey of academic and industry professionals regarding the preferred skillset of graduates of medical informatics programs Stud Health Technol Inform 200184(Pt 2):1028–1032. [PubMed] [Google Scholar]
- 5.Hersh W. Who are the informaticians? What we know and should know. J Am Med Inform Assoc. 2006;13(02):166–170. doi: 10.1197/jamia.M1912. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Gadd CS, Williamson JJ, Steen EB, Fridsma DB. (IMIA) on Education in Biomedical and Health Informatics. Methods Inf Med. 2010;49:105–120. doi: 10.3414/ME5119. [DOI] [PubMed] [Google Scholar]
- 7.Reiss-Brennan B, Brunisholz KD, Dredge C, Briot P, Grazier K, Wilcox A et al. Association of Integrated Team-Based Care With Health Care Quality, Utilization, and Cost. JAMA. 2016;316(08):826–834. doi: 10.1001/jama.2016.11232. [DOI] [PubMed] [Google Scholar]
- 8.Hastings SE, Suter E, Bloom J, Sharma K. Introduction of a team-based care model in a general medical unit. BMC Health Serv Res. 2016;16:245. doi: 10.1186/s12913-016-1507-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Commission on Accreditation for Health Informatics and Information Management Education – CAHIIM. Curricular Requirements. 2014 Health Informatics Master’s Degree Curriculum Requirements [cited 2018 Mar 13]Available from:http://www.cahiim.org/hi/curriculumrequire ments.html
- 10.Mantas J, Ammenwerth E, Demiris G, Hasman A, Haux R, Hersh W et al. Recommendations of the International Medical Informatics Association (IMIA) on Education in Biomedical and Health Informatics. Methods Inf Med. 2010;49:105–120. doi: 10.3414/ME5119. [DOI] [PubMed] [Google Scholar]
- 11.Global Health Workforce Council (GHWC) Global Academic Curricula Competencies for Health Information Professionals Chicago, IL: The AHIMA Foundation 2015 [cited 2018 Mar 13]Available from:http://www.ahima.org/about/global?tabid=council [Google Scholar]
- 12.Liyanagunawardena TR, Williams SA. Massive open online courses on health and medicine: review. J Med Internet Res. 2014;16(08):e191.. doi: 10.2196/jmir.3439. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Westra BL, Latimer GE, Matney SA, Park JI, Sensmeier J, Simpson RL et al. A national action plan for sharable and comparable nursing data to support practice and translational research for transforming health care. J Am Med Inform Assoc. 2015;22(03):600–607. doi: 10.1093/jamia/ocu011. [DOI] [PubMed] [Google Scholar]
- 14.Weaver CA, Warren JJ, Delaney C.Bedside, classroom and bench: collaborative strategies to generate evidence-based knowledge for nursing practice Int J Med Inform 200574(11–12):989–999. [DOI] [PubMed] [Google Scholar]
- 15.Värri A, Blake R, Roberts J, Fenton SH, Cleary M, Zacks SFet al. Transatlantic collection of health informatics competencies FinJeHeW 201608(2–3):127–136. [Google Scholar]
- 16.HITCOMP – Health IT Competencies [cited 2018 Mar 13]Available from:http://hitcomp.org/
- 17.Fischer MR, Bauer D, Mohn K.NKLM-Projektgruppe. Finally finished! National Competence Based Catalogues of Learning Objectives for Undergraduate Medical Education (NKLM) and Dental Education (NKLZ) ready for trial. GMS Z Med Ausbild 201532(3): Doc35. [DOI] [PMC free article] [PubMed]
- 18.Röhrig R, Stausberg J, Dugas M. GMDS project group “Medical Informatics Education in Medicine”. Development of national competency-based learning objectives “Medical Informatics” for undergraduate medical education. Methods Inf Med. 2013;52(03):184–188. doi: 10.3414/ME13-04-0001. [DOI] [PubMed] [Google Scholar]
- 19.Malone K, Supri S. A critical time for medical education: the perils of competence-based reform of the curriculum. Adv Health Sci Educ Theory Pract. 2012;17(02):241–246. doi: 10.1007/s10459-010-9247-2. [DOI] [PubMed] [Google Scholar]
- 20.Weinert FE, Smelser NJ, Baltes B.Competencies and key competencies: Educational perspective. International Encyclopedia of the Social and Behavioral SciencesIn:editors.4th ed.Amsterdam: Elsevier; 20012433–2436. [Google Scholar]
- 21.Boyatzis RE. Competencies in the 21 Manag Dev. 2008;27(01):5–12. [Google Scholar]
- 22.Phillips A, Yen PY, Kennedy M, Collins S. Opportunity and Approach for Implementation of a SelfAssessment Tool: Nursing Informatics Competencies for Nurse Leaders (NICA-NL) Stud Health Technol Inform. 2017;232:207–211. [PubMed] [Google Scholar]
- 23.Yoon S, Shaffer JA, Bakken S. Refining a self-assessment of informatics competency scale using Mokken scaling analysis. J Interprof Care. 2015;29(06):579–586. doi: 10.3109/13561820.2015.1049340. [DOI] [PubMed] [Google Scholar]
- 24.Prahalad CK, Hamel G.The core competence of the corporation. Harvard Business Review 1990. May-June.79–90.
- 25.Kulikowski CA, Shortliffe EH, Currie LM, Elkin PL, Hunter LE, Johnson TR et al. AMIA Board white paper: definition of biomedical informatics and specification of core competencies for graduate education in the discipline. J Am Med Inform Assoc. 2012;19(06):931–938. doi: 10.1136/amiajnl-2012-001053. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Arbeitskreis Deutscher Qualifikationsrahmen (AK DQR). Deutscher Qualifikationsrahmen für ein Lebenslanges Lernen22nd March, 2011 [cited 2018 Mar 13]. Available from:http://www.dqr.de/media/content/Der_Deutsche_Qualifikationsrahmen_fue_lebenslanges_Lernen.pdf
- 27.Breeden EA, Clauson KA. Development and implementation of a multitiered health informatics curriculum in a college of pharmacy. J Am Med Inform Assoc. 2016;23(04):844–847. doi: 10.1093/jamia/ocw023. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Tremblay M, Deckard GJ, Klein R. Health informatics and analytics – building a program to integrate business analytics across clinical and administrative disciplines. J Am Med Inform Assoc. 2016;23(04):824–828. doi: 10.1093/jamia/ocw055. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Krathwohl DR. A revision of Bloom’s Taxonomy: An Overview. Theory into Practice. 2002;41(04):212–218. [Google Scholar]
- 30.Ball MJ, Douglas JV, Hinton P Walker, DuLong D, Gugerty B, Hannah KJ . London: Springer; 2011. Nursing Informatics: Where Technology and Caring Meet. [Google Scholar]
- 31.Hübner U, Shaw T, Ball M.The TIGER Initiative. Hospital Healthcare Europe 2016174–177.[cited 2018 Mar 13]. Available from:http://www.hospit alhealthcare.com/it/tiger-initiative
- 32.Egbert N, Thye J, Schulte G, Liebe J, Hackl W, Ammenwerth E et al. An iterative methodology for developing national recommendations for nursing informatics curricula. Stud Health Technol Inf. 2016;228:660–664. [PubMed] [Google Scholar]
- 33.Australian Health Informatics Education Council (AHIEC) Health Informatics Scope, Careers and Competencies Version 1.9. 2011 [cited 2018 Mar 13]Available from:http://www.ahiec.org.au/docs/AHIEC_HI_Scope_Careers_and_Competencies_V1–9.pdf
- 34.bCanada’s Health Informatics Association, National Office Canada’s Health Informatics Association (COACH), Health Informatics Professional Core Competencies v3.0. Toronto, 2012 [cited 2018 Mar 13]Available from:https://www.coachorg.com/en/resourcecentre/resources/Health-InformaticsCore-Competencies.pdf
- 35.Gugerty B, Delaney CW.TIGER Informatics Competencies Collaborative (TICC). Final Report 2009[cited 2018 Mar 13]. Available from:http://tigercompetencies.pbworks.com/f/TICC_Final.pdf
- 36.Li S, Bamidis PD, Konstantinidis Set al. Prioritization of actions needed to develop the IT skills competence among healthcare workforce. PeerJ PrePrints 03e1137v3.2015[cited 2018 Mar 13]. Available from:https://dx.doi.org/10.7287/peerj.preprints.1137v3
- 37.Hübner U, Ball M, de Fátima H Marin, Chang P, Wilson M, Anderson C. Towards Implementing a Global Competency-Based Nursing and Clinical Informatics Curriculum: Applying the TIGER Initiative. Stud Health Technol Inf. 2016;225:762–764. [PubMed] [Google Scholar]
- 38.Bland JM, Altman DG. Cronbach’s alpha. BMJ. 1997;314:572. doi: 10.1136/bmj.314.7080.572. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.bAmerican Association of Colleges of Nursing Essentials of Doctoral Education for Advanced Nursing Practice Washington DC: American Association of Colleges of Nursing 2006 [cited 2018 Mar 13]Available from:http://www.aacn.nche.edu/dnp/Essentials.pdf [Google Scholar]
- 40.Parry D, Hunter I, Honey M, Holt A, Day K, Kirk R et al. Building an educated health informatics workforce – the New Zealand experience. Stud Health Technol Inf. 2013;188:86–90. [PubMed] [Google Scholar]
- 41.Chang P, Kuo MC, Ball MJ, Hannah KJ, Newbold SK.Chapter 25. Taiwan Model: Nursing Informatics Training. Nursing Informatics: Where Caring and Technologies MeetIn:et al. London: Springer; 2011411–428. [Google Scholar]
- 42.Strudwick G, Hardiker NR. Understanding the use of standardized nursing terminology and classification systems in published research: A case study using the International Classification for Nursing Practice(®) Int J Med Inform. 2016;94:215–221. doi: 10.1016/j.ijmedinf.2016.06.012. [DOI] [PubMed] [Google Scholar]
- 43.Brennan PF, Bakken S. Nursing Needs Big Data and Big Data Needs Nursing. J Nurs Scholarsh. 2015;47(05):477–484. doi: 10.1111/jnu.12159. [DOI] [PubMed] [Google Scholar]
- 44.Hackl WO, Rauchegger F, Ammenwerth E. A Nursing Intelligence System to Support Secondary Use of Nursing Routine Data. Appl Clin Inform. 2015;06(02):418–428. doi: 10.4338/ACI-2015-04-RA-0037. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Van Houdt S, Sermeus W, Vanhaecht K, De Lepeleire J. Focus groups to explore healthcare professionals’ experiences of care coordination: towards a theoretical framework for the study of care coordination. BMC Fam Pract. 2014;15:177. doi: 10.1186/s12875-014-0177-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Remus S. Advancing the Digital Health Discourse for Nurse Leaders. Stud Health Technol Inform. 2016;225:412–416. [PubMed] [Google Scholar]
- 47.bAmerican Nurses Association . 2nd ed. Silver Spring, MD: American Nurses Association; 2015. Nursing informatics: scope and standards of practice. [Google Scholar]
- 48.Kannry J, Sengstack P, Thyvalikakath TP, Poikonen J, Middleton B, Payne T et al. The Chief Clinical Informatics Officer (CCIO): AMIA Task Force Report on CCIO Knowledge, Education, and Skillset Requirements. Appl Clin Inform. 2016;07(01):143–176. doi: 10.4338/ACI-2015-12-R-0174. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Cummins MR, Gundlapalli AVI, Gundlapalli AV, Murray P, Park HA, Lehmann CU. Nursing Informatics Certification Worldwide: History, Pathway, Roles, and Motivation. Yearb Med Inform. 2016;(01):264–271. doi: 10.15265/IY-2016-039. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Gardner RM, Overhage JM, Steen EB, Munger BS, Holmes JH, Williamson JJ et al. Core content for the subspecialty of clinical informatics. J Am Med Inform Assoc. 2009;16:153–157. doi: 10.1197/jamia.M3045. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Ricciardi L, Mostashari F, Murphy J, Daniel JG, Siminerio EP. A national action plan to support consumer engagement via e-health. Health Aff (Millwood) 2013;32(02):376–384. doi: 10.1377/hlthaff.2012.1216. [DOI] [PubMed] [Google Scholar]
- 52.Moen A, Mæland LM Knudsen. Nursing informatics: decades of contribution to health informatics. Healthc Inform Res. 2013;19(02):86–92. doi: 10.4258/hir.2013.19.2.86. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Online Supplementary Material (PDF)
Online Zusatzmaterial (PDF)
Online Supplementary Material (PDF)
Online Zusatzmaterial (PDF)
Online Supplementary Material (PDF)
Online Zusatzmaterial (PDF)