Why decision support systems are important for medical education

Abstract

During the last decades, the inclusion of digital tools in health education has rapidly lead to a continuously enlarging digital era. All the online interactions between learners and tutors, the description, creation, reuse and sharing of educational digital resources and the interlinkage between them in conjunction with cheap storage technology has led to an enormous amount of educational data. Medical education is a unique type of education due to accuracy of information needed, continuous changing competences required and alternative methods of education used. Nowadays medical education standards provide the ground for organising the educational data and the paradata. Analysis of such education data through education data mining techniques is in its infancy, but decision support systems (DSSs) for medical education need further research. To the best of our knowledge, there is a gap and a clear need for identifying the challenges for DSSs in medical education in the era of medical education standards. Thus, in this Letter the role and the attributes of such a DSS for medical education are delineated and the challenges and vision for future actions are identified.

1. Introduction

During the last couple of decades, the inclusion of digital tools in health education has rapidly lead to a digital explosion. All the online interactions between learners and tutors, the description, creation, reuse and sharing of educational digital resources and the interlinkage between them in conjunction with cheap storage technology has led to an enormous amount of educational data.

As medical education is a unique type of education due to accuracy of information needed, continuous changing competences required and alternative methods of education used, it means that education assisting digital technologies should be revisited. Medical professionals in all educational levels – undergraduate, postgraduate and continues professional development – have the need for tailored education, while policy makers and organisations have to provide updated curricula and continuous medical education competences and skills needed with reports from organisations.

Medical education standards are mature enough in order to code the underlying medical education data [1] and the MedBiquitous blueprint [2] is capable of shaping the connection between different educational standards transformed from a vision to a reality. To this extent, education data mining is in its infancy, providing all the necessary techniques for analysis of such educational data.

The need of decision support systems (DSSs) in order to enhance the educational decision making is now more obvious than ever before. DSSs have been widely used in Medical and Health Sciences [3, 4] mainly for clinical decisions, but their use as tools for decision making in education are not widely spread.

Research initiatives for DSSs in education in general have existed for quite a few years [5], with the majority of them merely focusing on students and teachers and some on administration. There exist only a few frameworks that propose how to address DSSs in higher education, but they mainly focus on higher education systems and platforms [6].

To the best of our knowledge, there is a gap and a clear need for identifying the challenges for DSSs in medical education which will take into consideration existing educational standards and add value thereof.

Thus, in this Letter the role and the attributes of such a DSS for medical education are delineated and the challenges and vision for future actions are identified. The aim is not to describe a technical solution per se, but rather to reveal the needs and provide the qualities and guidance through a framework that could be used to develop DSSs specifically for use in medical education. Medical education standards and paradata could act as catalysts in the implementation of such framework.

The remainder of this Letter is structured as follows. In the following section, we provide a short overview of the role of educational data mining and existing DSSs for education in general, while in Section 3, we present a three-layered approach on the importance of DSSs in medical education. Last, but not least, in the discussion and conclusion section we briefly discuss the limitations of DSS in education and the future actions needed in order to realign DSS into daily medical education practice.

2. Background

Medical professionals require continuous and updated knowledge in their daily practice. To fulfil the gaps of required skills, the provided knowledge should be accurate and provided when, where, and how it is needed by the medical professionals [7]. The MedBiquitous Consortium has been developing a technology blueprint for medical education which supports collaborative technology systems to bridge the gaps between learners, educators, and certifiers and bring together activities, organisations, and resources [2]. Ultimately, this blueprint will seamlessly support the learner in ways that will improve patient outcomes and simplify the administrative work associated with lifelong learning and continuous improvement [8].

Based on those notions XML and Web services (and more recently semantic web and linked data based) standards have been released, or are under preparation for medical education, supporting the structured representation of medical education. Efforts identifying medical education standards include the mEducator space [1], in which standards are divided into six broad areas: of procedures for describing content items; of competencies; of evaluation; of quality assurance; of intellectual property rights (IPRs); and the learner's profile space. In addition to the above, the mEducator project proposed also the use of attention metadata [9] or paradata to reference data about user interactions with digital learning objects; this is now often used as a synonym for attention metadata [10, 11].

In addition, the MedBiquitous Consortium released a ‘map’ of standards showcasing the relationships between the different standards that it develops. Those standards include [8]: Activity Reporting; Competencies; Curriculum Inventory; Educational Achievement; Educational Trajectory; Healthcare Learning Object Metadata; Medical Education Metrics (MEMS); Performance Framework; Professional Profile; SCORM for Healthcare; Virtual Patients and so on.

The aforementioned medical education standards allow the description of medical education data which can be represented in a structured format identifying relationships and connections between them.

During the last couple of decades, the growth of ICT has led to an enormous creation of digital medical education resources that exist in institutional learning content management systems or other local and/or independent repositories. Medical educational data silos have started to be formed. Individual or small groups efforts have emphasised the use of web 2.0 technologies [12–17] for sharing and shaping high quality medical education resources, while other research groups have been investigating the likely role of semantic web and Linked Open Data on the enrichment of medical educational resources with additional knowledge and information [18–22]. In addition, Social Media have occupied a pivotal place within closed academic communities and medical associations when it comes to creating and exchanging medical educational resources [23, 24].

So the technological advancements exist in order to connect the existing medical education data from multiple institutions and share valuable information among them and feed DSSs in medical education.

DSSs make use of educational data mining techniques. Educational data mining techniques differs from traditional data mining techniques, in explicitly exploiting the multiple levels of meaningful hierarchy in educational data [25].

A lot of studies engaging educational data mining focus on the improvement of educational or learning design [26] while others focus on identifying the performance of the students [27]. Some research efforts identify the need for examining students learning behaviour in online learning environment [28], while the use of specific online learning content management system is central to the outcomes of educational data mining techniques [29].

A research review on educational data mining [30] identified five actors to use educational data mining (students, teachers, course developers/educational researchers, organisation, and administrators), and categorise the existing research outcomes according to the type of data they use or the environment they are research on: traditional education; web-based education/ e-learning; learning management systems; intelligent tutoring systems; adaptive educational systems; test/questionnaires; text/contents; and some do not belong in any of the above categories.

DSSs are well established in Medical and Health Science fields. Clinical DSSs are widely used in multiple medical fields [3, 4, 31–33] and even address cases with multiple conditions [34].

DSSs have started to be discussed before the establishment of eLearning and learning content management systems, mainly for schools, aiming at performance improvement, enhanced professionalisation and school renewal [5, 35]. DSS in education has been proposed for administration purposes to provide personal support for complex and managerial decisions [5, 36]. In some cases, DSSs are proposed to be used on top of education management information system to generate options for the decision makers to improve policy, strategy, planning, assessing, and monitoring of the educational systems [37], and to this extent, conceptual frameworks for DSSs have been proposed for higher education mainly identifying needs for students, teachers, and organisation based on organisation existing systems, platforms, and databases [6, 38, 39].

3. Fruits in the basket

The need of DSSs in medical education is urgent in order to provide help to the decision making process in different levels of the education process. Initially, the exact role and the challenges of DSS should be identified and in order for this to accurately be made, all the stakeholders involved in the learning process should be identified.

All the stakeholders involved (Fig. 1) in the learning process can use the wealth of educational data in different ways and for different decision making processes according to their requirements. Interested parties in this regard are the following:

• Medical students who need to track and measure their performance, and identify areas for improvement.

• Medical/health educational content providers who need to know how their content is used so as to improve and adapt their offerings to different markets.

• Medical schools/medical educational institutions and services that need to understand the profile of their students as well as the effectiveness of their teaching and learning methods, online learning resources, and tutors.

• Health education policy makers responsible for the evaluation and ranking of the supervised institutions according to global or even custom criteria.

Fig. 1.

Stakeholders benefit form DSSs and techniques in medical education

Taking into consideration the aforementioned roles, challenges for decision making focus on three layers: the educational, the institutional and the policy layers.

The educational layer (layer 1 in Fig. 1) engages all the decision support making regarding training and learning procedures. The main stakeholders in that level are the students and the medical/health educational content providers. Within that level a DSS utilise data from the students’ engagement with medical educational content, the interaction between them and them and the tutors, students’ digital assessments and detailed identification and use of provided educational content. So, a DSS should provide to the students information that will help them track their progress and focus on areas for improvement. Such information may include:

• Performance data (best/worse/average scores) about a specific test, book, subject, or combination of the above.

• Individual ranking in the student's educational context.

• Spider-type graphs displaying performance in selected thematic areas and highlight the student's strong and weak points.

• Students favourite learning means and techniques.

To this extent, medical/health educational content providers through DSS are able to find information for their content, based on learners’ interaction with it. Thus, such a system allows the tutor to take decisions on the actions needed to improve their material by providing information regarding the following:

• How the provided medical educational content is used, based on program, time frame, demographics etc.

• How do students perform based on region, time frame and demographics.

• How the use of the provided medical content does relates to the use of other materials or methods.

• What the feedback is on the provided medical educational content.

Within the institution layer the decision making processes engage data regarding the performance of the educational programs; data regarding the performance (in relation with engagement, satisfaction, and student assessment/feedback) of the educational material, learning designs comparisons. Curriculum learning outcomes and students learning outcomes could be mapped for institutional evaluation performance. Thus, the use of DSS will provide the following information:

• Students’ performance by module, educational program, learning resources, teaching and learning method, teacher etc.

• Students’ satisfaction by module, educational program, learning resources, teaching and learning method, teacher etc.

• Students’ engagement and online learning resources usage based on the time which each student engages with online material in comparison with the average.

• Evaluation and ranking of educational programs, learners, tutors and online learning resources.

• Data for visualisation tailored to different institutions needs and different types of used standards.

Medical/health education policy makers are always interested in efficient cost-effective ways of comparing and identify gaps in the medical education. DSS will need to be fed with data from a big number of institutions including data for engagement, satisfaction and student assessment/feedback, region, time and demographics, learning material use, evaluation and performance.

Based on that data DSS helps policy makers decide on the effectiveness of their strategy and tactics and proactively take important decisions by answering critical questions, such as

• How are institutions ranked by multiple criteria according to their performance and behaviour?

• How do learners perform on available kinds of medical education based on region, time, and demographics?

• What is the return on investment on learning materials such as textbooks, e-learning platforms, and online content?

As depicted in Fig. 2 the three layers are connected, and the layer on top cannot exist without the layer below. In the first layer, the two groups of stakeholders co-exist and for taking a decision based on the data available, both of the stakeholders should provide their part of the data in order the DSS to properly work. The second layer cannot get any decision from the DSS, unless all the data of the first level exist, while at the third level there is the prerequisite that different institutions provide their data for policy decision making.

Fig. 2.

DSS output within three layers of medical education

Medical education standards are mature enough in order to code the medical education data [1] and the MedBiquitous blueprint [2] which shape a connection between different educational standards. In order DSSs to provide a concrete results at the educational level all the MedBiquitous Consortium standards [8] will be used (Fig. 3): For example, the educational content itself (Virtual Patient, SCORM for Healthcare) should be organised so as to be able to share interactive computer programs that simulate real-life clinical scenarios for education and assessment purposes and learning resources, but it also need a standardised description of healthcare educational resources and activities (HLOM). To this extent, a common format for exchanging clinician contact, education, training, certification, and membership information (professional profile) is essential in order to monitor learners’ performance, engagement and satisfaction accomplished with a common format for the expected levels of performance related to a competency framework (performance framework). In the interest, both of the educational and the institutional layers are a common format for gathering and communicating evaluation data on healthcare education activities (MEMS), while a common format for documenting learner competency and entrustment across the continuum of health professions education (educational achievement) and for reporting professional education and certification related accomplishments (activity reporting) is essential for DSSs in medical education. For all the three layers (policy, institutional and educational layers), a common format for representing a list of competencies relevant to a profession or specialty is needed (competencies). A common format for curriculum data for benchmarking and educational research (curriculum inventory) and the tracking, planning, and audit of learners’ educational trajectory across medical schools and national organisations (educational trajectory) are necessary in order a DSS to provide valuable information for policy makers.

Fig. 3.

Data representation through MedBiquitous Consortium education standards in relation with the three layers

4. Conclusions

In this Letter, we have shaped the likely roles of DSSs in medical education together with their challenges and future actions. The proposed framework reveals the needs and is intended to provide guidance for future implementations in the scientific area of Medical Education Informatics. We showcase the linkage between existing medical education standards and paradata which are paramount for the implementation of such a framework. The whole picture of medical education challenges is decorated with technical standards in an attempt to emphasise how essential DSS might turn out to be in future years.

Ethical limitations in using DSSs in medical education could be easily raised, and even though ways of handling students’ privacy in small-scale research might be straightforward, institutions struggle to create such generic ethical frameworks. Solutions have been proposed in [40] along four axons: transparency, student control over data, right of access, accountability and assessment.

Existing educational data mining has been employed for a variety of contexts in education to date and many of the general purpose systems could potentially guide the design of DSS in medical education without considering the unique aspects of it. A holistic approach for a DSS for medical education will find its place in the different stakeholders use only if the data elaborated have a concrete structure. This is achievable through the existing medical education standards. In addition, linked mechanisms between the institutions are essential in order for a DSS to analyse widely useful data and help in decision making. Along this line for educational resources, the mEducator project [20, 22, 41, 42] proposed a number of best practices, along with open source mechanisms to interlink educational resources and monitor their evolution. Further research is needed on the interlinking between the paradata [10] of the different systems. To this extent, the expansion of learning analytics and educational data mining [43, 44] over the last few years looks promising for creating intelligent techniques to be used in DSSs. Another component that should be taken into consideration is the representation of data tailored to the different stakeholders in order to understand and gain the most out of it [45, 46]. This is probably the whole essence behind the big educational data and their recent explosion.

To conclude, it is believed that DSS for medical education will find its unique place in education practice in the years to come. The four different groups clustered in the three levels of medical education having a vested interested in it, they will value outputs of DSS for medical education in a different way and use it for different reasons and from a different perspective. Keeping the systems compliant with medical education technical standards will probably provide the pivotal point for the wider exploitation of the different data in the decision making process. To this extent, this Letter has put a small, but vital cornerstone towards that direction.

6. Funding and declaration of interests

Conflict of interest: none declared.