The Progress Test of the European Hematology Association: A New Tool for Continuous Learning

The European Hematology Curriculum, first launched in 2006, was created by the European Hematology Association (EHA) with the aim of harmonizing hematology training in Europe. Its goals were to define the different areas hematologists are expected to cover during their training, and to establish the minimum recommended levels of competence that a hematology trainee should attain.¹ The logical next step was the creation of the European Hematology Exam in 2015,² in the wake of several European medical associations that use similar examinations as part of their end of training certification. Since then, the number of candidates and countries participating has increased. Switzerland has included the examination as part of its final assessment toward specialization as a hematologist. During the COVID pandemic, participation in the examination continued unabated, reflecting its importance for hematologists across Europe and beyond.

EHA’s education platform (EHA Campus) offers opportunities for continuous learning for both trainees and specialists. Content is guided by the European Hematology Curriculum, which provides a structure for individual study and self-assessment. To complete this organized learning environment, a tool for objective assessment of knowledge during and after specialist training was needed. In the spring of 2020, EHA started offering a progress test: a longitudinal test based on equivalent evaluations given at fixed intervals, assessing developments in knowledge. The EHA Progress Test was inspired by an earlier version developed by the Swedish Hematology Association in 2013, which has become widely used by specialist trainees and specialists in Sweden. Noticeable pedagogical effects, like targeted study efforts in weak knowledge areas, changes in clinical rotations, and more have been reported in personal questionnaires. However, due to the limited number of Swedish trainees, data have not been published.

In undergraduate settings, progress testing has been used for a long time, first introduced at the University of Maastricht (“The Maastricht progress test”) in 1977.³ It is now widely used in many undergraduate programs all over the world. Students are given a test containing an extensive set of multiple-choice questions (MCQs) once or twice a year, serving both as an educational tool and as formal examinations of knowledge.⁴ The information gathered can also be used for (re)development of the curriculum and, in the case of collaboration between medical schools, comparison of results between institutions.⁵ In specialist training, progress tests have rarely been implemented. Instead, the focus has been on learning through clinical experience, which means that longitudinal knowledge testing has not been explored in most specialties. Trainees are mostly expected to build their knowledge base in a self-directed way by accumulating clinical experience, often with the help of a clinical tutor. There is extensive research on progress testing in undergraduate medical training, and the usefulness of the approach has been well validated. In postgraduate settings, there is a striking paucity of reports on such testing, even though attempts have been made in a few specialties.^6–9

EHA developed the Progress Test to serve both trainees and certified hematologists. It should lead to greater comprehension of their level of knowledge in the different sections of the European Hematology Curriculum, so that everyone can work on improving their knowledge in the weaker sections aiming at intrapersonal development.

To assess progress over time, the level of the test needed to be consistent from one session to the next. Hence, we decided to use the previous versions of the European Hematology Exam. The questions of the examination are written by a trained group of question writers and rigorously peer reviewed according to the guidelines formed by the Examination and Progress Test Working Group. The choice for MCQs allows for a large sample size. Even with limited testing time, it can assess knowledge in several areas with high reproducibility and can be used in a web based format.^10,11 The examination questions represent all 8 sections of the Curriculum and are written in English.² To guarantee independent judgment of the quality of the examination and the passing score definition, EHA collaborates with Cito, an internationally recognized professional research institute in the field of testing and educational measurement. Cito judged the European Hematology Exam to be a robust method of knowledge testing.² All 2017 exam questions were reviewed for the 2020 Progress Test by their authors to ensure they were still up to date, considering changes in diagnostics, treatment, and practice guidelines, and adapted if needed. Authors were also asked to provide a short explanation of why the answers were right or wrong with references to relevant literature or learning materials as feedback.

The same Progress Test was made available to EHA members twice a year, thus expanding the opportunity for everyone to take the test every year. The test was accessible on the EHA Campus (ehacampus.ehaweb.org) for one month, in March and November each year. After this period, participants were able to access their results and feedback for 3 months to guide their studies. The first Progress Test was made available in spring 2020, using the exam of 2017. The same test was again made available in fall 2020. A new Progress Test was created using the 2018 examination and made available spring and fall of 2021. To make results comparable over the years, participants were advised to take the test in a continuous time period (2.5 hours), in a room with minimal distraction. They had access to a Unit Reference Card that could be opened in a separate window. After taking the test, each participant could immediately access their results: correct answers, explanations, and suggested literature. A summary displaying the overall result and the result per section of the curriculum was sent to every participant within 3 weeks after closure of the test. Results were not expressed as pass or fail, underscoring that the Progress Test is not an examination but a formative assessment to be used as a continuous educational tool. Participants received a graph depicting their ability and a separate table showing whether they scored below (red cross) or at/above (green V) the expected end-of-training level for each section of the European Hematology Curriculum (Figure 1). In addition, since a significant group was expected to use the Progress Test as a preparation for the examination, participants were provided with the number of correct questions that would have been a pass in the EHA European Exam in the year the Progress Test was based on. All candidates were asked to provide demographic and training background information using an online survey. The candidates were also asked why they chose to take the Progress test and what their experience was.

Figure 1.

Representation of candidate performance. The white line is the expected end of training level.

Participation increased with each test, from 141 in spring and 156 in autumn 2020 to 185 in spring 2021. Two hundred fifty-eight participants took one test, 68 took two, and 19 took all three; 207 of them answered the survey. There were participants from 44 countries with 70% of the participants from Europe. Most were in specialty training and 26–35 years old. The participants found the Progress Test to be relevant to their hematology training, the vast majority being clinicians (86%–93%). Participants mentioned that some domains were beyond the scope of their practice: some only treated malignant diseases, and in other countries, disorders of thrombosis and hemostasis are not treated by hematologists. The European Hematology Exam and the Progress Test are made for European hematologists and based on the European Hematology Curriculum. Even so, writing questions that apply to hematology practice in the whole of Europe is a challenge. It is reassuring that such a high percentage of participants considers the Progress Test relevant to their training. The many participants from countries outside of Europe report the same experience. Half of the participants used it “to prepare for the specialist exam” and 54% “to use it as a learning tool.” Both reasons for participation are what EHA aimed for when constructing the Progress Test. The overall evaluation of the experience of the test scored 4.5 on a scale of 1–5. Most of the participants would have passed the examination with their scores on the Progress Test: 75.7%, 68.1%, and 66% in spring 2020, fall 2020, and spring 2021, respectively. Thrombosis and hemostasis were the section with the lowest scores.

The Progress Test can be used in different ways. It could be used by educators and hematologists in training as a basis for evaluation and to plan for future training goals, targeting areas where knowledge is below the expected level. For trained hematologists, this information could influence decisions on which courses or conferences one might choose to attend the following year. There are even examples of the test being used in a trivia-like setting, for example as a quiz. EHA can use (under)achievement in different sections of the Curriculum to guide the development of new educational materials. Using participants’ evaluations, feedback can be adapted.

Whichever way the Progress Test is used, individually or in groups, the objective of improving knowledge will still be achieved. Its increasing popularity will most likely enable more detailed analyses in the future.

AUTHOR CONTRIBUTIONS

MJW and GB participated in design and writing of the paper, and data analysis. AA, JB, CFL, JTN, MP, AR participated in writing of the paper. NvH and WH participated in writing of the paper and data analysis. BTH participated in data analysis.

DISCLOSURES

The authors have no conflicts of interest to disclose.