Advancing clinical reasoning in virtual patients – development and application of a conceptual framework

Abstract

Background: Clinical reasoning is a complex skill students have to acquire during their education. For educators it is difficult to explain their reasoning to students, because it is partly an automatic and unconscious process. Virtual Patients (VPs) are used to support the acquisition of clinical reasoning skills in healthcare education. However, until now it remains unclear which features or settings of VPs optimally foster clinical reasoning. Therefore, our aims were to identify key concepts of the clinical reasoning process in a qualitative approach and draw conclusions on how each concept can be enhanced to advance the learning of clinical reasoning with virtual patients.

Methods: We chose a grounded theory approach to identify key categories and concepts of learning clinical reasoning and develop a framework. Throughout this process, the emerging codes were discussed with a panel of interdisciplinary experts. In a second step we applied the framework to virtual patients.

Results: Based on the data we identified the core category as the "multifactorial nature of learning clinical reasoning". This category is reflected in the following five main categories: Psychological Theories, Patient-centeredness, Context, Learner-centeredness, and Teaching/Assessment. Each category encompasses between four and six related concepts.

Conclusions: With our approach we were able to elaborate how key categories and concepts of clinical reasoning can be applied to virtual patients. This includes aspects such as allowing learners to access a large number of VPs with adaptable levels of complexity and feedback or emphasizing dual processing, errors, and uncertainty.

Background

Clinical reasoning and related concepts such as clinical decision making or problem-solving have been major topics of healthcare education research. Clinical reasoning is a complex set of skills that encompasses the application of knowledge to collect and integrate information from various sources to arrive at a (working) diagnosis and management plan. The symptoms and findings of a patient have to be matched to a set of differential diagnoses to be able to arrive at a working or final diagnosis. Although clinical reasoning is a fundamental skill and one that has been the focus of research for over 30 years, it is not yet fully understood [1], [2] .

Healthcare students must acquire clinical reasoning skills and continue to build upon them in their clinical work [1]. However, experienced healthcare instructors often find it difficult to slow down and fully explain their clinical reasoning; reasons include the dynamic and often unconscious components of the reasoning process [1] and a lack of formal training [2].

During medical school clinical reasoning is often taught in bedside teaching courses or problem-based tutorials, but virtual patients are becoming a more important teaching activity to train and assess this skill to prepare students for real patient encounters [3], [4].

VPs in healthcare education are interactive computer-based programs that simulate real-life clinical scenarios [5]. The form of VPs varies greatly and ranges from basic text-based scenario descriptions to high-fidelity software simulations or virtual reality scenarios [6]. Evidence suggests their use in the form of interactive patient scenarios supports student clinical reasoning skills [7]. The first articles about using computers to simulate clinical reasoning for students were published in the 1970s [8] and VPs have gained popularity in healthcare curricula over the past 40 years. VPs have been integrated into undergraduate healthcare curricula in different formats, such as blended-learning scenarios [9], during clinical clerkships [10], or as assessment tools [11].

The knowledge gap this study intends to address was recognized years ago and remains largely unaddressed. Cook at al. suggested that VPs should be ideally suited to teach clinical reasoning, but that how learning occurs is not yet fully understood. Further, they concluded that there is insufficient evidence how specific VP design variations, such as content, authenticity, interactivity, or feedback, support clinical reasoning acquisition [7]. How VPs model and influence clinical reasoning and consequently, how VPs should be presented to healthcare students remain unanswered questions.

To advance this question, we decided to "zoom out" from VPs and VP technology as elaborated by Edelbring et al. [12] Thus, our aims were

1. to identify key concepts of learning clinical reasoning in general and

2. evaluate how these concepts are reflected in virtual patients to

3. elaborate recommendations on how to advance the learning of clinical reasoning with virtual patients.

Methods

We chose a qualitative approach based on Glaserian grounded theory [13] to explore the broad topic of clinical reasoning skills acquisition. Grounded theory is an inductive research methodology to understand a phenomenon and develop a theory which is anchored in the data. In an iterative process data collection, coding, and the development of the theory interact with each other [14]. Thus, grounded theory aligned with the focus of our research and allowed us to explore and include a wide range of data sources into our analysis. Figure 1 (Fig. 1) shows an overview of the study design. Following the grounded theory dictum "all is data" [13] we included a wide range of data sources, such as scientific literature, videos, or websites, into our analysis, since our approach was focused on synthesizing descriptions, frameworks, and teaching approaches of clinical reasoning. In a second step we explored how the emerged framework is represented in virtual patients and introduced conclusions on how to improve virtual patients accordingly.

Figure 1. Study design with two steps:

Exploration of learning clinical reasoning with framework development based on memos and concept maps, and application of the framework to virtual patients and elaboration of conclusions.

One researcher (IH), an experienced healthcare education researcher, conducted the study in close communication and cooperation with all authors.

Data collection and analysis

The main researcher started the grounded theory with the book "Developing clinical problem-solving skills" by Barrows et al. [15], which was coded completely. The book covers a broad range of teaching clinical reasoning aspects, and therefore we found it to be an ideal starting point for the analysis. In a cyclic process [16] the researcher applied theoretical sampling based on memoing to guide further data collection. Based on emerging codes she searched for data sources that could

1. further explain,

2. contrast, or

3. add a new perspective.

The Google search engine and PubMed were used as search tools for data sources, additionally, references of a data source were considered as potential new data sources. The search was not limited to a particularly time frame or source or authors.

Before coding a data source the researcher briefly scanned the abstract or summary and decided about its inclusions if at least one of the above described criteria was fulfilled.

For example, during coding the book by Barrows et al. the theme "role of the teacher" emerged, which required further elaboration. The researcher then identified and explored other sources such as an article by Eva [17] or the Massive Open Online Course (MOOC) of the University of Michigan about Instructional Methods in Health Professions Education [https://www.coursera.org/learn/instructional-methods-education] to further elaborate this theme.

If the emerging themes could not sufficiently be explored with articles or online resources, the researcher conducted interviews with different stakeholders, such as healthcare educators or researchers to further explore the themes. For example, "learning from errors" and "concept mapping in clinical reasoning" were further explored in interviews (see Attachment 1 for details about all conducted interviews). The interviews were colloquial conversations recorded in field notes; they were held online or face-to-face in a private setting.

A process schema with all data sources and why and when in the process they were included is available on request.

Before the core category - a core theme that integrates all lower level categories - emerged, the researcher open-coded each data resource; after that she applied selective coding to elaborate relations between concepts and further specify the nature of the categories. Codes were constantly compared, revised, and merged into concepts. During the process, the researcher composed concept maps and memos, which included the date of analysis, a brief summary, relations of key aspects, suggestions for further themes to explore, emerging ideas, and specific links to the data source to document the analysis.

Overall, 107 data sources, including six interviews, were coded; a full list can be found in Attachment 1.

Consistent with a theoretical sampling approach, the researcher continued the data collection until saturation was reached, i.e. no additional data were found to develop new themes in categories or new relationships between the themes. The memos and concept maps were then analyzed and a framework constructed based on the emerging categories, concepts, and relations.

Evaluation of virtual patients based on the framework

In a second step we transferred the framework to the virtual patient context and analyzed resources, such as VP-related literature, virtual patients and VP systems (a full list of data sources can be found in Attachment 2). The researcher analyzed virtual patients and authoring guidelines [http://www.virtualpatient.eu], [http://vpsystems.virtualpatients.net] based on the categories and concepts of the framework. The findings were complemented with an analysis of scientific articles which were extracted in a purposive literature search for each category.

The researcher then explored how concepts were implemented in these VPs and, similar to the grounded theory approach, documented the findings in memos and concept maps.

We discussed the developed concepts, theoretical saturation, the emerging framework, and our conclusions on virtual patients within an interdisciplinary panel of two healthcare educators, two healthcare professionals , two healthcare education researchers, an undergraduate medical student, and a computer scientist. The concept maps and memos served as a basis for these discussions.

Ethical approval

We received ethical approval from the Ethical Committee of the University of Munich for the interviews conducted for this study.

Results

During the grounded theory analysis we identified the core category "multifactorial nature of learning clinical reasoning". Based on the data we elaborated the following five categories with main actors: Learner-centeredness (learner), Patient-centeredness (patient), Psychological Theories (researcher), Teaching/Assessment (teacher), and Context (healthcare professional). Categories and their dedicated concepts are connected to the core category, but also interrelated with each other (see Figure 2 (Fig. 2)).

Figure 2. Framework of learning clinical reasoning with the five main categories, related concepts, and interrelations derived in the first step of the study.

Below, we will elaborate on the results of each category, give examples on how they are grounded in the data, and introduce findings from the analysis of the virtual patient presentations based on the framework. A summary of the results is shown in Attachment 3. Detailed concept maps for each category and an overall map showing the relations of the data sources and the concepts can be provided on request.

Context

Context emerged as an overarching concept that influences most aspects of clinical reasoning and learning [18]. In a clinical encounter patients and healthcare professionals interact in a certain environment, or context, with each other. Emotions influence this interaction and are an important component of the clinical reasoning process [19]. The learning context - where, with whom, and how the learning takes place - is also relevant [20]. In such a learning environment the balance between authenticity and cognitive load influences the learning experience [21], [22].

Time is an important aspect in different ways. First, it plays a role in a clinical encounter in terms of time pressure or when an encounter takes place - for example late at night or during the day. Secondly, learners need time to develop clinical reasoning skills [23], but, in learning contexts time may be restricted due to curricular constraints. And thirdly, time can be an indicator for efficient clinical reasoning (pattern recognition) or a deliberate analytical process.

In contrast to a real patient encounter the learning and the clinical context are separated in a VP environment. Time influences the clinical setting as well as the learning environment [24]. However, VPs are typically static – learners are either rewarded for a long and deliberate engagement with a VP or for a quick and time-efficient problem solving approach.

Learner-centeredness

Learner-centeredness emerged as an important category for the learning of clinical reasoning [2]. The concepts in this category are tightly interrelated with each other and other categories (see Figure 2 (Fig. 2)). In self-regulated learning (SRL) environments the learner is autonomous and controls his own learning activities and settings, which can increase his engagement and motivation [25]. Deliberate and active engagement is necessary to learn clinical reasoning skills [26]. Adaptability emerged as dynamically adjusting learning content, context [18], feedback [27], scaffolding [28], or other activities to the learners' needs and level of expertise [29]. For example, feedback can be adaptable in terms of who provides feedback – an instructor, peers, or the VP – and when, or how detailed it is given.

Learning analytics – i.e. the collection and analysis of learner activities – can form the basis for adapting a learning activity based on the learner's skill level in clinical reasoning. Also, learning analytics can be the basis of the feedback to the learner and recommend next activities [30].

Virtual patients are learner-centered, adaptable activities [31], [32] however, the degree of learner-centeredness is limited, because VPs are often used to standardize teaching and assessment [33].

Psychological Theories

This category is based on psychological theories and frameworks that have been developed to explain the clinical reasoning process. It includes dual processing, which encompasses analytic reasoning, pattern recognition (i.e. non-analytic reasoning), and a continuum between these two approaches [34]. Also, the concept of knowledge encapsulation, a process that supports the development of illness scripts [35] emerged in this category. Both, illness scripts and knowledge encapsulation are prerequisites for pattern recognition and are more prominent in experts than in novice learners. Another concept is metacognition [e.g. 25] - thinking or reflection about one's own thinking. It is an important strategy to reduce or process cognitive errors in clinical reasoning [36]. Evidence suggests that for developing expertise novices have to compare and contrast many patient cases, to encapsulate their knowledge and develop individual illness scripts [37].

In addition to seeing real patients at the workplace, learning with standardized patients or paper-based cases, virtual patients can be an answer to this requirement, for example by allowing deliberate practice.

To support analytical reasoning most VPs implicitly include components of an illness script, such as a problem list, differential diagnoses or a final diagnosis [38], which can also form the basis for authoring a VP scenario. Learners are either prompted to construct these components or they are automatically revealed in the scenario.

Teaching/Assessment

Assessment and teaching emerged as strongly interrelated with learning of clinical reasoning, but this category focuses more on the educator's view on what and how a learner should learn and be assessed.

Because clinical reasoning is a complex and non-linear process, evaluating and scoring learning or assessment activities is challenging [39]. Often there is no single correct answer and uncertainty and ambiguity need to be addressed.

Numerous methods to assess different steps in the clinical reasoning process have been described [40]; few, such as script concordance testing [41] or concept mapping [42] reflect the non-linear aspects of clinical reasoning.

Based on the data sources, we coded communication as a teaching/assessment concept, that is mainly related to teaching and assessment of clinical reasoning skills [43], although it is an important aspect for all interactions between actors. Typically, assessment of clinical reasoning skills at the workplace involves communication between a learner and an instructor or senior physician; for example, a learner presenting a summary statement of a patient or elaborating about possible differential diagnoses [43].

VPs are used as assessment tools, especially for formative assessments [44], and they also include a variety of methods to assess clinical reasoning skills, such as multiple choice questions or decision points. However, the non-linearity of clinical reasoning is a challenge for scoring and feedback and the quantitative methods often implemented in VPs, do not sufficiently reflect this.

Patient-centeredness

This category encompasses concepts related to patients, such as patient safety and errors, which may not be covered sufficiently in healthcare education [45]. Different types of errors, such as premature closure, related to different steps in the clinical reasoning process, may occur and consequently affect the patient [46]. Biases (e.g. gender or confirmation bias) or overconfidence with one's own decisions can cause such cognitive errors [36].

Another concept in this category is management decisions. Although related to illness scripts, the management of patients with the same disease is heterogeneous and management decisions need to be individually tailored to the patients' needs [18].

Virtual patients provide a safe environment for learners, patients and educators, which is crucial to allow learners to make errors and to learn from them without threatening a patients' safety [https://members.aamc.org/eweb/upload/Effective%20Use%20of%20Educational.pdf], [47].

The patient presentation in a VP varies from one-sentence descriptions, which neither address the patient by name nor give him a face, to more patient-centered representations [48], which include media of the patient and his/her reactions and emotions.

Attachment 3 summarizes the results of the second step of the qualitative exploration, in which we applied the elaborated framework to the state-of-the-art research and development of virtual patients. The Attachment 3 also includes open questions, which should be kept in mind when designing or providing virtual patient-based learning scenarios.

Discussion

We identified five categories and related concepts for "learning clinical reasoning" in a grounded theory approach. The developed framework (see Figure 2 (Fig. 2)) visualizes the main components and interrelations we have identified in our study. It is fairly complex, but we believe it adequately reflects the core category "multifactorial nature of learning clinical reasoning" and the diversity and richness of the topic.

Based on the five categories and concepts of the framework we suggest improvements in the design and use of VPs for learning clinical reasoning.

Context

The separation of the learning and clinical context in VP scenarios offers the opportunity to let the learners adjust both contexts separately to their needs. We suggest providing an individually increasingly challenging context of the virtual clinical encounter, for example by including irrelevant information or emotionally complex situations depending on the learner's skill level and goals. However, we consider it important to balance authenticity and cognitive load to not overburden the learners.

The often static nature of VPs does not reflect the dynamic character of clinical reasoning. Instead of considering the time on task as a fixed scoring component we suggest including it as component to adapt VP complexity. For example, the available time of the virtual clinical encounter could be limited (time-pressure) for more advanced learners or those who want to challenge themselves. To simulate a clinically authentic and time-pressured environment the learner can also be challenged to solve multiple VPs simultaneously.

Learner-centeredness

We propose a more flexible environment to allow learners to deliberately practice individually or in a team. Learners should be able to choose the level and type of feedback or adapt aspects such as context and content complexity. Computational models integrated in virtual patients can be the basis for such adaptable content

A VP system may recommend next activities, not necessarily limited to VPs, or further adaptions based on the analysis of learner's performance, level of self-direction, and self-defined learning goals for each step of the clinical reasoning process. Such an approach would enable learners to focus on individual areas of weakness. Continuous, timely, and specific feedback should be provided by the VP system, an instructor, or in form of summative peer or expert responses.

Psychological Theories

Although there are still many open questions concerning the exact nature of dual processing, there is evidence for a continuum or interaction between the process of pattern recognition and analytical reasoning. We envision a dynamic representation of the clinical reasoning process in VPs that allows learners to apply analytical reasoning, pattern recognition, or some combination thereof depending on the complexity of a VP, their skill level and goals.

We recommend creating a large pool of short and focused VPs to cover a variety of problems and allow learners to select VPs of varying clinical presentations and levels of complexity.

We recommend uncovering the implicit link between VPs and the illness scripts they are based on. This would engage the learner in actively and explicitly building their own illness scripts based on the VP scenarios. Such individually created illness scripts form stand-alone learning achievements, and learners may be encouraged to further develop and enrich them even outside a VP environment. To support pattern recognition learners could also be allowed or even encouraged to skip steps of the illness script building and make a diagnostic or management decision at any stage of the VP scenario.

Teaching/Assessment

To better reflect the non-linear character of clinical reasoning in VPs, we suggest assessing clinical reasoning as a multi-step process [3] using a variety of assessment and feedback methods for each step, including qualitative methods. For example, concept mapping could be an ideal method to visualize the VPs illness script. Consequently, that requires applying more complex scoring algorithms that take into account the diversity of such maps.

Patient-centeredness

A minimalistic patient presentation reduces the cognitive load for the learner, but does not adequately represent the patient in the clinical reasoning process. Also, it does not account for other factors that influence the clinical reasoning process, such as emotions, communication challenges, or biases.

On the other hand, media-rich VPs are more time-consuming for learners to work through, thus, contradicting the concept of working through many cases to develop illness scripts. To balance these two approaches, we recommend providing at least a basic description of the patient (including name, age, and some contextual information) and an image of the patient in all VPs and enrich some with additional media and elaborated descriptions to more adequately address the patients' role and emotional situations.

Virtual patients provide learners a safe environment in which they can learn at their own pace without harming a real patient. Therefore, we suggest including errors into VP scenarios and we envision advancing this concept by provoking errors and explicitly including potential causes, such as biases. In addition, immediate feedback, an elaboration of the error, and strategies for avoidance should be provided.

Finally, we suggest more explicitly covering the individual nature of management decisions, which can vary from patient to patient. Factors influencing these decisions should be elaborated.

Limitations

We are aware that our study has several limitations. Due to the large amount of data resources, especially literature, related to clinical reasoning, decision making and problem solving, our data sample size was relatively small, even if care was taken to ensure broad and covering sampling and theoretical saturation was discussed with an interdisciplinary panel of content matter experts.

We are also aware that the study design is unusual, since we did not discuss the emerging framework directly, but its application to virtual patients.

Despite our effort to form an interdisciplinary panel of experts representing a wide range of perspectives, we were unable to include patients.. However, patients are important stakeholders in the process of learning clinical reasoning and we suggest further research to include their view.

Except for the lack of patient's perspective we found the size and diversity of the expert panel (seven members) ideal for discussing the study. However, we cannot exclude that with a larger number of experts new perspectives might have arisen and influenced the final framework.

Conclusions

In our two-step approach and "zooming out" from virtual patients we developed a framework for learning clinical reasoning. In the second step we applied this framework to the world of virtual patients and drew conclusions on how clinical reasoning in VPs can be enhanced to be more effective learning resources.

Based on these conclusions we will develop a platform-independent open source clinical reasoning toolbox that can be integrated into VP scenarios. Furthermore, we plan to implement a large-scale, cross-institutional study to evaluate our approach. The results of this study will inform the further development of clinical reasoning in VPs.

Due to the general character of the framework, we believe that it is also applicable when developing or advancing clinical reasoning curricula or faculty development courses about clinical reasoning.

Funding

This project (IH) receives funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 654857 and AK is supported by internal funds at Jagiellonian University No K/ZDS/006367.

Acknowledgements

We would like to thank all students, researchers, clinicians, and educators who gave feedback in interviews, discussions, or the survey. We also thank Dr. Katja Kühlmeyer from the Institute for Ethics, History and Theory of Medicine at LMU Munich, who advised and supported us on the study design. Finally, we thank Prof. Leslie Fall, Prof. Martin Fischer, and Martin Adler for supporting the project.

Competing interests

The authors declare that they have no competing interests.

Erratum

The name of the second author Kononowicz was originally misspelled (Kononowic).