Putting the puzzle together: the role of ‘problem definition’ in complex clinical judgement

Abstract

CONTEXT

We teach judgement in pieces; that is, we talk about each aspect separately (patient, plan, resources, technique, etc.). We also let trainees figure out how to put the pieces together. In complex situations, this might be problematic. Using data from a drawing-based study on surgeons’ experiences with complex situations, we explore the notion of ‘problem definition’ in real-world clinical judgement using the theoretical lens of systems engineering.

METHODS

‘Emergence’, the sensitising concept for analysis, is rooted in two key systems premises: that person and context are inseparable and that what emerges is an act of choice. Via a ‘gallery walk’ we used these premises to perform analysis on individual drawings as well as cross-comparisons of multiple drawings. Our focus was to understand similarities and differences among the vantage points used by multiple surgeons.

RESULTS

In this paper we challenge two assumptions from current models of clinical judgement: that experts hold a fixed and static definition of the problem and that consequently the focus of the expert’s work is on solving the problem. Each situation described by our participants revealed different but complementary perspectives of what a surgical problem might come to be: from concerns about ensuring standard of care, to balancing personal emotions versus care choices, to coordinating resources, and to maintaining control while in the midst of personality clashes.

CONCLUSION

We suggest that it is only at the situation and system level, not at the individual level, that we are able to appreciate the nuances of defining the problem when experts make judgements during real-world complex situations.

INTRODUCTION

Complex patients impose mounting demands on clinicians’ ability to make sound and accurate judgements. Mounting demands occur because making judgements about what’s going on during complex situations becomes a ‘moving target’, as multiple actors with different perspectives change the evolution of the situation and consequently how it is defined. This complexity of how a problem is defined has not been the focus of medical education research efforts; our current knowledge about clinical judgement has mostly been built from exploring how the problem is solved. An exclusive focus on problem solving, as if problems were static, contradicts the nature of complex clinical situations and makes the notion of clinical judgement difficult to translate for workplace training purposes.

Clinical judgement has traditionally focused on problem solving as ‘the assessment of alternatives’ and the ‘selection between alternatives’.¹ Some authors have argued that accruing information is the key task in selecting the right alternative.² These claims support the ‘classical decision-making’ (CDM) model that views the decision maker as acting in a world of complete certainty.³ Other authors have argued that regardless of how much information we gather, there will always exist a level of irreducible uncertainty⁴ and therefore the focus should be on elaborating and improving options, rather than selecting between options.⁵

However, research in other domains^6–8 suggests that judgement during complex situations is centered on problem defining, not problem solving. In complex situations, how the problem is defined affects how experts adapt to what’s going on. Complex situations take place where conditions are constantly in flux, requiring the ability to adapt. Therefore, making judgements during complex situations does not come from extracting options out of a situation (as if an observer) but rather from living within the situation and grappling with the multiple ways in which it could be defined (as if an actor).^9,10 Furthermore, the existence of multiple ways of defining a complex problem suggests that complex problems are dynamic, and posits that iterative solutions do not just get us closer to ‘the’ solution, they actually redefine the problem. This feature is particularly evident in health care, where the multidisciplinary nature of complex clinical situations usually requires multiple individuals negotiating the definition of the problem. However, we currently do not know how experts in these situations go about defining the problem. If we know this, we will be able to better equip trainees with competences for more effective teamwork.

Theoretical framework

Systems concepts can inform approaches to researching expert judgement that include both problem definition and problem solving. Systems thinking is a way of seeing a situation in its entirety, so that we can perceive what enables the situation to be: interrelationships rather than individual parts, patterns of change rather than static snapshots.¹¹ In systems thinking, perspectives (i.e. participants’ interpretations) are responsible for shaping and defining the situation.¹² Perspectives change as the situation evolves, requiring the ability to recognise what emerges. Two premises define the concept of Emergence in systems thinking: that person and context are inseparable and that what emerges is an act of choice. Thus, the definition of the problem changes as the individual considers the situation from different vantage points and interacts with other elements of the system.^13,14 This approach helps to shed light on issues such as the positioning of the expert in relation to the situation and the implications of such positioning in the expert’s definition of the problem. In this paper, we describe different ways in which surgeons construct their definition of the problem from their positioning within the situation.

METHODS

The systems thinking approach incorporates analytical tools from a variety of systems disciplines.¹⁵ In our research, we rely on methods from systems engineering, particularly ‘rich pictures’.^16–19 Rich pictures are pictorial representations that attempt to capture a person’s perspective of a complex situation with all its interacting elements: things, ideas, people, character, feelings, beliefs and conflicts. Because they represent perspectives, rich pictures constitute a way of understanding how individuals see their place and role in relation to the situation. In our previous work we combined rich pictures with interviews to uncover the evolving and emerging non-procedural dimensions of complex and challenging situations in expert practice.^20,21 In this paper, we performed a secondary analysis on our original dataset to explore the notion of ‘problem definition’ in real-world clinical judgement. This secondary analysis employed the analytical steps used in our original study but focused on the systems thinking’s notion of ‘emergence’ as the sensitising concept for analysis. A summary of the data collection and data analysis procedures is provided next.

Data collection

The Research Ethics Board from Western University granted ethical approval to this study. Permission was given for the publication of the quotations and drawings by the participants via an explicit clause in the consent form. The original study involved the participation of five surgeons from different specialties, who each produced three drawings corresponding to three different complex surgical operations, for a total of 15 ‘rich pictures’. Data collection involved a two-stage process for each surgical case: (i) observation in the operating room of complex surgical operations, and (ii) a ‘rich picture’ drawing session about the same surgical operations. Observations (captured in the form of field notes) were conducted in order to get a sense of the environment and context of the particular situation. Observations also allowed us to formulate potential lines of questioning to be used in stage two.

Postoperatively, usually within the following 2 weeks, surgeons participated in a 60-minute drawing-plus-interview session. During this session the concept of a ‘rich picture’ was explained to the surgeons, who were then invited to take approximately 30 minutes to draw their perspective on the complexity of the situation and how it evolved. This drawing session was conducted in the privacy of a meeting room or office with only the researcher, who observed the surgery, and the surgeon present. The researcher left the room while the surgeon was drawing. After approximately 30 minutes the researcher returned to the room and, in order to understand the drawing from the surgeon’s perspective, recorded an interview with the surgeon. The surgeon was asked to describe the drawing and the researcher followed-up on any observations she may have made about the drawing itself or how it related to what she had observed intra-operatively. The discussion that followed frequently elicited new insights and understandings from the surgeon, who was encouraged to add more details to the drawing if he or she thought it necessary. This process was repeated for each of the three cases per surgeon.

Secondary data analysis

Secondary analysis of the dataset was performed in the form of a gallery walk.²² A gallery walk is a technique that promotes active engagement with data.

The drawings were displayed ‘gallery-style’ around a conference room. Two researchers performed the gallery walk by walking around the room, stopping at each drawing, engaging with each drawing for as much time as needed and writing analytical memos for individual drawings as well as for cross-comparison of multiple drawings. Memos were recorded in a custom-designed table that prompted researchers to identify motifs and to offer their interpretations of what those motifs meant individually and in relation to each other as well, as an overall interpretation of the potential stories playing out in each drawing. Our analysis was conducted in three stages.

Initially, we used the first premise of the notion of emergence (person and context are inseparable) to get at the question of: What was the whole case about? We looked at each drawing individually to get a sense of the vantage point used by the surgeon to describe each case. Our interpretation was then complemented with the field notes from the intra-operative observations and the explanation of the drawing given to us by the surgeon during the interview. We repeated the process for the three cases per surgeon.

For each participating surgeon, we performed a cross-comparison of the three drawings guided by the second premise of the notion of emergence (what emerges is an act of choice). Our focus at this stage was to understand similarities and differences among the vantage points used by multiple surgeons.

Finally, three participating surgeons joined us in a ‘return-of-findings^’ session designed to explore their resonance with the analytical results from the previous steps. Return-of-finding refers to follow-up meetings with the participants in which preliminary findings are discussed in order to achieve consensus in the interpretation and analysis of the data.

RESULTS

Complex surgical situations, like the ones we observed, lasted from 6 to 10 hours. How surgeons managed to solve the particular events did not feature centrally in their drawings. What seemed to be more important for them was how they made sense of what’s going on.

Making sense of what’s going on took the surgeons into drawing and describing the multiple vantage points from which they considered the particular surgical operation they drew. In the following case, three vantage points were used (Table 1): problem as procedural decision making (depicted as the ‘enemy MRI’, Fig. 1), problem as dealing with the medical team (depicted as the boxing gloves and big question mark, Fig. 1) and problem as a moral dilemma (depicted as a weight balance, Fig. 1). For labelling purposes we used S# to identify each surgeon and C# to identify each surgical case. For example, S1C2 corresponds to the second case (C2) drawn by surgeon S1.

Table 1.

Descriptions of the three vantage points used by surgeon S1 to describe case C2

Problem as procedural decision making	Problem as dealing with the medical team	Problem as a moral dilemma
‘Well, it’s complex because it was a large tumor; it was involving a number of different structures that need to be removed. It was undetermined exactly whether it was resectable prior, although I had a pretty good idea. It involved two different surgical teams…and those teams had to kind of play back and forth with each other at different points.’ (S1C2)	‘But the issue around that was a back story … because he was someone who does not have a lot of intelligence, the oncologist said “well we’re not going to treat him with chemotherapy before you operate on him because we don’t think he’s reliable”… and I felt very strongly that that was inappropriate because I thought that treating him with less than the standard of care just because he’s not as smart as the next person is a bad idea.’ (S1C2)	‘On one edge, I’ve written ‘‘the patient’’ and on the other edge I’ve got “the hospital, society, me” and the arrows are kind of going up and down ‘cause I kind of see that as teetering … what’s good for the patient may not necessarily be best for society because I’m wasting a lot of money by admitting him … and whether, you know, because it’s the right thing to do for the patient…but is it really the right thing to do for everyone else all involved?’ (S1C2)

Figure 1.

Rich picture of a complex surgical situation (C2) by participant S1. This situation depicted a disagreement between different services on how to provide care for a complex cancer patient along with its consequences

When special patient circumstances put into doubt the ability to ensure a standard of care, the drawing became a recurrent cycle. There is no beginning, no end. Question marks, unbalanced scales, boxing gloves, tombstones and killing knives were independent visual representations of the surgeon’s efforts in trying to understand what’s going on. Moreover, connecting arrows depicted the cyclical nature of the situation. The surgeon as part of the cycle appeared trapped in the question mark of an uncertain operating room, or as depicted in other cases, alone on an island or under the spotlight as he tried to compensate for social, institutional and personal factors. So what emerges for this particular surgeon during his cases are cycles of futility, particularly when dealing with terminal or mentally ill patients.

And then, I really started to get into this thing where sometimes we see this in these patients, where we kind of get into almost a futile cycle where we operate on them… they have a complication… and then, we have to go back to the operating room. You do this cycle a few times where it’s fine until finally someone kicks in and realises that this is futile. (S1C1)

When the situation is coloured by the surgeon’s personal health issues, balancing choices and emotions becomes somewhat surreal. The central theme of one surgeon’s drawing was himself as a child being picked up by an ambulance during a soccer game because his heart was failing (Fig. 2). Determining what’s going on in this situation became a quest between the past and the present. According to the surgical culture, surgeons should leave their emotions outside the door, but according to this surgeon, it only takes one case touching you at a deeper level to realise that such a quest exists for surgeons more often than not. In one of the cases depicted by this surgeon, images of the surgeon’s personal experiences blended with images of the patient’s issues, suggesting that what emerged for him was a balancing act that involved negotiating between what the situation afforded and what his emotions dictated.

This is the pathway to him getting better and getting back to the things he really loves… his daughter, football, hockey… This over here is me. This is me thinking about…he’s a young guy and so. Anyways, when I was younger, I had major heart problems. This is me being defibrillated back into regular rhythm… and this is me at a typical sports type thing where I couldn’t do anything and occasionally had to get called into an ambulance. So, knowing that, how much I can empathise with him and his needs… (S5C3)

Figure 2.

Rich picture of a complex surgical situation (C3) by participant S5. This situation depicted the impact that this particular patient had on the surgeon as he remembered having to suffer from the same illness

For other surgeons, their definition of the problem required being aware and internally negotiating other team members’ sense of the problem. According to one surgeon’s drawing, determining what’s going on was like ‘putting together pieces of a puzzle’. Those pieces, which appeared consistently in the drawing (Fig. 3), were depicted inside the operating room, outside in the waiting room and beyond the frontiers of two countries. According to this participating surgeon, ‘every actor has a piece and a key to the piece’, suggesting that what emerged for him was an act of coordination (like ‘juggling balls in the air’), as the same surgeon depicted in another of his drawings, between what the piece means and when and how to bring it in.

There are a lot of keys to the conduct of the operation … imaging system, the nurses, the industry reps, the engineers in Atlanta, the anaesthetist, myself and Dr X interplay back and forth and doing things together and a million steps that I’m trying to think of as we go through … I thought of what was the best picture to relay what the whole case is about and I put these pieces of a puzzle and then I figure the case is actually putting the pieces now together… you know you take all the pieces and you put them together in a bottle but it’s actually harder than that because it’s putting a ship together in a bottle but you can’t see the bottle… (S2C3)

Figure 3.

Rich picture of a complex surgical situation (C3) by participant S2. This situation depicted the multitude of people that the surgeon had to coordinate intra-operatively during a complex aneurysm surgery

Even though for our participant S2 above, every team member holds a ‘piece of the puzzle’, according to participant S3, ‘sometimes not all the team members come prepare[d] with the right piece’. In surgery, things need to keep moving. So when pieces of the puzzle do not fit, then team dynamics can veer to resemble episodes of the surgery in the form of cyclonic weather events, as this surgeon drew using multiple, dashed circular lines (Fig. 4). Adding team interpersonal issues to an already complex case completely changed the situation. What emerged was a team problem that challenged the surgeon’s ability to maintain control: a clash of personalities that made for ‘heavy sweating surgeons’, as depicted in the drawing.

Actually, when you [research assistant] fainted, myself, the green but keen first assistant Dr X and as I recall, the circulating nurse, so three individuals went to help you. You were able to get up and go onto this stretcher but then you didn’t look quite good so we got the porter and he got you down to Emergency. The surgery is sort of starting with negative gestalt. I’m sweating because I’m worried about you as well as the patient. Also I knew there would be [procedural] issues… And then, with the bickering between two team members, there was need to keep the peace and get people focused into a complex surgery. (S3C4)

Figure 4.

Rich picture of a complex surgical situation (C4) by participant S3. This situation depicted a disruption in the team dynamics provoked by a mistake made by one of the team members

A futile cycle, personal quest, building a puzzle, juggling balls in the air and dealing with cyclonic weather were some of the visual representations that surgeons used to answer the question of what emerges during complex operations. These stories portrayed what was going on for these surgeons while attempting to define the problems they were part of: ensuring standard of care independent of the patient’s circumstances, coordinating multiple and dispersing resources, balancing care choices with emotional impact, and maintaining control in spite of personality clashes.

DISCUSSION

Using the notion of emergence in this paper we challenged two assumptions from current models of expert clinical judgement: that experts hold a fixed and static definition of the problem and that the focus of the expert’s work is on solving the problem. Each situation described by our participants revealed different but complementary perspectives across situations and drawings of what a surgical problem might come to be: from concerns about ensuring standard of care, to balancing personal emotions versus care choices, to coordinating resources, to maintaining control while in the midst of personality clashes. Written as a list, those perspectives might suggest solution-seeking behaviours; however, the visual and verbal descriptions indicated that participants were most concerned with trying to make sense of what’s going on. In doing so, participants remained attentive to what emerged as the situation unfolded from their positioning within the situation.

In the systems literature, emergence has something to do with perceiving and choosing.¹⁰ For instance, if we take individual shapes or parts (two dots, a curve and a circle), we can organise them so that a recognisable whole (an image of a face) emerges. This organised whole of interrelated parts is what we call a system. Thus we choose to attribute new meanings to the shapes because we now perceive them as parts of this system. These new meanings emerge, both individually and collectively, because of their organisation as a system. In this way, what determines whether something is a whole or a part, and how the parts are related, depends on the person who is interested in it (i.e. it is an act of choice, not something ‘out there’ waiting to be discovered).²³ And this act of choice reflects the purpose of the person in thinking about, and giving attention to, the system – the type of activity that our participants engaged in.

If what emerges is an act of choice, we suggest that our study also offers the possibility for an alternative conversation, regarding the art of clinical judgement, in which the act of choice emerges metaphorically. Methodologically, we have suggested using visual methods to complement our attempts to construct meaning in research settings. In our research, we consciously used rich pictures²⁴ and we grounded this decision in the notion that we are constantly constructing meaning that is both visual and verbal. In complex situations, such construction of meaning may involve intuitive perception, which, some authors have argued, can be given form via aesthetics tools: ‘it is the appeal to our senses that enable us to make a value judgment’.²⁵ Although the use of visual data seemed to allow participants to experience a deeper engagement with the situation that enriched the data, the intensity of the data collection and data analysis processes influenced our sampling decisions. As a consequence, the findings of this small-scale study offer but one attempt at describing how the phenomenon of problem definition is perceived by experienced clinicians. Future research could explore the relevance of these findings outside the surgical setting and beyond a single clinician’s perspectives by interviewing multiple health care professionals around the same clinical case.

In conclusion, our study suggests that in complex situations, the judgement process of experienced clinicians seems to mainly revolve around problem definition, not problem solving. The notion of ‘emergence’ helped us appreciate the nuances of defining the problem. Particularly, we found that in trying to define the problem, experienced clinicians portrayed themselves as embedded in the situation. We therefore suggest that it is only at the situation level, not at the individual level, that we are able to foreground the importance of defining the problem when experts make judgements during real-world complex situations. It is at the situation level that experts are able to see themselves as part of constructing the story of what’s going on. In depicting themselves as actors, what emerged were purely their acts of choice.