Judging Whether a Patient is Actually Improving: More Pitfalls from the Science of Human Perception

Abstract

Fallible human judgment may lead clinicians to make mistakes when assessing whether a patient is improving following treatment. This article provides a narrative review of selected studies in psychology that describe errors that potentially apply when a physician assesses a patient's response to treatment. Comprehension may be distorted by subjective preconceptions (lack of double blinding). Recall may fail through memory lapses (unwanted forgetfulness) and tacit assumptions (automatic imputation). Evaluations may be further compromised due to the effects of random chance (regression to the mean). Expression may be swayed by unjustified overconfidence following conformist groupthink (group polarization). An awareness of these five pitfalls may help clinicians avoid some errors in medical care when determining whether a patient is improving.

INTRODUCTION

Rigorous follow-up is an important yet fallible element for effective medical care. Two of the many mistakes to avoid are incorrectly concluding that a patient is worsening (when the patient is not) or incorrectly concluding that a patient is improving (when the patient is not). The frequency of these two errors is unknown since rigorous data are rarely collected in everyday practice or published science. Some degree of error is inevitable because of the inherent limitations in perception exhibited by patients and reviewed earlier.¹The consequences from fallible patient self-report are hard to predict and can create either an unduly pessimistic or optimistic impression. The net result may lead to abandoning effective treatments (e.g., switching antibiotics when the patient was actually improving) or missed opportunities to discontinue needless treatments (e.g., persisting with acid suppressors when the patient actually had constipation).

Fallible patient self-report is not the only source of error at follow-up. Some medical errors reflect the propagation of mistakes that originate with the patient; for example, if a patient states "my knee pain is better after my arthroscopy", the clinician might be prone to exaggerate the effectiveness of the operation. However, another set of errors is created by the clinician since the professional providing the treatment is often the same person who checks whether the treatment was effective. This type of innate vested interest abounds in clinicians yet would not be accepted in athletes or other professionals.²Case studies suggest, moreover, that self-serving subjectivity is more easily recognized in others rather than oneself ^3,4and that objective conflict-of-interest declarations do not eliminate the problem.⁵

Clinicians may believe that they have reliable judgment about patient outcomes since they practice in an impartial manner. The science of cognitive psychology indicates, however, that human error occurs even without misguided incentives, deviant personalities, or financial conflict-of-interest. That is, fallible professional judgment can arise despite the best of intentions, insight, and integrity. The purpose of this narrative review is to summarize five concepts from psychological science that are standard in psychology textbooks and that might inform judgments made by clinicians who assess patients at follow-up (Table 1). We focus on specific pitfalls that have counterintuitive features, more than 500 citations on PsycINFO, relevance to health, yet rarely appear in standard medical textbooks or MEDLINE searches.

Table 1.

Avoiding Errors When Checking Patients at Follow-up

Error	Example	Solution	Example
Lack of double blinding	Doctor: “At a glance I can tell you’re better.”	Try to stay impartial	Doctor: “Are you feeling better, worse, or the same?”
Unwanted forgetfulness	Doctor: “What was your glucose before the diet?”	Use diaries and careful records	Doctor: “Record your glucose values for our next visit”
Automatic imputation	Doctor: “I’ve only seen patients do well with this.”	Be scrupulous about missing information	Doctor: “I ask my nurse to call every patient afterwards.”
Regression to the mean	Doctor: “Why didn’t the next surgery go as well?”	Anticipate repeated events as less extreme	Doctor: “Your recovery was exceptional but it may be less impressive next.”
Group polarization	Doctor: “She’s better.” Team: “We all agree.”	Encourage differing opinions	Doctor: “Set me straight if I’m wrong.” Team: “Some of us are not sure.”

Lack of Double Blinding

A lack of double blinding is an easily understood pitfall for clinicians because it is the counterpart of the placebo response for patients. The core issue is that preconceptions on the part of an evaluator can cause a participant to behave in ways that subtly reinforce those beliefs.⁶Telling teachers that their class is enriched with gifted students, for example, somehow leads to more gains in scholastic achievement than the average class during the same time interval.⁷Similarly, a psychiatrist would need almost super-human objectivity to check whether intense psychotherapy sessions improved the patient beyond the effects of standard treatment alone. Arguably, a degree of positive self-belief may be indispensable for sustaining a career during difficult times where patients are terminal and treatments are generally ineffective.

One classic demonstration about double blinding involved an elaborate study of young scientists who attempted to train genetically identical albino rats to run through a simple maze for a food pellet reward.⁸By random assignment, half the scientists were told that they had especially bright rats whereas the other half were told they had relatively dull rats. After training, each scientist initiated ten testing trials for their rat and recorded the number of successful completions of the maze. In accord with investigator bias, scientists assigned bright rats reported more successes on average than scientists assigned dull rats (2.3 vs. 1.5, p = 0.01). These results are especially interesting since the young scientists had no semblance of a financial conflict-of-interest and had received standardized instructions on the importance of scientific rigor.

Double blinding is an effective method for eliminating the conscious and subconscious distortions related to preconceptions in clinical science.⁹However, double blinding is not likely to become a major element in mainstream medicine since physicians need to know the details about individual patient treatments.¹⁰The major problems occur, perhaps, when a lack of double blinding is coupled to added intellectual traps such as confirmation bias.^11,12One corrective strategy is to invite second opinions from an impartial colleague.^13–15Group practices with patient hand-offs, in theory, might also help attenuate this bias if different physicians have different preconceptions.¹⁶Finally, formal third-party report-cards of patient outcomes may provide some impartial benchmarks to calibrate frequent, objective, and clinically important outcomes.¹⁷

Unwanted Forgetfulness

The fallible nature of the clinician's own memory may lead to further mistakes when evaluating a patient’s response to treatment. Clinicians sometimes forget simple items such as where they have parked their own car, yet recalling a medical patient is substantially more difficult because each patient has many features.^18,19The cognitive demands at follow-up become even more difficult due to the requirement for making paired comparisons (akin to remembering where a car was parked both today and last month) and the need for managing more than one patient (analogous to being a valet and remembering multiple cars parked on multiple different days). No wonder, for example, that assessing improvements in a patient’s rash might be difficult when running a follow-up clinic for patients with acne or psoriasis.²⁰

The field of memory science is rife with studies of fallibility, including one clever demonstration involving highly experienced professional air traffic controllers tested in their domain of expertise.²¹The basic task was to review a dynamic air traffic pattern presented on a standard instrument display panel and provide flight instructions to individual aircraft. The air traffic controllers were then questioned at random points and asked to recall the position and altitude of 5 designated aircraft from a field of about 13 at the time. The main finding was that air traffic controllers made many mistakes, particularly when trying to recall numerical altitudes rather than geographic positions (24 % vs. 16 %, p < 0.005). When surveyed, the air traffic controllers rated altitude and position as the two most important pieces of data for a safe recommendation.

One way to avoid unwanted forgetfulness is to maintain careful records and computerized reminders.^22,23Doing so can be laborious, cumbersome, and necessitates a reliable retrieval system.²⁴Automatic recording systems can be helpful, such as glucometers with built-in memory chips and cameras with digital images that can retrieve years of past data.²⁵Communication strategies that include electronic messaging can also provide an unambiguous method to return long afterwards to check what was and was not mentioned.^26,27Another corrective strategy is to write down during the initial patient contact the specific expectations that are intended at the time of subsequent follow-up.²⁸A final strategy is to foreshadow in dialogue with the patient the specific questions most likely to be asked at the next follow-up appointment.²⁹

Automatic Imputation

Patient follow-up is also marked by a degree of attrition whereby some doctors and patients do not meet for a second contact.³⁰Automatic imputation describes the widespread presumption that such missing observations are generally normal, unremarkable, and reasonably disregarded. In an emergency department, a clinician’s natural tendency might be to overlook the lack of a patient repeat visit, presume that all is fine, and believe that treatment was effective.³¹In a stroke unit, a medical intern might similarly be aware of a patient’s neurological deficits, presume that little further improvement is possible, and underestimate how much recovery occurs following a year of rehabilitation. In both cases, the gaps in hand-offs and discontinuities of care can promote an unduly optimistic or pessimistic impression of a patient’s outcome.³²

Experiments in Gestalt psychology provide remarkable examples of such unconscious automatic imputation. A classic demonstration involves optical illusions presented for about 2 minutes to university students on stimulus cards, such as a Kanizsa image (Fig. 1). After a small delay, participants are then asked to describe what they had seen.³³As expected, about 70 %–90 % of participants report seeing a white equilateral triangle in the Kanizsa image even though no such shape is actually present. The reason why people are misled is that a simple white triangle would nicely account for what is present and what is absent in an otherwise complex presentation. Apparently, human cognition can create false information even when distortions due to communication and memory are almost completely eliminated.

Figure 1.

Kanizsa figure containing an array of three black chevrons and three incomplete circles. Most observers see a white equilateral triangle that seems to float in front of the six black shapes on the page, although no such triangle is actually present.

The way to avoid automatic imputation is to be meticulous about detecting and correcting missing information. Sometimes the gap is an identified patient with a missing piece of data; for example, a pre-operative diabetic patient with a normal morning blood sugar, a delay in surgery, and a missing-but-mistakenly-presumed-normal evening blood sugar (that is actually quite low).³⁴At other times, the gap is that the entire patient is missing; for example, a patient who is discharged uneventfully after an asthma exacerbation but who is subsequently readmitted to another hospital for an asthma relapse. In medicine, wishful thinking predisposes clinicians to overlook both types of lapses.³⁵One promise of computerized medical records with integrated decision support is to mitigate such gaps though automatic monitoring and alerts.

Regression to the Mean

Perfect data would not resolve all problems related to follow-up assessments because of the additional bias of regression-to-the-mean, defined as the statistical tendency for extreme observations to subsequently attenuate due to the laws of probability.³⁶The counter-intuitive nature of this principle has contributed to the widespread popularity of futile remedies for centuries. The core problem arises because patients seek treatment when sick and clinicians respond with interventions to reduce suffering even though many illnesses are self-limited. At follow-up, both patient and clinician may attribute an observed recovery to the intervening treatment rather than the natural course of disease. Such faulty reasoning likely underlies the earlier popularity of many antiquated treatments including skull trepanation for migraine headaches, blood letting for pneumonia, and massive doses of vitamins for the common cold.

One demonstration of people's failure to account for regression-to-the-mean involves graduate students analyzing the flight instructor paradox.³⁷In essence, the paradox describes how pilots sometimes make an exemplary landing, receive praise, and then perform worse on their next flight.³⁸Similarly, pilots sometimes make a poor landing, receive criticism, and improve on their subsequent flight. A naïve interpretation might mistakenly conclude that positive reinforcement leads to complacency whereas negative reinforcement leads to diligence (contrary to learning theory). In the laboratory demonstration, sophisticated participants shown this paradox generally provided explanations based on mistaken beliefs about learning theory and no participant offered an explanation that mentioned regression-to-the-mean.

Regression-to-the-mean persists in medical care even if treatments are initiated on an elective basis.³⁹For example, patients who undergo bilateral carpal tunnel surgery often notice that the second operation does not go as well as the first operation.⁴⁰The disappointing outcomes might be mistakenly attributed to a worsening of patient disease or a decrease in quality of care. Another explanation, however, is that patients willing and able to undergo a second operation are an elite subset of patients who experienced above-average outcomes following initial surgery (and thus destined to do less well, on average, the next time). The complete solution to regression-to-the-mean would require infallible treatments; in the interim, clinicians need to continually remind themselves about the pervasive element of uncertainty influencing outcomes.^41–43

Group Polarization

The assessment of a patient’s course is often conducted as a group process; however, the presumed wisdom of the crowd is no panacea without independent thinking. Group polarization is defined as the unwanted tendency for people who share similar attitudes to become entrenched following mutual discussion. Some remarkable blunders in military combat have occurred when a leader is surrounded by sycophants or a mob mentality that leaves no room for dissenting opinions (which has, in the past, resulted in thousands of deaths).⁴⁴Similarly, students in naturopathic colleges can sometimes become increasingly distrustful of conventional medical vaccinations even though vaccines are not a part of their formal curriculum.⁴⁵Studies in psychology suggest, furthermore, that group polarization can also arise in repeated everyday tasks.

One controlled demonstration of group polarization involved college students playing small stakes blackjack in a casino like setting.⁴⁶By random assignment, some students played 20 rounds as isolated individuals with no dialogue between players. Other students played 20 rounds in a group setting where a consensus determined how much to wager on each game. The main finding was that the average wager increased by about 50 % following group dialogue compared to isolated performance (51 cents vs. 33 cents, p < 0.005). The reason behind this shift is that social dialogue is sometimes skewed by a few vocal, eloquent, or exceptional participants. Apparently, achieving a consensus among those with no strong prior beliefs does not always yield a simple average and can, instead, cause shifts in risk-taking attitudes.

The way to avoid group polarization is to start with sufficient diversity among the members so that errors are more likely to be canceled than reinforced.^47–49The optimal size of the group is likely a compromise of many factors, although some research suggests that even two added individuals are an improvement over one solitary judge.⁵⁰The best means of achieving a group consensus has not been established, and the same research suggests that forcing a consensus is not always necessary.⁵¹The tradition of scientific peer review is an implicit effort to avoid group polarization since the number of journal reviewers for a medical article is often far fewer than the number of authors listed on the article. A tradition of training in diverse locations is an analogous countermeasure as it allows the clinician to better distinguish truth from local opinion.⁵²

CONCLUSION

This article reviews five concepts from psychology that are relevant when a clinician is checking a patient’s response to treatment (Table 1). In many cases the patient's change is blatant and small fallibilities in judgment will not lead to faulty decisions. In other cases, however, the situation is uncertain and skilled judgment is crucial. Patients will prize a clinician who can reach the right decision in a swift manner. An awareness of specific patterns of mistakes might lead to better clinical outcomes and fewer complications in follow-up care. In contrast, some of the most difficult pitfalls to avoid are the ones that people do not recognize. Each of the pitfalls reviewed in this article has at least one solution that can be applied if clinicians are aware of their own fallibilities and plan to see patients following treatment.

The largest limitation of our review is that it is not a systematic review of all psychology research. The field is broad and variable, so that a formal meta-analysis would fail simple tests for heterogeneity. We selected concepts that have stood the test of time and thereby did not include recent findings that have early enthusiasm but not widespread replication. Our selective approach, moreover, was constrained so that only selected examples appeared for the underlying chain of reasoning characterized as comprehension, recall, evaluation, and expression. We focused on counter-intuitive concepts relevant to how people perceive changes rather than concepts important when evaluating one patient once. Finally, the dearth of clinical trials suggests the need for more future medical research examining such issues in everyday clinical domains.

Narrative reviews reflect rigorous science yet their synthetic structure involves subjective interpretation. Condensing a century of research into a succinct list for clinicians, therefore, raises countless choices on how many studies to exclude, which clinical analogues to offer, and what potential applications are possible. The current review provides a framework for understanding potential errors in judgment, some language for identifying otherwise nebulous misgivings, and scholarly background on the underlying basic science. The current review does not indicate the frequency of the errors, the effectiveness of potential countermeasures, and the ultimate impact of decision science on improving clinical outcomes. Those important issues need future research and the answers appear nowhere in the medical literature at present.