Opening the Black Box: Cognitive Strategies in Family Practice

Abstract

PURPOSE We wanted to describe the cognitive strategies used by family physicians when structuring the decision-making tasks of an outpatient visit.

METHODS This qualitative study used cognitive task analysis, a structured interview method in which a trained interviewer works individually with expert decision makers to capture their stages and elements of information processing.

RESULTS Eighteen family physicians of varying levels of experience participated. Three dominant themes emerged: time pressure, a high degree of variation in task structuring, and varying degrees of task automatization. Based on these data and previous research from the cognitive sciences, we developed a model of novice and expert approaches to decision making in primary care. The model illustrates differences in responses to unexpected opportunity in practice, particularly the expert’s use of attentional surplus (reserve capacity to handle problems) vs the novice’s choice between taking more time or displacing another task.

CONCLUSIONS Family physicians have specific, highly individualized cognitive task-structuring approaches and show the decision behavior features typical of expert decision makers in other fields. This finding places constraints on and suggests useful approaches for improving practice.

INTRODUCTION

Primary care physicians do not deliver important preventive and disease management interventions to patients as consistently as would be optimal.^1,2 Direct practice interventions, such as prompt and reminder systems, have been shown to be able to improve primary care physicians’ delivery of important interventions^3,4 but are little used and are often actively resisted in practice.⁵ Opportunistic delivery, that is, providing desired services at visits for other reasons, is widely regarded as essential to reducing underuse,⁶ although agreement is not universal.⁷ Although opportunistic delivery is effective, especially when supported by computerized reminders,⁸ its acceptance is, in practice, quite limited.⁹ Unfortunately, to date no intervention has been consistently successful in improving primary care decision making.¹⁰ We hypothesize that these decision-making interventions fit poorly with the structure of the decision task in real-world primary care, in part because cognitive strategies used by family physicians vary and are poorly understood.

Our purpose was to explore and characterize cognitive strategies used by family physicians to structure the decision tasks of the outpatient encounter. The conceptual framework guiding this investigation was the body of cognitive psychology literature describing the nature of expert decision making and the predictable influences that becoming expert have on cognitive process.¹¹ Our scope was limited to task-structuring strategies specifically; it was not an attempt to characterize all primary care decision making. Our overarching goal is to develop a foundation of cognitive science and human factors engineering to guide more realistically the design of practice interventions and changes, such as electronic medical records, prompt and reminder systems, and implementation of practice guidelines.

METHODS

Design

Our study was of a qualitative design using cognitive task analysis (CTA) interviewing approaches. CTA is a highly structured qualitative interview process designed to elicit the important components of expert decision making in the real-world environment.^12,13 Examples of its application are wide-ranging, including military command,¹⁴ firefighting,¹⁵ neonatal intensive care nursing,¹⁶ and endoscopic surgery.¹⁷ CTA was developed by Klein Associates (http://www.decisionmaking.com) for developing training materials for novices based on the processes of experts.

CTA is actually a collection of techniques that includes task diagram construction, simulation interviewing, and knowledge auditing.¹³ The task diagram is explicitly intended to obtain structuring information and was our primary mechanism of data collection. The simulation interview and knowledge audit interview were not used. The simulation interview was judged unlikely to capture the wide variety of cues used by primary care physicians—without these cues, their reaction was unlikely to correlate with actual behavior.¹⁸ The knowledge audit interview was judged likely to result in cataloguing the cues used in diagnosis of disease rather than exploring information processing.

Setting

Four clinical sites in the midwestern United States participated in this research. Three sites are community-based family practice clinics associated with an academic medical center. These clinics are staffed by about 1 dozen faculty members and supported by medical assistants, nurses, and clerical staff. Although there are medical students or residents in 2 of these practices, such learners were not included in the study. These sites provide the full spectrum of family practice services for infants, children, and adults of all ages. The fourth site is a clinic in a socioeconomically challenged urban setting that provides a broad range of basic health care support services, such as social work and Supplemental Food Program assistance, and education on a sliding fee scale for young people aged 12 to 21 years and their children. Faculty and residents from the university departments of family medicine, obstetrics and gynecology, and pediatrics staff the clinic.

Participants

We used a purposive sampling strategy for homogeneity of family physician specialty and active clinical practice in a community-based clinical office, and heterogeneity of years in practice and sex.¹⁹ All eligible physicians were contacted through multiple individual e-mails, telephone calls, announcements at departmental meetings, etc. One half of the 36 eligible physicians participated. Among those not participating, 3 were not interviewed because of illness or termination of their position during the study period; the remainder did not respond or were unable to schedule a time for the interview.

Interview Method

The primary investigator (RC) trained in CTA by using an interactive computer-based training kit from Klein Associates. The primary investigator then conducted all interviews. Sample interview questions are presented in Figure 1 ▶.

Figure 1.

Example probe questions used during cognitive task analysis interviews.

In each interview, the interviewer and physician worked one-on-one using a flip chart or chalkboard to develop cooperatively a task diagram of the various steps in which patient information is acquired and incorporated into decision making. Corrections and elaborations were encouraged throughout the interview. For each step, the type of information entering, filtering, processing, goal setting, and other aspects of cognitive processing were obtained using a series of probe questions.

After the task diagram was developed, both more specific and triangulating information was obtained. Questions were asked regarding how issues not directly related to the patient’s reason for visit get placed on the agenda. The example of a diabetic patient in the office for sinusitis who has not had a diabetic eye examination in the past year was often used to make the situation more concrete. By encouraging the physician to look at the process from a different perspective, the additional questioning often exposed things that were missed in the original task diagram.

The interviewer then asked physicians to make a list of helpers and detractors from this process of opportunistic care. That list revealed components of task structure that might otherwise have been overlooked. For example, if a poorly updated problem list was a detractor for the physician, then the problem list was considered to be a necessary component of that physician’s structuring of the cognitive task.

Data Collection

The primary investigator met the physician participant usually in the clinic to facilitate a natural context. Field notes were collected during each interview in the form of the evolving task diagram, which was drawn on a flip chart. The list of helpers and detractors, as well as opportunities for opportunistic care, was developed in the margins of this diagram. After the interview the primary investigator supplemented these field notes with a narrative typed record. These interpretive notes sought to summarize and explain the meaning of jottings recorded on the task diagram.²⁰ An example is presented in the Supplemental Appendix (which can be found online only at http://www.annfammed.org/cgi/content/full/3/2/144/DC1).

Analysis

Each investigator conducted multiple readings and reviews of all the collected data using the process of immersion crystallization to develop a comprehensive list of themes in the data.^21,22 In an iterative fashion during a series of 3 weekly, 2-hour meetings with all the investigators, we developed a comprehensive list of themes and then reduced these themes to a smaller set of thematic areas each with its subthemes.²² Next we reviewed the interview data from the physicians and actively sought disconfirming and confirming examples from these data based on our potential biases from a cultural review of categories.^23–25 Our analysis first identified a list of helpers and detractors to decision making in family medicine. We also found 3 major thematic areas that pervaded the data. By comparing the CTA task diagrams, we found recurring patterns of cognitive task structuring, which we winnowed down to 5. We then triangulated the 3 major themes and variations with the actual CTA diagrams that were developed during the original interviews with the research participants, which facilitated development of a table for comparing the 5 strategies based on their common features. Using these data and previous research from the cognitive sciences,¹¹ we also developed a model of novice and expert approach to decision making in primary care.

Verification

Member checking was conducted in 3 ways. First, the preliminary themes were presented for discussion at a departmental research faculty meeting. The themes elicited lively discussion, but there was widespread agreement among both those who had participated in CTA interviews and those who had not. We also presented our findings to a family medicine audience at a national meeting to assess the external validity of the findings. The identified themes resonated with the audience in their discussion and questions. Lastly member checking was conducted with the participants by e-mailing the main findings to get their feedback. All but 1 (who had left the practice) were contacted, and no reservations were expressed.

RESULTS

Participants were 18 family medicine faculty physicians with from 2 to 23 years of postresidency practice experience. One practiced at the nontraditional clinic; the others practiced at traditional sites. All participants reported their ethnicity as white. Age ranged from 29 to 52 years with a mean age of 40 years. Six participants were women and 12 were men.

Our initial review of the data led to the generation of a list of helpers and detractors to opportunistic care (Table 1 ▶). Although some elements were common across many physicians, at least 1 physician could usually be found who felt differently about any item. For example, some physicians saw a diabetic flow sheet as a wonderful helper, but others saw it as a bothersome additional bit of paperwork. One physician’s helper may be another’s detractor.

Table 1.

Helpers and Detractors of Opportunistic Care

Helpers	Detractors
Particular components of the chart	Disorganized, unavailable, or poorly fitting information sources
Organized, up-to-date, and clear records
Patient prompting	Time constraints
Good support from medical assistant	Vague, complex, long, or acute patient complaints
Time
Familiarity with the patient	Other demands on physician
Prompt and reminder system	Patient desire to discuss
Receptive patient	Noncontinuous care
Simple problems	Patient noncompliance
Cues in patient appearance
Physical environment
Recent educational experience
Potential regret

Note: both lists ranked in decreasing order of frequency of spontaneous mention.

By further reducing the initial themes, we identified 3 main thematic areas: the importance and pervasiveness of time pressure in task structuring, the high degree of variation in strategy, and the characteristic features of expert decision making, especially the automaticity and attentional surplus exhibited by expert family physicians. Automaticity refers to the effect achieved by years of practice and expertise wherein basic elements of the task are performed largely without conscious awareness, and attentional surplus refers to reserve capacity to handle additional problems in a given decision-making setting—in this case, during the patient encounter.

Time Pressure

Time pressure and the need to manage time emerged as the most common theme. All the traditional clinic physicians brought up time pressure as an important factor in their task structuring, either spontaneously or in response to the helpers and detractors line of questioning. The helpers include mechanisms by which time can be managed to enhance opportunistic care, and detractors reflect mechanisms that deter opportunistic care. Time pressure affected how physicians set priorities for individual patients and planed patient encounters from the start of the clinic day. The importance of time pressure was highlighted when contrasting the constraints of the traditional clinic with the constraints of the nontraditional clinic. Because the nontraditional clinic physician does not know when a patient is going to receive health care again, the priority is to cover as much as possible in a loose, flexible task structure, spending hours with the patient, if necessary, to make sure that everything is done.

Variation in Task-Structuring Strategies

The second theme was the widely varying strategies physicians used for structuring the encounter task under the constraints of time pressure. For illustrative purposes, 5 strategies are summarized in Table 2 ▶. For example, before clinic starts, a physician using the sticky note strategy prepares a sticky note for each patient based on the patient’s chief complaint that appears on the schedule and on information in the electronic medical record (Figure 2 ▶). This note serves to remind the physician of everything that needs to be asked, checked, and so on, for the patient’s complaint. This physician believes that only the patient’s immediate concerns can be addressed in the standard encounter and has set up this system to be sure other issues are covered in a regularly scheduled health maintenance examination.

Table 2.

Variation Among Task-Structuring Strategies

Strategy	Degree of Structure	Primary Source of Information	Key Information Acquisition Point(s)	Mechanism for Conducting Encounter	Management of Encounter	Time Orientation
Sticky note	High	Electronic medical record	Before clinic, beginning of the day, plan is made for the patient encounter based on the chief complaint.	Using sticky note list, previsit plan is instituted	Previsit plan is implemented; sticks to the recorded chief complaint and physician-planned list of issues	Regimented and stays on time
Template	Medium	Written templates	Continuous	Physician maintains binder with management templates based on age, sex, and disease status	Follows anticipated template	Regimented and stays on time
Familiar and fast	Low	Physician memory	During patient encounter from the patient	Mental processing using own memory and input from the patient	Attentional surplus used to address problems of limited nature, reschedule for complex issues	Rapid processing, with flexibility in agenda, but stays on time
Doorstep planning	Medium	Chart	Before patient encounter	Mental tally of agenda is organized	Patient’s complaints are first elicited and than addressed	Clock is a guide, goes over- time if particularly important patient care is needed
			Scan chart outside the examination room
Now or never	Low	All available sources eg, chart, WIC, social worker, clinic nurse, family or friend, etc	Continuous	Iterative list of needs based on information gathered before and during the encounter	Proactive with expectation for comprehensively addressing care needs	Flexible with no time constraints, though time intensive for the patient

WIC = Special Supplemental Nutrition Program for Women, Infants, and Children.

Figure 2.

Sticky note strategy.

Contrast this strategy with the doorstep-planning strategy, in which a physician constructs an explicit plan to bring out issues not related to the patient’s chief complaint. In the same clinic and under the same time pressure, these 2 physicians responded with highly distinct task structures. We found meaningful variations in several aspects of task structuring, as detailed in the column headings of Table 2 ▶.

Family Physicians as Expert Decision Makers

The third theme that emerged was the varying levels at which physicians exhibited cognitive behaviors that have been established in the literature as associated with novice or expert decision making. As would be expected,¹¹ interviews with more experienced physicians resulted in considerably less detailed information than interviews with less experienced physicians. The most experienced physicians’ cognitive task diagrams reflected not detailed descriptions but extremely general, almost nondescript processes, which in their generality encompassed great flexibility. In several cases, physicians felt frustrated trying to describe their task-structuring processes, which had become so deeply automatized as to be almost unavailable to conscious recall. This flexibility and automatic processing freed these physicians to take advantage of new systems, limited their searches and processes to those essential to the situation, and allowed them to respond well to unplanned situations and opportunities.

Based on our findings and previous work from decision science, we postulated a model of expert and novice decision making in primary care (Figure 3 ▶). This model provides a dichotomous depiction of how expert and novice family physicians vary in how they incorporate opportunistic care. Specific to opportunistic care, the decision-making efficiency of the expert generates an attentional surplus. The expert’s search for information is shorter and is limited to the crucial information. The decision processes of expert decision makers are partially automatized, thus requiring less attention than the decision processes of novices, and leave them additional attention for other tasks. Because the experts’ processes are also more flexible, they are more open to providing opportunistic care and have the attentional surplus to devote to it. This attentional surplus influences the perceived difficulty of performing opportunistic care, such as diabetes care during an acute care visit for an upper respiratory tract infection. Although an expert family physician would have attentional surplus, a novice physician will not. For the novice to address additional health issues or services, additional time will be required, or another activity will be displaced. In reality, physicians fall within a spectrum of attentional surplus, with experts having more and needing less time than novice physician decision makers.

Figure 3.

Model illustrating the dichotomy of expert vs novice decision makers in family medicine.

DISCUSSION

The variety of cognitive task-structuring strategies used by these primary care physicians may explain why practice change strategies typically meet with hit-or-miss success. If the physicians are not absolutely forced to use the intervention, they use or actively ignore it according to how it fits with their particular strategies (Table 2 ▶). For example, the doorstep-planning strategy physician will respond well to additional information provided by the medical assistant just before seeing the patient, because the information is delivered at a key information-acquisition point in this physician’s strategy. The physician using the sticky note strategy is likely to find the same information offered in the same manner to be distracting and actively ignore it, having already completed the plan for the encounter.

Cognitive task-structuring variability across physicians has important implications for the design and implementation of decision support systems and other practice change or guideline implementations efforts. Leading frameworks for practice improvement²⁶ provide a broad overview of the process of changing practice, delineate stages of change, and suggest numerous strategies helpful for those stages. These no-magic-bullet¹⁰ overviews postulate that practice change often fails because too few tools are applied. Our findings suggest, in contrast, that the many tools of practice change, even if correctly chosen, may fail because they are not designed with an adequate understanding of the underlying cognitive science.

Cognitive task structuring may help explain why individually tailored practice interventions to enhance preventive services delivery have been successful.²⁷ As described earlier, a system that delivers a clinical prompt from an electronic medical record during the encounter may change one physician’s decision making but be ignored as disruptive⁵ by others in the same setting; the system fits one’s strategy but clashes with another’s. An incentive that provides motivation to change practice, but requires enormous effort by the physician to change his or her strategy, may be passed up as not worthwhile. Further research into how physicians structure the complex cognitive task of primary care will be necessary. Efforts derived from the findings of human-factors–engineering research could illustrate whether any consistent or predictable results can be expected from anything but the most heavy-handed, compulsory practice change interventions.

The finding that expert family physicians’ cognitive task structuring shows features common among experts in other fields has potentially useful implications. A large body of research from cognitive psychology, with conclusions that are not always intuitive,²⁸ can be used to guide research in improving practice. Expert physicians, as do expert decision makers in other areas, rely heavily upon a rich set of flexible rules and automatic, unconscious decision processes that result in greater speed and efficiency. Cognitive science research and human-factors–engineering approaches that have been used successfully in other disciplines, particularly any involving opportunistic intervention in an established pattern of decision making, should be explored for their potential in increasing the effectiveness of efforts to improve expert decision making in primary care.

Several limitations of this study must be noted. First, our study sample was relatively homogeneous, and it is possible that the range of specific strategies is wider still among a broader spectrum of physicians. Additionally, the research tool did not address the affective and motivational components of decision making, but was confined to the information-related task-structuring aspects. Affect and motivation may influence the information-processing style that decision makers adopt.²⁹ Lastly, the focus was on the most common strategy used by the physician. Given the variable environment encountered in family practice, it is almost certain that these physicians showed varying degrees of flexibility and variation in their strategies as they adapted to the demands of individual situations.

Further research is clearly needed to confirm these findings in other settings, to establish fully the scope of the variation in cognitive strategies, both among physicians and in response to different situations, and to begin to elucidate the manner in which the powerful effect of the practice environment³⁰ interacts with and shapes the individual strategies. The end result suggested by this study is a model of the family physician as an expert decision maker with a highly individual cognitive strategy operating within the constraints of the environment.