Resilient Actions in the Diagnostic Process and System Performance

Abstract

Objectives

Systemic issues can adversely affect the diagnostic process. Many system-related barriers can be masked by ‘resilient’ actions of frontline providers (ie, actions supporting the safe delivery of care in the presence of pressures that the system cannot readily adapt to). We explored system barriers and resilient actions of primary care providers (PCPs) in the diagnostic evaluation of cancer.

Methods

We conducted a secondary data analysis of interviews of PCPs involved in diagnostic evaluation of 29 lung and colorectal cancer cases. Cases covered a range of diagnostic timeliness and were analyzed to identify barriers for rapid diagnostic evaluation, and PCPs’ actions involving elements of resilience addressing those barriers. We rated these actions according to whether they were usual or extraordinary for typical PCP work.

Results

Resilient actions and associated barriers were found in 59% of the cases, in all ranges of timeliness, with 40% involving actions rated as beyond typical. Most of the barriers were related to access to specialty services and coordination with patients. Many of the resilient actions involved using additional communication channels to solicit cooperation from other participants in the diagnostic process.

Discussion

Diagnostic evaluation of cancer involves several resilient actions by PCPs targeted at system deficiencies. PCPs’ actions can sometimes mitigate system barriers to diagnosis, and thereby impact the sensitivity of ‘downstream’ measures (eg, delays) in detecting barriers. While resilient actions might enable providers to mitigate system deficiencies in the short run, they can be resource intensive and potentially unsustainable. They complement, rather than substitute for, structural remedies to improve system performance. Measures to detect and fix system performance issues targeted by these resilient actions could facilitate diagnostic safety.

INTRODUCTION

A recent American Medical Association report¹ highlighted the need for further research in understanding and improving outpatient safety, especially in the area of delays in diagnosis. Measurement of system performance related to safety in the outpatient setting is critical to future quality improvement and care delivery efforts.^2;3 Measurement programs must therefore use methods that are sensitive to the large variety of factors that influence safety. These factors⁴ can be organized along a continuum^5;6 from the ‘blunt-end’ latent factors⁷ (which are controlled primarily by the facility’s administration, or are exogenous to the facility)^5;8;9 to the ‘sharp-end’ ¹⁰ active factors⁷ (over which the provider has some direct control).

This continuum is reflected in measurement frameworks proposed for quality and safety aspects of system performance.^5;6;8;11;12 For instance, structural measures assess infrastructure and policy;^11;12 process measures assess how well care delivery activities follow defined processes and policies;^2;12 and outcome measures assess the results of care delivery (including the impact on the health of patients and populations, and system throughput).^5;11 These frameworks illustrate how the quality of care delivered to the patient is a product of the whole system,¹³ not simply the result of activities by the sharp-end providers. Conceptually, this is described as a causal chain, where influence flows downstream from structure to process to outcome,⁴ and the downstream components are dependent on the ones upstream.¹³ However, this downstream propagation of upstream problems is not necessarily direct or simple. When practitioners are regularly confronted with safety risks, they sometimes develop new techniques or adaptations in how they address these challenges.^14;15 In doing so, they fill in the gaps between structural blunt-end factors and the demands of the dynamic situations on the frontlines.^16;17

Practitioners can thus contribute to system ‘resilience’, defined as the ability of a system to change modes of operation in response to changes in risk.^18;19 In complex, dynamic systems, resilience is a fundamental aspect of safety.^20;21 A resilient system is essential for safety in the outpatient setting, where care depends on coordination across multiple settings, and the ability of patients to follow a management plan.²² Coordination breakdowns are not uncommon, especially in the complex interface between primary care and subspecialty care for cancer.^23;24 We previously found diagnostic delays in cancer in a significant proportion of lung and colorectal cancer cases.^25;26 As sources of resilience, frontline providers are crucial to the organization’s ability to cope with potentially disruptive situations and risks.^16-18;27;28

System barriers impede expeditious diagnostic evaluation but might not be easily detected by downstream measures (such as time to diagnosis) because of mitigation by resilient actions of frontline providers.^18;29-31 In this study, we explored barriers to timely diagnostic evaluation of cancer and resilient actions of providers that attempted to mitigate these barriers. Our overall goal is to identify strategies used by providers to adapt to challenges in diagnostic evaluation, and to highlight the need for approaches to better detect system risks. This can improve understanding of the dynamics of safety in outpatient care and lead to identification of promising new directions for improving safety for cancer diagnosis.

METHODS

Setting, participants, and data collection

This study is a secondary analysis of data collected for a larger research study on the safety of the diagnostic process in cancer.³² The primary study identified cases of newly diagnosed lung and colorectal cancer using tumor registry data at two large integrated public healthcare systems. A sample of cases was selected based on types of indicators, then stratified based on timeliness of diagnosis, as determined by medical record review that assessed the time between initial presentation and final pathological diagnosis. Within this, cases were selected for interviews using maximum variation sampling, a technique to cover a wide range of case attributes relative to sample size.³³ See online supplementary appendix for more details.

Semi-structured interviews (audio recorded) were conducted with primary care providers (PCPs) closely involved in the case, and focused on the processes involved in diagnostic evaluation for these specific patients. The patient’s electronic medical record was available during interviews to facilitate recall. Institutional review board approval was obtained for this study.

Analysis

A multidisciplinary research team conducted a two-stage analysis of the interview transcripts to identify instances of providers performing resilient actions. In stage one, we used framework analysis³⁴ to identify barriers and instances of resilient actions in response to those barriers.³¹ The framework to define barriers was based on several factors identified in the literature as potentially detrimental to follow-up^26;35-39 and included the following categories: workload and task management; referral requirements and lead times; and coordination problems with specialty clinics and with patients. The framework for resilient actions was based on models of resilience in the literature.^29;31;40-45 We defined resilient actions as those that addressed an explicit safety barrier for that case³¹ via one or more of these capabilities: monitoring for and recognition of risks; foresight and anticipation; flexibility and the ability to respond; and/or uncertainty management and reflection. We focused only on actions initiated by the provider. Using these frameworks, a cognitive systems engineer member of the research team reviewed each transcript, identifying barriers and then instances of resilient actions in response to those barriers.

To estimate how often resilient actions were used to address barriers, we only counted actions that were mentioned regarding the case under study (not for any other patients mentioned by the provider). Similar instances were grouped together into categories which reflect general ‘resilient’ strategies³⁰ for using resources to facilitate diagnostic follow-up. These results were reviewed by another team member with clinical and human factors expertise.

Stage two was conducted to independently assess each instance in terms of how it related to the normal scope of work of the PCP in the system. Two physicians (both experienced in primary care and familiar with the hospital systems used in the study) reviewed each identified instance of resilient action, including information on the associated barrier and the case. They used a three-point scale to rate the extent to which the action would be considered standard or expected for that situation, where 1 meant the action was typical and expected; 2 meant that it involved somewhat more than the standard level of accommodation or effort; and 3 meant that the behavior was above and beyond the usual and expected (irrespective of whether the action was considered useful). The mean of the ratings was calculated for each instance of resilient action. In cases of disagreement by more than one point, a rating from a third physician was incorporated.

RESULTS

Twenty-six PCPs (22 physicians, 3 physician assistants, and 1 nurse practitioner) were interviewed; they had an average of 13 years’ experience in practice (SD=5.4), and 7 years at that facility (SD=9.3). Twenty-nine cases were analyzed: 17 colorectal and 12 lung cancers. The time from presentation of indicator to final pathological diagnosis ranged from 6 days to over 5 years (median=16 weeks, SD=85 weeks) for colorectal cases, and from 15 days to over 4 months for lung cases (median=7 weeks, SD= 5 weeks).

Most of the barriers identified were related to access to specialty services and coordination with the patient (ie, patient adherence). Barriers regarding referral requirements and lead times were related to 12 instances occurring in nine cases. Other barriers involving coordination with a specialty clinic were present in three instances (within two cases). Barriers of coordination with patients were related to 13 instances occurring in seven cases. Barriers regarding workload and task management were related to three instances occurring in three cases.

We identified 31 instances of resilient actions, ranging from 0 to 5 instances per case. In 59% of the cases (17 of 29) at least one resilient action was performed in an attempt to address a barrier. The mean physician ratings of the instances indicated many exceeded what would have been considered the standard level of effort (see table 1). In 40% of the cases (11 of 29) there was at least one instance rated 2.0 or greater. As shown in Figures 1 and 2, the resilient actions (and therefore the barriers to which they were in response) were present in cases with variable range of timeliness in diagnosis. Thus, cases with more timely diagnostic evaluation were not necessarily free from barriers.

Table 1.

Distribution of Instances across Mean Physician Rating

Mean Physician Rating		Number of Instances
1.0	(typical)	10
1.5		8
2.0	(somewhat more than standard)	6
2.5		5
3.0	(above and beyond)	2
	Total	31

Figure 1.

Instances of Resilient Actions for CRC Cases

Figure 2.

Instances of Resilient Actions for Lung Cases

The intraclass correlation coefficient⁴⁶ between the two raters is 0.527. Only three instances required the use of the third rater.

The 31 instances of resilient action are listed below (along with the mean physician rating of that instance), categorized by the resilience strategy they are instantiating and the barrier type they are in response to. Many of the actions illustrate strategies in dealing with patients or other parties (eg, specialists, clinic staff) in order to ensure timely diagnostic evaluation.

Strategy: using alternate pathways to achieve goal (obtain diagnostic testing)

Barrier: referral requirements and lead times

• A patient was found to have a suspicious colon mass on her CT scan, a positive FOBT (fecal occult blood test), and weight loss. The PCP determined that a colonoscopy was needed, but the wait time was a few months. The PCP used close colleagues in family practice service to have the patient admitted to the hospital in order to get a colonoscopy much sooner. (Mean rating of 3)

• A patient had a history of hemorrhoids and was found to have a positive FOBT. The PCP referred the patient to the rectal clinic, but maintained a suspicion that the problem may not be due to hemorrhoids. The PCP anticipated that the rectal clinic would be able to obtain a colonoscopy faster than would the PCP. (2.5)

Strategy: making pre-emptive requests (referrals)

Barrier: referral requirements and lead times

• To expedite a specialty visit, a provider ordered a pulmonary consult after an abnormal chest X-ray result, without waiting until a pending chest CT scan was completed. (2)

• Because of the high-risk factors for lung cancer, a provider ordered a pulmonary consult at the same time as ordering the CT scan (instead of waiting for the results).(1)

Strategy: contacting resource directly to try to get faster service

Barrier: referral requirements and lead times

• To obtain a faster colonoscopy appointment for a patient, the provider personally called the medical coordinator at the gastrointestinal (GI) clinic and requested expedited procedure. (2.5)

• The patient, an older woman with iron deficiency anemia, needed a colonoscopy, but there was a long wait time. To get her seen faster, the provider personally called the GI clinic to expedite the process. (2.5)

• To get a more rapid colonoscopy, the provider called GI. (2.5)

• To obtain a faster pulmonology consult, the PCP called the department directly (this PCP knew some of the attending physicians there). (2.5)

• A patient was experiencing chest pain and cough, but there was only one X-ray machine at this satellite clinic and appointment slots were sparse. Because the radiology department knew the provider only requests X-rays when really necessary (no annual screening X-rays), a slot was provided quickly. (1.5)

• In the reason field for an ASAP CT scan order request, a provider included the fact that the radiologist recommended a CT. (1)

• A patient had a positive FOBT and the provider determined that a colonoscopy was necessary. During an emergency department visit for abdominal distress, the patient’s abdominal CT scan result suggested a colon mass. The PCP included the CT scan results in the referral request for a colonoscopy to try to persuade the GI clinic to provide an earlier date for the procedure. (1)

Strategy: implementing methods to prevent needed follow-up being over-looked

Barrier: coordination with specialty clinics/resources

• A PCP, who does not usually get any information from the specialists about whether the patient is given an appointment or not, asked his patients to contact him if after a month they still have not received an appointment with the specialty clinic. (2)

• To stay informed about whether patients are getting diagnostic appointments or not, a provider scheduled visits with the patients every few months. (1.5)

Barrier: workload and task management

• A provider was going on sick leave for several weeks. In a patient’s notes, the provider included plans for the covering provider for what to do if the follow-up chest X-ray is abnormal. (2)

• To keep track of follow-up actions that a patient needs, a provider used post-it notes on the office wall as reminders. (2)

• To keep track of follow-up actions that a patient needs, a provider wrote addendums in the patient’s electronic chart, leaving the note unsigned to ensure it remained active and served as a reminder in the future. (1.5)

Barrier: coordination with patient

• A patient, with a history of successfully treated lung cancer, received an abnormal CT almost 2 years ago, but no follow-up studies. A new provider sifted through many years of medical records to identify that there was an abnormal CT that never received followed-up. (1)

Strategy: utilizing access in obtaining collaboration (patient adherence)

Barrier: coordination with patient

• A patient with a poor history of adherence with appointments was overdue for a CT scan. After his appointment, the PCP asked the nurse to escort him to the radiology department. (3)

• A patient had sometimes not shown up for appointments, but needed some lab work done (iron studies). The provider had established the habit of getting the lab drawn the same day to facilitate patient follow-up. (1.5)

• An older patient was hard of hearing. To review the patient’s chest X-ray with the patient face to face (as opposed to over the phone), the provider asked the patient to wait around until the X-ray was read later that day. (1.5)

• A patient called with a medication renewal request. The PCP saw that the patient had not been seen in a long time, and required him to come in for a visit. (1)

Strategy: recruiting other stakeholders (caregivers)

Barrier: coordination with patient

• Because the patient could experience some confusion during the pulmonary consult, the PCP called the patient’s family to make sure a family member could accompany the patient. (2)

• Because a patient was elderly and may have needed some assistance, the PCP called multiple times to reach the patient’s daughter and asked her to help him in getting a CT scan appointment (normally done by the patients themselves). (1)

Strategy: contacting collaborator (patient) about necessary actions

Barrier: coordination with patient

• A patient had a positive FOBT, but there were difficulties getting in touch with him. The office tried calling, mailing a letter, and mailing a certified letter to the patient. (1.5)

• A patient with an abnormal chest X-ray did not show up for the follow-up CT scan. The PCP was alerted to the no-show and called the patient to make sure he gets the CT scan. (1.5)

• A patient’s CT scan was abnormal. The PCP’s office tried to call the patient multiple times but was unsuccessful in reaching him, so they sent him a letter. (1)

Barrier: referral requirements and lead times

• Over 1 year ago, a patient was found to have a high risk for developing lung problems, but the patient had not received an X-ray until his return to this provider. The provider ordered an X-ray, which showed an abnormality. The provider called the patient the very next day to schedule a CT scan (instead of scheduling an appointment to discuss it) (1.5)

Strategy: guiding collaborator (patient) in the decision-making and planning process

Barrier: coordination with specialty clinics/resources

• To facilitate the coordination between a patient and the gastroenterology staff, the provider called the gastroenterology department from the exam room while the patient was still present. (2)

Barrier: coordination with patient

• In anticipation of possible fear and confusion an older and hard-of-hearing patient could experience during a pulmonary visit, his PCP explained the procedures and risks regarding the pulmonary assessment with the patient ahead of time. (1)

• The patient, an older man, had a fatalistic attitude about his health and was leaning against getting the colonoscopy. The PCP spent a large amount of time counseling the patient about how the procedure will help get information so the man could make an informed decision. (1)

• The patient and his wife were in denial about the possibility of the cancer coming back, believing that the diet and herbal supplements he had been using were preventing this. The PCP spent a lot of time with them helping them understand why cancer was a serious possibility. (1)

DISCUSSION

We analyzed interviews of PCPs involved in the diagnostic evaluation of 29 cases of lung or colorectal cancer to identify barriers to timely diagnosis and ‘resilient’ actions of providers aimed at mitigating those barriers. We identified 31 different instances of resilient actions. In 59% of the cases, the provider used at least one instance of resilient action, and in 40% of the cases at least one instance took effort beyond what is normal for that type of situation.

Resilient actions that we identified highlight the different ways in which providers attempt to facilitate safe and timely diagnostic evaluation despite challenges of distributed care in the outpatient setting. The presence of resilience actions in cases that had relatively shorter times to diagnosis demonstrates that completion of diagnostic evaluation in a timely manner does not indicate that a system is free from barriers. Rather, it suggests that in many cases, providers performed extraordinary actions to mitigate barriers and get diagnostic evaluation for their patients faster than what the system would have presumably delivered.

When the provider and/or staff are able to mitigate the barriers, the outcome and downstream process measures do not reflect any of the risks from the barriers and costs involved in addressing them.^27;31;47;48 For example, it is possible that the time to diagnostic evaluation would be the same for case A involving several barriers mitigated by the extraordinary efforts of the provider, and case B where no barriers were present at all.

Our findings suggest that in the context of the diagnostic process, reliance on timeliness of diagnosis as an indicator for system performance is insufficient. For example, among the more prompt cases in this study, we found both straightforward cases with no detected resilient actions, and problematic cases that required multiple actions, including some that went beyond usual and expected efforts. To the extent that outcome and downstream process measures are insensitive to resilient actions occurring in the organization, they are likely underestimating the patient safety risk and overestimating system performance. Such measures do not reflect the barriers that are mitigated via the effort of the providers, nor the costs of the extra expenditures of effort. Problems with diagnostic delays can be better detected by using assessment methods that are sensitive to barriers and mitigation efforts⁴⁹.

Such limitations of downstream measures support the argument that multiple measures are required to obtain a full picture of a system’s performance,^2;6;11 not just those capturing outcomes or only focusing on errors.⁵⁰ While the methods used in the study were developed for research purposes and not ready for use at a large scale, some higher level indicators for systems resilience have been proposed.^51;52 Nevertheless, all of the resilient actions involved extra efforts on the part of the provider and/or the primary care staff. As time pressures and work overload are already a problem with PCPs,^53;54 this might also adversely affect safety. Costs occur even when the resilient action does not succeed, and progress to diagnostic evaluation is slow. In our sample, many of the cases that took a long time for diagnostic evaluation also involved some type of resilient action.

Furthermore, these resilient actions can sometimes place burdens on another part of the system. For example, admitting a patient to the hospital to expedite a colonoscopy was done in the patient’s interest, but this is burdensome to the hospital. This might be cost-effective given the potential costs of delayed cancer treatment, but reliance upon such solutions is likely unsustainable. It illustrates the problems that can occur when adaptations are made without assessing and evaluating the consequences to other parts of the system⁵⁵.

The beneficial impact of one of these resilient actions is largely limited to one step in the care process for one patient. They are not substitutes for needed changes to a system’s structure. The role of resilient actions is to enable providers to deliver safe care in the context of new demands and challenges that the system has not yet been updated to handle.¹⁶ Supporting the human contribution to systems resilience is an essential part of the human factors approach to safety and system performance.⁵⁶ Accounting for resilient strategies in the design of work practices and technologies could lessen the additional effort required of providers as they use resilient actions to mitigate barriers.

More fundamentally, system resilience can be enhanced by supporting the blunt-end administration’s awareness of the new demands and challenges confronting the sharp-end providers. This can lead to better anticipation and adaptation of the system structure to current or anticipated risks^57;58, and prevent the opportunity costs generated when a system problem is temporarily mitigated instead of being escalated to blunt-end management for addressing at a structural level⁴⁸. Diagnostic evaluation for cancer in the outpatient setting is a complex problem that involves many different parties, as seen by the nature of the barriers we identified. Enhancing resilience in this area thus involves multiple parties working together to anticipate risks within a highly distributed system and to adapt system structure accordingly. It also demands a higher degree of dependence on the patient as an active partner.

Many of the strategies seen here are about using a more direct means of communication to switch from the default mode of interaction between parties to a new mode of interaction that can better support the coordination needed to address the system barrier. This corresponds to the principles of how to facilitate resilience in decentralized, distributed systems⁵⁹ like the outpatient setting, which include maintaining common ground (shared knowledge and expectations that facilitate coordination),⁶⁰ and being able to change how and where coordination is managed.^59;61 Our study has several limitations. Because our assessment was focused on accounts provided by the PCP, it addresses only the actions under the control of the PCP and the associated barriers. Because this study is a secondary analysis of previously collected interviews, additional data on factors relevant to the rationale for attempting a resilient action were not collected or incorporated. Although our assessment was informed by chart reviews and the provider’s account of actions, it is possible that some of the resilient actions were performed for reasons independent of actual clinical urgency. To overcome measurement related limitations, we used explicit criteria based on empirical studies in the literature to identify barriers and resilient actions, and incorporated independent assessment of the atypicality of each PCP’s actions. Another limitation is generalizability because our sample is from only two public institutions, and the focus is on a specific type of task (cancer diagnosis) for two particular types of conditions (colorectal and lung cancer). Nonetheless, these results shed light on general challenges of outpatient care, and the role of providers in deflecting the impact of upstream problems. Future research on the decision-making process of providers would need to address factors related to the difficulties or costs of the resilient action for the provider (ie, social resources, staff) and benefit (ie, clinical significance of faster diagnostic evaluation).

CONCLUSION

Our study illustrates and provides evidence on how downstream measures, such as timeliness of diagnosis, could fail to detect system problems. By highlighting the role of resilient actions of PCPs in facilitating timely diagnosis, we are identifying directions for new ways of facilitating diagnostic safety and assessment of system performance in the fragmented outpatient care setting. As healthcare continues to become more specialized and distributed across different types of providers, the role for measures to detect and fix system performance issues targeted by these resilient actions will become increasingly important in facilitating diagnostic safety.