Bridging the feedback gap: a sociotechnical approach to informing clinicians of patients’ subsequent clinical course and outcomes

Introduction

To improve their diagnosis and management skills, clinicians need consistent, timely and accurate feedback. Feedback helps clinicians become better calibrated, leading to more appropriate clinical decisions. Miscalibration—when clinicians’ confidence in the accuracy of their decisions does not align with their actual accuracy—may lead to overconfidence and diagnostic error.¹ Consistent structured feedback leads to improved outcomes such as accurate diagnosis of acute chest pain,² improved prehospital emergency care³ and lower costs of hospitalisation.⁴

Despite its benefits, significant gaps exist in delivering feedback to clinicians. In particular, clinicians do not consistently receive patient outcome feedback, that is, information on the subsequent clinical course and outcomes of patients that they have diagnosed and treated. For example, emergency department (ED) physicians in Canada reported receiving outcome feedback on only 15% of cases they encountered.⁵ Among US internal medicine residents, 58% reported almost never or only sometimes ultimately learning about their patients’ outcomes.⁶ Scientific knowledge on how to effectively provide clinicians with patient outcome feedback is underdeveloped. In contrast with traditional audit and feedback where clinicians receive aggregated metrics of clinical performance compared against explicit standards,⁷ patient outcome feedback provides clinicians with objective clinical information on their patient’s eventual diagnoses, treatment and clinical course. Rather than conferring black-and-white external judgements, patient outcome feedback provides clinicians with narratives regarding what happened to the patients they treated, which are crucial data for self-evaluation of clinical performance.

Developing effective feedback pathways is difficult. Fragmented healthcare systems with organisational and regulatory barriers, such as seen in the USA, make feedback-related information flow challenging. Even integrated healthcare networks such as the US Veterans Affairs health system⁸ and those in other countries, like the UK, Australia and Canada, only have sporadic disease-specific feedback programmes.⁹ To our knowledge, none of these systems have created universal fail–safe patient outcome feedback pathways to account for the numerous patient transitions between primary and subspecialty care, outpatient and inpatient care, general wards and intensive care units, medical and surgical care, and community hospitals and tertiary referral centres. Creating these feedback pathways presents significant logistical hurdles.

This paper discusses challenges to the development of systems for effective patient outcome feedback and proposes the application of a sociotechnical approach using health information technology (IT) to support the implementation of such systems. The concepts discussed herein are applicable not only to fragmented systems of care but also integrated health systems that plan to leverage the benefits of integration for providing effective clinician feedback.

Barriers to effective clinician feedback

To account for patients moving through a space-time continuum of care, we designate past care as ‘upstream’ and future care as ‘downstream’. Feedback on patient outcomes flows from a downstream source back upstream to a recipient. While we focus on clinicians as feedback recipients, feedback can be generated by any member of the downstream multidisciplinary team who cares for the patient (eg, specialist consultant, inpatient physician, practice colleague, care coordinator, social worker) and may even include the patient¹⁰ (figure 1). This conceptual approach is broad and is an extension of traditional handoffs during common patient transitions.

Figure 1.

Communication pathways for health information technology (IT)-supported patient outcome feedback. A patient initially diagnosed and/or treated by a clinician (upstream) subsequently undergoes care by other clinicians/care teams (downstream) who now have new knowledge of the patient’s clinical course and outcomes. This information is fed back to upstream clinician from a member of the downstream care team and/or the patient. A health IT-supported system facilitates this feedback and additional two-way communication that may result.

Feedback should be accurate, specific and concise, but to be most effective, it should be delivered and displayed in a manner that is timely, consistent, reduces cognitive load and encourages further dialogue.¹¹ Structural, psychological and interpersonal barriers to effective feedback communication exist between clinicians across healthcare settings.^{6 12–15} Some barriers are more common in fragmented healthcare systems while others are universal (table 1).

Table 1.

Barriers to effective clinician–patient outcome feedback that can be addressed by health information technology (IT)

Barriers to feedback	Features of potential health IT solutions
Barriers specific to fragmented healthcare systems
No consistent process for exchange of patient information for purposes of feedback across care settings and healthcare organisations	Allows for automated or semiautomated processes to deliver feedback between care settings and clinician groups
Lack of interoperability between different EHR platforms that hinder information exchange16	Health information exchanges can be leveraged to serve as a common platform between different EHR systems for purposes of feedback creation and delivery
Minimal infrastructure dedicated to sharing patient information across all clinicians involved in the care of a patient17	Health IT underpinning other existing care delivery infrastructure (eg, EHRs) can be adapted for the purpose of sharing patient outcomes
Barriers common to all healthcare systems
Structural barriers
Lack of resources to keep track of appropriate timing for feedback, correct feedback recipients/sources, and clinician contact information6 13 14	Automatically identifies appropriate feedback timeframe, correct feedback recipients, clinician contact information and manner of feedback delivery
Lack of time to conduct follow-up activities (eg, review EHR, seek out accepting clinicians) or provide feedback13 14	Supports a feedback system that is well integrated into clinicians' workflow to minimise effort required in delivering, receiving and processing feedback
Unreliable feedback delivery (eg, sent to the wrong clinician or address, delay with mailed letters)12	Supports electronic delivery of feedback to the correct clinician at the right time; can track delivery and receipt of feedback
Difficulty in delivering feedback on outcomes of individual patients (as opposed to aggregated patient data)12	If linked to existing EHR systems, capable of delivering feedback regarding individual patients
No clear guidelines as to content and manner of providing feedback, resulting in variable quality of communication14	Content and manner of generating and delivering feedback may be standardised to an extent given specific clinical settings or situations
Data security risks when delivering feedback especially to clinicians across organisations14	Can help maintain compliance with patient privacy laws by ensuring that information is received only by the intended recipient and data security is maintained throughout the feedback process
Psychological barriers
Reliance on individual motivations of upstream clinician to seek feedback (eg, clinical uncertainty, personal affinity for patient, concern for patient vulnerability)13	Does not rely on clinician motivations to generate and deliver feedback; since able to deliver feedback for all patients, can provide a more accurate perspective of clinicians' performance. Health IT-supported feedback systems championed by strong leadership promotes a culture that expects clinicians to deliver and anticipate feedback
Expectation of upstream clinician that they will automatically receive feedback if something untoward happens to the patient6	Changes expectations to that of universal feedback to clinicians for all patients, that is, no news is not good news
Perception of downstream clinicians that feedback is not effective or irrelevant to upstream clinicians15	Supports electronic methods to measure impact of feedback on certain behaviours of upstream clinicians (eg, reaccess of EHR after feedback is received); supports methods to collect data regarding upstream clinicians' perceptions of feedback; supports communication between clinicians for further discussion
Interpersonal barriers
Fear of possible conflict/retaliation and damage to professional relationships if negative feedback is delivered14 15	Mitigates conflict by normalising feedback through consistent delivery
Fear of medicolegal risks that may be incurred by upstream clinician14	Health IT-supported feedback systems can become an institutional norm and encourage clinicians to communicate in other ways to deepen mutual trust; communication between clinicians can potentially be incorporated into a quality improvement process, which may confer confidentiality and ‘peer-review privilege’ depending on applicable federal/state laws

EHR, electronic health record.

Decentralised systems of care present structural challenges to providing routine patient outcome feedback to clinicians, mostly because patient care and data are organised in silos lacking the necessary processes and infrastructure for sharing information. Data transfer may be tightly controlled not only by state/institutional regulators but also by developers of health IT systems. Information blocking constitutes practices that interfere with access, exchange or use of electronic health information¹⁶ and can be a significant barrier to clinician feedback. In recent years, efforts have been made in the USA and other developed nations such as countries in the European Union, the UK, Canada, Australia, New Zealand and South Korea to make information systems interoperable across organisations, resulting in the creation of health information exchanges. However, usage and usability of these exchanges remain low internationally.¹⁷ In the USA, electronic health record (EHR) vendors have created networks where different hospital systems using the same EHR can share patient information (eg, Epic Care Everywhere), however these networks have not been specifically used to support systematic patient outcome feedback.

Certain barriers to feedback are common to all healthcare systems including integrated systems,⁸ simply because feedback involves human communication. Obtaining and providing feedback requires time and effort from both upstream clinicians and downstream sources. Most downstream clinicians lack the necessary administrative resources and time to track patients whose outcomes need to be communicated to upstream clinicians, much less to compose and deliver the feedback.⁶ Many clinicians may also fear professional judgement and conflict, be reluctant to be viewed as assigning blame and worried about the discoverability of diagnostic discrepancies which may have medicolegal repercussions. Other clinicians may also perceive that feedback to colleagues is ineffective or irrelevant.¹⁵ Meanwhile, upstream clinicians often feel that the feedback they do receive is suboptimal. Feedback may be delivered to the wrong clinician or may be delivered late, making it difficult to remember clinical decisions.¹² There are also no clear standards for the content, format and manner of feedback delivery appropriate for specific clinical settings, thus feedback may be less effective.¹⁸

A sociotechnical approach using health IT to support effective clinician feedback

The current health IT infrastructure supporting care delivery in many countries can be harnessed to develop and sustain systematic processes to bridge feedback-related gaps between clinicians. In this section, we discuss how health IT, specifically the EHR, can be used to establish a feedback process that can both overcome barriers to feedback and be used to evaluate the feedback system itself. However, harnessing the benefits of health IT in the delivery of consistent patient outcome feedback to clinicians requires accounting for the complex adaptive sociotechnical healthcare system composed of clinician-users, their tasks/workflow, environment, organisations and technology as well the interactions between them. We thus also propose a set of essential sociotechnical considerations that need to be addressed for effective implementation of a health IT-supported feedback process.

Overcoming barriers to feedback using health IT

Health IT facilitates communication among clinicians and has the potential to break down practice silos within and between healthcare organisations. Table 1 shows the many ways in which health IT can address barriers to feedback.

Structural barriers, such as challenges in keeping track of upstream clinicians and their contact information, being reminded when feedback is due, having a standard format and method of communication and maintaining data security, can be addressed directly by health IT, facilitating timely, consistent, clear, accurate and secure feedback communication. Psychological and interpersonal barriers are more difficult to overcome and may not be addressed directly by health IT. However, having a consistent and reliable feedback system will generate learning and may itself be useful in changing the culture around feedback.¹⁹ Receiving feedback in a standard health IT-enabled format containing objective outcome information can help clinicians evaluate their own clinical decisions, promoting self-awareness and learning. Asynchronous health IT-supported feedback can enable clinicians to review patient outcomes privately, reducing the need for direct encounters that may be perceived as confrontational. Among US resident physicians, structured written feedback including diagnosis comparisons outlining reasons for any changes in diagnosis after care transitions was viewed positively and noted to contribute to improving clinical practice and identifying system vulnerabilities.²⁰ Among German primary care physicians, patient outcome feedback resulted in self-reflection and resolve to improve analytical reasoning and change clinical practice.²¹

Consistently relaying patient outcomes to upstream clinicians also eliminates reliance on individual motivations to seek feedback.¹³ If feedback becomes the norm across healthcare settings, that is, ‘this is just the way we do things around here,’ clinicians will communicate more freely in other ways (eg, phone calls and joint case conferences to discuss shared patients), eventually affecting clinician perceptions of feedback and deepening mutual trust. This would reduce fears of professional conflict and medicolegal implications. In a US feedback programme between a tertiary referral paediatric ED and community EDs, the downstream ED worked together with upstream EDs to modify feedback delivery processes to allay fears about medicolegal risks (eg, using secure electronic messaging).²²

Evaluating the feedback process and responding to feedback using health IT

Health IT can also support efforts to evaluate the feedback process itself by tracking and aggregating data on feedback reports created, delivered, received and viewed. These data will help clinicians and administrators determine whether the feedback process is functioning well and whether improvements need to be made. For instance, direct communication between clinicians prompted by feedback reports can be tracked and provide some evidence that feedback reports are being responded to by recipients.

To translate feedback into action and improvements, health IT-supported feedback systems require oversight and collaboration from clinical and administrative leaders on both sides of the feedback loop. A recent international systematic review showed that leaders need to gather and interpret data generated by the feedback process, encourage discussion of feedback information among their multidisciplinary staff, identify data trends that require system-wide improvements, and ensure that any medical errors uncovered are routed through the organisation’s usual safety/quality improvement process.²³ Leaders also need to harness organisational resources to support individual and group efforts to respond to feedback.

Using the EHR to create feedback loops

Although standalone health IT systems for feedback can be developed, a more immediate option is to use existing EHR systems to support feedback. One study used an EHR-generated templated feedback form delivered via the EHR’s messaging function to determine the effect of structured feedback to resident trainees who admitted patients overnight.²⁰ After the intervention, residents indicated improved comfort with sharing feedback and self-efficacy in identifying cognitive biases.

Commonly used EHR platforms can already automate several steps in the feedback process. For example, EHR platforms can: (1) generate patient lists based on prespecified criteria, (2) compile a database of upstream clinicians and their contact information, (3) create templated forms/letters, (4) insert data into forms/letters from patient records, (5) support secure electronic messaging between clinicians, and (6) track delivery/receipt of messages and access of patient records.²⁴ These functions can be readily adapted to implement and evaluate a formal feedback process within a healthcare organisation.

However, orchestrating an effective and sustainable feedback process cannot be accomplished merely by stringing EHR functions together. Downstream and upstream clinicians’ feedback goals and their competing tasks and responsibilities will drive implementation.¹¹ Feedback processes must minimise additional burden by being well integrated into clinicians’ workflow.²³ Healthcare organisations’ EHR systems, their interoperability, and the legal/policy environments in which they operate will also affect implementation and must be carefully considered.

Implementing a feedback process within a sociotechnical health system

To illustrate essential considerations for successful implementation, we use an eight-dimension sociotechnical model for studying health IT developed by authors Sittig and Singh. The eight dimensions include: hardware and software; clinical content, human–computer interface; people; workflow and communication; organisational policies and procedures; external rules, regulations and pressures; and system measurement and monitoring.²⁵ Prior to implementing a health IT-supported feedback system, this model can comprehensively identify barriers that need to be overcome and potential factors to be leveraged for success in each dimension (table 2). This requires partnerships between downstream clinicians and their upstream colleagues early in development to ensure that the design and implementation of the feedback system will fulfil common feedback goals, be well integrated into their workflow, allow for system measurement and monitoring, and comply with internal and external regulations. Collaboration at the outset can also help allay fears of professional judgement and interpersonal conflict by setting shared goals and expectations for a constructive feedback culture.

Table 2.

Considerations in implementing a health information technology-supported feedback system across eight sociotechnical dimensions

Sittig and Singh’s sociotechnical model dimensions25	Potential considerations
Hardware and software	What computing infrastructure is needed to run applications to implement the feedback system?
Clinical content	Who will determine the relevant clinical data to be included in feedback?
	What are the sources of clinical information to be delivered for feedback?
	How will the sources of clinical information be integrated seamlessly and meaningfully?
Human–computer interface	What are the aspects of the feedback system that the clinician-users can interact with?
	What factors need to be considered in designing the interface for clinicians generating feedback and clinicians receiving feedback?
People	Who are the personnel needed to generate and receive feedback?
	Who are the personnel needed to maintain and troubleshoot problems with the computing infrastructure for the feedback system?
Workflow and communication	What are the tasks to be accomplished to generate feedback?
	How can the feedback system be integrated in clinicians' and other stakeholders' respective workflow?
Internal organisational features	What aspects of the physical environment (eg, availability and placement of workstations) may affect delivery and receipt of feedback?
	What institutional policies may affect development and implementation of the feedback process?
External rules, regulations and pressures	What state and federal regulations may affect development and implementation of the feedback process?
System measurement and monitoring	How can effects of the feedback process be measured?
	How can problems with the feedback system be communicated to the necessary personnel?

For successful implementation, it is essential to consider each dimension. Basic hardware and software requirements are available in existing EHR platforms but additional innovations could facilitate more design features. Feedback content needs to be jointly decided by downstream and upstream clinicians by considering the balance between relevant information and resources required to extract and integrate data into a concise and coherent report. Highly specific feedback (eg, details of the clinical course) may be desirable but may be beyond the capabilities of the current IT platform to automatically populate reports and will be labor-intensive for clinicians to generate.

The design of the human–computer interface should be user-friendly; usability will enable clinicians to create, deliver and review feedback in a manner that eases their workload while enhancing the system’s sustainability over time. However, depending on the platform used, not all feedback system functions can be performed automatically by health IT, and thus humans will still be needed to fill gaps in the process. Designers of the feedback system will need to consider details of clinicians’ workflow so that the feedback process can be integrated seamlessly without undue burden to clinicians. Although participating in a systematic feedback process will require clinicians’ time, this is preferable to current inefficient methods and may yield improvements in practice and patient outcomes. Aside from clinicians, buy-in is needed from personnel integral to the smooth functioning of the system such as health IT experts necessary for system maintenance and troubleshooting.

Organisational policies (eg, data-sharing procedures) and external regulations (eg, patient privacy laws) will affect the feedback process; thus, support from administrators and institutional leadership, including legal counsel, will be necessary to establish institutional norms regarding feedback, ensure adherence to all applicable rules, and help assess and mitigate potential legal concerns. Implementing an EHR-supported feedback process across healthcare organisations, especially in fragmented healthcare systems, will also pose additional challenges related to lack of interoperability. Ultimately, clinicians and administrators need to work with informatics experts, EHR vendors and policymakers to solve this problem. In the short term, work-arounds may be possible through established public or private health information exchanges. Finally, strategies are needed to measure how well the system is functioning and its impact on outcomes. Certain metrics can be tracked such as reductions in unnecessary transfers or by conducting periodical surveys of clinician-users to monitor how feedback is used in practice.

Conclusions and future directions

Clinicians require effective feedback on individual patient outcomes to improve their diagnosis and management skills, but systems to support this type of feedback are underdeveloped. Health IT has the potential to overcome barriers to effective feedback in both fragmented and integrated healthcare systems. For health IT-supported patient outcome feedback systems to be effective, they must be thoughtfully integrated into the sociotechnical aspects of the healthcare environment. This will require collaboration between healthcare organisations, informatics experts, EHR vendors and policymakers. Studies are needed to identify and mitigate the challenges to feedback, including constraints related to clinician time and organisational resources. Future studies should evaluate the impact of health IT-supported patient outcome feedback systems on clinical diagnosis and patient outcomes. If significant benefits are found, such systems could be prioritised for implementation through policy changes. The proposed strategies discussed herein can support clinician participation and advance the health IT structure and processes to enable feedback for improving patient outcomes.

Ethics statements

Patient consent for publication

Not required.