Skip to main content
NCBI home page
Elsevier - PMC COVID-19 Collection logoLink to Elsevier - PMC COVID-19 Collection
. 2021 Mar 26;61(5):607–614. doi: 10.1016/j.jemermed.2021.03.014

Building Back Better: Applying Lessons from the COVID-19 Pandemic to Expand Critical Information Access

Hope EM Schwartz *,1, Nicholas R Stark †, Cathleen S Sowa †, Malini K Singh †, Christopher R Peabody †
PMCID: PMC7997598  PMID: 34108121

Abstract

Background

The Coronavirus disease 2019 (COVID-19) pandemic generated an unprecedented volume of evolving clinical guidelines that strained existing clinical information systems and necessitated rapid innovation in emergency departments (EDs).

Objectives

Our team aimed to harness new COVID-19-related reliance on digital clinical support tools to re-envision how all clinical guidelines are stored and accessed in our ED.

Methods

We used a design-thinking approach including empathizing, defining the problem, ideating, prototyping, and testing to develop a low-cost, homegrown clinical information hub: E*Drive. To measure impact, we compared web traffic on E*Drive to our legacy cloud-based folder system and conducted a survey of end-users using a validated health technology utilization instrument.

Results

Our final product, E*Drive, is a centralized clinical information hub storing everything from clinical guidelines to discharge resources. Clinical guidelines are standardized and housed within the high-traffic E*Drive platform to increase accessibility. Since launch, E*Drive has averaged 84 unique weekly users, compared with less than one weekly user on the legacy system. We surveyed 52 clinicians for a total response rate of 47%. Prior to the E*Drive rollout, 12.5% of ED clinicians felt confident accessing clinical information on the legacy system, whereas 76.6% of ED clinicians felt they could more easily access clinical information using E*Drive.

Conclusion

The COVID pandemic revealed vulnerabilities within our information dissemination system and presented an opportunity to improve clinical information delivery. Centralized web-based clinical information hubs designed around the clinician end-user experience can increase clinical guideline access in the ED.

Keywords: COVID-19, clinical guidelines, digital, innovation, information, design

Introduction

At the onset of the Coronavirus disease 2019 (COVID-19) pandemic, the volume of rapidly evolving clinical information necessitated innovation in emergency department (ED) information management (1, 2, 3). Among these innovations was our hospital's COVID-19 decision support tool, zsfgCOVID, a digital COVID-19 information source with up-to-date guidelines related to testing, personal protective equipment, and airway management (1). Epidemics and other public health crises can expose weaknesses in the organization and delivery of clinical care (4). Although COVID-19 clinical guidelines have tended to change more rapidly than guidelines in other areas of clinical care, the challenges presented by COVID-19 information management are broadly applicable.

Clinical guidelines are an important component of high-quality patient care (5,6). This is vital in the ED setting where clinicians must provide treatment in high-acuity situations (7). However, there is substantial variation in the use of clinical guidelines in emergency care (8). Several factors contribute to this variation, including inefficient access to relevant information, difficulty rapidly synthesizing information in variable formats, and challenges maintaining up-to-date information as practice guidelines evolve (9, 10, 11, 12, 13, 14, 15, 16, 17, 18).

Our team aimed to harness COVID-19-driven innovation to address key barriers in the utilization of clinical guidelines in our ED though the creation of E*Drive: a centralized, web-based clinical information hub that houses important and commonly used clinical information (https://edrive.ucsf.edu) (19). We used a design thinking approach that relied on rapid prototyping and end-user feedback to ensure ease of use and leadership engagement to ensure rigorous quality control. In this study, we describe an example of grassroots, clinician-driven systems improvement. We describe how our approach led to an innovative final product and share our impact and lessons learned to empower ED administrative and clinical leadership to replicate our efforts.

Materials and Methods

We approached building a web-based centralized clinical information hub using design thinking methodology, an iterative process centered on the end-user and using rapid prototyping to increase utility and usability of products in the health care setting (20). Steps in the design thinking process include: (1) empathizing with the end-user to understand their needs; (2) defining the problem; (3) ideating to generate potential solutions; (4) prototyping the product; (5) launching and testing the product in the natural environment (21).

Empathize

In the early months of the pandemic, clinicians on our team quickly realized the powerful applicability of our COVID-19 platform and received multiple requests to include non-COVID-related guidelines (1). Based on these requests and informal interviews with fellow clinicians while working on shift, our team brainstormed how we could apply the lessons we learned from the development of our COVID-19 platform to a broader clinical information tool. We learned that there was a desire to quickly access and that such a system did not exist at our institution. With the support of our departmental leadership, we set out to design a system for rapidly accessing hospital and department-specific clinical guidelines in-house, rather than using an expensive, third-party system.

Define

In informal interviews, clinicians reported difficulty accessing up-to-date guidelines and applying lengthy text-based guidelines in the acute care setting. To understand previous efforts to address this problem, we conducted a literature review of academic articles describing digital information systems for clinical guidelines using the search term “emergency department AND digital AND clinical AND (guidelines OR protocols).” Our PubMed search returned 35 results, two of which described platforms for digital clinical guideline storage. We also conducted a landscape analysis of publicly accessible ED clinical guideline platforms using a Google search with the search term “emergency department clinical guidelines protocols.” We reviewed the first 15 web-based platforms from U.S. academic centers and community hospitals to better understand the current state of how clinical guidelines are stored and accessed online (Appendix 1, available online). We found significant variation in quality and format with two key features that stood out as barriers to usability: first, clinical guidelines were stored in a number of formats ranging from PDF to Word documents that were not mobile friendly; second, clinical guidelines were housed in siloed websites, rather than integrated with other clinical information such as department announcements and discharge resources.

Ideate

We assembled an ED Clinical Guidelines Task Force consisting of an emergency medicine attending and resident, a medical student, and a website administrator to adapt our previously developed COVID-19 platform to serve as a broad clinical information hub. The goal of this group was to increase accessibility and usability, starting with the most utilized clinical guidelines. The team iterated on several early guidelines to define best practices for transforming lengthy text-based documents to a standardized, user-friendly algorithmic template (Appendix 2, available online), which were then applied to future rounds of development. These best practices are being used to ensure quality control moving forward.

Prototype

Our team used rapid-cycle, bottom-up end-user feedback to develop the initial prototype and make real-time improvements during the design process. We engaged ED leadership to ensure top-down support and address additional administrative concerns, including developing a rigorous change control process to ensure sustainability and retain clinical integrity of the guidelines during the reformatting process (Appendix 3, available online). The standardized protocol development process and key stakeholders are outlined in Figure 1 , with defined roles and estimated costs provided in Table 1 .

Figure 1.

Figure 1

Open in a new tab

Clinical guideline development: key stakeholders and change control process map.

Table 1.

Example Roles, Hours, and Costs to Develop a Web-Based Clinical Information Hub

Title Role Total Hours Estimated Cost
Project Manager Oversee project timeline, coordinate meetings, monitor progress, manage platform revisions 25–35 $40–50/h
Clinician Review and approve clinical guidelines, prioritize clinical guidelines to standardize 25–35 $150–250/h
Nonclinical Project Staff Draft redesigned clinical guidelines in collaboration with clinical team members
 40–60 $20–30/h
Site Administrator Regularly update platform with new guidelines 15–25 $20–30/h
Open in a new tab

One-time total cost range: $5850–$13,050.

Launch and Test

E*Drive was formally launched on October 13, 2020. A multimodal outreach and messaging campaign was employed, which included e-mail announcements and flyers with QR codes and the E*Drive web link distributed throughout the ED on workstations and other highly trafficked areas (Appendix 4, available online). A promotional video was also shown at multiple staff meetings and departmental conferences to build name recognition and generate enthusiasm for the launch of E*Drive (https://www.youtube.com/watch?v=MFEzGffOYfo).

Measure Impact

After launch, we used a quantitative survey to assess the usability and utility of E*Drive based on the Technology Acceptance Model (Appendix 5, available online) (22). The survey was distributed on the Qualtrics survey platform (Qualtrics Software Version: November 2020, Provo, UT), with participants recruited via e-mail outreach that included all ED attending, resident, and advanced practice providers (23). Results from these efforts are being used to measure impact and to refine the E*Drive product through continuous improvement, a key tenant of the design thinking process. This study was approved as exempt research by the University of California, San Francisco Institutional Review Board (Study #20-30875).

Results

Final Product

E*Drive is a centralized clinical information hub accessible at https://edrive.ucsf.edu and built on a Drupal web platform (Version 9; 2020) (24). The landing page contains five broad categories of clinical information, including clinical guidelines, reporting forms, COVID-19 guidelines, antibiotics, and discharge navigation. Rapidly changing announcements, which include important updates about COVID-19, equipment, and other hospital-wide announcements that need to be quickly and broadly circulated to all ED staff, are also located on the landing page (Figure 2 ).

Figure 2.

Figure 2

Open in a new tab

E*Drive landing page (https://edrive.ucsf.edu).

Within the clinical guidelines section, the E*Drive site is optimized to access relevant guidelines within two clicks, and guidelines are standardized and optimized for accessibility in the acute care setting (Figure 3 ). Guidelines are created in Google Slides so they can be created, added, and modified with no specialized training.

Figure 3.

Figure 3

Open in a new tab

Exemplar protocol pre and post standardization: cervical spine clearance.

GCS = Glasgow Coma Scale score; CT = computed tomography; MRI = magnetic resonance imaging; ED = emergency department.

Impact

Using internal ED operations data, we compared the use of E*Drive to the legacy cloud-based folder system for clinical guideline storage. Prior to the E*Drive rollout, the cloud-based site housing clinical guidelines received 47 unique page views from December 2019 to December 2020, for an average of 0.13 views per day or less than one view per week. Since launch on October 13, 2020, E*Drive has averaged 12 unique views per day and 84 views per week. Given an average daily staffing volume of 35 providers, this means about 34% of providers access E*Drive on a given day, on average.

Fifty-two clinicians responded to the survey, including 29 attending physicians, 17 resident/fellow physicians, and 5 advanced practice providers. This response rate represents 47% of 110 clinical providers who have worked in our ED since the launch of E*Drive. Of all clinicians surveyed, 12.5% felt confident accessing clinical information on the internal cloud-based folder system prior to the E*Drive rollout (Table 2 ); 76.6% agreed or strongly agreed that they were able to access clinical information using E*Drive more easily; 70.3% agreed or strongly agreed that E*Drive was helping them do their job more efficiently; and 78.1% found the E*Drive platform understandable and easy to navigate. Clinical guidelines were reported to be the most useful component of the E*Drive platform (50.8%), followed by antibiograms (29.7%) and COVID-19 guidelines (26.6%). On a scale of 1–10, the average likelihood of recommending use of E*Drive to a colleague was 8.02 (95% CI 7.49–8.55). End-user feedback collected in the Qualtrics survey after rollout suggested the following areas for improvement in future iterations of the E*Drive platform: enhanced searchability and sitemap for clinical guidelines, expansion to hospital-wide clinical guidelines, and relevant phone numbers (which were subsequently added to E*Drive within 1 month of launch).

Table 2.

Quantitative End-User Feedback Survey Results

Question Strongly Agree
 Agree
 Disagree
 Strongly Disagree
n % n % n % n %
Prior to the E*Drive platform, I felt confident accessing clinical information on the wiki system at ZSFG. 6 12% 2 4% 29 57% 14 27%
Since the launch of the E*Drive platform, I am able to more easily access clinical information at ZSFG. 21 41% 28 55% 2 4% 0 0%
Accessing E*Drive is useful in helping me do my job more efficiently. 14 27% 31 61% 6 12% 0 0%
I find the E*Drive platform understandable and easy to navigate. 19 37% 31 60% 2 4% 0 0%
Daily
 Multiple × /Week
 Once/Week
 Once/Month
 Never
n % n % n % n % n %
I use the E*Drive platform approximately: 3 6% 8 15% 18 35% 18 35% 5 10%
Open in a new tab

Discussion

Using design thinking, we developed a novel clinical information system for increasing access to clinical guidelines in our ED. The E*Drive platform significantly increased clinician confidence in accessing clinical information, and the majority of users found it easy to use and integrate into their clinical practice. The impact of E*Drive was also supported by usage data, which showed that E*Drive significantly increased the proportion of providers accessing clinical guidelines, compared with our prior clinical information system.

Compared with existing online clinical information platforms, E*Drive stands apart as a low-cost, clinician-designed, and highly customizable platform that can be rapidly updated to incorporate real-time information. Like our COVID-19 platform, E*Drive was developed in-house using open-source software rather than relying on a third-party developer. This approach allowed us to create system-level change at low cost with a small, agile team, including ED clinicians with first-hand clinical expertise, operational support for rapid execution, and a direct line to ED leadership (Table 1). The design-thinking approach allowed us to bridge bottom-up end-user feedback and top-down leadership requirements to maximize ease of use while retaining rigorous clinical standards. Our formal change-control process has allowed us to continue to standardize, upload, and widely disseminate protocols nearly instantaneously, as dictated by changing circumstances during the COVID-19 pandemic.

Key to the usability of E*Drive are three unique features that set our platform apart from comparable systems: standardization, integration, and accessibility. Other clinical information hubs reviewed in our landscape analysis link to documents in a variety of non-standardized formats, including text-based and visual guidelines, requiring downloads to access, which may present a barrier to real-time clinical use. The E*Drive platform makes use of rigorous guidelines for flow, color, and location of information to ensure that guidelines are easy to synthesize and apply in the acute care environment (Appendix 2).

In building E*Drive, we integrated multiple streams of commonly accessed clinical information, including redesigned clinical guidelines, COVID-19 information, local antibiograms, and daily announcements on a single landing page. Our platform also employs an open-access and mobile friendly design (Figure 2). Site design emphasized minimizing clicks, and guidelines are displayed in JPEG format so that mobile users can easily enlarge information on their devices, further increasing ease of access in a busy ED setting.

Lastly, we harnessed COVID-19 digital innovation as a call-to-action illustrating the power of frontline clinicians to create meaningful systems change. By integrating the E*Drive platform with our pre-existing COVID-19 information platform, we leveraged momentum from COVID-19-necessitated practice changes to create buy-in for a new clinical information hub. The challenges of delivering emergency care during a global pandemic has provided a testing-bed and unprecedented momentum for clinical digital innovation.

Limitations

There are several limitations to the current study. First, 47% of clinical providers in the ED responded to our survey, introducing potential nonresponse bias that could skew the reported relative utility of the E*Drive product. However, objective usage data support our conclusion that the E*Drive platform has increased the usability of clinical guidelines. Second, this is a single-center study, and variation in clinical systems and department culture could affect implementation at other institutions. However, we used open-source and low-cost resources to build the E*Drive platform, and we share our process, lessons learned, and next steps for application to other institutions to encourage replication (Appendix 6, available online). Third, E*Drive currently houses ED-specific guidelines with limited inpatient protocols. Including inpatient protocols on our platform is an important area of ongoing work to improve care transitions between ED and inpatient services and to optimize care for admitted patients boarding in the ED. Lastly, we do not know whether increased accessibility to clinical information impacted clinical practice among providers at our institution. Future studies should investigate whether greater access to clinical guidelines from E*Drive has a direct correlation with improving provider adherence to guidelines.

Conclusion

The COVID-19 pandemic called for rapid innovation in the use of digital tools to streamline access to clinical information. We applied lessons learned from COVID-19 to create a centralized clinical information hub with standardized clinical guidelines to increase accessibility in our ED. Using a design thinking process, we built a flexible, in-house, low-cost platform that is scalable and reproducible. The creation and implementation of E*Drive demonstrates that design thinking methodology, including rapid prototyping and soliciting clinician end-user feedback, can help improve accessibility and use of clinical guidelines in the ED.

Article Summary

1. Why is this topic important?

Clinical guideline utilization is associated with improved outcomes in the acute care setting; however, numerous barriers in existing clinical information systems have limited clinicians’ ability to access and utilize clinical guidelines.

2. What does this study attempt to show?

We aimed to use a design-thinking approach to build an in-house, customizable, low-cost clinical information hub to increase access to clinical guidelines in the emergency department (ED). We built on momentum from COVID-19 pandemic-related practices changes and increased reliance on digital clinical support tools to generate momentum for broader systems change.

3. What are the key findings?

Our final product, E*Drive, is an open-access, web-based platform housing everything from standardized clinical guidelines to discharge resources. Web-traffic usage data showed that the E*Drive platform increased the frequency with which clinicians accessed clinical guidelines. Clinicians reported significantly increased ability to access and use clinical guidelines in their clinical practice.

4. How is patient care impacted?

By increasing access to clinical guidelines in the ED, E*Drive has the potential to impact patient care through the application of clinical guidelines.

Footnotes

Supplementary material associated with this article can be found, in the online version, at doi:10.1016/j.jemermed.2021.03.014.

Appendix. Supplementary materials

mmc1.docx (3.7MB, docx)

References

  • 1.Stark N., Kerrissey M., Grade M., Berrean B., Peabody C. Streamlining care in crisis: rapid creation and implementation of a digital support tool for COVID-19. West J Emerg Med. 2020;21:1095–1101. doi: 10.5811/westjem.2020.7.48537. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Lai L., Sato R., He S., et al. Usage patterns of a web-based palliative care content platform (PalliCOVID) during the COVID-19 pandemic. J Pain Symptom Manage. 2020;60:e20–e27. doi: 10.1016/j.jpainsymman.2020.07.016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Noble J., Degesys N.F., Kwan E., et al. Emergency department preparation for COVID-19: accelerated care units. Emerg Med J. 2020;37:402–406. doi: 10.1136/emermed-2020-209788. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Krausz M., Westenberg J.N., Vigo D., Spence R.T., Ramsey D. Emergency response to COVID-19 in Canada: platform development and implementation for eHealth in crisis management. JMIR Public Health Surveill. 2020;6:e18995. doi: 10.2196/18995. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Ziegler D.W., Agarwal N.N. The morbidity and mortality of rib fractures. J Trauma. 1994;37:975–979. doi: 10.1097/00005373-199412000-00018. [DOI] [PubMed] [Google Scholar]
  • 6.Rudin R.S., Fischer S.H., Shi Y., et al. Trends in the use of clinical decision support by health system–affiliated ambulatory clinics in the United States, 2014–2016. Am J Accountable Care. 2019;7(4) https://www.ajmc.com/view/trends-in-the-use-of-clinical-decision-support-by-health-system-affiliated-ambulatory-clinics-in-the-united-states-20142016 Available at: [Google Scholar]
  • 7.Taufique Z., Timen M., Swartz J.L., Friedmann D.R. Adherence to subspecialty guidelines in the emergency department. Laryngoscope. 2020 Oct 26 doi: 10.1002/lary.29202. [Epub ahead of print] [DOI] [PubMed] [Google Scholar]
  • 8.Ebben R.H., Vloet L.C., Verhofstad M.H., Meijer S., Groot J.A.M., van Achterberg T. Adherence to guidelines and protocols in the prehospital and emergency care setting: a systematic review. Scand J Trauma Resusc Emerg Med. 2013;21:9. doi: 10.1186/1757-7241-21-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Francke A.L., Smit M.C., de Veer A.J.E., Mistiaen P. Factors influencing the implementation of clinical guidelines for health care professionals: a systematic meta-review. BMC Med Inform Decis Mak. 2008;8:38. doi: 10.1186/1472-6947-8-38. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Centers for Disease Control and Prevention (CDC) Public Health Informatics Office (PHIO) 2020. Adapting clinical guidelines for the digital age.https://www.cdc.gov/csels/phio/clinical-guidelines/index.html Published June 11Available at: [Google Scholar]
  • 11.Woolf S.H., Grol R., Hutchinson A., Eccles M., Grimshaw J. Clinical guidelines: potential benefits, limitations, and harms of clinical guidelines. BMJ. 1999;318:527–530. doi: 10.1136/bmj.318.7182.527. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Sutton R.T., Pincock D., Baumgart D.C., Sadowski D.C., Fedorak R.N., Kroeker K.I. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med. 2020;3:17. doi: 10.1038/s41746-020-0221-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Correa V.C., Lugo-Agudelo L.H., Aguirre-Acevedo D.C., et al. Individual, health system, and contextual barriers and facilitators for the implementation of clinical practice guidelines: a systematic metareview. Health Res Policy Syst. 2020;18:74. doi: 10.1186/s12961-020-00588-8. https://health-policy-systems.biomedcentral.com/articles/10.1186/s12961-020-00588-8 Available at: [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Van de Velde R. Framework for a clinical information system. Int J Med Inform. 2000;57:57–72. doi: 10.1016/s1386-5056(99)00062-3. [DOI] [PubMed] [Google Scholar]
  • 15.Ciccarese P., Caffi E., Boiocchi L., Quaglini S., Stefanelli M. A guideline management system. Stud Health Technol Inform. 2004;107:28–32. [PubMed] [Google Scholar]
  • 16.Chan W.V., Pearson T.A., Bennett G.C., et al. ACC/AHA special report: clinical practice guideline implementation strategies: a summary of systematic reviews by the NHLBI Implementation Science Work Group: a report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines. J Am Coll Cardiol. 2017;69:1076–1092. doi: 10.1016/j.jacc.2016.11.004. [DOI] [PubMed] [Google Scholar]
  • 17.Jun J., Kovner C.T., Stimpfel A.W. Barriers and facilitators of nurses’ use of clinical practice guidelines: an integrative review. Int J Nurs Stud. 2016;60:54–68. doi: 10.1016/j.ijnurstu.2016.03.006. [DOI] [PubMed] [Google Scholar]
  • 18.Mathe J.L., Martin J.B., Miller P., et al. A model-integrated, guideline-driven, clinical decision-support system. IEEE Software. 2009;26:54–61. [Google Scholar]
  • 19.University of California San Francisco (UCSF). E*Drive. Available at: https://edrive.ucsf.edu/. Accessed November 8, 2020.
  • 20.Roberts J.P., Fisher T.R., Trowbridge M.J., Bent C. A design thinking framework for healthcare management and innovation. Healthcare. 2016;4:11–14. doi: 10.1016/j.hjdsi.2015.12.002. [DOI] [PubMed] [Google Scholar]
  • 21.Altman M., Huang T.T.K., Breland J.Y. Design thinking in health care. Prev Chronic Dis. 2018;15:E117. doi: 10.5888/pcd15.180128. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Davis F.D. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q. 1989;13:319–340. [Google Scholar]
  • 23.Qualtrics. Home page. Available at: https://www.qualtrics.com/. Accessed November 8, 2020.
  • 24.Drupal. Home page. Available at: https://www.drupal.org/. Accessed November 8, 2020.

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

mmc1.docx (3.7MB, docx)

Articles from The Journal of Emergency Medicine are provided here courtesy of Elsevier

RESOURCES