Opportunities to Apply Human-centered Design in Health Care With Artificial Intelligence–based Screening for Diabetic Retinopathy

Abstract

Diabetic retinopathy (DR) is a leading cause of blindness. Artificial intelligence (AI) has been proposed to provide a novel opportunity to increase screening for DR. While it is paramount to ensure AI has adequate technical capabilities to perform accurate screening, it is also important to assess how to best implement such technology into clinical practice. Human-centered design offers a methodology to understand the real-world context and behaviors of individuals, engage stakeholders, and rapidly prototype and test solutions, enhancing usability and avoiding unintended consequences. This review describes the methodology of human-centered design, examining how it has been used within a variety of health care contexts, with a particular focus on how it has been used to implement an AI-based DR screening program. Further research is needed to understand the best strategies to implement and evaluate AI in health care.

INTRODUCTION

Diabetic retinopathy (DR) is a leading cause of blindness around the world, however, early detection and treatment can prevent visual impairment by over 90%.¹ Unfortunately, more than half of patients with diabetes have been estimated to miss recommended eye screening guidelines.² Artificial intelligence (AI) has been poised to provide a novel and effective opportunity to increase screening for DR, with studies reporting up to 100% sensitivity and 91.1% specificity for detecting DR.^3,4

While much of the research on AI for DR screening has focused on examining its technical capabilities, there is a gap in understanding how this technology may be implemented clinically. An algorithm that is accurately predictive does not always translate into success in a real-world clinical environment.^5,6 Successful implementation of technology involves understanding and designing for the various stakeholders involved, such as ophthalmologists, optometrists, primary care partners, clinic managers, staff, and patients. Human-centered design (HCD) offers a methodology to understand the real-world context and behaviors of individuals, engage stakeholders, and rapidly prototype and test solutions.⁷ This iterative methodology offers the opportunity to continuously evaluate whether designed processes or products are prioritizing user needs by actively engaging users throughout the development process, enhancing usability, and avoiding unintended consequences.⁸ In this paper, we discuss how HCD and its implications in health care can be applied to AI-based screening for DR.

DEFINING HUMAN-CENTERED DESIGN

HCD is a methodology that was born from the design community as they shifted from viewing the role of a designer as one to make an existing idea more attractive to customers to seeing designers as integral participants who are involved from the onset to develop ideas that meet customer needs first.⁹ Tim Brown, a pivotal expert in HCD and chair at IDEO, one of the largest design firms that has contributed significantly to sharing the practice of HCD, defines the HCD methodology as one that “must ultimately pass through 3 spaces…Inspiration, Ideation, and Implementation.”⁹ Recognizing that the process is nonlinear and iterative, he admits that it can feel chaotic to those who are unfamiliar with the process, but that returning to each space several times throughout the course of a project, using key values of empathy, integrative thinking, optimism, experimentalism, and collaboration, can lead to a refinement that drives innovation.⁹ While the framework of Tim Brown for HCD will be used to foundationally structure the following discussion, it is also important to acknowledge the various definitions of HCD. These definitions share a similar goal of understanding user needs through iteration throughout the entire process by seeking feedback from stakeholders in a process that is very time and resource-intensive but can reduce the risk of failure.¹⁰

While HCD is the product of the design and engineering industry, it has since been adapted for use in the health care space, given how problems in health care often require innovative, empathy-driven solutions. While traditional health services research uses methods with the goal of developing evidence-based interventions, this structure does not always guarantee successful implementation as it often favors what works for the average user based on posited theories and can neglect creative solutions inspired and tested by actual users; however, because HCD explores various stakeholder’s desires and motivations to build proposed innovations, the process may result in ideas that are more creative, desirable and feasible to implement in a sustainable way.^11,12 Prior studies comparing HCD interventions to traditional interventions have demonstrated that HCD interventions result in greater user satisfaction, effectiveness, and usability.¹³

One 2017 scoping review emphasizes how HCD is well suited to innovate for problems in the global health space, citing 21 articles that used HCD to develop interventions, such as those to encourage healthy lifestyle behaviors, increase health insurance accessibility for children, and provide health education.¹⁴ The review also highlights the distinction between traditional scientific research reporting and HCD, given the latter often has open-ended processes that embrace ambiguity to drive innovation in a way that can limit reproducibility and lead to different approaches in measuring impact.¹⁴ Another article highlights the growing prominence of HCD in health care, providing an overview of the practice of HCD and presenting concrete ways on how HCD promotes global health equity through their Medic Mobile project.¹⁵ A more recent review from 2021 identified 82 studies that used HCD to tackle health care problems, such as designing interventions for smoking cessation, providing support to patients with diabetes, and designing web-based vision screening for children with amblyopia,¹⁶ demonstrating how this field has grown rapidly in just a few years.

Examples of Human-centered Design Methodology in Health Care: Inspiration

Embarking on the “Inspiration Phase” of HCD involves understanding the user needs and identifying the problem to which the design process will be applied. In one example, patients, family members, health center staff, community health workers, and community leaders involved in 8 tuberculosis treatment units were interviewed to better understand the challenges to tuberculosis medication adherence, resulting in identifying common themes and insight statements that ultimately inspired the creation of a digital platform for tuberculosis management, 99DOTS.¹⁷ Other examples of how the Inspiration phase has been used to design for health care challenges include a study where stakeholder interviews were conducted to understand inequities in prenatal care,¹⁸ observations were performed to map the patient experience of attending a health care appointment, revealing the frustrations associated with waiting to be seen,¹⁹ and workshops were held to identify design opportunities to reduce diagnostic disparities.²⁰

Examples of Human-centered Design Methodology in Health Care: Ideation

The “Ideation Phase” involves processes to brainstorm and develop ideas targeted at meeting user needs identified in the Inspiration Phase. In the 99DOTS tuberculosis study, the ideation phase included 5 brainstorming sessions which elucidated 127 unique ideas that were sorted into common themes that drove the development of 16 prototypes of pill packaging to encourage medication adherence.²¹ This collaborative approach reduced the stigma associated with tuberculosis, which was shown to be an important challenge to medication adherence.²¹ As another example, inviting various stakeholders such as parents, nurses, physicians, designers, engineers, and students was key in the ideation phase of a project to propose ideas to improve the experience within a neonatal intensive care unit.²² Additionally, another study incorporated users in group design sessions to assist with hypertension management in chronic kidney disease, resulting in the generation of several prototype ideas that were evaluted to maximize benefits to stakeholders while also minimizing negative consequences.²³

Examples of Human-centered Design Methodology in Health Care: Implementation

The “Implementation Phase” utilizes findings from the other 2 phases to deploy a potential solution and to seek user feedback to continuously refine the intervention. Returning to the 99DOTS intervention for tuberculosis medication adherence, assessing the implementation and feasibility of the intervention was nested within a randomized trial that demonstrated a 98% reported adherence to treatment following the intervention, with the majority of patients and health workers indicating in post-intervention surveys that the intervention was convenient and fostered connections between patients and health care workers.²⁴ Another randomized control trial protocol plans to evaluate the implementation of a vaping cessation tool for teens developed using cocreation.²⁵ One article discussed the challenges to implementing and measuring the impact of HCD interventions, presenting several case studies where a key tenet in implementing an HCD intervention was the routine review of data and active assessment of what worked well and what did not to guide real-time adaptation of the intervention.²⁶

It is important to understand that these 3 phases of “Inspiration, Ideation, and Implementation” are nonlinear, as the process of HCD requires continuously returning to each phase and iterating on the design after elucidating user feedback to refine the solution.

USING HUMAN-CENTERED DESIGN TO IMPLEMENT ARTIFICIAL INTELLIGENCE-BASED SCREENING FOR DIABETIC RETINOPATHY

A previous paper from our group discusses how the initial Inspiration and Ideation Phases of HCD were utilized in developing an AI-based screening for DR.²⁷ The focus of this article is to discuss the HCD implementation of this program in an endocrinology clinic, which was built from the collaboration between the University of Illinois’s Institute for Healthcare Delivery Design (IHDD), Endocrinology Department, and Ophthalmology Department.

Given the Endocrinology Department’s high volume of patients with diabetes and the impact a DR diagnosis may make on diabetes medication management, in addition to the Ophthalmology Department’s investment in identifying preventable blindness from DR, the decision was made to implement the AI screening program for DR in endocrinology. Understanding how to implement an AI screening for DR program within endocrinology that would meet staff and patient needs required recurring weekly meetings, hundreds of hours of observations, and several workshops involving various stakeholders. Through these sessions, it became clear that a screening program would need to minimize the burden on clinical staff, provide clear guidance on how to use the AI technology, identify patients to prioritize, integrate within existing clinical workflows, and facilitate how screening results would translate into ophthalmology or optometry referrals. One example of an Implementation Phase deliverable included a Medical Assistant Workflow Guide that underwent several iterations based on observations from the IHDD team, feedback from the Endocrinology Clinic Manager, and clinical guidance from physicians.

To practice the iterative values of HCD, weekly meetings with key stakeholders within the endocrinology, ophthalmology, and IHDD teams have been held to discuss unintended consequences, challenges, and successes to the implementation of the screening program. A current challenge to implementation includes identifying how to troubleshoot a high percentage of “ungradable” images produced by the AI screening. This challenge is being addressed by observing screening practices and meeting with the AI device team to provide insights into how to continue to iterate and refine the program. The HCD process encourages us to examine multiple avenues that result in ungradable images and identify factors that may not have been previously considered.

Future steps for evaluating the implementation include surveying patient perceptions regarding AI to screen for DR, interviewing staff, and defining metrics to measure implementation success. The findings from this evaluation will help to further refine the screening program. It is important as well to note that findings from the HCD process are not intended to be fully generalizable between different populations although the methodology is generalizable. Because a strength of HCD is the depth to which qualitative exploration identifies specific user, stakeholder, and system needs, the solution that is generated will be individually tailored. Thus, successful implementation in one system may not always present identically in a different system. There is tremendous value in dedicating time and resources to laying the groundwork for HCD, as the process is extensive given the cycles of iterations guided by specific user feedback and the inherent openness to pivoting to allow for the refinement of an intervention. From the initiation of our project to the beginning of implementation, it has taken 11 months of collaboration, which will continue to extend into future months as the Implementation Phase requires frequent assessment and refinement as well.

PARTNERSHIP IS KEY IN USING HUMAN-CENTERED DESIGN IN HEALTH CARE

A foundational step to implementing HCD to tackle health care problems is establishing a partnership, not just with health care stakeholders but also with design partners. Senior design strategists at IHDD have been instrumental in developing the AI screening program for DR. While some articles cited throughout this manuscript were written in partnership with design organizations such as The Better Lab (https://www.thebetterlab.org/), The MeasureD Lab (https://measured.design/), Hopelab (https://hopelab.org/), and IDEO (https://www.ideo.com/) additional options to collaborate with designers may be available through partnering with design schools to provide a more accessible option to health care systems already situated within academic institutions.¹¹

FUTURE STEPS TO CONSIDER WHEN USING HUMAN-CENTERED DESIGN FOR THE IMPLEMENTATION OF ARTIFICIAL INTELLIGENCE IN HEALTH CARE

In considering the implementation of AI in health care, there are limitations to the technology that need to be evaluated. Given the fact that an AI algorithm is only as effective as the data it is trained on, it is important to evaluate the inputs and outputs of the technology. Just as new medications must undergo rigorous phase IV testing and regulations, evaluating the safety and efficacy of AI technology when deployed in a clinical setting is important. The lack of regulatory guidelines for the use of AI in healthcare means that data sets that train the algorithms may remain unstandardized and beget ethical implications surrounding issues such as data ownership and deployment.²⁸

To test AI technology, it is, in additon, crucial to understand how the algorithm works. The INTRPRT guideline [which includes themes of incorporation (IN), interpretability (IN), target (T), reporting (R), prior (PR), and task (T)] is one proposed framework aimed at increasing transparency in understanding how algorithms function. It adapts HCD methodology to create transparency in the development of machine learning algorithms to evaluate if the resulting algorithms effectively address the needs of their intended users.²⁹ This approach ultimately provides an avenue to refine and validate an algorithm before adoption for use in clinical decision-making.

Not only can HCD increase transparency surrounding AI technology and assist in the implementation of AI technology in a clinical setting, but also HCD’s foundational value to design with, rather than for, users has been suggested to increase patient trust in health care interventions when patient perspectives are prioritized in the process of problem-solving.³⁰ As demonstrated by the examples discussed in this article, applying HCD to health care challenges in ophthalmology can provide an effective methodology to drive innovation.