Patient and Provider Experience with Artificial Intelligence Screening Technology for Diabetic Retinopathy in a Rural Primary Care Setting

Abstract

Introduction:

The development of autonomous artificial intelligence for interpreting diabetic retinopathy (DR) images has allowed for point-of-care testing in the primary care setting. This study describes patient and provider experiences and perceptions of the artificial intelligence DR screening technology called EyeArt by EyeNuk during implementation of the tool at Western Maine Primary Care in Norway, Maine.

Methods:

This non-randomized, single-center, prospective observational study surveyed 102 patients and 13 primary care providers on their experience of the new screening intervention.

Results:

All surveyed providers agreed that the new screening tool would improve access and annual screening rates. Some providers also identified initial challenges in incorporating the tool into the primary care visit (31%). Patients expressed a favorable view of the service, sharing an openness to being screened more regularly (75%) and a desire to have screenings performed at Western Maine Primary Care going forward (81%).

Discussion:

Patients were generally favorable about their experience with the new DR screening technology. Providers indicated challenges due to the limited availability of trained medical assistant photographers during the initial implementation of DR screening, as well as timing issues in coordinating screening with regular office appointments.

Conclusions:

This study supports further investigation of this technology in primary care, particularly in areas with challenges to care access. The potential benefits of this innovative tool in caring for people with diabetes includes improving access to retinopathy screenings and supporting wider detection of vision-threatening retinopathy.

Diabetic retinopathy (DR) is the leading cause of preventable blindness among adults in the developed world.¹ Widespread detection and early treatment of retinopathy could prevent 77% to 90% of permanent blindness associated with diabetes.^2,3 Despite measurement and performance incentives, screening rates have consistently been suboptimal, with the national rate hovering at 64.8%.^4,5 Typically, screenings must take place at a specialty practice for eye care. Access to care, cost, and misunderstanding the potential implications of untreated retinopathy are some of the barriers to acceptable screening rates.⁶

Efforts to increase screenings have involved performing retinal photography with specialized cameras within the primary care setting. These images have traditionally been sent to eye-care specialists for interpretation. The immediate availability of screening results at the point of care increased rates of timely referral follow-up for specialty evaluation and treatment of DR.⁷ The development of autonomous artificial intelligence (AI) for DR image interpretation has been progressing.^8–11 A review of several large peer-reviewed research studies found that AI screening can detect DR at a rate at least equal to both optometrists and general ophthalmologists, with more than 90% sensitivity for detecting more than mild to vision-threatening disease.^8–11 Early implementation of autonomous DR screening revealed promising results with respect to feasibility and validity in real-world clinical settings internationally.¹² This technology was granted 510(k) clearance by the Federal Drug Administration in August 2020.¹³ The Centers for Medicare and Medicaid Services approved a reimbursement code for clinical use, and AI DR screening is now qualified to meet Health Effectiveness Data and Information Set standards for quality care.^14,15

At Western Maine Primary Care (WMPC), screening rates for DR averaged 55.08% over the past 6 years (2015–2020).¹⁶ At this rate, more than 500 patients annually were not receiving appropriate screening. In 2020, WMPC was awarded an Ignite grant through the MaineHealth Innovation Center to purchase a Canon CR2-AF camera. This grant enabled WMPC to start retinopathy screening at the point of care using EyeArt (EyeNuk, Woodland Hills, CA) in 2021. Establishing this new service consisted of management, technical, legal, supply chain, and institutional review board review. It also involved contracting, camera acquisition, staff training, design and implementation of clinical workflow, documentation and referral process, and quality control. This study describes patient and primary care provider (PCP) experiences and perceptions of the AI DR screening technology during the initial implementation of the tool at WMCP.

METHODS

This non-randomized, single-center, prospective observational study surveyed 102 patients and 13 PCPs about their experience with a new DR screening intervention. This intervention detects DR in the primary care setting using autonomous AI for image interpretation. This cross-sectional study used patient and PCP survey data collected from a single practice located in Norway, ME, over 10 months (March 2021 to January 2022). This study was deemed exempt by the Institutional Review Board at Maine Medical Center (registration #1695750–1). To support studying the feasibility and acceptability of using this technology in primary care, Ignite grant funds covered screenings at no cost to the first 100 eligible patients. The study timeline was determined by the time needed to perform the first 100 screens on eligible patients.

Setting

WMPC is located in Norway, a small town in the rural county of Oxford, Maine. The practice serves 15 towns with a service area of 32 000 people. In 2021, the practice had approximately 31 000 visits and served 13 374 unique patients. Of the current patients at the practice, 1410 patients were diagnosed with type I or type II diabetes mellitus. WMPC has 23 providers, of which 14 are PCPs. The practice has 34 medical assistants (MAs). The region has 1 local ophthalmologist and 3 to 4 optometrists. Initially, 3 MAs and 1 PCP at WMPC were trained to use the camera and software by the vendor. Two additional MAs were trained during the study.

Screening Process

Patients were eligible for screening based on the health maintenance reminder (no prior screening for DR within 1 year) in the electronic health record. Any adult patients with diabetes but without diagnosed DR were eligible. After visiting their PCP, eligible patients were seen by an MA in a private room with the camera. The MA provided the screening, which took only a few minutes and did not require pupil dilation.

Data Collection and Analysis

Patient survey data was collected between March 2021 and November 2021. All patients who participated in the DR AI screening technology at WMPC during this time were invited to complete an anonymous survey immediately after being screened. No screened patients declined to complete the survey. After using the technology with patients for 9 months, PCPs were invited to complete a one-time anonymous survey soliciting their opinions and experience. This survey process took place in January 2022. All surveys were developed and stored in REDCap. The survey tools were developed by the study team. The goal of the surveys was to understand the feasibility of implementation from the PCPs perspective, as well as the perception and experience of patients, and whether the tool would improve access to care. Descriptive analysis of the findings were performed using R-Studio (version 3.6.2).

RESULTS

There was a 100% response rate among currently practicing PCPs at the practice (Table 1).

Table 1.

Provider Perspectives on Experience Using Artificial Intelligence Screening Technology for Diabetic Retinopathy (N = 13)

Provider perspectives on screening technology	Disagree/Strongly disagree, No. (%)	Neutral, No. (%)	Agree/Strongly agree, No. (%)
I find the diabetic retinopathy technology easy to use*	0 (0)	6 (75)	2 (25)
My patients have had a positive experience with the diabetic retinopathy screening technology in our practice	0 (0)	2 (15)	11 (85)
I find it easy to incorporate use of the diabetic retinopathy screening technology into visits with my patients who have diabetes	4 (31)	1 (8)	8 (61)
Use of the diabetic retinopathy screening technology in our practice improves access to screening for my patients	0 (0)	0% (0)	13 (100)
Use of the diabetic retinopathy screening technology in our practice will improve annual screening rates among our patients.	0 (0)	0% (0)	13 (100)
Provider’s rating of overall experience	Negative/Very negative, No. (%)	Neutral, No. (%)	Positive/Very positive, No. (%)
How would you rate your overall experience with the diabetic retinopathy technology?	0 (0)	4 (31)	9 (69)

^*5 respondents left this question blank

Although 14 PCPs generally practice at WMPC, 1 PCP was on maternity leave and unavailable to participate during the study period. In addition to quantitative survey responses, PCPs shared comments and explanations on how they saw the tool improving access. One PCP said about the tool, “Helps us triage patients to ophthalmology if they screen positive.” Another noted, “I have many patients who have challenges getting a diabetic eye exam in the community due to either transportation / psychosocial barriers, or lack of available providers. This DM [diabetic mellitus] retinopathy screen increases access for these patients.” Others noted challenges with implementing the tool in the primary care setting, with most PCPs noting issues with scheduling screenings with limited MA availability, “I would like to get trained in this device. Setting up the exam is sometimes difficult as it requires several steps. Find MA, talk to MA, have MA check schedule, get back to me, then talk to patient again.”

There was a 100% response rate to the patient-experience survey (Table 2). Patient’s comments were overwhelmingly positive, including notable praise for the MAs performing the exam and the ease of the experience. Patient’s comments included “Awesome, very quick and efficient,” “Glad you are doing this!” “Awesome addition to the practice,” and “MA did a super job!”

Table 2.

Patient Perspectives on the Artificial Intelligence Screening Technology for Diabetic Retinopathy (N = 102)

Prior experience with screening	No. (%) 102 (100)
Before today, approximately how many times in the past have you been screened for diabetic retinopathy?
Never	46 (45)
1–2 times	23 (23)
3–5 times	11 (11)
More than 5 times	14 (14)
I don’t know	8 (8)
Since being diagnosed with diabetes, would you say that you receive diabetic retinopathy screenings each year?
Yes, each year I receive screening	18 (18)
Yes, although I may have missed a year or two	21 (21)
No, I tend not to receive annual screenings	36 (35)
I was diagnosed in the past year/this is my first screening	15 (15)
I don’t know	12 (12)
Where did you typically receive your diabetic retinopathy screening in the past?
Ophthalmologist or optometrist in the greater Norway region	32 (31)
Ophthalmologist or optometrist in the greater Portland region	2 (2)
This is my first screening	36 (35)
I don’t know	13 (13)
Other	19 (19)
Future behaviors	102 (100)
With the availability of diabetic retinopathy screening in our practice, will this change where you will obtain your annual screening going forward?
Yes, I’d like to have my screening done here each year going forward	83 (81)
No, I’d like to go back to seeing my ophthalmologist or optometrist for my annual screenings	8 (8)
I don’t know	11 (11)
With the availability of diabetic retinopathy screening in our practice, will this change how often you will obtain your annual screening going forward?
Yes, I will probably be screened more regularly going forward	77 (75)
No, I always get my annual screenings, regardless of location	11 (11)
I don’t know	14 (14)
Barriers	99 (97)*
What barriers have you faced to receiving your annual diabetic retinopathy screening in the past?
Difficult to make an appointment with an ophthalmologist or optometrist	19 (19)
Difficult to drive to my ophthalmologist or optometrist appointment	3 (3)
I have not encountered any barriers	44 (44)
I was diagnosed with diabetes in the past year/this is my first screening	20 (20)
Other	6 (6)
Other - No insurance/finances	3 (3)
Other - COVID-19 complications	1 (1)
Other - Pupil dilation and other exam-associated discomfort	3 (3)
Patient experience	99 (97)*
How would you rate your overall experience with the diabetic retinopathy technology?
Negative/Very negative	0 (0)
Neutral	6 (6)
Positive/Very positive	93 (94)

^*3 respondents left these questions blank

DISCUSSION

The findings showed that patients viewed the service favorably, based on both survey results and written comments. The results suggest that most patients (75%) were open to being screened more regularly, and many patients (45%) welcomed the first opportunity to be screened. Overall, PCPs gave a slightly less favorable assessment (69%), although all PCPs expressed support for the service offering and optimism that it may improve care (100%). Some PCPs indicated trouble accessing the screening during office hours. This problem was possibly due to the limited availability of trained MA photographers during the initial implementation of DR screening, as well as timing issues in coordinating screening with regular office appointments.

Favorable patient ratings may have been biased by the offering of no-cost exams for this study. Comparison with a cohort subject to a Current Procedural Terminology charge for the service might be of interest. Approximately 3% of participants cited cost/finances as a barrier to standard screening. Other barriers included difficulty with scheduling or transportation to an eye appointment (22%), and a need for pupil dilation or other discomfort related to the exam (3%). This program may offer an alternative to overcome these barriers. Key learnings included the need for adequate technical skill training of photographers before service adoption, and the need for an adequate number of MA photographers to meet the need posed by eligible patients in a busy office.

CONCLUSIONS

This prospective observational study surveyed patients and PCPs about their experience of a new AI screening intervention to detect DR in the primary care setting in rural Maine. The study found patients had an overall positive experience with the service, whereas PCPs were generally supportive and optimistic for future impacts and indicated trouble accessing the screening service during office hours. Future studies are planned and could include assessments of real-world correlation of AI interpretation and eye-specialist assessment of DR severity, timeliness of referral follow-up, and impact on screening rates. Other areas of interest include cost-benefit analysis of AI versus eye-specialist screening programs. These generally positive findings support further studies into the use of AI technology for DR screening, particularly in areas with challenges to care access. The potential benefit of this innovative tool to the care of people with diabetes includes improving access to retinopathy screenings, enhanced self-management of diabetes, and a wider detection of vision-threatening retinopathy.