Cost-effectiveness of Autonomous Point-of-Care Diabetic Retinopathy Screening for Pediatric Patients With Diabetes

Key Points

Question

Is autonomous artificial intelligence diabetic retinopathy screening more cost-effective than standard eye care screening examination performed by health care professionals?

Findings

This economic evaluation used decision analysis to model the cost-effectiveness of detecting and treating diabetic retinopathy and its sequelae among children with type 1 and type 2 diabetes and found an incremental cost-effectiveness ratio of $31 for type 1 diabetes and $95 for type 2 diabetes for each additional case of diabetic retinopathy identified compared with standard practice.

Meaning

These results suggest that when more than 23% of patients adhere to diabetic retinopathy screening recommendations, autonomous artificial intelligence screening is the preferred strategy and cost-saving for the patient and family.

Abstract

Importance

Screening for diabetic retinopathy is recommended for children with type 1 diabetes (T1D) and type 2 diabetes (T2D), yet screening rates remain low. Point-of-care diabetic retinopathy screening using autonomous artificial intelligence (AI) has become available, providing immediate results in the clinic setting, but the cost-effectiveness of this strategy compared with standard examination is unknown.

Objective

To assess the cost-effectiveness of detecting and treating diabetic retinopathy and its sequelae among children with T1D and T2D using AI diabetic retinopathy screening vs standard screening by an eye care professional (ECP).

Design, Setting, and Participants

In this economic evaluation, parameter estimates were obtained from the literature from 1994 to 2019 and assessed from March 2019 to January 2020. Parameters included out-of-pocket cost for autonomous AI screening, ophthalmology visits, and treating diabetic retinopathy; probability of undergoing standard retinal examination; relative odds of undergoing screening; and sensitivity, specificity, and diagnosability of the ECP screening examination and autonomous AI screening.

Main Outcomes and Measures

Costs or savings to the patient based on mean patient payment for diabetic retinopathy screening examination and cost-effectiveness based on costs or savings associated with the number of true-positive results identified by diabetic retinopathy screening.

Results

In this study, the expected true-positive proportions for standard ophthalmologic screening by an ECP were 0.006 for T1D and 0.01 for T2D, and the expected true-positive proportions for autonomous AI were 0.03 for T1D and 0.04 for T2D. The base case scenario of 20% adherence estimated that use of autonomous AI would result in a higher mean patient payment ($8.52 for T1D and $10.85 for T2D) than conventional ECP screening ($7.91 for T1D and $8.20 for T2D). However, autonomous AI screening was the preferred strategy when at least 23% of patients adhered to diabetic retinopathy screening.

Conclusions and Relevance

These results suggest that point-of-care diabetic retinopathy screening using autonomous AI systems is effective and cost saving for children with diabetes and their caregivers at recommended adherence rates.

This economic evaluation assesses the cost-effectiveness of detecting and treating diabetic retinopathy and its sequelae among children with type 1 diabetes and type 2 diabetes using artificial intelligence diabetic retinopathy screening vs standard screening by an eye care professional.

Introduction

Youths and adults with diabetes are at risk for diabetic retinopathy (DR), which can lead to vision loss.¹ The prevalence of DR among youths with type 1 diabetes (T1D) and type 2 diabetes (T2D) ranges from 4% to 13%.^2,3,4 Despite recommendations from the American Diabetes Association and American Academy of Ophthalmology for yearly screening, adherence remains low.⁵ Although the prevalence is low among youths, the risk of developing DR is high. Diabetic retinopathy is present in up to 50% of patients with T1D 28 years or more after diagnosis^6,7 and already present at the time of diagnosis in 12% to 19% of patients with T2D.⁴

Digital teleophthalmology systems that use nonmydriatic cameras have been implemented to improve DR screening rates^8,9 and are cost-effective.¹⁰ Digital fundus photography can be efficiently and safely performed without pupil dilation, including in the pediatric setting.^{3,11,12,13,14} Recently, the US Food and Drug Administration (FDA) approved the first autonomous artificial intelligence (AI) diagnostic system to detect DR in adults. With this system, a minimally trained operator guided by an image-quality AI takes retinal images with a nonmydriatic fundus camera, and these images are subsequently assessed in real time at the point of care (POC) for the presence or absence of DR.¹⁵ In 2020, this form of FDA-validated autonomous AI became part of the American Diabetes Association standard of care for DR screening.¹⁶

Our goals were to develop a patient-oriented decision model to estimate the cost savings of using autonomous AI DR screening in youths compared with a clinic-based ophthalmoscopy examination by an eye care professional (ECP). Using decision analysis, we modeled the cost-effectiveness of detecting and treating DR and its sequelae among children with diabetes. We also approached this model from the perspective of the patients and their caregivers to evaluate expected cost savings because lower cost sharing by the patient is associated with increased adherence and improved outcomes.¹⁷

Methods

In this economic evaluation, we developed a decision analysis model in which the number (proportion) of DR cases ascertained was defined as effectiveness and patient and family out-of-pocket cost was defined as the cost. This study followed recommendations for the publication of technology assessments¹⁸ and economic evaluations¹⁹ and was performed from March 2019 to January 2020. This study did not require institutional review board review because it did not involve human participant research.

This model assumed that a child (age <21 years) previously diagnosed with T1D or T2D was under regular care by a pediatrician or pediatric endocrinologist and began with no known eye disease. The perspective was the family’s, meaning that out-of-pocket costs and similar burdens were the focus. This model did not incorporate the cost of time off from work for parents and missed school days for children or the cost to payers, parents’ employers, or other third parties. The mode had 2 alternatives: autonomous AI retinopathy screening and clinician-based screening examination performed by an ECP (ophthalmologist or optometrist). Because screening is recommended with an annual frequency,²⁰ 1 year was the interval of focus.

Outcomes

The focus of the model was the diagnosis of clinically actionable, referable DR. The outcome is expressed as probability of having DR (true positive) given a screening strategy. We did not consider downstream (>1 year) morbidity or costs associated with DR except for an extra ECP visit within the year for those diagnosed with DR by AI screening. This choice was biased against AI screening examinations because early detection would appear as being more costly owing to this extra ECP visit, whereas the alternative would not benefit from cost savings downstream when more severe DR is diagnosed in patients who were not screened.

Statistical Analysis

TreeAge software, version 2020 (TreeAge Software LLC) was used for this analysis. When choices were made, we biased the model against autonomous screening.

Decision Tree

A simple tree (Figure 1) was the best model structure for this problem because (1) the decision alternatives were simple, (2) there was no need to model change in disease over time, and (3) there was a simple time horizon.²⁰ According to the assumptions of the model, youths with diabetes have the options of standard ophthalmologic (ECP) or autonomous AI screening for DR. With either of these alternatives, the possibility exists that they do not get screened (lack of adherence). If a patient chose autonomous AI screening and tested positive for DR, they were referred to an ECP for further examination; therefore, the model assumes that patients who screen positive by autonomous AI also incur the cost of an ophthalmologic examination. Table 1 shows the parameters of the tree.

Figure 1. Schematic of Decision Tree.

Patients enter the model with the following options: autonomous artificial intelligence (AI) or eye care professional (ECP) screening. In the latter, they undergo or do not undergo an examination. The screening result may be positive or negative, with concomitant outcomes. If they do not undergo an examination, they either have or do not have diabetic retinopathy. If the AI screen is not diagnosable or the result is positive, patients are referred to an ECP; otherwise, outcomes are as indicated. The payoffs in this figure focus on true-positive results and out-of-pocket costs for the examination and, in the case of a positive ECP examination result, downstream treatment of diabetic retinopathy (1 visit).

Table 1. Parameter Values for the Decision Model.

Parameter	Base case estimate	Sensitivity		Reference
		Low	High
Out-of-pocket cost, $
Autonomous AI	0	0	100	15
ECP visit	35	0	500	21
Treating retinopathy	94	0	10 000	22,23
Probability of a patient pursuing ECP screening	0.20	0.15	0.90	9,24,25,26
Probability that a patient with a positive screening result will follow up with an ECPa	0.95	0	0.95	27
Prevalence of retinopathy in the population
Type 1 diabetes	0.09	0.06	0.14	1,2,3,4,28
Type 2 diabetes	0.14	0.09	0.50
Relative odds of undergoing autonomous AIb	76	0.5	100	9,26
Sensitivity
Autonomous AI	0.87	0.001	0.99	15
ECP examination	0.35	0.001	1.00	29,30
Specificity
Autonomous AI	0.91	0.001	0.99	15
ECP examination	0.95	0.001	1.00	29,30,31
Diagnosability of autonomous AIc	0.96	0.80	0.99	15

Abbreviations: AI, artificial intelligence; ECP, eye care professional.

^aProbability is expressed as a function of the probability of obtaining ECP screening and the relative odds of seeking care from an ECP if the AI screening result is positive vs being referred in the first place. That is, the probability of ECP screening of 0.2 = odds of 0.25 × 76 = odds of 19 = probability of 0.95 = probability that a patient with a positive AI screening result will follow up with an ECP.

^bWith respect to ECP screening.

^cDiagnosability is defined as the fraction of all patients undergoing any type of screening who receive a valid diagnostic result (negative or positive result for diabetic retinopathy).

Out-of-Pocket Costs

In both AI and ECP screening, patients incur the cost of subspecialty care with their endocrinologist, but because the automated AI screening is performed as a part of the endocrinology visit, there is no additional copayment for DR screening; thus, the base case cost for AI screening is $0. A high cost of $100 is included for AI screening because there may be additional charges to the patients depending on their insurance coverage and deductible.²¹ However, when a patient seeks care from an ECP, an additional copayment is assumed for subspecialty care; thus, the base case cost was $35. The estimated out-of-pocket cost in the model for an ECP visit ranged from $35 to $500 based on the copayment amount and insurance deductible. Because of the short time horizon, no discount rate was used.

The medical cost of treating DR is estimated to range from $0 to $10 000 depending on age of the patient and severity of disease^22,23 and based on treatment for 1 year. This range assumes that most (80%) patients will not visit the ECP for treatment ($0), some will require a few extra ECP visits for treatment, and a small percentage (12%) will require close ECP follow-up and monthly DR treatment for 1 year (12 visits). In the baseline scenario, we estimated that 8% of patients would possibly reach a maximum deductible of $600 (maximum of $10 000 in sensitivity analysis). In the model, we used a base case value of $94, which was weighted by the probabilities of DR severity and the consequent likelihood of treatment.³²

Screening

The prevalence of DR is reported to be 5.6% among youths with T1D and 9.1% among youths with T2D,¹ with other studies^2,3,28 suggesting rates ranging from 4% to 13%. The sensitivity analyses used broad ranges.

Adherence to regular DR examinations in adults is as low as 15% and as high as 80%.^9,24,25,26 In youths, DR screening adherence ranges from 35% to 72%.⁵ A previous study²⁷ found that with positive findings on routine ophthalmologic examination in adults with diabetes, only 29% had appropriate follow-up recorded compared with a telehealth screening group in which 95% had appropriate ECP follow-up recorded after screening positive for DR.

Individuals are more likely to undergo examination with POC DR screening compared with traditional referral systems. Furthermore, patients who have positive findings on POC screening are more likely to obtain an ECP examination.^{8,9,26,27,33,34} The base case value of 76 matches the relative odds of autonomous screening (probability of 0.95 equals odds of 19) to that of ECP screening (probability of 0.20 equals odds of 0.25) (19/0.25 = 76). Use of relative odds rather than raw probabilities would give meaningful sensitivity analysis results.³⁵

Screening by autonomous AI is more sensitive (87%) than the reported sensitivity (33%-34%) of ECP screening examinations.^15,29,30 The specificity of autonomous AI is 91% compared with 95% for ECP dilated eye examination.^15,29,30 A previous study³¹ that evaluated the specificity of telemedicine interpretation of retinal images by optometrists, ophthalmologists, and retinal specialists reported a range of specificity of 70% to 100%.

Diagnosability is defined as the fraction of all patients undergoing any type of screening who receive a valid diagnostic result (negative or positive result for DR) rather than an output (eg, cannot determine or insufficient image quality to determine). The diagnosability of the autonomous AI system was 96%, whereas the diagnosability for an ECP was 95% to 100% when using indirect ophthalmoscopy.³¹ In our model, if an AI examination was not diagnosable, the child was referred to an ECP. The probability of follow-up was assumed to be the same as if the examination result were positive.

Outputs

The decision tree (Figure 1) produces expected values: outcomes (DR identified or dollars spent) were multiplied by their probabilities of occurrence, those products were added at a chance (circular) node, and the process continued upstream to the first node of the strategy (autonomous AI or ECP). A true-positive result is the expected proportion of all patients who would test true positive (true-positive proportion [TPP]), which is also the screening effectiveness. Because all patients comprise the denominator, the TPP will be smaller than sensitivity. The highest value that the TPP can attain is the prevalence, either 0.09 (T1D) or 0.14 (T2D). In the case of costs, the calculation is the expected mean payment after adjustment for adherence to the patient for either strategy. This calculation is called the patient payment.

The incremental cost-effectiveness ratio (ICER) is defined as the mean patient payments required to find 1 additional case of DR (a true-positive result) beyond what the current ECP strategy would find. The ICER represents the difference in cost divided by the difference in effect between the 2 options.

After base case analyses, we performed sensitivity analyses (eAppendix in the Supplement) to evaluate whether the conclusions were sensitive to the range of values available in the literature. Sensitivity analysis in cost-effectiveness studies is the same as assessing the effect of uncertainty that CIs have in evaluating a point estimate: if the base case value of a parameter is far from the threshold value derived by the decision model, the conclusion is that uncertainty in the value of that parameter does not affect the conclusion of the cost-effectiveness model.

Results

Analysis was performed using the base case assumptions described above that incorporated the probabilities and thresholds listed in Table 1. The resulting cost-effectiveness analysis is given in Table 2.

Table 2. Cost-effectiveness of Standard ECP Screening vs Autonomous AI Screening in Patients With Type 1 and Type 2 Diabetes.

Screening	Patient payment for each strategy, mean, $	Difference in cost, $	Effectiveness, TPP	Difference in effect	ICER, $
Type 1 diabetes
ECP screening	7.91	0.61	0.006	0.02	31
Automated AI screening	8.52		0.03
Type 2 diabetes
ECP screening	8.20	2.65	0.01	0.03	95
Automated AI screening	10.85		0.04

Abbreviations: AI, artificial intelligence; ECP, eye care professional; ICER, incremental cost-effectiveness ratio (difference in cost divided by difference in effect); TPP, true-positive proportion.

Effectiveness was defined as the TPP (Table 3). For T1D, the expected TPP was expected to be 0.006 (of the prevalence of 0.09) for ECP screening and 0.03 by autonomous AI. For T2D, the expected TPP was 0.01 for the ECP strategy and 0.04 for autonomous AI. The low ECP TPP was attributable to the low probability (0.20) of patients keeping their ECP appointment.

Table 3. Summary of the Expected Values for the Strategies and Disease Conditions.

Variable	Type 1 diabetes				Type 2 diabetes
	Autonomous AI		ECP		Autonomous AI		ECP
	DR	No DR	DR	No DR	DR	No DR	DR	No DR
Absolute proportions
Positive examination result	0.03	0.01	0.006	0.009	0.04	0.01	0.01	0.009
Negative examination result	0.06	0.90	0.08	0.90	0.10	0.85	0.10	0.85
Total	0.09	0.91	0.09	0.91	0.14	0.86	0.14	0.86
Relative ratesa
Positive examination result	0.29	0.01	0.07	0.01	0.28	0.01	0.07	0.01
Negative examination result	0.71	0.99	0.93	0.99	0.72	0.90	0.93	0.99
Total	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00

Abbreviations: AI, artificial intelligence; DR, diabetic retinopathy; ECP, eye care professional.

^aSensitivity and specificity.

On the basis of the cost-effectiveness analysis, in the base case scenario, out-of-pocket mean patient payment by ECP screening was expected to be $7.91 for patients with T1D and $8.20 for those with T2D. The mean patient payment for AI was estimated to be higher, at $8.52 for T1D and $10.85 for T2D. The source of the variation in mean patient payment was explored in the sensitivity analysis. In the base case (20% adherence scenario), the ICERs were $31 (T1D) and $95 (T2D).

Although all variables were tested (eAppendix in the Supplement), only the sensitivity analysis (Table 2 and Figure 2) for adherence found a meaningful threshold effect. Because adherence with the ECP examination increased above the base case value of 20% and more patients attended their DR screening, the mean patient payment increased also and the ECP examination became relatively more expensive (Figure 2). For the threshold of 23% of patients visiting an ECP and adhering to the diabetic eye examination, autonomous AI screening was associated with cost savings and detected more cases (difference in true-positive rate of 0.027 in favor of autonomous AI). In other words, the threshold whereby each strategy led to the same mean patient payment was at 0.23 adherence for parents taking their child to the ECP or having the AI screening performed. As adherence increased, more patients obtained ECP screening, and the mean patient payment for ECP testing continued to increase. For example, in a 50% adherence scenario, AI would be cost saving as the ICER is negative (−$953); thus, AI screening would dominate ECP screening. Autonomous AI was both cost saving and more effective. Overall, as the probability of patients obtaining ECP screening increased, the ICER increasingly favored AI screening.

Figure 2. One-way Sensitivity Analysis for Probability of Eye Care Professional (ECP) Screening.

AI indicates artificial intelligence; ICER, incremental cost-effectiveness ratio.

Discussion

Diabetic retinopathy screening programs that use autonomous AI recently became available and were cleared for use in the US by the FDA. We present, to our knowledge, the first cost-effectiveness analysis of the use of AI screening for pediatric diabetes from the patient and caregiver perspective. We found that, based on published data, autonomous AI is more effective (as shown by the higher TPPs: 0.03 for T1D and 0.04 for T2D for autonomous AI vs 0.006 for T1D and 0.01 for T2D for ECP) and cost saving than standard referral-based strategies of ECP screening examinations. Although the model showed that autonomous AI screening had higher mean patient payment in the baseline adherence scenario, that higher payment was attributable to the poor adherence of patients in visiting the ECP when referred; therefore, on average, few families may be paying for these visits. Through sensitivity analyses, we found that if more patients were to actually visit the ECP, the mean ICER would decrease, and AI would be more cost saving for the patient and family. Quality of care measures, such as the Healthcare Effectiveness Data and Information Set created by the National Committee of Quality Assurance and the Merit-Based Incentive Payment System created by the Centers for Medicare & Medicaid Services, require at least 80% of patients with diabetes to receive screening to meet the quality measure. At this adherence level and greater, the mean ICER is negative and autonomous AI saves costs. No other variables demonstrated a threshold above or below where ECP screening would dominate.

This analysis is a patient-perspective model and did not address system, societal, or other third-party costs. Convenience was not explicitly modeled in this analysis but can be inferred from the autonomous AI POC convenience and immediate result. At the child’s diabetes clinic appointment, the parent has a choice to complete the AI-based DR screening at that visit with no additional copayment or accept a referral for an ECP examination in the future with an additional copayment and time off from school and work. Although autonomous AI may be more convenient for the patient, we biased the cost-effectiveness model against autonomous AI. With the implementation of AI in the endocrinology clinic (rather than a referral), more people can be screened, and thus both the system cost of screening all patients and the mean patient payment increase as more patients follow through on screening recommendations and pay for screening. Furthermore, the model included adding the mean patient payment of the ECP screening examination to the autonomous AI patient payment because individuals with positive findings on AI screening (true-positive or false-positive results) will be referred for an ECP examination. Because the prevalence of DR is low among pediatric patients with diabetes, it is unlikely that children will have a positive screening result from autonomous AI and have to undergo an ophthalmologic evaluation.

According to the cost-effectiveness analysis, the mean patient payment of ECP screening was similar for T1D ($7.91) and T2D ($8.20) at the baseline scenario, with 20% adherence. However, the patient payment of autonomous AI screening was greater for youths with T2D ($10.85) than those with T1D ($8.52) in this baseline scenario. This difference may be associated with the prevalence of DR being higher among youths with T2D compared with youths with T1D. Youths with T2D would have a positive result more often from the autonomous AI and, therefore, would be more likely to incur the additional cost of an ECP visit. The ICER was higher for youths with T2D ($95) because there was a higher prevalence of T2D and thus a larger number of children referred for ECP screening, which incurs cost.

In this model, AI was assumed to increase the probability of an individual undergoing screening and paying for it compared with baseline screening practices. Although the out-of-pocket cost for an individual patient for an ECP visit was estimated to be higher than for an in-office POC autonomous AI screening because the model was biased toward ECP screening, ECP screening is the preferred option in the low-adherence baseline scenario. The sensitivity analysis showed that, when adherence with recommended ECP screenings increased, the mean patient payment of standard ECP examinations increased and the mean patient payment for autonomous AI decreased. From the patient perspective, POC autonomous AI screening is advantageous because it can be performed in the diabetes clinic setting, saving the patient and their caregiver another physician visit in most cases. Furthermore, the demonstrated sensitivity of autonomous AI screening was superior to that of the ECP examination.^15,29,30 The benefit of POC DR screening has been shown in other centers, with more patients undergoing screening and those who screen positive being more likely to attend ophthalmologic follow-up.^9,26,27 For patient care and improved long-term outcomes, this strategy is superior to the high rates of nonadherence with ECP screening examinations.²⁵ Models of cost sharing and adherence in diabetes care demonstrate that lower out-of-pocket costs and reduced cost sharing improves adherence and outcomes, further supporting this cost-effective strategy.^17,36,37

Limitations

This study has limitations. In contrast to other models of DR screening that incorporate long-term complications, costs of vision loss, and quality-adjusted life-years saved, we did not assess downstream costs of complications because these occur later in the pediatric population. This analysis only focused on DR, and there may be other advantages for a child to see an ophthalmologist if there are other vision concerns. Furthermore, because this analysis represents the patient perspective, we did not evaluate system or societal costs of technology implementation or personnel or office finances; these costs should be considered in further analysis and modeling.

Conclusions

The results suggest that DR screening using autonomous AI systems is effective and cost saving for children with diabetes and that use of POC screening with immediate results will increase DR screening rates, thus improving care and providing early identification of disease. Future cost-effectiveness and cost-savings analysis models should analyze the use of these innovative AI systems from the health care system perspective.

Supplement.

eAppendix. Peds Automated DR Screening