A Multi-Center Diabetes Eye Screening Study in Community Settings: Study Design and Methodology

Abstract

Purpose

Diabetes is the leading cause of new cases of blindness among adults aged 20–74 years within the United States. The Innovative Network for Sight Research group (INSIGHT) designed the Diabetic Eye Screening Study (DESS) to examine the feasibility and short-term effectiveness of non-mydriatic diabetic retinopathy (DR) screening for adults with diabetes in community-based settings.

Methods

Study enrollment began in December 2011 at four sites: an internal medicine clinic at a county hospital in Birmingham, Alabama; a Federally-qualified community healthcare center in Miami-Dade County, Florida; a university-affiliated outpatient pharmacy in Philadelphia, Pennsylvania; and a medical home in Winston-Salem, North Carolina. People 18 years or older with previously diagnosed diabetes were offered free DR screening using non-mydriatic retinal photography that was preceded by a brief questionnaire addressing demographic information and previous eye care use. Visual acuity was also measured for each eye. Images were evaluated at a telemedicine reading center by trained evaluators using the National Health System DR grading classification. Participants and their physicians were sent screening report results and telephoned for a follow-up survey 3 months post-screening to determine whether participants had sought follow-up comprehensive eye care and their experiences with the screening process.

Results

Target enrollment at each site was a minimum of 500 persons. Three of the four sites met this enrollment goal.

Conclusion

The INSIGHT/DESS is intended to establish the feasibility and short-term effectiveness of DR screening using non-mydriatic retinal photography in persons with diabetes who seek services in community-based clinic and pharmacy settings.

Introduction

Diabetes is the leading cause of new cases of blindness among adults aged 20–74 years in the United States.^1–4 In 2005–2008, 4.2 million (28.5%) of Americans with diabetes aged 40 years or older had diabetic retinopathy (DR) and of these, 655,000 (4.4%) had advanced DR that could lead to severe vision loss.¹ The number of people with DR is expected to increase more than 3-fold by 2050, creating an immense and costly public health problem.^4–7 Successful management of DR includes early diagnosis, tight glycemic and blood pressure control, and medical treatment.^8–12 Clinical trials demonstrating the efficacy of these interventions led the American Academy of Ophthalmology, the American Diabetes Association and the American Optometric Association to recommend annual dilated fundus examinations to reduce the risk of vision loss.^13–15 Despite the risk of vision loss, only 50–60% of people with diabetes, and even fewer in some low-income populations, follow this recommendation, and about 50% of persons are diagnosed too late for treatment to be optimally effective.^16–18

Screening for DR using a non-invasive, non-mydriatic (no dilating drops needed) fundus camera has become an accepted screening method for people with diabetes in many parts of the world.^19–21 These cameras provide digital fundus images that are graded for the presence of DR. The timely identification of ocular disease in people with diabetes can aid in successful management of DR and prevention of vision loss. Those who screen positive (have findings that are not normal) can then be referred for a comprehensive dilated eye examination and management.

The Innovative Network for Sight Research (INSIGHT) Diabetic Eye Screening Study (DESS) was designed to examine the feasibility and effectiveness of non-invasive, non-mydriatic DR screening for people with diabetes in community-based clinic and pharmacy settings, specifically primary care health clinics and pharmacies. The study targeted communities with large, underserved, minority populations at high risk of DR. The community-based approach to screening was chosen for its potential to reach and screen a larger population of those with diabetes than screenings performed in eye care provider offices or in other tertiary care settings.

Materials and methods

Study design overview

The purpose of the INSIGHT/DESS was to conduct a prospective, multi-site evaluation of the feasibility of DR screening with non-mydriatic retinal imaging in adults with diabetes in a variety of primary care clinic and pharmacy settings serving minority, high-risk populations. At baseline, we administered a questionnaire, measured presenting distance visual acuity, and performed non-mydriatic ocular imaging. Images were read at a telemedicine reading center, and results were mailed to patients and their primary care physicians. Three months after screening results were made available to participants, and participants were contacted by telephone for a follow-up questionnaire to evaluate their experience with the screening process and to determine subsequent eye care use. The tenets of the Declaration of Helsinki were upheld throughout this study and all participating centers received site-specific ethics approval. Written informed consent included consent to contact participants’ primary care physician.

Specific aims

The INSIGHT/DESS had five specific aims: (1) to determine the average number of individuals screened on a monthly basis; (2) to determine the rate of DR (by eye and individual), and corresponding severity; (3) to determine the rate of individuals who show signs of additional ocular pathology including cataract, glaucoma, or corneal conditions; (4) to determine the rate of individuals who schedule recommended follow-up eye care; and, (5) to determine individuals’ self-reported experiences with the screening process.

Organizational structure

The INSIGHT network includes four institutions, the University of Alabama at Birmingham (UAB), the University of Miami, Wills Eye Hospital (WEH), and Johns Hopkins University with Wake Forest University, collaborating with the Centers for Disease Control and Prevention (CDC) Vision Health Initiative in the Division of Diabetes Translation. The executive committee, which guided the study, consisted of the four principal investigators plus a CDC scientific collaborator. The Administrative and Coordinating Center and Telemedicine Department were based at WEH and the Data Management and Analysis Center was based at UAB.

Study setting and recruitment

DR screening was conducted at four locations beginning in December 2011. Major eligibility criteria included adults 18 years or older with previously diagnosed diabetes (type 1 or 2) who resided in one of the community locations. The recruitment process and unique setting of each site is described below and summarized in Table 1. All sites were chosen to target high-risk, minority populations with diabetes.

Table 1.

Overview of each site in the Innovative Network for Sight Research Diabetic Eye Screening Study, United States.

Site	Camera placement	State	Patients recruited, n	Majority race/ethnicity
University of Alabama at Birmingham	Internal medicine clinic at county hospital	Alabama	612	African American
University of Miami	Community health center	Florida	622	Hispanic/Latino
Wills Eye Hospital	Outpatient pharmacy at university	Pennsylvania	500	Caucasian
Johns Hopkins University	Outpatient medical home	North Carolina	180	African American

Site 1: University of Alabama at Birmingham

The UAB site was based in the Internal Medicine Clinic of Cooper Green Mercy Health System in Birmingham, Alabama. The health system, located in the most populated county in Alabama, is Alabama’s only county-owned, community health system with outpatient and urgent care clinics, offering care to all residents of Jefferson County regardless of ability to pay. The majority of patients have no health insurance. DR screening for the study was performed as part of usual care for diabetic patients who presented to the clinic. The vast majority of patients who seek care through the health system are African American.

Site 2: University of Miami

The University of Miami site was the Jessie Trice Community Health Center, a federally-qualified community healthcare center serving the uninsured or underinsured in Miami-Dade County, Florida. Participants, who had to be residents of Miami-Dade County, were recruited via flyers in the community, flyers posted at the community center, and referral by local physicians. English, Spanish and Haitian Creole speaking patients were eligible at this site.

Site 3: Wills Eye Hospital

The WEH site was an outpatient pharmacy in Philadelphia, Pennsylvania, located within the Jefferson Health System used primarily by patients and employees of Thomas Jefferson University. Pharmacy personnel directed individuals picking up diabetic medication to the screening location. Family practice physicians with offices in the same building as the pharmacy referred individuals to the study after being made aware of the program. Flyers were posted in the pharmacy and were given to family practice physicians to increase recruitment. Additionally, advertisements were placed in local newspapers. English and Spanish speaking individuals were eligible at this site.

Site 4: Johns Hopkins University

The initial site for John Hopkins University was a pharmacy in a rural town in New Mexico serving a population primarily of Hispanic or American Indian ethnicity. Due to inability to recruit patients, the site was changed to the Wake Forest School of Medicine-affiliated Downtown Health Plaza (DHP) located within downtown Winston-Salem, North Carolina, primarily serving low-income residents of the surrounding area. The DHP includes primary and specialty-care clinics as well as an onsite pharmacy. DHP physicians and pharmacy staff serving individuals with diabetes referred individuals to the screening study.

Sample size

The target sample size for the INSIGHT/DESS was a minimum of 500 at each site. The sample size was selected to allow for comparison of feasibility and effectiveness between study sites.

Study questionnaire and visual acuity

Following informed consent, a research staff member administered a questionnaire to each participant. Collected variables included participant contact information, sex, age, race, ethnicity, insurance status, time since diabetes diagnosis, primary care physician (PCP) visit in past year, hemoglobin A1c, timing of last dilated eye exam, smoking status, permission to contact PCP, request for eye care provider contact information, request for scheduling assistance, and history of cataract surgery. Walk-in distance visual acuity was measured for each eye with the participant’s current distance correction (if worn) using the Titmus V2 vision screener (Sperian Protection Optical Inc, Chester, VA, USA).

Ocular imaging

Following the questionnaire and walk-in distance visual acuity testing, ocular imaging was performed. A chin rest and forehead rest were used to position the individual in front of the camera. To minimize interference from room lighting, either a drape covering the participant’s head or darkened room was used during the imaging. Trained technicians used a non-mydriatic Nidek Model AFC-230 camera (Nidek Inc, Fremont, CA, USA) with auto-focus in accordance with the camera’s standard operating instructions. Three photos were taken of each eye (anterior segment, nasal fundus, and temporal fundus), for a total of six images per participant. If images were blurry or the participant blinked, additional images were taken until a satisfactory image was obtained. Images were generated using NavisLite software (Nidek Inc) and uploaded to a Health Insurance Portability and Accountability Act (HIPPA)-compliant secure website administered by the WEH Telemedicine Reading Center. Technicians were prompted to record a number of demographic and health variables into the online database at the time of photography.

Image grading

At the WEH Telemedicine Reading Center, a HIPPA-compliant proprietary software program was used for image reading and report generation for each site. The study’s grading system for DR was based on the National Health System DR grading classification system and is reproduced in Table 2 ²² We selected this system because it is widely used in telemedicine screening programs. Other ocular pathology was recorded as present or absent based on external and fundus images. Noted abnormalities included hypertensive changes, cotton wool spots, branch retinal vein occlusion, central retinal vein occlusion, branch retinal artery occlusion, central retinal artery occlusion, vitreous opacity, nevus, macular degeneration, scar, cataract, corneal changes, and cup-to-disc ratio >0.6. Certified image readers at the WEH Telemedicine Reading Center determined preliminary grades within 48 hours of image upload to the WEH secure website. All images showing signs of DR or other ocular pathology were further reviewed by an ophthalmologist at the Reading Center.

Table 2.

Innovative Network for Sight Research Diabetic Eye Screening Study image grade, description, and recommendations based on the National Health System Grading Classification System and American Academy of Ophthalmology Recommendations, United States.

Grade	Description	Recommendation
R0	No DR	Re-evaluate in 12 months with either eye care specialist or photographic screening
	None
	Isolated cotton wools spots (≥1) in the absence of any microaneurysm or hemorrhage
R1	Background DR	Refer to eye care provider
	≥1 microaneurysm(s)
	≥1 retinal hemorrhage(s)
	Any exudates caused by DR
R2	Pre-proliferative DR	Refer to ophthalmologist promptly
	Intra-retinal microvascular abnormality
	Venous beading
	Venous loop or reduplication
	Multiple deep, round or blot hemorrhages
R3	Proliferative DR	Refer to ophthalmologist promptly
	New vessels on the disc
	New vessels elsewhere
	Pre-retinal or vitreous hemorrhage
	Pre-retinal fibrosis with or without tractional retinal detachment due to DR
M	Maculopathy	Refer to ophthalmologist promptly
	Exudate within 1DD of the center of the fovea
	Circinate or group of exudates within the macula
	Any microaneurysm or hemorrhage within 1DD of the center of the fovea only if associated with a best visual acuity 20/40 or worse
P	Photocoagulation	Refer to eye care provider
	Focal/grid to macula
	Peripheral scatter
U	Unclassifiable/ungradable	Refer to eye care provider
	Due to poor photographic location, focus, or contrast

DR, diabetic retinopathy; DD, disc diameter.

Based on recommendations from the American Academy of Ophthalmology for DR follow-up, screening reports were automatically generated as determined by degree of DR (Table 2).¹³ Patients with any of the six images deemed unreadable due to poor photographic quality (focus, location, contrast, etc.) were advised to follow-up with repeat imaging or an eye care provider due to unreadable images.

Notification of findings and referrals

All study participants were notified of their examination results, and results were also sent to participants’ primary care provider if they had requested this at the time of screening. For individuals whose reports recommended prompt referral to an eye care provider due to DR or maculopathy (grades R2, R3 and M in Table 2), a site-specific research coordinator telephoned the participant within 48 hours of receiving the report to advise them of the recommendation. The coordinator also offered the participant assistance with scheduling an appointment with an eye care provider. Up to five attempts were made to reach the participant by phone. A letter was also mailed to the participant and the participant’s primary care provider (when consent was previously given) with the results and recommendations of the screening.

For participants whose reports recommended normal (non-urgent) referral or follow-up (grades R0, R1, P and U on Table 2), a letter was sent to the participants describing the screening results. For abnormal screening results designated for normal referral (grades R1, P and U), the letter encouraged the participant to seek an appointment for a dilated eye examination “within the next few months.” For normal screening results (grade R0), the letter encouraged the participant to seek an appointment for a dilated eye examination on an annual basis.

Telephone follow-up questionnaire

Site-specific research staff administered a follow-up questionnaire by telephone approximately 3 months after the screening results were distributed to each participant. The follow-up time frame was selected to give participants ample opportunity to follow-up with their primary care physician and/or eye care provider, should they choose to do so. The questionnaire addressed participants’ experience with the screening process, whether they would be willing to pay for the screening in the future, their recommendations for improving the screening, what they were told about the outcomes of their screening, and follow-up eye care use or plans for eye care use since the screening. At least five telephone-call attempts were made to the participant before the participant was designated as unable to be reached for questionnaire administration.

Data management

At the time of the initial retinal imaging and questionnaire, a unique numerical identification was generated for each participant. Each site entered participant data from screening forms and follow-up questionnaires into a secure web-based data entry system stored at the Data Management and Analysis Center. Each site implemented technician training and quality control procedures to ensure data were correctly entered in this master database.

All images with their corresponding identification number and all fields from the DR screening report were uploaded and securely stored at the Reading Center. Once enrollment closed and all reports were generated, the report results were electronically transmitted to the Data Management and Analysis Center for merging with data from the screening forms and follow-up questionnaires. Site-specific databases were made available at each site to facilitate analysis of site-specific data. A network-wide analysis of data was performed to address network-wide aims.

Results

Target enrollment at each site was a minimum of 500 persons. Three of the sites achieved or exceeded this minimum while the remaining site enrolled 180 persons. Details for each site can be found in Table 1. The majority of patients participating were female (63.1%) and of a racial/ethnic minority (88%). While there were site differences in populations, the mean age at each site was similar at 53–55 years. Overall, 12% of patients had one or more images that were deemed ungradable.

Overall, recruitment by site tended toward older median age and female population (Table 3). Additionally, African Americans made up the largest racial/ethnic group at all sites except 1, which had a larger Hispanic/Latino population. All sites recruited more under/uninsured population than the county demographic average. The reported rate of dilated eye examinations within the past year by site ranged from 25.5–52.8%.

Table 3.

Demographic and insurance results of the Innovative Network for Sight Research Diabetic Eye Screening Study, United States; comparison of recruitment sites to county data.

	Miami, Florida		Birmingham, Alabama		Winston-Salem, North Carolina		Philadelphia, Pennsylvania
	Miami-Dade County	Study site	Jefferson County	Study site	Forsyth County	Study site	Philadelphia County	Study site
Population, n	2,496,435	608	658,466	600	350,670	180	1,526,006	506
Health insurance coverage, %	70.2	22.6	87.2	29.5	82.5	51.7	85.5	79.2
Median age, years	38.2	56.0	37.1	54.0	37.2	56.0	33.5	54.0
Female, %	51.6	65.6	52.6	65.5	52.5	66.3	52.8	43.8
Race/ethnicity, %
White	73.8	1.3	53.0	14.5	97.8	21.1	41.0	18.8
Black or African American	18.9	33.9	42.0	84.3	26.0	68.9	43.4	68.2
American Indian and Alaska Native	0.2	0	0.3	0	0.4	0.6	0.5	0
Asian	1.5	0.3	1.4	0.5	1.9	0	6.3	4.6
Hawaiian and other Pacific Islander	0	0	0	0	0.1	0.6	0	0
Hispanic	65.0	41.1	3.9	0.3	11.9	8.3	12.3	2.8
Haitian	Not listed	11.5	Not listed	0	Not listed	0	Not listed	0.2
Cuban	34.3	11.0	0.1	0	0.2	0	0.3	0.2
Othera	2.4	0	1.1	0.3	2.2	0.6	2.8	5.3

^aMutiracial or no data available.

US Census Bureau: State and County QuickFacts (2010 data except health insurance 2009–2013 5-Year American Community Survey). http://quickfacts.census.gov, accessed February, 2015.

Quality assurance

Several measures were implemented to ensure quality of the data collection and image reading. Staff members and telemedicine specialists trained technicians at each site to administer the questionnaire and to operate the camera. Staff members observed as technicians administered the questionnaire and photography on a semi-weekly basis. To ensure that data were correctly entered, a double-entry system was used.

To ensure consistent grading quality, 10% of images labeled normal by the readers were randomly selected and reviewed by ophthalmologists to ensure correct classification of normal images. No images labeled normal by the image readers were found to have signs of pathology. To evaluate intra-grader reliability (repeat grading by the same person), 138 patients (828 images, both normal and with pathology) were randomly selected for re-evaluation. The proportion of these images found to have DR mirrored that of the overall study population, with roughly 25% of individuals having some level of DR (Table 4). The image reader who graded the original images re-graded these selected images, masked to her previous grades, with a minimum of 3 months between her first and second grade. The kappa coefficient was used to determine the level of agreement between the original and second grades, corrected for agreement by chance alone. The intra-grader kappa coefficient for DR findings was 0.72, (95% confidence interval, CI, 0.62–0.83) with 88.8% agreement. Metrics on variability of each grader over time was not performed, as all readers had years of experience.

Table 4.

Final results from fundus image subset evaluated for quality control, Innovative Network for Sight Research Diabetic Eye Screening Study, United States.

Diabetic retinopathy diagnosis	n (%)
	(N = 138 patients)
No DR	208 (75.4)
Background DR	36 (13.0)
Pre-proliferative DR	2 (0.7)
Proliferative DR	4 (1.5)
Ungradable	26 (9.4)

DR, Diabetic retinopathy.

The same procedure and images were used to evaluate inter-grader reliability. The inter-grader kappa coefficient for DR findings was 0.62 (95% CI 0.51–0.73) with agreement of 84.1%.

Discussion

The INSIGHT/DESS represents a uniquely designed, multi-center study evaluating diabetic eye screening using a non-mydriatic camera in community-based settings in the US. Strengths of the study include the standardization of the screening protocol and questionnaire, use of a single central reading center, and inclusion of multiple geographic locations and racial/ethnic populations, and different community-based settings (e.g. clinics, pharmacy). Quality assurance measures allowed for accurate data entry and image reading. Locating the study within different clinical and pharmacy settings will allow for evaluation of feasibility and effectiveness across different care settings. The urban settings proved to be more feasible, in this study, than the more rural screening site. Both outpatient clinics and the outpatient pharmacy setting were able to recruit and screen the desired number of patients over the study time frame. However, the out-patient clinic venues had the greatest success. The sites primarily seemed to recruit from the underserved, as reflected in the low health care coverage compared to the counties’ coverage. We hypothesize that these urban settings could potentially sustain long-term screening and follow-up tele-ophthalmic diabetic evaluations. In a more rural setting, perhaps a different approach would better serve the community and provide better use of resources.

Limitations of the study include its reliance on participant self-reporting information such as time since diabetes diagnosis and eye care use, the potential of poor image quality leading to missed diagnosis of pathology, and loss to follow-up for the telephone survey. Data on qualified individuals who declined screening were not collected, nor do we know what proportion of individuals with diabetes in these communities had access to the study. It is possible that the study may not be generalizable to other regions, racial populations, or care settings. While based on the National Health Service grading classification system and American Academy of Ophthalmology follow-up recommendations, the study’s grading and recommendation methodology have not been previously validated.

In collaboration with the CDC Vision Health Initiative, the INSIGHT network is evaluating the effectiveness and feasibility of diabetic eye screenings in primary care clinics and pharmacies, targeting a diverse, medically underserved portion of the diabetic population that often does not undergo annual DR screening. The DESS will provide information on the prevalence and severity of diabetic eye disease and additional ocular pathology for individuals in these community-based clinic and pharmacy settings. Results of the study can potentially be used by public health researchers and clinicians in studies, as well as used on a large scale for underserved populations.