Medical Record Validation of Self-Reported Eye Diseases and Eye Care Utilization among Older Adults

Abstract

Purpose

Vision impairment is an important public health concern. Accurate information regarding visual health and eye care utilization is essential to monitor trends and inform health policy interventions aimed at addressing at-need populations. National surveys provide annual prevalence estimates but rely on self-report. The validity of self-reported information regarding eye disease has not been adequately explored.

Methods

This cross-sectional study compared self-report of eye care utilization and eye disease with information obtained from medical records. The study population was 2,001 adults aged 70 years and older who completed the Behavioral Risk Factor Surveillance System’s Visual Impairment and Access to Eye Care Module. Cohen’s kappa (κ) was used to assess agreement.

Results

Agreement between self-report and medical records was substantial for eye care utilization (κ=0.64) and glaucoma (κ=0.73), moderate for macular degeneration (κ=0.40) and diabetic retinopathy (κ=0.47), and slight for cataracts (κ=0.18). Self-report tended to overestimate the number of subjects who visited an eye care provider in the previous year, and underestimated the prevalence in all but one (glaucoma) of the four eye diseases evaluated.

Conclusions

Though agreement was substantial for self-report of eye care utilization, results of the current study suggest that national estimates based on self-report overestimate eye care utilization.

INTRODUCTION

Vision impairment is an important public health concern that affects more than 3.6 million Americans age 40 and older and results in significant disability and annual economic costs estimated at over $51 billion.^1,2 The most common age-related eye diseases are age-related macular degeneration, cataract, diabetic eye conditions and glaucoma.¹ As the population ages, the number of Americans at risk for age-related eye diseases is increasing.³ Moreover, the prevalence of diabetes in the United States (US) has more than doubled over the past 20 years,⁴ and is expected to continue to grow into the foreseeable future,⁵ dramatically increasing the numbers at risk for diabetic retinopathy. Thus, accurate information on visual health and eye care utilization is essential to monitor trends and inform health policy interventions aimed at addressing the needs of these populations. Recent US general population prevalence estimates of visual impairment and utilization of eye care services have relied on national population based surveys, including the National Health and Nutrition Examination Survey (NHANES),^6,7 which includes an eye examination, the Behavioral Risk Factor Surveillance System (BRFSS)⁸ and the National Health Interview Survey (NHIS).⁹ Both the BRFSS and NHIS are based on self-report; however, the reliability and validity of self-reported visual impairment and utilization of eye care services information has not been fully explored.

Relatively few investigations have examined self-reported vision related outcomes with respect to validity.^10–15 High self-report accuracy has been shown for cataracts among a large nationally representative sample.¹¹ Among a cohort of Latinos 40 and older, low sensitivity and high specificity were reported for four self-reported eye diseases (macular degeneration, cataract, diabetic eye conditions and glaucoma).¹⁵ Among diabetic populations, slight and fair agreement for the frequency of eye care utilization has been reported.^13,14 Previous studies examining agreement between patient self-report and information obtained from medical records have suggested a number of patient and provider characteristics that potentially influence agreement.^10–28 Agreement has been shown to be high for some conditions, e.g., diabetes mellitus,^17,21,22,25 but poor for others, e.g., arthritis.²⁵ Poor agreement has been shown to be influenced by patient factors including increasing age,^21–23 male gender,^22,23 non-White race,^13,26,27 low educational attainment,^11,22 impaired cognitive function,²¹ and increasing number of comorbidities;^21–23 still, other research has reported that many of these factors do not play a significant role in the accuracy of self-reported medical information.^{11,16,20,25,28}) There is continued uncertainty regarding the potential limitations of self-reported vision health information. The study’s objective is to evaluate the reliability of self-reported eye care utilization and vision health information obtained from older adults who completed the BRFSS Visual Impairment and Access to Eye Care Module.

METHODS

The research followed the tenets of the Declaration of Helsinki. Informed consent was obtained from study subjects after explanation of the nature and possible consequences of the study. The study was approved by the Institutional Review Board of The University of Alabama at Birmingham.

Study population

The study sample consisted of participants in a population-based, prospective cohort study on older drivers in north central Alabama (Owsley et al., submitted 2012). The source population for the study sample consisted of adults aged ≥ 70 years old who resided in Jefferson County, Alabama or the border areas of contiguous counties. Potential participants were randomly identified from contact information available through a list of persons obtained from a direct marketing company (Pinpoint Technologies, Tustin CA).²⁹ We then confirmed driver’s license status through the Alabama Department of Public Safety, and eliminated those from the target population who did not hold Alabama licenses. Potential participants were randomly selected from the final list, received a letter, and were telephoned to confirm that they were eligible to participate. Persons who stated they had an Alabama license and had driven within the last three months, were ≥ 70 years old, and spoke English were invited for participation. Participants were enrolled between October 2008 and August 2011. Eighty-four percent of those who met eligibility criteria consented to participate.

Demographic, general health, and co-morbidity information was obtained during an in-person interview. All participants completed the 2009 BRFSS vision module which consists of nine questions aimed at assessing visual impairment, eye disease, access to eye care, eye care insurance, and eye examination; in addition, the BFRSS diabetes module question regarding retinopathy was included. Table 1 lists the specific eye disease and access to eye care questions considered in the current study. Subjects were asked permission for copies of their eye medical records to be obtained and provided contact information (name, telephone number and business address) for their eye care provider (ophthalmologist or optometrist). A researcher highly experienced at abstracting eye medical records and masked to the responses provided by subjects reviewed medical records and extracted information about the date of the most recent examination and diagnoses.

Table 1.

Vision health questions administered to study participants

Module/question	Multiple choice responses
Visual Impairment and Access to Eye Care
When was the last time you had your eyes examined by any doctor or eye care provider?	Within the past month (anytime less than 1 month ago) Within the past year (1 month but less than 12 months ago) Within the past 2 years (1 year but less than 2 years ago) 2 or more years ago Never Don’t know / Not sure Refused
Have you EVER been told by a doctor or other health care professional that you:
NOW have cataracts?*	Yes No Don’t know / Not sure Refused
Had glaucoma?
Had age-related macular degeneration?
Diabetes
Has a doctor ever told you that diabetes has affected your eyes or that you have retinopathy?	Yes No Don’t know / Not sure Refused

^*provide for the response “Yes, but had them removed”

Variable definitions

Subjects were given seven choices for the question regarding the time of last eye exam (Table 1). Responses of “not sure” (N=7) were not considered in the analysis because equivalent information was not available from medical records. Similar categories were created based on medical record information by deriving the time of last eye exam, i.e., the time difference between the date of subject interview and the date of last examination listed on the medical record. To make results comparable to previous research,⁸ three mutually exclusive categories (< 1year, 1–2 years, and > 2years) were used for comparisons.

Subjects were given four options for questions regarding past diagnosis of glaucoma, macular degeneration, and diabetic retinopathy (Table 1). Responses of “not sure”, or “refused” were not considered in the analysis. For the cataract question there is an additional choice of “yes, but had them removed”, and for purposes of data analysis the two affirmative answers were combined into one “yes” category. Medical record information was abstracted by an experienced coder masked to subject responses. The following coding scheme was used. In general, disease definitions utilized were very broad. Each eye was coded separately. A participant was designated as having the condition if present in one or both eyes. An eye was coded as having an intra-ocular lens insertion (IOL) if IOL presence was used to characterize the eye in chart notes. Cataract was designated for eyes where chart notes included the term “cataract” regardless of type or severity level, or if IOL information was present in the medical record. Glaucoma was designated for eyes where notes made reference to the presence of glaucoma (any type); in addition, presence of ocular hypertension and glaucoma suspect were documented but not included in the primary analysis. Macular degeneration was designated for eyes where notes made reference to the term macular degeneration (including all disease severity descriptors such as “early”, “mild”. “moderate”, “intermediate”, “geographic atrophy”, “exudative”, “CNV” (choroidal neovascularization), “dry” and “wet”). Macular degeneration was also coded for eyes where the chart notes included macular drusen, hypo- or hyper-pigmentation, or pigment mottling. Diabetic retinopathy was designated for eyes where the terms diabetic retinopathy or diabetic macular edema were used, regardless of severity level.

Statistical analysis

Cohen’s kappa (κ) coefficients and their 95% confidence intervals (CIs) were calculated and used to determine agreement between information obtained from self-report and that from medical record. Because of the ordinal nature of the last eye exam variable, weighted kappa was used to assess agreement while simple kappa was used to assess agreement for eye diseases. Kappa values can range from −1 to 1 and qualitative interpretation has been previously characterized as: < 0 = no agreement; 0–0.20 = slight; 0.21–0.40 = fair; 0.41–0.60 = moderate; 0.61–0.80 = substantial; and 0.81–1.0 = almost perfect.³⁰ Race specific (White and African American) kappa coefficients were compared and the Chi-square test of equality used to test for significant differences.

RESULTS

There were 2,001 subjects in the study (Table 2). Average age was 77 years and ranged from 70 to 98. The majority of subjects were male (56.5%) and of White race (82.1%) with the balance almost entirely African Americans. Most subjects judged their general health to be very good or excellent (59%); nonetheless, most reported co-morbidities, including high blood pressure (65.4%), arthritis (54.5%), and heart problems (39.9%). Medical records were successfully obtained for the vast majority of the sample (1,873 of 2,001, 93.6%). The majority were from ophthalmologists (61.9%) and the remainder, optometrists (38.1%). Reasons for not obtaining records were: subject unable to provide a name for their eye care provider (N=59), no previous eye exam on record (N=38), eye exam date occurred after the interview date (N=27), the ophthalmologist’s or optometrist’s office never provided records in spite of repeated requests (N=2), and the subject refused to give permission (N=2).

Table 2.

Demographic and health characteristics of study participants (N=2,001)

Mean age (SD)	77 (4)
Median age	76
Age range	70–98
Gender, N (%)
Male	1,130 (56.5)
Female	871 (43.5)
Race/ethnicity, N (%)
White	1,643 (82.1)
African American	349 (17.4)
Hispanic	4 (0.2)
Asian	3 (0.2)
Other	2 (0.1)
General health N, (%)
Excellent	353 (17.6)
Very good	828 (41.4)
Good	630 (31.5)
Fair	164 (8.2)
Poor	26 (1.3)
Five most prevalent co-morbidities, N (%)
High BP	1,303 (65.4)
Arthritis	1,087 (54.5)
Heart problems	798 (39.9)
Hearing	644 (32.3)
Cancer	628 (31.4)
Type of provider medical record obtained from, N (%)	1,873 (93.6)
Ophthalmologist	1,159 (61.9)
Optometrist	714 (38.1)

There was substantial agreement between the question regarding time since last eye exam and information obtained from medical records (κ_W=0.64, 95% CI 0.60–0.68) (Table 3). Subjects tended to judge their most recent eye care examination as having occurred more recently than it did. For example, 79.4% of subjects reported their last eye exam as occurring within the previous year but medical records indicated 71.0%; likewise, 9.8% of subjects reported their last eye exam as having occurred more than two years ago but medical records indicated 12.8%.

Table 3.

Agreement between self-report and medical record for time of last eye exam

	Self-Report Time Eyes Last Examined
Medical record	< 1 year, N (%)	1–2 years, N (%)	> 2 years, N (%)	Total, N (%)
< 1 year	1,282 (86.5)	37 (18.2)	7 (3.9)	1,326 (71.0)
1–2 years	153 (10.3)	113 (55.7)	37 (20.3)	303 (16.2)
> 2 years	47 (3.2)	53 (26.1)	138 (75.8)	238 (12.8)
Total, N (%)	1,482 (79.4)	203 (10.9)	182 (9.8)	1,867 (100)

Weighted Kappa=0.64 (0.60–0.68)

Table 4 presents agreement with medical record for the questions regarding diagnosis for four major eye diseases. Agreement for cataracts or intra-ocular lens insertion was slight (κ=0.18, 95% CI = 0.13–0.23) with 80.8% concordance. For subjects reporting and not reporting cataracts, percent agreement was 97.8% and 15.4%, respectively. Cataracts prevalence based on self-report was 79.4% and 95.1% based on medical record. Agreement for glaucoma was substantial (κ=0.73, 95% CI = 0.68–0.78) with 94.8% concordance. For those reporting and not reporting glaucoma, percent agreement was 75.5% and 97.2%, respectively. Glaucoma prevalence based on self-report was 11.0% and 10.8% based on medical records. Agreement for macular degeneration was moderate (κ=0.40, 95% CI = 0.35–0.46) with 86.1% concordance. For those reporting and not reporting macular degeneration, percent agreement was 73.9% and 87.2%, respectively. Macular degeneration prevalence based on self-report was 8.5% and 18.0% based on medical records. Agreement for diabetic retinopathy was moderate (κ=0.47, 95% CI = 0.35–0.59) with 97.2% concordance. For those reporting and not reporting diabetic retinopathy, percent agreement was 58.1% and 98.1%, respectively. Diabetic retinopathy prevalence based on self-report was 2.3% and 3.2% based on medical record.

Table 4.

Comparison of self-report with medical record for major eye diseases

	Self-Report
Medical Record	Yes, N (%)	No, N (%)	Total, N (%)
Cataracts or IOL
Yes	1,442 (97.8)	324 (84.6)	1,766 (95.1)
No	33 (2.2)	59 (15.4)	92 (5.0)
Total, N (%)	1,475 (79.4)	383 (20.6)	1,858 (100)
κ=0.18 (95% CI = 0.13–0.23)
Glaucoma
Yes	154 (75.6)	46 (2.8)	200 (10.8)
No	50 (24.5)	1,611 (97.2)	1,661 (89.3)
Total, N (%)	204 (11.0)	1,657 (90.1)	1,861 (100)
κ=0.73 (95% CI = 0.68–0.78)
Macular degeneration
Yes	116 (73.9)	218 (12.8)	334 (18.0)
No	41 (26.1)	1,481 (87.2)	1,522 (82.0)
Total, N (%)	157 (8.5)	1,699 (91.5)	1,856 (100)
κ=0.40 (95% CI = 0.35–0.46)
Diabetic Retinopathy
Yes	25 (58.1)	35 (1.9)	60 (3.2)
No	18 (41.9)	1,788 (98.1)	1,806 (96.8)
Total, N (%)	43 (2.3)	1,823 (97.7)	1,866 (100)
κ=0.47 (95% CI = 0.35–0.59)

Agreement between self-report and medical record for eye care utilization and most eye diseases were similar for Whites and African Americans (Table 5). However, for macular degeneration there was significantly less agreement for African Americans than Whites (κ=0.14 versus κ=0.42, p<0.01). Very few African Americans (3.6%) reported macular degeneration but of those that did, 63.6% did not have the diagnosis listed on their medical records, compared to 23.5% among White participants. There was no significant difference in agreement between African Americans and Whites for any other eye diseases examined; though, there were dissimilar patterns of discordance by race for glaucoma diagnosis. For example, among African Americans and Whites who self-reported no to the glaucoma question, 7.2% and 2.0% respectively, had medical record information indicating that they did have glaucoma; among African Americans and Whites who self-reported yes to the glaucoma question, 13.9% and 30.5%; respectively, did not have medical record information that confirmed the diagnosis.

Table 5.

Comparison of agreement for eye care utilization and eye diseases by racial group

	White (κ)	African Americans (κ)	p-value*
Last eye exam**	0.65	0.61	0.43
Eye diseases
Cataract	0.19	0.14	0.44
Glaucoma	0.70	0.76	0.28
Macular degeneration	0.42	0.14	<0.01
Diabetic retinopathy	0.44	0.51	0.60

^*χ2 test of equality of κ coefficients

^**weighted κ

DISCUSSION

The current study provides an evaluation of self-reported eye care utilization and eye disease among older adults by examining response agreement with information obtained from medical records. Agreement was substantial for the question regarding time of last eye examination; however, the results suggest that eye care utilization estimates based on this question will overestimate the number of respondents who received an eye exam within the previous year and underestimate the number receiving an eye exam greater than two years ago. Qualitatively, for the four eye disease questions investigated, agreement was substantial for one, moderate for two and slight for the other.

The study’s prevalence estimates for cataracts are much greater than previous estimates for a similarly aged population.^{8,11,12,31,32} This is primarily due to the present study combining responses for current cataract and past cataract removal. Nonetheless, the concordance for cataracts in the current study was comparable to previous research.^11,12 A large proportion (85%) of the current study’s subjects who reported no cataracts, in fact had medical record information indicating cataracts; as might be expected, most (92.9%) of these participants had not had an IOL. Poor agreement may be due to eye care providers noting a diagnosis of cataract in the early stages of opacity, yet because the vision impairment is minor or the patient cites no vision complaints, the provider does not believe it necessary to inform patients of the early condition.

The self-reported prevalence of glaucoma in the current study was similar to those reported in recent analyses of similarly aged adults, one that that relied on 2005 through 2008 BRFSS data,⁸ and the other, 2002 NHIS data.³² Two conditions related to glaucoma, but diagnostically distinct from it, are ocular hypertension and glaucoma suspect. In caring for patients diagnosed with ocular hypertension or glaucoma suspect and in discussing their disorder with them, it would not be surprising if eye care providers may mention the term “glaucoma”. Thus it would not be surprising if some patients got the impression that they had glaucoma. Broadening the classification of glaucoma to include glaucoma suspect (N=104) and ocular hypertension (N=26) increased the number of subjects diagnosed with glaucoma-related conditions from 200 (10.8%) to 330 (17.7%). Agreement for this broad diagnostic category with self-report of glaucoma was reduced from 0.73 to 0.60 suggesting that those with ocular hypertension or glaucoma suspect diagnoses tended to correctly understand that they did not have glaucoma. Of the 200 subjects with medical record information indicating glaucoma, 165 (77.0%) correctly self-reported their glaucoma diagnosis. For the 130 subjects with medical record information indicating glaucoma suspect or ocular hypertension, 109 (83.9%) correctly self-reported no glaucoma diagnosis. Thus, a large proportion of participants with a medical record diagnosis of glaucoma, glaucoma suspect and ocular hypertension were able to accurately report, suggesting that self-report of glaucoma is valid.

Self-reported prevalence of macular degeneration was slightly lower than those of a recent analysis of similarly aged adults that relied on 2005 through 2008 BRFSS data,⁸ but similar to self-reported prevalence among adults 75 and older participating in the 2002 NHIS.³² Differences may be due to the current study’s inclusion of only current drivers. Since the ability to obtain a license to drive a motor vehicle and one’s decision to actually be a driver is influenced by vision,³³ it is likely that participants had better vision health on average compared to similarly aged individuals regardless of licensure status from the general population.

The results indicated moderate agreement for the question regarding diabetic retinopathy. A recent study which investigated the prevalence of diabetic retinopathy using 1997–2010 BRFSS data found that although the prevalence of diabetes had increased over the study period, the prevalence of age-adjusted vision impairment and diabetic retinopathy among those reporting diabetes had decreased from 23.7% to 16.7%.³⁴ It is encouraging that the anticipated increase in diabetic retinopathy has not occurred, but the authors note that the unclear validity of the self-report of vision impairment was a study limitation. Results of the current study indicate under-reporting for self-reported diabetic retinopathy but do not provide information as to whether reporting is differential with regard to diabetes diagnosis, subject age, or time period of report.

The current study found a significant difference in agreement between African Americans and Whites for self-reported macular degeneration with medical record (κ=0.14 versus κ=0.43, p<0.01). Previous research is ambiguous regarding the role of race in the accuracy of self-reported medical conditions.^13,16,26,27 Racial differences in health literacy³⁵ might contribute to African Americans being less familiar with eye disease such as glaucoma.³⁶ An analysis of focus groups of ophthalmologists and optometrists from a similar geographic region as the current study found that a majority (74%) of their comments were negative about older African Americans’ attitudes about vision and eye care. In general, the eye care professionals were concerned that older African Americans did not make eye care a priority and did not fully understand the importance of preventive strategies and available treatments.³⁷ Recent research has supported the proposition that physician-patient communications play an important role in race based health disparities,^38–41 which may have contributed to the agreement patterns reported in the current study for macular degeneration and glaucoma.

Early detection and treatment of eye diseases is important in maintaining eye health and decreasing disability.^42–44 Mobility and quality of life are negatively impacted by vision impairment resulting in heavy personal burdens and high costs to society; thus, early identification and treatment are the keys to decreasing eye disease, vision impairment and blindness.^45–48 Consequently, population based information on eye care utilization is valuable in assessing the quality and determinants of eye care. In the present study agreement between self-report and medical record for this variable was substantial and the proportion of subjects who reported having been to an eye care provider in the previous year was comparable to previous research.⁸ Nonetheless, subjects’ medical records suggested that participants were overestimating eye care visits that occurred within one year and underestimating visits that occurred after two years. Because age-related eye diseases are often asymptomatic, current recommendations for older adults, i.e., those >65 years of age, from the American Academy of Ophthalmology and the American Optometric Association are to receive a comprehensive dilated eye examination every one to two years.^49,50

Strengths of this study are as follows. The current study was a large, population-based examination of agreement between self-report and medical record on eye care utilization and major age-related chronic eye conditions among adults aged 70 years and older. Medical records were available for a large proportion of subjects and the person abstracting medical charts was masked to subject responses. In addition, all medical records were obtained from ophthalmologists and optometrists.

One important methodological consideration is how to handle self-reported responses of “not sure.” Few subjects self-reported “not sure” for time of last eye examination (N=7) and slightly more responded “not sure” for questions regarding cataracts (N=15), glaucoma (N=11), macular degeneration (N=17), and diabetic retinopathy (N=7). The current analysis omitted subjects who responded in this manner from the respective prevalence calculation. Based on information obtained from subjects’ medical records, 14/15 (93.3%) had cataracts, 7/11 (63.6%) had glaucoma, 5/17 (29.4%) had macular degeneration and 2/7 (28.6%) had diabetic retinopathy. To include those responding “not sure” in the analysis would require assuming that either none or all had the condition in question, thereby driving prevalence estimates down or up, respectively. However, given the small number of subjects (< 1%) in the current study who self-reported “not sure” to any of the eye disease questions, the impact on results was minimal.

The current study did not consider education level or cognition. Increased educational attainment has been reported to increase the validity of self-reported diagnoses that result in hospitalizations,¹¹ and for some chronic conditions, e.g., diabetes, hypertension, myocardial infarction, and stroke.²¹ Some studies have reported that educational attainment does not play a significant role in self-report validity among elderly subjects,^10,25,16 and the relationship between level of cognition and the validity of self-reported medical information is not well established.^20,25,28 Nonetheless, relatively few (6.4%) participants did not complete high school, and only 2.4% performed less than normal on cognition assessment.

Only licensed drivers were included in the sample but the National Highway Traffic Safety Administration estimates that approximately 90% of older adults are licensed drivers.⁵¹ Thus, the prevalences of eye diseases and vision impairment are likely lower in the study population than in the general population which would result in conservative estimates. And finally, the study population did not include a meaningful number of participants (< 1%) who were not White or African American and did not include any participants less than 70 years of age.

In summary, the current study suggests that national estimates based on self-report underestimate eye disease and overestimate eye care utilization in the 70 years of age and older population. Future research should investigate these associations in other populations, e.g., < 70 years of age, Hispanic and Asian American populations.