Validity of self reported eye disease and treatment in a population-based study: The Los Angeles Latino Eye Study

Abstract

Purpose

To examine the validity of self-reported eye disease, including cataract, age-related macular degeneration (AMD), glaucoma, and diabetic retinopathy (DR), and self-reported surgical treatment for cataract and DR in the Los Angeles Latino Eye Study (LALES).

Design

Population-based cross-sectional study

Participants

6357 Latinos age 40+ years from the LALES

Methods

Participants underwent a detailed interview, including survey questions about ocular health, diagnoses and timing of last eye examination, and a standardized clinical examination. Self report was compared to examination to determine sensitivity and specificity by length of time since last eye examination. Stepwise logistic regression was used to determine factors associated with inaccurate self report.

Main Outcome Measures

Sensitivity and specificity were calculated for four self reported eye diseases (cataract, AMD, glaucoma, DR) and for surgical treatment of cataract and DR. Odds ratios (OR) were determined for factors associated with inaccurate self report underestimating eye disease and treatment.

Results

For each disease, sensitivity and specificity in those who reported their last eye examination as <1 year ago were: 36.8%, 92.5% for cataract; 37.7%, 96.3% for glaucoma; 5.1%, 98.9% for AMD; and 25.7%, 94.2% for DR. Self report was less accurate with increasing time since last eye examination. Inaccurate self report was independently associated with better visual acuity (OR=2.4), <2 comorbidities (OR=1.7), last eye exam/visit 1–5 years ago and ≥5 years ago (OR=2.3 and 4.9, respectively), and less education (OR=1.3 for 7–12 years and 1.7 for <7 years). Of 88 participants surgically treated for cataract who reported an eye examination <1 year ago, sensitivity and specificity of self-reported surgical history were 90.9% and 99.9%. Of the 31 participants treated for DR (laser/surgery) and reporting an eye examination <1 year ago, sensitivity and specificity of self-reported surgical history were 19.4% and 99.6%.

Conclusions

Among Latinos, self reporting of eye disease and surgical history provides a significant underestimate of the disease burden. This may lead to significant misclassification in vision research if self report alone is used to identify persons with eye disease.

Large epidemiologic studies rely heavily on self reporting of medical history, diagnoses, and treatment in collecting data for population-based analyses. These participant responses form the basis of reports of prevalence and incidence of conditions, severity of disease, and treatment effect. Patients’ self report of disease is a cornerstone of almost all types of clinical research, from large-scale survey data such as the National Health and Nutrition Examination Survey (NHANES) and the National Health Interview Survey (NHIS) to small-scale studies of specific disease states such as toxoplasmosis or strabismus. Even in day-to-day clinical practice, we rely on patients’ self report of not only their own but also their family medical history. And we use this information to make broad conclusions about disease causation, progression, and association with environmental factors, sometimes resulting in significant policy changes. For this reason, the validity of self reporting of presence of disease or history of treatment has been assessed in a number of diseases, ranging from cancer¹ to mental health² to periodontic disease,³ but it has rarely been examined in eye disease. Furthermore, the validity of self reporting of eye disease and treatment has never been assessed in a primarily Latino population, the largest and fastest growing minority group in the United States.

The purpose of this study was to assess the validity of self reporting of four major eye diseases and two common treatment modalities in a population-based sample of Latinos who are primarily Mexican-American, with the goal of assessing the validity of this method as the basis for the assessment of the burden of ocular disease in this population. We hypothesized that accuracy of self reporting of eye disease would be lower in participants with less visual impairment, less comorbidities, a longer time from last dilated eye examination, and lower education.

Materials and Methods

Study Design

The study population consisted of self-identified Latinos of predominantly Mexican ancestry living in La Puente, California. Details of the study design, sampling plan, and baseline data have been presented elsewhere.⁴ To briefly summarize, a door-to-door census of all dwellings was conducted within 6 census tracts in La Puente. All residents who were eligible (self-identified Latinos 40 years or older at the time of the census, living in the specified census-tracts) were informed of the study and invited to participate in both a home interview and a clinic examination.^4,5 The Los Angeles Latino Eye Study (LALES) population was found to be comparable to the Los Angeles County and national Latino population 40 years and older in age and racial distribution; 95% were defined as Mexican-American (participant, parents, or grandparents born in Mexico).⁴ Of the 7789 persons determined to be eligible, 6357 (82%) chose to participate in the study. Of those, 6131 participants completed both the in-home interview and a clinic eye examination, 215 completed both the in-home interview and a home eye examination, and 11 completed the examination at the clinic but did not complete the in-home interview.

Institutional review board approval was obtained from the Los Angeles County/University of Southern California Medical Center Institutional Review Board. All study procedures adhered to the principles recommended in the Declaration of Helsinki for research involving human subjects. All work was compliant with the Health Insurance Portability and Accountability Act.

Interview Data

After informed consent was obtained, eligible individuals underwent a comprehensive in-home interview to obtain information about sociodemographics, medical history, and ocular history. Socio-demographic factors included in the interview were age, gender, country of birth, years of education, marital status, income level, insurance, and employment status. Acculturation was measured using the Cuellar 9-item Acculturation Rating Scale for Mexican Americans (low ≤1.9 vs. high >1.9, based on whether the participant spoke, read, or wrote Spanish and/or English and what the preferred language was). ⁷ Medical history included hypertension, diabetes, number of comorbidities (<2 vs. ≥2), and perceived general health (excellent/very good/good vs. fair/poor) and general vision (excellent/very good/good vs. fair/poor). Participants were questioned regarding the date of their last physical and eye examination (within 12 months, 1–5 years ago, over 5 years ago, never). To determine ocular disease and treatment history, the following questions were asked: “Has a doctor ever told you that you had cataracts/macular degeneration/glaucoma/diabetic eye disease in either eye? If so, which eye? Did you ever have cataract surgery? Did you ever have laser treatment or surgery for your diabetic eye disease?”

Clinical Examination Data

Participants were then invited and scheduled for a detailed eye examination, which was performed in a standardized manner at the LALES local eye examination center. In the clinic examination, participants were assessed for the following ocular conditions: cataract, age-related macular degeneration (AMD), glaucoma, and diabetic retinopathy (DR). Diagnosis of cataracts was based on a standardized grading of lens changes at the time of the eye examination.⁸ Diagnosis of AMD was based on drusen type and the existence of retinal pigmentary abnormalities, classified as early AMD or advanced AMD. Two graders at the Wisconsin Ocular Epidemiology Grading Center performed masked grading using a modification of the Wisconsin Age-Related Maculopathy Grading System.⁹ Open-angle glaucoma was defined by characteristic optic nerve and visual field abnormalities, diagnosed by two independent glaucoma specialists.¹⁰ Diagnosis of DR was made in persons with diabetes mellitus using modified grading protocols from the Early Treatment Diabetic Retinopathy Study adaptation of the modified Airlie House classification.¹¹

Statistical Analysis

Sensitivity and specificity of self reporting were calculated for the four self-reported eye diseases (cataract, AMD, glaucoma, and DR) as well as for cataract surgery and laser/surgical treatment of DR in either the right or the left eye, stratified by length of time since last complete eye examination/eye care visit (<1 year ago, 1–5 years ago, >5 years ago). 1929 patients who did not respond to the questionnaire or reported never having had an eye examination (Cohort 1) or eye surgery (Cohort 2) were excluded from all analyses.

Separate univariate analyses and forward stepwise multivariate logistic regression (validated by backward stepwise elimination) were used to obtain odds ratios (OR) and confidence intervals (CI) determining demographic, medical, and ocular history factors associated with inaccurate self report underestimating presence of the aforementioned eye diseases. For these analyses, cases were those who reported they did not have any of the four diseases but on clinical examination were found to have at least one) and laser/surgery for cataract and DR. Controls were those who correctly reported that they did have one or more of the four diseases. An alpha of 0.2 was used for entrance and exit criteria in stepwise selection, and the initial variables entered into the model were as follows: age, gender, education, marital status, employment, acculturation, vision insurance status, household income, perceived general health condition, perceived general eye condition, history of diabetes, number of comorbidities, time in months since last visit to any doctor, and time in months since last eye care visit/completed eye exam. Analyses were conducted at the 0.05 significance level (all P-values two-sided) using SAS version 9.1 software (SAS, Inc, Cary, NC).

RESULTS

The demographics of the LALES study population have been reported elsewhere.⁴ As indicated in Table 1, of the 6357 participants, 6106 responded positively or negatively to the survey questions about history of cataract, 6028 responded positively or negatively to questions about history of AMD, 6095 responded positively or negatively to questions about history of glaucoma, and 1040 responded positively or negatively to questions about history of DR (after confirming a history of diabetes mellitus). Of these, complete data on treatment and examination based diagnosis was present in 5913 participants with cataract, 5825 with AMD, 6095 with glaucoma, and 998 with DR.

Table 1.

Participants in the Los Angeles Latino Eye Study with a Positive Diagnosis by Self Report and/or Clinical Examination Stratified by Eye Disease

Eye Disease	Participants responding to survey questions about eye disease	Participants undergoing clinical examination	Participants diagnosed with disease by self report	Participants diagnosed with disease by clinical examination	Participants diagnosed with disease by self report AND clinical examination
Cataract	6106	5913	494	1068	287
AMD	6028	5825	48	574	17
Glaucoma	6095	6095	183	283	70
DR	1040¥	998	136	516	108

AMD = age-related macular degeneration; DR = diabetic retinopathy

^¥Questions about DR were only asked of participants who responded positively to survey questions about history of diabetes mellitus.

Table 1 also provides a breakdown of participants diagnosed by self report versus clinical examination in each disease category: 494 participants self reported a positive history of cataract, 48 self reported a positive history of AMD, 183 self reported a positive history of glaucoma, and 136 self reported a positive history of DR. Within this same group, 1068 participants were diagnosed with cataract on clinical examination, 574 were diagnosed with AMD, 283 were diagnosed with glaucoma, and 516 were diagnosed with DR. Therefore, the number self reporting a history of disease that was confirmed by clinical examination was 287 for cataract, 17 for AMD, 70 for glaucoma, and 108 for DR.

Table 2 summarizes the self-reported history of disease stratified by time since last examination excluding patients who never reported having had an eye examination or eye care (Cohort 1). It also summarizes the self-reported history of cataract surgery or DR surgery, excluding patients who never reported having had eye surgery (Cohort 2).

Table 2.

Numbers of participants who self-reported history of Eye disease (Cohort 1) and self-reported cataract surgery/laser or other surgery for diabetic retinopathy (Cohort 2) stratified by time since last eye examination or last eye care visit.

Time since Last Eye Examination or Eye Surgery	Study Cohort 1: Self-reported -history of eye disease	Study Cohort 2 Self-reported history of Cataract Surgery/laser or other surgery for DR
<1 year	1975	84/12
1–5 years	2011	47/4
>5 years	442	3/0
Total	4428	134/16

Study Cohort 1: Self-reported eye disease (cataract, age-related macular degeneration, glaucoma, and diabetic retinopathy[DR]). Patients who never reported having had an eye examination or eye care were excluded from Cohort 1

Study Cohort 2: Self-reported cataract, laser or other surgery for DR. Patients who never reported having had eye surgery for cataract or DR were excluded from Cohort 2

A total of 1929 patients were excluded from all analyses.

Overall, Cohort 1 sensitivity of self report of eye disease for cataract, AMD, glaucoma, and DR was low even in patients who received an eye examination less than 1 year prior (range 5.1%–36.8%), as seen in Table 3. The specificity of self report for eye disease was high, ranging from 92.5%–99.7% no matter the length of time since last eye exam/visit. Participants with AMD had the lowest sensitivity (2.7%) but the highest specificity (99.7%) of self report among all groups.

Table 3.

Sensitivity and Specificity of Self-reported Eye Disease in Participants in the Los Angeles Latino Eye Study Stratified by Time since Last Eye Examination (n=4428)

Time since last eye examination or eye care visit	Cataract	Glaucoma	AMD	DR
Sensitivity (%)
<1yr	36.8	37.7	5.1	25.7
1–5 yrs.	21.8	36.8	2.7	18.2
>5 yrs.	11.1	5.1	-	4.8
Specificity (%)
<1yr	92.5	96.3	98.9	94.2
1–5 yrs.	95.5	98.5	99.5	94.9
>5 yrs.	98.1	98.8	99.7	95.7

AMD = age-related macular degeneration, DR = diabetic retinopathy

As shown in Table 4, stepwise multivariate logistic regression analysis to identify independent factors associated with inaccurate self report of eye disease (n=1019 cases) vs. accurate self report (n=482 controls) revealed that better visual acuity (OR=2.4, 95%CI 1.7–3.4), <2 comorbidities (OR=1.7, 95%CI 1.2–3.3), last complete eye examination over 1 year ago (OR range = 2.3–4.9), and lower number of years of education, 7–12 years and <7 years (OR=1.3/1.7, 95%CI 1.02–2.1/1.1–2.7, respectively), were significantly associated with greater inaccuracy of self reporting of diagnosis. Time since last complete/dilated eye examination was further defined as 1–5 years ago (OR=2.3, 95%CI 1.7–3.0) or 5 or more years ago (OR=4.9, 95%CI 2.2–10.9), both significantly associated with greater inaccuracy of self reporting consistent with our P=0.05 cutoff.

Table 4.

Independent Risk Factors^* Associated with Inaccurate Self-reported Eye Disease (n=1019) Compared to Accurate Self-reported Eye Disease (n=482) in participants in the Los Angeles Latino Eye Study (n=1501)

Independent Risk Factors		Total N	Odds Ratio (95%CI)	P-value
General vision
Fair/Poor/Very Poor/Blind		973	1
Excellent/Very Good		528	2.4 (1.7–3.4)	<0.001
Comorbidities¶
≥2		937	1
<2		564	1.7 (1.2–3.3)	0.001
Last Eye Examination/Eye Care Visit
<1 year ago		650	1
1–5 years ago		729	2.3 (1.7–3.0)	0.003
5 or more years ago		122	4.9 (2.2–10.9)	<0.001
Education
>12 years		182	1
7–12 years		590	1.3 (1.02–2.1)	0.009
<7 years		729	1.7 (1.1–2.7)	0.024

CI = Confidence Interval

^*Based on a multivariate forward stepwise regression, with backward stepwise validation.

^¶Number of self-reported co-morbidities from the following list: arthritis, stroke/brain hemorrhage, high blood pressure, angina, heart attack, heart failure, asthma, skin cancer, other cancer, back problems, hearing problems.

As shown in Table 5, of the 88 participants surgically treated for cataract (right eye) and reporting an eye exam/eye care visit <1 year ago, sensitivity of self reporting was 90.9% and specificity 99.9%. Of 31 treated for DR (laser/surgery, right eye) and reporting an eye exam <1 year ago, sensitivity was 19.4% and specificity was 99.6%. These numbers remain relatively consistent for the cataract surgery group reporting their last eye exam as 1–5 years ago; for those undergoing laser/surgery for DR, sensitivity rates continue to fall the longer ago their last reported eye exam, while specificity remains constant (Table 5). Data (not shown in Table 5) for the left eye was very similar. In stepwise multivariate analysis of treatment for cataract and DR, none of the factors evaluated was found to be significantly associated with inaccurate report of cataract surgery or laser/surgery for DR (data not shown).

Table 5.

Sensitivity and Specificity of Self-reported Eye Surgery Compared to Eye Examination Findings in the Right Eye of Participants in the Los Angeles Latino Eye Study^*

Time since last eye examination or eye care visit	Number Treated Based on Examination	Number Self Reporting Treatment	Sensitivity (%)	Specificity (%)
Cataract Surgery
<1 year ago	88	84	90.9	99.9
1–5 years ago	59	47	78.0	100
>5 years ago	3	3	-	-
Laser or other surgery for Diabetic retinopathy
<1 year ago	31	12	19.4	99.6
1–5 years ago	6	4	16.7	99.9
>5 years ago	0	0	-	-

^*Data from the left eye was similar to data in the right eye.

DISCUSSION

Self reporting, either by questionnaire or interview, is one of the most commonly used methods of obtaining medical data. Numerous studies have examined the validity of self-reported interview data, but few have examined this in regard to eye disease and none in a primarily Latino population. In our cohort, all four diagnoses – cataract, AMD, glaucoma, and DR – had a low sensitivity and high specificity of self reporting. This indicates that not self reporting the presence of one of these conditions was not a good measure of whether or not a person had the disease, but if a participant did report the presence of one of these conditions it was likely to be accurate. While sensitivity of self report decreased with greater length of time since last eye exam/eye care visit, surprisingly, we found a low sensitivity even among those who reported a complete eye examination/eye care visit within the last year. In these participants, our data show that less than 40% were unable to accurately report their ocular diagnosis. Along these lines, a positive self report did not reliably confirm a diagnosis of eye disease in this population even in those who had been seen by an eye care provider within the last year, supporting the need for having follow-up clinical examination. Self report of laser or surgery for diabetic retinopathy followed this same pattern, although self report of cataract surgery was much more reliably accurate with high sensitivity and specificity. A possible limitation of our findings is that there may be a discrepancy between what participants were previously told by their eye doctor vs. what was diagnosed by the LALES clinical examination, indicating that some cases of inaccurate reporting may result from undiagnosed eye disease rather than poor recall.

Comparison with Other Studies

In the population-based Beaver Dam Eye Study, Linton et al¹² examined the validity of self and surrogate-reported diagnosis of cataract and AMD in a primarily white population in rural Wisconsin. Their findings were similar to ours in that both self and surrogate-reporting of cataract and AMD had a low sensitivity and a high specificity, though they found sensitivity of self reporting to be greater when done in-person rather than by telephone or by surrogate.¹² As in our study, the sensitivity of self reporting of AMD was much lower than that of cataract, perhaps indicating less patient understanding and awareness of this disorder as compared with the more prevalent and familiar diagnosis of cataract. Overall, our results confirm these previous findings, indicating that the true prevalence of most major eye diseases is likely underestimated via methods of self reporting. Also, given that a large amount of public health outreach has been devoted to diagnosing and treating cataracts, especially among poor and immigrant populations both domestically and abroad, our data may support that this push to increase education is indeed effective in creating awareness of this diagnosis and its treatment.

Of note, while the purpose of this study was to examine this research question in a primarily Latino population, our results were concurrent with those of the Beaver Dam study in a similar economic demographic but different ethnic group. Such uniformity of findings across groups supports extrapolation of these results to other populations, though additional studies are needed to confirm that these findings are indeed upheld across ethnicities.

Few studies have evaluated accuracy of self reporting of eye surgery; one other study from the Salisbury Eye Evaluation project investigated accuracy of reporting of cataract surgery in particular for self and/or sibling and found, as we did, a high sensitivity and specificity.¹³ In larger studies of a variety of other medical conditions, recall of hospitalizations and surgery has been reasonably accurate.¹⁴ It has been suggested that longer length of hospital stay was associated with more accurate recall (and that underreporting was more common than over-reporting); given that hospitalization is not required for most eye surgeries, this finding is in line with our data showing a low sensitivity of self reporting of laser/surgery for diabetic retinopathy. This was not the case with cataract surgery, but as mentioned above the greater public awareness of this disorder may help improve recall and accuracy of self reporting. Additionally, cataract surgery takes place in a surgical suite, either in a hospital-affiliated or ambulatory surgical center, as opposed to laser treatment for diabetic retinopathy which is usually performed in a clinic setting. The association with change of location may also boost patient recall.

Only one other study, of patients from general eye and retina clinics within the University of Chicago Hospitals,¹⁵ has looked at self reporting of prior laser treatment of diabetic eye disease. In this clinic-based sample, participants were very accurate (sensitivity and specificity = 96%) in reporting past laser treatment.¹⁵ While these results differ from ours, these differences may reflect a selection bias in that persons who are obtaining care within the healthcare system are more likely to provide an accurate self report than those who either do not obtain or are unable to access care.

This is also evident in the results of our multivariate analysis, since participants who had a complete/dilated eye exam within the last year were more likely to accurately report their diagnoses than those whose last examination was 1–5 years ago or greater than 5 years ago; the odds of accurately self reporting eye disease significantly decreased by a substantial amount with increasing time since last eye exam. As expected, we found that those with better visual acuity and fewer comorbidities more often inaccurately reported having one of the four eye diseases. It is understandable that healthier people with better vision and fewer comorbidities are less likely to be seen regularly by a physician of any type and, therefore, may be less aware of eye diseases that are mild or currently not visually significant, such as early cataracts, glaucoma, AMD, and early DR. If such patients are reporting excellent/good vision, they or their primary care providers may not prioritize a visit with an eye care provider. This highlights the need for increased awareness among adult Latinos and their healthcare providers of the importance of routine eye health maintenance visits; guidelines from the American Academy of Ophthalmology recommend comprehensive eye exams for adult persons with no risk factors every 2–4 years for those ages 40–54, every 1–3 years for those ages 55–64, and every 1–2 years for those over age 64.¹⁶

As we hypothesized, those with less years of education more often inaccurately reported the presence of eye disease. This may reflect a lack of understanding secondary to poor communication and lower health literacy, again emphasizing the need for language- and education-appropriate health care discussions in these populations. These results were consistent with those of Bowie, et al in the Salisbury Eye Evaluation project mentioned above, where more education was also found to be significantly predictive of accurate report of cataract or cataract surgical history on behalf of a sibling.¹³

Study Limitations

One limitation of our study is that this population is drawn from a single community with 94.7% of Mexican ancestry, so our results may not be broadly generalizable to all Latinos within the US. However, as noted above, our findings were similar to those of studies published on other non-Latino populations, so these findings may be more widely applicable both inside and outside of the Latino community. Additionally, since we validated participants’ self report of diagnosis and treatment by clinical examination at our LALES centers instead of using previous medical records, there may be a discrepancy between what participants were told previously and our examination. While we took great care to use standardized diagnostic techniques, we realize some cases of inaccurate reporting may result from undiagnosed eye disease rather than poor recall, and this could introduce error into our methods.

Finally, as mentioned in the methods section, this analysis examines factors associated with inaccurate self reporting that underestimates eye disease, since our number of participants that overestimated eye disease, or reported history of disease when none was found at clinical examination, were too few to allow for a separate analysis of substantive power. Investigation of overestimation of eye disease can also provide important insight into predictors of inaccurate self reporting in this population, however, and we are working on future analyses stratified by diagnosis to further elucidate these associations.

In summary, while we continue to rely heavily on self reporting of eye disease and treatment in vision research on Latinos, we should take into account that this may provide a gross underestimate of the burden of eye disease in this group. If used to answer a study question designed to identify persons with disease, our data suggest that questionnaires reliant on self report are inadequate as they provide poor prevalence measures in this population. When possible, clinical examination or other objective measures should be incorporated in addition to patient self report. When this is not possible, researchers should be cognizant of factors that are associated with inaccurate self report in their populations and take such factors into account in data analysis and conclusions. Methods to increase patient awareness of eye disease, including emphasis on the importance of routine ocular health maintenance examinations, may help to improve the accuracy of self reporting in this population, which in turn can enhance the quality of care these patients receive.