Instrument Development of the UNC Dry Eye Management Scale

Abstract

Purpose

Dry eye disease (DED) is a common ocular disease that can have adverse effects on quality of life. Our aim was to develop a single-item questionnaire that is reliable, patient-driven, and clinic-friendly to assess DED symptoms and their effect on quality of life in order to help support the management of patients with DED.

Methods

An initial dry eye questionnaire was created and administered to 18 patients with DED followed by a 15-minute cognitive interviewing session. This questionnaire was then refined using feedback obtained from the cognitive interview and was termed the University of North Carolina Dry Eye Management Scale (UNC DEMS). Field testing was then performed on 66 patients (46 with DED and 20 without DED) to determine the validity and test re-test reliability of the UNC DEMS compared to the current gold standard, the Ocular Surface Disease Index (OSDI). Pearson correlation coefficients were calculated between the UNC DEMS, OSDI, and other DED measures to assess criterion-related validity. Reliability coefficients were estimated for test-retest reliability.

Results

Comparing the UNC DEMS to the OSDI across all study participants, the correlation coefficient was 0.80 (p < 0.001). Comparing the UNC DEMS to the OSDI in the DED group, the correlation coefficient was 0.69 (p < 0.001). The test-retest reliability coefficient of the UNC DEMS was estimated to be 0.90.

Conclusion

The UNC DEMS is a valid, reliable questionnaire that can be efficiently administered in a busy clinical practice and can be used to support the management of patients with DED.

INTRODUCTION

Dry eye disease (DED) is a common ocular disease that affects 5–17% of the U.S. population with prevalence increasing with age.^1–3 It is a chronic disease characterized by symptoms such as ocular pain, grittiness, burning, foreign body sensation, tearing, and sensitivity to light.^{4, 5} Multiple studies have confirmed that DED adversely affects a person’s quality of life (QOL).^{1, 5–9} Specifically, DED symptoms can affect a person’s ability to read comfortably, concentrate, operate a computer, drive, or even perform basic work tasks.¹ Moreover, the adverse effects on QOL typically become more pronounced with disease chronicity and symptom severity.⁹ Two recent studies used utility assessments, a type of QOL measure, to quantify DED’s effect on a patient’s QOL. The studies demonstrated that patients with moderate to severe dry eye have utility scores comparable to those who have chronic conditions such as kidney failure requiring dialysis, severe angina, and hip fractures.^{3, 10} This finding is strong evidence that QOL and symptom severity are inter-related. Both need to be addressed when treating patients with dry eye to maximize health outcome and patient-physician satisfaction.

With the increasing recognition that DED symptoms diminish QOL, multiple dry eye questionnaires have been developed to evaluate both dry eye symptoms and their effect on QOL. Many of these questionnaires, however, have either not undergone sufficient validity and reliability testing or are not specific to DED.^{4, 6} In fact, a recent review of dry eye questionnaires revealed only two validated and reliable disease-specific questionnaires that accounted for both DED symptoms and QOL: the Ocular Surface Disease Index (OSDI)¹¹ and the Impact of Dry Eye on Everyday Life (IDEEL).^{12, 13} The OSDI and IDEEL have both proven extremely valuable in assessing dry eye symptoms and disease effect on QOL in clinical trials.^{2, 4–6, 8, 9, 14} The IDEEL, however, contains 57 items and the OSDI, a 12-item questionnaire, involves a complex, time-consuming scoring algorithm.^{11, 12, 14} Therefore, both questionnaires are time-consuming to complete and score, and may have limitations being implemented into a busy clinical setting.^{11, 12, 14}

In addition to the administrative and scoring burdens of the OSDI and IDEEL, clinical markers for DED, such as fluorescein staining, tear break-up time (TBUT), and Schirmer testing, have high inter-observer biases, lack standardization among ophthalmologist providers, and correlate poorly with subjective symptoms.^{4, 6, 7, 15} Consequently, physician and patient ratings of dry eye status and its effect on QOL are often discordant. It is essential that physicians take this discordance into account when attempting to treat a patient’s dry eye as clinical signs alone may not fully capture the patient’s experience with DED. If a busy clinical practice does not allow a clinician the necessary time to administer the OSDI or IDEEL, however, this discordance may not be detected and a patient’s symptoms may go untreated.^{1, 5, 7, 13} Thus, there is a pressing need to produce a more efficient and reliable patient-driven instrument that physicians can use to help guide therapy and management of DED.³ In response to this need, our research team developed a single-item dry eye questionnaire called the University of North Carolina Dry Eye Management Scale (UNC DEMS). Following the PROMIS® guidelines for instrument development as outlined below, we created the UNC DEMS to simultaneously assess both DED symptoms and their effect on QOL in dry eye patients.

METHODS

PROMIS® Standards

The Federal Drug Administration’s (FDA’s) publication of guidelines for instrument development for patient-reported outcomes measures (PROs) in 2006 spurred the paradigm shift favoring the proper development and implementation of PROs in policy and health systems.^16,17 The Patient-Reported Outcome Measurement Information System (PROMIS®), a National Institute of Health (NIH) funded initiative, has established a set of nine standards for instrument development, refinement, and field-testing of PROs for research and clinical use.¹⁸ The UNC DEMS (FIGURE 1), a one-item, graded scale (1–10), was developed using these PROMIS® standards as a guide.

Figure 1.

The University of North Carolina Dry Eye Management Scale (UNC DEMS).

Instrument Development and Psychometric Evaluation of the UNC DEMS

To date in this on-going effort, we have followed the first seven PROMIS® standards for UNC DEMS instrument development. The first five PROMIS® standards : (1) Defining target concept and conceptual model, (2) Generating and design of individual items, (3) Constructing item pool, (4) Determining item bank properties, and (5) Field-testing and instrument format¹⁸ are detailed in a prior publication by the authors.¹⁹ In brief, we used a comprehensive literature search of dry eye symptoms and disease-effect on QOL combined with direct consultations with multiple ophthalmologists and dry eye patients to create an initial dry eye questionnaire. This questionnaire was administered to 18 patients with DED (ICD9 Code: 375.15) followed by a 15-minute cognitive interviewing session. The UNC DEMS was then refined using feedback obtained from the cognitive interviews. A final version of the UNC DEMS was produced (FIGURE 1).

Validity and Reliability Testing

Following the sixth and seventh PROMIS® standards (validity and reliability testing),¹⁸ we began field-testing on a larger cohort of patients to determine the validity and test re-test reliability of the UNC DEMS compared to the current gold standard, the OSDI. Using the initial validation study of the OSDI as a guide, we determined that 50 patients would be an adequate sample size to estimate the intra-class correlation to within 0.06 using a one-sided 95% confidence interval if the true correlation is 0.9 (to determine reliability of the questionnaire); in addition, we determined that this sample size would provide 98% power to detect a 0.50 correlation with the OSDI and 89% power to detect a 0.40 correlation with artificial tear use.¹¹ Therefore, we aimed to recruit 75 patients (with the goal of at least two-thirds of those patients fully completing the study). Ultimately, a total of 66 patients were recruited into the study, of which 46 had dry eye (ICD9 Code: 375.15) and 20 were controls without DED. To be included in the dry eye group, patients had to be 18 years of age or older with a known diagnosis of dry eye. These patients must have experienced dry eye symptoms within the last 3-months prior to enrolling in the study. Control patients were healthy and without ocular surface disease or vision correction surgery. All study participants were English speaking as the UNC DEMS was available only in English. Exclusion criteria for both dry eye and control patients included: intraocular surgery within the past 90-days, history of corneal transplant or neurotrophic keratitis, dry eye secondary to Stevens-Johnson Syndrome and/or cicatricial pemphigoid, severe conjunctival goblet cell loss or scarring conditions, congenitally absent meibomian or lacrimal glands, or active ocular infection such as blepharitis or lid margin inflammation. This study was approved by the University of North Carolina Institutional Review Board prior to enrolling patients.

During a regular clinic visit, consent was obtained from qualified study patients who were then asked to complete the DEMS (the word “UNC” was omitted from the questionnaire in this field test to prevent biases), OSDI, and a short survey. In this survey, patients were asked to report if they have used artificial tears, frequency of artificial tear use, and their subjective rating of their dry eye disease status as either normal, mild-to-moderate, or severe. The study investigators also obtained tear breakup time (TBUT) and fluorescein corneo-conjunctival staining (FUL-GLO®, Akorn, MD). Average TBUT OU was determined by taking three consecutive measurements of TBUT in each eye and then using the mean of these six measurements for our analyses. A fluorescein corneo-conjunctival staining score was determined for each eye using the Oxford Grading Scale (OGS); the average of the OD and OS score for each patient was used for our statistical analyses. An attending ophthalmologist then performed a complete slit-lamp examination and provided his or her assessment of the patient dry eye disease status as being normal, mild-to-moderate, or severe. Of note, the ophthalmologist performing this assessment was blinded to the UNC DEMS and OSDI patient-reported outcomes. Using his or her clinical judgment and experience, the ophthalmologist also recorded the presence or absence of ocular conditions commonly associated with DED, including chalasis, meibomian gland dysfunction (MGD), superficial punctate keratitis (SPK), lid wiper epitheliopathy (LWE), and poor lid apposition. After at least a week had passed since the clinic visit, the study participants were asked to complete the DEMS and OSDI a second time. The post-1-week forms were sent to the patients either via regular mail or by a web-link in an email. Participants were able to submit their responses either by mail or online, depending on their preferences. All participants were compensated with a redeemable $25 gift card.

Statistical Analysis

To assess criterion-related validity of the UNC DEMS, we estimated the Pearson correlation coefficients and their 95% confidence intervals using Fisher’s z transformation, between the DEMS, the OSDI and other DED measures. Additionally, we plotted the DEMS against the OSDI at the clinic visit and fit a simple linear regression with 95% prediction intervals. To assess construct validity, we compared the mean DEMS scores between the DED and control patients as well as between groups based on patient and physician ratings of DED status. For these comparisons, we used nonparametric Kruskal-Wallis tests. For the comparisons across the ratings groups, we first conducted an overall test to compare across the three groups and, only if the overall test was significant at the 0.05 level, we then conducted all pairwise comparisons. For test-retest reliability, we used only data from patients who provided follow-up measurements and we applied linear mixed models to estimate the reliability coefficient along with 95% bootstrapped confidence intervals. Additionally, we created a limits of agreement plot²⁰ for repeated measurements of the DEMS, including only dry eye patients to avoid artificially inflating the number of zero differences. Finally, we compared the DEMS scores across groups of patients based on the presence or absence of ocular conditions using Kruskal-Wallis tests, and we fit a simple linear regression to assess the association of the DEMS score with the total number of ocular conditions. All analyses were conducted in SAS, version 9.3 (SAS Institute, Cary, NC).

RESULTS

Population Demographics

A total of 66 patients participated in the UNC DEMS validity and reliability study; 46 were dry eye patients and 20 were control participants free of ocular disease. Patients in the dry eye and control groups were similar in age range and numbers of men and women (Table 1). Artificial tear use was higher in the dry eye group than in the control (mean: 2.53 times/day versus 0.15 times/day). Physicians and patients both rated the DED disease status as normal, mild-to-moderate, or severe (Table 2). Physicians were less likely to rate patient’s DED status as severe than were patients.

Table 1.

Patient Demographics

	Control Patients N = 20	Dry Eye Patients N = 46	Total Patients N = 66
Ethnicity, n (%)
African American	2 (10.0)	10 (21.7)	12 (18.1)
Caucasian	18 (90.0)	33 (71.7)	51 (77.2)
Other/Unknown	0 (0.0)	3 (6.5)	3 (4.5)
Sex, n (%)
Female	15 (75.0)	37 (80.4)	52 (78.7)
Male	5 (25.0)	9 (19.5)	14 (21.2)
Age, mean (SD)	62.8 (13.1)	61.6 (13.1)	62.4 (13.0)

Table 2.

Patient & Physician Rating of Disease Severity

	Control Patients N = 20 n (%)	Dry Eye Patients N = 46 n (%)	Total Patients N = 66 n (%)
Patient Rating of Disease Severity:
Normal	18 (90.0)	4 (8.6)	22 (33.3)
Mild-to-Moderate	2 (10.0)	26 (56.5)	28 (42.4)
Severe	0 (0.0)	14 (30.4)	14 (21.2)
No Rating Provided	0 (0.0)	2 (4.3)	2 (3.0)
Physician Rating of Disease Severity:
Normal	18 (90.0)	5 (10.8)	23 (34.8)
Mild-to-Moderate	2 (10.0)	39 (84.7)	41 (62.1)
Severe	0 (0.0)	2 (4.3)	2 (3.0)

Validity Testing

Figure 2 presents a scatter plot of the DEMS against the OSDI at the clinic visit. The DEMS is correlated to the OSDI across all study participants at an estimated coefficient of 0.80 (95% CI [0.69, 0.87], p < 0.001). The DEMS was correlated to the OSDI in the dry eye group at an estimated coefficient of 0.69 ([0.49, 0.81], p < 0.001). Table 3 presents the correlations of the DEMS and OSDI with DED clinical measures. Overall, the DEMS and OSDI have similar moderate, but significant, correlations with artificial tear usage, fluorescein staining, and TBUT.

Figure 2.

Scatter plot of Dry Eye Management Scale (DEMS) by Ocular Surface Disease Index (OSDI) at the clinic visit with estimated regression line and 95% prediction interval.

Table 3.

Pearson Correlations of DEMS and OSDI with DED Parameters over all study participants (N = 66)

	DEMS	OSDI
OSDI	0.80 (p < 0.001)	1.00
Freq. of Artificial Tear Use	0.43 (p < 0.001)	0.39 (p = 0.001)
Average Oxford Score OU	0.39 (p = 0.001)	0.42 (p <0.001)
Average TBUT OU	−0.26 (p = 0.032)	−0.38 (p = 0.002)

Note: DEMS = Dry Eye Management Scale, OSDI = Ocular Surface Disease Index, DED = Dry Eye Disease, TBUT = Tear break-up time, OU = Both Eyes

In the control group, the mean DEMS score was 1.85 ± 1.72, whereas in the dry eye group it was 5.73 ± 2.15 (p < 0.001). In the control group, the mean OSDI score was 9.49 ± 14.18 as compared to 37.18 ± 23.24 in the dry eye group (p < 0.001). The average DEMS and OSDI scores both varied significantly (p<0.001 for each) across patients with normal, mild-to-moderate, and severe DED (Table 4).

Table 4.

DEMS and OSDI Scores by DED Severity^‡

	Mean DEMS Score ± SD	Mean OSDI Score ± SD
Patient Disease Severity Rating
Normal (n = 22)	1.45 ± 0.72	7.34 ± 7.50
Mild-to-Moderate (n = 28)	5.40 ± 1.41	36.32 ± 22.04
Severe (n = 14)	7.86 ± 1.32	51.41 ± 17.79
Physician Disease Severity Rating
Normal (n = 22)	2.50 ± 2.03	14.60 ± 19.21
Mild-to-Moderate (n = 40)	5.68 ± 2.46	38.25 ± 23.30
Severe (n = 2)	5.50 ± 2.12	23.39 ± 17.01

Note: DEMS = Dry Eye Management Scale, OSDI = Ocular Surface Disease Index, DED = Dry Eye Disease.

^‡p-value < 0.001 for comparing both the DEMS and OSDI across both patient and physician ratings groups. For patient ratings, all pairwise comparisons between the 3 severity rating groups for each of the DEMS and OSDI resulted in p<0.04. For physician ratings, comparison of mild-to-moderate to normal rating groups resulted in p<0.001 for both DEMS and OSDI; pairwise comparisons between Severe group and other two groups resulted in p>0.05 for both DEMS and OSDI.

Test-Retest Reliability

Fifty-seven of 66 patients (86.4%) completed the post-1-week follow-up DEMS; 55 patients (83.3%) also completed the post-1-week follow-up OSDI. The test-retest reliability coefficient of the UNC DEMS was estimated to be 0.90 [0.84, 0.95]. By comparison, the OSDI’s reliability coefficient was 0.81 [0.70, 0.91]. Among dry eye patients, the mean difference between the post-1-week and clinic DEMS score was −0.24 units, and agreement was very good with only one value outside of the limits of agreement (Figure 3).

Figure 3.

Limits of agreement plot for dry eye patients with repeated Dry Eye Management Scale measurements (N=40).

Other Findings

On average, patients who were noted to have SPK on exam had a DEMS score 2.80 points higher than did those without SPK (p < 0.001). Patients with poor lid apposition had an average DEMS score 2.56 points higher than did those without poor lid apposition, but this difference was not statistically significant (p = 0.090). There was no statistically significant evidence of an association between DEMS score and presence of chalasis, MGD, or LWE. When all five of these clinical findings were taken together, there was a statistically significant association between the DEMS score and the total number of findings present on exam. On average, the DEMS score increased by 0.66 points for each additional finding (p = 0.016).

DISCUSSION

Our results indicate that UNC DEMS is a valid and reliable questionnaire that can help meet the growing need for a patient-centric measure of DED that is efficiently administered in a busy clinical setting. In the present study, the DEMS demonstrated excellent test-retest reliability in a moderate sample size of both dry eye patients and controls. Some may suggest that the test-retest reliability is subject to question because patients filled out a mailed or online version of the questionnaire (rather than completing the questionnaire again in the clinic). Completing a mailed or emailed version of the questionnaire in a non-clinical setting, however, should reduce or have no effect on the test-retest reliability of the questionnaire, rather than artificially inflating it. Thus, our estimate of 0.90 for the reliability coefficient is most likely a conservative estimate. In addition, our estimate of 0.81 for the reliability coefficient for the OSDI is consistent with the estimate of 0.82 initially reported by Schiffman, et al.¹¹ Our finding of similar reliability coefficients in the OSDI further strengthens our confidence in the test-retest reliability of the DEMS.

Our validity testing showed that the DEMS has a clear ability to discriminate between normal, mild-to-moderate, and severe DED status as defined by the patient. In addition, the DEMS was able to discriminate between normal and mild-to-moderate disease as defined by the clinician. The ophthalmologists who graded disease severity in this study only labeled two patients as having severe disease despite the fact that 14 patients rated their own disease status as severe. This finding is in accordance with the findings of two studies showing that clinicians routinely underestimate both patient symptoms and disease effect on QOL.^{21, 22} Such results further highlight the discrepancy between patient and physician interpretation of disease severity in the dry eye population.

In general, correlation coefficients < 0.35 are considered low correlations, coefficients 0.36 – 0.67 are considered moderate, and coefficients > 0.68 are considered strong.²³ Using these definitions, the DEMS score correlated strongly with the OSDI score, the current gold standard of symptom and QOL measurement in dry eye patients. In addition, the DEMS showed moderate correlation with frequency of artificial tear use. The DEMS showed low-to-moderate correlation with clinical measures of DED status (fluorescein staining and TBUT), which is consistent with the clinical correlations of other PROs in DED such as the OSDI and IDEEL.^{11, 12}

It is important to note that even though scores of the UNC DEMS and OSDI are highly correlated, the UNC DEMS is not a replacement for either the OSDI or IDEEL. Both the OSDI and IDEEL will likely remain valuable patient-reported measures in the clinic and especially in clinical trials.^{2, 4–6, 8, 9, 14} Although both of these latter instruments have served as excellent measures of DED’s effects on a patient’s daily living in clinical trials, both are multi-item questionnaires that have a relatively high burden of administration for patients, and consume valuable clinic time for scoring.^{11, 12, 14} Alternatively, the DEMS is a single-item questionnaire and may prove especially useful in these time-limited clinical settings. In addition, because the DEMS is a validated, reliable questionnaire, it may also serve a valuable role as an end-point in clinical trials alongside the OSDI and IDEEL, especially if either time or funding of such trials is limited.

One additional advantage of the UNC DEMS is that it includes a time frame (1-week) for patient reporting. Many current dry eye surveys neglect to include any type of time frame, which in turn could lead to a wide variation in patient responses depending on whether they are trying to report current symptoms, average symptoms, or most severe symptoms.⁴ For example, when a time-frame is not included in a questionnaire, some patients may choose to report their worst symptoms since their last clinical visit whereas other patients may only report what their specific symptoms are that day. The one-week time frame in the UNC DEMS provides a reference point that allows patients to account for the fluctuations in their symptom severity over the past week while also covering a small enough period of time that patients can easily recall their symptoms and the overall effect on their QOL.

Our study is not without limitations. First, our sample size consisted of 66 patients at the UNC Ophthalmology Clinic. Although the recruited patient population was very diverse, our institution-specific research and moderate sample size may limit the generalizability of our questionnaire. Therefore, we encourage other researchers to perform trials at their respective institutions to test for broader generalizability of this tool. Second, the DEMS showed discriminative capability between normal, mild-to-moderate, and severe disease status as reported by the patients in this study as well as between normal and mild-to-moderate physician-reported disease severity. Its ability to discriminate severe disease status (as determined by physicians), however, is unclear, since the ophthalmologists in this study labeled only two patients as having severe disease. More research should be done to further validate this questionnaire’s discriminative capability. Finally, if the DEMS is to be used as a disease-monitoring tool, physicians need to know what score change on the scale would constitute a minimal clinically important difference (MCID). The results of this study do not clearly demonstrate the MCID of the DEMS; a study at UNC, however, is currently underway to determine the MCID.

The UNC DEMS is currently undergoing further research. In addition to the UNC study mentioned above, we have also commenced a study of a culturally appropriate DEMS questionnaire in Spanish. These two studies together will help fulfill the final two PROMIS® guidelines for PRO development of Interpretability and Language Translation/Cultural Adaption.¹⁸

In conclusion, the UNC DEMS is a unique, single-item questionnaire that provides a snapshot of a patient's overall experience – symptoms and quality of life – with dry eye in the past week. It has excellent test-retest reliability as well as strong validity. Unlike more complicated, time-consuming questionnaires, the DEMS is easily understood and requires little time to administer in the clinic.¹⁹ As dry eye is a chronic condition requiring both self-management as well as physician-management, the UNC DEMS offers a bridge between patient and provider in working together to manage this challenging disease. Our questionnaire is available with instructions for use at https://www.med.unc.edu/ophth/for-patients/clinical-specialties/unc-dry-eye-management-scale. It is our aim that this tool will assist physicians and their patients in the monitoring and management of dry eye disease in clinical practice.

Appendix A: COGNITIVE INTERVIEWING

Cognitive Interviewing Methods

After the initial questionnaire was developed, two members of the research team who had been trained in cognitive interviewing administered the UNC DEMS and the OSDI to 18 patients with dry eye disease (DED) and subsequently performed cognitive interviewing on those patients. We selected a broad range of subjects in order to include an adequate representation of sex, race, age and severity differences in DED patients. After completing the UNC DEMS and the OSDI, patients were asked several questions by the interviewers about the UNC DEMS. A list of these questions is provided at the end of this Appendix. The interviewers recorded patient responses to cognitive interviewing and the investigative team used the responses to guide the modification of the UNC DEMS. The first 7 subjects completed the initial questionnaire (Appendix Figure 1), and the subsequent subjects received a modified version of the questionnaire. All patients were asked similar cognitive interviewing questions regardless of which questionnaire they received. Patient responses ultimately guided the creation of the final version of the questionnaire (Appendix Figure 2).

Cognitive Interviewing Results

Of 19 patients recruited, 18 agreed to participate in our study. The median age of participants was 65 (range 47–90); 50% of participants were female. Seventy-two percent of participants were Caucasian, 17% were African-American, and 11% were of other race. In general, patient responses from the cognitive interviewing process indicated that patients understood and liked the new UNC DEMS questionnaire. However, responses from the first seven patients interviewed indicated that patients were selecting a number based on the examples of daily life (which were initially provided in the tags underneath the number scale) rather than based on symptom severity and overall effect on quality of life. For example, a patient may describe his or her symptom severity and disease-effect on quality of life as very mild, but because dry eye occasionally blurred vision while reading, corresponding to a higher number, such patients would select 6 on the scale instead of a lower number reflecting their current assessment of symptoms and quality of life. This patient understanding resulted in a clustering of scores around the middle of the scale. We used this patient feedback to move the examples of daily life from the tags underneath the numbers to the main text of the questionnaire. Other modifications of the scale were also based on patient responses to interview questions but were more minor. For example, we changed the term describing symptom severity in the 7–8 tag from “severe” to “very bothersome” as well as adding the phrase “doing things you enjoy” as an example of daily life. At this time, we also changed the questionnaire’s name from the Dry Eye Symptom Severity Score to the UNC Dry Eye Management Scale (DEMS).

After these modifications were made, we administered a modified UNC DEMS to another 11 patients. Responses from these patients indicated that the changes made to the scale increased the comprehensibility of the questionnaire. In addition, after we had moved the examples of daily life to the questionnaire’s main text, patients found it easier to select a number based on the combined effect of symptom severity and DED’s overall influence on quality of life over the last week.

During this entire cognitive interviewing process, several subjects also indicated a desire to discuss the details of their DED and UNC DEMS score with their ophthalmologist. Therefore at the conclusion of the cognitive interviewing, we added the question “Is there anything else you would like your doctor to know about your eyes?” to the UNC DEMS (Appendix Figure 2). This supplemental question was placed underneath the scale of the UNC DEMS to serve as a springboard for discussion with the physician about a patient’s dry eye status.

Question Template Used During Cognitive Interviewing

1. What do you think the question is asking about?

2. I asked you to use a 1 to 10 scale.

   1. Can you describe what you are thinking of when you think of a 1?
   2. And what about the 10—what are you thinking of?
   3. What about the middle? What are you thinking of when you think of the 5?
3. People usually say the symptoms of dry eyes are pain, burning, grittiness, “feeling like something’s in your eye”, tearing, and sensitivity of light. I’d like to take each of these, and ask what they mean to you?

   1. First, let’s start with pain. How does pain from dry eyes feel to you?
   2. Next, let’s talk about burning. How does burning from dry eyes feel to you?
   3. Let’s move on to grittiness. How does grittiness from dry eyes feel to you?
   4. Ok, now “feeling like something’s in your eye”. What does “feeling like something’s in your eye” caused by dry eyes feel like to you?
   5. How about tearing? How does tearing caused by dry eyes feel to you?
   6. Now our last symptom—sensitivity to light. How does sensitivity to light caused by dry eyes feel to you?
   7. Do you experience any other symptoms from dry eyes?____________________ How does __(previous answer)__ feel to you?

4. In the questionnaire, we asked about how your dry eye symptoms affected your daily life. What does daily life mean to you?

   Follow-up: Can you give me an example of daily life?

5. When you picked a number on the scale, were you thinking more about the dry eye symptoms, the effect on your daily life, or both?

   Follow-up: When you say __(previous answer)__, tell me what comes to mind.

6. How do your dry eye symptoms affect your daily life?

7. We gave examples to give you an idea of what the numbers on the scale might mean. Do you think the examples we gave match the numbers on the scale?

   Follow-up: Can you think of any other examples for the numbers?

8. We used the words “mild” “moderately” and “severe” to describe your dry eye symptoms.

   1. Do you feel the word “mild” matches a 3–4 on the scale?
   2. Do you feel the word “moderately” matches a 5–6 on the scale?
   3. Do you feel the word “severe” matches a 7–8 on the scale?
9. Similarly, we used the words “hardly” “sometimes” and “frequently” to describe how the symptoms may have affected your daily life.

   1. When the scale says “hardly affects your daily life” does that match a 3–4 on the scale?
   2. When the scale says “sometimes affects your daily life” does that match a 5–6 on the scale?
   3. When the scale says “frequently affects your daily life” does that match a 7–8 on the scale?

10. Does this scale let you say how good or bad your eyes are feeling?

11. How easy or hard is it to remember how your dry eyes affected you in the past week?

    Follow-up: How do you remember this?

12. Did the 10 point scale let you say enough about what you are feeling that you didn’t need to answer the other questions we gave you [the OSDI] OR was answering the other questions helpful?

13. Is there anything else we’ve forgotten or that you think we need to know to make this a good scale in describing your dry eye symptoms and the effect on your daily life?

Appendix A Figure 1: Draft version of The University of North Carolina Dry Eye Management Scale (UNC DEMS).

Appendix A Figure 2: The University of North Carolina Dry Eye Management Scale (UNC DEMS).

Appendix B: Instructions for Administering the UNC DEMS

The UNC DEMS was administered during the validation process. The following is the method and language currently used to administer the UNC DEMS:

The UNC DEMS should be administered at each outpatient visit to patients with a history of dry eye disease, or to any patient with dry eye symptoms such as redness, burning, irritation, foreign body sensation, and/or tearing. When the eye care provider gives the UNC DEMS to the patient, say the following, "Please read the UNC Dry Eye Management Scale.” The provider should then point to the examples in the boxes and say, "Read the examples below and please select the one number that best represents your symptoms and their effect on your quality of life over the last week." The patient should then be given a few seconds to read the scale and circle the one number that best answers the question.

The UNC DEMS is available at the following link for non-profit clinical and research use: https://www.med.unc.edu/ophth/for-patients/clinical-specialties/unc-dry-eye-management-scale. It is also provided below for your reference.

Appendix B Figure 1: The University of North Carolina Dry Eye Management Scale (UNC DEMS).