Development of the Northwestern Esophageal Quality of Life Scale: A Hybrid Measure for Use Across Esophageal Conditions

Abstract

OBJECTIVES

Measures of health-related quality of life (HRQOL) in chronic esophageal conditions such as gastroesophageal reflux disease, eosinophilic esophagitis, and achalasia are widely used to measure this important patient-reported outcome. We seek to leverage these existing measures to create a hybrid measure of esophageal illness HRQOL (the Northwestern Esophageal Quality of Life—NEQOL), allowing for broad use across diseases while maintaining sensitivity to nuances of a specific condition.

METHODS

A three-step, mixed-methods process per FDA guidelines for patient-reported outcome (PRO) development was followed: review and consolidation of existing HRQOL measure items into a single questionnaire, reliability and validity analyses (principle components factor analysis, Cronbach alpha, Guttman split-half, inter-item correlation, test–retest correlation, and Pearson’s correlation with related constructs) based on responses from a representative sample of esophageal illness patients, and individual structured cognitive interviews with patients for item refinement and reduction.

RESULTS

An initial 30-item measure was created. Two-hundred twelve patients completed the reliability and validity portion of the study, and 15 completed cognitive interviews. Factor analysis and item-reduction resulted in 11 items being removed from the NEQOL prior to patient interviews. Construct validity was supported by moderate and significant correlations with psychological distress and general HRQOL. Test–retest reliability was excellent. Following patient interviews, an additional 5 items were removed because of floor effects or participant feedback yielding a 14-item, single scale measure of HRQOL.

CONCLUSIONS

Although more research is warranted, the NEQOL is a reliable and a valid hybrid measure of disease-specific HRQOL across several chronic esophageal conditions.

INTRODUCTION

Esophageal disorders, ranging from gastroesophageal reflux disease (GERD), which affects up to 1/3 of the population (1), to less common but still prevalent conditions such as achalasia, eosinophilic esophagitis (EoE), Barrett’s esophagus, and dysphagia, are associated with significant disease burden, disability, and cost (2–4). Health-related quality of life (HRQOL) is a multidimensional construct that captures the physical, mental, social, and emotional aspects of a patient’s life and how health status impacts these domains. HRQOL can be readily measured through patient self-report and correlates with outcomes in a wide range of digestive diseases (5–9). There are two approaches to the measurement of HRQOL—generic and disease specific—with pros and cons to each approach (10). Generic measures such as the MOS Short Form 36 (SF-36) or National Institutes for Health (NIH) Patient Reported Outcomes Measurement Information System (PROMIS) are useful because they allow for comparison of HRQOL across disease groups (e.g., GERD vs. diabetes) and against the general population. Disease-specific tools are also useful in that they yield important nuances around QOL in the esophageal population that would not otherwise be detected.

Disease-specific HRQOL measures exist for GERD (11,12), achalasia (13), eosinophilic esophagitis (7), dysphagia (14), and Barrett’s esophagus (15) but are not without limitation—one major concern is that the existing measures combine severity of disease with social/emotional impact, making it difficult to isolate the true impact of disease on HRQOL, as severe symptoms do not necessarily correlate with worsened quality of life (8,16). The second major concern about existing esophageal measures is that most of them are not transportable across clinical and research settings due to length, inefficiency of administration (e.g., having to give a different questionnaire to each presenting group), or they lack the complexity preferred by either the researcher or the clinician.

Our aim was to develop an easy-to-use, valid measure of esophageal HRQOL that addresses many of the existing concerns: (1) expeditiously and sufficiently captures its important domains specific to esophageal conditions; (2) delineates HRQOL from disease-specific symptom severity, and (3) can be used more broadly across diseases of the esophagus.

MATERIALS AND METHODS

Scale Development and Data Collection

Following recommended guidelines for patient-reported outcome (PRO) measures development (FDA, 2009) (17), this study consisted of a three-step process for patient-reported outcome development (Figure 1).

Figure 1.

Three-step process for PRO development of NEQOL. HRQOL, health-related quality of life; NEQOL, Northwestern Esophageal Quality of Life; PRO, patient-reported outcome; PCFA, Partial Confirmatory Factor Analysis.

Step 1

Existing measures of HRQOL for GERD, dysphagia, achalasia, EoE, and Barrett’s esophagus were reviewed for content by three study investigators (TT, LK, and JP): Gastrointestinal Quality of Life Index (18), Quality of Life in Reflux and Dyspepsia Patients (19), GERD Analyzer (20), GERD Assessment Scales (21), Nepean Dyspepsia Index (22), MD Anderson Dysphagia Inventory (23), Laryngopharyngeal Reflux HRQOL scale (24), Reflux-QUAL (25), Eosinophilic Esophagitis QOL-A (7), Achalasia QOL (13), and the GERDQ (26). Relevant themes related to HRQOL were identified, and items from each questionnaire grouped by theme. Next, items were evaluated for redundancy, quality, ease of understanding, and relevance to the HRQOL construct. Items that were deemed to be poor quality, difficult to understand, or not relevant to the HRQOL construct (i.e., symptom rating questions) were removed. The study investigators’ recommendations were compared for consensus.

Step 2

The preliminary measure was given to a representative sample of patients with a confirmed diagnosis of GERD, Barrett’s esophagus, EoE, achalasia, or dysphagia of unknown etiology for a minimum of 6 months. Participants between the ages of 18 and 70 were recruited from a university-based outpatient gastroenterology practice and via a research dedicated website (www.researchmatch.org). After informed consent was obtained, participants completed a series of online questionnaires:

Inclusion/Exclusion Screening

Esophageal diagnosis, GI comorbidity (e.g., irritable bowel syndrome, inflammatory bowel disease, or other GI condition), serious mental illness (e.g., schizophrenia and bipolar disorder), or other chronic illness was recorded. Patients who did not report an esophageal condition or had medical or mental health co-morbidity were excluded from the study to eliminate potential HRQOL confounders related to other conditions.

Demographic and Clinical Information

Gender, age, race, ethnicity, education, income, geographic region of U.S., hometown population, misdiagnosis status (yes/no), number of outpatient GI visits in past year, number of current GI medications, use of dietary treatment, use of dietitian, number of endoscopic procedures in past year, use of therapist/counselor for GI illness, treatment adherence over past week rating (0–100), symptom severity rating (0–10), symptom expectation rating (0–10), symptom control rating (0–10), treatment efficacy rating (0–10), treatment satisfaction rating (0–10), physician satisfaction rating (0–10).

Northwestern Esophageal Quality of Life Scale (NEQOL)

The preliminary version of the NEQOL was constructed after review of validated measures of HRQOL for esophageal conditions. The initial version of the NEQOL contains 30 items rated on a 5-point Likert scale (very true–not at all true). Items aim to capture relevant domains of HRQOL including social function, emotional distress, eating impact, sleep, and financial burden. Each item is coded from 0 to 4, and a total score is obtained by summing all items.

Centers for Disease Control (CDC) Healthy Days HRQOL Scale

The Centers for Disease Control healthy days is a standard measure of HRQOL (27). The measure contains three modules that may all be used or be used individually: 4-item core module, 5-item activity limitations module, and 5-item healthy days symptom module. For this study, the core and symptom modules were used.

Impact of Illness Scale (IIS) (28)

The IIS is a 9-item measure of the degree of psychosocial impact of an illness. Questions are answered on a 4-point Likert scale from not at all to fully. Scores range from 0 to 27 with higher scores denoting a greater negative impact. The IIS demonstrates good reliability (α =0.88 to 0.93) and validity.

NIH PROMIS Anxiety and Depression Scales (29)

The NIH PROMIS scales are validated measures of psychological distress. The anxiety scale contains seven items and the depression scale contains eight items on a 5-point Likert scale (never to always). Higher scores indicate greater anxiety and depression.

After completing the study, participants had the option to provide contact information to participate in the test–retest portion of the study approximately 2 weeks after completing the questionnaires. Those who opted to participate were sent the NEQOL to complete a second time within 14 days of completing step 2. Duration between time 1 and time 2 was recorded, as well as any changes to treatment and general ratings of symptom severity and control on a 10-point scale that were also collected at first administration of the NEQOL.

Step 3

After initial reliability analyses, validity, principle components factor analysis, and item reduction were completed on the preliminary NEQOL, we conducted cognitive interviews with patients in order to verify comprehension and relevance of the NEQOL questions. Individuals aged 18–70 with a confirmed diagnosis via medical record of GERD, Barrett’s esophagus, achalasia, dysphagia NOS, and eosinophilic esophagitis for at least 6 months were recruited. Individuals were invited to participate during an outpatient visit to the gastroenterology clinic, or over the phone, by a graduate student in clinical psychology (AB). Participants completed a phone pre-screen for exclusion criteria above then came in to the Division of Gastroenterology Academic Offices for an individual study session. After providing consent, participants answered demographic questions and then participated in an audio-recorded interview. During the interview, they responded to the NEQOL items and answered follow-up questions to determine item comprehension and overall quality of the measure.

Statistical analysis

Data from the online system for step 2 were exported into SPSS v.22 for analyses. Preliminary descriptive statistics evaluated the data for normal distribution. Reliability of the NEQOL was evaluated via Cronbach alpha and the Guttman statistic for split-half reliability. Validity was measured by correlations with established HRQOL measures (Centers for Disease Control Healthy Days and IIS) and measures of psychological distress (NIH PROMIS). Principle components factor analysis with Varimax rotation evaluated the factor structure of the NEQOL. Inter-item correlations were evaluated with cutoff scores set for r >0.80 or <0.20 to indicate items for potential removal from the questionnaire. Test–retest reliability was evaluated with Pearson’s correlations between time 1 and time 2 of the NEQOL administration. Paired samples t-tests evaluated any changes in NEQOL scores, illness severity, and symptom control between assessment periods, and independent samples t-tests evaluated differences in NEQOL scores for participants who made treatment adjustments compared with those without adjustments.

Individual interviews for step 3 were transcribed for content analysis. For each participant, survey items were classified as clear or confusing. Following analysis of all items, a tally was made in order to determine which items were clear and straight-forward to everyone and which were confusing to many or some participants. Participants were asked whether any changes should be made to the items, and those suggestions were noted. Ceiling and floor effects were examined by determining whether greater than or equal to 80% of participants answered not at all true or very true to items. Each item was individually presented to study investigators with interview feedback for item refinement and reduction.

RESULTS

Step 1

After review of existing disease-specific HRQOL measures, a total of 30 items were included in the initial NEQOL.

Step 2

Three hundred and eighty-seven potential participants visited the study website and completed screening questions. Of these, 4 were excluded for not having the proper esophageal diagnosis, 15 for not having their diagnosis for a minimum of 6 months, 103 for having a co-morbid GI diagnosis (e.g., inflammatory bowel disease), and 17 for having a co-morbid serious mental illness (n =139 excluded). Of participants who met the inclusion criteria (n =240), 233 completed all study questionnaires (97% completion rate). The study sample was primarily non-Hispanic Caucasian, female, college educated, and recruited from online sources (Table 1). The most common diagnosis was GERD, followed by EoE and Achalasia. No significant differences existed between recruitment sources (online versus clinic) for NEQOL scores (P =0.90); hence, the entire sample was pooled for analyses.

Table 1.

Demographic and clinical characteristics of initial validation sample

Characteristics	N=233
Demographic
Age	44.0 (13.2)a
Gender
Male	29.8% (71)
Female	70.2% (167)
Race
Caucasian	93.8% (225)
African American	0.8% (2)
Asian	1.3% (3)
American Indian/Alaskan Native	0.8% (2)
Multi-racial	2.1% (5)
Other	1.3% (3)
Ethnicity
Hispanic	3.0% (8)
Non-Hispanic	97.0% (219
Education
Less than college degree	26.2% (62)
College degree or higher	73.8% (175)
Income
Less than $50,000 per year	28.9% (67)
More than $50,000 per year	71.1% (165)
Recruitment source
Outpatient clinic	17.6% (42)
Online	82.4% (191)
Clinical variable
Diagnosis
Achalasia	12.9% (30)
Barrett’s esophagus	5.6% (13)
Dysphagia NOS	2.6% (6)
Eosinophilic esophagitis	24.0% (56)
GERD	45.9% (107)
More than 1 of the above	9.0% (21)
Illness severity (out of 10)	6.72 (2.06)
Symptom control past month (out of 10)	7.39 (2.52)
Treatment effectiveness (out of 10)	7.63 (2.58)
Treatment satisfaction (out of 10)	7.56 (2.75)
Satisfaction with MD relationship (out of 10)	7.94 (2.94)
Treatment adherence past week (out of 100%)	83.22 (24.89)
Number of outpatient visits past year	2.91 (4.79)
Number of endoscopies past year	0.91 (1.29)
Number of current medications	1.16 (0.86)
Current dietary treatment	39.1% (91)
History of seeing dietitian for illness	19.1% (45)
History of seeing therapist for Illness	6.8% (16)

GERD, gastroesophageal reflux disease.

^aMean (s.d).

Reliability analyses of the 30-item NEQOL yielded excellent internal consistency (Cronbach’s α=0.96) and split-half reliability (Guttman statistic=0.94). Principle components factor analysis yielded a five-factor structure; however, the majority of items fell on Factor 1 (eigenvalue=15.24). Inter-item correlations indicated three items with poor correlations (<0.20), with most other items on the scale and three items with high inter-item correlations (>0.80) with at least one other question; five of these items were subsequently removed from the scale. The item “I worry that I may develop esophageal cancer in the future” was retained based on study investigator opinion of the importance to capture this concern, especially among patients with Barrett’s esophagus and achalasia. Study investigators (TT, LK, and JP) evaluated the remaining items for redundancy for further reduction or modification. The intermediate version of the NEQOL subsequently contained 19 of the original 30 items to be evaluated via cognitive interviews.

The NEQOL demonstrated excellent construct validity, supported by strong positive correlations with the Centers for Disease Control healthy days, IIS, and the NIH PROMIS Scales (Table 2). Overall, moderate correlations existed (0.47–0.76), indicating separate yet related constructs. In addition, the NEQOL significantly correlated with disease-specific scores of illness severity (negative relationship) and with symptom control and treatment effectiveness (positive relationship; Table 2). HRQOL was also associated with poorer physician–patient relationship rating and lower treatment satisfaction.

Table 2.

Construct validity correlational measures of the NEQOL

	NEQOL score	P
CDC healthy days
General health rating	0.51	0.000
Days poor physical health	0.60	0.000
Days poor mental health	0.47	0.000
Days limitation due to health	0.50	0.000
Impact of illness scale	0.76	0.000
PROMIS—anxiety	0.54	0.000
PROMIS—depression	0.53	0.000
Illness severity	−0.39	0.000
Symptom control past month	0.54	0.000
Treatment effectiveness past month	0.54	0.000
Treatment satisfaction	0.60	0.000
MD–patient relationship quality	0.32	0.000

CDC, Centers for Disease Control; NEQOL, Northwestern Esophageal Quality of Life; PROMIS, Patient Reported Outcomes Measurement Information System.

Ninety-two participants completed the test–retest portion of the study. Of these, 69 (75%) reported no change in their esophageal treatment since the first assessment. On average, participants rated their illness severity at 5.35(2.15) and their symptom control at 5.65(2.50) out of 10 over the previous 30 days. Test–retest reliability was above acceptable cutoffs (r =0.88, P <0.001), indicating excellent temporal stability in NEQOL scores between administration points with a mean time between administrations of 32 days. A significant decrease in self-rated illness severity existed between time 1 and time 2 (6.61 vs. 5.59, P <0.01); no significant changes existed for symptom control and overall QOL. Participants who made treatment changes reported significantly poorer symptom control over the past 30 days (P =0.02) but reported better HRQOL than those whose treatment remained stable (P =0.03); illness severity did not change.

Step 3

A total of 15 people participated in the cognitive interviews. The sample consisted primarily of middle-aged, non-Hispanic Caucasians and was evenly distributed by gender (Table 3). Most attained a college or a post-graduate education, and the majority were employed at a full or a part-time status. The majority were diagnosed with EoE and achalasia, with fewer diagnosed with GERD or dysphagia. We were unable to recruit any patient with Barrett’s esophagus because available patients were either ineligible because of being over age 70 or had received corrective surgery and no longer had Barrett’s esophagus.

Table 3.

Demographic characteristics of cognitive interview participants

Demographic	N=15
Diagnosis
Eosinophilic esophagitis	5
Achalasia	5
GERD	3
Dysphagia	2
Symptom rating
None	5
Mild	3
Moderate	5
Severe	1
Very severe	1
Education
Decline to answer	1
High school	2
Some college	2
College	6
Postgraduate	4
Employment
Full-time	8
Part-time	3
Retired	1
Unemployed	2
Disability	1
Age (years)
Mean	55.3
S.D.	13.3
Time since diagnosis (years)
Mean	13
S.D.	10.4
Length of interview
Mean	35.4
S.D.	16.4
Sex (%)
Female	60
Male	40
Race (%)
Non-Hispanic White	86.7
Other	13.3

GERD, gastroesophageal reflux disease.

Overall, five items described as vague, redundant, or demonstrating significant floor effects were removed from the NEQOL resulting in a 14-item final version (Appendix A). Other items were reworded based on participant feedback (i.e., Item 7, “I’m concerned eating will make me feel ill”, the word “ill” was changed to “sick”, because of ambiguity expressed by participants regarding the word “ill”; Item 12, “I avoid social activities that I once enjoyed” was modified to say “I limit my social activities (e.g., hobbies, exercise, sex) because of my esophageal condition” in order to provide examples; Item 17, “I feel frustrated” was modified to say “I feel angry or frustrated about my esophageal condition”, in order to incorporate a range of emotions expressed by participants during the interviews.

DISCUSSION

We found the NEQOL to be a reliable and a valid measure of health-related quality of life in a large cohort of patients with GERD, EoE, achalasia, dysphagia, and Barrett’s esophagus. This study took established disease-specific esophageal HRQOL measures and consolidated them into a 14-item survey that allows for rapid assessment in a clinical setting or cross-disease comparisions in a research study. Measures of internal consistency, split-half reliability, test–retest reliability, and construct validity were at or above established guidelines for scale development.

Food and Drug Administration guidelines for the development of reliable and valid patient-reported outcomes outline a three-phase paradigm to meet standards of acceptability: (1) initial qualitative inquiry with patients affected by the condition to guide question development, (2) administration of the measure to a sufficient and a representative sample in order to conduct quantitative psychometric analyses, and (3) qualitative inquiry via cognitive interviews to ensure that feasibility, acceptability, and construct validity are sufficient. This study follows Phases 2 and 3 of these guidelines. In lieu of Phase 1, the initial qualitative inquiry to inform question development, established and validated questionnaires guided development of the initial questionnaire.

The development of the NEQOL as a “hybrid” measure between generic and disease-specific measures of HRQOL allows for broad use across diseases while maintaining sensitivity to the nuances of symptoms of a specific disease. Hybrid measures, such as this one, have been developed in other health domains including chronic illness as a whole (30), heart disease (31), and liver disease (32). The NEQOL has the potential for both clinical and research utility; further validation is warranted in larger, more racially diverse samples and among conditions that were poorly represented in the current study (i.e., Barrett’s esophagus).

The NEQOL correlated with several disease-specific scores including self-reported symptom severity, symptom control, and general HRQOL. Poorer HRQOL was modestly correlated with increased esophageal symptoms, poorer symptom control, and reduced treatment effectiveness. These findings are similar to preexisting literature on the relationship between HRQOL and these patient outcomes (33–35). In addition, patient satisfaction with treatment and their relationship with their physician were also negatively associated with HRQOL scores. Future studies should evaluate the NEQOL with other disease-specific measures of HRQOL such as the GERDQ and Achalasia QOL to confirm these relationships.

Several limitations should be taken into consideration when interpreting our results. In part 2 of the study, the majority of patients were recruited online (82.4%), and therefore we were unable to confirm their diagnosis. However, comparison of NEQOL scores between clinic and online sources demonstrated no significant differences between groups. In addition, the sample was primarily Caucasian, non-Hispanic, and highly educated; hence, caution should be applied when using the NEQOL in diverse populations. Finally, as we were unable to recruit any patient with Barrett’s esophagus for part 3, we should use caution when applying the NEQOL to this diagnosis group. As such, additional validation is recommended before it is fully ready for clinical or research purposes.

The NEQOL is designed as a rapid assessment tool of HRQOL in clinical settings. More detailed, disease-specific measures remain the preferred method of evaluating HRQOL for research purposes until the NEQOL is sufficiently evaluated across multiple samples. For studies evaluating several esophageal disorders, use of the NEQOL versus generic measures of HRQOL has merit following further validation in more diverse samples. In summary, the NEQOL is an easy-to-use, valid measure of Esophageal HRQOL that addresses concerns related to existing measures, capturing domains specific to several diseases of the esophagus while delineating HRQOL from symptom severity.

Study Highlights.

WHAT IS CURRENT KNOWLEDGE

• Health related quality of life (HRQOL) correlates with outcomes in a wide range of digestive diseases and can be measured through patient self-report.

• Two types of HRQOL measures exist—generic and disease specific—and both have strengths and limitations.

• Generic measures allow for comparisons across disease groups but may overlook nuances of symptoms associated with particular conditions.

• Disease-specific measures exist for gastroesophageal reflux disease (GERD), achalasia, eosinophilic esophagitis, achalasia, dysphagia, and Barrett’s esophagus but have significant limitations because of the inclusion of symptom severity, which does not always correlate with worsened quality of life.

WHAT IS NEW HERE

• The Northwestern Esophageal Quality of Life (NEQOL) is a “hybrid” measure between generic and disease-specific measures that allows for broad use across diseases while maintaining sensitivity to the nuances of symptoms of a specific disease.

• The NEQOL is a reliable and a valid measure of HRQOL that is easy to administer in clinical settings with potential utility in research.

APPENDIX A

Final version of the NEQOL

	Not at all true	Somewhat true	Neither true nor untrue	Somewhat true	Very true
1 Because of my esophageal condition, I find eating and/or food to be much less enjoyable because of the changes I have had to make to my eating habits (e.g., remove foods from my diet, eat smaller meals, eat slowly).
2 I worry something more serious is wrong with my esophagus.
3 I am concerned other people may notice my esophageal condition and make comments or pass judgments.
4 I feel tired, worn out, or fatigued because of my esophageal condition.
5 I’m concerned eating will make me feel sick.
6 I worry that I may develop esophageal cancer in the future.
7 I feel ashamed because of my esophageal condition.
8 I have trouble sleeping at night.
9 I feel uncomfortable eating outside my home (e.g., at a restaurant, a friend’s house).
10 I limit my social activities (e.g., hobbies, exercise, sex) because of my esophageal condition.
11 Because of my esophageal condition I feel generally unwell much of the time.
12 I feel worried and/or sad most days because of my esophageal condition.
13 I feel angry or frustrated about my esophageal condition.
14 I cannot perform my usual daily activities at home, work, or school as often as I would like because of my esophageal condition.