Patient-reported outcome measures in dysphagia: a systematic review of instrument development and validation

SUMMARY

Objective: Patient-reported outcome (PRO) measures are commonly used to capture patient experience with dysphagia and to evaluate treatment effectiveness. Inappropriate application can lead to distorted results in clinical studies. A systematic review of the literature on dysphagia-related PRO measures was performed to (1) identify all currently available measures and (2) to evaluate each for the presence of important measurement properties that would affect their applicability.

Design: MEDLINE via the PubMed interface, the Cumulative Index of Nursing and Allied Health Literature, and the Health and Psychosocial Instrument database were searched using relevant vocabulary terms and key terms related to PRO measures and dysphagia. Three independent investigators performed abstract and full text reviews. Each study meeting criteria was evaluated using an 18-item checklist developed a priori that assessed multiple domains: (1) conceptual model, (2) content validity, (3) reliability, (4) construct validity, (6) scoring and interpretation, and (7) burden and presentation.

Results: Of 4950 abstracts reviewed, a total of 34 dysphagia-related PRO measures (publication year 1987–2014) met criteria for extraction and analysis. Several PRO measures were of high quality (MADS for achalasia, SWAL-QOL and SSQ for oropharyngeal dysphagia, PROMIS-GI for general dysphagia, EORTC-QLQ-OG25 for esophageal cancer, ROMP-swallowing for Parkinson's Disease, DSQ-EoE for eosinophilic esophagitis, and SOAL for total laryngectomy-related dysphagia). In all, 17 met at least one criterion per domain. Thematic deficiencies in current measures were evident including: (1) direct patient involvement in content development, (2) empirically justified dimensionality, (3) demonstrable responsiveness to change, (4) plan for interpreting missing responses, and (5) literacy level assessment.

Conclusion: This is the first comprehensive systematic review assessing developmental properties of all available dysphagia-related PRO measures. We identified several instruments with robust measurement properties in multiple diseases including achalasia, oropharyngeal dysphagia, post-surgical dysphagia, esophageal cancer, and dysphagia related to neurological diseases. Findings herein can assist clinicians and researchers in making more informed decisions in selecting the most fundamentally sound PRO measure for a given clinical, research, or quality initiative.

INTRODUCTION

Impaired swallowing or dysphagia refers to a sensation of food or liquid being delayed or hindered in its passage from the mouth to the stomach. It is often classified anatomically as either oropharyngeal or esophageal. Available studies estimate prevalence in community-dwelling persons over age 50 years to be between 15% and 22%,^1–3 while 40% to 60% of assisted living facility and nursing home residents are reported to have feeding difficulties.^3,4 True prevalence of dysphagia is difficult to estimate due to high degree of overlap with other esophageal symptom classes such as heartburn, regurgitation, chest pain, and globus sensation.⁵ Measurement of treatment effectiveness for these disorders can be challenging even in well-designed clinical trials as many patients with dysphagia also have superimposed esophageal or extra-esophageal symptoms of gastroesophageal reflux disease (GERD). Patient-reported outcome (PRO) measures provide a method of capturing the patient experience and can be helpful for use in comparative effectiveness studies.

PRO measures are defined as ‘any report of the status of a patient's health condition that comes directly from the patient without interpretation of the patient's response by a clinician or anyone else’ and are key elements of patient-centered outcomes research.⁶ Including the patient experience related to treatment benefit and harm is now obligatory in the U.S. preapproval regulatory setting, and the US Food & Drug Administration recommends the use of PRO measures as part of the approval process for pharmacological products and devices.⁷ PRO measures have also been highlighted in the National Institutes of Health (NIH) Roadmap and their use is now required in most NIH-funded randomized clinical trials.⁸ Hence, there has been a marked increase in the numbers of dysphagia-related PRO measures. Significant methodological inconsistency exists because of complexity of proper PRO measure development, which is important because the use of questionnaires that do not adhere to psychometrically robust standards can yield spurious data and incorrect conclusions.⁹

A recent systematic review analyzing PRO measures in the surgical treatment of patients with esophageal cancer was published, but did not critically analyze the developmental properties and was focused on primarily identifying the measures that were being used in measuring quality of life after surgery for esophageal carcinoma.¹⁰ The objectives of this study were to perform a comprehensive systematic review of the literature on all dysphagia-related PRO measures in adults and to carefully evaluate each instrument for the presence of key developmental measurement properties. Moreover, this study aims (1) to identify high-quality PROs related to dysphagia and (2) to assist clinicians and researchers in making more informed decisions in selecting the most fundamentally sound PRO measure for a given clinical, research, or quality initiative.

METHODS

Search strategy

An expert librarian searched MEDLINE via the PubMed interface, the Cumulative Index of Nursing and Allied Health Literature, and the Health and Psychosocial Instrument database in July 2015 using relevant vocabulary and key terms related to PRO measures and swallowing disorders. Some examples of keywords used included swallowing, swallowing disorder, ineffective swallow, aspiration, pharyngeal disease, pharyngeal dysfunction, esophageal dysmotility, and feeding difficulty. There were no publication date restrictions in the search parameters and the search was limited to English language publications. Reference lists of included articles and reviews related to measurement of dysphagia were hand-searched to identify potentially pertinent articles.

Study selection

Inclusion and exclusion criteria were developed in consultation with an expert panel that included a statistician with expertise in measurement theory, a research librarian, systematic review methodologists, and researchers and clinicians who treat and study dysphagia-related conditions. Three investigators (DOF, KH, and EN) independently reviewed abstracts for all identified studies, and those meeting predetermined criteria were advanced to full text review. Abstracts that did not provide adequate information to determine eligibility were also advanced to the full text review phase.

Data extraction

Studies that met criteria for full text review underwent data extraction by one reviewer, and a second reviewer independently verified data accuracy. We extracted the basic characteristics of selected PRO measures including name, acronym of the PRO measure, year(s), aims of the measure, population targeted and involved in instrument development, number of patients, mean age and gender distribution, development setting (e.g. tertiary care, community) and country, language, number of items and subscale(s), and response options (e.g. Likert, visual analog scale).

PRO measure assessment

Three investigators (DOF, RS, and DAP) independently assessed each study's methodology using a criteria checklist developed a priori (Table 1).¹¹ In brief, the checklist was designed to help systematic reviewers identify components deemed important to the development of PRO measures including conceptual model, content validity, reliability, construct validity, scoring and interpretation, and respondent burden and presentation. Definitions of these concepts are provided in Table 2.

Table 1.

‘Checklist’ of key characteristics to consider when evaluating a patient-reported outcome (PRO) measure. Instructions: Please indicate, in the ‘score’ column, whether or not the information provided in the citation/source document meets each of the criteria (0 = criterion not met, 1 = criterion met).

	Score
Conceptual model
1. Has the PRO construct to be measured been specifically defined?
2. Has the intended respondent population been described?
3. Does the conceptual model address whether a single construct/scale or multiple subscales are expected?
Content Validity
4. Is there evidence that members of the intended respondent population were involved in the PRO measure's development?
5. Is there evidence that content experts were involved in the PRO measure's development?
6. Is there a description of the methodology by which items/questions were determined (e.g. focus groups, interviews)?
Reliability
7. Is there evidence that the PRO measure's reliability was tested (e.g. test–retest, internal consistency)?
8. Are reported indices of reliability adequate (e.g. ideal: r ≥ 0.80; adequate: r ≥ 0.70; or otherwise justified)?
Construct validity
9. Is there reported quantitative justification that single scale or multiple subscales exist in the PRO measure (e.g. factor analysis, item response theory)?
10. Is the PRO measure intended to measure change over time? If YES, is there evidence of both test–retest reliability AND responsiveness to change? Otherwise, award 1 point if there is an explicit statement that the PRO measure is NOT intended to measure change over time.
Criterion validity
11. Are there findings supporting expected associations with existing PRO measures or with other relevant data?
12. Are there findings supporting expected differences in scores between relevant known groups?
Scoring and Interpretation
13. Is there documentation how to score the PRO measure (e.g. scoring method such as summing or an algorithm)?
14. Has a plan for managing and/or interpreting missing responses been described (i.e. how to score incomplete surveys)?
15. Is information provided about how to interpret the PRO measure scores [e.g. scaling/anchors, (what high and low scores represent), normative data, and/or a definition of severity (mild → severe)]?
Respondent burden and presentation
16. Is the time to complete reported and reasonable? OR, if it is NOT reported, is the number of questions appropriate for the intended application?
17. Is there a description of the literacy level of the PRO measure?
18. Is the entire PRO measure available for public viewing (e.g., published with the citation, or information provided about how to access a copy)?

Table 2.

Definitions of domains deemed important to the development of PRO measures.

Domain	Explanation
Conceptual model	Conceptual model provides a rationale for and description of the concepts and target population that a measure is intended to assess.
Content validity	Content validity refers to evidence that a PRO measure's domain(s) is appropriate for its intended use. Items and conceptual domains should be relevant to the target population's concerns. It should include input directly from patients and also from content experts. There should be a clear description of the process by which included questions were derived.
Reliability	Reliability is the degree to which scores are free from random (measurement) error.
	Internal consistency reliability, the degree to which segments of a test (e.g. individual items) are associated with one another, reflects precision at a single time point.
	Test–retest reliability refers to the reproducibility of scores over two administrations, typically in close temporal proximity, among respondents who are assumed not to have changed on the relevant domains.
	Traditionally cited minimum levels for reliability coefficients are 0.70 for group-level comparisons and 0.90 to 0.95 for individual comparisons. Reliability estimates lower than these conventions should be justified in the context of the proposed PRO measure's intended application.
Construct validity	Construct validity refers to whether a test measures theoretic constructs or traits and directly affects the appropriateness of the measurement-based inferences. Several different forms exist and are outlined below.
	Dimensionality: Empirical demonstration of dimensionality (e.g. factor analysis) provides evidence of whether a single scale or multiple subscales exist in the PRO measure.
	Responsiveness to change (longitudinal validity) is the extent to which a PRO measure detects meaningful change over time when it is known to have occurred. It is predicated on demonstration of both test–retest reliability (stability when no change is expected) and clinically meaningful change when it is expected.
	Convergent validity is the degree to which a PRO measure's scores correlate with other questionnaires that measure the same construct or with related clinical indicators (e.g. diagnostic test). A priori hypotheses about expected associations between a PRO measure and similar or dissimilar measures should be documented.
	Known-groups validity is the degree to which a PRO measure is able to differentiate between groups that empiric evidence has shown to be different (e.g. cases and controls).
Interpretability and scoring	Interpretability is the degree to which the meaning of the scores can be easily understood. A description of how to score the measure should be provided (e.g. summation, algorithm).
	Missing responses are a common occurrence in both clinical and research settings and can affect an end-users ability to interpret results. A prespecified plan for dealing with missing responses can mitigate the risk of bias from missing data.
	Scaling is important for interpretability. Cross-sectional and longitudinal changes in scores need to be contextualized to allow interpretation of their meaning. Ideally scaling should be based on what represents a clinically important or patient-important change in the construct being measured.
Burden and presentation	Burden refers to the time, effort, or other demands placed on respondents or those administering the instrument. This includes number and complexity of items. Literacy level needed to understand and complete the measure is another important consideration of burden. Most experts recommend items be at the sixth-grade reading level or lower; however, this criterion should be contextualized to the intended target population. It is also important to be able to view all the questions included in a measure to ensure that it is useful to the end user.

Each reviewer was trained and calibrated on appropriate application of the checklist using a methodology described separately.¹¹ They were then independently tasked with evaluating all identified PRO measures. Upon completion, reviewers met to discuss and come to consensus on scoring discrepancies. A senior investigator adjudicated remaining discrepancies.

Data synthesis

Data from unique PRO measures demonstrated heterogeneity in constructs, methodology, and intended purpose and therefore were not appropriate for aggregation or meta-analysis. Instead, individual PRO characteristics, measurement properties, and functionality were summarized independently.

RESULTS

Figure 1 describes the disposition of studies identified for the review. Of the 4950 studies identified, 37 studies provided developmental data on 34 dysphagia-related PRO measures with publication years ranging from 1987¹² to 2014¹³ (Table 3). Common reasons for exclusion were lack of relevance and description of de novo PRO measure development or validation.

Fig. 1.

Disposition of studies identified for this review.

Table 3.

Measurement aims, target population, and item characteristics for patient-reported outcome measures related to dysphagia.

Instrument	Year	Measurement aim	Target population	Language	Number of items/subscales	Response options	Subscales
CS-BED	1987	To design a self-administered questionnaire regarding benign esophageal diseases and test it on a population with known esophageal disease and mimicking disorders	Patients with esophageal disease	Danish	29 items	Dichotomous	NA
NDS	1992	To assess and quantify patient report of eating capacity for items of food	Patients with benign esophageal stricture due to reflux, Barrett's, following surgery, or scleroderma	English	Nine items	Dichotomous	NA
QL-EsoCa	1992	To gain more insight into the (change in) quality of life of patients with operable esophageal carcinoma, by investigating the relations between several quality of life indicators and the changes in QL	Patients who were surgically treated for esophageal carcinoma	English	39 items in five subscales	Four-point scale (1–4)	Physical symptoms Psychological distress Global evaluation Activity level Swallowing problems
EORTC-QLQ-OES24	1996	To develop an EORTC questionnaire module for patients with esophageal cancer that is suitable for QOL assessments before, during and after any single treatment or combination of treatments including chemotherapy, external beam radiotherapy, esophagectomy, intubation, laser treatment, tumor necrosis with ethanol or diathermy, or brachytherapy	Patients with esophageal cancer	English	24 items in six subscales with five-single items	Four-point scale (1–4)	Dysphagia Deglutition Eating Indigestion Pain Emotional
SWAL-QOL	2000	To develop and validate a patient-based, dysphagia-specific quality of life questionnaire to augment information on treatment variations and treatment effectiveness using psychometric tests and patient input	Patients with mechanical or neurologic oropharyngeal dysphagia due to a broad range of etiologies (cancer, vascular, degenerative or other neurologic disease, chronic medical condition, trauma, obstructive respiratory disease, dementia, other or unknown)	English	44 items in 10 domains and symptom scale	Five-point scale (1–5)	Burden Eating Duration Eating Desire Food Selection Communication Fear Mental Health Social Fatigue Sleep Symptoms
SSQ	2000	To design an inventory to measure the severity of oral-pharyngeal dysphagia, to determine its reliability and validity, and to examine its sensitivity to change in response to therapy	Patients with neuromyogenic oropharyngeal dysphagia in whom the severity of dysphagia had been stable for at least 3 months	English	17 items in four domains	VAS (0– 100); five-point scale (1 item)	Dysphagia Drooling Factor 3 Factor 4
MDADI	2001	To design a reliable and valid self-administered questionnaire that can be used to determine the effects of dysphagia (swallowing disability) on the quality of life of patients with head and neck cancer	Patients with a neoplasm of the upper aerodigestive tract	English	20 items in four subscales	Five-point scale (1–5)	Global Emotional Functional Physical
UWQOL-R	2001	To design a global measure of post-treatment quality of life in head and neck cancer patients	Patients after treatment for head and neck cancer	English	10 domains (each with four options)	Dichotomous	Pain Appearance Activity Recreation Swallowing Chewing Speech Shoulder Taste Saliva
Bazaz	2002	To define the incidence and severity of dysphagia over a 12-month postoperative period	Patients who underwent anterior approach spine surgery	English	2 items	Four-point scale (nonesevere)	NA
SWAL-CARE	2002	To develop a patient reported outcome to assess quality of swallowing care and patient satisfaction	Patients with mechanical or neurologic oropharyngeal dysphagia due to a broad range of etiologies	English	15 items in three subscales	Five-point scale (1–5)	Clinical Information General Advice Patient Satisfaction
EORTC-QLQ-EOS18	2003	To test the reliability and validity of the EORTC esophageal cancer module in patients undergoing treatment for esophageal cancer	Patients with newly diagnosed esophageal squamous cell or adenocarcinoma	English* (available in other languages)	18 items in four subscales with six single items	Four-point scale (1–4)	Dysphagia Eating Esophageal pain Reflux
MADS	2005	To develop a measure of disease-specific measure of health-related quality of life for use in patients with achalasia that was valid and reliable and would be useful as an outcome measure in clinical trials	Patients with achalasia	English	10 items	Dichotomous (y/n) and three- to five-point scales	NA
DSST	2006	To assess geriatric patients’ assessment of their clinical symptoms of dysphagia by means of a customized dysphagia screening tool and the usefulness of this assessment to health care professionals	New admissions to long-term care/subacute rehabilitation center	English	Nine items in two subscales	Dichotomous (y/n)	Liquids Foods
MDQ-2W	2006	To validate a self-reported, stem-and-leaf questionnaire similar to the MDQ, to reflect symptoms of dysphagia over a two week period	Patients referred to endoscopy for dysphagia or seen in an outpatients tertiary care clinic	English	29 items	NR	NR
FACT-E	2006	To validate FACT-esophageal in patients with esophageal cancer	Cohort A: esophageal cancer patients with histologically proven resectable squamous or adenocarcinoma of the esophagus or gastroesophageal junction, 18 years of older, English speaking and estimated survival >3 months; Cohort B: esophageal cancer patients scheduled for adjuvant chemoradiation before surgery using same criteria as Cohort A and were medically fit for chemoradiation therapy	English	44 items in six subscales	Five-point scale (0–4)	Physical well-being Functional well-being Social/family well-being Emotional well-being Swallowing Eating
MDQ	2007	To construct and validate a simple instrument to measure dysphagia using items from previously validated questionnaires, encompassing a spectrum of domains including dysphagia onset, frequency and severity, dysphagia for a variety of foodstuffs, previous episodes of food impaction and odynophagia	Outpatients from the esophagus clinic or upper endoscopy suite	English	13–27 items	Dichotomous (y/n), Likert Nonhierarchical, VAS	NA
SDQ	2007	To develop and validate a new self-reported swallowing disturbance questionnaire to identify swallowing disturbances in patients with Parkinson's disease	Patients with Parkinson's disease	English	15 items	Four-point Likert scale (0–3) Dichotomous (y/n; 1 item)	NA
SDQ-D	2011	To develop assess the accuracy of SDQ for detecting swallowing problems among patients with other etiologies besides Parkinson's disease and determine an optimal cutoff score for identifying those who need further evaluation for likely dysphagia	Patients referred for swallowing disturbance	English	15 items	Four-point Likert scale (0–3) Dichotomous (y/n; 1 item)	NA
EORTC-QLQ-OG25	2007	To improve assessment of HRQL in patients with upper gastrointestinal cancer by producing a single EORTC questionnaire module to measure HRQL in patients with esophageal or gastric cancer, including tumors of the esophago-gastric junction	Patients with a histological diagnosis of esophageal or gastric cancer including tumors of the esophago-gastric junction	English* (available in multiple languages)	25 items in six subscales	Five-point scale (0–4)	Dyphagia Eating restrictions Reflux Odynophagia Pain and discomfort Anxiety
	2012	To provide reference values regarding symptoms common among esophageal and gastric cancer patients, based on an unselected adult population
EAT-10	2008	To assess validity and reliability of a comprehensive dysphagia outcome instrument designed for a broad range of dysphagia that is efficient to administer and easily scored	Patients with voice and swallowing disorders	English	10 items	Five-point scale (0–4)	NA
DYMUS	2008	To define and validate a survey to assess swallowing function in patients with multiple sclerosis	Patients with multiple sclerosis	English	10 items in two subscales	Dichotomous (y/n)	Solid food dysphagia Liquid dysphagia
SDI	2010	To test whether a short-term induction monotherapy with budesonide was superior to placebo in achieving histologic remission of active eosinophilic esophagitis and investigated the reversibility of symptoms	Patient over 14 years with histologically confirmed eosinophilic esophagitis	English	Two items	Four- and five-point scales	NA
MDQ-30	2010	To develop and validate a tool for use in clinical trials that will measure esophageal dysphagia with respect to types of symptoms, symptom frequency and severity occurring in the 30 days prior to survey administration	Adults with esophageal disorders	English	28 items in three subscales	Dichotomous, Likert scale, nonhierarchical	Dysphagia Heartburn Regurgitation
DSS	2010	To validate and assess the reliability of the Dutch version of the DHI & MDADI and the Dysphagia Severity Scale in oncological patients with oropharyngeal dysphagia	Patients with oropharyngeal dysphagia due to oncological disorders	Dutch	One item	Visual analog scale (0–100)	NA
SSQ-hn	2010	To validate a self-reported swallowing-specific inventory, the Sydney Swallow Questionnaire as a means of evaluating swallowing impairment in head and neck cancer patients	Patients who received curative surgical treatment for oral and oropharyngeal cancer	English	17 items in two subscales	VAS (0– 100); five-point Likert scale (1 item)	General Quality of life
SLS	2010	To develop a valid and reliable tool for the assessment of oropharyngolaryngeal manifestations of scleroderma in order to obtain a quantifiable and repeatable measure	Patient with scleroderma	Italian, English	39 items in five subscales	Four-point Likert scale (1–4)	Impairment Swallow Voice Multifield Quality of Life
ESQ	2011	To develop an evaluative integrative patient self-report measure that provides a clinical descriptive ‘snapshot’ of common esophageal symptoms and to appraise the reliability and validity of this instrument in clinical setting	Patients presenting with esophageal or throat complaints	English	30 items (frequency, severity) in three domains	Severity: six-point Likert scale (0–5); Frequency: five-point Likert scale (0–4)	Esophageal dysphagia Globus sensation Gastro-esophageal reflux
ROMP-swallowing	2011	To develop a new questionnaire to assess the three domains of speech, swallowing, and saliva controls in individuals with Parkinson's Disease in order to capture symptoms at the levels of functioning and activities, as well as participation	Patients with Parkinson's disease	English, Dutch	23 items (seven items in swallowing domain)	Five-point Likert scale (1–5)	Speech Swallowing Saliva Control
DHI	2012	To develop a patient-reported outcomes tool that is clinically efficient, easy for most patient populations to complete, uses concrete statements supplied from patient complaints, and has daily-use practicality to measure the emotional, physical, and functional effects of dysphagia on the quality of life in individuals with a variety of medical diagnoses affecting swallowing	Patients with dysphagia due to a broad range of dysphagia etiologies	English	25 items in three domains	Seven-point interval scale (1–7)	Emotional Functional Physical
DSQ	2012	To design and validate an easy-to-use patient self-report dysphagia questionnaire and to apply it to a group of patients undergoing anterior cervical disc surgery	Patients undergoing anterior cervical spine surgery	Swedish, English	Five items	Variable Likert scale (0–4)	NA
SOAL	2012	To develop and validate a laryngectomy-specific questionnaire to investigate swallowing function	Patients after laryngectomy	English	17 items	Three-point Likert scale (0–2)	NA
DSQ-EoE	2013	To develop and field test a PRO for dysphagia in adolescents and young adults with eosinophilic esophagitis that meets FDA guidelines and could be used as a reliable outcome measure in future clinical trials	Adolescents and adults with eosinophilic esophagitis	English	Three items	Dichotomous (2) and five-point Likert scale (1)	NA
EEsAI	2014	To develop and validate a PRO instrument for adult eosinophilic esophagitis patients	Patients with eosinophilic esophagitis	English, French, German	45 items in five domains	Variable (NR)	General domain Symptoms food dependent Symptoms food independent Comorbidities Medications
PROMIS-GI	2014	To develop the NIH PROMIS GI symptom scales to be a standardized, rigorously developed, electronically administered set of PRO measures that span the breadth and depth of GI symptoms, and can be used across all GI disorders for clinical and research purposes	Patients with a broad range of GI disorders and complaints	English	102 items in eight subscales (seven items in the swallowing subscale)	Five-point categorical response scale(1–5)	Abdominal pain Gas/bloating Diarrhea Constipation Bowel incontinence/soiling Reflux Swallowing Nausea/vomiting

Bazaz, Bazaz score for dysphagia after anterior cervical spine surgery; CS-BED, clinical symptoms in benign esophageal disease; DHI (1), Deglutition Handicap Index; DHI (2), Dysphagia Handicap Index; DSS, Dysphagia Severity Scale; DSST, dysphagia-specific screening tool; DSQ (1), Dysphagia Short Questionnaire; DSQ-EoE, Dysphagia Short Questionnaire for eosinophilic esophagitis; DYMUS, dysphagia in multiple sclerosis; EAT-10, Eating Assessment Tool-10; EORTC, European Organization for Research and Treatment of Cancer; EORTC-QLQ-OES18, European Organization for Research and Treatment of Cancer Quality-of-Life with esophageal cancer 18 items; EORTC-QLQ-OES24, European Organization for Research and Treatment of Cancer Quality-of-Life with esophageal cancer 24 items; EORTC-QLQ-OG25, European Organization for Research and Treatment of Cancer Quality-of-Life with esophageal Cancer 25 items; ESQ, esophageal symptoms questionnaire; EEsAI, Symptom-based Index for adults with eosinophilic esophagitis; FACT-E, functional assessment of cancer therapy esophageal cancer subscale; MADS, measure of achalasia disease severity; MDADI, M.D. Anderson dysphagia inventory; MDQ, Mayo Dysphagia Questionnaire; MDQ-30, Mayo Dysphagia Questionnaire-30 Day; MDQ-2w, Mayo Dysphagia Questionnaire – 2 week; NDS, new dysphagia score; PROMIS-GI, Patient-Reported Outcomes Measurement Information System Gastrointestinal Symptom Scales; QL-Eso-CA, quality of life in patients with resected esophageal cancer; ROMP-swallowing, Radboud Oral Motor Inventory for Parkinson's Disease; SOAL, swallowing outcome after laryngectomy; SDI, Straumann Dysphagia Index; SDQ, swallowing disturbance questionnaire; SDQ-D, swallowing disturbance questionnaire in dysphagia; SLS, Scleroderma Logopedic Scale; SSQ, symptom inventory for oropharyngeal dysphagia; SSQ-hn; Sydney Swallowing Questionnaire for head and neck cancer; SWAL-CARE, swallowing quality of care; SWAL-QOL, swallow quality of life questionnaire; UW-QOL-s, UW quality of life swallowing domain.

Increasingly more instruments have been introduced over time, with a peak incidence between 2006 and 2010 (Figure 2). In order of frequency, countries of PRO measure development were United States (14), United Kingdom (3), Netherlands (3), Canada (2), Italy (2), Israel (2), Australia (1), Denmark (1), Sweden (1), and Switzerland (1) or involved multinational collaboration (4) (Table 4). Most were devised at tertiary care medical centers. Sample size of patients used to construct these measures varied from 49¹⁴ to 995.¹⁵ Mean or median participant age ranged from 22 years in a PRO measure focused on eosinophilic esophagitis¹⁶ to 79.9 years in a screening tool for geriatric dysphagia.¹⁷ In all, 91% (31/34) instruments described sample gender distribution, of which most (74%, 23/31) had a male preponderance (range: 26%–78.5%; Table 4). While most measures were designed based on classical test theory principles, two PRO measures employed item response theory techniques.^15,18

Fig. 2.

New PRO measures published over time.

Table 4.

Characteristics of participants and setting in the development of currently published patient-reported outcome measures related to dysphagia.

Article	Instrument	Study population	Setting	N	Distribution of pathology	Mean age (SD), range	Male (%)	Country
Andersen et al. (1987)12	CS-BED	Four groups of consecutive patients with (1) typical stable, exercised-induced angina pectoris, (2) benign esophageal disease, (3) operated on for gallstones or receiving medical treatment for gastroduodenal ulcer, (4) normal subjects	Departments of Thoracic Surgery, Gastroenterology, Statistics, and Cardiology at Rigshospitalet, Copenhagen	N = 135	N = 34 typical, stable exercise-induced angina N = 30 with benign esophageal disease N = 30 operated on for gallstones or medical treatment for gastroduodenal ulcers N = 41 normal subjects admitted for minor surgery with no former admissions because of esophageal diseases	Group 1: 54 (NR), 39–73 Group 2: 55 (NR), 28–78 Group 3: 59 (NR), 30–83 Group 4: 46 (NR), 19–80	80% 60% 44% 55% Overall = 60%	Denmark
Dakkak and Bennett (1992)14	NDS	Patients with endoscopically diagnosed benign esophageal strictures of varying severity	Hull Royal Infirmary, Gastrointestinal Unit, Kingston Upon Hull	N = 49	N = 41 reflux N = 5 reflux (Barrett) N = 2 Postsurgical N = 1 scleroderma	NA	NA	UK
van Knippenberg et al. (1992)19	QL-Eso-Ca	Patients with operable esophageal cancer	Department of Surgery, Erasmus University Hospital, Rotterdam	N = 62	Esophageal cancer (N = 52 gastric pull-up; N = 5 colon interposition, N = 5 combination)	58.1 (9.9)	67.7%	Netherlands
Blazeby et al. (1996)22	EORTC-QLQ-OES24	Patients with esophageal cancer	University Department of Surgery, Bristol Royal Infirmary, Bristol; Department of Thoracic Surgery, Heartlands Hospital, Birmingham; ENT Department Sahlgrenska Hospital, University of Gothenburg, Gothenburg; Departmento de Oncologia, Hospital de Navarra, Pamplona	Item generation = 12 [Patients (10), esophageal surgeon (1), medical epidemiologist (1)] Interviews = 22 Pre-testing = 132	N = 154 Esophageal cancer	Item generation: NA Interviews: NA Pretesting: 67(NR), 51–88	NA NA 70%	UK Sweden Spain
McHorney et al. (2000)28 (Phase I)	SWAL-QOL	Patients with mechanical or neurologic oropharyngeal dysphagia with a) VFSS documented oropharyngeal dysphagia as judged by experienced SLP and b) stability in the dysphagia as judged by SLP and clinician	Outpatient speech, neurology, and other medical clinics and from ENT and other clinics of VA hospitals, their affiliated university medical centers and free-standing medical centers from Madison, WI and Chicago, IL	Focus groups = 67 (52 patients, 15 caregivers)	Mechanic or neurologic oropharyngeal dysphagia	65.8 (15.4)	68.2%	US
McHorney et al. (2000)28 (Phase II)	SWAL-QOL	Patients with mechanical or neurologic oropharyngeal dysphagia with a) VFSS documented oropharyngeal dysphagia as judged by experienced SLP and b) stability in the dysphagia as judged by SLP and clinician	Patients recruited from Madison and Tomah, WI and Chicago, IL	N = 106	Chronic medical condition (5), degenerative neurologic disorder (10), dementia (1), head and neck cancer (41), neurological disorder (1), obstructive respiratory disorder (5), other (12), trauma (2), unknown (11), vascular disorder (18)	67.9 (10.7)	76.5%	US
McHorney et al. (2002)30 (Phase III)	SWAL-QOL	Controls – volunteers from general community, a hospital volunteer department, and an elder-hostel education program without oropharyngeal disorders	Outpatients from VA, and affiliate university hospitals or free-standing hospitals in Madison WI, Chicago IL, Little Rock AR, Houston TX, Atlanta GA, and Long Beach CA	Validation = 386 Controls = 40	Cancer (109), vascular disease (61), degenerative neurological disease (49), other neurologic disease (36), obstructive respiratory disease (23), trauma (15), chronic medical condition (17), dementia (4), other reason (26), unknown (46)	Cases: 66.1 (13.2) Controls: 72.9 (NR), 45–95	Cases: 78.5% Controls: NR	US
Wallace et al. (2000)29	SSQ	Two groups of patients with oropharyngeal dysphagia and a group of comparably aged controls without dysphagia. Group 1: confirmed stable neuromyogenic oropharyngeal dysphagia; Group 2: dysphagia due to Zenker's diverticulum referred for surgery	Department of Gastroenterology, St. George Hospital, Sydney	Group 1 = 48 Group 2 = 11 Control = 19	Group 1: CVA brainstem (12), CVA other (2), Parkinson's disease (12), movement disorder other (4), myopathy (3), myasthenia gravis (1), ALS (6), brainstem lesion other (2), idiopathic (3) Group 2: Zenker's diverticulum	Group 1: 70 (13), 30–96 Group 2: 66 (13), 36–84 Controls: 62(16), 31–94	Group1: 60% Group 2: 54.5% Controls: 42%	Australia
Chen et al. (2001)24	MDADI	Patients with neoplasm of upper aerodigestive tract undergoing evaluation by SLP	MD Anderson Cancer Center, Houston, TX	Focus group = 6 (Faculty, fellow, SLPs) Administration = 100	Squamous cell carcinoma (82), benign neoplasm (15), nonsquamous malignancy (3)	58 (10), 21–80	76%	US
Weymuller et al. (2001)25	UW-QOL-R	All new patients with diagnosis of head and neck cancer	Department of Otolaryngology, University of Washington, Seattle, WA	N = 500 with head and neck cancers	Initial questionnaire developed in 13 patients with stage III or IV oropharyngeal cancer (1997), 6 in surgical group and 7 in nonsurgical group	Surgical group: 40–64 Nonsurgical group: 40–65	66% 71%	US
Bazaz et al. (2002)53	Bazaz	Patients who underwent anterior cervical spine surgery	Spine Institute and Department of Orthopedic Surgery, University Hospitals of Cleveland, Case Western Reserve University, Cleveland	N = 224	ACDF (123), ACCF (97), anterior cervical plate revisions (2), combined ACDF-ACCF (1), fibular strut graft revision (1)	51.8 (12.2)	51.8%	US
McHorney et al. (2002)30	SWAL-CARE	Patients with mechanical or neurologic oropharyngeal dysphagia with a) VFSS documented oropharyngeal dysphagia as judged by experienced SLP and b) stability in the dysphagia as judged by SLP and clinician	Outpatients from VA, and affiliate university hospitals or free-standing hospitals in Madison, WI, Chicago, IL, Little Rock, AR, Houston TX, Atlanta GA, and Long Beach CA	Validation = 386 Controls = 40	Cancer (109), vascular disease (61), degenerative neurological disease (49), other neurologic disease (36), obstructive respiratory disease (23), trauma (15), chronic medical condition (17), dementia (4), other reason (26), unknown (46)	Cases: 66.1 (13.2) Controls: 72.9, (NR) 45–95	Cases: 78.5% Controls: NR	US
Blazeby et al. (2003)23	EORTC-QLQ-OES18	Two groups of patients with newly diagnosed esophageal squamous cell or adenocarcinoma. Group A consisted of patients undergoing potentially curable treatment and Group B included patients receiving treatment with palliative intent	Multiple hospitals internationally in Germany, UK, France, Switzerland, Australia, Spain, and Sweden coordinated by the EORTC	N = 591	Group A: Esophageal cancer with potentially curable treatment = 267 (esophagectomy = 95, chemotherapy + radiation = 172) Group B: Esophageal cancer being treated with palliative intent = 224 (endoscopic palliation = 96; chemotherapy + radiation = 128)	Age (years) A  B <50    12% 6% 50–59   26% 21% 60–69    37% 31% 70–79  23%  29% >80   1%  13%	Group A: 81% Group B: 75%	UK France Germany Sweden Australia Spain Other
Urbach et al. (2005)18	MADS	Patients with achalasia recently seen in outpatient specialty clinic	Mail-out questionnaires to patients seen at specialty clinic at University of Toronto, Toronto General Hospital, Toronto, Ontario	Item generation = NR Administration = 70	Achalasia = 70	50.34 (17.2), 18–87	50%	Canada
Boczko (2006)17	DSST	Geriatric persons newly admitted to a long-term or subacute rehabilitation facility	Long-term and subacute rehabilitation facility (Department of Speech-Language Pathology and Swallowing Center, Jewish Home and Hospital Lifecare System, NY)	N = 199	New admissions to a large long-term care/subacute rehab facility with patient diagnoses including systemic disorders (32.7%), orthopedic/muscular (27.6%), cardiovascular (25.1%), cognitive/emotional (21.6%), gastrointestinal (16.1%), pulmonary (16.5%), cancer (6.5%), central nervous system (6.0%), and other (12.1%)	79.9, (NR) 50–98	37.2%	US
Peloquin et al. (2006)40	MDQ-2W	Patient referred to endoscopy for dysphagia or seen in outpatient tertiary care clinics	Outpatient gastroenterology and ENT Clinics, Mayo Clinic, Rochester, MN	Feasibility = 20 Reproducibility = 139 Concurrent validity = 182	Patients with dysphagia symptoms	59.6 (NR)	45%	US
Darling et al. (2006)20	FACT-E	Patients with esophageal cancer; surgical patients (cohort A) or patients receiving induction chemoradiotherapy before surgery (cohort B)	Division of Thoracic Surgery, Toronto General Hospital, University of Toronto and Division of Thoracic Surgery, Dalhousie University, Halifax	Qualitative = 18 Field test = 38 Validation: Cohort A = 54; Cohort B = 29	NR NR Cohort A = Resectable esophageal or GEJ cancer Cohort B = Resectable + fit for chemoradiation	NR NR Cohort A: 62 (11) Cohort B: 58 (11)	NR NR Cohort A: NR Cohort B: 75.8%	Canada
Grudell et al. (2007)37	MDQ	Unselected patients with a variety of esophageal disorders recruited from outpatient clinics	Two outpatient clinics specialized in esophageal disorders (Esophagus Clinic or the Upper Endoscopy Suite; Mayo Clinic, Rochester, MN)	Item refinement = 13 (5 GIs, 8 outpatients) Pilot = 8 Validation = 70 Reproducibility = 70	NR NR Dysphagia = 36 Dysphagia = 36	NR 67.3 (14.7) 60.6 (13.2) 57.7 (14.6)	NR 62% 54% 50%	US
Manor et al. (2007)35	SDQ	Patients with Parkinson's disease referred to SLPs in Movement Disorder Unit	Tertiary care outpatient clinic (Movement Disorder Unit- Parkinson Center, Voice and Swallowing Disorders Clinic- Department of Otolaryngology HANS, Tel-Aviv Sourasky Medical Center, Tel-Aviv)	N = 57	Parkinson's disease = 57	69 (10)	73.4%	Israel
Cohen et al. (2011)45	SDQ-D	Patients who were referred for swallowing evaluation	Voice and Swallowing Unit in the Tel-Aviv Medical Center	N = 100	Neurologic disorder (40), head and neck tumor +/− XRT (30), GI disease (8), and no diagnosis (22)	61 (3.5)	54%	Israel
Lagergren et al. (2007)21	EORTC-QLQ-OG25	Patients with esophageal or stomach cancer including those of the gastroesophageal junction	Seven academic sites in four different European countries	N = 300	Esophageal cancer (148), Gastroesophageal junction (66), Stomach (85)	Median 63 (27–85)	69%	France Germany UK Sweden
Schaaf et al. (2012)54	EORTC-QLQ-OG25	Randomly selected sample of the Swedish population between 40 and 79 years of age	Upper Gastrointestinal Research, Department of Molecular Medicine and Surgery, Karolinska Institute, Stockholm	N = 4910	Cancers = 413 (prostate = 138, breast = 72, urogenital = 26, leukemia and lymphoma = 19, lung = 4, upper gastrointestinal = 13, lower gastrointestinal = 45, skin = 39, brain = 5, other = 15, metastasis = 5)	65 (NR), 40–79	69.9%	Netherlands
Belafsky et al. (2008)41	EAT-10	Broad range of patients with dysphagia including those with oropharyngeal and esophageal phase dysphagia	Outpatient Voice and Swallowing Clinics (Dept. of Otolaryngology, UC-Davis and Medical College of Georgia)	Item generation = NR Item refinement: Cases = 235 Controls = 100 Validity analysis = 46	NR Reflux disease (66), voice disorder (50), head and neck cancer history (42), esophageal phase dysphagia (26), oropharyngeal dysphagia (50) Zenker's diverticulum (23), reflux disease (14), esophageal stricture (9)	NR Cases: 62 (14) Controls: 48 (16)  65 (16)	NR Cases: 54% Controls: 53%  45.7%	US
Bergamaschi et al. (2008)38	DYMUS	Patients with multiple sclerosis being seen for maintenance visits	Outpatients seen at four different centers: Neurological Institute Mondino of Pavia, Neurological Department and the Neurology Unit of Catania, Neurological Department of Genoa ASL, S. Raffaele Institute of Milan	Item generation = NR Administration = 226	Multiple sclerosis: relapsing remitting = 164, secondary progressive = 49, primary progressive = 13; swallowing problems = 59	40.5 (NR), 21–71	26%	Italy
Straumann et al. (2010)31	SDI	Patients older than 14 years with histologically confirmed eosinophilic esophagitis	Department of Gastroenterology, University Hospital Basel, Basel; Department of Gastroenterology, Inselspital, University Hospital Bern, Bern	N = 36	Eosinophilic esophagitis = 36 Cases = 18 Controls = 18	33.1 (13.1) 38.2 (12.4)	94% 78%	Switzerland
McElhiney et al. (2010)42	MDQ-30	Unselected patients referred to outpatient gastroenterology clinics for a variety of complaints	Outpatients referred to Esophagus Clinic or General Gastroenterology Clinic, Mayo Clinic, MN for upper endoscopy or colonoscopy	N = 414 Feasibility = 20 Reproducibility = 216 Concurrent validity = 178	17 patients had dysphagia 22% with dysphagia, 104 not using PPI, 112 using PPI Not using PPI (102) and using PPI (76)	61.5 (NR), 19–101 53.7 (15) 61.5 (14) 58.1 (15.5)	46% 30% 53% 43%	US
Speyer et al. (2011)26	DSS	Patients diagnosed with oropharyngeal dysphagia due to oncological disorders	Outpatient clinic for dysphagia, Department of Otorhinolaryngology-Head and Neck Surgery and MAASTRO clinic in Academic Hospital, Maastricht University Medical Center	N = 76	Oropharyngeal dysphagia due to oncological disorders	Range: 45–83 Men: 64 Women: 61	75%	Netherlands
Dwivedi et al. (2010)27	SSQ-hn	Patients who received curative treatment with surgery with or without radiation and/or chemotherapy for oral and oropharyngeal cancer	Outpatients at Royal Marsden Hospital, London, UK	N = 54/62 responded	Oral cavity (16), oropharynx (38)	58.6 (9.7), 35.9–80	64.8%	UK
Vitali et al. (2010)39	SLS	Patients with systemic sclerosis disease	Ospedale Maggiore, Immunology Clinical Unit	Focus group = NR Pilot = 28 Phase 2 = 16 Phase 3 = 15 Administration: Cases = 86, Controls = 40	Systemic sclerosis (84) and healthy subjects (40)	NR NR NR Cases 57 (12.7) Controls 56.1 (7.5)	NR NR NR Cases 19%, Controls 20%	Italy
Kwiatek et al. (2011)44	ESQ	Adult patients with physician appointments to discuss esophageal and throat symptoms	Northwestern Memorial Faculty Foundation Gastroenterology and Otolaryngology outpatient clinics, Chicago, IL	N = 211 Validation = 134	Globus (33), dysphagia (52), globus and dysphagia (4), globus and reflux (9), globus and other (3), dysphagia and reflux (20), reflux and other (4), globus and dysphagia and other (24)	NR (NR), 18–83	39.8%	US
Kalf et al. (2011)34	ROMP-swallowing	Adult outpatients with Parkinson's disease, including patients with a form of (probable) atypical Parkinsonism	Departments of Rehabilitation and Epidemiology, Biostatistics, and HTA, and Scientific Institute for Quality of Healthcare, Nijmegen Centre for Evidence Based Practice, Radboud University, Nijmegen, and Department of Neurology, Donders institute for Brain, Cognition and Behavior, Nijmegen	N = 178	Idiopathic Parkinson's disease = 130, atypical Parkinsonism = 48	64 (9.8)	63%	Netherlands
Silbergleit et al. (2012)43	DHI	Patients presenting with dysphagia	Outpatients to Henry Ford Hospitals in Southeast Michigan	Prelim version = 77 subjects Final version = 214 subjects and 74 controls Test–retest = 63	Prelim PRO: Neurological disorders (40), head and neck disorders (10), esophageal abnormalities (6), other (18) Final PRO: head and neck disorders (76), neurological impairment (72), GERD (23), esophageal abnormalities (6), respiratory disorders (9), other (28) Test–retest: head and neck disorder (22), neurological disorder (26), GERD (7), esophageal abnormality (1), other (7)	Female 60.3 (NR), 25–89 Male 62.6 (NR), 24-94 Cases: Female 60.3 (16.5), 20–92; Male: 65.5 (12.8), 19–96 Controls: Female 55.8 (12.9), 30–86; Male 53.5 (13.7), 30–80 Final: Female 60.3 (NR), Male 65.5 (NR)	Alpha 57.1% Beta 48.6% Final 36.5%	US
Skeppholm et al. (2012)33	DSQ	Patients with stable dysphagia recruited from dysphagia unit (1st validation study) and outpatients undergoing anterior cervical spine surgery for degenerative disk disease (2nd validation study)	First: Dysphagia Unit, Department of Speech-Language Pathology, Karolinska University Hospital, Solna Second: Stockholm Spine Center, Löwenströmska Hospital, Upplands Väsby, Stockholm, Sweden and Department of Orthopedics, Uppsala University Hospital, Uppsala, Sweden	First validation = 45 Second validation = 111	Malignancy of neck region (21), neurological disease (14), miscellaneous (4), xerostomia (2), unknown (4) Patients with degenerative disk disease (111)	64.8 (10.4) 46.9 (6.7)	NR 50%	Sweden
Govender et al. (2012)36	SOAL	Noncomplaining volunteers without any history of dysphagia or associated illness; patients after total laryngectomy; and patients at least 3 months post head and neck chemo-radiation and with known dysphagia	Speech and Language Therapy, University College London Hospital National Health Service Foundation Trust	Phase I: Question development = 16 (SLP = 6; laryngectomees = 10) Phase II: Volunteers = 20 Laryngectomees = 19 Chemoradiation = 19	Phase I: laryngectomies = 10 Phase II: laryngectomies = 19; head and neck cancer treated with chemoradiation = 19	63 (NR) Vol. 63 (NR), 44–83; Laryng: 66 (NR), 48–80; Radiation; 61 (NR) 41–92)	80% 50% 52.6% 57.9%	UK
Govender et al. (2016)46	SOAL	Patients over the age of 18 who had undergone laryngectomy and were a minimum of 3 months past their last oncologic treatment, and who had no known head and neck recurrent disease	Head and Neck Cancer Centre, University College London Hospital, National Health Service Foundation Trust	Main sample = 110 Test–retest = 15	Laryngectomy only = 16, laryngectomy and radiotherapy = 65, laryngectomy and chemoradiotherapy = 25, missing data = 17	Main sample 66 (9.1) Test–retest 65.2 (4.04)	94% 73.3%	UK
Dellon et al. (2013)16	DSQ-EoE	Adolescents and adult patients with eosinophilic esophagitis	Pediatric patients were enrolled from a large community-based practice and an academic practice, and adults were enrolled from two academic centers	Phase I = 20 (adults = 10, adolescents = 10) Phase II = 35/37 (adults = 18, adolescents = 17)	Confirmed eosinophilic esophagitis	22 (7.2), 12–43 23.6 (10.7), 12–45	50% 54%	US
Schoepfer et al. (2014)13	EEsAI	Adult patients with eosinophilic esophagitis in need of an EGD for initial diagnosis, for confirming a suspected diagnosis, or for monitoring previously diagnosed eosinophilic esophagitis	One ambulatory care clinic and seven hospitals in Switzerland and the United States	Item generation: patient surveys = 135, focus groups = 27, semistructured interviews = 24 Evaluation = 153 Validation = 120	Confirmed eosinophilic esophagitis	NR Median 38 (IQR 29–46), 17–71 Median 40.5 (IQR 31–49), 19–80	NR 72.5% 60.8%	Switzerland US
Spiegel et al. (2014)15	PROMIS-GI	Patients seeking care for an active GI symptom, including abdominal pain, gas/bloating, nausea, vomiting, diarrhea, incontinence, constipation, dysphagia, or acid reflux	University, community, and Veterans Administration institutions around the US	Qualitative = 130 Psychometric = 865	NR	NR	NR	US

Constructs measured

Both condition-specific and general dysphagia-related PRO measures were identified (Table 3). Condition-specific measures were heterogeneous and included esophageal cancer,^19–23 oropharyngeal dysphagia related to cancer,^24–27 oropharyngeal dysphagia,^28–30 eosinophilic esophagitis,^13,16,31 dysphagia related to anterior spine surgery,^32,33 Parkinson's disease,^34,35 benign esophageal disease,¹² total laryngectomy-related dysphagia,³⁶ esophageal stricture,¹⁴ achalasia,¹⁸ esophageal disorders,³⁷ multiple sclerosis,³⁸ and systemic sclerosis.³⁹ Seven were designed to assess patients with general dysphagia,^15,40–45 and one was a screening tool for dysphagia among geriatric patients.²³

Five swallowing questionnaires are subscales within broader PRO measures.^{15,20,25,34,39} Three validated a pre-existing PRO measure in a new target population.^16,27,45 Two validated a pre-existing PRO measure using a different recall period,^40,42 and two further refined previously developed health-related quality of life measures related to esophageal cancer.^21,23

Developmental characteristics

Developmental characteristics of individual PRO measures varied considerably. In all, 47% of instruments (16/34) met at least one criterion in each of the six domains. While no measure met all criteria, there were several methodologically robust measures that demonstrated adequacy in 16 (Patient-Reported Outcomes Measurement Information System Gastrointestinal Symptom Scales (PROMIS-GI), swallow quality of life questionnaire (SWAL-QOL), Sydney Swallow Questionnaire (SSQ), Swallowing Outcomes After Laryngectomy questionnaire (SOAL)),^15,28,29,36 15 (Measure of Achalasia Disease Severity (MADS), European Organization for Research and Treatment of Cancer Quality-of-Life with esophageal Cancer 25 items (EORTC-QLQ-OG25), European Organization for Research and Treatment of Cancer Quality-of-Life with esophageal cancer 18 items (EORTC QLQ-OES18), swallowing quality of care (SWAL-CARE)),^18,21,23,30 and 14 (M.D. Anderson dysphagia inventory (MDADI), Radboud Oral Motor Inventory for Parkinson's Disease (ROMP)-Swallowing)^24,34 of 18 assessed developmental measurement properties. Figure 3 provides an itemized, schematic overview of the developmental measurement properties and functionality for the 34 identified dysphagia-related PRO measures. Findings are summarized below.

Fig. 3.

Summary comparison of measurement properties among identified patient-reported outcome measures.

Conceptual model

Each PRO measure provided some description of the construct it was designed to measure and its intended respondent population. Most (32/34) prespecified whether the measures were expected to have a single scale or multiple subscales (Figure 3).

Content validity

Less than half (44%, 15/34) of dysphagia-related PRO measures directly involved target populations in developing item content (e.g. focus groups or interviews) (Figure 3). In contrast, content experts were involved in questionnaire content development for all instruments. One-third (11/34) of outcome measures clearly described the methodology by which their content (e.g. questions, responses) was derived.

Reliability

Overall, reliability was not evaluated in the development of 24% (8/34) PRO measures (Figure 3). All measures that tested reliability were found to have adequate indices (r ≥ 0.70) or justified lower values.

Types of reliability tested differed. Some instruments demonstrated test–retest reliability (SSQ, Dysphagia Short Questionnaire (DSQ), Mayo Dysphagia Questionnaire (MDQ), Mayo Dysphagia Questionnaire–2 week (MDQ-2w), Eating Assessment Tool-10 (EAT-10), Mayo Dysphagia Questionnaire-30 Day (MDQ-30)),^{29,33,37,40–42} while others showed internal consistency (dysphagia-specific screening tool (DSST), MADS, quality of life in patients with resected esophageal cancer (QL-Eso-Ca), EORTC QLQ-OG25, EORTC QLQ-OES18, swallowing disturbance questionnaire (SDQ), dysphagia in multiple sclerosis (DYMUS), swallowing disturbance questionnaire in dysphagia (SDQ-D)).^{17–19,21,23,35,38,45} Several measures demonstrated both internal consistency and test–retest reliability (PROMIS-GI, functional assessment of cancer therapy esophageal cancer subscale (FACT-E), MDADI, Dysphagia Severity Scale (DSS), SSQ, SWAL-QOL, SWAL-CARE, ROMP-Swallowing, SOAL, Scleroderma Logopedic Scale (SLS), DHI, esophageal symptoms questionnaire (ESQ)).^{15,20,24,26–28,30,34,36,39,43,44}

Responsiveness

Responsiveness to change is a measurement property espoused by most instruments. However, only 15% (5/34) of PRO measures met this criterion by either demonstrating responsiveness (FACT-E, SWAL-QOL, SSQ, EAT-10)^20,28,29,41 or were not applicable by nature of being intended for cross-sectional screening (DSST).¹⁷

Construct validity

Four forms of construct validity were evaluated: dimensionality, responsiveness to change (longitudinal validity), convergent validity, and known-group validity (Table 4). An empirical justification of dimensionality (single or multiple subscales within a measure) was provided for 35% (12/34) of dysphagia-related PRO measures (Figure 3). Evidence of convergent and known-group validity was shown in 79% (27/34) and 59% (20/34) of measures, respectively.

Interpretation and scoring

There was significant variability in scoring approach. While usability presumes understanding an instrument's scoring system, 18% of studies (6/34) lacked documentation on how to score the PRO measures (clinical symptoms in benign esophageal disease (CS-BED), EORTC QLQ-OES24, Straumann Dysphagia Index (SDI), MDQ, MDQ-2w, MDQ-30).^{12,22,31,37,40,42} Furthermore, only 12% (4/34) of PRO measures provided a plan for how to score incomplete surveys (QL-Eso-Ca, SSQ, MADS, SOAL). For example, both the SOAL and Measure of Achalasia Disease Severity (MADS) scores are deemed invalid if there are missing item responses.^18,36,46 In the Sydney Symptom Inventory for Oropharyngeal Dysphagia Questionnaire (SSQ), surveys are discarded if 15 of 19 questions are not completed.²⁹ In contrast, the Quality of Life and Esophageal Cancer questionnaire (QL-Eso-Ca) provides a more detailed algorithm for missing responses.¹⁹

The majority of developmental studies (82%, 28/34) provided some information on how to interpret PRO measure scores using scaling or severity cut-offs (Figure 3). Most instruments used simple summation for scoring, with higher scores representing greater degrees of dysphagia. In contrast, PROMIS-GI calibrated each scale using two-parameter item response theory graded response modeling and scored on a T metric with a mean of 50 and standard deviation of 10 in the US general population.¹⁵ This allows scales to be converted into a percentile score, where each respondent is compared against the US general population on an easily interpretable scale. Several PRO measures did not meet any criteria in the scoring and interpretation domain, which makes them impractical for application by other researchers or clinicians (CS-BED, EORTC QLQ-OES24, MDQ, MDQ-2w, SDI, MDQ-30).^{12,22,31,37,40,42}

Burden and presentation

Most PRO measures were considered to have reasonable degree of burden to the patient based on cutoff of 30 questions determined by three reviewers (DP, RS, and DF). PRO measures having greater than 30 questions were considered to impose a higher burden on patient. Four exceptions were the 44-item Swallowing Quality of Life Questionnaire (SWAL-QOL), the 39-item QL-Eso-Ca, the 44-item Functional Assessment of Cancer Therapy Esophageal Cancer Subscale (FACT-E), and the 38-item ESQ.^19,20,28,44 Few dysphagia-related PRO measures provided an estimation of the literacy level needed for their comprehension and completion (PROMIS-GI, DSQ-EoE, SWAL-QOL, MDQ-30, DHI).^{15,16,28,42,43} Access to all items and answer choices is essential to an outcome measure's usability. However, of developmental studies only 65% (22/34) provided a readily accessible method of viewing the complete instrument.

DISCUSSION

PRO measures are the cornerstone to systematically collecting patient-centered data and monitoring treatment outcomes. Although there are many avenues available for testing the physical act of swallowing (e.g. videofluoroscopic swallow study, manometry, and upper endoscopy), PRO measures are critical to understanding the psychosocial aspects of dysphagia and are the only modality that provides patient perception on how the disease or treatment is affecting their quality of life.⁴⁷ These standardized measures are therefore valuable tools for demonstrating treatment effectiveness and improving medical care for patients with dysphagia.

Given their importance in both clinical and research settings, there has been a significant increase in number of dysphagia-related PRO measures since the first measure was introduced in 1987.¹² Some PRO measures are disease specific such as those focused on achalasia, post-operative dysphagia from spine surgery, Parkinson's disease, multiple sclerosis, eosinophilic esophagitis, or malignancy, while others are designed to assess a broader, diverse population. The appropriateness of using a particular PRO measures is predicated on its intent and developmental measurement properties. To our knowledge, this is the first systematic review evaluating the measurement properties of the entire breadth of dysphagia-related PRO measures in the adult literature. It rigorously evaluated each instrument for the presence of fundamental developmental characteristics.

Overall, the 34 dysphagia-related PRO measures identified demonstrated significant variability in their developmental rigor. No outcome measure satisfied all criteria. Yet, many are rigorously developed and of high quality, as evidenced by possessing nearly all measurement properties in the six assessed domains. These included condition-specific measures for esophageal cancer (FACT-E, EORTC QLQ-OG25, EORTC QLQ-OES18),^20,21,23 upper aerodigestive neoplasm-attributable oropharyngeal dysphagia (MDADI),²⁴ mechanical and neuromyogenic oropharyngeal dysphagia (SWAL-QOL, SSQ, SWAL-CARE),^28–30 achalasia (MADS),¹⁸ and eosinophilic esophagitis (DSQ-EoE).¹⁶ In addition, there was one general dysphagia PRO measure with exceptional developmental characteristics that was devised as part of the NIH PROMIS program (PROMIS-GI).¹⁵ Before applying even these high-quality measures in clinical or research settings, it is imperative that end users carefully consider their intent and developmental characteristics. A well-designed measure used for unintended applications or in inappropriate target populations can lead to distorted results.

There were several developmental strengths in currently available dysphagia-related PRO measures. Each measure defined the construct (i.e. outcome) it planned to measure as well as the intended target population. Ideally, this target population should be enrolled in sufficient numbers to provide adequate power for its development and/or validation. Dysphagia instruments varied widely in enrollment from 49¹⁴ to 995¹⁵ target participants. It is generally recommended that variable and subject sampling are optimized for factor latent variable analysis-based methods and/or that there be greater than 100 participants involved in validation.⁴⁸ Overall, 11 studies achieved this standard (Table 4). Adequacy, applicability, and generalizability of measures that include few individuals from the target population in development should be questioned.

Another strength is that most PRO measures adequately assessed some aspect of reliability and construct validity. Of the four forms of construct validity considered, the majority of instruments tested for convergent and/or known-group validity. However, surprisingly few adequately evaluated either dimensionality (35%) or responsiveness to change (12%), which may have several important implications. Dimensionality provides evidence for the presence or absence of subscales within the overall measure and is particularly relevant when attempting to justify the existence of subscales and to ensure they represent discrete concepts. This can have important implications for scoring algorithms. For instance, the MDADI questionnaire included four subscales (global, emotional, functional, and physical aspects of quality of life in patients with head and neck cancer) but did not perform statistical justification (e.g. factor analysis) to prove the subscales were not measuring overlapping aspects of the same construct.²⁴ End users may try to report individual subscale scores when, in fact, each may not empirically represent a unique concept.

A critical omission in these instruments is lack of demonstrable responsiveness to change. Responsiveness is defined as the ability of a measure to detect clinically important changes over time.⁴⁹ Unless intended for cross-sectional or screening purposes, one of the most common applications of PRO measures is monitoring change in the construct (i.e. dysphagia) over time to assess comparative effectiveness of interventions or the natural history of the condition. To adequately show responsiveness to change, an instrument must have stability when no change is expected (test–retest reliability) and exhibit change in score when it is expected (responsiveness). Of all the identified PRO measures, only 12% established this parameter. Based on initial developmental studies, dysphagia-related PRO measures without demonstrated responsiveness may not be appropriate for and give spurious results in clinical trials and other comparative effectiveness studies.

Another global concern regarding available dysphagia-related PRO measures is an insufficient degree of patient-centeredness. The content of 56% instruments was derived from content experts without direct input from patients. The foundation of PRO measures is the target population perspective and experience. Thus, omitting patients at this stage compromises the content validity and fidelity of scores. It creates a condition in which patients answer questions designed by and based on the experience and opinions of content experts who do not live with their particular condition.

Most dysphagia-related PRO measures provide some instructions on how to score the survey with majority using a simple summation scoring system. The majority also provided some information on how to interpret the PRO measures scores in the way of scaling/anchors or cut-offs for severity. This allows clinicians and researchers to assign qualitative meaning to quantitative scores. A key limitation to PRO measures is score interpretability. It is often difficult to determine what a change in score translates to or whether it is clinically meaningful. Several strategies exist to determine what a clinically important change or patient important change represents for a given outcome measure.⁵⁰ A commonplace practice is to evaluate the statistical significance of a change in score. This approach introduces potential to equate statistically and clinically significant change, which can lead to flawed conclusions. Omission of information on score interpretation altogether represents a weakness in many dysphagia-related PRO measures and limits their usefulness in clinical and research applications.

Incomplete questionnaires are common occurrences in both clinical practice and research. Although many techniques exist to address incomplete data, only 12% (4/34) of PRO measures provided a plan for scoring and managing incomplete surveys (QL-Eso-Ca, SSQ, MADS, SOAL). This is important as missing data, especially in clinical trials, can introduce significant bias. Readability of PRO measures is also a critical as respondents who do not understand the questions and answer options are less likely to complete a given questionnaire. The average reading comprehension level of English-speaking adults in the United States is estimated at the seventh- or eighth-grade level.⁵¹ A recent study examining readability of four commonly used dysphagia-related PRO measure questionnaires (EAT-10, MDADI, SWAL-CARE, SWAL-QOL) found each had a reading grade level of ninth grade or higher.⁴⁷ Only 15% (5/34) of dysphagia-related PRO measures in this systematic review described literacy level (PROMIS-GI, DSQ-EoE, SWAL-QOL, MDQ-30, DHI). Developers should consider readability as an important testable construct since poor readability may affect validity, reliability, and sensitivity.⁴⁷

There are limitations to this review process. Despite the careful design, the search may not have captured all available literature as these types of studies are often poorly indexed. Hand-searches were used to mitigate this limitation. We also limited our search to English publications. Applicable PRO measures may have been published in other languages that this review did not capture. There is also the risk of subjectivity in scoring PRO measure characteristics. Every effort was made to minimize this risk by using three independent reviewers for each instrument considered. We also ensured strict adherence to PRISMA guidelines in reporting this systematic review.⁵²

CLINICAL IMPLICATIONS

We critically evaluated developmental properties of 34 dysphagia-related PRO measures and identified several high quality PRO measures that were rigorously developed in various disease states. Condition-specific measures that possessed measurement properties in all six measured domains were as follows: esophageal cancer (FACT-E, EORTC QLQ-OG25, EORTC QLQ-OES18),^20,21,23 upper aerodigestive neoplasm-attributable oropharyngeal dysphagia (MDADI),²⁴ mechanical and neuromyogenic oropharyngeal dysphagia (SWAL-QOL, SSQ, SWAL-CARE),^28–30 achalasia (MADS),¹⁸ eosinophilic esophagitis (DSQ-EoE),¹⁶ and general dysphagia (PROMIS-GI).¹⁵ This systematic review will help clinicians and researchers in choosing the most developmentally sound PRO measure for research or clinical applications.

CONCLUSIONS

PRO measures are essential for systematically collecting patient-centered data and monitoring treatment outcomes in patients with dysphagia. A surge in the number of these measures has generated difficulty for clinicians and researchers in choosing the best PRO measure for their particular intent. This is the first systematic review of current dysphagia-related PRO measures. There are many well-designed, high-quality dysphagia-related PRO measures. However, several thematic deficiencies exist in the current literature, namely, lack of: (1) direct patient engagement in content development, (2) empirically justified dimensionality, (3) demonstrable responsiveness to change, (4) plan for interpreting missing responses, and (5) literacy level assessment. Care must be taken to understand the developmental characteristics of PRO measures before selecting and advocating their use in research or clinical applications.