Abstract
This White Paper by the European Society for Swallowing Disorders (ESSD) reports on the current state of screening and non-instrumental assessment for dysphagia in adults. An overview is provided on the measures that are available, and how to select screening tools and assessments. Emphasis is placed on different types of screening, patient-reported measures, assessment of anatomy and physiology of the swallowing act, and clinical swallowing evaluation. Many screening and non-instrumental assessments are available for evaluating dysphagia in adults; however, their use may not be warranted due to poor diagnostic performance or lacking robust psychometric properties. This white paper provides recommendations on how to select best evidence-based screening tools and non-instrumental assessments for use in clinical practice targeting different constructs, target populations and respondents, based on criteria for diagnostic performance, psychometric properties (reliability, validity, and responsiveness), and feasibility. In addition, gaps in research that need to be addressed in future studies are discussed. The following recommendations are made: (1) discontinue the use of non-validated dysphagia screening tools and assessments; (2) implement screening using tools that have optimal diagnostic performance in selected populations that are at risk of dysphagia, such as stroke patients, frail older persons, patients with progressive neurological diseases, persons with cerebral palsy, and patients with head and neck cancer; (3) implement measures that demonstrate robust psychometric properties; and (4) provide quality training in dysphagia screening and assessment to all clinicians involved in the care and management of persons with dysphagia.
Keywords: Measures, Diagnostic accuracy, Validity, Reliability, Responsiveness, Psychometrics
Introduction
The terms Dysphagia and swallowing disorders are used interchangeably in the literature; however, achieving international consensus on defining dysphagia is more challenging. Despite there being no universally accepted definition, a commonly used description of dysphagia is the dysfunction of one or more parts of the swallowing apparatus including the mouth, the tongue, the oral cavity, the pharynx, the airway, and the oesophagus and its upper and lower sphincters, often due to anatomical or structural deficiencies or abnormalities [1]. Swallowing is not only essential for nutritional intake, but it is also directly involved in the management of internal secretions from the upper and lower aerodigestive tracts (e.g. saliva, bile, and nasal, tracheal or bronchial secretions). Dysphagia is associated with a broad spectrum of underlying congenital or acquired conditions which is influenced by both age and comorbid diseases [2]. Dysphagia is one of the leading causes of death and morbidity for older persons, children and adults with neurological disorders (e.g. cerebral palsy, stroke, dementia), or head and neck cancer patients, due to serious complications such as malnutrition, dehydration, and aspiration pneumonia [3, 4].
Dysphagia is a symptom or collection of symptoms resulting from underlying impairments and disorders in the highly integrated neuromotor sequence of swallowing, from forming a cortical decision through to the realization of motor action with associated sensory modulation, defined by series of events that involve the digestive and upper respiratory tract and the interaction of various anatomical structures along the pathway [5]. Dysphagia may modify the preparation, propulsion, and/or transit of the bolus through the upper digestive pathways and reflects abnormalities in transit and/or airway protection. A distinction can be made between oropharyngeal and oesophageal dysphagia.
Not having reached consensus on a universal definition of the construct of dysphagia impacts on making an accurate estimation of the prevalence of dysphagia, as it influences what criteria need to be met to fall within the parameters of the definition. A systematic review on the prevalence of dysphagia in the general population indicated marked variability in prevalence rates, which ranged between 2.3 and 16% [6]. Another review demonstrated that the prevalence of dysphagia after stroke and in patients with Parkinson’s disease, traumatic brain injury, and community-acquired pneumonia was high (8.1–80%, 11–81%, 27–30%, and 91.7%, respectively) [7]. The high variability in the retrieved prevalence data can be explained by discrepancies regarding dysphagia definitions, underlying diseases being at different stages with variable sequelae and using different measures to determine the presence of dysphagia. The impacts of definitional issues on determining accurate prevalence rates are twofold. First, definitional inconsistencies will have an impact on aspects of validity, reliability, and generalizability of prevalence numbers. Second, screening or assessments of dysphagia operationalizes the underlying construct being measured, and definitional inconsistencies will invariably influence what and how dysphagia is being measured. Despite the implications of definitional challenges, given the high prevalence of dysphagia [6, 7], the serious complications associated with it, and costs to healthcare [8], systematic screening of at-risk populations such as acute stroke patients should be central to healthcare strategies to improve early intervention and clinical outcomes in these clinical populations. Moreover, the high prevalence of dysphagia underscores the need for standardized, reliable, and valid methods for the screening and assessment of dysphagia.
The purpose of screening is to identify persons at risk of dysphagia. Patients who are identified as being at risk through screening are typically referred for further assessment and, where needed, related assessment including oral health and gastroesophageal reflux. Both screening and assessment need to be evidence-based and feasible to administer. Only screening with sufficient diagnostic performance and assessments with robust psychometric properties should be used clinically and in research. For example, using screening tools with poor diagnostic performance may result in missing people at risk for dysphagia (false positives) or lead to unnecessary use of limited resources by assessing those who are not at risk of dysphagia (false negatives). Similarly, implementing assessments with poor psychometric qualities will undermine evidence-based practice and research as current health status or intervention effects cannot be objectified if measures lack robustness in validity, reliability, or responsiveness [9, 10]. Selecting the best evidence-based screening and assessments for dysphagia is an essential step in the management of dysphagia.
The purpose of this white paper by the European Society for Swallowing Disorders (ESSD) is to report on the current state of screening and non-instrumental assessment for dysphagia in adults. The measures that are available, how to select screening tools and assessments, and gaps in research that still need to be addressed in future research will be discussed.
Screening
Screening in a Nutshell
Screening is generally accepted as the first step in the management of dysphagia by identifying patients at risk for swallowing problems (see Fig. 1). Being identified as at risk of dysphagia following screening indicates the need for further assessment. A great variety of different types of screening has been described in the literature. Many screening tools consist of trial swallows using water in various aliquots or a range of different viscosities. Other forms of screening utilize combined protocols of trial swallows and pulse oximetry, pulse oximetry alone, evaluating clinical features (e.g. abnormal gag, voice alteration, or volitional cough), cervical auscultation, and elements of medical history (e.g. recurrent episodes of pneumonia or cough elicitation tests) [11, 12]. In addition, screening tools may use different endpoints to define being at risk, such as penetration, aspiration, pharyngeal delay, or dysphagia.
Screening tools using multiple swallow trials may differ in the order of selected viscosities or selected volumes per swallow trial. For example, the Toronto Bedside Swallowing Screening Test or TOR-BSST [13] consists of two steps; the first step is screening for abnormalities in voice quality and tongue movement, and the second step consists of ten consecutive teaspoons of water. Any failed item during either the first or second step constitutes a positive screen result and indicates an increased risk for dysphagia and a need to referral for assessment. Authors of water swallow screening tools may argue that the rationale for screening is solely to identify persons at risk of dysphagia and, therefore, no other consistencies need to be included[14].
Conversely, the volume–viscosity swallowing test or V-VST [15], another commonly used screening, uses three different viscosities (referred to as nectar, liquid, and pudding) and three volumes (5, 10 and 20 ml), resulting in a maximum of nine swallow trials. Whereas the TOR-BSST screens for being at risk for dysphagia only, the V-VST provides additional oral intake advice. Still, as only single trials are used per volume–viscosity combination, any advice on oral intake should be made with caution, as using a limited number of swallow trials can underestimate the aspiration risk in persons with dysphagia, especially when using boluses with a thin liquid consistency [16]. Even when a screening tool uses several trials of the same volume and viscosity, for example the Gugging Swallowing Screen or GUSS [17], any advice on oral intake should nevertheless be supported by findings from further assessment.
There is still no consensus on the order in which different viscosities should be offered when expanding the water protocol with, for example, thickened liquids, puddings, or solid boluses. Some authors start with thickened liquids using increased patient safety in terms of reduced frequency of aspiration as rationale, whereas other authors prefer starting with thin liquids referring to the heightened risk of residue when using higher viscosities [18]. Also, the use of different volumes may impact screening results; consecutive sips with large volumes in patients without overt airway responses or voice changes seem to rule out risk of aspiration, whereas small volumes with single sips indicate aspiration when clinical signs are present [14].
All patients at risk of dysphagia should be screened. Common populations at risk are frail older persons, stroke patients, patients with progressive neurological diseases (e.g. Parkinson’s disease or dementia), children and adults with non-progressive neurodevelopmental conditions (e.g. cerebral palsy), and patients with head and neck cancer [7, 19, 20]. Patients failing screening should be referred for further assessment by a dysphagia expert. However, patients having an established risk for dysphagia or manifest signs of dysphagia through case history may be referred for further assessment without being subjected to screening. If patients cannot be screened due to their cognitive state or health condition, screening should be postponed, usually resulting in recommended nil per os while waiting for screening. Some patient groups, however, may follow different screening and assessment pathways as no improvement in their condition is to be expected, for example, patients who may be at risk of dysphagia and have progressive cognitive decline (e.g. dementia). The lack of consensus and guidelines on how to manage these vulnerable populations result in moral dilemmas that need to be resolved on a case-by-case basis between healthcare practitioners and patients’ family. In comparison, screening is usually recommended to be performed within the first 24 h following the stroke [21]; however, there is no international consensus on what should be the critical timeframe for screening in other diagnostic groups.
Selection Criteria for Screening
Screening needs to meet the following feasibility criteria before being implemented in clinical settings: simple administration, use of non-invasive methods to minimize distress to patients, and straightforward to train clinicians involved [22, 23]. While most screening tools are freely available, some, like the TOR-BSST, can only be used after attending an online training module to improve standardization and consistency in rating.
Screening tools need to be valid, reliable, and show good diagnostic performance. Screening tools can be developed and validated for specific target populations (e.g. patients after stroke, patients with head and neck cancer or frail older persons). The use of certain screening tools may not be recommended in other patient groups than the populations for which the screen was originally developed and validated for; validity of a screening must be confirmed again when its use is extended to other clinical populations. Sufficient intra-rater, inter-rater, or test–retest reliability is often expressed by an intra-class correlation coefficient (ICC) or weighted Kappa equal to or greater than 0.70 [24]. Diagnostic performance of a screening tool is determined by comparison with a reference test, usually fiberoptic endoscopic evaluation of swallowing (FEES) or videofluoroscopic evaluation of swallowing (VFS). According to the international COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN), this is also referred to as criterion validity [25]. The reference test is considered the ‘gold standard’ and hypothesized to have no false negative or false positive results. In other words, the ‘gold standard’ correctly indicates the presence of dysphagia when the person truly has dysphagia (true positives) and identifies subjects as not having dysphagia when they truly do not have dysphagia (true negatives). As such, it is not recommended to use another screening or clinical assessment as reference test unless considered a ‘gold standard’. Furthermore, both reference tests and screening tools may use different endpoints. For example, when administering a FEES, the occurrence of aspiration may be used as endpoint to confirm the diagnosis of dysphagia, whereas a water swallow screening may use coughing or a wet voice as endpoint for identifying a patient at risk of dysphagia. As different endpoints yield different diagnostic performance, clinicians and researchers must predefine and select endpoints before determining diagnostic performance.
Diagnostic performance comprises elements such as sensitivity, specificity, positive and negative predictive value, positive and negative likelihood ratio, and diagnostic odds ratio (Table 1) [26]. As applied to the topic at hand, sensitivity and specificity refer to how well the screening tool identifies the presence and absence of dysphagia, respectively, whereas predictive values are the probability that a positive or negative test result is accurate. As such, sensitivity and specificity provide clinicians information about which screening to select, whereas predictive values inform the persons being tested about the chances of actually having dysphagia.
Table 1.
Term | [Acronym] | Definition | Formulaa |
---|---|---|---|
Prevalence | [PR] | Proportion of subjects in a population having a disease | Diseased / Total population |
Sensitivity | [Se] | Proportion of reference test positive (diseased) subjects who test positive with the screening tool | Se = TP / (TP + FN) |
Specificity | [Sp] | Proportion of reference test negative (healthy) subjects who test negative with the screening tool | Sp = TN / (TN + FP) |
Positive Predictive Value | [PPV] | Proportion of positive results in statistics and diagnostic tests that are true positive results | PPV = TP / (TP + FP) |
Negative Predictive Value | [NPV] | Proportion of negative results in statistics and diagnostic tests that are true negative results | NPV = TN / (TN + FN) |
Positive Likelihood Ratio | [LR +] | Probability of a person who has the disease testing positive divided by the probability of a person who does not have the disease testing positive | LR + = Se / (100 – Sp) |
Negative Likelihood Ratio | [LR-] | Probability of a person who has the disease testing negative divided by the probability of a person who does not have the disease testing negative | LR- = (100 – Se) / Sp |
Diagnostic Odds Ratio | [DOR] | Odds of a positive test in those with disease relative to the odds of a positive test in those without disease | DOR = LR + / LR- |
Error | % of observations that were misclassified by the model | (FP + FN) / (TP + TN + FP + FN) | |
Accuracy | % of observations that were correctly classified by the model |
(TP + TN) / (TP + TN + FP + FN) = 100 – error |
|
Precision | Proportion of TP to all positive predictions | TP / (TP + FP) = PPV |
aTP = True Positive = Identified patient; FP = False Positive = Healthy person identified as patient (Not identified healthy person); TN = True Negative = Identified healthy person; FN = False Negative = Patient identified as healthy person (Not identified patient)
Depending on work environment and requirements, different cut-offs for diagnostic performance may be used. For example, if assessment after screening is expensive or invasive, clinicians may prefer to minimize the number of false positives and use a cut-point with high specificity. Conversely, if the penalty for missing a case is high (e.g. the disease is fatal but can be treated successfully), a clinician may prefer to maximizing the number of true positives by using a cut-point with high sensitivity. In general, tests with high sensitivity have low specificity; they are good for identifying actual cases of the disease but often at the expense of high rates of false positives. For example, results from a meta-analysis showed that increasing volumes in water swallow tests resulted in higher sensitivity but lower specificity when screening for aspiration in stroke patients [27]. There are, however, no strict guidelines on the optimal cut-offs for diagnostic performance. Some diagnostic reviews only included screening tools with a high sensitivity (at least 70%) and at least a moderate specificity (at least 60%; e.g. [11, 12]). Even so, guidelines on what cut-off scores to use remain ambiguous.
A receiver operating characteristics (ROC) curve, a probability curve, visualizes the trade-off between sensitivity and specificity by presenting the true positive rate (sensitivity) as function of the false positive rate (100 minus specificity). Conversely, the area under the curve (AUC) can be interpreted as the average value of sensitivity for all possible values of specificity or the average value of specificity for all possible values of sensitivity [28]. AUC represents a measure of separability; how well is the model capable of distinguishing between two classes or groups, for example, between persons with or without dysphagia? The COSMIN guidelines stipulate that an AUC of at least 0.70 (or 70%) is required to provide evidence for sufficient criterion validity [24].
Non-Instrumental Assessment
Assessment in a Nutshell
After a patient has been screened for dysphagia and identified as being at risk for dysphagia, further assessment is required (Fig. 1). Both ‘gold standard’ assessments VFS and FEES can diagnose aspiration (including silent aspiration) and other physiological problems in the pharyngeal phase. However, access to instrumental assessment may be restricted and no international consensus exists regarding which visuoperceptual or software-based measures to use for analysis of these video recordings. Moreover, there is insufficient psychometric evidence to recommend any individual measure as valid and reliable to interpret VFS and FEES recordings [29].
Another step after screening is non-instrumental clinical assessment by a dysphagia expert. In the process of clinical assessment of dysphagia, many different aspects may be distinguished, for example, medical history: conducting a physical examination; the subjective description of the swallowing problem or patients’ complaints [30]; and the expert’s clinical observations during the interview and examination process. Clinical assessment may have the following purposes, single or in combination: to identify possible causes of the swallowing problems; to estimate the safety of swallowing and risk of aspiration; to support decisions on oral or alternative feeding routes; to identify the need for further assessment (e.g. instrumental assessment); and to establish baseline data for future comparisons after intervention or during the course of a disease [22]. Clinical assessment may also address patient needs that are specific to certain clinical populations. For example, in patients with head and neck cancer, the effects of radiation and chemotherapy may be important and need to be described in detail.
Clinical assessment may refer to a large variety of assessments, each of which may be describing different aspects of dysphagia. In the absence of systematic reviews, textbooks and current opinion papers provide different overviews. While there is currently no comprehensive overview of clinical assessments, most literature seems to agree on the following four categories: 1) assessment of cognition and communication; 2) evaluation of the oral, laryngeal, and pharyngeal anatomy, physiology, and function (including cranial nerve examination); 3) oral intake and nutritional status; mealtime observations; and 4) intervention trials (e.g. bolus modification, postural adjustments and/or swallow manoeuvers) [22, 31].
Another form of non-instrumental clinical assessment involves patient self-report measures as part of the multidimensional assessment of dysphagia. Patient self-report measures target two distinct but related concepts: functional health status (FHS) versus health-related quality of life (HR-QoL). FHS refers to the influence of a given disease (e.g. dysphagia) on particular functional aspects, whereas HR-QoL is the unique personal perception of someone’s health, taking into account social, functional, and psychological issues [32].
Patient-Reported Measures
Patient-reported measures play an important role in patient-centred healthcare and can improve communication, patient engagement, and self-efficacy as patients may be more involved in goal setting [33–35]. The use of a patient-reported measure, however, may be limited by patients who experience cognitive and/or language comprehension deficits. Recently, a framework (patient-reported outcome measures or PROM-cycle) was developed to support the selection and implementation of patient-reported measures in individual patient care and for quality improvement [35]. The PROM-cycle describes eight steps necessary for systematic selection and implementation of PROMs in a cyclic approach and provides links to, for example, relevant tools and existing guidelines, examples from clinical practice and methodological considerations.
Patient self-report measures in dysphagia often combine both FHS and HR-QoL in the same measure, even though both are considered two distinct concepts. Consequently, disease-related functioning cannot be distinguished from disease-related quality of life as subjectively experienced by patients. Further, psychometric reviews on patient self-evaluation (e.g. [36, 37]) and psychometric evaluations on most commonly used self-report measures in dysphagia (e.g. [38–42]) identified that all existing measures have either poor psychometric properties or lack comprehensive evaluation and reporting of psychometric properties. This flags an urgent need for research. For example, even though the Eating Assessment Tool (EAT-10) is one of the most commonly used FHS measure in dysphagia, recent evidence using contemporary Item Response Theory1 (IRT; Rasch analyses [43, 44]) did not support its use in either research or clinical practice [38, 40–42]. Another frequently used FHS questionnaire is the Sydney Swallow Questionnaire (SSQ [45]), which is yet to be evaluated using IRT.
Among those self-report measures mainly targeting HR-QoL are the Swallowing Quality of Life questionnaire (SWAL-QOL [46]), the Dysphagia Handicap Index (DHI [47]), the Deglutition Handicap Index (DHI [48]), and the MD Anderson Dysphagia Inventory (MDADI [49]). The SWAL-QoL, like the EAT-10, is one of few measures that has been evaluated using Rasch analyses. As an outcome of the evaluation, it was recommended to further investigate the SWAL-QoL’s underlying structure and psychometric characteristics prior to continued clinical use [39, 50]. For most self-report measures, however, even though their use is common practice, their psychometric robustness is yet to be established using IRT [36, 37].
The use of self-report measures for oral health has gained importance in recent years. Poor oral health and concomitant dysphagia are important risk factors of aspiration pneumonia [51]. Oropharyngeal colonization by respiratory pathogens has shown to play a key role in the pathophysiology of aspiration pneumonia [52–56]. A recent systematic review reporting on patient-reported measures for oral health in adults identified 20 English-language multiple-item questionnaires summarized into four domains: oral function, orofacial pain, orofacial appearance, and psychosocial impact [57]. Examples of frequently used self-report measures in oral health include the Oral Health Impact Profile (OHIP [58]) and the Oral Health Questionnaire for Adults [59]. Most self-reported measures in oral health, however, need further psychometric evaluation before supporting their implementation in healthcare and research [57].
Given that gastroesophageal reflux disease (GERD) is common in populations presenting with dysphagia [2, 60, 61], self-report on GERD may form part of dysphagia assessment. Many examples can be found in the review by Bolier, Kessing [62] including, for example, the widely used and validated GERD Impact Scale (GIS [63]) and the Reflux Disease Questionnaire (RDQ [64]). Still, in the absence of further psychometric evaluations, many of these measures may not meet psychometric quality criteria [65].
Assessment of Anatomy and Physiology
Before assessing the anatomy and physiology of the swallowing act, some higher cortical functions that act as precursors to oral feeding need to be evaluated. These include, but are not limited to, the patient’s alertness, responsiveness, cognition and language skills, as well as head and trunk control (i.e. motor control) [66]. In the absence of standardized formal assessments, input–output reasoning will inform further clinical assessment of functional anatomy (Table 2–3) whereby innervation (input) from a muscle or functional groups of muscles (effectors) will, in turn, result in deglutition-related actions (output).
Table 2.
Input aAfferents | Effector | Outputa Efferents |
---|---|---|
V2 (maxillary nerve), V3 (lingual nerve: branch of inferior alveolar nerve of the mandibular nerve) | Lips | VII (Labial sphincter) |
V3 (lingual nerve: branch of mandibular nerve) | Tongue | XII (Oral control of bolus) |
V3 (mandibular nerve) | Jaw | V motor (Mastication) |
V, IX | Soft palate | V, X (Palate function) |
V | Mouth and cheeks | V motor, VII (Oral control of bolus) |
IX | Base of tongue | XII (Propulsion into oropharynx) |
X | Epiglottis (lingual side) | X (Laryngeal sphincter function) |
X (superior laryngeal nerve) | Epiglottis (laryngeal side) | X (Laryngeal sphincter function) |
X (superior laryngeal nerve) | Glottis and supraglottal larynx | X (Laryngeal sphincter function) |
X (inferior laryngeal nerve) | Subglottal larynx | X (Laryngeal sphincter function) |
X | Cervical trachea | X (Cough reflex) |
V, IX, X | Naso-oropharynx | IX, X (Velopharyngeal sphincter function) |
X | Hypopharynx | X (Propulsion, pharyngeal squeeze) |
aCranial Nerves: I Olfactory nerve, II Optic nerve, III Oculomotor nerve, IV Trochlear nerve, V Trigeminal nerve, VI Abducens nerve, VII Facial nerve, VIII Vestibulocochlear nerve, IX Glossopharyngeal nerve, X Vagus nerve, XI Accessory nerve, XII Hypoglossal nerve
Table 3.
Functional group | Motor Input | Effector | Output |
---|---|---|---|
Innervationa | (Main) Muscles | (Main) Actions in Deglutition | |
Masticatory | CN V |
Lateral / Lateral pterygoid Masseter Temporalis |
Mastication Closure oral cavity Mandible: raise |
Facial | CN VII |
Buccinator Orbicularis oris |
Lip/mouth: seal Bolus: push towards teeth |
Intrinsic tongue | CN XII |
Inferior / Superior longitudinal Transverse Verticalis |
Tongue: shorten – lengthen, narrow – broaden, tip up –down, concave – convex bow tongue Bolus: preparation, formation, positioning, transport |
Extrinsic tongue | CN X | Palatoglossus |
Tongue: protrude – retract, lower – raise Bolus: preparation, formation, positioning, transport Seal oral cavity |
CN XII |
Genioglossus Hyoglossus Styloglossus |
||
Suprahyoid | CN V |
Anterior belly of digastric Mylohyoid |
Hyoid: lower – raise, protract – retract, stabilize Mouth floor: stabilize, elongate Mandible: lower |
CN VII |
Posterior belly of digastric Stylohyoid |
||
C1 [via CN XII] | Geniohyoid | ||
Infrahyoid | C1 [via CN XII] | Thyrohyoid |
Hyoid: lower, stabilize Mouth floor: stabilize, elongate Larynx: raise – lower, stabilize |
C1-C3 [via Ansa cervicalis CN XII] |
Omohyoid Sternohyoid Sternothyroid |
||
Palatal | CN V | Tensor veli palatine |
Soft palate: raise – retract, lower, brace, tense Oropharynx entrance: widen Posterior tongue: raise Uvula: raise Seal back of oral cavity from oropharynx Seal nasopharynx |
CN X |
Levator veli palatin Palatoglossus |
||
CN XI [via CN X] | Uvular | ||
Pharyngeal | CN IX | Stylopharyngeus |
Palate: lower Pharynx: raise, shorten Larynx: raise Seal oral cavity Seal nasal cavity Narrow pharyngeal lumen Bolus: transport Oesophageal sphincter: most distal component (of pharyngo-oesophageal segment or PES) |
CN X |
Palatopharyngeus Salpingopharyngeus Superior / Middle / Inferior pharyngeal constrictor |
||
Laryngeal | CN X |
Aryepiglottic Lateral / posterior cricoarytenoid Oblique / Transverse arytenoid Thyroarytenoid Thyroepiglottic |
Vocal folds: adduct – open Arythenoids cartilages: approximate to epiglottis Epiglottis: lower Aryepiglottic foldsb |
aCranial Nerves: I Olfactory nerve, II Optic nerve, III Oculomotor nerve, IV Trochlear nerve, V Trigeminal nerve, VI Abducens nerve, VII Facial nerve, VIII Vestibulocochlear nerve, IX Glossopharyngeal nerve, X Vagus nerve, XI Accessory nerve, XII Hypoglossal nerve
bLaryngeal sphincter function or squeeze: true vocal folds, false vocal folds, aryepiglottic folds
A strict order of events provides clinical markers that guide clinicians during assessment with a functional–anatomic pathway of motor outputs activated by sensory inputs (Table 2) [67]. Therefore, the anatomy and physiology that underpins the swallowing act need to be understood to allow for the assessment thereof. Swallowing is a highly integrated neuro-sensorimotor function, realized by organs (effectors) affiliated with different systems and inter-linked with other functions, so that swallowing, breathing, and phonoarticulation alternate according to hierarchically correlated pathways [68]. Swallowing is the result of external (sight and smell) and internal sensory stimuli (touch, temperature, and taste), which converge at the meduallary swallowing centre, from where the motor efferents of the bulbar cranial nerves (CNs) start: CN V (Trigeminal nerve: motor), CN VII (Facial nerve), CN IX (Glossopharyngeal nerve), CN X (Vagus nerve), CN XII (Hypoglossal nerve), and cervical spinal nerves C1-C3. The CN XI (accessory nerve) carries motor fibres for the sternocleidomastoid muscle and trapezius muscles (spinal root), and parasympathetic preganglionic visceral fibres (cranial root) that join the CN X reaching the soft palate, pharynx, larynx, and oesophagus. The brainstem swallowing centre is widely connected and receives cortical, subcortical, thalamic, limbic and cerebellar descending modulatory afferents, which transform the semi-reflex nature of swallowing into spatiotemporal integrated muscular contractions (muscular patterns) [5]. The neurological structures (including cortex, extrapyramidal and pyramidal system, and cranial nerves) are contained in the neurocranium (protective shell surrounding the brain and brain stem), while the swallowing effectors are contained in the viscerocranium (skeleton supporting facial structure). At this level, the upper parts of the respiratory and digestive tract are arranged in parallel and conjugate in the pharyngeal cavity, which becomes a common pathway for both air and bolus transport. In this way, the pharynx acts alternately as an organ of the respiratory tract (chamber for air passage) as well as an organ of the digestive tract (peristaltic contracting chamber for bolus passage).
Anatomy of the upper digestive tract allows the organization of functional chambers operating in a concatenated series [69]. In the oral cavity, the bolus is prepared (minced and mixed with saliva) as a consequence of coordinated actions between facial muscles, cheeks, teeth, jaws, tongue, and oral floor muscles [70]. When the bolus is transported into the pharyngeal cavity, a reflex reconfiguration of the pharynx occurs; the oropharynx transforms from a respiratory pathway to a swallow pathway [71]. The pharynx acts as a digestive lumen, generating contractile and peristaltic activities with the aim of transporting the bolus to the oesophagus [72]. A strict hierarchy between vital primordial reflex activities determines the arrest of respiration (apnea), thus allowing the pharyngeal transportation of the bolus [73]. The reconfiguration of the pharynx is defined by glossopalatal opening, velopharyngeal closure, upper and anterior hyoid laryngeal excursion, and upper oesophageal sphincter opening.
Articulation and voice quality are checked during speech and sustained vowel production and provide information about vocal fold or glottis closure. Furthermore, facial appearance and expression may indicate concerns about orofacial musculature, presence of surgical scarring, asymmetry, and inadequate labial seal (drooling). Touch, temperature, and taste sensibility can be tested on the lips, mucosa of the oral cavity, tongue (body, base, margins), fauci, soft palate, and posterior wall of the oropharynx using low-tech devices such as a wisp of cotton or tongue blade. The same low-tech devices can be used to test thermal and gustatory sensibility by wetting them in hot or iced water, or in salty, sweet, bitter, sour, or sparkling solutions.
Tone and strength of the tongue can be assessed by eliciting resistance against a tongue blade, while mobility is assessed by administering non-articulatory or articulatory praxis tasks. The opening of the jaws (temporomandibular joint) is closely examined and masticatory muscles are evaluated for its praxis and strength. The pterygoid muscles are tested during lateral mandibular movements with the jaw applying resistance against a hand, while the masseter and temporal muscles can be palpated during contraction. The symmetrical soft palate movement is evaluated using a simple articulatory task (i.e. by sustaining the vowel /a:/) or during the process of testing sensibility. The gag reflex can be elicited by stimulating the back of pharynx with a tongue blade/probe. Finally, the presence of the pharyngeal reflex is elicited by asking patients to perform a dry swallow while the clinician holds three fingers on the external hypo-laryngeal axis; the index finger is placed on the hyoid bone, the middle finger is placed on the thyroid notch, and the ring finger is placed on the cricoid ring. There should be an antero-superior displacement of these structures of about 2 to 2.5 cm during a dry swallow. Table 3 provides additional examples of clinical assessment of functional anatomy and physiology of the swallowing act.
Clinical Swallowing Evaluation
A clinical swallowing evaluation (CSE) is typically performed after screening for swallowing problems to provide additional information to ascertain the relative risk for aspiration and to direct clinical decision making [74]. CSE findings are important in determining whether an instrumental assessment may be required (if not already performed) and to decide which compensatory strategies and postural manoeuvres need to be prioritized for trialling to inform dysphagia therapy planning and patient management. At least a tentative diagnosis must be made and a management plan agreed upon following a CSE [31]. However, aspiration and other physiological problems in the pharyngeal phase of swallowing can only be diagnosed through directly observation using instrumental assessments [75–77].
The initial step in a CSE is to compile a thorough case history with the patient and/or carer. This involves a careful review of the medical history, if available, a history of the present condition (e.g. onset, duration of difficulties, symptoms), current swallowing status (e.g. method of oral intake and current diet, strategies that help swallowing, easiest and most difficult food consistencies to swallow), and factors that might influence management (e.g. comorbidities, cognition, food restrictions and nutritional status, presence of gastroesophageal reflux, and cultural preferences) [30]. Common examples of the many available measures used while compiling a case history are the Functional Oral Intake Scale (FOIS; [78]) and the Mini-Mental State Examination (MMSE; [79]) or Mini-Cog [80] to screen for cognitive impairment.
After compiling the case history, the clinician forms a preliminary clinical hypothesis that is tested during ensuing assessment. Only a few standardized CSE tests are available across dysphagia populations. The Mann Assessment of Swallowing Ability (MASA [81]) is an example of a standardized CSE. The MASA was validated on stroke patients and has adaptations for patients with head and neck cancer (MASA -C [82]). While conducting a CSE, the clinician also completes a clinical observation of the patient noting body posture, head posture at rest, level of alertness, and ability to follow assessment instructions, upper respiratory tract secretions, and the patient’s ability to manage his/her saliva. In addition, an orofacial examination is conducted that includes a cranial nerve examination. Oral health may be assessed using a measure such as the Oral Health Assessment Tool (OHAT [83]).
Trial feeding is only considered an appropriate management strategy if the patient has an adequate voluntary cough and is able to manage his/her own secretions. If, however, the patient is drowsy, medically unstable, and unable to swallow saliva requiring suctioning, then trial feeding is deemed too high risk and should not be attempted. Self-feeding is preferred over feeding by carers or health care providers, to mimic typical eating and drinking patterns. As mentioned earlier, there are opposing views regarding which consistency to trial first. Some clinicians prefer to begin trials with small 5 ml volumes of water, gradually increasing the volume before moving to thicker consistencies. The rationale is that water, if aspirated, combined with good oral hygiene will be absorbed into the lungs without the immediate risk for the development of pneumonia [84, 85]. Other clinicians prefer to start with thickened consistencies so as to reduce the risk for aspiration [15]. Trials with fluids that promote sensation, for example carbonation and sweet and sour bolus, may be included in CSE [86–88]. The temperature of the bolus may also be manipulated with colder boluses used to increase sensation [89]. Thicker consistencies may be trialled later as they may give rise to residue in the pharynx which, in turn, can lead to aspiration [18]. When assessing chewing it is important to evaluate the patient’s ability to manage solids and to identify risk of choking also. Assessments such as the Test of Mastication and Swallowing of Solids (TOMASS [90]) may be useful.
When possible, a CSE should also include mealtime observation. This is particularly useful in patients with cognitive impairments where feeding and swallowing difficulties may be influenced by fluctuating levels of attention, fatigue, or environmental factors (e.g. background noise or other distractors). Observations such as the patient’s ability to self-feed, the need to use adaptive eating utensils, duration of the meal, and the presence of fatigue are noted. Examples of available standardized CSEs based on mealtime observation are the McGill Ingestive Skills Assessment (MISA [91]) for older persons with feeding difficulties, and the Dysphagia Disorder Survey (DDS; [92]) developed and validated for persons with developmental disabilities. Both measures have demonstrated promising, robust psychometric properties [91, 93, 94].
During the CSE, individualized trials are conducted to investigate the effectiveness of compensatory strategies the person may already have put in place or to test new strategies and examine which are most effective for safe and efficient oral intake. Compensatory strategies comprise changing what the person eats (i.e. diet modification) and changing how the person eats (e.g. head posture, alternate liquid/solid swallows, volume control). The efficacy of implementing adaptive equipment (e.g. beakers, modified spoons, straws) in promoting safe oral intake should also be examined.
Ancillary to conducting CSE is evaluation such as pulse oximetry and cervical auscultation. Pulse oximetry measures peripheral capillary oxyhemoglobin saturation and is used to detect a decrease in saturation, which is indicative of aspiration during swallowing [95]. Cervical auscultation is hypothesized to differentiate between normal and abnormal swallowing by listening to the sound of swallowing and swallowing-related respiration using a stethoscope placed on the neck [96]. However, there is contention regarding diagnostic accuracy of pulse oximetry in predicting aspiration and current evidence does not support its use [95]. Cervical auscultation, although commonly used, is equally contentious and there is limited evidence to support its use [96].
Limitations of the CSE are that protocols vary and that there is variability in practice even within the same clinical settings and between populations [97–99]. However, its key strength is providing low-cost clinical information to the multidisciplinary team in the immediate management of patients with dysphagia, especially if clinical teams do not have access to instrumental assessments.
Challenges and Minimum Requirements for Assessment
A core outcome set (COS) is an agreed minimum set of outcomes that should be measured and reported in clinical trials of a specific disease or target population [10]. When selecting outcome measures as part of a COS, four criteria need to be considered [10]. First, both the construct to be measured (e.g. FHS, HR-QoL) as well as the target population (e.g. age, diagnosis) must be clearly defined. Second, existing measures must be identified by, for example, performing literature searches or using recent systematic reviews. Third, the quality of all measures must be evaluated both in terms of quality of psychometric properties as well as feasibility for administering the measure within particular setting. Fourth, one measure must be selected for each outcome or construct in a COS (Fig. 1).
The COSMIN group established an international consensus-based taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcome measures [25]: guidelines for using the taxonomy, including a standardized appraisal tool for rating methodological quality of psychometric studies [100], criteria for evaluating psychometric quality of assessments [101, 102], and a rating system for synthesizing and grading psychometric evidence [101, 102]. The taxonomy distinguishes nine psychometric properties across three domains (Table 4): validity (content validity, structural validity, cross-cultural validity, hypothesis testing for construct validity and criterion validity), reliability (internal consistency, reliability and measurement error), and responsiveness. When selecting an assessment, all nine psychometric properties are considered when identifying the assessments with the most robust psychometric properties.
Table 4.
Domain | Measurement property | Definitiona |
---|---|---|
Validity | Degree to which an instrument measures the construct(s) it purports to measure | |
Content validity | Degree to which the content of an instrument is an adequate reflection of the construct to be measured | |
Structural validity | Degree to which the scores of an instrument are an adequate reflection of the dimensionality of the construct to be measured | |
Hypotheses testing for construct validity | Degree to which the scores of an instrument are consistent with hypotheses based on the assumption that an instrument validly measures the construct to be measured | |
Cross-cultural validity | Degree to which the performance of the items on a translated or culturally adapted instrument are an adequate reflection of the performance of the items of the original version of the instrument | |
Criterion validity | Degree to which the scores of an instrument are an adequate reflection of a ‘gold standard’ | |
Reliability | Degree to which the measurement is free from measurement error | |
Internal consistency | Degree of the inter-relatedness among the items | |
Reliability (intra-rater, inter-rater, test–retest) |
Proportion of the total variance in the measurements which is because of ‘true’ differences among patients | |
Measurement error (intra-rater, inter-rater, test–retest) | Systematic and random error of a patient’s score that is not attributed to true changes in the construct to be measured | |
Responsiveness | Ability of an instrument to detect change over time in the construct to be measured | |
Interpretabilityb | Degree to which one can assign qualitative meaning to an instrument’s quantitative scores or change in scores |
aDefinitions derived from Mokkink, Prinsen [24]
bInterpretability is considered an important characteristic of a measurement instrument but is not a psychometric property
Assessments that have poor psychometric properties should not be used in either clinical settings or research. Minimum requirements for including an outcome measure in a COS are high-quality evidence for good content validity and good internal consistency [10]. However, as recent psychometric reviews have identified missing or indeterminate evidence for many psychometric properties of existing assessments in dysphagia (e.g. [29, 36, 37]), often only preliminary conclusions on the psychometric robustness of the assessments can be drawn based on current available psychometric evidence in the literature.
Content validity, the degree to which the content of an assessment is an adequate reflection of the construct to be measured, is considered the most important psychometric property within the COSMIN taxonomy [102]. Concerns around content validity are raised when assessments that have been developed and validated for specific target populations are used in different populations. For example, the MD Anderson Dysphagia Inventory (MDADI) is a dysphagia-specific quality of life questionnaire targeting patients with head and neck cancer [49], whereas the Dysphagia in MUltiple Sclerosis questionnaire (DYMUS) is a FHS self-report assessment for adults with multiple sclerosis [103]. Assessments like the MDADI and DYMUS that have been developed and validated for a specific clinical population cannot be used in another population without the risk of compromising its content validity [102]. For an assessment to have good content validity [102], all items need to be aligned with the construct of interest in a specific population and context of use (relevance), all key concepts of the construct need to be covered (comprehensiveness), and instructions, items and response options need to be understandable to the target population as intended (comprehensibility).
Problems may also occur when using a translated version of an original assessment, without undertaking the required steps to ensure good cultural validity. The proposed framework of translation by Koller, Kantzer [104] is based on a review of influential translation guidelines and summarizes the reconciliation processes that needs to be undertaken when translating measures. The framework stipulates that through a process of reconciliation, two or more independent forward translations are merged into one single translation using decision criteria on comprehensibility, cultural appropriateness, grammar, and use of relevant and consistent terminology. Next, cross-cultural validity needs to be determined by comparing the performance of items from a translated or culturally adapted assessment with the original assessment version [24].
Finally, aspects of feasibility need to be taken into consideration when deciding on which assessments to select for measuring outcomes. Feasibility refers to the practicability of using an assessment in its intended setting, taking into account constraints of time, money, and interpretability [10]. Other important factors related to feasibility include, for example, patient’s cognitive functioning and physical ability, patient’s and clinician’s ability to comprehend items, ease of administration, duration of administering the assessment, simplicity of score calculation, accessibility of required equipment in different settings, copyright requirements, and associated costs to acquire and administer the assessments [10]. Another consideration include administering, where appropriate, assessments via telehealth instead of face-to-face to increase accessibility to healthcare in rural and remote areas [105], to increase service reach, reduce costs [106], and more recently to continue providing safe health services during the COVID-19 pandemic.
Recommendations and Future Research
Selection of Screening and Assessments
Where access to instrumental assessment is restricted, the use of clinical assessments becomes exponentially important when evaluating patients’ illness trajectories. There are many screening and non-instrumental assessments available for evaluating dysphagia in adults (see Table 5). However, even though measures may be part of daily clinical practice, their use may not be warranted due to poor diagnostic performance or lacking robust psychometric properties.
Table 5.
Domain | Screening and assessmenta | Acronym | Respondent | Reference |
---|---|---|---|---|
Standardized screening | ||||
At risk of swallowing problems |
Gugging Swallowing Screen Toronto Bedside Swallowing Screening Test Volume-viscosity swallowing test |
GUSS TOR-BSST V-VST |
Clinician Clinician Clinician |
Trapl et al., 2007 Martino et al., 2009 Clavé et al., 2008 |
Standardized assessments | ||||
Cognition & Communication | Mini-Cog | – | Clinician | Borson, 2000 |
Mini-Mental State Examination | MMSE | Clinician | Folstein et al., 1975 | |
Oral intake status | Functional Oral Intake Scale | FOIS | Clinician | Crary et al., 2005 |
Dysphagia-related quality of life & Functional health status |
Dysphagia Handicap Index MD Anderson Dysphagia Inventory Sydney Swallow Questionnaire Swallowing Quality of Life questionnaire |
DHI MDADI SSQ SWAL-QOL |
Patient Patient Patient Patient |
Silbergleit et al., 2012 Chen et al., 2001 Dwivedi et al., 2010 McHorney et al., 2002 |
Oral health status |
Oral Health Assessment Tool Oral Health Impact Profile Oral Health Questionnaire for Adults |
OHAT OHIP – |
Clinician Patient Patient |
Chalmers et al., 2005 Slade et al., 1994 WHO, 2013 |
Gastroesophageal reflux disease |
GERD Impact Scale Reflux Disease Questionnaire |
GIS RDQ |
Patient Patient |
Jones et al., 2007 Shaw et al., 2001 |
Swallowing | Mann Assessment of Swallowing Ability | MASA | Clinician | Mann et al., 2002 |
Mastication | Test of Mastication and Swallowing of Solids | TOMASS | Clinician | Huckabee et al., 2018 |
Mealtime observation |
McGill Ingestive Skills Assessment Dysphagia Disorder Survey |
MISA DDS |
Clinician Clinician |
Lambert et al., 2003 Sheppard et al., 2014 |
Non-standardized assessments | ||||
Anatomy / Cranial nerve integrity | Clinical examination of the tongue, hard and soft palate, teeth, gums, oral mucosa, trigeminal (V), facial (VII), glossopharyngeal (IX), vagal (X), and hypoglossal (XII) cranial nerves. | |||
Oral motor skills / Physiology |
Clinical examination of oral muscle strength, range, tone, steadiness, accuracy and coordination. Mealtime observation including observation of drooling or sialorrhea, mastication, eating speed, cough or choking, oral residue, head and body positioning. |
|||
Trial feeding Intervention trials / Compensatory strategies |
Bolus modification, postural adjustments and/or swallow manoeuvers |
aNo international consensus exists on which screen or assessment for dysphagia in adults is preferred. In addition, many screens and assessments have unknown or poor psychometric properties. The presented list of screens and assessments does not provide a comprehensive overview, but examples of common clinical practice.
Therefore, based on this ESSD white paper, the following recommendations are made (Fig. 1):
Discontinue the use of non-validated dysphagia screening tools. Instead, use screening tools with good diagnostic performance, good reliability and validity and tools that meet feasibility criteria.
Implement screening using tools that have optimal diagnostic performance in selected populations that are at risk of dysphagia. These populations include, but are not limited to, stroke patients, frail older persons, patients with progressive neurological diseases (e.g. Parkinson’s disease or dementia), persons with cerebral palsy, and patients with head and neck cancer.
Discontinue the use of measures with insufficient or poor psychometric properties. Instead, use measures that demonstrate robust psychometric properties that meet psychometric quality and feasibility criteria.
Provide quality training in dysphagia screening and assessment to all clinicians involved in the care and management of persons with dysphagia.
Future Research and Clinical Practice
There are great challenges for future research in the screening and assessment of dysphagia in adults:
Existing measures in dysphagia with incomplete or missing evaluations of psychometric properties (i.e. validity, reliability, and responsiveness), need urgent and ongoing study.
New measures need to be developed by using contemporary psychometric standards and methods, such as item response theory in combination to classic test theory.
To ensure adequate content validity, studies should be conducted at the onset of developing a new measure to reach consensus on underlying definitions of constructs and to ensure that the items to be include are relevant and comprehensible and the measure is comprehensive in capturing the underlying construct(s).
There is a need for robust studies that examine the costs and benefits of screening and assessment by telehealth versus face-to-face administration.
There is an urgent need to reach international consensus on several crucial topics. Delphi studies2 would be an appropriate method to aim for consensus among experts and stakeholders on the following topics [107, 108]:
A universally accepted definition dysphagia?
How to define and measure severity of dysphagia?
What is the minimum set of outcomes (COS) that should be measured and reported in dysphagia?
Which outcome measures should be selected for each outcome in the minimum set of outcomes (COS)?
What are the critical time points for administering dysphagia screening and assessment?
Instrument development and international consensus will enhance research designs and outcome measurement in future clinical trials and subsequently lead to improved health care pathways for patients with dysphagia. There is a need for standardized screening and assessment, and consistency in terminology and definitions used in research and clinical practice. When addressing the validity, reliability, and responsiveness of outcome measures, both measurement properties and criteria for good psychometrics should be defined (e.g. the COSMIN framework). Finally, when referring to dysphagia, a common language is needed to define its symptoms and promote consistent use of terminology in screening, assessment and interventions.
Biographies
Renée Speyer
PhD
Reinie Cordier
PhD
Daniele Farneti
PhD
Weslania Nascimento
PhD
Walmari Pilz
PhD
Eric Verin
PhD
Margaret Walshe
PhD
Virginie Woisard
PhD
Authors Contribution
All authors contributed, read, and approved the manuscript.
Funding
Open access funding provided by University of Oslo (incl Oslo University Hospital).
Compliance with Ethical Standards
Conflict of interest
The authors declare no potential conflicts of interest.
Research Involving Human and Animal Participants
This study does not involve human participants or animals.
Footnotes
Methodologies and interpretation of Classic Test Theory (CTT) findings are easier to interpret than those of Item Response Theory (IRT); however, the CTT framework has some limitations. In IRT the unit of analysis and results are not restricted to the test population, whereas in CTT the evaluation of psychometric properties is specific to the test population. In addition, IRT evaluates the reliability of each individual item, while CTT assesses the performance of a measure as a whole [43]. IRT models estimate both item and person parameters within the same model, calculate person-free parameter estimation and item-free trait level estimation, and identify optimal scaling of individual differences based on the evaluation of differential item functioning [44].
The Delphi technique is a structured process which aims to develop group consensus on a defined topic through a series of survey rounds [107]. The same participants complete each round (although some individuals may discontinue participating). Rounds are held until consensus is reached (or it becomes apparent that consensus cannot be reached). Results from previous rounds are used to inform participants about changes that may facilitate consensus in subsequent rounds [108]. Using the Delphi technique enables large number of individuals across diverse locations and areas of expertise to be included anonymously, thus avoiding domination of the consensus process by one or a few experts [107].
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Shaker R. Oropharyngeal dysphagia. Gastroenterology & Hepatology. 2006;2(9):633–634. [PMC free article] [PubMed] [Google Scholar]
- 2.Roden DF, Altman KW. Causes of dysphagia among different age groups. Otolaryngol Clin North Am. 2013;46(6):965–987. doi: 10.1016/j.otc.2013.08.008. [DOI] [PubMed] [Google Scholar]
- 3.Martino R, et al. Dysphagia after stroke. Stroke. 2005;36(12):2756–2763. doi: 10.1161/01.STR.0000190056.76543.eb. [DOI] [PubMed] [Google Scholar]
- 4.Vergis EN, et al. Pneumonia in long-term care: a prospective case-control study of risk factors and impact on survival. Arch Intern Med. 2001;161:2378–2381. doi: 10.1001/archinte.161.19.2378. [DOI] [PubMed] [Google Scholar]
- 5.Jean A. Brain stem control of swallowing: neuronal network and cellular mechanisms. Physiol Rev. 2001;81(2):929–969. doi: 10.1152/physrev.2001.81.2.929. [DOI] [PubMed] [Google Scholar]
- 6.Kertscher B, et al. Prevalence of oropharyngeal dysphagia in the Netherlands: a telephone survey. Dysphagia. 2015;30(2):114–120. doi: 10.1007/s00455-014-9584-z. [DOI] [PubMed] [Google Scholar]
- 7.Takizawa C, et al. A systematic review of the rrevalence of oropharyngeal dysphagia in stroke, Parkinson’s disease, Alzheimer’s disease, head injury, and pneumonia. Dysphagia. 2016;31(3):434–441. doi: 10.1007/s00455-016-9695-9. [DOI] [PubMed] [Google Scholar]
- 8.Attrill S, et al. Impact of oropharyngeal dysphagia on healthcare cost and length of stay in hospital: a systematic review. BMC Health Serv Res. 2018;18(1):594. doi: 10.1186/s12913-018-3376-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Johnston, B.C, et al. Chapter 18: Patient-reported outcomes, in Cochrane handbook for systematic reviews of interventions, J.P.T. Higgins and J. Thomas, Editors. 2019, Cochrane.
- 10.Prinsen CAC, et al. How to select outcome measurement instruments for outcomes included in a “Core Outcome Set” – a practical guideline. Trials. 2016;17(1):449. doi: 10.1186/s13063-016-1555-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Bours, G.J.J.W, et al. Bedside screening tests vs. videofluoroscopy or fibreoptic endoscopic evaluation of swallowing to detect dysphagia in patients with neurological disorders systematic review. J Adv Nurs. 2009;65(3):477–93. [DOI] [PubMed]
- 12.Kertscher B, et al. Bedside screening to detect oropharyngeal dysphagia in patients with neurological disorders: an updated systematic review. Dysphagia. 2014;29(2):204–212. doi: 10.1007/s00455-013-9490-9. [DOI] [PubMed] [Google Scholar]
- 13.Martino R, et al. The Toronto Bedside Swallowing Screening Test (TOR-BSST) Stroke. 2009;40(2):555–561. doi: 10.1161/STROKEAHA.107.510370. [DOI] [PubMed] [Google Scholar]
- 14.Brodsky MB, et al. Screening accuracy for aspiration using bedside water swallow tests. Chest. 2016;150(1):148–163. doi: 10.1016/j.chest.2016.03.059. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Clavé P, et al. Accuracy of the volume-viscosity swallow test for clinical screening of oropharyngeal dysphagia and aspiration. Clin Nutr. 2008;27(6):806–815. doi: 10.1016/j.clnu.2008.06.011. [DOI] [PubMed] [Google Scholar]
- 16.Baijens LWJ, et al. FEES protocol derived estimates of sensitivity: aspiration in dysphagic patients. Dysphagia. 2014;29(5):583–590. doi: 10.1007/s00455-014-9549-2. [DOI] [PubMed] [Google Scholar]
- 17.Trapl M, et al. Dysphagia bedside screening for acute-stroke patients: the Gugging Swallowing Screen. Stroke. 2007;38:2948–2952. doi: 10.1161/STROKEAHA.107.483933. [DOI] [PubMed] [Google Scholar]
- 18.Newman R, et al. Effect of bolus viscosity on the safety and efficacy of swallowing and the kinematics of the swallow response in patients with oropharyngeal dysphagia: white paper by the European Society for Swallowing Disorders (ESSD) Dysphagia. 2016;31(2):232–249. doi: 10.1007/s00455-016-9696-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Speyer R, et al. Prevalence of drooling, feeding and swallowing problems in cerebral palsy across the lifespan: systematic review and meta-analysis. Dev Med Child Neurol. 2019;61:1249–1258. doi: 10.1111/dmcn.14316. [DOI] [PubMed] [Google Scholar]
- 20.van der Maarel-Wierink CD, et al. Subjective dysphagia in older care home residents: a cross-sectional, multi-centre point prevalence measurement. Int J Nurs Stud. 2014;51:875–881. doi: 10.1016/j.ijnurstu.2013.10.016. [DOI] [PubMed] [Google Scholar]
- 21.Summers D, et al. Comprehensive overview of nursing and interdisciplinary care of the acute ischemic stroke patient: a scientific statement from the American Heart Association. Stroke. 2009;40:2911–2944. doi: 10.1161/STROKEAHA.109.192362. [DOI] [PubMed] [Google Scholar]
- 22.Speyer R. Oropharyngeal dysphagia: screening and assessment. Otolaryngol Clin North Am. 2013;46(6):989–1008. doi: 10.1016/j.otc.2013.08.004. [DOI] [PubMed] [Google Scholar]
- 23.Donovan NJ, et al. Dysphagia Screening: State of the Art: Invitational Conference Proceeding From the State-of-the-Art Nursing Symposium, International Stroke Conference 2012. Stroke. 2012;44:e24–e31. doi: 10.1161/STR.0b013e3182877f57. [DOI] [PubMed] [Google Scholar]
- 24.Mokkink, L.B., et al., COSMIN methodology for systematic reviews of Patient‐Reported Outcome Measures (PROMs)-User manual (version 1.0). 2018.
- 25.Mokkink LB, et al. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol. 2010;63(7):737–745. doi: 10.1016/j.jclinepi.2010.02.006. [DOI] [PubMed] [Google Scholar]
- 26.Bouter LM, Zielhuis G, Zeegers M, Diagnostic and prognostic research. Bohn. Houten: Stafleu van Loghum; 2018. pp. 171–201. [Google Scholar]
- 27.Chen P-C, et al. Systematic review and meta-analysis of the diagnostic accuracy of the water swallow test for screening aspiration in stroke patients. J Adv Nurs. 2016;72(11):2575–2586. doi: 10.1111/jan.13013. [DOI] [PubMed] [Google Scholar]
- 28.Zhou XH, Obuchowski NA, McClish DK, Statistical methods in diagnostic medicine. Hoboken. NJ: John Wiley & Sons Inc; 2011. [Google Scholar]
- 29.Swan K, et al. Psychometric properties of visuoperceptual measures of videofluoroscopic and fibre-endoscopic evaluations of swallowing: a systematic review. Dysphagia. 2019;34(1):2–33. doi: 10.1007/s00455-018-9918-3. [DOI] [PubMed] [Google Scholar]
- 30.Scharitzer M, et al. Taking the history in patients with swallowing disorders: an international multidisciplinary survey. Abdominal Radiology. 2017;42(3):786–793. doi: 10.1007/s00261-016-0931-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Cichero J. Respiration and swallowing. In: Cichero J, Murdoch B, editors. Dysphagia: foundation, theory and practice. Inc: John Wiley & Sons; 2006. pp. 92–111. [Google Scholar]
- 32.Ferrans CE, et al. Conceptual model of health-related quality of life. J Nurs Scholarsh. 2005;37(4):336–342. doi: 10.1111/j.1547-5069.2005.00058.x. [DOI] [PubMed] [Google Scholar]
- 33.Porter I, et al. Framework and guidance for implementing patient-reported outcomes in clinical practice: evidence, challenges and opportunities. Journal of Comparative Effectiveness Research. 2016;5(5):507–519. doi: 10.2217/cer-2015-0014. [DOI] [PubMed] [Google Scholar]
- 34.Reuben DB, Tinetti ME. Goal-oriented patient care — an alternative health outcomes paradigm. N Engl J Med. 2012;366(9):777–779. doi: 10.1056/NEJMp1113631. [DOI] [PubMed] [Google Scholar]
- 35.van der Wees PJ, et al. Development of a framework with tools to support the selection and implementation of patient-reported outcome measures. Journal of Patient-Reported Outcomes. 2019;3(1):75–75. doi: 10.1186/s41687-019-0171-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Speyer R, et al. Psychometric properties of questionnaires on functional health status in oropharyngeal dysphagia: a systematic literature review. Biomed Res Int. 2014;2014:1–11. doi: 10.1155/2014/458678. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Timmerman AA, et al. Psychometric characteristics of health-related quality-of-life questionnaires in oropharyngeal dysphagia. Dysphagia. 2014;29(2):183–198. doi: 10.1007/s00455-013-9511-8. [DOI] [PubMed] [Google Scholar]
- 38.Cordier R, et al. Evaluating the psychometric properties of the Eating Assessment Tool (EAT-10) using Rasch analysis. Dysphagia. 2017;32(2):250–260. doi: 10.1007/s00455-016-9754-2. [DOI] [PubMed] [Google Scholar]
- 39.Cordier R, et al. Using Rasch analysis to evaluate the reliability and validity of the Swallowing Quality of Life Questionnaire: an item response theory approach. Dysphagia. 2018;33(4):441–456. doi: 10.1007/s00455-017-9873-4. [DOI] [PubMed] [Google Scholar]
- 40.Hansen T, Kjaersgaard A. Item analysis of the Eating Assessment Tool (EAT-10) by the Rasch model: a secondary analysis of cross-sectional survey data obtained among community-dwelling elders. Health Qual Life Outcomes. 2020;18(1):139. doi: 10.1186/s12955-020-01384-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Wilmskoetter J, et al. Construct validity of the Eating Assessment Tool (EAT-10) Disabil Rehabil. 2019;41(5):549–559. doi: 10.1080/09638288.2017.1398787. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Kean J, et al. An introduction to item response theory and Rasch analysis: application using the Eating Assessment Tool (EAT-10) Brain Impairment. 2018;19(1):91–102. doi: 10.1017/BrImp.2017.31. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Linacre, J.M., Rasch-model computer programs: Program manual 3.92.0. 2016, Chicago: Mesa-Press.
- 44.Reise S, Henson J. A discussion of modern versus traditional psychometrics as applied to personality assessment scales. J Pers Assess. 2003;81(2):93–103. doi: 10.1207/S15327752JPA8102_01. [DOI] [PubMed] [Google Scholar]
- 45.Dwivedi RC, et al. Validation of the Sydney Swallow Questionnaire (SSQ) in a cohort of head and neck cancer patients. Oral Oncol. 2010;46(4):e10–e14. doi: 10.1016/j.oraloncology.2010.02.004. [DOI] [PubMed] [Google Scholar]
- 46.McHorney CA, et al. The SWAL-QOL and SWAL-CARE outcomes tool for oropharyngeal dysphagia in adults: III. Documentation of reliability and validity Dysphagia. 2002;17(2):97–114. doi: 10.1007/s00455-001-0109-1. [DOI] [PubMed] [Google Scholar]
- 47.Silbergleit AK, et al. The Dysphagia Handicap Index: development and validation. Dysphagia. 2012;27(1):46–52. doi: 10.1007/s00455-011-9336-2. [DOI] [PubMed] [Google Scholar]
- 48.Woisard, V., M.P. Andrieux, and M. Puech, Validation of a self-assessment questionnaire for swallowing disorders (Deglutition Handicap Index). Revue de Laryngologie - Otologie - Rhinologie, 2006. 127(5). [PubMed]
- 49.Chen, A.Y., et al. The development and validation of a dysphagia-specific quality-of-life questionnaire for patients with head and neck cancer the M D Anderson Dysphagia Inventory. Arch Otolaryngol Head Neck Surg. 2001;127(7):870–6. [PubMed]
- 50.Simpelaere IS, et al. Validation to the Dutch version of the Swallowing Quality-of-Life Questionnaire (SWAL-QoL) and the adjusted DSWAL-QoL (aDSWAL-QoL) using item analysis with the Rasch model: a pilot study. Health Qual Life Outcomes. 2017 doi: 10.1186/s12955-017-0639-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.van der Maarel-Wierink CD, et al. Risk factors for aspiration pneumonia in frail older people: a systematic literature review. J Am Med Dir Assoc. 2011;12(5):344–354. doi: 10.1016/j.jamda.2010.12.099. [DOI] [PubMed] [Google Scholar]
- 52.van der Maarel-Wierink CD, et al. Meta-analysis of dysphagia and aspiration pneumonia in frail elders. J Dent Res. 2011;90(12):1398–1404. doi: 10.1177/0022034511422909. [DOI] [PubMed] [Google Scholar]
- 53.van der Maarel-Wierink CD, et al. Oral health care and aspiration pneumonia in frail older people: a systematic literature review. Gerodontology. 2013;30(1):3–9. doi: 10.1111/j.1741-2358.2012.00637.x. [DOI] [PubMed] [Google Scholar]
- 54.Noguchi S, et al. Using oral health assessment to predict aspiration pneumonia in older adults. Gerodontology. 2018;35:110–116. doi: 10.1111/ger.12324. [DOI] [PubMed] [Google Scholar]
- 55.Ortega O, Clavé P. Oral hygiene, aspiration, and aspiration pneumonia: from pathophysiology to therapeutic strategies. Current Physical Medicine and Rehabilitation Reports. 2013;1:292–295. [Google Scholar]
- 56.Seedat, J. and C. Penn, Implementing oral care to reduce aspiration pneumonia amongst patients with dysphagia in a South African setting. South African Journal of Communication Disorders, 2016. 63(1). [DOI] [PMC free article] [PubMed]
- 57.Mittal H, et al. Patient-reported outcome measures for adult dental patients: a systematic review. Journal of Evidence Based Dental Practice. 2019;19(1):53–70. doi: 10.1016/j.jebdp.2018.10.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 58.Slade GD, Spencer AJ. Development and evaluation of the Oral Health Impact Profile. Community Dent Health. 1994;11(1):3–11. [PubMed] [Google Scholar]
- 59.World Health Organization, Oral health surveys: basic methods. 2013, World Health Organization. p. 125.
- 60.Cho SY, et al. Prevalence and risk factors for dysphagia: a USA community study. Neurogastroenterol Motil. 2015;27(2):212–219. doi: 10.1111/nmo.12467. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61.Kendall KA. Airway closure delay: the predominant pathophysiology in reflux-associated dysphagia. Otolaryngology-Head and Neck Surgery. 2019;160(5):885–890. doi: 10.1177/0194599818824302. [DOI] [PubMed] [Google Scholar]
- 62.Bolier EA, et al. Systematic review: questionnaires for assessment of gastroesophageal reflux disease. Dis Esophagus. 2015;28(2):105–120. doi: 10.1111/dote.12163. [DOI] [PubMed] [Google Scholar]
- 63.Jones R, Coyne K, Wiklund I. The Gastro-oesophageal reflux disease Impact Scale: a patient management tool for primary care. Aliment Pharmacol Ther. 2007;25(12):1451–1459. doi: 10.1111/j.1365-2036.2007.03343.x. [DOI] [PubMed] [Google Scholar]
- 64.Shaw MJ, et al. Initial validation of a diagnostic questionnaire for gastroesophageal reflux disease. Am J Gastroenterol. 2001;96(1):52–57. doi: 10.1111/j.1572-0241.2001.03451.x. [DOI] [PubMed] [Google Scholar]
- 65.Vakil NB, et al. Systmatic review of patient-reported outcome instruments for gastroesophageal reflux disease symptoms. Eur J Gastroenterol Hepatol. 2013;25:2–14. doi: 10.1097/MEG.0b013e328358bf74. [DOI] [PubMed] [Google Scholar]
- 66.Murry T, Carrau RL. Clinical manual of swallowing disorders. Singular Thomson Learning: San Diego; 2001. [Google Scholar]
- 67.Miller, J.A., Sensory system: peripheral and central connections, in The neuroscientific principles of swallowing and dysphagia. 1999, Singular San Diego.
- 68.Farneti, D. and E. Genovese, Correlations between bedside and instrumental endoscopic parameters in determining severity of dysphagia: an integrated clinical evaluation of safety and efficiency. Acta Otorhinolaryngologica Italica, [In press]. [DOI] [PMC free article] [PubMed]
- 69.German, R.Z. and J.B. Palmer, Anatomy and development of oral cavity and pharynx. GI Motility online, 2006.
- 70.Morquette P, et al. Generation of the masticatory central pattern and its modulation by sensory feedback. Prog Neurobiol. 2012;96:340–355. doi: 10.1016/j.pneurobio.2012.01.011. [DOI] [PubMed] [Google Scholar]
- 71.Kahrilas JK, et al. Oropharyngeal accommodation to swallow volume. Gastroenterology. 1996;111:297–306. doi: 10.1053/gast.1996.v111.pm8690194. [DOI] [PubMed] [Google Scholar]
- 72.Tsumori N, et al. Morphologic characteristics of the superior pharyngeal constrictor muscle in relation to the function during swallowing. Dysphagia. 2007;22(2):122–129. doi: 10.1007/s00455-006-9063-2. [DOI] [PubMed] [Google Scholar]
- 73.Grillner S, et al. Mechanisms for selection of basic motor programs roles for the striatum and pallidum. Trends Neurosci. 2005;28(7):364–370. doi: 10.1016/j.tins.2005.05.004. [DOI] [PubMed] [Google Scholar]
- 74.Walshe M, Ryan M, Regan J. Screening for dysphagia: perspectives and practices of speech and language therapists in the Republic of Ireland. Folia Phoniatr Logop. 2017;69(5–6):226–238. doi: 10.1159/000486031. [DOI] [PubMed] [Google Scholar]
- 75.Farneti D, Turroni V, Genovese E. Aspiration: diagnostic contributions from bedside swallowing evaluation and endoscopy. Acta Otorhinolaryngol Ital. 2018;38:511–516. doi: 10.14639/0392-100X-1967. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 76.Virvidaki IE, et al. Swallowing and aspiration risk: a critical review of non instrumental bedside screening tests. Journal of Clinical Neurology. 2018;14(3):265–274. doi: 10.3988/jcn.2018.14.3.265. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 77.McCullough GH, et al. Utility of clinical/swallowing examination measures for detecting aspiration post-stroke. J Speech Lang Hear Res. 2005;48:1280–1293. doi: 10.1044/1092-4388(2005/089). [DOI] [PubMed] [Google Scholar]
- 78.Crary MA, Mann GD, Groher ME. Initial psychometric assessment of a functional oral intake scale for dysphagia in stroke patients. Arch Phys Med Rehabil. 2005;86(8):1516–1520. doi: 10.1016/j.apmr.2004.11.049. [DOI] [PubMed] [Google Scholar]
- 79.Folstein MF, Folstein SE, McHugh PR. “Mini-mental state.” A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12:189–198. doi: 10.1016/0022-3956(75)90026-6. [DOI] [PubMed] [Google Scholar]
- 80.Borson S, et al. The Mini-Cog: a cognitive 'vital signs' measure for dementia screening in multi-lingual elderly. Int J Geriatr Psychiatry. 2000;15:1021–1027. doi: 10.1002/1099-1166(200011)15:11<1021::aid-gps234>3.0.co;2-6. [DOI] [PubMed] [Google Scholar]
- 81.Mann G. The Mann Assessment of Swallowing Ability. New York: Singular/Thomson Learning; 2002. [Google Scholar]
- 82.Carnaby GD, Crary MA. Development and validation of a cancer-specific swallowing assessment tool: MASA-C. Support Care Cancer. 2014;22(3):595–602. doi: 10.1007/s00520-013-2011-4. [DOI] [PubMed] [Google Scholar]
- 83.Chalmers JM, et al. The Oral Health Assessment Tool — Validity and reliability. Aust Dent J. 2005;50(3):191–199. doi: 10.1111/j.1834-7819.2005.tb00360.x. [DOI] [PubMed] [Google Scholar]
- 84.Gillman A, Winkler R, Taylor NF. Implementing the free water protocol does not result in aspiration pneumonia in carefully selected patients with dysphagia: a systematic review. Dysphagia. 2017;32(3):345–361. doi: 10.1007/s00455-016-9761-3. [DOI] [PubMed] [Google Scholar]
- 85.Langmore SE. Why I like the free water protocol. Perspectives on Swallowing and Swallowing Disorders (Dysphagia) 2011;20(4):116–120. [Google Scholar]
- 86.Palmer PM, et al. Effects of a sour bolus on the intramuscular electromyographic (EMG) activity of muscles in the submental region. Dysphagia. 2005;20(3):210–217. doi: 10.1007/s00455-005-0017-x. [DOI] [PubMed] [Google Scholar]
- 87.Sdravou K, Walshe M, Dagdilelis L. Effects of carbonated liquids on oropharyngeal swallowing measures in people with neurogenic dysphagia. Dysphagia. 2012;27(2):240–250. doi: 10.1007/s00455-011-9359-8. [DOI] [PubMed] [Google Scholar]
- 88.Cola PC, et al. Taste and temperature in swallowing transit time after stroke. Cerebrovascular Diseases Extra. 2012;2(1):45–51. doi: 10.1159/000339888. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 89.Teismann IK, et al. Tactile thermal oral stimulation increases the cortical representation of swallowing. BMC Neurosci. 2009;10(1):71–71. doi: 10.1186/1471-2202-10-71. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 90.Huckabee ML, et al. The Test of Masticating and Swallowing Solids (TOMASS): reliability, validity and international normative data. Int J Lang Commun Disord. 2018;53(1):144–156. doi: 10.1111/1460-6984.12332. [DOI] [PubMed] [Google Scholar]
- 91.Lambert HC, et al. McGill Ingestive Skills Assessment (MISA): development and first field test of an evaluation of functional ingestive skills of elderly persons. Dysphagia. 2003;18(2):101–113. doi: 10.1007/s00455-002-0091-2. [DOI] [PubMed] [Google Scholar]
- 92.Sheppard JJ, Hochman R, Baer C. The Dysphagia Disorder Survey: validation of an assessment for swallowing and feeding function in developmental disability. Res Dev Disabil. 2014;35(5):929–942. doi: 10.1016/j.ridd.2014.02.017. [DOI] [PubMed] [Google Scholar]
- 93.Hansen T, Lambert HC, Faber J. Validation of the Danish version of the McGill Ingestive Skills Assessment using classical test theory and the Rasch model. Disabil Rehabil. 2012;34(10):859–868. doi: 10.3109/09638288.2011.624249. [DOI] [PubMed] [Google Scholar]
- 94.Speyer R, et al. Psychometric characteristics of non-instrumental swallowing and feeding assessments in pediatrics: a systematic review using COSMIN. Dysphagia. 2018;33(1):1–14. doi: 10.1007/s00455-017-9835-x. [DOI] [PubMed] [Google Scholar]
- 95.Britton D, et al. Utility of pulse oximetry to detect aspiration: an evidence-based systematic review. Dysphagia. 2018;33(3):282–292. doi: 10.1007/s00455-017-9868-1. [DOI] [PubMed] [Google Scholar]
- 96.Lagarde MLJ, Kamalski DMA, van den Engel-Hoek L. The reliability and validity of cervical auscultation in the diagnosis of dysphagia: a systematic review. Clin Rehabil. 2016;30(2):199–207. doi: 10.1177/0269215515576779. [DOI] [PubMed] [Google Scholar]
- 97.Bateman C, Leslie P, Drinnan MJ. Adult dysphagia assessment in the UK and Ireland: Are SLTs assessing the same factors? Dysphagia. 2007;22(3):174–186. doi: 10.1007/s00455-006-9070-3. [DOI] [PubMed] [Google Scholar]
- 98.Martino, R., G. Pron, and N. Diamant, Oropharyngeal dysphagia: surveying practice patterns of the speech language pathologist. Dysphagia, 2004. 19(3). [DOI] [PubMed]
- 99.Rumbach A, Coombes C, Doeltgen S. A survey of Australian dysphagia practice patterns. Dysphagia. 2018;33(2):216–226. doi: 10.1007/s00455-017-9849-4. [DOI] [PubMed] [Google Scholar]
- 100.Mokkink LB, et al. COSMIN risk of bias checklist for systematic reviews of patient-reported outcome measures. Qual Life Res. 2018;27(5):1171–1179. doi: 10.1007/s11136-017-1765-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 101.Prinsen CAC, et al. COSMIN guideline for systematic reviews of patient-reported outcome measures. Qual Life Res. 2018;27(5):1147–1157. doi: 10.1007/s11136-018-1798-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 102.Terwee CB, et al. COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study. Qual Life Res. 2018;27(5):1159–1170. doi: 10.1007/s11136-018-1829-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 103.Bergamaschi R, et al. The DYMUS questionnaire for the assessment of dysphagia in multiple sclerosis. J Neurol Sci. 2008;269(1–2):49–53. doi: 10.1016/j.jns.2007.12.021. [DOI] [PubMed] [Google Scholar]
- 104.Koller M, et al. The process of reconciliation: evaluation of guidelines for translating quality-of-life questionnaires. Expert Rev Pharmacoecon Outcomes Res. 2012;12(2):189–197. doi: 10.1586/erp.11.102. [DOI] [PubMed] [Google Scholar]
- 105.Speyer R, et al. The effects of telehealth by allied health professionals and nursing in rural and remote areas: A systematic review and meta-analysis. J Rehabil Med. 2018;50(3):225–235. doi: 10.2340/16501977-2297. [DOI] [PubMed] [Google Scholar]
- 106.Burns CL, et al. Implementation of speech pathology telepractice services for clinical swallowing assessment: an evaluation of service outcomes, costs and consumer satisfaction. J Telemed Telecare. 2019;25(9):545–551. doi: 10.1177/1357633X19873248. [DOI] [PubMed] [Google Scholar]
- 107.Boulkedid R, et al. Using and reporting the Delphi method for selecting healthcare quality indicators: A systematic review. PLoSONE. 2011;6:e20476. doi: 10.1371/journal.pone.0020476. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 108.Tapio P, et al. The unholy marriage? Integrating qualitative and quantitative information in Delphi processes. Technol Forecast Soc Chang. 2011;78:1616–1628. [Google Scholar]
- 109.Massey, B.T., Physiology or oral cavity, pharynx and upper esophageal sphincter. GI Motility online, 2006.
- 110.Damodaran O, et al. Cranial nerve assessment: a concise guide to clinical examination. Clin Anat. 2014;27:25–30. doi: 10.1002/ca.22336. [DOI] [PubMed] [Google Scholar]
- 111.Gray, H, et al. Gray's Anatomy: The anatomical basis of clinical practice. 39th ed. 2005, Edinburgh: Elsevier Churchill Livingstone. p. 1627.
- 112.Whiting PF, et al. QUADAS-2: A revised tool for the Quality Assessment of Diagnostic Accuracy Studies. Ann Intern Med. 2011;155(8):529–536. doi: 10.7326/0003-4819-155-8-201110180-00009. [DOI] [PubMed] [Google Scholar]