Dynamic Imaging Grade of Swallowing Toxicity (DIGEST): Scale Development and Validation

Abstract

Purpose

NCI’s Common Terminology Criteria for Adverse Events (CTCAE) is the universal framework for toxicity reporting in oncology trials. We sought to develop a CTCAE-compatible modified barium swallow (MBS) grade for the purpose of grading pharyngeal dysphagia as a toxicity endpoint in cooperative group organ preservation trials for head and neck cancer (HNC). We hypothesized that a 5-point CTCAE-compatible MBS grade (“DIGEST”) based on the interaction of pharyngeal residue and laryngeal penetration/aspiration ratings is feasible and psychometrically sound.

Methods

A modified Delphi exercise was conducted for content validation, expert consensus, and operationalization of DIGEST criteria. Two blinded raters scored 100 MBS conducted before or after surgical or non-surgical organ preservation. Intra- and inter-rater reliability were tested by weighted Kappa. Criterion validity against OPSE, MBSImP™©, MDADI, and PSS-HN was assessed with one-way ANOVA and post hoc pairwise comparisons between DIGEST grades.

Results

Intra-rater reliability was excellent (weighted Kappa=0.82–0.84) with substantial to almost perfect agreement between raters (weighted Kappa=0.67–0.81). DIGEST significantly discriminated levels of pharyngeal pathophysiology (MBSImP™©: r=0.77, p<0.0001), swallow efficiency (OPSE: r=−0.56, p<0.0001), perceived dysphagia (MDADI: r=−0.41, p<0.0001), and oral intake (PSS-HN diet: r=−0.49, p<0.0001).

Conclusions

With the development of DIGEST, we have adapted MBS rating to the CTCAE nomenclature of ordinal toxicity grading used in oncology trials. DIGEST offers a psychometrically sound measure for HNC clinical trials and investigations of toxicity profiles, dose-response, and predictive modeling.

INTRODUCTION

Dysphagia, a dose-limiting toxicity of head and neck radiotherapy, is the primary functional endpoint of many national and international organ preservation trials (e.g., ECOG3311, PATHOS, ORATOR, HN002, DARS)^1–5 for locoregionally advanced stage head and neck cancer (HNC). Refinements in minimally invasive surgical techniques and conformal methods of radiation delivery hold promise to mitigate the burden of dysphagia. Yet, our ability to quantify relative functional advantages of novel treatments in clinical trials is fully contingent on the measurement paradigm. Measurement of dysphagia is complex. It is generally agreed that multiparametric panels of dysphagia measures should comprise patient-reported outcomes and clinician-reported indices of swallowing function. Among clinician-graded methods, videofluoroscopy (also known as a modified barium swallow study [MBS]) is widely regarded as the gold-standard for examination of oropharyngeal swallowing function and has been described in more than 140 HNC reports indexed in PubMed since 1982. Diverse and often unvalidated MBS endpoints are reported in these trials, and only one National Cancer Institute (NCI) multi-site cooperative group HNC trial (E2399) used the MBS to assess swallow outcomes. As MBS is now increasingly adopted as an endpoint measure in contemporary NCI network and international HNC trials, a robust yet streamlined MBS measure that aligns with toxicity reporting structure is paramount to standardize efforts.

The NCI’s Common Terminology Criteria for Adverse Events (CTCAE [Table 1]) serves as the universal framework for toxicity reporting in oncology trials. CTCAE offers a standardized language and criteria for clinician-rated toxicity grades. CTCAE grades dysphagia as a function of dietary restrictions, dysphagia symptoms, and enteral/parental nutrition requirements but does not assess swallowing according to MBS parameters. These broad criteria (symptoms, diet, and feeding tube) of the extant clinical CTCAE dysphagia grade are not fully sensitive to physiologic swallowing impairments, as it is well-documented that patients often elect to eat and refuse gastrostomy in the setting of clinically significant dysphagia and aspiration on MBS.^6,7 Similarly, there are patients who rely on tube feedings for reasons other than dysphagia such as mucosal toxicity, salivary dysfunction, and food aversion from dysguesia. We sought to develop and psychometrically validate a CTCAE-compatible MBS rating for the purpose of grading pharyngeal dysphagia as a toxicity endpoint in cooperative group organ preservation trials for HNC. Because safety and efficiency are widely regarded as primary constructs of interest in swallow evaluations on MBS, we hypothesized that a 5-point CTCAE-compatible MBS grade (Dynamic Imaging Grade of Swallowing Toxicity or “DIGEST”) based on the interaction of pharyngeal residue and laryngeal penetration/aspiration ratings is feasible and psychometrically sound.

Table 1.

Common Terminology Criteria for Adverse Events v.4.03 Dysphagia Item

CTCAE v.4.0332
Dysphagia: A disorder characterized by difficulty in swallowing.
1 Symptomatic, able to eat regular diet
2 Symptomatic and altered eating/swallowing
3 Severely altered eating/swallowing; tube feeding or TPN or hospitalization indicated
4 Life-threatening consequences; urgent intervention indicated
5 Death

METHODS

Scale development

We first developed a conceptual framework for DIGEST (Figure 1) to establish the key determinants of pharyngeal phase dysphagia contributing to health compromise. A priori objectives of DIGEST included those listed in Table 2. We convened a panel of nine expert clinician researchers with a minimum of 10 years in specialized oncology clinical practice. The panel included six clinician scientists who serve as Principal Investigators in HNC outcomes studies using MBS endpoints and three senior clinicians (Board Certified Specialists in Swallowing and Swallowing Disorders) with HNC dedicated practices who participate in MBS data collection for head and neck trials. For content validation, an 11-item survey was administered to each panelist to assess construct relevance. A modified Delphi exercise was then conducted over three sessions (10 hours) for expert consensus and to operationalize DIGEST criteria against CTCAE benchmarks.

Figure 1. Conceptual model for DIGEST Scale Development.

Abbreviation: DIGEST, Dynamic Imaging Grade of Swallowing Toxicity

Table 2.

Objectives of DIGEST Scale Development

1. Grade pharyngeal stage of swallowing compatible with NCI CTCAE ordinal toxicity grading framework for oncology trials to ease interpretation among interdisciplinary researchers

2. Incorporate components of established MBS measures to encourage and ease adoption by speech pathologists

3. Leverage recent efforts to standardize MBS procedures

4. Limit DIGEST to two constructs of pharyngeal dysphagia to streamline measurement

5. Provide single summary grade for pharyngeal stage of swallow function

DIGEST

DIGEST utilizes two component scores to quantify pharyngeal bolus transit: 1) safety profile and 2) efficiency profile. A priori, the Penetration-Aspiration Scale (PAS) and an estimation of the percentage of pharyngeal residue were selected as the primary measures of safety and efficiency, respectively. Modifiers were developed during the consensus exercise to operationalize these measures to the CTCAE framework. Safety and efficiency profile criteria of DIGEST are illustrated in Figure 2. The summary DIGEST rating aligns with NCI’s framework for toxicity reporting in oncology trials and assigns a global rating of pharyngeal swallowing function according to the interaction of the safety and efficiency profile scores (0=no pharyngeal dysphagia, 1=mild, 2=moderate, 3=severe, 4=life threatening).

Figure 2. Bolus scoring criteria for DIGEST.

Footnote: Grade 1=mild, Grade 2=moderate, Grade 3=severe, Grade 4=life threatening

Abbreviation: DIGEST, Dynamic Imaging Grade of Swallowing Toxicity

Psychometric analysis

Records of MBS conducted at MD Anderson Cancer Center between 2005 and 2013 were queried in a computerized departmental MBS database. Patients with history of organ preservation for laryngeal and/or pharyngeal carcinoma were eligible for inclusion. Swallow studies in patients with history of recurrent or second primary malignancy of the head and neck at the time of MBS were excluded as well as those treated with “open” transcervical or transmandibular HNC surgery. One hundred MBS were then randomly selected from eligible cases. The standard MBS protocol included: 2 trials each of 5-mL, 10-mL, and self-administered cup sip volumes of thin liquid barium (Varibar®, Bracco Diagnostics Inc., Monroe Township, New Jersey, USA), barium pudding (Varibar®, Bracco Diagnostics Inc., Monroe Township, New Jersey, USA), and cracker coated in barium paste. To ensure that MBS were sufficiently diverse to test the DIGEST scale’s discrimination, eligible cases were proportionally sampled to only comprise 10% pre-treatment swallow studies, which often reveal relatively normal swallow function. Additionally, MBS were proportionally sampled 2:1 for abnormal PAS scores, defined as a score greater than or equal to three. Two blinded raters scored the 100 MBS conducted before or after surgical or non-surgical organ preservation. Thirty-two studies were re-sampled randomly and re-rated to assess intra-rater reliability.

Statistical analysis

Weighted Kappa was used to assess intra- and inter-rater reliability. Criterion validity against Oropharyngeal Swallow Efficiency (OPSE),⁸ Modified Barium Swallow Impairment Profile (MBSImP™©),⁹ M.D. Anderson Dysphagia Inventory (MDADI),¹⁰ and Performance Status Scale for Head and Neck Cancer Patients (PSS-HN)¹¹ was assessed with one-way ANOVA and post hoc pairwise comparisons between DIGEST grades using the CONTRAST statement in PROC GENMOD procedure with a Wald chi-square statistic option. Weighted Kappa was used to assess agreement between CTCAE grades assigned by the expert panel and a lab rater’s post hoc analysis of DIGEST scores. SAS version 9.2 (SAS Institute Inc., Cary, NC) and S-Plus version 8.04 (TIBCO Software Inc., September 3, 2008) were used to carry out the computations for all analyses.

RESULTS

Patient characteristics

Table 3 displays demographic information for the 100 patients included in the sample. The mean patient age at time of MBS was 61 (range: 47 to 84), and 82% of the patients were male. Tumor subsites followed expected distributions, with more than half of MBS sampled from oropharyngeal cancer patients. The majority (72%) were treated with chemoradiation (induction, concurrent, or sequential); the remainder received radiation alone, conservation surgery, or conservation surgery and adjuvant radiation or chemoradiation.

Table 3.

Patient Characteristics (n = 100)

	No. of Patients (%)
Sex
Male	82 (82)
Female	18 (18)
Age (mean)	61
Primary Site
Hypopharynx	7 (7)
Larynx	24 (24)
Oropharynx	59 (59)
Nasopharynx	6 (6)
Unknown Primary	4 (4)
T Classification
0	4 (4)
1	16 (16)
2	43 (43)
3	25 (25)
4	12 (12)
N Classification
0	29 (29)
1–2a	14 (14)
2b	23 (23)
2c	27 (27)
3	7 (7)
Treatment Combination
RT + chemo*	72 (72)
RT alone	4 (4)
Surgery + adjuvant RT**	14 (14)
Surgery***	10 (10)

^*Includes induction, concurrent, and sequential chemotherapy

^**Includes post-operative radiation and chemoradiation

^***Includes induction chemotherapy in one patient

Content validity

The expert panel unanimously agreed or strongly agreed on survey that the severity of pharyngeal phase dysphagia can be graded according to the safety and efficiency of bolus transport on MBS and that safety and efficiency are the primary attributes describing bolus transport in the pharyngeal phase of swallowing. Likewise, there was unanimous agreement that pharyngeal residue is a surrogate measure for the efficiency of bolus transport through the pharynx. The overwhelming majority reported agreement to strong agreement that the degree of airway penetration/aspiration measures the safety of bolus transport; one panelist reported disagreement on this item. All panelists reported that PAS and ordinal residue grades are relevant or very relevant measures of pharyngeal phase swallowing safety and efficiency, respectively.

Reliability

Intra-rater reliability was excellent in both raters (weighted Kappa=0.82–0.84). Agreement on efficiency between the two raters was almost perfect (weighted Kappa = 0.81) and substantial on safety (weighted Kappa = 0.67). Inter-rater agreement on the summary DIGEST grade was substantial (weighted Kappa = 0.67).

Criteria validity

DIGEST significantly discriminated levels of pharyngeal pathophysiology (MBSImP™©: r=0.77, p<0.0001) and swallow efficiency (OPSE: r=−0.56, p<0.0001), as depicted in Figure 3a and 3b. Perceived dysphagia (MDADI; r=−0.41, p<0.0001) and oral intake (PSS-HN diet: r=−0.49, p<0.0001) were significantly negatively correlated with DIGEST (Figure 3c and 3d).

Figure 3. DIGEST by modified barium swallow study (A, B) and patient-reported outcome (C, D) criterion measures.

DIGEST significantly discriminated levels of pharyngeal pathophysiology (MBSImP™©: r=0.77, p<0.0001), swallow efficiency (OPSE: r=−0.56, p<0.0001), perceived dysphagia (MDADI: r=−0.41, p<0.0001), and oral intake (PSS-HN diet: r=−0.49, p<0.0001).

Abbreviations: DIGEST, Dynamic Imaging Grade of Swallowing Toxicity, MBSImP™©, Modified Barium Swallow Impairment Profile, OPSE, Oropharyngeal Swallow Efficiency, MDADI, MD Anderson Dysphagia Inventory, PSSHN, Performance Status Scale-Head and Neck Cancer

Construct validity

Agreement between the CTCAE grades assigned to MBS by panelists and post hoc DIGEST scores assigned by lab raters was substantial (weighted Kappa = 0.78), suggesting that bolus anchored MBS criteria assigned by DIGEST reflect the toxicity framework of CTCAE (Figure 4).

Figure 4. Construct validity agreement.

Agreement between the global CTCAE grades assigned to MBS by panelists and post hoc DIGEST scores assigned by lab raters was substantial (weighted Kappa = 0.78).

Abbreviations: CTCAE, Common Terminology Criteria for Adverse Events, DIGEST, Dynamic Imaging Grade of Swallowing Toxicity

DISCUSSION

Swallowing is a top functional priority ranked by patients before and after HNC treatment^12,13 and is an independent driver of quality of life in survivorship.¹⁴ Pharyngeal dysphagia also significantly contributes to major secondary morbidity in survivorship, including pneumonia and malnutrition¹⁵. For these reasons, dysphagia is now considered a major functional endpoint in many cooperative group trials of both surgical and non-surgical organ preservation strategies for locoregionally advanced stage HNC.^1,4 Despite its adoption as a routine clinical procedure, there remains no consensus on an optimal MBS parameter to use as an endpoint measure in these trials. Herein, we describe the development and validation of a novel and simple CTCAE-compatible MBS-graded measure of pharyngeal stage dysphagia – DIGEST. Adapting commonplace videofluoroscopic measurement parameters to the CTCAE nomenclature of ordinal toxicity grading in oncology trials, DIGEST offers a shared interdisciplinary language for MBS-anchored clinician-grading of pharyngeal dysphagia between investigators running oncology trials and speech pathologists leading the MBS examinations.

Tremendous progress has been made to standardize the MBS examination since its introduction and adoption into clinical practice over three decades ago.^9,16–18 As with any imaging measure, DIGEST ratings can only be reliably assigned with adherence to a standard MBS protocol. Critical elements of standardization include, among many, the contrast agent, the bolus protocol, frame rate of image acquisition, and patient instruction. A uniform bolus protocol must be efficient to minimize radiation exposure (per ALARA principle) yet feature a sufficient range of consistencies to assess swallow capacity. Martin-Harris et al. developed and statistically assessed a consensus-derived, optimal bolus protocol consisting of measured-volume and natural “cup sip” presentations of thin liquid barium (Varibar®, Bracco Diagnostics Inc, Monroe Township, NJ, USA), pudding thick, and a dry solid bolus.⁹ DIGEST was accordingly developed to align with this type of bolus protocol using standardized contrast agents (Varibar®, Bracco Diagnostics Inc, Monroe Township, NJ, USA). The effect of testing additional bolus types on the psychometric properties of DIGEST has not been evaluated and should be considered by investigators or clinics that use alternate bolus protocols in their practice. Speech Pathology researchers outside of the United States who do not have access to standardized contrast agents developed for videofluoroscopy have worked in partnership to ensure a standard level of barium concentration of videofluoroscopy agents employed within large multi-center trials that plan to incorporate MBS analysis such as DIGEST.¹⁹

DIGEST is a bolus-anchored functional outcome measure designed to reflect NCI’s CTCAE framework for grading toxicities of cancer therapy. Because of our highly focused goal to develop a simple MBS-derived toxicity grade, we opted to measure two bolus-anchored constructs of pharyngeal bolus transit: 1) swallowing safety (i.e., penetration/aspiration) and 2) swallowing efficiency (i.e., residue). Our validation results suggest that bolus-anchored measures accurately reflect the degree of pharyngeal swallowing dysfunction as measured by physiologic and temporal MBS indices. DIGEST is not, however, intended to replace more elegant, validated measures of biomechanical, kinematic, physiologic and temporal parameters of the swallow that are critical to characterize patterns of dysfunction and pathophysiology of dysphagia. Likewise, DIGEST was developed with the intention of grading the pharyngeal stage of swallowing, the phase most commonly impacted by organ preservation regimens. The validation sample included patients treated with transoral methods of surgical organ preservation (transoral laser microsurgery and transoral robotic surgery) and non-surgical organ preservation regimens (radiation alone and chemoradiotherapy). Thus, DIGEST provides an MBS grading platform for cancers of the larynx, pharynx, and unknown primary HNCs commonly triaged to organ preservation modalities but is not likely a representative measure of global swallow function for patients with significant degrees of oral or esophageal dysfunction.

DIGEST was developed with the intent of mapping existing, widely adopted measures of bolus flow to the CTCAE framework of toxicity grading. Component measures that required no special analysis equipment or spatiotemporal imaging calculations were ideal for DIGEST to assign rank to both swallowing safety/airway protection and efficiency function. First, the popular PAS score was selected as the candidate measure of safety/airway protection.²⁰ PAS modifiers were applied on consensus of the expert panel to operationalize the PAS for the purpose of summary grading in DIGEST, specifically accounting for the amount and frequency or pattern of aspiration events not quantified in the existing PAS criteria of airway entry, depth of invasion, and clearance.

Pharyngeal residue was selected as the candidate parameter to represent swallowing efficiency. Pharyngeal residue can be quantified in many ways. A variety of residue metrics have been proposed and studied, including indices of residue location,^21,22 ordinal ratings of residue,^9,23,24 and estimations of percent residue²³ in discrete pharyngeal spaces (i.e., valleculae or piriform sinuses)²⁵ or the entire oropharyngeal tract.⁸ Ordinal residue measures were attractive for DIGEST as they are inherently compatible with the ordinal CTCAE scaling. Ordinal grades of pharyngeal residue are reliably measured, typically using an estimation of percent residue to define upper and lower limits of each grade. A cut-point of roughly 50% residue is widely suggested to represent significant impairment in bolus clearance, defining the upper ranges of severity on the popular and rigorously validated MBSImP™©.^9,25,26 Lower limits of residue are defined qualitatively in some scales as “coating”⁹ and more quantitatively estimated by others as <10%^25,26 or <25%.²⁷ Collating the many options to ordinally grade residue, considering their psychometrics, relative advantages and vetting these through the expert panel, the ordinal grading selected for DIGEST was: <10%, 10–49%, 50–90%, and >90% with modifiers to assign a pattern of residue across bolus types. The normalized residue ratio scale (NRRS) was recently validated as a more precise method to quantify percent residue using post-processing image segmentation in ImageJ to calculate residue in discrete regions of the pharynx (valleculae and piriform sinuses).²⁸ NRRS is a valuable paradigm in MBS measurement but was not incorporated into DIGEST due to the measurement specifications and validation of the NRRS as a single-swallow measure.

The summary DIGEST grade is an ideal MBS-derived endpoint measure for oncology trials (i.e., grade 1= mild, grade 2= moderate, grade 3= severe, and grade 4= life threatening pharyngeal dysphagia) that reflects the combined impairment in swallowing safety and efficiency. Component safety and efficiency scores, however, might be valuable in clinical reporting to describe a profile of the pharyngeal swallow analogous to the TNM classification of tumors or GRBAS classification of voice profiles.²⁹ For instance, a DIGEST profile of S0 E1 D1 reflects safe (S0) but mildly inefficient (E1) pharyngeal bolus transit, representing an overall mild pharyngeal dysphagia (D1). Whereas, a DIGEST profile of S3 E1 D3 reflects a swallow with severe safety compromise (S3) and mild inefficiency (E1) of bolus transit, equating to an overall severe pharyngeal dysphagia. DIGEST profiles might also be mapped to dysphagia pathophysiology and appropriate therapies to help to refine or prioritize treatment algorithms for distinct subtypes of dysphagia.

In this work, we deliver a reliable, validated ordinal MBS grade of pharyngeal stage dysphagia. Compatible with NCI’s CTCAE toxicity grading system, DIGEST offers a streamlined MBS measure that is sensitive to the needs of multi-site oncology trials. DIGEST was developed as a functional outcome measure. As such, DIGEST provides a clinician-rated measure of the functionality of uncompensated pharyngeal bolus transit on videofluoroscopy. MBS penetration/aspiration and pharyngeal residue profiles have been mapped to the CTCAE framework using clearly defined and scaled parameters. For ease of clinical interpretation, these grades are summarized qualitatively in Table 4. To use DIGEST as a clinical decision making tool, the clinician must also consider a host of concomitant factors including patients’ respiratory status, general wellness, mental functioning, and compensatory abilities. As such, while we are hopeful that DIGEST offers an ideal tool for risk stratification, DIGEST alone cannot be used to render clinical decisions about oral intake and dysphagia therapy. Finally, in research trials, our preferred approach is to pair DIGEST with complementary functional measures of oral intake (e.g., PSS-HN or Functional Oral Intake Scale [FOIS])³⁰ and patient-reported outcome questionnaires (e.g., MDADI or EAT-10)³¹ as the complex relationships of clinician-graded and patient-reported swallowing outcome measures has been robustly described.

Table 4.

DIGEST Safety and Efficiency Profiles

DIGEST	CTCAE criteria	Safety grade*	Efficiency grade*	Safety		Efficiency
0	No toxicity	S0	E0	Safe	&	Efficient
1	Mild	S0	E1–E2	Safe	&	Mild to moderate inefficiency
		S1	E0–1	Mildly unsafe	&	Efficient or mildly inefficient
2	Moderate	S0	E3	Safe	&	Severely inefficient
		S1	E2–E3	Mildly unsafe	&	moderate to severe inefficiency
		S2	E0–2	Moderately unsafe	&	Efficient or mild-moderate inefficiency
3	Severe	S0–S1	E4	Safe or Mildly unsafe	&	Profoundly inefficient
		S2	E3–E4	Moderately unsafe		Severe to profound inefficiency
		S3	E0–E3	Severely unsafe	&	Efficient or Mild to severe inefficiency
		S4	E0–E2	Profoundly unsafe		Efficient or Mild to moderate inefficiency
4	Life-threatening	S3	E4	Severely unsafe	&	Profoundly inefficient
		S4	E3–E4	Profoundly unsafe	&	Severely to profoundly inefficient

^*Criteria for safety and efficiency grades are detailed in Figure 2

CONCLUSIONS

With the development of DIGEST, we have adapted MBS rating to the CTCAE nomenclature of ordinal toxicity grading used in oncology trials. DIGEST offers a psychometrically sound measure for HNC clinical trials and investigations of toxicity profiles, dose-response, and predictive modeling.