Inter-rater agreement of novel high-resolution impedance manometry metrics: bolus flow time and esophageal impedance integral ratio

Abstract

Background

Novel high-resolution impedance manometry (HRIM) metrics of bolus flow time (BFT) and esophageal impedance integral (EII) ratio have demonstrated clinical utility, though the reliability of their analysis has not been assessed. We aimed to evaluate the inter-rater agreement of the BFT and EII ratio.

Methods

HRIM studies including five upright, liquid swallows from 40 adult patients were analyzed by two raters using a customized MATLAB program to generate the BFT and EII ratio. Inter-rater agreement was assessed using the intraclass correlation coefficient (ICC) for median values generated per patient and also for all 200 swallows.

Key Results

The ICC (95% confidence interval, CI) for BFT was 0.873 (0.759 – 0.933) for median values and 0.838 (0.778 – 0.881) for all swallows. The ICC (95% CI) for EII ratio was 0.983 (0.968 – 0.991) for median values and 0.905 (0.875 – 0.928) for all swallows. Median values for both BFT and EII ratio were similar between the two raters (p-values 0.05).

Conclusions & Inferences

The BFT and EII ratio can be reliably calculated as supported by generally excellent inter-rater agreement. Thus, broader utilization of these measures appears feasible and would facilitate further evaluation of their clinical utility.

Abbreviated abstract

Excellent inter-rater agreement was demonstrated between two raters for novel high-resolution impedance manometry metrics including the bolus flow time (BFT) and esophageal impedance integral (EII) ratio. As both of these measures have demonstrated clinical utility through enhanced symptom association over other HRIM metrics, the reliable calculation of the BFT and EII ratio supports their broader application to the evaluation of esophageal disease.

Introduction

Esophageal manometry is the primary method to assess esophageal motility. When high-resolution manometry is combined with multichannel intraluminal esophageal impedance sensors (high-resolution impedance manometry, HRIM), the esophageal function evaluation can be enhanced by assessing the interplay between esophageal motility and bolus transit.¹ Novel HRIM paradigms have incorporated intraluminal impedance with esophageal pressure measures to derive esophageal pressure-flow metrics.^2–5 Among these, metrics that predict flow across the esophagogastric junction (EGJ), the bolus flow time (BFT), or residual bolus following a swallow, the esophageal impedance integral (EII) ratio were developed and validated among healthy controls utilizing simultaneous videofluoroscopy.^{6, 7} Both the BFT and EII ratio demonstrated promise with enhanced symptom-association over other manometric measures among patients with achalasia and also with non-obstructive dysphagia without a major esophageal motility disorder.^8–10 However, inter-rater reliability of the analysis to generate the BFT and EII ratio has not been assessed and thus the aim of this study was to evaluate the inter-rater agreement of the BFT and EII ratio among a cohort of patients evaluated for dysphagia.

Methods

Subjects

Patients that completed HRIM for an indication of dysphagia were retrospectively identified for inclusion from the esophageal manometry registry of the Esophageal Center of Northwestern Medicine. A total of 10 patients were selected from four groups based on Chicago Classification diagnoses to generate a total study population of 40 unique patients: 1) normal motility, 2) EGJ outflow obstruction, 3) Type II achalasia, and 4) Type II achalasia (pre-treatment diagnosis) evaluated following intervention with pneumatic dilation (n = 3), laparoscopic myotomy (n = 5), or per-oral endoscopic myotomy (n = 2).¹¹ Patients were excluded if a hiatal hernia ≥ 3 cm was present or for previous foregut surgery among groups 1, 2, and 3.

HRIM protocol and analysis

Manometry studies were completed using a 4.2-mm outer diameter solid-state assembly with 36 circumferential pressure sensors at 1-cm intervals and 18 impedance segments at 2-cm intervals (Medtronic Inc, Shoreview, MN). The HRIM protocol included a 5-minute baseline recording, 10 5-ml swallows in a supine position, and five 5-ml swallows in the upright position using 0.45% saline for test swallows at 20–30 second intervals. The 10 supine swallows were utilized to designate an esophageal motility diagnosis via the Chicago Classification v3.0, (but were not subjected to further analysis for inter-rater agreement).¹¹ The 5 upright swallows (as previous findings demonstrated better symptom-association with upright than supine swallows) were subjected to analysis for inter-rater agreement assessment by two raters blinded to patient-clinical details (ZL and WK).¹⁰ Both raters were experienced in HRM analysis and proficient with MATLAB^™ (The MathWorks Inc., Natick, MA, U.S.A.). One rater is the primary designer of the customized MATLAB program and has exclusively performed the previously reported analysis on BFT and EII ratio.^6–10 The other rater was newly introduced to the BFT and EII ratio analysis paradigm and underwent a one-on-one tutorial lasting approximately 60 minutes was provided for instruction of the MATLAB program and analysis paradigms. Additionally, supervised analysis of 40 patient-swallows was performed by the new rater prior to initiating the study protocol of independent analysis.

The integrated relaxation pressure (IRP) was assessed using Manoview v3.0.1. The HRIM data for each subject were exported to MATLAB^™ to apply to a customized program for analysis of BFT and EII ratio as previously described; Figure 1.^{9, 10}

Figure 1. The bolus flow time and esophageal impedance integral (EII) ratio.

A) The EII ratio was calculated by dividing the measurement region of interest (red-dashed box), which entailed the swallow to the completion of peristalsis or 12 seconds if peristalsis was absent, into two impedance domains: swallow (Z1) and post-swallow (Z2). The amount of bolus present (the EII) within each domain was quantified by measuring the impedance-pixel density (impedance value x time x axial length). The EII ratio was then calculated as the ratio of residual bolus volume (EII-Z2) relative to the intra-esophageal bolus volume immediately following the swallow, but before the deglutitive contraction (EII-Z1): EII ratio = (EII-Z2)/(EII-Z1). The area within the black-dashed box is enlarged in the top panel of B. B) The BFT was measured by positioning three impedance and three manometry signals through the EGJ at 1-cm intervals with the distal signals positioned within the hiatus as identified by crural contractions, top panel. The bolus presence time (BPT) at the EGJ was determined as the time from which the impedance dropped to 90% of the nadir until the impedance returned to 50% of the impedance baseline (middle panel). The BFT was then derived as the sum of all periods during the BPT when a flow-permissive pressure gradient (i.e. when the esophageal pressure was greater than both the crural and intra-gastric pressure signals) was present (bottom panel). If the impedance drop was not greater than 50% at each axial location and/or a flow-permissive pressure gradient was not achieved, the BFT was considered to be zero. Figure used with permission from the Esophageal Center at Northwestern. CD – crural diaphragm.

Statistical analysis

Inter-rater agreement was assessed using the intraclass correlation coefficient (ICC; two-way, mixed model for absolute agreement) and interpreted as 0–0.2: poor agreement; 0.3–0.4: fair agreement; 0.5–0.6: moderate agreement; 0.7–0.8: strong agreement; and >0.8: almost perfect agreement. The ICC was applied to both the median values generated from the five upright swallows for each patient (“median values”) and to all 200 analyzed swallows (“all swallows”), acknowledging that the statistical assumption of independence was not met when applied to all swallows. Median values were also compared between raters using the Wilcoxon-signed ranks test, which assumed a 5% level of statistical significance.

Results

Subjects

Forty patients, ages 20–85 years (mean +/− standard deviation 56 +/− 18 years), 21 (53%) female, were included. The treated achalasia group was evaluated at a median (inter-quartile range) 13 (8 – 27) months following treatment. Motility patterns among the post-treatment achalasia group included absent contractility (2), type II achalasia (3), type III achalasia (3), and ineffective esophageal motility (2).

Inter-rater agreement

Median (per-patient) values generated by one rater were median (inter-quartile range) 16 (7–22) mmHg for IRP, 1.3 (0 – 3) seconds for bolus presence time, 0.2 (0–1.1) seconds for BFT, and 0.33 (0 – 0.52) for EII ratio and were similar (i.e. all p-values > 0.05) than those generated by the other rater: 15 (8–22) mmHg for IRP, 1.6 (0.1 – 3.2) seconds for BPT, 0.7 (0–1.6) seconds for BFT, and 0.30 (0.05 – 0.51) for EII ratio.

Intraclass correlation coefficients demonstrated strong to almost perfect agreement of each metric whether evaluating median values (n = 40) or all upright swallows (n = 200); Table 1. Evaluation of ICCs when analyzing all swallows among each patient group (n = 50 swallows per group) demonstrated moderate to almost perfect agreement for each metric; Table 2.

Table 1. Intraclass correlation coefficients (ICC).

Values represent ICC (95% confidence interval). IRP – integrated relaxation pressure. BPT – bolus presence time. BFT – bolus flow time. EII – esophageal impedance integral.

A. All
All patients	Median values (n = 40)	All upright swallows (n = 200)
IRP	0.993 (0.987 – 0.996)	0.993 (0.990 – 0.994)
BPT	0.959 (0.919 – 0.979)	0.915 (0.887 – 0.936)
BFT	0.873 (0.759 – 0.933)	0.838 (0.778 – 0.881)
EII ratio	0.983 (0.968–0.991)	0.905 (0.875 – 0.928)

Table 2. Intraclass correlation coefficients (ICC) by patient sub-types.

Values represent ICC (95% confidence interval) for all 50 swallows among each patient group (i.e. 5 upright swallows x 10 patients per group). IRP – integrated relaxation pressure. BPT – bolus presence time. BFT – bolus flow time. EII – esophageal impedance integral.

	Normal motility	EGJ outflow obstruction	Type II achalasia -Treatment naïve	Type II achalasia - treated
IRP	0.985 (0.974 – 0.992)	0.994 (0.990 – 0.997)	0.993 (0.972 – 0.997)	0.982 (0.969 – 0.990)
BPT	0.828 (0.697 – 0.902)	0.814 (0.653 – 0.898)	0.945 (0.904 – 0.969)	0.891 (0.808 – 0.938)
BFT	0.718 (0.507 – 0.839)	0.731 (0.526 – 0.847)	0.505 (0.147 – 0.715)	0.714 (0.483 – 0.840)
EII ratio	0.600 (0.299 – 0.773)	0.965 (0.939 – 0.980)	0.967 (0.934 – 0.982)	0.971 (0.904 – 0.968)

Discussion

We evaluated the inter-rater agreement of novel HRIM metrics, the BFT and EII ratio, between two raters among patients evaluated for dysphagia and demonstrated strong to almost perfect agreement between the raters.

The HRIM metrics of pressure-flow metrics generated via the automated impedance manometry (AIM) analysis have demonstrated potential benefits by detecting abnormalities in patients with non-obstructive and post-fundoplication dysphagia.^3–5 A previous study evaluating inter-rater agreement of the AIM analysis pressure-flow metrics between five raters of variable analytic experience demonstrated almost perfect agreement when applied to 50 patients evaluated for gastroesophageal reflux.¹² Among the 200 upright swallows analyzed, mean ICCs of ≥0.9 for each pressure-flow metric, as well as the IRP, were reported. Although among fewer raters, the current study demonstrated similar excellent measures of inter-rater agreement between an experienced and recent trained rater with the BFT and EII ratio.

Although the sub-group analysis by patient group was not our primary objective and was thus limited by sample sizes, it demonstrated slightly lower degrees of agreement (although still moderate) with EII ratio among normal motility and BFT among treatment-naïve achalasia. Re-evaluation of discordant cases suggested disagreement with EII ratio was often related to the determination of the end of peristalsis. With BFT, disagreement was often related to the designation of the EGJ and crural sensors, particularly when esophageal shortening (and thus movement of the LES relative to the HRIM catheter) occurred. This could be addressed by a program upgrade to account for LES movement, such as methods similar to an LES pressure sleeve.

While this study demonstrated that BFT and EII ratio analysis could be performed reliably among multiple raters, the generalizability of this study remains limited by the restricted availability of the required customized MATLAB program. Future incorporation of these metrics into HRIM-analytic software could offer broader utilization. Additionally, although the study could be limited by the relatively small patient cohorts, we attempted to select patient-groups that encompassed clinically-important groups that exhibit diverse peristaltic and EGJ properties. However, further evaluation of BFT and EII ratio analysis among more heterogeneous esophageal motility cohorts remains needed.

In conclusion, the BFT and EII ratio were able to be generated with generally excellent inter-rater agreement. Given the previously reported potential for clinical utility of these measures, the demonstration of reliable calculation of the BFT and EII ratio suggests they could be more broadly applied and further the evaluation of their clinical utility.

Key Points.

• Novel high-resolution impedance manometry (HRIM) metrics, the bolus flow time (BFT) and esophageal impedance integral (EII) ratio have demonstrated potential clinical utility through enhanced symptom association over other HRIM metrics.

• Almost perfect inter-rater agreement was demonstrated between two raters for both the BFT and EII ratio.

• Reliable calculation of the BFT and EII ratio supports broader application of these metrics to further assess their role in clinical evaluation of esophageal disease.

Abbreviations

AIM

automated impedance manometry

BPT

bolus presence time

BFT

bolus flow time

CD

crural diaphragm

EGJ

esophagogastric junction

EII

esophageal impedance integral

HRIM

high-resolution impedance manometry

ICC

intraclass correlation coefficient

IRP

integrated relaxation pressure