The Development of Quality Measures for the Performance and Interpretation of Esophageal Manometry

Abstract

Background and Aims

Esophageal manometry (EM) is the gold standard for the diagnosis of esophageal motility disorders. Variations in the performance and interpretation of EM result in discrepant diagnoses and unnecessary repeated procedures, and may negatively impact patient outcomes. A method to benchmark the procedural quality of EM is needed. The primary aim of this study was to develop quality measures for performing and interpreting EM.

Methods

The RAND/University of California, Los Angeles Appropriateness Methodology (RAM) was utilized. Fifteen experts in esophageal manometry were invited to be a part of the panel. Potential quality measures were identified through a literature search and interviews with experts. The expert panel ranked the proposed quality measures for appropriateness via a two-round process on the basis of RAM.

Results

Fourteen experts participated in all processes. A total of 29 measures were considered; 17 of these measures were ranked as appropriate and related to competency (2), pre-procedure (2), procedure (3) and interpretation (10). The latter 10 were integrated into a single composite measure. Thus, 8 final measures were determined to be appropriate quality measures for EM. Five strong recommendations were also endorsed by the experts, however they were not ranked as appropriate quality measures.

Conclusions

Eight formally validated quality measures for the performance and interpretation of EM were developed on the basis of RAM. These measures represent key aspects of a high-quality EM study and should be uniformly adopted. Evaluation of these measures in clinical practice is needed to assess their impact on outcomes.

BACKGROUND & AIMS

Esophageal manometry is the gold standard for evaluating esophageal motor function and diagnosing esophageal motor disorders.¹ This procedure is performed by placement of a transnasal catheter with recording sites along its length into the stomach in order to measure pressure events in the esophagus following test swallows. Esophageal manometry was interpreted using conventional line tracings until the advent of high-resolution esophageal manometry (HREM) in the 1990s which enabled clinicians to interpret esophageal pressure via a refined spatiotemporal display of pressure topography. Easier identification of anatomic landmarks and superior inter-rater diagnostic agreement are some of the many advantages of HREM over conventional line tracing.^1,2 These advances in esophageal manometry have improved the diagnostic capabilities and management of esophageal motility disorders making esophageal manometry an indispensable clinical test and research tool.^1,3

Performing esophageal manometry in a standardized manner is essential to accurate interpretation as variations in the performance of manometry (technical aspects, patient factors, and quality of technician and interpreter) influence manometric parameters and can result in discrepant and inaccurate diagnoses.^4–7 In addition, HREM, the preferred technology to evaluate motility disorders, is a costly technology.⁸ Repeat studies due to initial misinterpretation may negatively impact patient outcomes and result in misdiagnosis and avoidable healthcare costs. Hence, there is a recognized need to standardize the protocol of esophageal manometry and to grade procedural performance objectively. However, a reliable method to distinguish a high quality manometry study does not currently exist.

In the era of value-based healthcare, development of quality measures that benchmark procedural performance and the management of diseases is required. The National Quality Forum has introduced an initiative to establish a framework for quality measures to ensure that they are scientifically acceptable, usable and feasible.⁹ Distinct from guidelines, quality measures are held to a higher standard, representing key aspects of care that must be adhered to; non-adherence to a quality measure reflects suboptimal care.¹⁰ Although esophageal manometry has been shown to be a clinically useful test, a method to measure the procedural quality of esophageal manometry does not exist.¹¹ As such, developing measures that define a high quality esophageal manometry study is essential in order to reduce the discrepancy in performance of procedures and benchmark performance systematically.

The primary aim of our study is to utilize the RAND University of California Los Angeles Appropriateness Methodology (RAM)¹² to develop validated quality measures for performing and interpreting esophageal manometry.

METHODS

This study was approved by the Northwestern University institutional review board. We utilized RAM, a well-described method used to develop quality-of-care measures, to develop quality measures for esophageal manometry [Figure 1].^10,12–14

Figure 1.

Study design based on RAND UCLA Appropriateness Methodology (RAM).

Recruitment of the Expert Panel

Fifteen experts in esophageal manometry were invited to participate in this study. The main selection criteria in nominating the expert panel included leadership in the field of esophageal manometry, geographic diversity and diversity of practice type and setting. According to RAM constructs, the expert panel should include 7 to 15 members, so as to be large enough to permit diversity of representation while still being small enough to allow all members to be involved in the group discussion. Prior to the ranking process a questionnaire was sent to those who agreed to participate, in order to understand current practices and variations in esophageal manometry use in the United States.

Compilation of Potential Quality Measures

Potential quality measures were generated through 1) an extensive literature review (RY) identifying thirty relevant scientific papers including large randomized controlled trials, retrospective studies and systematic reviews related to esophageal manometry from the past ten years and 2) interviews with experts in the field.

Round 1: Initial ranking of potential quality measures

The list of potential measures with specific instructions for ranking was sent to the expert panel members via electronic mail for the first round of rankings based on the panelists’ personal judgment. A measure was considered appropriate if adherence was critical to performing and interpreting an esophageal manometry study, regardless of cost or feasibility of implementation. The measure should apply to the average patient who presents to the average physician at an average hospital. Finally, measures may not always provide benefit to an individual study but should be beneficial to the overall performance and interpretation of esophageal manometry.

Each measure was ranked on a nine-point interval scale in which a score of 1 signified definitely inappropriate, 5 signified uncertain/equivocal appropriateness, and 9 signified definitely appropriate. Panelists were also given the opportunity to suggest wording modifications to improve the validity of the measure or to suggest a new measure. Summary statistics and agreement was assessed for each individual potential quality measure.

Round 2: Discussion of potential quality measures and re-ranking

At a face-to-face meeting of all panelists (May 2015 in Washington, DC), a summary of round 1 rankings with aggregated summary statistics as well as evidence based literature for review was provided to each member of the expert panel. Selected quality measures were discussed to identify opportunities to improve wording and discuss the evidence. New measures could also be proposed. Following discussion of each measure, the panelists independently re-ranked each measure for appropriateness. The rankings were compiled, and summary statistics were again calculated for each individual measure.

Scoring for Appropriateness

Appropriateness was determined based on median rankings for appropriateness and the dispersion of rankings. Agreement for a panel of fourteen members was defined by twelve or more panelists’ rankings falling in the same three-point range (i.e., 1–3, 4–6, or 7–9). If three or more of the rankings were in disparate categories this was considered to be indicative of disagreement. A measure was considered appropriate if there was agreement for rankings in the range of 7–9 for a measure, and was deemed to be equivocal if the median ranking was in the 4–6 range or if there was overall disagreement for the measure. If the median ranking was in the range of 1–3 and there was agreement amongst the panel, the measure was considered to be inappropriate.

RESULTS

Of the fifteen nominated physicians, 14 accepted and participated in all processes. The panel was comprised of 2 female and 12 male clinicians and researchers in the fields of gastroenterology and gastrointestinal surgery from 13 academic institutions across the country with a median of 18 (range 9 to 36) years of experience in performing and interpreting esophageal manometry [Table 1]. Individual practice patterns were assessed through an initial questionnaire and are presented in Table 2.

Table 1.

Baseline demographics of expert panel members (N=14)

Clinial Parameter	Respondents, n (%)
Gender (Female)	2 (14%)
Age (Years):
• 40–54	• 7 (50%)
• 55–70	• 6 (43%)
• > 70	• 1 (7%)
Specialty:
• Gastroenterologist	• 3 (21%)
• Gastroenterologist, Motility Expert	• 9 (64%)
• Gastrointestinal Surgeon	• 2 (14%)
Practice Type (Academic)	14 (100%)
Number of States Represented	12

Table 2.

Expert Panel’s Current Practice Patterns for Esophageal Manometry

Practice Parameter	Respondents, N (%)
Personnel performing procedure*:
• Technician	• 5 (36%)
• Nurse	• 11 (79%)
• Physician	• 1 (7%)
• Other	• 1 (7%)
Minimum level of education of qualified personnel:
• High-school diplomma/General Educational Development       certification	• 2 (14%)
• College graduate	• 1 (7%)
• Medical assistant certification	• 2 (14%)
• Nursing certification	• 9 (64%)
Physician signing off on final results of esophageal manometry*:
• Gastroenterologist	• 2 (14%)
• Gastroenterologist, Motility expert	• 12 (86%)
• Gastrointestinal Surgeon	• 2 (14%)
Procedures referred by*:
• Gastroenterologist	• 14 (100%)
• Gastroenterologist, Motility expert	• 5 (36%)
• Gastrointestinal Surgeon	• 6 (43%)
• Internist/Primary care provider	• 9 (64%)
• Pulmonologist	• 1 (7%)
Esophageal manometry system*:
• High-resolution manometry	• 14 (100%)
• Conventional manometry	• 1 (7%)
High-resolution manometry systems*:
• Sandhill	• 4 (29%)
• Sierra Scientific	• 3 (21%)
• Given	• 10 (71%)
• Medical Measurement System	• 1 (7%)
Procedure setting*:
• Ambulatory center	• 3 (21%)
• Outpatient clinic	• 7 (50%)
• Inpatient gastroenterology lab	• 7 (50%)
Anesthetic used*:
• None	• 2 (14%)
• Lidocaine spray	• 2 (14%)
• Lidocaine gel	• 12 (86%)
Informed consent routinely obtained and documented	9 (64%)
Supine wet swallows performed routinely:	12 (86%)
• 5–10 per study	• 6 (50%)
• 11–15 per study	• 6 (50%)
• > 15 per study	• 0 (0%)
Upright wet swallows performed routinely:	5 (36%)
• 5–10 per study	• 2 (40%)
• 11–15 per study	• 2 (40%)
• > 15 per study	• 1 (10%)
Multiple rapid swallows performed routinely	6 (43%)
Mulitple water swallow performed routinely	2 (14%)
Provocative measures utilized routinely	3 (21%)
Time alotted between swallows:
• 10–30 seconds	• 8 (57%)
• > 30 seconds	• 6 (43%)
Classification scheme utilized*:
• Classic classification scheme	• 5 (36%)
• Chicago Classification v3.0 scheme	• 12 (86%)
Parameters interpreted routinely*:
• Esophagogastric Junction (EGJ)	• 14 (100%)
• EGJ morphology	• 11 (79%)
• Integrated relaxation pressure	• 13 (93%)
• Peristalsis	• 14 (100%)
• Pressurization	• 13 (93%)
• Contractile Pattern	• 14 (100%)
EGJ relaxation parameters measured*:
• 4 second Integrated relaxation pressure	• 12 (86%)
• Single sensor nadir pressure	• 1 (7%)
• 3 second nadir pressure	• 1 (7%)
• E-sleeve nadir pressure	• 1 (7%)
Contractile pattern parameters measured*:
• Contractile front velocity	• 4 (29%)
• Distal latency	• 13 (93%)
• Distal contractile integral	• 13 (93%)
• Peak amplitude	• 1 (7%)
• Intrabolus pressure	• 2 (14%)
Parameters included in manometry report, routinely:
• Clinical diagnosis	• 14 (100%)
• Chicago classification diagnosis	• 13 (93%)
• Summary of results	• 14 (100%)
• Tabulated manometry results	• 11 (79%)
• Treatment recommendations	• 4 (29%)
• Follow-up recommendations	• 5 (36%)
• Communication to referring provider	• 13 (93%)

^*Questions with more than one response permissible.

Twenty-six potential quality measures were initially proposed based on literature review and interviews with experts. Based on the round 1 rankings, 12 measures were identified as appropriate, 2 as inappropriate, and the remaining 12 were discussed during round 2. In addition, 3 newly proposed measures were discussed during round 2.

Ultimately, a total of 17 potential quality measures related to competency (2), pre-procedure (2), procedure (3), and interpretation (10) were determined to be appropriate [Table 3]. These latter 10 measures were combined into a single composite measure. In the end, 8 appropriate quality metrics, as well as 8 inappropriate and 4 equivocal measures for esophageal manometry were developed [Supplementary Tables 1 & 2; Figure 2].

Table 3.

Appropriate Quality Measures for Esophageal Manometry

Category	Appropriate Quality Measure for Esophageal Manometry	Median Score (min, max)
Competency	1. IF esophageal manometry is performed,     THEN the technician must be competent to     perform esophageal manometry. *	9 (2,9)
	2. IF a physician is considered competent to     interpret esophageal manometry, THEN     the physician must interpret a minimum     number of esophageal manometry studies     annually.*	9 (5,9)
Pre-procedural	3. IF a patient is referred for esophageal     manometry, THEN the patient should have     undergone an evaluation for structural     abnormalities prior to manometry.+	8 (3,9)
	4. IF an esophageal manometry is performed,     THEN informed consent must be obtained     and documented.*	8 (3,9)
Procedural	5. IF an esophageal manometry study is     performed, THEN a time interval of at least     30 seconds should occur between     swallows.	9 (4,9)
	6. IF an esophageal manometry study is     performed, THEN at least 10 wet swallows     should be attempted.	9 (6,9)
	7. IF an esophageal manometry study is     performed, THEN at least 7 evaluable wet     swallows should be included.	7 (3,9)
Interpretation	8. IF an esophageal manometry study is     interpreted, THEN a complete procedure     report should document	Composite Measure
	a. Reason for referral	a. 9 (9,9)
	b. Clinical diagnosis	b. 9 (9,9)
	c. Diagnosis according to formally validated       classification scheme*	c. 9 (5,9)
	d. Documentation of formally validated       classification scheme used*	d. 9 (5,9)
	e. Summary of results	e. 9 (9,9)
	f. Tabulated results including:	f. 9 (6,9)
	i. Upper esophageal sphincter activity+	f(i) 9 (2,9)
	ii. Interpretation of esophagogastric       junction relaxation	f(ii) 9 (8,9)
	iii. Documentation of pressure inversion       point if technically feasible+	f(iii) 8.5 (7,9)
	iv. Pressurization pattern	f(iv) 9 (7,9)
	v. Contractile pattern	f(v) 9 (6,9)
	g. Technical limitation (if applicable) +	g. 9 (9,9)
	h. Communication to referring provider	h. 9 (8,9)

^*Indicates measures ranked with disagreement in Round 1 and discussed during Round 2.

⁺Indicates newly proposed measures that were discussed and ranked in Round 2.

Figure 2.

Process to determine appropriatness of proposed quality measures.

Quality Measure, QM

The expert panel also strongly recommended adherence to five aspects of esophageal manometry; these however were not considered to meet criteria for appropriateness and thus were not ranked as appropriate quality measures [Table 4].

Table 4.

Strong Recommendations by the Expert Panel for Esophageal Manometry

A technician should perform a minimum of 20 observed esophageal manometry studies in order to establish competency.

A technician should perform a minimum of 50 esophageal manometry studies annually in order to maintain competency.

A physician should interpret a minimum of 50 esophageal manometry studies annually in order to mainain competency

High-resolution manometry should be utilized when performing an esophageal manometry study.

Esophagogastric junction relaxation should be measured by calculation of the integrated relaxation pressure.

Quality Measures

Competency (Quality Measures (QM) 1 & 2)

The quailty of the individual performing and interpreting the study was considered to be integral to esophageal manometry and the majority of panelists agreed that the technician and interpreting physician should demonstrate competency for esophageal manometry (QM 1 & 2). To date competency for esophageal manometry performance and interpretation has not been measured or validated and while this requires formal assessment, the expert panel believes that procedural volume is an important aspect of competency. Thus, although not determined to be appropriate quality measures, the experts made three strong recommendations with regards to a minimum procedural volume required for competency: 1) A technician should perform a minimum of 20 observed esophageal manometries in order to establish competency, 2) A technician should perform at least 50 esophageal manometry studies annually in order to maintain competency, and 3) A physician should interpret at least 50 esophageal manometry studies annually to maintain competency.

Initially, establishing a minimum required level of technician education was proposed as a quality measure. This measure was ranked with significant disparity across the panel. Based on discussions during round 2, it was agreed by the panel that level of education is an inappropriate quality measure.

Pre-procedure (QM 3)

Esophagael manometry should not be performed in certain scenarios including in the setting of an esophageal obstruction from a tumor or severe stricture; as such it was agreed that an evaluation for structural abnormalities of the esophagus should be performed prior to eosphageal manometry (QM 3).¹ It was recommended that this evaluation may be performed via upper gastrointestinal endoscopy, esophagram or other form of structural evaluation of the esophagus.^3,15

In round 1 there was disagreement for whether obtaining and documenting informed consent prior to the procedure was associated with a high quality esophageal manometry study. At baseline, 64% of our experts routinely obtain and document informed consent. Considerable discussion for this measure was held in round 2 and ultimately was agreed by twelve panelists that informed consent should be requisite (QM 4). Many experts agreed that informed consent protects and informs the patient and aids in shared decision making. Minor and major risks related to manometry exist and many experts felt that informed consent should be documented for all invasive procedures that harbor potential risks.^16,17

Although all panelists recommended that HREM be utilized over conventional line tracing in practice and 93% rely solely on HREM, disagreement existed for whether HREM should be required. Concerns about mandating HREM included access across all practice types, high cost, and catheter durability. Furthermore, experts agreed that while HREM is better able to identify key anatomic landmarks and esophageal motor patterns, conventional line tracings could also be used to perform high quality exams.^1,18,19 Panelists recognized that the medical community has historically been slow to embrace new technologies such as HREM and replace established modalities such as conventional line tracing. As a result, requiring the use of high resolution technology for manometry studies was ranked equivocally, however the experts do strongly recommend that HREM be used if available when performing esophageal manometry.

Procedure (QMs 5, 6, 7)

It was unanimously agreed that at least 30 seconds should be allowed between each swallow in order to allow the resting lower esophageal sphincter pressure to return to baseline and avoid deglutitive inhibition of esophageal motor activity (QM 5).^1,20 While current classification algorithms are based on ten water swallows in the supine position, studies have demonstrated that interpretation is possible if at least 7 swallows are adequate; thus, it was the group consensus that at least 10 supine wet swallows should be attempted with inclusion of at least 7 evaluable swallows (QMs 6 & 7).^21,22

Patient position is understood to be an important component of esophageal manometry. Traditionally, esophageal peristalsis was assessed in the supine position, in part to zero the pressure. With HREM, zeroing of pressure is not necessary and thus the procedure can be performed in the upright position. In addition, assessing esophageal motility in the upright position may eliminate vascular artifact encountered in the horizontal position. Of note, compared to supine, normative thresholds are reduced in the sitting position.^23,24 Despite the recognized value of both positions, panelists felt that mandating this for all studies is impractical. For instance, if a study in the upright position clearly demonstrates achalasia in a patient who is uncomfortable or at aspiration risk in the supine position, a high-quality manometry in the upright position may be sufficient. For these reasons this metric was considered to be inappropriate.

All three measures related to provocative measures were ranked as inappropriate. At baseline a minority of panelists obtain multiple rapid swallows (43%), multiple water swallows (14%) and provocative measures (21%) when performing esophageal manometry studies. While there is emerging evidence supporting the ability of multiple rapid swallows to assess esophageal peristaltic reserve and multiple water swallows and solid bolus swallows to induce esophageal pressurization if a subtle outflow obstruction is present, the panelists felt that insufficient data exists to standardize the use of provocative measures for all studies.^25–29 Additionally, provocative measures are time consuming, do not commonly alter diagnoses or strategies, may be difficult for the average physician at the average institution to perform, and may not be tolerated by all patients. Thus, while certainly useful for clinical investigation, provocative measures are refined specific measures whose universal adoption may be impractical and the application of these maneuvers should be individualized.

Interpretation (Composite QM 8)

Ten individual quality measures were considered with regards to interpretation of esophageal manometry. There was unanimous agreement for inclusion of the following items in the procedure report: reason for referral, clinical diagnosis, summary of results, tabulated results and communication to referring provider. Mandated documentation of recommendations for follow-up and treatment was considered inappropriate; panelists agreed that esophageal manometry is a diagnostic aid to be used in concert with other clinical information to make treatment and follow-up recommendations and that by itself is insufficient to serve this purpose.

While the Chicago Classification is the most recent validated schema for manometric interpretation of esophageal motility disorders, only 86% of the panel use this algorithm in their every day clinical practice.³⁰ After discussion, the group recognized that classification schema may evolve and be improved over time and thus it was agreed that a specified interpretation scheme should not be mandated. However, all esophageal manometry studies must be interpreted according to a formally validated scheme and, moreover, the utilized scheme should be documented.

With regards to tabulated results, specific parameters were considered. The UES was discussed at length. Identification of the UES is imperative to the measurement of other manometric parameters, however there is insufficient data to support the interpretation of the UES pressure; as such it was agreed that, at a minimum, visible activity of the UES be documented.¹ Panelists agreed that pressurization and contractile patterns should be interpreted. Similarly, EGJ relaxation should be interpreted, and while several parameters of measurement exist, it is the general recommendation of the panel that the integrated relaxation pressure be used.³¹ In discussing identification of the EGJ, the group recommended identification of the pressure inversion point (PIP) as it is imperative to ensuring intragastric probe placement and the relative location of the lower esophageal sphincter and crural diaphragm. This is important in detecting achalasia, defining hiatus hernia and describing post-surgical anatomy.^32,33 The panel recognizes that in certain cases identification of the PIP is not technically feasible in which case it is important to note the catheter location if possible (i.e. intrathoracic or intraabdominal). Realizing that studies may be limited and of poor quality for reasons beyond the technician or physician, e.g. poor patient cooperation, the panelists agreed that in such scenarios the existence of and reason for technical limitation should be documented (QM 8).

CONCLUSIONS

Despite its critical importance in the diagnosis and management of esophageal motility disorders, features of a high-quality esophageal manometry have not been formally defined. Standardizing key aspects of esophageal manometry is imperative in order to ensure the delivery of high-quality care. We report the development of quality measures for the performance and interpretation of esophageal manometry utilizing a formal validated methodology.

We initially proposed 26 potential quality measures through a review of the literature and interviews with experts in the field. On the basis of RAM, fourteen experts in esophageal disorders ultimately ranked 17 measures as appropriate; 10 measures were incorporated into a composite measure, yielding a final total of 8 appropriate quality measures for the performance and interpretation of esophageal manometry.

The goal of this work was not an attempt to promote specific practice guidelines, but rather to propose baseline quality measures by which payers, institutions, physicians and patients may assess the quality of esophageal manometry. Ultimately, such measures may be incorporated into national repositories and linked to public reporting and reimbursement.

These eight appropriate quality measures are considered absolutely necessary in the performance and interpretation of esophageal manometry [Table 3]. In particular measures #3 to #8 are clinically feasible and measurable, and should serve as an initial framework to benchmark quality and reduce variability in esophageal manometry practices. Based on this, prior to performing a manometry study an evaluation for underlying structural esophageal abnormalities should be performed and documented (#3) and informed consent should be obtained and documented (#4). During the manometry study, a minimum of 10 attempted swallows and 7 evaluable swallows should be performed and documented (#6 & #7). Interpretation of an esophageal manometry study should at a minimum document the parameters outlined in #8. While measures related to competency (#1 & #2) were ranked as highly appropriate, further work is needed to determine metric specifics prior to implementation in clinical practice.

Despite agreement on these measures, we did find that variation in esophageal manometry practices exist even amongst our panel of fourteen experts and believe that greater differences are present across all practices, once again emphasizing the need for quality measures for esophageal manometry. There were additional measures considered to have high importance that were not ranked as appropriate for various reasons, including challenges to implementation across all practice types. Some measures were considered integral to research and clinical practice at a center of excellence, however of less relevance when considering the average patient at the average institution.

This process highlighted areas for future research. For instance, an esophageal manometry study may be compromised in the hands of an incompetent technician, regardless of technology quality. This was considered to be of such high importance that the experts felt it imperative to recommend a minimum procedural volume for competency based on expert opinion alone. A formal determination of what constitutes competency for esophageal manometry is needed and the working group is currently pursuing the development of a validated competency assessment tool for esophageal manometry. Additionally, interpretation of anatomic landmarks is included as a quality metric; as such, standardizing templates for manometry analysis and documentation is of interest. These measures were developed for the clinical practice of esophageal manometry in the United States, and simulating this measure development process in other countries should be considered. Additionally, these measures were developed solely for esophageal manometry; quality metrics for other forms of esophageal motility testing such as high-resolution impedance manometry and pH monitoring do not currently exist. Lastly, the impact of these process quality measures on patient outcomes in clinical practice needs to be examined.

In summary, this is the first report of using RAM to develop physician-led valid quality measures for esophageal manometry. As the US healthcare system transitions from a volume based to a value and quality based system, this work is an initial and important step in improving the quality of care of esophageal disorders.