Advanced Systems to Assess Colonoscopy

INTRODUCTION:PREVENTION OF COLORECTAL CANCER BY COLONOSCOPY

Colorectal cancer is the second major cause of cancer-related death in the United States.¹ The long time involved in progression of mucosal dysplasia from a small polyp to an invasive cancer, the mucosal shedding of molecules and cells into stool during this process, and the ability to image the colon mucosa using direct inspection or x-ray techniques are features that make early detection and prevention of colorectal cancer possible. Various screening tests, such as digital rectal examination, fecal occult blood testing, double-contrast barium enema, and colonoscopy, have increasingly contributed to the detection of polyps and early cancers. Among these tests, colonoscopy is the most accepted screening method for the detection of colorectal cancer or its precursor lesions and is the only colorectal cancer screening and surveillance technology that allows for diagnostic and therapeutic operations in one procedure. Colonoscopy has contributed to a marked decline in the number of colorectal cancer-related deaths.

THE PROBLEM WITH COLONOSCOPY: NOT ALL COLORECTAL CANCERS ARE PREVENTED

Recent data suggest, however, that there is a significant miss rate for detection of even large polyps and cancers.^2–8 Examples include a double-procedure study from the author’s institution where colonoscopy failed to detect 4 out of 5 individual colorectal cancers detected by CT colography,³ a double-cohort study where colonoscopy detected fewer colorectal cancers than CT colography,⁸ a population-based case control study in Canada where colonoscopy did not prevent death from right-sided colorectal cancer,⁴ and a screening study from Germany where the repeat colonoscopy findings were similar to the case-control study findings in Canada.⁶ The conclusion from these and other studies (for a comprehensive summary, see Hewett and colleagues⁵) is that the protective effect of colonoscopy, when used in routine clinical practice, has not lived up to the expectations raised by carefully controlled prospective research studies. Furthermore, the protective effect seems minor or absent for right-sided cancers and at best approximately 70% for left-sided cancers.

ASSUMPTIONS: FACTORS THAT MAY EXPLAIN FAILURES OF COLONOSCOPY

Several factors may contribute to the miss rate. In general, these factors can be divided into those related to the patient, the equipment used, and the endoscopist performing the procedure. A cooperative patient, either due to voluntary control of the patient or due to a moderate amount of sedatives and analgesics, is a requirement for a successful endoscopic examination. Similarly, a colonic anatomy allowing passage of the colonoscope to the cecum is assumed—this is the case in nearly all patients. The patient-related factors that may lower miss rates consist mainly of two important actions: first, discontinuation of any nutrients other than clear liquids for a defined time before the procedure (most often 1–2 days), and, second, strict adherence to a bowel cleansing program1 to 2 days before the procedure. The desired end result is a colon free of any solid food with either no liquid content or small amounts of highly diluted stool and gastrointestinal juices that are easily aspirated. Although no truly objective measurements for judging colonic preparation exist, a semiquantitive subjective scoring system is used by most endoscopists.⁹ The equipment-related protective factors are variable and less dominant than in the past, given the overall quality of the currently available commercial endoscopes. Nevertheless, there are real differences between endoscopes of different manufacturers that can affect the protective effect of colonoscopy. The endoscopist-related protective factors consist mainly of skill set, the inspection time, and the effort exerted to inspect as much of the visible mucosa as possible. At present, skill set is defined as having completed a minimal number of procedures during a formal fellowship; a subset of these procedures should include certain endoscopic diagnostic and therapeutic instrumentations. Formal testing of the acquired skill set does not take place. There is debate about what constitutes optimal inspection time; however, the American Society for Gastroentintestinal Endoscopy (ASGE) and American College of Gastroenterology (ACG) in a consensus document in 2006 suggest that independent of patient, equipment, and endoscopist, at least 6 to 10 minutes should be spent during the withdrawal phase on careful inspection of all visible colonmucosa.⁹ The third endoscopist-related factor is the effort to inspect as much of the visible mucosa as possible. Effort is different from skill set. Meticulous effort means that by using all options available, such as torque, lateral (left/right) and vertical (up/down) tip deflexion, aspiration, washing of mucosa, retroflexion, and repeatedly moving through tight angulations, the endoscopist tries to inspect the entire colon mucosa. Current equipment allows inspection of most (>90%–95%) of the colon mucosa (the visible mucosa) during a routine screening colonoscopy in a normal 50-year-old patient if all these techniques are used as required during the procedure. A complete inspection (100% of colon mucosa) is unusual with current endoscopic equipment; inspection of less than 90% to 95% in a well-cleansed colon of a normal 50-year-old patient should lead to questioning the skill set or the level of effort of the endoscopist.

DEFINING QUALITY OF COLONOSCOPY

Metrics that are Being Collected

For each colonoscopic procedure, some data are collected. At the most basic form, data collection consists of a handwritten or dictated free text report with or without a few images in the form of separate photographs. To get paid, a set of billing codes is available, frequently in a separate practice management system. In cases when specimens are obtained, another piece of paper, such as a letter from a pathology laboratory, may hold the final histologic diagnosis. Additional data, in particular data related to quality of the procedure, may not be collected. Meaningful data extraction to examine quality is not possible because this would be prohibitively expensive and provide little or no useful information.

In the most detailed and comprehensive form, detailed electronic data are available with structured information about the indication for the colonoscopy, preprocedure education and instructions, adherence to the bowel preparation regimen, the procedure with all its details, digital images of key anatomic locations and findings, any complications related to the procedure, recovery and discharge, any histologic findings from specimens removed, and suggested follow-up. At present, sophisticated systems allowing all of this are mostly available in large gastroenterology group practices and academic centers. Sometimes practice-, research-, and quality-related data are collected in separate electronic applications; sometimes—and ideally—all these data types are collected in a single application that serves all needs and prevents redundant data collection activity. In reality, in 2010 most endoscopists, whether or not solo practitioners, in a small single specialty group, or in a large multispecialty center, have implemented data collection methods somewhere between basic paper-based and highly comprehensive, digital formats.

Given these descriptions of the wide variety of data collection methods coupled with (1) an absence of any central (ie, federal, state, societal, or insurance) requirements regarding which data should be collected, (2) the voluntary nature of establishing some kind of prospective data collection, (3) the lack of a minimum set of universally agreed-on data types worthy to be collected, (4) the lack of a financial incentive to collect data that would prove quality, and (5) the possible legal ramifications of collecting data that can possibly incriminate the physicians performing the procedures, it is not surprising that most if not all data that currently are collected regarding colonoscopy are simple, limited in quantity, easy to obtain, inexpensive to collect, and not reflecting what actually happened during the procedure.

In 2006, the ASGE and ACG published a consensus report in which these organizations listed a set of minimal criteria related to quality as recommendations.⁹ Of these recommendations the following four intraprocedure recommendations are not related to pre-existing disease conditions and best define skill set and effort of the endoscopist during colonoscopy:

1. Cecal intubation rates: visualization of the cecum by notation of landmarks and photo documentation of landmarks should be documented in every procedure.

2. Detection of adenomas in asymptomatic individuals (screening): adenomas should be detected in at least 25% of men and at least 15% women more than 50 years old.

3. Withdrawal times: mean withdrawal time should be at least 6 minutes in colonoscopies with normal results performed in patients with intact colons.

4. Mucosally based pedunculated polyps and sessile polyps less than 2 cm in size should not be sent for surgical resection without an attempt at endoscopic resection or documentation of endoscopic inaccessibility.

The problem with these four recommendations is that they do not reflect the eventual result of colonoscopy—the final state of colon preparation after removal of remaining fecal material, the amount of mucosa inspected, and the completeness of removal of all lesions. If cecal intubation is documented by a good-quality image, then there is solid evidence of the fact that the entire colon was traversed; if an image is not available, the opinion of the endoscopist can be relied on. Finding one or more polyps is not a guarantee that other polyps are not missed. Similarly, spending 6 minutes during withdrawal is not a guarantee that all mucosa was cleaned as needed and inspected. Finally, an attempt at polypectomy of a lesion less than 2 cm is not the same as being able to remove a polyp less than 2 cm without remaining polypoid tissue in (nearly) all cases.

In summary, current intraprocedural quality measures are subjective and do not reflect the effort of the endoscopist to clean, inspect all mucosa, and remove all abnormalities (ie, they do not reflect true quality of colonoscopy). In addition, these limited data provide a false sense of measuring or providing quality and allow easy manipulation (eg, remove one polyp and delay endoscope removal until a withdrawal time of 6 minutes is reached) of data toward an apparently favorable outcome.

Metrics that Should be Collected

Only two things matter when it comes to colonoscopy and colorectal cancer: Was colorectal cancer prevented in the patients who underwent colonoscopy (less morbidity)? and Did the patient live longer due to the intervention (less mortality)? If cancer is prevented but patients do not experience a better quality or longer life, screening is not indicated. Assuming that prevention of colorectal cancer does lead to longer, good quality of life, in my opinion, there are two choices to evaluate colonoscopy (Fig. 1). First, when, where, and how colonoscopy fails to prevent colorectal cancer can be measured by calculating the frequency of colorectal cancer despite colonoscopy (CCdC) or the interval colorectal cancer rate. This is not a trivial task—it requires long-term follow-up of a large study population and requires detailed data about the condition of the patient and specimens removed from the patient. Second, whether or not a high-quality colonoscopy was performed can be measured based on evaluation of the entire colonoscopy instead of a limited data set, as is currently the case.

Fig. 1.

Options to measure true quality of colonoscopy. The first option is to determine over time whether or not colorectal cancer occurs despite colonoscopy (A). The second option is to develop technology that can measure whether or not a high-quality colonoscopy was performed based on the entire colonoscopy (B). CLEAR is an acronym for Clean, Look Everywhere and Abnormality Removal, features that define quality of colonoscopy.

Assessing whether or not colonoscopy prevents colorectal cancer

At the Mayo Clinic, the author and colleagues have created a large database spanning from 1992 to 2009.¹⁰ This database contains all endoscopy, diagnosis, and pathology information about all patients seen at the institution. As data are currently under review, the results are not disclosed in this article. Several important conclusions, however, are drawn based on the results.

1. First, CCdC is not a random event. Among many factors and features examined, the endoscopist doing the procedure is of key importance because some seem more often involved in CCdC cases than others.

2. Second, most cases of CCdC seem to be truly missed tumors not rapidly growing de novo tumors.

3. Third, these tumors seem to have similar features than tumors that are not missed, suggesting once more that they are truly missed and not rapidly growing de novo tumors.

4. Fourth, withdrawal time duration was not related to number of CCdC, neither was polyp detection rate.

5. Finally, the protective effect for some endoscopists against CCdC extends beyond 3 years and may even extend beyond 5 years, the longest period the author and colleagues have studied thus far.

These preliminary data, from the largest and most detailed study done so far, clearly point to the endoscopist as the key factor determining whether or not a patient develops colorectal cancer in the first 3 to 5 years after a colonoscopy.

Assessing whether or not colonoscopies are of high quality

Measuring failure of prevention of colorectal cancer as the ultimate outcome provides objective, meaningful data but takes many years of careful observation and detailed clinical data acquisition. In addition, patients who underwent a suboptimal colonoscopy may develop and die from colorectal cancer during the observation period, thus not benefiting at all from the quality-control efforts. To address this issue related to colonoscopy, the author and colleagues developed a second approach that uses the entire procedure to determine quality. Instead of taking a few convenient, easily measurable, multiple, procedure-based features, such as cecal intubation rate, adenoma detection rate, and average withdrawal time, the author and colleagues proposed obtaining detailed quality-related information from every colonoscopy representing the entire procedure. Instead of providing an average, subjective, surrogate rating of quality for an endoscopist, which is meaningless for an individual patient, the author and colleagues proposed providing an objective, detailed report of an individual procedure reflecting the actual quality provided to an individual patient.

How is achieving an objective, detailed report for every individual patient proposed? The solution is algorithm-based, automated analysis of the video stream representing the entire procedure. The author and colleagues realize that other important quality features cannot be derived from the video stream, such as preoperative instructions, the amount of discomfort of a patient during the procedure, reasons for not removing polyps, and any follow-up instructions. All of these features are important but become irrelevant in the presence of a poor-quality colonoscopy.

For colonoscopy, one needs to measure from insertion until removal of the endoscope whether or not the endoscopist instituted all efforts reasonably possible to CLEAR the colon of all lesions during the procedure. CLEAR is an acronym that reflects the three important aspects of colonoscopy, which the author and colleagues think define quality:

1. Clean—the patient should adhere to the colon preparation instructions and the endoscopist should remove remaining fecal material. The end result at the time of withdrawal should be a good or excellent prepared colon as defined in the ASGE and ACG guidelines.

2. Look Everywhere—the endoscopist has to actively look behind every fold (working the folds) and move or remove remaining stool to achieve as close to 100% inspection of the colon mucosa.

3. Abnormality Removal—the endoscopist has to remove polyps by biopsy forceps, snare polypectomy, or other modalities; lesions left behind may develop into a malignant lesion before a next screening or surveillance procedure is performed.

For each of these three key aspects of colonoscopy, one or more metrics need to be developed to provide a meaningful quality report that truly reflects how well the colonic preparation was after removal of remaining fecal material, how much of the colonic mucosa was well seen, and how completely any premalignant lesions were removed.

ENDOSCOPIC MULTIMEDIA INFORMATION SYSTEM

During the past 7 years, the author and colleagues have developed an automated, innovative system that uses computer-based algorithms to analyze the image stream generated during colonoscopy for specific metrics.¹¹ This system is named the Endoscopic Multimedia Information System (EMIS). EMIS at present does not interfere with actual colonoscopy because the same image stream analyzed by computer is also displayed on a monitor, allowing a colonoscopist to view the colonic mucosa and perform diagnostic and therapeutic procedures as indicated. The ultimate goal of EMIS, however, is completely automated real-time (ie, during a procedure) analysis of colonoscopy with feedback to the endoscopist to confirm that specific quality milestones have been achieved or to drive endoscopist behavior toward achieving these milestones in case this seems not to happen. EMIS consists of several components, each critical for pursuing the ultimate goal.

EM-Capture

The first component of EMIS is EM-Capture, a set of fully automated, real-time endoscopy video stream capture and file-generation algorithms.¹² The algorithms determine whether or not the image frames are derived from an endoscope with its tip containing the video camera inside the patient. The algorithms use a combination of frame-derived color, movement, and shape aspects in real time to determine absence or presence of an inside-the-patient state. Since May 2007, the author and colleagues have gradually expanded the number of endoscopy rooms equipped with this software from 2 to 13 with outstanding results: under specific conditions, the algorithms remove nearly 100% of all leading outside-the-patient frames and no trailing outside-the-patient frames as programmed. Recording stops after a few continuous minutes of outside-the-patient recording to allow for removal of polyps that are too big to pass via the working channel, change in endoscope, lens cleaning, and so forth. Initially, the author and colleagues recorded a few minutes of the video stream after the endoscope had been removed from the patient to verify that the entire colonocopy procedure has been captured; currently, the author and colleagues record all inside-the-patient images until the time point that the endoscope passes through the anus while verifying that no repeat insertion of the endoscope occurs over the next minutes. In addition, the author and colleagues have verified the number of procedures recorded by the system over a 4-day time span with the number of procedures performed according to endoscopy practice data: comparison showed that the automated, inside-the-patient technology captures every procedure. Comparison showed also that endoscopy practice data—despite best attempts—contained at predictable regularity errors, such as incorrect room assignment and incorrect start and end times of the procedure.

EM-Capture runs on a robotic workstation; this system has no keyboard, mouse, or monitor and is managed remotely by another set of algorithms running on a central server, EM-Central (discussed later). The workstation consists of inexpensive common off-the-shelf hardware: a Core 2 Duo CPU, 4 to 8 GB RAM, two 250-GB or larger hard drives, and a video capture card for total costs, including installation of less than $1500 per endoscopy room. Three cables connect it to (1) 110 V, (2) the image processor of the endoscope, and (3) the intranet. The EM-Capture algorithms run as a component of the operating system; therefore, anyone logging on or off remotely does not interrupt image capture or video file generation. Video file size is variable depending on the length of the procedure. At present, MPEG-2 video files are generated consisting of 30 720 × 480 pixel color images per second, which results in approximately 1 GB of hard drive space per 20 minutes. Video file capture in high definition format requires substantially more hard drive space per video file.

To summarize, EM-Capture automatically detects when an endoscope enters a patient and provides a collection of video files that represent all endoscopies performed in the rooms where EM-Capture is installed. With EM-Capture, there is a record of every endoscopy from start to finish.

EM-Central

The second component of EMIS is EM-Central, a set of control and scheduling algorithms that reside on a central server.¹³ As with EM-Capture on the robotic workstations, EM-Central operates autonomously: it gathers information about the state of the workstations, schedules specific tasks at specific times, and sends e-mails to the programming staff if any of the operating conditions from any of the workstations or the server itself are out of predefined bounds. The EM-Central server also functions as a Web server allowing review of the state of EMIS and access to the various functions of EM-Central via an Internet browser. Currently EM-Central has five main functions:

1. A summary of all captured video files is available under the tab, “Captured Videos.” Files can be listed in tabular format or in graphic outline per room and per time period. For instance, a complete overview of a single day, week, month, or year is available per room or for the entire unit (Fig. 2).

2. Any automated annotations are shown in the second tab, “Annotated Videos.” Drill downs are possible for the annotations and specific images identified (eg, end of insertion, appendix, or instruments) are shown when detected by the algorithms.

3. The state of EMIS hardware and the software programs running on the various workstations and server are shown in the tab, “Machine Status.” This allows verification in a single view that all workstations, server, and software are up and running and that enough storage capacity is available.

4. The “Events Log” tab shows a summary of all the actions taken by the central server, such as moving files from capture location to storage server, moving files between workstations, and deleting files.

5. Under “Accounts Management,” the manager of the system can set access privileges for people who have access to the system. Depending on role, more or fewer of the system features are available to individual users.

Fig. 2.

Screen shot of room use chart for July 2, 2008. Eight endoscopy rooms are listed; blue bars represent the time periods when inside-the-patient events occurred that were detected by EM-Capture and captured and saved as a video files.

EM-Manual

The third component of EMIS is EM-Manual, a manual annotation program.¹⁴ EM-Manual allows extracting single frames or a series of frames as a clip; a clip is a video fragment between by two specific frames (ie, the start and end frames). Using minimal standard terminology (MST), as developed by the European Society of Gastrointestinal Endoscopy, anyone with endoscopic expertise can annotate video files. All MST-based annotations are individual frame based: either the annotation states something about the entire frame (eg, showing a specific anatomic location) or part of a frame is selected using an ellipse and the annotation is limited to that part of the frame (eg, “appendix” in Fig. 3).

Fig. 3.

Screen shot of EM-Manual: frame mode. The left upper frame shows properties of the video, video segments (eg, segmentation based on speech recognition if voice annotation was used during manual video file capture to define colon segments), and annotation tools. The right upper frame shows the consultation panel, allowing annotation of images or components of images. The lower frame shows the various annotation options available. The image component appendix is marked by an ellipse and annotated using MST as “Normal” and Site “Appendix.”

In addition to MST-based annotations, EM-Manual allows incorporation of custom annotations, used these for two purposes. The first use is to annotate specific features for which algorithms are being developed for machine-based feature recognition. For instance, frames showing the appendiceal orifice are annotated to show the opening of the appendix in the cecum. As MST-based annotations, these are frame-based. The second use is to support specific studies; each study can be supported by one or more study specific tabs. Examples include tabs for overall colon preparation score (according to ASGE and ACG⁹), segmental colon preparation after endoscopic cleansing (Boston bowel preparation scale¹⁵), key time points (end of insertion, maximal insertion, and end of procedure), and quality of retroflexion (Fig. 4). Several of these annotations are video based, some cover the entire video (colon preparation), whereas others may be specific to a small segment (retroflexion).

Fig. 4.

Screen shot of EM-Manual: video mode. The upper right frame shows the video panel and the lower frame the Boston bowel preparation scale (BBPS) annotation panel. Rapid review of video files is possible in forward and backward play mode. Speed can be increased or decreased in steps of 2 from normal to 32× normal forward speed or 2× normal backward speed using the “>>” and “<<” buttons. Jumping 10 seconds forward or backward is easily achieved using “>|” and “|<” buttons. Jumps to start and end of the video file can be achieved using “|-“ and “-|” buttons. Jump to a specific frame can be accomplished by entry of the frame number next to “Jump to.” Graphic annotation panels are preferred where possible; in the panel “Prep Out” in the lower frame, a colon model is used for right, transverse, and left colon annotation for BBPS annotation.

Annotations are stored in a project folder that resides in the same directory as the video file being annotated. Inside the project folder reside all the annotations, the individual frames that are annotated, and any clips generated from the video file (Fig. 5). All annotations are stored in a single extensible markup language (XML) file with extension, APRO (for Arthemis PROject; the original code name of the project¹⁴). All images are stored in an IMG folder. All clips are stored in a CLIP folder; because a clip is generated from a minimum of two frames, each of which may have annotations, each clip has its own APRO folder with all annotations and images related to the frames inside the clip.

Fig. 5.

EM-Manual (APRO) file organization. Video files are stored as *.mpg files. All annotations are stored in a project folder that has the file name of the video file and the file type, “PROJECT.” Within the project folder reside the *.APRO2 file (a second more advanced version of an earlier *.APRO format) and subfolders with images selected for annotation and clips generated. A CLIP folder contains the actual clip and the CLIP PROJECT folder; as for the video project folder, the CLIP PROJECT folder contains the clip *.APRO2 file and an image folder. All annotations are contained within the *.APRO2 files and are combined with the video file and images by loading an *.APRO2 file into EM-Manual.

All annotations for a series of video files in a single folder can be extracted from the APRO files in the project folders using a dedicated program that generates a single comma-separated values (CSV) file. The CSV file can be loaded into any statistical package for analysis.

EM-Automated

The fourth component of EMIS is EM-Automated, a software package containing algorithms for automatic extraction of endoscopic features.¹¹ Currently, EM-Automated runs postprocedure (ie, analysis starts after the procedure has been completed). The reason for this is twofold: first, the algorithms take more time than is available during the procedure; and second, the hardware costs to allow real-time annotation until recently were exorbitantly high. Faster algorithms and more powerful workstations at lower cost, however, soon will allow the first algorithms to run in real time. The ultimate goal is to create a real-time version that reliably assesses features related to quality and provides feedback during a colonoscopy.

To achieve the ultimate goal, EM-Automated needs to complete several steps. First, it needs to determine whether or not the endoscope is inside a patient; currently this is achieved by EM-Capture.¹² Second, it needs to determine whether or not the endoscope is inside a colon; for postprocedure processing, this is irrelevant because the author and colleagues only run the program on video files obtained during colonoscopy. The endoscopy rooms, however, are used for a mix of procedures, including esophagogastroduodenoscopy. For real-time analysis and feedback, the algorithms either need to be instructed that the video stream is from a colonoscopy, or algorithms need to be developed that detect in real time the type of the endoscopy. This work is ongoing. Third, it needs to determine whether or not the images are clear or blurred; analysis of blurred images is unlikely to reveal quality features.¹⁶ Fourth, it needs to determine which part of the clear image consists of colon mucosa and which part consists of remaining fecal material.¹⁷ Finally, it needs to extract several features that are associated with a high-quality colonoscopy and, if these are not present, provide some form of feedback (auditory or visually) to the endoscopist. Table 1 lists several features for which the author and colleagues have developed algorithms; some run only as postprocedure algorithms, some are already converted to the real-time environment. Some of the preliminary results are discussed.

Table 1.

Features for which algorithms have been developed

Feature	Measurement
Image quality	Clear or not clear (blurred)
Stool	Present or absent
Location	Colon segments or unique location
Movement direction	Forward or backward
Speed	Fast or slow
Maximal intubation	Cecum, appendix, terminal ileum, or other
View direction	Forward or lateral
Space-occupying lesion	Present or absent
Instruments	Present or absent
Quadrant coverage histogram	Numeric score
3-D mucosal inspection estimate	Numeric score

EMIS RESULTS

The system is still in development; existing algorithms are being optimized, new algorithms are in development, the infrastructure is being altered to allow real-time annotation with feedback, some algorithms are targeted for graphics processing unit (GPU) instead of central processing unit (CPU) processing, and proprietary modules are being replaced by open source modules. The latter two changes are implemented to lower the cost of the hardware and software to make EMIS affordable for every endoscopy unit. Despite that EMIS is far from complete, the author and colleagues already have made several observations.

Peer Review: EM-Capture Combined with EM-Manual

The author and colleagues have created a huge database of randomly captured as well as specific colonoscopist-derived colonoscopies over the past 5 years using both Fujinon and Olympus equipment. These files are used not only as source for training and test images for algorithm development but also as material for peer review by a varying group of endoscopists. Three preliminary results, currently pending review, can be drawn from analysis of this video file database.

1. There is a large variation in the practice of colonoscopy. In some colonoscopies, endoscopists spend great effort in cleaning remaining fecal material whereas in others most debris is left untouched. In some colonoscopies, endoscopists spend great effort working the folds whereas in others, a single, straight pull backward is observed. Therapeutic maneuvers for similar polypoid lesions can vary from cold biopsy, hot biopsy, cold snare, and hot snare to mucosal resection.

2. There is consensus among endoscopists as assessed by peer review regarding important aspects of quality. Preliminary peer review data show that endoscopists agree strongly when a colonoscopy is performed with high quality.¹⁸ Agreement among endoscopists is less marked when certain features of quality are present but others not (eg, excellent fecal material removal without working the folds or inspecting angulations). This likely reflects a difference of endoscopists’ expectations, in other words, a difference in individual definition of what defines high quality.

3. Some endoscopists are significantly and persistently better in performing high-quality colonoscopy examinations than others. Again, these data are under review, but the conclusion is obvious: endoscopists are not all equal.

These preliminary results strongly support the author and colleagues’ conviction: an objective review and feedback process—ideally during the examination—is needed of every colonoscopy to guarantee that every patient undergoes an examination that includes a minimal set of quality deliverables.

Computer-Based Review: EM-Capture Combined with EM-Automated

In fall of 2010, the author and colleagues will start rollout at the Mayo Clinic of a system that performs completely automated video stream capture and postprocedure analysis. The system consists of EM-Capture, EM-Central, and a postprocedure version of EM-Automated. EM-Central will monitor EM-Capture and, when EM-Capture is done for the day, direct EM-Automated to execute several algorithms that will derive metrics regarding the following aspects of colonoscopy:

• Image quality—fraction of images that are clear in each specific phase of the procedure

• Mucosal preparation—fraction of clear images with stool and within images with stool the fraction of the image that shows stool (Fig. 6)

• Maximal extent—time point when insertion phase ends and withdrawal phase starts

• Appendix—time points and images when appendix is detected

• Back and forth movements—the number of times the endoscope changes direction in forward-backward direction

• Quadrant coverage histogram¹⁹—a score that consists of the number of times the endoscope tip traverses all quadrants of the colon for a predefined lateral deflection from the central axis; this score is the author and colleagues’ first effort at measuring working the folds (Fig. 7).

Fig. 6.

Example of automated stool detection. Region 1 is clear mucosa; regions 2, 3, and 4 are mucosa covered with stool.

Fig. 7.

Quadrant coverage histogram: Circumferential withdrawal: A and C show the spiral shape of the movement of the tip of the endoscope. B and D show the visual field in red. A and B reflect views in which the distant proximal colon is visible; C and D reflect views in which the distant proximal colon is absent and only colon wall is seen. Position 1 shows a left lateral quadrant view, position 2 a superior quadrant view, position 3 a right lateral quadrant view, and position 4 an inferior quadrant view; endoscopic views are from bottom to top of figure. The green bars in A and B indicate the distant proximal colon visible in all positions; hatched green bars in C and D indicate the distant proximal colon that is not visible in any position. The quadrant coverage histogram algorithm counts the number of times the tip of the endoscope passes through all four quadrants at a specific deflexion from the center. (From Liu D, Cao Y, Tavanapong W, et al. Quadrant coverage histogram: a new method for measuring quality of colonoscopic procedures. Conf Proc IEEE Eng Med Biol Soc 2007;1:3470-3; with permission.)

For each of these metrics, the author and colleagues have developed, tested, and implemented computer-based algorithms. In each case, the same method was followed: a set of video files or images was created that contained the features for which algorithms were being created. These were divided in training and test bed. The training files or images were used to develop and train the algorithms; the test bed was used to determine the sensitivity and specificity of the mature algorithms. The author and colleagues created algorithms for an extensive list of features, not all of which will be implemented in the first rollout of EM-Capture combined with EM-Manual (see Table 1).

Each of these metrics can be studied per time or anatomic—if known—segment, providing far more accurate, objective information about the procedure than any other method. A copy of several images (eg, one clear image every 15 seconds) and images of key events, such as maximal extent and appendix, can automatically be stored to provide a succinct summary of the procedure without taking up much disk space. In addition, gaming the system, easily achieved under current guidelines (eg, watching the clock for 6 minutes during withdrawal), will be difficult if not impossible. Saving the entire video file, if only for 2 to 4 weeks, to allow review by health care providers or patients is likely to encourage high-quality examinations because nobody wishes to create a semipermanent record of a low-quality colonoscopy.

Whatever the final implementation of the EMIS technology, future studies will need to investigate whether or not (1) EMIS can measure differences in quality of colonoscopy and (2) whether or not a high score from EMIS means that a high-quality colonoscopy was done. The EM-Manual derived scores will be compared with the EM-Automated derived scores in formal, blinded research studies.

Computer-Based Feedback: EM-Capture and Real-Time EM-Automated

In 2012, the author and colleagues plan to roll out a system that does image capture, quality analysis, and polyp detection in real time. This requires three major advances. First, a new computer infrastructure needs to be created that permits advanced, complex real-time image analysis at an affordable price. Such an infrastructure does not yet exist. However, tremendous advances in hardware and software—multiprocessor CPU chips, multithreading 64-bit operating systems, multiprocessor GPU chips, and new software to effectively and easily use these massively multithreaded manycore GPU chips—are occurring while at the same time costs are rapidly decreasing. Second, a fast polyp detection technique needs to be developed. The author and colleagues, as well as others, have developed several polyp detection techniques, some based on shape, some based on texture. None of these techniques, however, are either fast enough or of high enough accuracy to allow real-time detection when using low-cost capture stations. Affordable, multiprocessor CPU/GPU systems will allow implementing polyp detection techniques in real time. Third, a timely and easily understood real-time feedback method needs to be developed. Feedback needs to be provided as soon as possible when a persistent deviation from predefined guidelines occurs. Alternatively, feedback should not detract and interfere with normal workflow. The author and colleagues are working on two types of feedback: visual, by providing overlay projection on the monitor visible to all in the endoscopy room, and auditory, by providing a voice message via earphone for private, endoscopist-only feedback. Finally, the message should be simple and clear, and the endoscopist should be able to respond to the message with a predictable endoscopic action that is considered to improve quality and can be measured using EMIS technology. Fig. 8 provides a schematic view of the real-time solution. A first effort at real-time feedback will occur in 2010: the author and colleagues are planning to provide real-time feedback related to the fact that retroflexion in the rectum has been performed and, if this is the case, whether or not the images show clear images of the entire mucosa surrounding the endoscope coming through the anal canal.

Fig. 8.

Real-time analysis. During real-time analysis, the video stream is analyzed first for inside/outside patient state. If the endoscope is inside a patient, the images are kept in the buffer and analyzed for protruding lesions and quality criteria. Feedback is provided either visually or auditory; the feedback and the response to the feedback are stored and can be imported into the electronic medical record (EMR) if desired.

FUTURE PLANS

Preliminary data from manual review of colonoscopy video files show that most endoscopists try to remove most polyps that are detected and visualized. Not all polyps seem to be removed and not all polyps are completely removed, but there may be reasons for leaving behind polyps (eg, patient to have surgery, patient on anticoagulation, and so forth) or part of polyps (eg, patient becomes hemodynamically unstable and procedure is aborted) that are not obvious from only analyzing the video stream. All endoscopists who have shown interest in the author and colleagues’ system seem to agree on one feature they wished were available: a metric that shows how much of the colon mucosa actually was inspected. To estimate this, the author and colleagues have developed a new method that generates a 3-D image out of 2-D images.²⁰ That which cannot be seen cannot be measured accurately, but by using a surface interpolation technique and several logical assumptions, the author and colleagues think that those areas of the colon that may have escaped inspection can be estimated with surprising usefulness and clarity. The colon is able to be displayed in 3-D similar to CTcolography and then virtual inspection—fly through—allowed of the simulated colon with an estimate of mucosa visualized as well as not visualized given a specific endoscope lens and presence or absence of tip deflection (ie, simulating the effort to look behind folds [Fig. 9]). The goal is to make this technique available in real-time providing periodic feedback (eg, once every several minutes or once every colon segment) to the endoscopist and thus provide the endoscopist the opportunity to go back and inspect areas previously missed.

Fig. 9.

Estimation of mucosal surface area seen. (A) A 2-D view of a relatively straight colon. (B) A 3-D representation in 2-D of (A). (C) A 3-D representation of (A) before fly through of the tip of the endoscope (gray/white cylinder). (D) A 3-D representation of (A) after fly through. Pink areas represent mucosa that can be inspected before fly through (C) and has been inspected after fly through (D) without tip deflection. Green areas represent mucosa that has not been inspected. The simulation estimates that in a relatively straight colon approximately 93% of the mucosal surface can be inspected with fly through without lateral tip deflection (D). (From Hong D, Tavanapong W, Wong J, et al. 3D Reconstruction of colon segments from colonoscopy images. In: IEEE Int’l Conf on Bioinformatics and Bioengineering. Taiwan 2009; p. 53–60.)

The ultimate goal of the author and colleagues’ work is to develop an automated, real-time, video stream–based analysis system that provides feedback to the endoscopist as needed to guarantee that each endoscopic procedure is of high quality. The system should be transparent for those who perform a high-quality procedure and provide suggestions to those who are not meeting established, scientifically verified minimal criteria. By off-loading repetitive, calculation-intensive algorithms to coprocessors on common off-the-shelf graphic cards and using—where possible— open source software components, the author and colleagues plan to keep the system affordable. All data, including any feedback provided and the response of the endoscopist to the feedback, will be recorded and can be—if desired—stored as part of a permanent electronic medical record.

The author and colleagues need to prove that such a system decreases the incidence of CCdC, provides all patients with a high-quality colonoscopy, and hopefully turns suboptimal endoscopists into good or excellent colonoscopists. An additional beneficial side effect of the technology likely will be that the interval between colonoscopies can be extended due to more effective and complete clearing of the colon, thereby reducing costs. Therefore, all parties involved—the patient, the endoscopist and the payers—in the long run stand to benefit from the system.

SUMMARY

In summary, the protective effect of colonoscopy is dependent on patient-, equipment-, and endoscopist-related factors. Of these three factors, endoscopist- and patient-related factors show most variation and are most difficult to assess. At present, objective methods to assess adequacy of the colonic preparation, the acquired skill set of the endoscopist (either at the end of training or after several years in practice), the true inspection time during withdrawal, and the effort exerted by the endoscopist to inspect all visible mucosa do not exist.

The author and colleagues present two methods to measure a more accurate estimation of quality of colonoscopy. The first method is to study the long-term outcome of all patients for a specific endoscopist; the measurement is CCdG, and the goal for the endoscopist is to detect and treat all precursor lesions and, therefore, to have no patients with CCdG. Because this method relies on long-term follow-up and does not provide protection for patients of endoscopists who perform less than optimal quality colonoscopy, the author and colleagues have developed a second method that automatically assesses a combination of intraprocedural factors that are commonly thought to greatly influence overall quality and outcome of a colonoscopic procedure. This second method currently is postprocedure, but the author and colleagues are converting it to real time (ie, during the procedure) with feedback to the endoscopist as needed. Finally, a combination of these two methods, as part of a rigorous prospective study, is required to validate that real-time analysis and feedback protects patients in the long term from CCdC.