Background
Colonoscopy is currently the preferred screening modality for prevention of colorectal cancer. Indeed most US citizens without specific risk factors for colorectal cancer are now advised to undergo a screening colonoscopy around age 50. This recommendation assumes that all large (>1 cm) polyps and colorectal cancers are identified during this initial screening procedure. However, recent data suggest that there is a significant miss-rate for detection of both large polyps and cancers. There are no objective data to explain why some large polyps and cancers are missed. We are developing a totally novel approach to automatically obtain objective documentation of a colonoscopic procedure.1 Here we present a software tool that allows rapid manual annotation of videos obtained during colonoscopy.
Hypothesis
Objective measurements regarding the appearance of the colonic mucosa, as well as the ability of the endoscopist to inspect the colonic mucosa and perform procedures can be derived from video files using computer algorithms.
Aims
Our aims are (1) to develop manual multimedia annotation software to annotate colonoscopy video files for endoscopic findings as well as diagnostic and therapeutic procedures, and (2) to develop a library of annotated images to be used to test and/or train an automated annotation system.1
Methods
Digitized Video Signal Capture
A moveable, prototype workstation was developed that consists of a PC with battery, a surge protector and a Dazzle DVC 150 or 200 video NTSC to MPEG-2 signal encoder. This workstation digitally captures and stores the complete video file generated during colonoscopy. The prototype system also has the ability to capture audio signals, which can be provided as needed via a switch on the chest of the endoscopist. Approximately one GigaByte of digital image file is created for every 20 minutes of colonoscopy. The Mayo Clinic IRB approved capture – in an anonymous fashion – of digital video files during routine endoscopic practice. Patients provide verbal consent and sign a HIPAA waiver.
Preprocessing Step
Natural language techniques, available as Open Source material, are used to convert the audio annotations present in the video files into text. To indicate progression throughout the procedure, standard statements were used, that can be recognized by the software and are used for file segmentation. Instrument recognition is based on the specific features of instrument insertion: it always is performed via the instrument channel which has an exit at a fixed location relative to the CCD camera, a specific cylindrical shape of the cable, and a direction of the shape towards the center of the image.
Manual Annotation Software
We created a user friendly, minimal learning requiring software tool that allows the user to rapidly find images based on audio annotation, colon segment, or the presence of an instrument within the image. Annotations are divided in diagnostic and therapeutic types using the European Society of Gastrointestinal Endoscopy (ESGE) Minimal Standard Terminology (MST).
Results
Using the capture system we created approximately 300 digital video files during endoscopy; approximately 250 of these files contain digitized images of a complete colonoscopic procedure. We are able to segment colonic video files into shots representing colon segments, such as rectum, sigmoid, descending colon, etc. We have used the fixed, expected location for instruments when present as the basis for an image recognition algorithm that detects the presence of instruments with high sensitivity and specificity, and thus the procedures performed by the endoscopist. Images can be selected from video files, specific image areas can be selected using configurable shapes (circles, ovals) and annotated using MST. Video clips can be created by defining the leading and last image of the video segment of interest. A file management system was implemented that allows all extracted images, annotations and video clips to be saved within a video file-related project folder.
Conclusion
Our software allows rapid creation of a video endoscopy library with classification of diagnostic findings and procedures based on the internationally accepted MST nomenclature developed by the ESGE. The library will be used to test and/or train our automated annotation system which is currently in development.1
Acknowledgement
This work is partially supported by the US National Science Foundation Grant No. 0092914, IIS-0513777, IIS-0513809, and IIS-0513582 and the Mayo Clinic
References
- 1.Hwang S, Oh J, Lee JK, et al. Automatic Measurement of Quality Metrics for Colonoscopy Videos. Proc. of The 13th ACM International Conference on Multimedia; 2005. pp. 912–921. [Google Scholar]