Abstract
Despite dramatic innovation in medical imaging and information system technologies, the radiology report has remained stagnant for more than a century. Structured reporting was created in the hopes of addressing well-documented deficiencies in report content and organization but has largely failed in its adoption due to concerns over workflow and productivity. A number of political, economical, and clinical quality-centric initiatives are currently taking place within medicine which will dramatically change the medical landscape including Pay for Performance, Evidence-Based Medicine, and the Physician Quality Reporting Initiative. These will collectively enhance efforts to improve quality in reporting, stimulate new technology development, and counteract the impending threat of commoditization within radiology. Structured reporting offers a number of unique opportunities and advantages over traditional free text reporting and will provide a means for the radiology community to add value to its most important service deliverable the radiology report.
Key words: Quality, reporting, pay for performance, evidence-based medicine
Historical Perspective
The radiology report is in many respects the single-most important basis on which radiologists are judged by their clinical colleagues.1 In the current practice environment, patients have little, if any, direct contact with the diagnostic radiologist interpreting their imaging exams, and their perception of the radiologist’s work is largely an extension of their referring clinicians’ perceptions. With the advent of picture archiving and communication systems (PACS), physicians are no longer held captive by the physical constraints of the medical imaging department and can now access imaging studies remotely, often without accessing the formal report issued by the radiologist. As a result, direct interactions between clinicians and radiologists have decreased dramatically with the adoption of filmless imaging.2 In this atmosphere, radiologists have begun to refine and redefine their service deliverables––with the radiology report as the most valuable element.
Although imaging technologies (the tools of the radiology trade) have undergone dramatic evolution during the past century, the radiology report has remained surprisingly static. The earliest radiology report, Eine Neue Art von Strahlen,3 was published in 1896 by Wilhelm Röntgen, who introduced a novel technology that could “see through human flesh” and would dramatically change future medical diagnosis. The occupational requirements for “roentgen photography” were yet to be determined, with photographers, electricians, physicists, and physicians with varied backgrounds all volunteering their services. After a period of adjustment not uncommon with new technologies, this discovery led to the creation of a new medical specialist––the roentgenologist––whose unique abilities were defined by the quality of the written report.
Speech has served as the principle input for reporting over the past century, with technological advances in the form of transcription aids. The earliest dictation machine was in use by the 1910s, imprinting recordings of the spoken word onto a wax cylinder for playback and transcription.4 These dictation devices were the mainstay for more than three quarters of a century until the advent of digital dictation, which today is giving way to speech recognition software that provides instantaneous computer-generated transcription. Although these advances have led to improved operational efficiency and timeliness in report turnaround, the end product has nonetheless remained static, both in content and structure. For better or worse, the free text (prose) radiology report has been the main deliverable for radiologists for over a century. The question to ask is whether this free text fixation is the result of report optimization, inertia, or cainophobia.
In order to answer this question, we must first explore the clinical, technical, and psychological aspects of medical reporting and determine whether an alternative approach is both clinically warranted and technically feasible. Here, the words of Winston Churchill seem timely: “There is nothing wrong with change, if it is in the right direction.”
Clinical Perspective
The attributes of a “good” radiology report can has been described by the eight Cs,1,5 although two additional critical components must also be included in the list (Table 1).
Table 1.
Clarity |
Correctness |
Confidence |
Concision |
Completeness |
Consistency |
Communication |
Consultation |
Timeliness |
Standardization |
The first C is clarity, which has been reported to be the single-most valuable attribute in a radiology report.6,7 The second is correctness, which is affirmed in the American College of Radiology (ACR) Standard for Communication,8 which states, “the radiology report should contain a precise diagnosis whenever possible.” The third C is confidence, which describes the level of certainty that can be attributed to the observed findings. The fourth is concision––the ability to report findings with brevity. It has been noted that the length of the radiology report tends to vary inversely with the confidence and preparation of the radiologist.9 The fifth C is completeness, which allows the clinician to derive the maximum amount of significant clinical information associated with the findings. Not all radiologic findings are deemed clinically relevant, and the distinction of clinical significance is often left to the referring clinician, so the completeness of reporting becomes crucial.10 The sixth C is consistency, which is important in ensuring that components remain the same throughout the report. If a finding is reported as “right sided” in the body of the report, this should be repeated in the impression of the report to avoid confusion.
The seventh and eighth Cs are communication and consultation, which take on greater importance in the current radiology practice. With the heightened focus and attention on medical errors and adverse clinical outcomes,11,12 communication of emergent and clinically unexpected findings has taken on greater importance and necessitates documentation within the radiology report.13,14 Proactive consultation between clinician, radiologist, and patient has been shown to enhance clinical outcomes and reduce medico-legal liability.15–17 This is particularly relevant to mammography, where the Mammography Quality Standards Act (MQSA) has mandated communication and consultation requirements for breast cancer imaging.18
Two additional report attributes of a “good” radiology report are timeliness and standardization. Timeliness is an essential component of ensuring that clinical management is expedient. With the widespread adoption of computerized imaging and information systems technologies (e.g., PACS), access to medical imaging data is both ubiquitous and instantaneous. With the advent of teleradiology and speech recognition software, medical imaging providers can provide reports on a 24/7 basis, placing greater emphasis on the timeliness of service. The last report attribute, standardization, is perhaps the most challenging for the radiology community as a whole. This requires the creation, acceptance, and widespread adoption of a universal lexicon, which would improve the quality and accuracy of radiology communication.19,20 In reality, however, this is no small undertaking and has been attempted by a number of initiatives including RadLex, BI-RADS, the Systematized Nomenclature of Medicine, the Unified Medical Language System, the Fleischner Society Glossary, and Digitial Imaging Communications in Medicine.21 Unfortunately, at the present time, none of these lexicons are comprehensive in nature and cover the full spectrum of anatomy, pathology, and modality within medical imaging.
Psychological Perspective
Mark Twain said, “It’s not the progress I mind, it’s the change I don’t like.” Resistance to imposed change is engrained in the human psyche. Because change is an essential element in the modern workplace, an entire discipline of the social and psychological sciences now focuses on change management. Change management is the process of assisting individuals and operations in transitioning from one system to another.22 The major challenges to successful technology adoption are as much behavioral as technical.23 People (or organizations) with low “psychological ownership” in such change can actively resist implementation of superior technologies and induce failure. Effective leadership and strategic planning must therefore anticipate and proactively address this innate resistance to change.
One of the common misconceptions is that because technology investments consist in part of tangible objects (hardware and software) and then the technology is fully implemented at installation. This overlooks the important fact that technology does not operate in isolation but requires people to maintain performance and achieve the desired goals. When these “human issues” are overlooked, technically sound applications often fail. The key to successful change management lies in understanding the combined importance of psychological, technical, political, and operational issues. This was emphasized by Gardner, who said24 “In my opinion, the success of a project is perhaps 80% dependent on the development of the social and political interaction skills of the developer and 20% or less on the implementation of the hardware and software technology.”
Oddly enough, most human-oriented change theories have been derived from mathematics and physics. Watzlawick et al.25 selected two theories derived from mathematical logic a framework of “first- and second-order” changes. First-order change was defined as a variation in the manner in which processes or procedures have been previously performed, thereby leaving the system relatively unchanged. Second-order change was defined as a more dramatic alteration in the overall system, with a redefinition or reconceptualization in the manner in which processes or procedures are performed. Golembiewski et al.26 went on to add a third level of change, defined as “middle-order” change.
A relevant example of these three levels of change can be seen with current mammography reporting strategies. The advent of digital dictation represented a first-order change and provided consulting physicians with direct access to the audio file of the radiologists’ transcribed report. The overall reporting process (from the radiologist’s viewpoint) was essentially unchanged, but the new technology afforded access to report data that previously did not exist with conventional dictation. A middle-order change occurred with the introduction of the BIRADS system from the ACR. This reporting system created standardization of mammography reports using an established lexicon and categorization of findings with recommendations for clinical and imaging follow-up. This entailed a change in radiologist workflow and report creation without adoption of new technology. The third-order change in mammography reporting has occurred with the adoption of speech recognition technology, which calls for the integration of new software into the RIS, allowing the radiologist to create and edit the mammography report directly onto a computer workstation (thereby eliminating the paper report). This has the advantage of creating a finalized report at the time of image interpretation, while creating both technical and workflow challenges for the radiologist. Each successive order of change brings an escalating level of modification to existing workflow and technology.
An alternative to the more abstract models of Watzlawick and Golembiewski has been proposed by Lorenzi and Riley.23 This model divides change into microchange and macrochange. A microchange simply involves a modification to an existing process or technology, whereas a macrochange encompasses a major change or revision. If we were to use the example of a PACS, a microchange involves a software upgrade, and a macrochange involves a replacement of one vendor’s PACS with another. One must remember, however, that perspectives vary on what constitutes a micro- or a macrochange. A PACS administrator may view a recent software upgrade as a minor change to the system and accept it easily. A radiologist on the other hand, who is comfortable with the existing electronic hanging protocols, may object vehemently to a “new and improved” upgrade that dramatically changes these hanging protocols and alters his/her workflow. As a result, these two professionals view the same software upgrade in far different terms, with differing levels of acceptance.
If we want to introduce innovation to reporting in the form of structured reporting, we must take these important principles into account and develop effective strategies to counteract the natural and inevitable human resistance to change.
Stakeholder Perspective
If one wants to affect change in radiology reporting, it is critical to remember that unique differences exist in individual stakeholder perspective (Table 2). The principal stakeholders in reporting include the radiologist (who creates the report), the clinician (who acts on the report), the administrator (who is tasked with analyzing reports), the patient (who is the principle party in the recorded data), and the payer (who determines the economic value of the report). Within each stakeholder group, variability exists as to quantifying report quality. Radiologists, for example, are a heterogeneous group and are comprised of general and sub-specialists, academic and community practitioners, and salaried and fee-for-service providers. Each if these distinguishing characteristics may play a role in how radiologists perceive report quality and their willingness to embrace change. Certain common perceptions do exist, and these serve as the focal point of discussion.
Table 2.
Radiologist: End-user workflow and productivity |
Clinician: Timeliness and quality of report content |
Administrator: Compliance with regulations and standards |
Patient: Communication and accessibility of reporting content |
IT: Technology and data integration |
Payer: Data mining and outcomes analysis |
The radiologist community has been relatively recalcitrant to reporting changes and is, to a large extent, driven by concerns over workflow and productivity optimization. As the size, complexity, and number of medical imaging studies have been rapidly increasing; the radiologist community has turned to workflow-enabling technology to address these demands. Examples of workflow-enhancing technologies include PACS, computerized multiplanar reconstructions, and computer-aided detection software (CAD), which assist in the steps of image storage and presentation, processing, and interpretation, respectively. To date, one could argue that effective workflow-enhancing technology does not exist for reporting. In fact, the most important technology innovation within reporting in the last decade is speech recognition software, which has been reported to decrease radiologist productivity,27,28 largely due to additional time requirements associated with radiologist report editing. As a result, the creation of a workflow-enhancing reporting alternative would be viewed in a positive light by the radiologist community, assuming that comparable report quality standards are maintained.
The clinical community is also heterogeneous in nature and divided among several lines; the most important of which relate to occupation, education, and training. For a given radiology report (e.g., mammography), different perspectives exist for a primary care physician, surgeon, medial oncologist, and nurse practitioner. Some clinicians (e.g., surgeon) are apt to review imaging and report data in tandem, while others (e.g., primary care physician) more likely to review report data without image correlation. As a result of these differences, many clinicians would see value in the ability to customize report data in a manner specific to his/her own unique preferences.
Regardless of these occupational and individual clinician differences, almost all clinicians place a high priority on report timeliness, content quality, and communication of pertinent findings. A reporting system that addresses these clinical priorities through automated (and verifiable) delivery, documented critical results reporting, and customizable content creation will be viewed in a positive light. Several studies to date have shown that most clinicians prefer itemized structured reports to their free text, prose counterparts.29,30 As a result, new reporting strategies should strive to provide customized data presentation states, in a manner deemed of greatest value to the clinical end-user.
Hospital and departmental administrators may demonstrate some variability in keeping with institutional variations yet share several commonalities. Perhaps the most important priority to this group as it relates to reporting is compliance with communitywide standards and governmental regulations. This is most prevalent in mammography, due to the aforementioned MQSA requirements, which mandate timely report communication, documentation of receipt and recommended follow-up, and calculation of reporting accuracy (through positive and negative predictive values). Even though mammography has a standardized lexicon in the form of BI-RADS, compliance is far from universal, and the derived report data is often not in a standardized format. As a result, many of the MQSA-mandated requirements are often manual in nature, requiring significant resource expenditures (in personnel time) and subject to human error. A new reporting strategy would have combined clinical, medico-legal, and economic value if it were to automate many of these reporting requirements through prospective creation of standardized data, formulation of structured databases, automated data mining, and derived outcomes analysis. The ability to create structured databases and perform automated data mining has a multitude of applications relating to research, education and training, and establishment of evidence-based medicine guidelines.
Patient empowerment is a concept gaining great attention in the medical literature,11,12,31,32 with a shift from the traditional provider-centric model of medical practice to a patient-centric model. In a patient-centric practice model, patients would be extended specific rights which empower them and provide the means with which to take control over their medical decision-making. The five rights intrinsic to this “Patient Bill of Rights” include self determination, informed consent, consultation, notification, and designation. All of these rights are directly applicable to the radiology report, and the data within the radiology report needs to be presented to the patient in an understandable and easily accessible manner. The ability to present corresponding data from the aforementioned structured report database provides an important mechanism for patients and third party payers to access clinical outcome data, which is essential to making well informed and educated decisions. The derived data can be used to assist in the determination of exam appropriateness, selection of institutional and individual service providers, and risk/benefit analysis associated with imaging and interventional procedures. An educated and well informed consumer is an asset to the medical (imaging) community and can serve as an important stimulus to continued technology and clinical innovation.
Another stakeholder is the information technology (IT) specialist, who plays a vital role in medical imaging today, with the widespread adoption of computerized information systems technologies (e.g., RIS, PACS, EMR). One of the many responsibilities of the IT specialist is to ensure data integration and accessibility. In the case of the medical imaging, the imaging data, report data, and clinical data must all be accessible to the end-user, preferably in a single application. Unfortunately, in current reporting system, this consists of three distinct datasets, which are derived from three different forms of technology: the imaging modality (imaging data), the PACS (report data), and the EMR (clinical data). An ideal solution would be to combine all three forms of data into a single report, thereby allowing the end-user to simultaneously access all relevant imaging, report, and clinical data at a single source, independent of technology integration.
Innovation Opportunities
The ongoing focus of cost savings in healthcare has led to the perception of radiology as a commodity, where selection of service provider is largely based upon cost. This commoditization trend has been accelerated by technology development; in the form of globalization of imaging services (e.g., teleradiology), increased data exchange (e.g., DICOM), and the creation of decision support technologies (e.g., CAD). The optimal strategy for counteracting this commoditization trend is the creation of quality metrics and standards throughout the medical imaging practice, which will provide an objective and reproducible means for qualitative differentiation. The single-most important service deliverable which the radiology community has to advance this effort is the radiology report, which will serve as the focal point of quality-centric innovation in the near future.33
Fortunately for the medical community at large, a number of timely political, economic, and clinical initiatives have recently taken place which serves as a catalyst for quality-centric innovation throughout medical practice. On the political front, the IOM publications11,12,31 calling for sweeping quality-centric medical reforms have led to a number of legislative actions including the Medicare Modernization Act of 2004, the Medicare Value Purchase Act of 2005, and the Tax Relief and Healthcare Act of 2006. These bills have called for the development of value-based purchasing programs under Medicare, creation of Pay for Performance (P4P), acquisition of electronic information technology, and the public dissemination of quality information to consumers. The net effect of these combined initiatives is the promotion of quality throughout healthcare through financial incentives, creation of an electronic infrastructure, and quality-centric data repositories.
In response to this legislation, the Physician Quality Reporting Initiative was created, which identifies specific report quality indicators which are required in order to qualify for P4P reimbursements. Over time, the number of reimbursable report quality indicators will increase, creating greater economic incentives for imaging providers to modify report content. The documentation of quality indicators in medical reporting is not exclusive to radiology and has been advocated in several other medical disciplines including gastroenterology,34,35 pathology,36 and cardiology.37 As the number and scope of these report quality indicators continues to expand, the standardization of medical reporting will follow. Use of nonstandardized language has been shown to degrade the quality and accuracy of radiology communication20,38 while the introduction of BI-RADS (a breast imaging lexicon) has resulted in reduced variability and improved clarity of communication.39
The incorporation of these report quality indicators is also an integral component of evidence-based medicine (EBM), which has been defined as the process of systematically finding, appraising, and incorporating contemporary research findings into clinical decision-making. The principle goal of EBM is to provide a conscientious scientific basis for clinical decision making. In doing so, EBM serves as a methodologic strategy to streamline and objectify the decision-making process, by utilizing data derived from the peer-reviewed scientific literature, databases, and clinical trials.
While EBM serves as the impetus for data-driven best practice guidelines, its applicability within radiology is currently limited by the non-standardized radiology report. Without report standardization, it is impractical to perform large-scale report data mining; which serves as a prerequisite to future efforts in clinical research, outcomes analysis, education, and training and the establishment of best practice guidelines.
In order to meet these combined political, economic, and clinical demands, it is essential that the radiology community transitions to structured reporting, but this will not be an easy task. A number of clinical, technical, and psychological challenges must be first addressed in order to gain wide-scale acceptance within the radiologist community. In the upcoming series of articles, we will explore these challenges and use data-driven studies to identify the optimal approach for future innovation.
Acknowledgement
This work was supported by an educational/research grant from Fuji Medical Systems, USA.
References
- 1.Reiner BI, Siegel EL, Knight N. Radiology reporting: past, present, and future: the radiologist perspective. J Am Coll Radiol. 2007;5:313–319. doi: 10.1016/j.jacr.2007.01.015. [DOI] [PubMed] [Google Scholar]
- 2.Reiner B, Siegel E, Protopapas Z, et al. Impact of filmless radiology on the frequency of clinician consultations with radiologists. AJR. Am J Roentgenol. 1999;173:1169–1172. doi: 10.2214/ajr.173.5.10541082. [DOI] [PubMed] [Google Scholar]
- 3.Röntgen WK. Eine Neue Art von Strahlen. Würzburg, Germany: Medicophysical Institute of the University of Würzburg; 1896. [Google Scholar]
- 4.Reiner BI, Siegel EL, Knight N. The evolution of the radiology report and the development of speech recognition. In: Reiner BI, Siegel EL, Weiss DL, editors. Electronic Reporting in the Digital Medical Enterprise. Great Falls: Society for Computer Applications in Radiology; 2003. pp. 1–7. [Google Scholar]
- 5.Armas RR. Qualities of a good radiology report [letter] AJR. Am J Roentgenol. 1998;170:1110. doi: 10.2214/ajr.170.4.9530077. [DOI] [PubMed] [Google Scholar]
- 6.Lafortune M, Breton G, Baudouin JL. The radiological report: what is useful for the referring physician? Can Assoc Radiol J. 1988;39:140–143. [PubMed] [Google Scholar]
- 7.McLoughlin RF, So CB, Gray RR, et al. Radiology reports: how much descriptive detail is enough? AJR. Am J Roentgenol. 1995;165:803–806. doi: 10.2214/ajr.165.4.7676970. [DOI] [PubMed] [Google Scholar]
- 8.American College of Radiology: ACR Standard for communication: diagnostic radiology. In: Standards 2000–2001. Reston, VA: American College of Radiology, 2000, pp 1–3
- 9.Hall FM. Language of the radiology report: primer for residents and wayward radiologists. AJR. Am J Roentgenol. 2000;175:1239–1242. doi: 10.2214/ajr.175.5.1751239. [DOI] [PubMed] [Google Scholar]
- 10.Rothman M. Malpractice issues in radiology: radiology reports [letter] AJR. Am J Roentgenol. 1998;170:1108–1109. doi: 10.2214/ajr.170.4.9530073. [DOI] [PubMed] [Google Scholar]
- 11.To Err is Human: Building a Safer Health System. Washington DC: National Academy Press; 2000. [PubMed] [Google Scholar]
- 12.Crossing the Quality Chasm: A New Health System for the 21st Century. Washington DC: National Academy Press; 2001. [PubMed] [Google Scholar]
- 13.Singh H, Arora HS, Vij MS, et al. Communication outcomes of crtical imaging results on a computerized notification system. J Am Med Inform Assoc. 2007;14:459–466. doi: 10.1197/jamia.M2280. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Hanna D, Griswold P, Leape LL, et al. Communicating critical test results: safe practice recommendations. Jt Comm J Qual Patient Saf. 2005;31:68–80. doi: 10.1016/s1553-7250(05)31011-7. [DOI] [PubMed] [Google Scholar]
- 15.Badley BW. A clinician’s expectation of the radiologist. J Can Assoc Radiol. 1977;28:285–286. [PubMed] [Google Scholar]
- 16.Dijk CN, Leeuw PA. Imaging from an orthopedic point of view: what the orthopedic surgeon expects from the radiologist. Eur J Radiol. 2007;62:2–5. doi: 10.1016/j.ejrad.2007.01.016. [DOI] [PubMed] [Google Scholar]
- 17.Mozumdar BC, Jones G. Medico-legal issues in radiological consultation. Radiol Manage. 2003;25:40–43. [PubMed] [Google Scholar]
- 18.US Food and Drug Administration (2009) http://www.fda.gov/CDRH/MAMMOGRAPHY
- 19.Weiss DL, Langlotz CP. Structured reporting: patient care enhancement of productivity nightmare? Radiology. 2008;249:739–747. doi: 10.1148/radiol.2493080988. [DOI] [PubMed] [Google Scholar]
- 20.Khorasani R, Bates DW, Teeger S, et al. Is terminology used effectively to convey diagnostic certainty in radiology reports? Acad Radiol. 2003;10:685–688. doi: 10.1016/S1076-6332(03)80089-2. [DOI] [PubMed] [Google Scholar]
- 21.Hall FM. The radiology report of the future. Radiology. 2009;251:313–316. doi: 10.1148/radiol.2512090177. [DOI] [PubMed] [Google Scholar]
- 22.Lorenzi NM, Riley RT. Organizational issues equals change. Int J Med Inform. 2003;69:197–203. doi: 10.1016/S1386-5056(02)00105-3. [DOI] [PubMed] [Google Scholar]
- 23.Lorenzi NM, Riley RT. Managing change: an overview. J Am Med Inform Assoc. 2000;7:116–124. doi: 10.1136/jamia.2000.0070116. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Gardner D. Davies keynote lecture. Proceedings of the Computer-based Patient Record Institute. Washington: CPRI; 1998. [Google Scholar]
- 25.Watzlawick P, Weakland JH, Fisch R. Change: Principles of Problem Formation and Problem Resolution. New York: Norton; 1974. [Google Scholar]
- 26.Golembiewski RT, Billingsley K, Yeager S. Measuring change and persistence in human affairs: types of change generated by OD designs. J Appl Behav Sci. 1976;12:133–157. doi: 10.1177/002188637601200201. [DOI] [Google Scholar]
- 27.Bhan SN, Coblentz CL, Norman GR, et al. Effect of voice recognition on radiologist reporting time. Can Assoc Radiol J. 2008;59:203–209. [PubMed] [Google Scholar]
- 28.Rana DS, Hurst G, Shepstone L, et al. Voice recognition for radiology reporting: is it good enough? Clin Radiol. 2005;60:1205–1212. doi: 10.1016/j.crad.2005.07.002. [DOI] [PubMed] [Google Scholar]
- 29.Plumb AA, Grieve FM, Khan SH. Survey of hospital clinciains’ preferences regarding the format of radiology reports. Clin Radiol. 2009;64:386–394. doi: 10.1016/j.crad.2008.11.009. [DOI] [PubMed] [Google Scholar]
- 30.Naik SS, Hanbridge A, Wilson SR. Radiology reports: examining radiologist and clinician preferences regarding style and content. Am J Roentgenol. 2001;176:591–598. doi: 10.2214/ajr.176.3.1760591. [DOI] [PubMed] [Google Scholar]
- 31.Achieving a New Standard for Care. Washington DC: National Academy; 2004. [Google Scholar]
- 32.Reiner BI. Quantifying radiation safety and quality in medical imaging. Part I: creating the infrastructure. J Am Coll Radiol. 2009;8:558–561. doi: 10.1016/j.jacr.2009.04.007. [DOI] [PubMed] [Google Scholar]
- 33.Reiner BI, Siegel EL. Decommoditizing radiology. J Am Coll Radiol. 2009;3:167–170. doi: 10.1016/j.jacr.2008.11.004. [DOI] [PubMed] [Google Scholar]
- 34.Lieberman DA, Faigel DO, Logan JR, et al. Assessment of the quality of colonoscopy reports: results of a multi-center consortium. Gastrointest Endosc. 2009;69:645–653. doi: 10.1016/j.gie.2008.08.034. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Lieberman D, Nadal M, Smith RA, et al. Standardized colonoscopy reporting and data system: report of the Quality Assurance Task Group of the national Colorectal Cancer Roundtable. Gastrointest Endosc. 2007;65:757–766. doi: 10.1016/j.gie.2006.12.055. [DOI] [PubMed] [Google Scholar]
- 36.Mammen JJ, Tuthill JM. Structured data in pathology reports: overcoming challenges with new tools. AMIA Annu Symp Proc. 2008;6:1041. [PubMed] [Google Scholar]
- 37.Sundaram B, Patel S, Bogot N, et al. Anatomy and terminology for the interpretation and reporting of cardiac MDCT: part I, structured report, coronary calcium screening, and coronary artery anatomy. Am J Roentgenol. 2009;192:574–583. doi: 10.2214/AJR.08.1177. [DOI] [PubMed] [Google Scholar]
- 38.Kong A, Barnett GO, Mosteller F, et al. How medical professionals evaluate expressions of probability. N Engl J Med. 1986;315:740–744. doi: 10.1056/NEJM198609183151206. [DOI] [PubMed] [Google Scholar]
- 39.Lazarus E, Mainiero MB, Schepps B, et al. BI-RADS lexicon for US and mammography: inter-observer variability and positive predictive value. Radiology. 2006;239:385–391. doi: 10.1148/radiol.2392042127. [DOI] [PubMed] [Google Scholar]