Abstract
BACKGROUND
Reporting standards promote clarity and consistency of stress myocardial perfusion imaging (MPI) reports, but do not require an assessment of post-test risk. Natural Language Processing (NLP) tools could potentially help estimate this risk, yet it is unknown whether reports contain adequate descriptive data to use NLP.
METHODS
Among VA patients who underwent stress MPI and coronary angiography between January 1, 2009 and December 31, 2011, 99 stress test reports were randomly selected for analysis. Two reviewers independently categorized each report for the presence of critical data elements essential to describing post-test ischemic risk.
RESULTS
Few stress MPI reports provided a formal assessment of post-test risk within the impression section (3%) or the entire document (4%). In most cases, risk was determinable by combining critical data elements (74% impression, 98% whole). If ischemic risk was not determinable (25% impression, 2% whole), inadequate description of systolic function (9% impression, 1% whole) and inadequate description of ischemia (5% impression, 1% whole) were most commonly implicated.
CONCLUSIONS
Post-test ischemic risk was determinable but rarely reported in this sample of stress MPI reports. This supports the potential use of NLP to help clarify risk. Further study of NLP in this context is needed.
INTRODUCTION
Approximately 4 million stress tests are performed annually in the United States, nearly 90% of which are done with cardiac imaging, such as single photon electron computed tomography (SPECT) or other nuclear myocardial perfusion imaging (MPI) methods [1]. Based on stress test results, clinicians make an assessment of post-test ischemic risk that is critical for directing downstream care. For example, whether stress MPI findings are low, moderate or high-risk determines the appropriateness of coronary angiography and revascularization [2,3].
Clinicians’ assessment of risk starts with the nuclear cardiology report. Many of the required reporting elements for stress MPI are designed to help clinicians determine post-test risk [4,5]. Unfortunately, recent evidence from a large accrediting body found that between 20 and 50 percent of reports in accredited facilities do not adhere to reporting standards [6]. Even given a structured description of ischemic lesions, one study showed that referring providers frequently underestimate the extent of ischemia [7]. So while efforts to accelerate adoption of stress MPI reporting standards are likely to improve the overall clarity and consistency of reports, this may not ensure that the clinical significance of results are understood by referring providers.
Data analytic tools such as natural language processing (NLP) could be used to augment providers understanding of stress MPI results. NLP has previously been used in radiology and other fields to process free text medical reports in order to facilitate billing, assess the quality of reporting and, as it relates to the current study, underscore pertinent or high-risk findings [8]. Using this approach, an NLP tool could conceivably be used to interpret stress MPI reports, extracting unstructured information to provide structured estimates of post-test ischemic risk.
An NLP approach to estimate risk is only viable, however, if the underlying reports include all of the critical data elements required for accurate risk estimation. Accordingly, in our study, we used a national sample of stress MPI reports from the Veterans Health Administration (VA) to assess whether these reports contained all the necessary data elements for determining risk, namely the post-test risk used to guide appropriateness of coronary revascularization [2]. For comparison, we also hoped to quantify the percentage of reports that explicitly commented on the clinical significance of the results.
METHODS
Setting, patient population, and data source
Figure 1 describes how we identified and acquired our sample of stress MPI reports. Given our focus on ischemic risk estimation, we sought to enrich our sample corpus with reports from patients likely to have abnormal findings. We therefore limited our initial document search to VA patients who underwent cardiac catheterization as captured by the VA Clinical Assessment Reporting and Tracking Program (CART) [9]. In these patients, we searched the VA Corporate Data Warehouse (CDW) for any stress test report between January 1, 2009 and December 31, 2011 based on Current Procedural Terminology (CPT) and International Classification of Diseases Ninth Revision (ICD-9) codes for myocardial nuclear imaging (78451–78454, 78460, 78461, 78464, 78465, 78472, 78473, 78481, 78483, 78491, 78492, 89.44). Reports that did not contain VA station number (facility identifier), clinical history, report text, or impression text were excluded.
FIGURE 1.
Acquisition of a sample of stress myocardial perfusion imaging reports within the VA corporate data warehouse
Figure 1 describes how the corpus of stress MPI reports was acquired. Given the focus on post-test risk estimation, the authors limited the initial search to only VA patients who underwent cardiac catheterization as captured by the VA CART Program (120,580 patients). The records of these patients were then searched using the VA Corporate Data Warehouse (CDW) for any stress test report between January 1, 2009 and December 31, 2011 based on CPT and ICD-9 codes for myocardial nuclear imaging (45,321 reports). Reports that did not contain station number, clinical history, report text, impression text or relevant text strings (e.g. “Spect”, “Myoview”, etc.) were excluded (22,981 reports). From this larger corpus, 4000 reports were randomly selected using proportional stratification based on VA station. To eventually allow iterative improvement of an NLP tool, these 4000 documents were partitioned into 20 groups of 200 documents. One of those groups was randomly selected for a pilot analysis. 113 nuclear imaging reports were randomly selected from this group, of which 99 were stress nuclear MPI studies.
To confine the corpus to stress MPI reports only, we included only documents containing one of the following text strings, “spect”, “pet”, “technetium”, “myoview”, “sestamibi”, “tc-99”, tetrofosmin”, “thallium”, “radionuclide”, “isotope”, “radiopharm”, or “radioisotop” or the space-separated strings “mci ” or “ tc “ anywhere in the fields of name of test, report text, or impression text.
The CART database used to define our study population contained a total of 112,784 catheterization procedures between 1/1/2009 and 12/31/2011, carried out in 112,580 distinct patients. Within this date range, 45,321 of these patients underwent 45,363 cardiac stress tests. Using the preceding inclusions and exclusions to search for stress MPI reports in the VA CDW, our initial document corpus contained 22,981 reports collected from 22,981 distinct patients across 68 separate VA stations. From this larger corpus, we randomly selected 4000 reports using proportional stratification based on VA station. To eventually allow iterative improvement of an NLP tool, these 4000 documents were partitioned into 20 groups of 200 documents.
Before we could embark on developing an NLP tool, we needed to understand the composition and quality of the stress MPI reports captured by our search. Namely, we hoped to determine whether reports consistently contained enough unstructured data for an NLP tool to estimate post-test risk. Therefore, we randomly selected one group of 200 documents for a pilot analysis. The randomly generated document corpus included 113 studies; 14 of these studies were not nuclear myocardial perfusion studies. The remaining 99 reports from 44 different VA stations were included in our analysis.
Assessment of Stress MPI Documentation Quality
We focused our efforts on documentation of stress MPI findings and our ability to determine post-test ischemic risk based on the description of relevant imaging findings. Critical data elements for risk estimation, listed in Table 1, were defined by collating required and recommended inputs from the 2009 guidelines for standardized reporting of nuclear MPI with clinically important elements from the 2009 appropriateness criteria for coronary revascularization [2,4]. This list is consistent with recommendations from the Intersocietal Accreditation Commission for nuclear cardiology, nuclear medicine and PET laboratories (IAC Nuclear/PET, formerly ICANL) [5]. And these data elements represent the key ingredients for a natural language processing system to determine patients’ post-test risk from a nuclear MPI report.
TABLE 1.
description of critical data elements in a stress nuclear MPI report
| Data Element | Example descriptors |
|---|---|
| Perfusion defect: Infarct | Infarct; Scar; Fixed; Non-reversible; present during rest and stress |
| Perfusion defect: Ischemia | Ischemia; Reversible; present during stress only; worse with stress; inducible; impaired perfusion reserve |
| Perfusion defect size | Small, Medium, Large; Number of segments (17 segment model); Percentage of myocardium (estimated or calculated) |
| Perfusion defect location | 17 segment model; Anterior, Lateral, Inferior, Septal, etc. |
| Perfusion defect severity | Mild, Moderate, Severe |
| Summed rest/stress/difference | Numerical score: 0 to 68 |
| Transient ischemic dilation | Numerical score (e.g. 1.10); Present or Absent |
| LV systolic function | Ejection fraction (5–80%); Mildly, moderately or severely depressed |
| Risk assessment | Low risk, Moderate risk, High risk |
Describes the critical data elements included in the authors’ analysis of stress test reports. Critical data elements included: the presence or absence of ischemia, including implied terms like “scar” for infarct or “reversible defect” for ischemia; the size, location, and severity of a perfusion abnormality; summed rest, stress, and difference scores; presence or absence of transient ischemic dilation; description of left ventricular systolic function, including a quantitative ejection fraction or qualitative descriptors such as “moderately depressed” or “normal”; a formal post-test risk assessment by the report author.
Two reviewers (AL, SB) independently analyzed the 99 stress MPI reports in our sample. Reviewers examined each test report for the presence or absence of critical data elements, both within the impression alone and within the report in its entirety. The presence or absence of critical data elements was adjudicated according to prespecified rules.
Regarding the presence or absence of a perfusion abnormality, if the words “ischemia” or “infarct” were not explicitly used, the reviewers attempted to identify terms that might otherwise be used to describe these findings (e.g. “scar” to describe infarct). We called these ‘implicit’ descriptions. In terms of the characteristics of a perfusion abnormality, we looked for descriptors of size, location, and severity of the lesion. We also assessed each report for the presence or absence of summed rest (SRS), stress (SSS), and difference (SDS) scores as well as the presence or absence of transient ischemic dilation (TID). With regards to left ventricular systolic function, we accepted either a quantitative ejection fraction or qualitative descriptors such as “moderately depressed” or “normal.” Finally, we assessed the reports for the presence or absence of a formal risk assessment by the interpreting physician.
Within this framework, the reviewers then independently graded each report for its ability to confer post-test ischemic risk based on the 2009 appropriateness criteria for coronary revascularization [2]. If the appropriateness of revascularization could be determined based on available report elements, we considered post-test ischemic risk to be “determinable.” Specifically, risk was considered determinable if the report described the following basic elements: presence or absence of ischemia; description of ischemic lesion if present; description of left ventricular systolic function (Figure 2).
FIGURE 2.
the ability to determine post-test ischemic risk based on critical data elements in a stress nuclear MPI report
Figure 2 describes how critical data elements were used to assess whether ischemic risk was “determinable” or “not determinable”. If a formal ischemic risk assessment was not provided in the stress test report, the authors assessed the extent to which risk was “determinable.” Risk was considered “determinable” if the report described the following basic elements: presence or absence of ischemia; description of ischemic lesion if present; description of left ventricular systolic function. The inset table provides reasons why risk was not determinable based on review of the impression section alone (n=25) and considering the entire report (n=2).
After initial ratings were made by each reviewer, final grades were assigned based on consensus between the reviewers. Interrater agreement was assessed by Cohen’s kappa calculations.
RESULTS
Impression-Only Review
Results with respect to both the impression section alone and the full report are summarized in Table 2. In review limited to the ‘impression’ section of 99 stress MPI reports, the presence or absence of ischemia was directly reported in 70 (71%) reports. In an additional 21 (21%) reports, ischemia was described in other terms, most commonly as a “reversible” defect (10 of 21). Meanwhile, the presence or absence of infarct was directly reported in 22 (22%) reports and implicitly reported in another 35 (35%) reports using terms such as “scar” (9 of 35) and “fixed” defect (10 of 35). Of the 67 cases (68%) where ischemia or infarct was reported, 65 (94%) commented on the location, 37 (57%) the severity, and 36 (55%) the size of the abnormality. An ejection fraction was reported in 79 (80%) reports. Sum stress, rest or difference scores were provided in 6 (6%) reports and an assessment of transient ischemic dilation was provided in 13 (13%) cases. A formal post-test ischemic risk assessment was provided in 3 (3%) reports. When a formal statement of risk was not explicitly provided, ischemic risk was determinable in 71 (74%) cases. Of the 25 (26%) cases where risk was not determinable, inadequate description of LV systolic function (9 of 25), inadequate description of ischemia (5 of 25), or both (11 of 25) were the most common deficiencies in reporting (Figure 2).
TABLE 2.
incidence of critical data element reporting in a sample of Stress Nuclear MPI reports
| Critical Data Element | Impression | Entire Report |
|---|---|---|
| Perfusion defect: Ischemia | 91 of 99 | 95 of 99 |
| Perfusion defect: infarct | 57 of 99 | 65 of 99 |
| Perfusion defect size | 36 of 67 | 44 of 67 |
| Perfusion defect location | 65 of 67 | 67 of 67 |
| Perfusion defect severity | 37 of 67 | 49 of 67 |
| SRS/SSS/SDS Scores | 6 of 99 | 13 of 99 |
| Transient Ischemic Dilation | 13 of 99 | 36 of 99 |
| LV Systolic Function | 79 of 99 | 99 of 99 |
| Risk Assessment | 3 of 99 | 4 of 99 |
| Risk Determinable | 74 of 96 | 93 of 95 |
Describes critical data element reporting in our sample of stress nuclear MPI reports. The frequency with which critical data elements were described is reported, separately considering the entire report and the impression section of the report. If a formal risk assessment was not provided, the authors assessed the extent to which post-test risk was “determinable” according to the aforementioned criteria (Figure 2). The number of reports for which risk was considered “determinable” is also provided.
Full Report Review
Analyzing the 99 reports in their entirety, the presence or absence of ischemia was directly reported in 72 (73%) reports, while an additional 23 cases (23%) used other terms such as a “reversible” defect (15 of 23). Meanwhile, the presence or absence of infarct was directly reported in 25 (25%) reports and implicitly reported in another 40 cases (40%), using the terms “scar” (11 of 40) and “fixed” defect (11 of 40). Of 67 cases (68%) where an ischemic or infarct lesion was described, 44 (66%) commented on the size, 49 (73%) the severity, and 67 (100%) the location of the abnormality. An ejection fraction was given in 99 (100%) studies. Sum stress/difference scores and TID were reported in 13 (13%) and 36 (36%) cases, respectively, an improvement from the impression alone. Ischemic risk was judged to be determinable in 97 (98%) cases based on critical elements in the report. Of the two cases where risk could not be determined from report elements, one case did not adequately describe LV systolic function and the other case contained no description of an ischemic lesion. A formal post-test risk assessment was provided in 4 (4%) reports.
Interrater Agreement
There was strong agreement between the reviewers in the individual assessment of the measured elements of stress test reporting (2519 of 2574 measures; 97.9%; Cohen’s kappa 0.89). There was also strong agreement in the individual assessment of determinable risk from stress reports (189 of 198 measures; 95.5%; Cohen’s kappa 0.81).
DISCUSSION
Our study is a critical first step in describing the content of stress MPI reports as it relates to the determination of post-test risk. We found that 98% of reports in a sample from the Veteran’s Health Administration included all of the critical data elements required for estimation of risk but only 4% included a direct statement about this risk. These findings highlight an opportunity for NLP solutions to assist with clinical interpretation of stress test reports.
Stress myocardial perfusion imaging techniques are among the most commonly used non-invasive methods to further define risk of coronary disease in patients with an intermediate or borderline high pretest probability [10]. Concise but comprehensive reporting of test findings is essential for promoting appropriate downstream treatment decisions. In particular, it is critical that referring providers be able to estimate post-test ischemic risk since this affects the appropriateness of more invasive diagnostic testing and treatment [2,3,11].
Recent data suggest that ordering providers underestimate ischemia when reading stress MPI reports [7]. This is troubling in that it suggests that referring providers may not be able to accurately estimate their patient’s post-test risk based on their review of stress test results. Who then can make an assessment of risk for the patient? While it is not required by reporting standards, accrediting agencies for stress MPI recommend that interpreting physicians comment on the clinical significance of perfusion results – and specifically, the risk – in the “overall impression” section of the report [4]. But many “readers” are reluctant to supersede the role of the referring clinician, who knows the patient and the pretest probability, in making such a calculation. With this in mind, it is not surprising that less than 5% of reports in our sample contained a comment on risk.
Our results are heartening if you consider that the basic data elements necessary for estimating post-test ischemic risk are present in nearly 100% of reports. Data mining tools, such as natural language processing, could be trained to automatically extract the relevant text from imaging reports and synthesize this data in a coherent way. Similar efforts have led to the extraction of ejection fraction data from unstructured echocardiography reports [12]. Natural language processing has been used to identify high risk findings within radiology reports, such as those pertaining to appendicitis, thromboembolic disease and premalignant lesions [8]. In the case of stress imaging reports, NLP could not only be used to identify basic information about systolic function (e.g. ejection fraction), but also direct and indirect descriptors of ischemia and relevant modifiers such as size, location and severity.
If an NLP tool could be designed to pluck these elements out of stress test reports, one could even imagine automating a basic assessment of post-test ischemic risk. We imagine this tool would not only assist both referring and interpreting physicians in guiding appropriate clinical care for patients after stress MPI testing, but it would inform research and quality efforts. Referrals for cardiac catheterization after stress testing could be routinely assessed for appropriateness. Disparities between expected and measured risk could be studied. You could even study how the calculation of post-test risk itself affects outcomes: do patients with high-risk stress MPI findings have better outcomes because of their greater likelihood to receive PCI?
Unfortunately, our data also suggest that designing an NLP tool for unstructured stress imaging reports may be challenging, since roughly 1 in 4 reports did not disclose critical report elements in the impression section. Analyzing an entire report with NLP requires complicated rules to relate separate text clauses spread throughout the document, whereas focusing on a single section can simplify these functions. NLP tools for imaging reports are therefore somewhat limited by the complexity of syntax and how the language within the report is structured.
This is why structured reporting is so essential to overall improvement of report quality. A structured report is written using standardized definitions and content in a predictable and clinically relevant format, as opposed to more free form reporting. While structured reporting constrains information entry, attenuating efficiency and overall satisfaction among users of these documents, recipients of imaging reports have been shown to prefer structured reports to free text presumably because such standardization promotes consistency and clarity [13,14,15]. Cardiac imaging societies, as well as other imaging subspecialty societies, have endorsed structured reporting as a means of improving the quality of reports [16]. With appropriate input and buy-in from local stakeholders, it has been shown that standardized reporting can be implemented successfully and to the satisfaction of imaging specialists [17].
Still, uptake of structured reporting related to nuclear stress imaging reports has been slow. A recent study by Maddux et al. showed that only about 60% of accredited institutions are compliant with IAC Nuclear/PET reporting standards [6]. This percentage for non-accredited institutions is likely even lower. Improving compliance with reporting of required data elements – many of which are designed to augment clinician’s assessment of post-test risk – will be essential to the success of NLP tools for processing risk. Special attention should be paid to the impression section, not only because this would simplify NLP systems as described above, but because this is an essential section of the report and the first place ordering providers look for information [18,19].
This study should be considered in light of the following limitations. First, the study population was limited to Veterans seeking care within the VA, and our results are perhaps not generalizable to other care provision settings. Second, we lacked data on institutional, patient and provider factors that may have contributed to a report’s adequacy, such as the experience and specialty of the report author. Future studies may seek to identify reporting provider and system-level characteristics associated with higher quality reporting. Third, we restricted our analysis to those reporting elements most directly applicable to the determination of risk and subsequent clinical care. Additional study may identify gaps in other technical aspects of reporting that relate to procedural safety. Fourth, our analysis does not consider that increased attention to appropriateness of invasive coronary procedures may, over time, result in a greater proportion of reports incorporating a risk assessment. A logical next step in this work would be to examine temporal trends in the reporting of post-test risk. Finally, our work does not directly inform the impact of poor documentation on subsequent clinical care. Future work should seek to understand the impact of poor documentation quality on the quality of downstream care.
In conclusion, this study demonstrates that most stress MPI reports do not provide an estimate of ischemic risk even though these reports contain the basic data required for such a summary interpretation. Broader application of structured data reporting will improve the quality and consistency of stress test documentation. In doing so, these efforts may also facilitate using natural language processing systems to compute and reliably communicate post-test ischemic risk.
Supplementary Material
New Knowledge Gained.
In a sample of stress MPI reports in the VA, two independent reviewers agreed that reports often contained the basic elements necessary to make a summary interpretation of post-test risk, but these summary results were rarely reported by interpreting providers. This suggests an opportunity for Natural Language Processing (NLP) tools to assist with providing a summary interpretation. Greater standardization of report content and syntax will assist these efforts.
Acknowledgments
Funding
This project was supported by funding from VA Health Services Research & Development (HSR&D-QUERI RRP 12-448). Dr. Levy receives funding from National Institutes of Health (NIH) T32 Training Grant 5T32-HL-007822-19.
Dr. Shah is supported by an Agency for Healthcare Research and Quality award (5K12HS022998).
ABBREVIATIONS
- ASNC
American Society of Nuclear Cardiology
- CART
VA Clinical Assessment Reporting and Tracking Program
- CDW
VA Corporate Data Warehouse
- CPT
Current Procedural Terminology
- IAC
Intersocietal Accreditation Commission
- ICD-9
International Classification of Diseases, Ninth Revision codes
- MPI
myocardial perfusion imaging
- NLP
natural language processing
- PCI
percutaneous coronary intervention
- SPECT
single photon electron computed tomography
- SDS
summed difference score
- SRS
summed rest score
- SSS
summed stress score
- TID
transient ischemic dilation
- VA
Veterans Health Administration
Footnotes
Disclosures
No relevant disclosures from any of the authors.
Contributor Information
Andrew E. Levy, Email: Andrew.levy@ucdenver.edu.
Nishant R. Shah, Email: nishmd@gmail.com.
Michael E. Matheny, Email: michael.matheny@vanderbilt.edu.
Ruth M. Reeves, Email: ruth.reeves2@va.gov.
Glenn T. Gobbel, Email: glenn.t.gobbel@vanderbilt.edu.
Steven M. Bradley, Email: Steven.Bradley@allina.com.
References
- 1.Ladapo JA, Blecker S, Douglas PS. Physician Decision Making and Trends in the Use of Cardiac Stress Testing in the United States. Ann Intern Med. 2014 Oct 7;161(7):482–90. doi: 10.7326/M14-0296. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Patel MR, Dehmer GJ, Hirshfeld JW, Smith PK, Spertus JA. ACCF/SCAI/STS/AATS/AHA/ASNC 2009 Appropriateness Criteria for Coronary Revascularization. Circulation. 2009 Mar 10;119(9):1330–52. doi: 10.1161/CIRCULATIONAHA.108.191768. [DOI] [PubMed] [Google Scholar]
- 3.Patel MR, Bailey SR, Bonow RO, Chambers CE, Chan PS, Dehmer GJ, et al. ACCF/SCAI/AATS/AHA/ASE/ASNC/HFSA/HRS/SCCM/SCCT/SCMR/STS 2012 Appropriate Use Criteria for Diagnostic Catheterization. JACC. 2012 May 29;59(22):1995–2027. doi: 10.1016/j.jacc.2012.03.003. [DOI] [PubMed] [Google Scholar]
- 4.Tilkemeier PL, Cooke CD, Grossman GB, McCallister BD, Ward RP. ASNC Imaging Guidelines for Nuclear Cardiology Procedures. J Nucl Cardiol. 2009;16(4):650–0. [Google Scholar]
- 5.The IAC standards and guidelines for Nuclear/PET accreditation. Ellicott City, MDFinal Report. International Accreditation Commission, Nuclear/PET; 2015. Accessed via: http://www.intersocietal.org/nuclear. [Google Scholar]
- 6.Maddux PT, Farrell MB, Ewing JA, Tilkemeier PL. Improved compliance with reporting standards: A retrospective analysis of Intersocietal Accreditation Commission Nuclear Cardiology Laboratories. J Nucl Cardiol. 2016 Nov 9;3:350. doi: 10.1007/s12350-016-0713-y. [DOI] [PubMed] [Google Scholar]
- 7.Trägårdh E, Höglund P, Ohlsson M, Wieloch M, Edenbrandt L. Referring physicians underestimate the extent of abnormalities in final reports from myocardial perfusion imaging. EJNMMI Res. 2012;2(1):27. doi: 10.1186/2191-219X-2-27. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Pons E, Braun LMM, Hunink MGM, Kors JA. Natural Language Processing in Radiology: A Systematic Review. Radiology. 2016 Apr 18;279(2):329–43. doi: 10.1148/radiol.16142770. [DOI] [PubMed] [Google Scholar]
- 9.Maddox TM, Plomondon ME, Petrich M, Tsai TT, Gethoffer H, Noonan G, et al. A national clinical quality program for Veterans Affairs catheterization laboratories (from the Veterans Affairs clinical assessment, reporting, and tracking program) Am J Cardiol. 2014 Dec 1;114(11):1750–7. doi: 10.1016/j.amjcard.2014.08.045. [DOI] [PubMed] [Google Scholar]
- 10.Fihn SD, Gardin JM, Abrams J, Berra K, Blankenship JC, Dallas AP, et al. 2012 ACCF/AHA/ACP/AATS/PCNA/SCAI/STS Guideline for the Diagnosis and Management of Patients With Stable Ischemic Heart Disease. JACC. 2012 Dec;60(24):e44–e164. doi: 10.1016/j.jacc.2012.07.013. [DOI] [PubMed] [Google Scholar]
- 11.Wolk MJ, Bailey SR, Doherty JU, Douglas PS, Hendel RC, Kramer CM, et al. ACCF/AHA/ASE/ASNC/HFSA/HRS/SCAI/SCCT/SCMR/STS 2013 Multimodality Appropriate Use Criteria for the Detection and Risk Assessment of Stable Ischemic Heart Disease. JACC. 2014 Feb 4;63(4):380–406. doi: 10.1016/j.jacc.2013.11.009. [DOI] [PubMed] [Google Scholar]
- 12.Garvin JH, DuVall SL, South BR, Bray BE, Bolton D, Heavirland J, et al. Automated extraction of ejection fraction for quality measurement using regular expressions in Unstructured Information Management Architecture (UIMA) for heart failure. J Am Med Inform Assoc. 2012 Jan 1;19(5):859–66. doi: 10.1136/amiajnl-2011-000535. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Rosenbloom ST, Denny JC, Xu H, Lorenzi N, Stead WW, Johnson KB. Data from clinical notes: a perspective on the tension between structure and flexible documentation. J Am Med Inform Assoc. 2011 Mar 1;18(2):181–6. doi: 10.1136/jamia.2010.007237. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Schwartz LH, Panicek DM, Berk AR, Li Y, Hricak H. Improving Communication of Diagnostic Radiology Findings through Structured Reporting. Radiology. 2011 Jul 1;260(1):174–81. doi: 10.1148/radiol.11101913. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Naik SS, Hanbidge A, Wilson SR. Radiology Reports. Am J Roentgenol. 2001 Mar;176(3):591–8. doi: 10.2214/ajr.176.3.1760591. [DOI] [PubMed] [Google Scholar]
- 16.Douglas PS, Hendel RC, Cummings JE, Dent JM, Hodgson JM, Hoffmann U, et al. ACCF/ACR/AHA/ASE/ASNC/HRS/NASCI/RSNA/SAIP/SCAI/SCCT/SCMR 2008 Health Policy Statement on Structured Reporting in Cardiovascular Imaging. JACC. 2009 Jan 6;53(1):76–90. doi: 10.1016/j.jacc.2008.09.005. [DOI] [PubMed] [Google Scholar]
- 17.Larson DB, Towbin AJ, Pryor RM, Donnelly LF. Improving Consistency in Radiology Reporting through the Use of Department-wide Standardized Structured Reporting. Radiology. 2013 Apr 1;267(1):240–50. doi: 10.1148/radiol.12121502. [DOI] [PubMed] [Google Scholar]
- 18.Bosmans JML, Weyler JJ, De Schepper AM, Parizel PM. The Radiology Report as Seen by Radiologists and Referring Clinicians: Results of the COVER and ROVER Surveys. Radiology. 2011 Apr;259(1):184–95. doi: 10.1148/radiol.10101045. [DOI] [PubMed] [Google Scholar]
- 19.Wackers FJT. The art of communicating: The Nuclear Cardiology Report. J Nucl Cardiol. 2011 Jun 25;18(5):833–5. doi: 10.1007/s12350-011-9417-5. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.