Abstract
Background
Posterior cerebral ischemic stroke and its underlying causes can be easily misdiagnosed in routine practice. Therefore, more than a third of positive cases can be easily missed during routine CT image reporting unless expert neuroimaging radiologists carefully report it.
Objective
To assess the inter-rater agreement level between senior residents and general radiologists and a specialized expert neuroradiologist when detecting and diagnosing posterior cerebral circulation calcification.
Methods
This was a cross-sectional observational study. A total of fifteen senior radiology residents (SRRs) and fifteen general radiologists (GRs) at four different hospitals in Riyadh, Saudi Arabia, were included in the study. A total of four CT-scanned brain cases with the presence of posterior circulation calcification (PCC) with different degrees of severity and one brain case with negative PCC were selected. These cases were predefined by expert neuroradiologists at our center. The cases were uploaded into the picture archiving and communication systems (PACS) at four different centers as outsider cases. These cases were then randomly assigned to the participating SRRs and GRs for reporting. All radiologists were blinded to the findings of the cases. Inter-observer agreement was assessed using the weighted kappa coefficient of agreement (k) between the two groups.
Results
The cerebral calcification misdiagnosis rate for the SRRs and GRs was > 93% for most of the positive cases. There was 1) poor inter-observer agreement between the SRRs and GRs for the detection of severe posterior cerebral calcification(PCC) in a negative stroke case (agreement for misdiagnosis, k = 0.93; correct diagnosis, k = 0.00), 2) poor inter-observer agreement for mild PCC in a negative stroke case (agreement for misdiagnosis, k = 0.93; correct diagnosis, k = 0.00), 3) moderate PCC in a positive posterior stroke case (agreement for misdiagnosis, k = 0.92; correct diagnosis, k = 0.00), and 4) poor interobserver agreement for severe PCC in a positive posterior cerebral stroke case (agreement for misdiagnosis, k = 0.846; correct diagnosis, k = 0.00). There was excellent agreement between the SRRs and GRs when reporting negative cases of PCC and stroke.
Conclusion
Our study concluded that most of the SRRs and GRs missed the diagnosis of posterior cerebral calcification in the presented cases.
Abbreviations: PCC, Posterior circulation calcification; SRR, Senior Radiology Resident; GR, General Radiologist; NECT, Non-enhanced computed tomographic; CT, Computed tomography; PACS, Picture archiving communicating system
Keywords: Radiologist, Posterior cerebral calcification, Agreement level, Residents
1. Introduction
Stroke is considered one of the main reasons for death and disability globally [1]. A stroke can occur in the anterior or posterior part of the brain. The diagnosis of the anterior stroke diagnosis is well established among radiologists and mostly seen via cross-section imaging techniques. The definition of a posterior cerebral ischemic stroke is an infarction within vascular regions that receive blood supply from the vertebrobasilar arterial system [2]. Most posterior cerebral ischemic strokes are caused by arterial atherosclerosis and calcification. Unfortunately, posterior cerebral ischemic stroke and its underlying causes can be easily misdiagnosed. Therefore, more than a third of positive cases can be easily missed during routine CT image reporting unless expert neuroimaging radiologists carefully report it.
Furthermore, according to Banerjee, Stone, and Werring (2018), posterior brain cerebral circulation strokes are missed three times more than anterior circulation strokes [3]. Arterial calcification is considered a significant risk factor for ischemic stroke [4]. Most stroke patients develop ischemic symptoms, including sensory loss, visual field defects, and vertigo [3]. Vertebrobasilar artery calcification is commonly attributed to hypertension, age, smoking, and diabetes [5].
Non-enhanced computed tomographic (NECT) scanning is a superior technology that can detect vascular calcification [1]. Regardless of its drawbacks, NECT is widely available and a fast diagnostic modality, which enforces its common application for the detection of ischemic and hemorrhagic stroke [6]. However, interpreters can miss posterior cerebral circulation calcification (PCC) for several reasons, such as a lack of knowledge and experience, as well as interpreting too many cases during long and variable duties. From observing numerous brain CT scan reports for patients examined between 2015 and 2021 in our center, we observed that most positive PCC cases were not noted or mentioned in corresponding reports. Therefore, it was essential to address this issue in a scientific manner and examine the medical interpreters’ abilities while blinded to the reporting of the positive and negative PCC cases.
It was assumed that PCC detection, diagnosis, and interpretation by cross-sectional imaging could be missed by both SRRs and GRs. Therefore, the purpose of this study was to determine how frequently PCC is misdiagnosed by SRRs and GRs because they are the first interpreters of CT brain scans in emergency departments. Moreover, we also assessed the PCC detection rates of SRRs, and GRs compared to an expert neuroimaging radiologist.
2. Materials and methods
2.1. Study design
The study was conducted at four radiology departments in Riyadh, Saudi Arabia. Ethical approval was obtained from the Institutional Review Board of King Saud University Medical City. The study design was cross-sectional and observational, where the researchers sought consecutive patients with posterior strokes and PCC cases that required interpretation and definition by expert neuroradiologists.
2.2. Patient selection and review of radiology reports
Four plain CT-scanned brain cases with positive PCC and one CT-scanned brain with negative posterior circulation calcification were selected and defined by two senior expert radiologists at our center. The cases were uploaded into the (PACS) of four different centers as outsider cases. These cases were introduced into routine radiology cases and randomly assigned to fifteen GRs and fifteen SRRs for daily reporting. All the participating radiologists were blinded to all cases and unaware of the previous reports. Their reports were collected, and the participants were later interviewed regarding the missing diagnosis of posterior circulation calcification.
2.3. Degree of PCC in the preselected cases
The degrees of PCC for the preselected cases were 1) “severe” for the first case without posterior ischemic stroke, 2) “mild” for the second case without the presence of ischemic stroke, 3) “moderate” for the third case with the presence of posterior cerebral stroke, and 4) “severe” for the fourth case with posterior cerebral stroke. The fifth case was free of calcification and ischemic stroke.
2.4. Scan technique and interpretation
The brain imaging was undertaken on either a 64-detector CT scanner (Discovery CT750 HD; GE Healthcare, Chicago, IL) using a conventional CT technique (120 kV, 90 mAs, and 0.5 slice thickness) or a dual-source CT scanner (Somatom Definition Flash; Siemens; Germany) using a similar conventional CT technique (120 kV, 250 mAs, and 0.5 slice thickness).
2.5. Statistical analysis
All statistical analyses were carried out using SPSS software, Version 27.0.0.0 (IBM Corp., Armonk, NY). A descriptive analysis of the experience of the participating groups was provided. The percentage of participants missing the diagnoses among both groups was also provided. To determine interobserver agreement for the detection of PCC, a weighted kappa statistic was used for the analysis of interobserver agreement between the SRRs and GRs. All weighted kappa value assessments were presented based on the agreement of a misdiagnosis and a correct diagnosis.
3. Results
The SRRs group had a three-year mean for radiological experience, and the GRs group included board-certified radiologists with a seven-year mean for radiological experience. Of all the participating radiologists, 76.7% missed diagnosing and reporting PCC, 16.7% reported only one case with posterior calcification out of four cases, and 6.7% reported three cases with PCC out of the four cases (see Table 1).
Table 1.
Number of participating radiologists.
| Detected Calcification | No. of Participating Radiologists | Percentage |
|---|---|---|
| 0 | 23 | 76.7 |
| 1 | 5 | 16.7 |
| 3 | 2 | 6.7 |
| Total | 30 | 100 |
The detection rate was slightly higher for the GRs group: 73.3% missed reporting calcification, 20% detected only one case, and 6.7% detected three cases. Of the SRRs, 80% missed detecting and reporting PCC, and 13.3% reported a single case. Like the GR, 6.7% of the SRRs detected three cases with PCC (p = 0.003; see Table 2).
Table 2.
Comparison of the detection rate of PCC between GRs and SRRs.
| Targeted Group | Missed (PCC) | Single Detected Case | Three Detected Cases | Total | |
|---|---|---|---|---|---|
| General Radiologists | Count | 11 | 3 | 1 | 15 |
| Percentage | 73.3% | 20% | 6.7% | 100% | |
| Senior Radiology Residents | Count | 12 | 2 | 1 | 15 |
| Percentage | 80% | 13.3% | 6.7% | 100% | |
Note: P = 0.003
The inter-rater agreements for the PCC cases with different degrees of calcification and positive and negative posterior ischemic stroke detection are provided in Table 3. There was 1) poor agreement for PCC detection and severe PCC in the negative stroke case (agreement for misdiagnosis, k = 0.93; correct diagnosis, k = 0.00); 2) poor interobserver agreement for mild posterior cerebral calcification in the negative stroke case (agreement for misdiagnosis, k = 0.93; correct diagnosis, k = 0.00); 3) poor interobserver agreement for moderate posterior cerebral calcification in the positive posterior stroke case (agreement for misdiagnosis, k = 0.92; correct diagnosis, k = 0.00); and 4) poor interobserver agreement for severe posterior cerebral calcification in the positive posterior cerebral stroke case (agreement for misdiagnosis, k = 0.846; correct diagnosis, k = 0.00). In contrast, the SRR and GR showed 100% agreement in reporting the normal CT brain case free of calcification.
Table 3.
Inter-rater agreement between SRRs and GRs.
| General Radiologists |
Total | Weighted kappa |
||||||
|---|---|---|---|---|---|---|---|---|
| Missed Diagnosis | Correct Diagnosis | |||||||
| Missed Diagnosis | Correct Diagnosis | |||||||
| CASE 1 | SENIOR RESIDENTS | Missed Diagnosis | N | 13 | 1 | 14 | 0.929 | 0.000 |
| % | 92.9% | 100.0% | 93.3% | |||||
| Correct Diagnosis | N | 1 | 0 | 1 | ||||
| % | 7.1% | 0.0% | 6.7% | |||||
| CASE 2 | SENIOR RESIDENTS | Missed Diagnosis | N | 13 | 1 | 14 | 0.929 | 0.000 |
| % | 92.9% | 100.0% | 93.3% | |||||
| Correct Diagnosis | N | 1 | 0 | 1 | ||||
| % | 7.1% | 0.0% | 6.7% | |||||
| CASE 3 | SENIOR RESIDENTS | Missed Diagnosis | N | 12 | 2 | 14 | 0.923 | 0.000 |
| % | 92.3% | 100.0% | 93.3% | |||||
| Correct Diagnosis | N | 1 | 0 | 1 | ||||
| % | 7.7% | 0.0% | 6.7% | |||||
| CASE 4 | SENIOR RESIDENTS | Missed Diagnosis | N | 11 | 2 | 13 | 0.846 | 0.000 |
| % | 84.6% | 100.0% | 86.7% | |||||
| Correct Diagnosis | N | 2 | 0 | 2 | ||||
| % | 15.4% | 0.0% | 13.3% | |||||
| CASE 5 | SENIOR RESIDENTS | Missed Diagnosis | N | 0 | 15 | 15 | 0.000 | 1.000 |
| % | 0.0% | 100% | 100% | |||||
| Correct Diagnosis | N | 0 | 0 | 0 | ||||
| % | 0.0% | 0.0% | 0.0% | |||||
4. Discussion
This study highlighted that several degrees of PCC severity can be misdiagnosed or underreported by SRRs, and GRs based on CT brain scans. The goal of this study was to investigate the PCC detection rate by GRs and SRRs for comparison with an expert neuroradiologist’s readings. This study found low PCC detection rates for both groups, and the misdiagnosis rate was more than 93%. Posterior circulation calcification was occasionally reported in cases that involved posterior cerebral stroke, which increases the severity of PCC, and the detection rates for calcification were slightly higher in the GRs group compared to the SRRs group.
An error rate of 4% was previously reported from daily practice radiology reports [7]. The impact of misinterpreting or misidentifying PCC is significant and causes delays in medical and surgical treatment plans. Radiological errors increase the mortality and morbidity of patients with stroke, especially in neuroradiological emergency cases [8]. Most previous studies have summarized causes for radiologists’ errors made when identifying and interpreting PCC. These include the patient’s clinical history, the availability of previous studies, the environment of the reporting room, and the radiologist’s level of vigilance [9].
In our study, the participating radiologists were interviewed to clarify the reasons behind the missed diagnoses of PCC. Due to heavy workloads, the participants stated that they only reported the main exam findings. Another reason for not reporting calcification was a lack of clinical significance. The minority of the participants believed PCC was common in most patients, especially after the sixth decade of life, so there was no reason to report it. Calcification was reported exceptionally well in CT angiography cases, where stenosed arteries can be illustrated. In contrast, PCC was successfully noticed by some of the participants. Most of them were aware of the negative consequences of posterior calcifications and confirmed that calcification can be the main reason for posterior cerebral stroke. They also believed that posterior circulation calcifications can be associated with other vascular calcifications, and thus the patient can be referred for further investigations.
Several factors have been proposed in the literature to reduce PCC neuroimaging errors, including improving radiologists’ knowledge, well set-up systems and working conditions, turnaround times for reporting, double reporting techniques, and good communication between referring physicians and reporting radiologists [10]. The following are recommended to reduce errors in PCC detection and diagnosis: the use of PACS facilities to assess posterior fossa vasculature with proper adjustment of window width and level, as well as radiologists paying more attention to brain parenchyma in posterior circulation territory that are affected by ischemic change. Posterior fossa ischemia cases must include the presence or absence of vasculature calcifications. A scoring system for posterior brain cerebral circulation calcification should be established to accurately grade the severity of calcification.
Our study included some limitations. First, we analyzed the detection of PCC in limited cases. Other important morphological variables were not considered, such as the location of the calcification, the degree of stenosis, and parenchyma enhancement. We included only five cases; however, our aim was to measure the feasibility of PCC detection by several SRR and GR. Therefore, we limited our study to five preselected cases with cerine pathology. Furthermore, CT angiography was not provided for the sample cases, as most emergency cases are diagnosed based on plain CT brain scans. To the best of our knowledge, this is the first study in Saudi Arabia that has assessed inter-rater agreement between a large number of SRRs and GRs for the radiologic description of PCC. We recommend future longitudinal studies to identify potential sources of error at multiple centers with more morphological variable data.
5. Conclusion
Our study concluded that there was a significant misdiagnosis of PCC by both the GRs and SRRs groups. PCC was frequently missed, even in the case of positive posterior ischemic stroke. There was no interobserver agreement for the detection of PCC compared to an expert reader among both groups. The GRs showed fair agreement compared to the SRRs. Therefore, it is important to increase the awareness of PCC and clarify the negative impacts of such vascular calcifications. Furthermore, interobserver agreement could be improved by establishing a diagnostic criterion for PCC and double reporting technique with an expert neuroradiologist. This would reduce the number of misdiagnosis cases.
Funding
None to declare.
CRediT authorship contribution statement
MJA and MAF contributed to study conceptualization and review final manuscript. MA, FKA, DMA, AMA and AA contributed to study design and data collection. MJA contributed to data interpretation and analysis. MA and MJA contributed to manuscript writing, editing and preparation. The presenting work was check and submitted by MJA. All authors reviewed the final draft.
Conflict of interest
The authors disclose no conflict of interests, and the work was not supported or funded by any medical company.
Acknowledgments
The authors are grateful to the Medical Imaging Department Administration and PACS team at KSMC. The authors would like to thank all Medical and supporting staff of the Radiology Department at KSMC for their help. In addition, the authors would like to thank the Deanship of Scientific Research at Prince Sattam bin Abdulaziz University for the support by providing statistical packages for analysis.
Ethics approval and consent to participate
This study was conducted in accordance with the declaration of Helsinki. The studies involving human participants were reviewed and approved by the Ethics Committee at king Saud Medical city (ref no. H1RI-09-Mar21-01). Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.
References
- 1.Venkat A., et al. Factors associated with stroke misdiagnosis in the emergency department: a retrospective case-control study. Neuroepidemiology. 2018;51(3–4) doi: 10.1159/000491635. 123–27. [DOI] [PubMed] [Google Scholar]
- 2.Nouh A., Remke J., Ruland S. Ischemic posterior circulation stroke: a review of anatomy, clinical presentations, diagnosis, and current management. Front. Neurol. 2014;5:1–16. doi: 10.3389/fneur.2014.00030. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Banerjee G., Stone S.P., Werring D.J. Posterior circulation ischaemic stroke. BMJ. 2018;361:1–7. doi: 10.1136/bmj.k1185. [DOI] [PubMed] [Google Scholar]
- 4.Gökçal E., et al. The association of vertebrobasilar calcification with etiological subtypes, stroke recurrence and outcome in acute brainstem ischemic stroke. Neurol. Neurochir. Pol. 2018;52(2) doi: 10.1016/j.pjnns.2017.09.010. 188–93. [DOI] [PubMed] [Google Scholar]
- 5.Van Der Toorn J.E., Engelkes S.R., Kamran Ikram M., Arfan Ikram M., Vernooij M.W., Kavousi M., Bos D. Vertebrobasilar artery calcification: prevalence and risk factors in the general population. Atherosclerosis. 2019:46–52. doi: 10.1016/J.Atherosclerosis.2019.05.001. [DOI] [PubMed] [Google Scholar]
- 6.Jensen-Kondering U. Hyperdense artery sign on computed tomography in acute ischemic stroke. World J. Radiol. 2010;2(9):354. doi: 10.4329/wjr.v2.i9.354. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Caranci F., Tedeschi E., Leone G., Reginelli A., Gatta G., Pinto A., Squillaci E., Briganti F., Brunese L. Errors in neuroradiology. Radiol. Med. 2015;120:795–801. doi: 10.1007/S11547-015-0564-7. [DOI] [PubMed] [Google Scholar]
- 8.Cirillo L., Leonardi M., Dall’Olio M., Princiotta C., Stafa A., Simonetti L., Toni F., Agati R. Complications in the treatment of intracranial aneurysms with silk stents: an analysis of 30 consecutive patients. Interv. Neuroradiol. 2012;18(4):413–425. doi: 10.1177/159101991201800407. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Brady A., Laoide R.O., McCarthy P., McDermott R. Discrepancy and error in radiology: concepts, causes and consequences. Ulst. Med. J. 2012;81:3–9. [PMC free article] [PubMed] [Google Scholar]
- 10.Grassi R., Rea G., Scaglione M., Brunese L., Scialpi M. Imaging of tracheocele: report of three cases and review of the literature. Radiol. Med. 2000;100:285–287. [PubMed] [Google Scholar]