Abstract
Background
Administrative health data are valuable in health research and disease surveillance, but have low to moderate sensitivity in identifying transient ischaemic attacks (TIA) in the emergency department (ED). We aimed to identify the predictors of coding accuracy for TIA.
Methods
The study population was obtained from two ongoing studies on the diagnosis of TIA, minor stroke and stroke mimic. ED charts were manually reviewed by a stroke neurologist to obtain the clinical diagnosis, patient characteristics and content of physician documentation. Administrative data codes were compared with the chart-adjudicated diagnosis to determine cases of misclassification by administrative data. Univariable regression was used to evaluate candidate predictors of disagreement, and the significant variables were tested in a multivariable model to obtain an adjusted estimate of effect.
Results
Among 417 patients (39.1% TIA, 37.2% minor stroke and 23.7% stroke mimics), there were 122 cases of disagreement between adjudications and administrative data codes for the diagnosis of TIA. The majority of disagreement (n=103/122, 84.4%) arose from adjudicated TIA cases that were misclassified as non-TIA in administrative data coding. There were 78 (18.7%) charts with documented uncertain diagnosis, and 73 (17.5%) charts had no definite diagnosis. The relative risk of disagreement between chart adjudication and administrative data coding when the final diagnosis was uncertain or absent was 1.82(1.36, 2.44) and the risk difference was 18.5%. Multivariable logistic regression analyses confirmed this association using different case definition algorithms.
Conclusions
In suspected patients with TIA and minor stroke presenting to the ED, physician documentation was the dominant factor in coding accuracy, supporting the concept that physicians are active participants in administrative data coding. Strategies to improve chart documentation are predicted to have a positive effect on coding accuracy.
Keywords: Transient ischemic attack, emergency medicine, administrative data
Strengths and limitations of this study.
Understanding the predictors of data accuracy has high relevance in improving the low to moderate sensitivity of transient ischaemic attack (TIA) coding in the emergency department (ED) and may shed insight into the coding accuracy of other conditions primarily treated in the outpatient setting.
ED charts were manually reviewed by a fellowship-trained stroke neurologist to determine the final clinical diagnosis and the factors associated with administrative data miscoding.
We evaluated predictors related to patients, physicians and the patient–physician interaction, such as decision for advanced neuroimaging, consultation of a specialist and time of presentation, including factors that can be acted on to improve administrative data accuracy.
The study population was obtained from two observational studies on the diagnosis of TIA and minor stroke versus stroke mimic using serum biomarkers and neuroimaging, and most patients were discharged from a single university teaching hospital, limiting the generalisability of our results to other jurisdictions and healthcare models.
Introduction
Patient interactions with healthcare systems generate large amounts of routinely collected data, generally termed ‘administrative data.’ These data are widely used for disease surveillance and research.1 Because they are not primarily generated for research purposes, studies using these data can be vulnerable to selection and misclassification bias.
Data accuracy can be influenced by physician factors, patient characteristics, as well as the patient–physician interaction. Coding accuracy has been shown to be influenced by prospective payment systems,2 structural factors, such as the size, geographical location and specialty of the hospital or health organisation,3 4 coder characteristics, expertise, continuing education,4 as well as physicians.5 A reabstraction study of 1829 medical records from 21 Veterans Administration hospitals found that 62% of coding inaccuracies could be attributed to physicians missing or inappropriately including procedures and diagnoses, using inadequate terminology, and calling inactive diagnoses active.5 In addition, the accuracy of administrative data codes is dependent on the disease under study, the study time period, the International Classification of Diseases (ICD) iteration used,6 the case definition algorithm (choice of ICD codes and their positions)7 8 and the clinical setting.8 9
Administrative data identify patients with suspected transient ischaemic attacks (TIA) discharged from the emergency department (ED) with low to moderate sensitivity and positive predictive value.8 9 Multiple publications have addressed the accuracy of cerebrovascular disease coding using the ICD, but data on the factors influencing the quality of coding are sparse.10 The diagnosis of TIA is based on patient history and clinical judgement, and is highly challenging because symptoms are transient and there are no pathognomonic, physiological, laboratory or radiological findings. Therefore, physician documentation may have more severe consequences on the quality of TIA coding.
We aimed to study the associations between patient characteristics and physician factors, including physician specialty and chart documentation, and the accuracy of TIA coding in ED administrative data. We also evaluated whether clinical factors were associated with differences in the content of physician documentation.
Methods
Study population
The study population was obtained from two ongoing Canadian studies: SPECTRA (Spectrometry for Transient Ischaemic Attack Rapid Assessment) and DOUBT (Diagnosis Of Uncertain-origin Benign Transient neurological symptoms). SPECTRA aims to identify a blood biomarker to differentiate TIA and minor strokes from mimics and enrols patients within 24 hours after symptom onset. DOUBT is a neuroimaging study of patients with TIA, minor strokes and stroke mimics and enrols patients within 7 days after symptom onset. Minor stroke is defined as National Institute of Health Stroke Scale score ≤3. Recruitment for both studies largely took place in a tertiary care teaching hospital with an ED volume of about 78 000 visits per year, including 905 annual acute stroke consults. Residents of Alberta, Canada enrolled between 1 December 2013 and 30 October 2015 with at least one ED visit prior to the enrolment date of SPECTRA (within 24 hours) and DOUBT (within 7 days) were included.
ED chart adjudication
All ED charts were manually reviewed by a stroke neurologist (AYXY), blinded to the administrative data coding, to obtain the ED clinical diagnosis. The full ED record was reviewed, including documentation from all physicians (emergentologists and consultants) and allied healthcare members who interacted with the patient. The adjudicated diagnosis of TIA was made based on the WHO time-based criteria.11 For example, a patient with symptoms lasting <24 hours but an MRI showing evidence of tissue ischaemia, like a small diffusion weighted imaging lesion, was adjudicated as a TIA and specifically not a stroke. A senior stroke neurologist (SBC) independently reviewed a random sample of 30 charts to determine interabstractor reliability and Cohen’s kappa was calculated.
Patient clinical characteristics, time of presentation (after-hours presentation is between 16:30 and 7:30) and stroke risk factors (history of ischaemic strokes or TIA, intracerebral haemorrhage, congestive heart failure, coronary artery disease, hypertension, diabetes mellitus, dyslipidaemia, atrial fibrillation, peripheral vascular disease and active smoking) were abstracted. Missing risk factors were imputed to be absent because negative findings on patient’s history are often not documented in medical charts. The burden of risk factors was categorised as none, one risk factor, or two or more. We recorded whether neuroimaging was completed in the ED. We defined abnormal imaging as presence of tissue ischaemia, intracerebral haemorrhage or evidence of vascular disease in the relevant territory (intracranial occlusion or ≥50% stenosis and/or extracranial occlusion or ≥50% stenosis or <50% in the presence of an ulcerated plaque or thrombus). The ED charts of four patients were missing and the SPECTRA and DOUBT case report forms were reviewed instead. In order to replicate the Canadian coding standards, only documentation dated and timed prior to the disposition from the ED was evaluated.12
When evaluating the content of physician documentation, we determined whether there was documentation of a clear final diagnosis, documentation of uncertainty or a lack of a definite diagnosis. As per coding standards, we obtained the final diagnosis from the consultant or admitting service’s notes. If no consultant was involved during the ED visit, the final diagnosis was obtained from the ED physician’s note. Documentation of uncertainty was defined as recording one or more possible diagnoses, for example, ‘TIA versus migraine versus seizure’, or lack of specification if the diagnosis was a TIA or stroke, for example, ‘posterior circulation ischaemia.’ A statement of ‘rule out stroke’ or ‘query TIA’ was not considered a diagnostic uncertainty because no differential diagnosis was explicitly stated. Lack of definitive diagnosis was defined as documentation of a symptom complex, for example, ‘transient aphasia, resolved’ or ‘weakness not yet diagnosed,’ or documentation of imaging findings, for example, ‘carotid stenosis’, as a diagnosis without an indication of symptomatic or asymptomatic (incidental) status.
In order to illustrate how quality of physician documentation may influence coding accuracy, table 1 shows the three different documentation styles of a hypothetical scenario: 82-year-old woman with history of atrial fibrillation presents with 15 min of aphasia and family noted right hand clumsiness now resolved, examination normal. CT and angiography showed no acute ischaemia, but there is a left middle cerebral artery, M2 branch occlusion. We asked a coding specialist from our tertiary care hospital to provide the ICD codes and accompanying interpretations for each documentation style.
Table 1.
Examples of coding from physician documentation of diagnostic impressions
| Documentation | ICD-10-CA codes* | Interpretation of codes | |
| Physician 1: clear diagnosis |
82F known AF, likely cardioembolic left hemispheric TIA. Differential includes seizures. CT/CTA: no stroke, left MCA-M2 occlusion |
G45.8 (MP) Q R56.88 (OP) I66.0 (OP) I48.90 (OP) |
Other transient cerebral ischaemic attacks and related syndromes Seizure—queried Occlusion and stenosis of MCA AF, unspecified |
| Physician 2: uncertainty |
82F history of AF, transient aphasia and right arm weakness. Cerebral ischaemia vs seizure. CT/CTA: no stroke, left MCA-M2 occlusion |
R47.0 (MP) R29.8 (OP) Q I63.5 (OP) Q R56.88 (OP) I66.0 (OP) I48.90 (OP) |
Aphasia Right arm weakness Cerebral infarction—queried Seizure—queried Occlusion and stenosis of MCA AF, unspecified |
| Physician 3: lack of definite diagnosis |
82F with AF, transient aphasia and right arm weakness. CT/CTA: no stroke, left MCA-M2 occlusion |
R47.0 (MP) R29.8 (OP) I66.0 (OP) I48.90 (OP) |
Aphasia Right arm weakness Occlusion and stenosis of MCA AF, unspecified |
*These represent real codes generated by a coding specialist in a tertiary care hospital based on the hypothetical patient information.
AF, atrial fibrillation; CT/CTA, CT and angiography; ICD-10-CA, International Classification of Disease 10th Canadian iteration; MCA-M2, middle cerebral artery, M2 branch, MP, main problem; OP, other problem; TIA, transient ischaemic attack.
National Ambulatory Care Reporting System
The National Ambulatory Care Reporting System (NACRS) database contains discharge diagnoses for all ED visits in Alberta coded as one ICD 10th Canadian iteration (ICD-10-CA) code for the ‘main problem’ per ED visit, up to 10 ‘other problems’ codes.13 A prefix ‘Q’ can be combined with any diagnostic codes in order to flag that the diagnoses were queried by the physician. The main-position algorithm defined TIA as any case with TIA codes (G45.x except G45.4) in the main position without querying prefix Q. The any-position algorithm defined TIA as any case with TIA codes in any diagnostic position with or without coding prefix Q. Ischaemic cerebrovascular disease was defined as coding for either TIA or ischaemic stroke (H34.1, H34.2, I63.x and I64.x) in the main position.14 The Alberta unique personal healthcare number was used for deterministic linkage with NACRS. When linkage to multiple ED visits occurred, including between-ED transfers, the last visit prior to enrolment was retained for analysis.
The SPECTRA and DOUBT studies received approval from the University of Calgary institutional review board for research using human subjects and written informed consent was obtained from all patients enrolled. The current study received approval from the University of Calgary institutional review board for research and a waiver of consent was granted (REB15-2943).
Statistical methods
Descriptive statistics were used to determine the frequency of baseline characteristics. We calculated the relative risk and risk difference of having a clearly documented final diagnosis and an accurate administrative data code. For the logistic regression analyses, charts with documented uncertainty and those without a definite diagnosis were evaluated separately because the former is unavoidable in clinical medicine and cannot be eliminated, while the latter can be acted on. To identify the factors associated with misclassification in administrative data, we used univariable logistic regression to estimate the ORs for predetermined clinically relevant variables: age (continuous), sex, ABCD2 (Age, Blood pressure, Clinical, Duration, Diabetes) score ≥4, ongoing symptoms in the ED, vascular risk factors (0, 1 or ≥2), neurovascular imaging in ED, abnormal finding on neurovascular imaging, neurology consultation in ED, weekend presentation, after-hours presentation, uncertain diagnosis and no definite diagnosis. Multivariable logistic regression with all variables significantly associated with disagreement in the univariable analysis (p<0.05 for significance) was performed to obtain an adjusted estimate of effect. Because we predicted the content of physician documentation to strongly influence coding accuracy, the logistic regression analyses were repeated in the subgroup of patients with a clear final diagnosis documented. The primary outcome was disagreement between adjudicated ED diagnosis and the main-position NACRS algorithm. Sensitivity analyses were performed using the any-position algorithm for TIA as well as main-position algorithm for the diagnosis of cerebral ischaemia. We also evaluated predictors of documentation of a clear final diagnosis. All analyses were performed using STATA V.14.0 (STATA Corp).
Results
Among 417 patients included, the adjudicated diagnoses showed 163 (39.1%) TIA, 155 (37.2%) mild ischaemic strokes and 99 (23.7%) stroke mimics. Interabstractor agreement between the two stroke neurologists was 76.7% (κ=0.50) for the diagnosis of TIA. Figure 1 shows the distribution of cases of disagreement between chart adjudication and the NACRS main-position algorithm for the diagnosis of TIA. Among 163 chart-adjudicated cases of TIA, there were 60 (36.8%) charts that were correctly coded as a TIA in the main position and there were 103 (63.2%) charts that were incorrectly coded as non-TIA. Among 254 chart-adjudicated cases of non-TIA (minor strokes or mimics), there were 235 (92.5%) charts that were correctly coded as a non-TIA in the main position and 19 (7.5%) charts that were incorrectly coded as a TIA. The total number of incorrectly coded charts was 122. Most of the incorrect coding was explained by cases of chart-adjudicated TIA that were incorrectly coded as non-TIA (103/122 84.4%). Figure 2 shows the cases of disagreement between chart adjudication and the NACRS any-position algorithm. Finally, the cerebral ischaemia algorithm identified 202 cases of TIA and minor stroke and there were 138 cases of disagreement for this diagnosis (figure 3).
Figure 1.
Disagreement between ED chart adjudication and NACRS main-position algorithm for the diagnosis of TIA. ED, emergency department; NACRS, National Ambulatory Care Reporting System; TIA, transient ischaemic attack.
Figure 2.
Disagreement between ED chart adjudication and NACRS any-position algorithm for the diagnosis of TIA. ED, emergency department; NACRS, National Ambulatory Care Reporting System; TIA, transient ischaemic attack.
Figure 3.
Disagreement between ED chart adjudication and NACRS main-position algorithm for the diagnosis of cerebral ischaemia. ED, emergency department; NACRS, National Ambulatory Care Reporting System.
Table 2 shows the frequency of patient and physician characteristics presented by agreement between ED adjudicated diagnosis and NACRS main-position algorithm. As a combined measure, when the final diagnosis was either uncertain or absent, the relative risk of disagreement between chart adjudication and administrative data coding was 1.82 (95% CI 1.36 to 2.44) in comparison to those with a clearly documented final diagnosis. The risk difference was 18.5%.
Table 2.
Frequency of baseline characteristics presented by agreement between ED chart adjudication and NACRS main-position algorithm, n (%)
| Agreement (N=295) | Disagreement (N=122) | |
| Median age (IQR) | 67 (22) | 65.5 (20) |
| Male sex | 151 (51.2) | 70 (57.4) |
| ABCD2 ≥4 | 216 (73.2) | 91 (74.6) |
| Ongoing symptoms in ED | 195 (66.1) | 50 (41.0) |
| Vascular risk factors | ||
| 0 | 77 (26.1) | 36 (29.5) |
| 1 | 69 (23.4) | 23 (18.9) |
| 2 or more | 149 (50.5) | 63 (51.6) |
| Neurovascular imaging in ED | 280 (94.9) | 114 (93.4) |
| Abnormal neurovascular imaging | 95 (32.2) | 39 (32.0) |
| Neurology consult | 249 (84.4) | 92 (75.4) |
| Weekend presentation | 58 (19.7) | 22 (18.0) |
| After-hours presentation | 137 (46.4) | 57 (46.7) |
| Documented diagnostic uncertainty | 39 (13.2) | 39 (32.0) |
| Failure to document a diagnosis | 50 (17.0) | 23 (18.9) |
ABCD2, Age, Blood pressure, Clinical, Duration, Diabetes; ED, emergency department; NACRS, National Ambulatory Care Reporting System.
Table 3 shows the logistic regression analysis evaluating predictors of disagreement between chart-adjudicated diagnoses and the different NACRS algorithms. Univariate analyses showed that ongoing symptoms and neurology consultation in the ED were predictive of agreement, and documentation of diagnostic uncertainty was associated with disagreement between adjudicated diagnosis and the TIA main-position algorithm. Multivariable analysis showed that physician’s diagnostic uncertainty was associated with disagreement, OR 3.71 (95% CI 2.16 to 6.36), whereas ongoing symptoms in the ED was associated with less disagreement, OR 0.32 (95% CI 0.20 to 0.51). Documentation of diagnostic uncertainty and lack of a definitive final diagnosis were the only factors associated with the disagreement between chart adjudication and administrative data for the TIA any-position algorithms (OR 3.16 (95% CI 1.82 to 5.47) and 2.51 (95% CI 1.42 to 4.45), respectively) and the cerebral ischaemia main-position algorithms (OR 3.26 (95% CI 1.93 to 5.50) and 2.55 (95% CI 1.49 to 4.37), respectively). In the subgroup of patients with a clear final diagnosis documented (table 4), the association between clinical variables and administrative data accuracy start to emerge: ongoing symptoms in the ED and having a neurologist evaluation in the ED were associated with higher accuracy of coding and male sex was associated with lower accuracy.
Table 3.
Univariable and multivariable logistic regression analysis (OR (95% CI)) of candidate predictors of disagreement between ED adjudicated diagnosis and NACRS coding
| TIA NACRS main-position algorithm (N=122) |
TIA NACRS any-position algorithm (N=106) |
Cerebral ischaemia NACRS main-position algorithm (N=138) |
||||
| Univariable | Multivariable | Univariable | Multivariable | Univariable | Multivariable | |
| Age | 1.00 (0.98 to 1.01) | 1.00 (0.99 to 1.02) | 1.00 (0.98 to 1.01) | – | ||
| Male sex | 1.28 (0.84 to1.96) | – | 1.10 (0.70 to 1.71) | 1.08 (0.72 to 1.63) | – | |
| ABCD2 ≥4 | 1.07 (0.66 to 1.74) | – | 0.94 (0.57 to 1.54) | – | 0.97 (0.61 to 1.53) | – |
| Ongoing symptoms in ED | 0.36 (0.23 to 0.55) | 0.32 (0.20 to 0.51) | 0.65 (0.42 to 1.02) | – | 0.91 (0.60 to 1.38) | – |
| Vascular risk factors | – | – | – | |||
| 0 | Ref | Ref | Ref | |||
| 1 | 0.71 (0.39 to 1.32) | 0.62 (0.33 to 1.17) | 0.63 (0.35 to 1.13) | |||
| ≥2 | 0.90 (0.55 to 1.48) | 0.71 (0.42 to 1.17) | 0.68 (0.42 to 1.10) | |||
| Neurovascular imaging in ED | 0.76 (0.31 to 1.85) | – | 0.62 (0.26 to 1.51) | – | 0.63 (0.27 to 1.46) | – |
| Abnormal neurovascular imaging | 0.99 (0.63 to 1.55) | – | 0.94 (0.58 to 1.51) | – | 0.94 (0.60 to 1.45) | – |
| Neurology consult | 0.57 (0.34 to 0.95) | 0.77 (0.44 to 1.35) | 1.03 (0.58 to 1.82) | – | 0.62 (0.37 to 1.03) | – |
| Weekend | 0.90 (0.52 to 1.55) | – | 0.62 (0.34 to 1.15) | – | 0.67 (0.39 to 1.16) | – |
| Night shift | 1.01 (0.66 to 1.54) | – | 1.33 (0.86 to 2.08) | – | 1.18 (0.78 to 1.77) | – |
| Documented diagnostic uncertainty | 3.08 (1.86 to 5.13) | 3.71 (2.16 to 6.36) | 2.49 (1.48 to 4.19) | 3.16 (1.82 to 5.47) | 2.59 (1.57 to 4.28) | 3.26 (1.93 to 5.50) |
| Failure to document a diagnosis | 1.14 (0.66 to 1.97) | – | 1.83 (1.06 to 3.13) | 2.51 (1.42 to 4.45) | 1.88 (1.12 to 3.14) | 2.55 (1.49 to 4.37) |
Bold indicates p<0.05.
ED, emergency department; NACRS, National Ambulatory Care Reporting System; TIA, transient ischaemic attack.
Table 4.
Univariable and multivariable logistic regression analysis (OR (95% CI)) of candidate predictors of disagreement between ED adjudicated diagnosis and NACRS coding among patients with a clear final diagnosis documented
| TIA NACRS main-position algorithm (N=60) |
TIA NACRS any-position algorithm (N=48) |
Cerebral ischaemia NACRS main-position algorithm (N=65) |
||||
| Univariable | Multivariable | Univariable | Multivariable | Univariable | Multivariable | |
| Age | 1.00 (0.98 to 1.02) | – | 1.01 (0.99 to 1.03) | – | 1.01 (0.99 to 1.02) | – |
| Male sex | 1.79 (1.01 to 3.16) | 2.09 (1.15 to 3.80) | 1.28 (0.69 to 2.35) | – | 1.21 (0.71 to 2.06) | – |
| ABCD2≥4 | 0.89 (0.48 to 1.64) | – | 0.76 (0.40 to 1.47) | – | 1.01 (0.56 to 1.85) | – |
| Ongoing symptoms in ED | 0.32 (0.18 to 0.57) | 0.37 (0.20 to 0.66) | 0.46 (0.25 to 0.84) | 0.46 (0.25 to 0.84) | 0.73 (0.43 to 1.25) | – |
| Vascular risk factors | – | – | – | |||
| 0 | Ref | Ref | Ref | |||
| 1 | 0.89 (0.39 to 2.04) | 0.66 (0.28 to 1.57) | 0.69 (0.30 to 1.59) | |||
| 2 or more | 1.25 (0.65 to 2.40) | 0.74 (0.37 to 1.46) | 1.23 (0.66 to 2.30) | |||
| Neurovascular imaging in ED | 0.58 (0.21 to 1.64) | – | 0.44 (0.16 to 1.25) | – | 0.87 (0.29 to 2.65) | – |
| Abnormal neurovascular imaging | 0.74 (0.40 to 1.36) | – | 0.67 (0.34 to 1.34) | – | 0.78 (0.43 to 1.40) | – |
| Neurology consult | 0.31 (0.17 to 0.56) | 0.36 (0.19 to 0.68) | 0.96 (0.44 to 2.08) | – | 0.32 (0.18 to 0.58) | 0.32 (0.18 to 0.58) |
| Weekend | 0.94 (0.46 to 1.90) | – | 0.97 (0.45 to 2.09) | – | 0.64 (0.30 to 1.35) | |
| Night shift | 0.86 (0.50 to 1.49) | – | 1.17 (0.64 to 2.14) | – | 0.85 (0.50 to 1.45) | |
Bold indicates p<0.05.
ABCD2, Age, Blood pressure, Clinical, Duration, Diabetes; ED, emergency department; NACRS, National Ambulatory Care Reporting System; TIA, transient ischaemic attack.
Evaluation of physician documentation showed that only 266 (63.8%) charts had a clear final diagnosis documented, despite a high number of neurology consultations (n=341, 81.8%), neuroimaging (n=394, 94.5%) and vascular imaging (n=347, 83.2%) completed in the ED. Uncertainty was documented in 78 (18.7%) charts and 73 (17.5%) charts had no definitive diagnosis. We did not identify any factors strongly associated with documentation of a clear final diagnosis (see online supplemental table 1). Having two or more vascular risk factors was statistically significantly associated with having a clear final diagnosis, but the magnitude of the association was small and the CIs were wide (OR 1.64 (95% CI 1.03 to 2.63)).
bmjopen-2016-015234supp001.pdf (354.3KB, pdf)
Discussions
Physician documentation was the dominant factor influencing the accuracy of the administrative data coding for patients with TIA and minor stroke discharged from the ED. The risk difference was 18.5%, suggesting that close to one in five charts with coding misclassifications could have improved accuracy if a definitive diagnosis was documented or uncertainty removed. When only considering charts with a clear final diagnosis, the influence of clinical factors on data accuracy start to emerge. These factors, ongoing symptoms in the ED, having a neurological consultation completed in the ED, and male sex, likely influence coding accuracy by improving clinical diagnostic accuracy.
Medical charts are important for documenting patient’s clinical status, communication among healthcare providers and have medicolegal value. Physicians report that the ‘Assessment and plan’ section is most valued and often reviewed first.15 In our study, although 82% of the patients received a specialist evaluation, close to one in five charts had no definite diagnosis documented and another one in five charts documented multiple competing diagnoses without a clear indication of the working diagnosis. While diagnostic uncertainty cannot be eliminated in clinical medicine, our study highlights the importance of clearly conveying the working diagnosis and outlining the thought process behind the investigations and management plan. In clinical practice, physicians often infer diagnoses made by another physician even if it is not explicitly stated. However, non-physician administrative data coders do not have the medical background to make such judgements. Coders are mandated to exclusively use physician documentation to code diagnoses and comorbidities. They may use laboratory or imaging reports to add specificity, but they cannot code a diagnosis unless it is documented by a physician. Therefore, we purposely strictly evaluated what was written in the chart and showed that its content influenced coding accuracy. In addition, better documentation may enhance communication between the healthcare team and improve patient outcome. A recent study found that formal chart documentation of acute kidney injury, and thus proper coding of the condition in administrative data, was associated with lower mortality rates, even after adjustment for severity of the illness.16
Our results show that documentation is not affected by patient characteristics (age, vascular risk factor burden and severity of the symptoms), investigation results (diagnostic imaging or specialist consultation), or the time of presentation to ED. It remains unclear why one chart is better documented than another. Electronic health records have the potential to improve documentation quality with their advantage of being more standardised in structure and better data accessibility and readability, but their ability to truly improve data quality is still unclear.17 Further, current electronic record systems are designed to focus on document composition without facilitating data synthesis, which further increases the information burden physicians are face with.15 Resident physicians report spending more time in reading and documenting medical health records than being at the bedside and raise concerns over the lack of feedback on their charting.18 19 One solution is to encourage, teach and provide incentives for improving the quality of health records to avoid losing important messages in the noise.20 Specifically, data from the USA demonstrate that physicians do not receive adequate training in disease nosology and coding methods.21 In certain European countries, physicians are responsible for administrative data coding, but the degree of formal training is variable. An Italian study evaluating physicians’ intercoder agreement for stroke codes defined an ‘expert coder’ as a neurologist who underwent at least three half-day or full-day ICD-coding training courses.22 In comparison, Canadian coding specialists are trained in a 2-year postsecondary education programme.23 Prior publications have shown that education programmes on documentation and inpatient coding processes improve the accuracy of diagnoses, comorbidities and complications coding in administrative data on both medical and surgical services.24 25
Our results are consistent with other studies reporting the influence of physician factors on administrative data coding accuracy. Higher quality chart documentation has been reported to be associated with higher agreement between coders and adjudicators in identifying comorbidities of patients admitted for carotid endarterectomy.26 Another study in the outpatient setting found that 43% of the primary diagnosis codes were inaccurate in general internal medicine clinics and physician diagnostic error was an important contributor.27
In the subset of charts that underwent a second independent review, we showed that the agreement was 76.7%. This is consistent with a study conducted in the Stanford TIA Clinic that reported a 72% overall agreement among fellowship-trained stroke neurologists.28 The subjectivity in TIA diagnosis, especially during the initial health encounter, is impossible to eliminate. Demonstration of cerebral ischaemia using magnetic resonance diffusion-weighted imaging in clinically-diagnosed patients with TIA is variable (30% to 70%) and associated with longer symptom duration.29 A tissue-based diagnosis for TIA has been proposed, where the diagnosis is defined by the absence of tissue ischaemia.30 However, because diffusion-weighted imaging is not routinely used in clinical practice, applying a tissue-based TIA diagnosis as reference standard would negatively impact the generalisability of our results.
Our study has some limitations. We had limited information on individual physician or coder characteristics, such as their role (trainee, attending and subspecialty), level of training and experience. Our retrospective study design did not allow us to capture a detailed record of patient symptoms, such as the speed of onset and types of symptoms, which may impact both diagnostic and coding accuracy and result in residual confounding.23 31 We used a convenience sample of non-consecutive patients presenting to the ED with acute neurological symptoms and the majority of the patients were discharged from a single tertiary care centre. This resulted in a high proportion of cases with neurological consultations in the ED and limits the generalisability of our results to other settings. Finally, although patient diagnosis does not influence physician reimbursements in Canadian acute care hospitals, payment systems and financial incentives can influence diagnosis coding.32 We did not have access to longitudinal payment information to test this hypothesis.
Conclusions
Clear documentation of a final diagnosis was associated with improved administrative data accuracy for TIA and minor strokes in the ED. Although our data do not reveal the reason why one chart is better coded than another, an intuitive solution is to educate physicians and other providers who generate medical information about proper documentation techniques, disease nosology and coding standards. Prospective studies evaluating the effects of techniques to improve physician documentation, such as education programmes, standardised discharge summaries or automated coding algorithms, on coding accuracy are needed. Administrative data are continuously generated—at a cost to the healthcare system—to help decision-makers monitor, evaluate and plan the provision of health services as well as for research. Improving their accuracy is in the best interest of patient care.
Supplementary Material
Acknowledgments
We acknowledge the special contribution of Ms Chris Makar, Coding Coordinator, Data Collection, Foothills Medical Centre.
Footnotes
Contributors: AYXY contributed to the study concept and design, acquisition, analysis and interpretation of the data, as well as drafting and revising the manuscript. HQ contributed to the study concept and design, data acquisition and interpretation, critical revision of the manuscript and supervision. ADMR contributed to the study concept and design, data interpretation, critical revision of the manuscript and supervision. GOW contributed to the data acquisition, administrative support and manuscript revision. MDH contributed to the study concept and design, data analysis and interpretation, critical revision of the manuscript and supervision. SBC contributed to the study concept and design, data interpretation, critical revision of the manuscript and supervision.
Funding: This work was supported by Alberta Innovates Health Solutions (AYXY holds a Clinician-Fellowship Award, grant number 201500087). SPECTRA was funded by Genome Canada and Genome Alberta, and DOUBT was funded by the Canadian Institutes of Health Research.
Competing interests: None declared.
Ethics approval: University of Calgary Institutional Review Board for Research (REB15-2943).
Provenance and peer review: Not commissioned; externally peer reviewed.
Data sharing statement: The dataset supporting the results and conclusions of this article is available upon request.
References
- 1. Yu AY, Holodinsky JK, Zerna C, et al. . Use and utility of administrative health data for stroke research and surveillance. Stroke 2016;47:1946–52. 10.1161/STROKEAHA.116.012390 [DOI] [PubMed] [Google Scholar]
- 2. Assaf AR, Lapane KL, McKenney JL, et al. . Possible influence of the prospective payment system on the assignment of discharge diagnoses for coronary heart disease. N Engl J Med 1993;329:931–5. 10.1056/NEJM199309233291307 [DOI] [PubMed] [Google Scholar]
- 3. Rangachari P. Coding for quality measurement: the relationship between hospital structural characteristics and coding accuracy from the perspective of quality measurement. Perspect Health Inf Manag 2007;4:3. [PMC free article] [PubMed] [Google Scholar]
- 4. Santos S, Murphy G, Baxter K, et al. . Organisational factors affecting the quality of hospital clinical coding. Him J 2008;37:25–37. 10.1177/183335830803700103 [DOI] [PubMed] [Google Scholar]
- 5. Lloyd SS, Rissing JP. Physician and coding errors in patient records. JAMA 1985;254:1330–6. 10.1001/jama.1985.03360100080018 [DOI] [PubMed] [Google Scholar]
- 6. O'Malley KJ, Cook KF, Price MD, et al. . Measuring diagnoses: ICD code accuracy. Health Serv Res 2005;40:1620–39. 10.1111/j.1475-6773.2005.00444.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Benesch C, Witter DM, Wilder AL, et al. . Inaccuracy of the International Classification of Diseases (ICD-9-CM) in identifying the diagnosis of ischemic cerebrovascular disease. Neurology 1997;49:660–4. 10.1212/WNL.49.3.660 [DOI] [PubMed] [Google Scholar]
- 8. Hall R, Mondor L, Porter J, et al. . Accuracy of administrative data for the coding of acute stroke and TIAs. Can J Neurol Sci 2016;43:765–73. 10.1017/cjn.2016.278 [DOI] [PubMed] [Google Scholar]
- 9. Johnsen SP, Overvad K, Sørensen HT, et al. . Predictive value of stroke and transient ischemic attack discharge diagnoses in The Danish National Registry of Patients. J Clin Epidemiol 2002;55:602–7. 10.1016/S0895-4356(02)00391-8 [DOI] [PubMed] [Google Scholar]
- 10. McCormick N, Bhole V, Lacaille D, et al. . Validity of diagnostic codes for acute stroke in administrative databases: a systematic review. PLoS One 2015;10:e0135834 10.1371/journal.pone.0135834 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Investigators W. The World Health Organization MONICA Project (monitoring trends and determinants in cardiovascular disease): a Major international collaboration. WHO MONICA Project Principal Investigators. J Clin Epidemiol 1988;41:105–14. [DOI] [PubMed] [Google Scholar]
- 12. Canadian Institute for Health I. Canadian coding standards for version 2015 ICD-10-CA and CCI 2016 . https://secure.cihi.ca/estore/productSeries.htm?pc=PCC189 (accessed 22 June 2016).
- 13. Canadian Institute for Health Information. National Ambulatory Care Reporting System (NACRS) Metadata. https://www.cihi.ca/en/types-of-care/hospital-care/emergency-and-ambulatory-care/nacrs-metadata (accessed 6 Feb 2017).
- 14. Kokotailo RA, Hill MD. Coding of stroke and stroke risk factors using international classification of diseases, revisions 9 and 10. Stroke 2005;36:1776–81. 10.1161/01.STR.0000174293.17959.a1 [DOI] [PubMed] [Google Scholar]
- 15. Koopman RJ, Steege LM, Moore JL, et al. . Physician information needs and electronic health records (EHRs): time to reengineer the clinic note. J Am Board Fam Med 2015;28:316–23. 10.3122/jabfm.2015.03.140244 [DOI] [PubMed] [Google Scholar]
- 16. Wilson FP, Bansal AD, Jasti SK, et al. . The impact of documentation of severe acute kidney injury on mortality. Clin Nephrol 2013;80:417–25. 10.5414/CN108072 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Wang N, Yu P, Hailey D. Description and comparison of quality of electronic versus paper-based resident admission forms in Australian aged care facilities. Int J Med Inform 2013;82:313–24. 10.1016/j.ijmedinf.2012.11.011 [DOI] [PubMed] [Google Scholar]
- 18. Neri PM, Redden L, Poole S, et al. . Emergency medicine resident physicians' perceptions of electronic documentation and workflow: a mixed methods study. Appl Clin Inform 2015;6:27–41. 10.4338/ACI-2014-08-RA-0065 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Oxentenko AS, West CP, Popkave C, et al. . Time spent on clinical documentation: a survey of internal medicine residents and program directors. Arch Intern Med 2010;170:377–80. 10.1001/archinternmed.2009.534 [DOI] [PubMed] [Google Scholar]
- 20. Hirschtick RE. A piece of my mind. John Lennon's elbow. JAMA 2012;308:463–4. 10.1001/jama.2012.8331 [DOI] [PubMed] [Google Scholar]
- 21. Lorence DP, Ibrahim IA. Disparity in coding concordance: do physicians and coders agree? J Health Care Finance 2003;29:43–53. [PubMed] [Google Scholar]
- 22. Leone MA, Gaviani P, Ciccone G. Inter-coder agreement for ICD-9-CM coding of stroke. Neurol Sci 2006;27:445–8. 10.1007/s10072-006-0721-9 [DOI] [PubMed] [Google Scholar]
- 23. Hennessy DA, Quan H, Faris PD, et al. . Do coder characteristics influence validity of ICD-10 hospital discharge data? BMC Health Serv Res 2010;10::9910–696399. 10.1186/1472-6963-10-99 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Spellberg B, Harrington D, Black S, et al. . Capturing the diagnosis: an internal medicine education program to improve documentation. Am J Med 2013;126:739–43. 10.1016/j.amjmed.2012.11.035 [DOI] [PubMed] [Google Scholar]
- 25. Momin SR, Lorenz RR, Lamarre ED. Effect of a documentation improvement program for an academic otolaryngology practice. JAMA Otolaryngol Head Neck Surg 2016;142:533–7. 10.1001/jamaoto.2016.0194 [DOI] [PubMed] [Google Scholar]
- 26. So L, Beck CA, Brien S, et al. . Chart documentation quality and its relationship to the validity of administrative data discharge records. Health Informatics J 2010;16:101–13. 10.1177/1460458210364784 [DOI] [PubMed] [Google Scholar]
- 27. Peabody JW, Luck J, Jain S, et al. . Assessing the accuracy of administrative data in health information systems. Med Care 2004;42:1066–72. 10.1097/00005650-200411000-00005 [DOI] [PubMed] [Google Scholar]
- 28. Castle J, Mlynash M, Lee K, et al. . Agreement regarding diagnosis of transient ischemic attack fairly low among stroke-trained neurologists. Stroke 2010;41:1367–70. 10.1161/STROKEAHA.109.577650 [DOI] [PubMed] [Google Scholar]
- 29. Kidwell CS, Alger JR, Di Salle F, et al. . Diffusion MRI in patients with transient ischemic attacks. Stroke 1999;30:1174–80. 10.1161/01.STR.30.6.1174 [DOI] [PubMed] [Google Scholar]
- 30. Albers GW, Caplan LR, Easton JD, et al. . Transient ischemic attack--proposal for a new definition. N Engl J Med 2002;347:1713–6. 10.1056/NEJMsb020987 [DOI] [PubMed] [Google Scholar]
- 31. Calanchini PR, Swanson PD, Gotshall RA, et al. . Cooperative study of hospital frequency and character of transient ischemic attacks. IV. the reliability of diagnosis. JAMA 1977;238:2029–33. [PubMed] [Google Scholar]
- 32. Seiber EE. Physician code creep: evidence in Medicaid and State Employee Health Insurance billing. Health Care Financ Rev 2007;28:83–93. [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
bmjopen-2016-015234supp001.pdf (354.3KB, pdf)