A surveillance study to determine the accuracy of mild traumatic brain injury diagnosis in an emergency department: protocol for a retrospective cohort study

Abstract

Introduction

Previous literature confirms that a mild traumatic brain injury (mTBI) may result in long-term emotional impacts and, in vulnerable subgroups, cognitive deficits. The accurate diagnosis of mTBI and its written documentation is an important first step towards providing appropriate and timely clinical care. Surveillance studies involving emergency department (ED) and hospital-based data need to be prioritised as these provide incident mTBI estimates. This project will advance existing research findings by estimating the occurrence of mTBI among those attending an ED and quantifying the accuracy of mTBI diagnoses recorded by ED staff through a comprehensive audit of ED records.

Methods and analysis

Retrospective chart reviews (between June 2015 and June 2016) of electronic clinical records from an ED in Sydney (New South Wales, Australia) will be conducted. The study population will include persons aged 18–65 years who attended the ED with any clinical features potentially indicative of mTBI. The WHO operational criteria for the clinical identification of mTBI cases is the presence of: (1) a Glasgow Coma Scale (GCS) of 13–15 after 30 min postinjury or on presentation to hospital; (2) one or more of the following: post-traumatic amnesia (PTA) of less than 24 hours’ duration, confusion or disorientation, a witnessed loss of consciousness for ≤30 min and/or a positive CT brain scan. We estimate that 30 000 ED attendances will be screened and that a sample size of 500 cases with mTBI will be identified during this 1-year period, which will provide reliable estimates of mTBI occurrence in the ED setting.

Ethics and dissemination

The study was approved by the Northern Sydney Local Health District Ethics Committee. The committee deemed this study as low risk in terms of ethical issues. The written papers from this study will be submitted for publication in quality peer-reviewed medical and health journals. Study findings will be disseminated via presentations at national/international conferences and peer-reviewed journals.

Strengths and limitations of this study.

• This study will provide previously unavailable data on the number and proportion of patients attending the emergency department (ED) of a major metropolitan Australian hospital with mild traumatic brain injury (mTBI), and on the number and proportion with clear documentation of mTBI in their medical records.

• Study findings may educate ED staff and/or improve clinical practice.

• The current study will determine the rates of mTBI diagnosis among ED attendances only, and the findings will not be generalisable to the wider community, as it will not capture those cases treated by primary healthcare providers (eg, general practitioners) or alternatively where medical attention was not sought, which may contribute to an underestimation of population-based mTBI incidence.

• As this is not a prospective study, the availability and accuracy of relevant documented clinical information in ED medical records is essential to mTBI identification in this study, and inadequacies in this information may limit conclusions about mTBI in some cases.

Introduction

A WHO review of hospital treated mild traumatic brain injury (mTBI) reported an annual incidence in the range of 100–300/100 000.¹ Because most mTBI is not treated in hospital, the true population incidence of mTBI has been estimated to range between 600/100 000 and 749/100 000 per year.^{1 2} Current literature reaffirms that mTBI may result in long-term emotional disorders,^3–5 and in vulnerable subgroups, cognitive deficits.⁶ mTBI is now being recognised as a major health concern.⁷ Common postconcussive symptoms associated with but not specific to mTBI, are classified as physical (eg, headache, blurring of vision), behavioural (eg, irritability, anxiety) and cognitive (eg, difficulty with memory).³ Although postconcussive symptoms usually resolve within days or weeks, research literature indicates subjective reporting of physical, cognitive and emotional symptoms for several months or years postinjury in a subgroup of individuals.^8–12 Consequences for these individuals may include reduced functional ability, heightened emotional distress and delayed return to work or school.^{5 13 14} However, early identification and subsequent early intervention, such as through education and support for the guided resumption of activities, significantly increases social participation and decreases the severity of postconcussive symptoms.¹⁵ The major limitation of research in this area is that mTBI cases are often underdiagnosed and thus under-reported. Not knowing the true incidence and prevalence of mTBI renders it challenging to allocate resources and inform evidence-based healthcare planning.³

Emergency department (ED) assessment is an important primary point of medical contact for early diagnosis, a key element in the management of mTBI for a significant number of patients.^16–18 However, an accurate clinical identification of patients with mTBI in ED is complicated by varied criteria used for diagnosis.^19–21 Furthermore, variations in diagnostic terminology and diagnostic coding make it difficult for mTBI to be identified through an administrative database.^{19 22–24} A prospective cohort study of all patients presenting to an urban academic ED in the USA over 6 months²⁴ revealed inaccurate identification of patients with mTBI using International Classification of Diseases, Ninth Revision criteria (ICD-9 codes) assigned on discharge. These ICD-9 codes were associated with a significant number of false-positive and false-negative code assignments. Powell et al ²² found that despite patients reporting symptoms consistent with an mTBI diagnosis when interviewed by study personnel, the diagnosis of mTBI was frequently absent from ED medical records. Instead, it appears the ED staff were more focused on ruling out severe brain injury for patients with a likely mechanism for TBI. This approach perhaps reflected the primary mission of the ED, that is, to stabilise and treat serious injuries, and time constraints inherent in ED practice may also have influenced findings. However, it was concluded that persons with a possible mTBI on arrival at ED were more likely not to be diagnosed.²²

Moreover, Cassidy et al reported that 24% of people injured in a motor vehicle crash have a diagnosable mTBI,⁷ and these authors concluded mTBI to be a major health concern in the long term.⁷ Falls and motor vehicle crashes are leading causes of TBI; however, the true distribution of injury mechanisms for mTBI is not known. Given the lack of good-quality published studies on mTBI following motor vehicle crashes, there is an obvious gap in knowledge in this respect.³ Currently, there is limited empirical evidence as to whether persons with mTBI are accurately identified, diagnosed and recorded in Australian ED records (including in the largest state of New South Wales). The need to establish a reliable surveillance system to monitor and inform evidence-based healthcare planning and effective treatment, prevention and rehabilitation strategies for mTBI has been repeatedly emphasised.² Consequently, this mTBI surveillance study will aim to move the research forward in this area. It will involve an electronic clinical record review employing WHO diagnostic criteria²⁵ and secondary criteria previously used by Meares et al ²⁶ to identify the number of patients with evidence of mTBI who were seen at an ED of a major Sydney metropolitan hospital. The study objectives are: (1) to estimate overall, age-specific and sex-specific occurrence rates of ED attendances with mTBI; (2) to assess the extent to which mTBI cases identified via chart review are explicitly identified and documented by ED staff using recorded medical diagnoses and diagnostic codes and to characterise the terminology and codes currently being used in these cases; (3) to assess the proportion of mTBI cases that occur due to falls, motor vehicle crashes or other mechanisms of injury and (4) to describe sociodemographic, injury-related and admission-related characteristics of individuals who attended ED with mTBI based on either confirmed WHO diagnostic criteria or on suggestive but insufficient/ indeterminate evidence using secondary criteria. The proposed study will clarify the nature and scope of mTBI by reporting accurate estimates of its occurrence in an ED setting as well as identifying limitations in current ED diagnosis, documentation and management (eg, assessment, discharge instructions) of mTBI. Therefore, findings from this study have the potential to improve long-term patient outcomes, inform the use of health resources and promote management consistency for the mTBI patient population.

Methods

Sample selection

The proposed study will employ a retrospective surveillance system to determine if an mTBI occurred and a diagnosis was documented by ED staff. The data source for this study will be electronic clinical records related to ED attendances with or without an associated hospital admission at Royal North Shore Hospital (RNSH) over a 1-year period (between June 2015 and June 2016). RNSH is a large hospital in metropolitan Sydney, New South Wales NSW, Australia, serving a population of 213 000 inhabitants in 2016, across four local government areas. The overall number of RNSH ED attendances in the study year was approximately 80 000, and of these, 30 000 were aged 18–65 years old.

Inclusion and exclusion criteria

The inclusion criteria (see figure 1) are adults aged 18–65 years who presented to the ED within 24 hours postinjury with any clinical features indicative of mTBI²⁷ based on the WHO criteria for mTBI^{22 25} which are: (1) Glasgow Coma Scale (GCS) of 13–15 after 30 min postinjury or on presentation to hospital; (2) at least one of the following: post-traumatic amnesia (PTA) of less than 24 hours duration, confusion or disorientation, a witnessed loss of consciousness for 30 min or less and/or positive CT brain scan indicating intracranial injuries not requiring neurosurgery.^{2 25 28} The exclusion criteria include: penetrating brain injury; moderate/severe TBI, spinal cord injury and pre-existing cognitive impairment.^{22 26} Persons who attended ED with head trauma but did not meet the WHO criteria will be classified as indeterminate mTBI if they meet one of the following secondary criteria: (1) if assessed for PTA and obtains optimal scores (ie, 18 out of 18) on the Abbreviated Westmead Post-traumatic Amnesia Scale (A-WPTAS)¹⁸ or (2) if presented to the ED for postconcussive symptoms,²⁹ transient neurological deficits or queried loss of consciousness. Although the literature²⁷ suggests that a diagnosis of mTBI should not be based only on postinjury symptoms, these cases may nevertheless reflect the difficulty in obtaining accurate diagnostic information (ie, WHO criteria) from ED records or the mildest injuries, where manifestations of mTBI resolve prior to the arrival of the medical personnel or presentation to hospital.²⁶ Indeed, these cases are the most difficult to identify.

Figure 1.

Screening and data collection flow chart. Grey boxes indicate study selection process for the identification of mild traumatic brain injury (mTBI). ED, emergency department; GCS, Glasgow Coma Scale; MVA, motor vehicle accident; PTA, post-traumatic amnesia; SCI, spinal cord injury; TBI, traumatic brain injury.

Patient screening and mTBI case identification protocol was developed with input from all investigators. Patient information will be extracted by the research team from the information management system, FirstNet, a module of the Health Electronic Medical Record that is used in New South Wales (figure 1). A limitation of FirstNet is that only a principal diagnosis can be recorded and guidelines are not explicit on whether symptoms or a diagnosis is to be entered.³⁰ If mTBI is the diagnosis of interest, and the patient is not categorised accordingly, they will not be identified through FirstNet. To increase the accuracy of identifying possible patients with mTBI, all ED presentations meeting study age and interval period will be reviewed by a two-step process (figure 1). First, individuals with any possible indicators of mTBI as their discharge diagnosis, will be identified by the research team, using the following key search categories: (1) a diagnosis of head trauma, head injury or brain injury; (2) a mechanism of injury consistent with a possible mTBI (eg, transport-related accident, assault or fall); (3) an injury description that includes head lacerations, bruising, swelling, facial fractures or other musculoskeletal injuries or (4) mTBI-related symptoms, such as pain, neurological or behavioural symptoms. Triage presenting information will then be reviewed and the individual excluded if there is no traumatic mechanism involved or if the mechanism of injury is not consistent with mTBI.

Second, for the possible cases identified in step 1, a thorough review of all documentation available in clinical records will confirm if individuals meet the relevant WHO diagnostic criteria for mTBI. The review will also document whether alcohol and illicit drug usage was involved in the injury.^{18 22 26} The audit of ED records will help obtain information (if available) regarding GCS scores, with mTBI defined as a GCS score of 13–15 after 30 min postinjury or on hospital admission. Information from the A-WPTAS, a validated measure of PTA, will also be collected. The A-WPTAS scale includes eye opening, motor and verbal components from the GCS and a test of ability to recall three picture cards to measure amnesia.¹⁸ Other measures, which will be collected to confirm occurrence of mTBI include: signs of confusion or disorientation, a witnessed loss of consciousness of 30 min or less and a positive finding from brain CT scan or skull X-ray. Lastly, the presence of postconcussive symptoms is also recorded, whether in isolation (indeterminate mTBI) or in association with other indicators.

Two chart auditors (IP and KVV) have been trained in study inclusion and exclusion criteria and case identification, and pilot testing conducted to work through disagreements or differences of opinion until the two auditors arrived at a common understanding and were extracting the same cases at initial screening (per cent agreement of 91%). The first stage of the screening procedure was then assigned to KVV and the second stage to IP, so each step will be consistently undertaken by the same chart auditor. If the chart auditors are unsure about a record, all clinical evidence will be referred to and reviewed by the study investigators (BG, SM) on a weekly basis. The data will be collected using data collection sheets, and will be subsequently entered into a secure online platform, called Research Electronic Data Capture (REDCap).

The proposed retrospective surveillance system will also allow us to obtain sociodemographic, injury-related and admission-related data for individuals identified as having confirmed mTBI based on either WHO diagnostic criteria or suggestive, but insufficient/indeterminate evidence using secondary criteria. Variables will include: postcode, age, sex, ethnicity date of injury, mechanisms of injury (eg, motor vehicle crash, falls, sports-related) and hospital admission status.

Further, for this study, mTBI-related diagnoses (ie, brain injury, mTBI, concussion, postconcussive symptoms or syndrome) listed in the ED medical records and the assigned diagnostic codes (SNOWMED and International Classification of Disease, 10th Revision (ICD-10) codes) will be collected through retrospective review to determine whether an mTBI diagnosis was or was not documented by ED staff.

Sample size

A 1-year audit of clinical and electronic ED records from RNSH will achieve substantial sample numbers. We base this assumption on a previous study by Meares (data not published), which showed that annual patient presentations to another major New South Wales ED in 2010 were 54 473, increasing in 2011 to 56 903. Meares reported in an audit of electronic and clinical medical records that between April and September 2010, there were 19 084 attendances of individuals aged between 18 and 65 years of age, and between April and September 2011, there were 20 024 attendances. The proportion identified with mTBI was 1.1% (n=228) between April and September 2010, and between April and September 2011, there were 1.3% (n=252). Therefore, for this study, we estimate that approximately 30 000 ED attendances will be screened and that 500 mTBI cases will be identified from RNSH medical records over a 1-year period. This sample size provides high precision of estimation of mTBI incidence in the ED setting (95% CIs of ±0.2% for the overall point estimate and ±1.6% for a study stratum that includes only 1% of the total sample).

Outcomes

The primary outcome measure is the rate of mTBI diagnosis among ED attendances (aged 18–65 years) at RNSH over a 1-year period, based on confirmed WHO diagnostic criteria from ED records. Secondary outcomes are: (1) the proportion of mTBI cases that occurred as a result of a motor vehicle crash versus those injured in a fall or sports-related incident and (2) characterising the sociodemographic and other characteristics (eg, age, sex, ethnicity, hospital admission status) of individuals attending the ED with an mTBI diagnosis confirmed based on WHO diagnostic criteria versus suggestive but indeterminate evidence of mTBI using secondary criteria (3) characterise how mTBI diagnoses are noted and recorded by ED staff and explore the management of those injuries in the ED setting.

Data analysis plan

Demographic and clinical data will be summarised using means and SD for continuous variables and frequencies or percentages for categorical variables. Overall, age-specific and sex-specific rates of mTBI over the 1-year period will be calculated with 95% CIs. While FirstNet allows possible mTBI cases to be identified in the first instance, an audit of hospital records will allow an identification of mTBI based on meeting ≥1 of WHO criteria and/or secondary criteria. Among ED attendances identified as involving an mTBI through retrospective chart audit, the accuracy of the ED working diagnosis, documented by clinicians in the medical records, and the accuracy of diagnostic coding entered in Systematized Nomenclature of Medicine—Clinical Terms or SNOMED CT (used within hospital electronic medical records) will be examined. Further, the ICD diagnoses for inpatient admissions of mTBI will also be compared in separate analyses. Rates of agreement and misclassification will be quantified and the kappa statistic for agreement computed.

Sociodemographic, injury-related and admission-related characteristics of mTBI cases will be described. Analyses will be conducted using SPSS and/or SAS statistical software.