Detection of Subtle Cognitive Changes after mTBI Using a Novel Tablet-Based Task

Abstract

This study examined the potential for novel tablet-based tasks, modeled after eye tracking techniques, to detect subtle sensorimotor and cognitive deficits after mild traumatic brain injury (mTBI). Specifically, we examined whether performance on these tablet-based tasks (Pro-point and Anti-point) was able to correctly categorize concussed versus non-concussed participants, compared with performance on other standardized tests for concussion. Patients admitted to the emergency department with mTBI were tested on the Pro-point and Anti-point tasks, a current standard cognitive screening test (i.e., the Standard Assessment of Concussion [SAC]), and another eye movement–based tablet test, the King-Devick^® (KD). Within hours after injury, mTBI patients showed significant slowing in response times, compared with both orthopedic and age-matched control groups, in the Pro-point task, demonstrating deficits in sensorimotor function. Mild TBI patients also showed significant slowing, compared with both control groups, on the Anti-point task, even when controlling for sensorimotor slowing, indicating deficits in cognitive function. Performance on the SAC test revealed similar deficits of cognitive function in the mTBI group, compared with the age-matched control group; however, the KD test showed no evidence of cognitive slowing in mTBI patients, compared with either control group. Further, measuring the sensitivity and specificity of these tasks to accurately predict mTBI with receiver operating characteristic analysis indicated that the Anti-point and Pro-point tasks reached excellent levels of accuracy and fared better than current standardized tools for assessment of concussion. Our findings suggest that these rapid tablet-based tasks are able to reliably detect and measure functional impairment in cognitive and sensorimotor control within hours after mTBI. These tasks may provide a more sensitive diagnostic measure for functional deficits that could prove key to earlier detection of concussion, evaluation of interventions, or even prediction of persistent symptoms.

Introduction

Traumatic brain injury is an emerging major health concern, with 1.7 million cases arising every year.¹ Of these diagnoses, the Centers for Disease Control and Prevention (CDC) reports that 1.3 million patients are admitted and released from the emergency department (ED), with 75% defined as mild traumatic brain injury (mTBI) or concussion, most commonly occurring from falls, motor vehicle accidents, and assaults.^1,2 Despite significant advances over the past several years, the diagnosis or detection and clinical management of recovery after mild head injury remain areas of intense study. Symptoms from mTBI typically present as minor deficits in neurological function, with no structural indications of abnormality using standard clinical imaging (e.g., computed tomography [CT] scan), resulting in difficult detection of underlying neuropathological changes using conventional methods.^3–5 Typically, symptoms subside in natural recovery within 3 months after injury; however some patients experience at least one post-concussion symptom persisting beyond 3 months.^5–8

As a result of the challenges in quickly and accurately identifying severity of damage immediately after mTBI, neuropathological changes contributing to poor outcome largely go unnoticed, with some patients experiencing long-term cognitive–behavioral deficits after discharge from the ED.^3,9–12 However, investigating the presence of post-concussive symptoms persisting longer than 3 months after mTBI has been controversial, and it is likely that other factors, such as comorbidities and pre-injury disorders, contribute to recovery and outcome after injury.^5,8,13,14 Nonetheless, a diagnostic tool that is both sensitive and clinically useful in the detection of a concussion and its severity immediately after injury is essential to improve diagnosis. A tool such as this also may be critical in differentiating conditions, evaluating interventions, or identifying other laboratory biomarkers that may improve outcome and recovery in patients suffering from concussion.¹⁵

Although current standard neuropsychological methods of concussion assessment have been validated and remain clinically useful, they often lack the sensitivity to immediately detect or distinguish subtle functional deficits due to injury.^4,16 Primary limitations of these evaluations include the subjectivity in self-reporting, practice effects, and administrator reliability, as well as cultural, second language, and educational influences. Recent reports using more advanced neuroimaging techniques show great promise in quantifying and detecting subtle physical abnormalities associated with mild TBIs.^17–21 However, these tools are expensive, require trained personnel, and are not yet standardized in clinical diagnostic procedures.^17,18 Therefore, there remains a critical need for the development of an objective behavioral biomarker with the ability to detect severity and predict potential occurrence of persistent deficits that is portable, easily accessible, and inexpensive for quick diagnosis.

One objective method used to study effects of concussion involved measuring simple and choice reaction times.²² This method, however, is not convenient for use in EDs and/or sideline assessment in sports. An additional objective method that has been employed in the evaluation of brain physiology and dysfunction is the tracking and evaluation of saccadic eye movements.^23,24 Eye tracking is a well-established technique in detecting sensitive functional and behavioral deficits by measuring rapid oculomotor movements through variations of simple tasks.²⁵ Two primary tasks in eye tracking assessment used to detect subtle cognitive changes by assessing reflexive and voluntary motor control are the pro-saccade and anti-saccade tasks, respectively.^25–27 In a typical pro-saccade task, an observer fixates on a spot on a computer screen and when a visual target appears in a peripheral location makes a saccadic eye movement to it. In an anti-saccade task, the observer sees the same sequence (fixation spot and then target), but now makes a saccadic eye movement to a location opposite to the location of the target. These tasks can separately measure sensorimotor function by natural reflexive eye movements towards stimuli (pro-saccade), and cognitive function by voluntary control or suppression of these reflexive responses (anti-saccade).^28–30 Maintenance of attention and cognitive control of automatic responses in the anti-saccade task is a sensitive and reliable measure that offers insight into executive function and thus frontal lobe physiology.^30–32 Additionally, the neural circuitry and dynamic properties of saccades are well known and have been well-studied in parallel with other models systems, such as coordinated hand movements, presenting an ideal model for investigating underlying brain function in disease.^33–36

Recently, there has been growing interest in the diagnostic potential of eye-tracking technology for the detection of mTBI. Previous research has shown promising results in concussion assessment using eye-tracking devices, with measureable deficits one week to one year post-injury.^37–43 Although saccadic oculomotor deficits appear to be a likely functional biomarker for concussion, eye tracking devices require trained personnel and lack the accessibility and convenience needed for diagnoses in EDs and/or sideline assessment in sports. In an attempt to resolve this problem, the King-Devick^® (KD), a tablet-based task measuring attention and eye movements through rapid number naming (see the Measures section in Methods), has shown some potential in assessing the cognitive effects of concussion assessment in sports.^44–46 Because KD and other tasks have proven effective for sports-related brain injuries, they may prove useful in detecting mTBIs in EDs. However, KD and other tasks have only demonstrated sensitivity at short time intervals post-injury; consequently, the task has yet to be validated clinically in additional settings at longer time intervals post-injury, such as the ED.

Additionally, KD and other tasks also typically rely on initial baseline measurements that are unavailable in testing non-sport related populations, such as emergency assessment.^47,48 It is important to note that neuropsychological tests, such as the Immediate Post-Concussion Assessment and Cognitive Test (ImPACT), are not used in the ED to diagnose mTBI and thus their sensitivity and specificity remain unknown. These tests instead are used in evaluating sports concussion in a post-game outpatient assessment by a neuropsychologist or other sports health professional.

The aim of this study is to examine the potential of a novel, touch-based approach utilizing methods, paradigms, and analyses similar to eye and hand movement tracking tasks to detect functional cognitive deficits within hours after a mild traumatic brain injury.⁴⁹ Using current tablet technology, this tool was developed and administered with an iPad-based application and is comprised of two tasks (Pro-point and Anti-point; see the Measures section in Methods) modeled after the standard pro-saccade and anti-saccade tasks used in saccadic eye tracking assessment.⁴⁹ The Pro-point task measures stimulus-driven visuomotor function using stimulus-directed hand-point movements. The Anti-point task, with identical visual stimuli and requiring identical hand-point movements, measures cognitive functions through the additional task requirements of inhibition and voluntary control. Specifically, the Anti-point task demands inhibition of the stimulus-directed hand-point movements and generation of a willful or internally-based (“opposite”) hand-point response, thus recruiting the frontal lobe and measuring cognitive control.^50,51 Each pointing task typically takes less than a minute to complete its 48 trials.

These tasks have previously shown a capacity for detecting small but significant cognitive changes in sport-related head injuries after soccer ball heading and could potentially show further diagnostic and predictive ability in clinical settings, such as the ED.⁴⁷ To investigate this, we administered the tasks to patients in the ED who met the diagnostic criteria for mTBI in accordance with the CDC's 2003 report, as well as two control groups, including an orthopedic control group and age-matched control group.² We assessed performance on the Pro-point and Anti-point tasks, a current standard cognitive screening test, specifically, the Standard Assessment of Concussion (SAC), and the KD test.^52–54 We investigated whether the Pro-point and Anti-point tablet-based tasks could detect sensorimotor and cognitive impairment, respectively, in patients admitted into the ED within hours after an mTBI and whether these novel quick measures of performance are more sensitive than other currently available measures.

Methods

Ethics statement

The present study was in approved by the Institutional Review Board at the University of Texas Health Science Center at Houston and is in compliance with the principles expressed in the Declaration of Helsinki. Informed consent for participation in this research study, as well as access to medical records in the ED were obtained from all patients before testing.

Subjects

A total of 30 subjects between the ages of 18–65 years of age were recruited into three groups, including an mTBI group (M), an orthopedic control group (O), and a normal uninjured control group (N). Of the M sample, 11 subjects (average age, 33 years; three female) were recruited from the ED at Memorial Hermann Hospital (Houston, TX) who met the diagnostic criteria for mTBI as defined by the CDC. The inclusion criteria for mTBI included: non-penetrating head trauma, no loss of consciousness or loss of consciousness under 30 min, Glasgow Coma Scale score 13–15, and no abnormalities on a CT scan.^2,55 Additionally, less than 16 h post-injury was also part of the inclusion criteria.

Control subjects included seven orthopedic patients (O group; average age, 31 years; no females) and 12 normal subjects (N group; average age, 33 years; six female). The O subjects were patients recruited in the ED at Memorial Hermann Hospital (Houston, TX) with injuries classified as orthopedic and excluding the head to control for the environment of the ED and the occurrence of a traumatic injury within 16 h. The N control subjects were age-matched to the M group and had little to no history of mild or more serious traumatic brain injury. Group N participants were tested in a controlled environment in the adjacent Medical School building.

Additional inclusion criteria for participants in all groups were blood alcohol concentration less than 0.08 and no pain medications within 24 h. All subjects also were queried regarding their race/ethnicity, visual status, loss of consciousness status (M group only), history of concussion, and cause of injury (M group only; see Table 1 for demographic information). Answers from ED patients were confirmed by attending physicians and/or present family members. Additionally, the injuries of the O group and the M group were assessed to obtain an Injury Severity Score (ISS).⁵⁶ The ISS is an anatomical scoring system that provides quantitative analysis of injury severity in patients with multiple bodily injuries and is commonly used in the ED. All subjects' ED medical records were obtained and assessed for an ISS by a trained clinician based on the presence and severity of non-head injuries (O group) and head injuries and non-head injuries combined (M group).

Table 1.

Demographic Information of Participants

Variable	Normal controlsn = 12	Orthopedic controlsn = 7	mTBIn = 11	p
Age at injury, mean (SD)	33 (15.0)	31 (11.6)	33 (16.5)	0.96/0.25/0.67
Gender (male:female)	6:6	7:0	8:3	0.23/0.89/0.55#
History of concussion (yes:no)	1:10+	0:7	4:7	0.12/0.60/0.30#
LOC under 30 min, n (%)	NA	NA	54.5%
Injury to assessment interval, hours (SD)	NA	5.9 (5.2)	5.9 (5.2)	0.98/0.49
Education, mean (SD)	16.5 (3.1)	13.7 (3.5)	14.09 (2.1)	0.07/0.78/0.99
Cause of injury (mTBI)
Motor vehicle accident			54.5%
Fall			18.2%
Struck by/against			18.2%
Assault			9.1%
Injury Severity Score, mean (SD)		4.7 (3.3)	5.3 (5.1)	0.80/0.86

+One subject's history information was not available

#p value obtained from a generalized linear model run with group (normal controls [N], orthopedic controls [O], mild traumatic brain injury group [M]) as a fixed factor and using a logit link function and a binomial distribution for the dependent variables gender and history of concussion.

All p values are obtained using a mixed model analysis with group (N,O,M) as the fixed factor and subject as the random factor.

Order of p values: All subjects/Only subjects with data from all tests/Only subjects with data from all tests, grouped into un-concussed (N and O) and concussed (M) groups for receiver operating characteristic analysis.

mTBI, mild traumatic brain injury; SD, standard deviation; LOC, loss of consciousness; NA, not applicable.

There were no significant differences between groups in age, education level, history of concussion, or gender (Table 1). In addition, there were no significant differences between the O and M groups in injury severity (determined by the ISS) or the injury to assessment interval (Table 1); however, one O participant was within 17 h post-injury at time of assessment. None of the main findings reported changed with this subject's removal. Task administration did not interfere with treatment or standard hospital procedures.

Measures

Pro-point and Anti-point tasks

Both the Pro-point and Anti-point tasks (Fig. 1) were presented on a tablet screen, which displayed a fixation circle (radius 1.4 cm) surrounded by four outlined square boxes (0.8 cm). To start a trial, the participant is required to place their index finger on the fixation circle. Subsequently, with a fixed delay (280 msec), a target stimulus (white square, 0.8 cm) randomly appears in one of the four boxes. The participant is then required to either touch the square box containing the target stimulus (Fig. 1A, Pro-point task) or touch the square box opposite of where the target stimulus appeared (Fig. 1B, Anti-point task) as quickly and accurately as possible. The two tasks are presented in separate blocks until 48 correct trials are completed for each task. Subjects made very few errors in these pointing tasks (average 3.6%).

FIG. 1.

Representation of display and correct response for the Pro-point and Anti-point tasks on a tablet screen. Trial initiation touch is indicated by the hand with numeral 1. Response touch is indicated by the hand numeral 2. (A) Pro-point task showing correct response from fixation circle (white) to target stimulus (blue circle). (B) Anti-point task showing correct response from the fixation circle (white) to square box opposite of target stimulus (blue circle). Color image is available online at www.liebertpub.com/neu

We recorded two key measures of performance on these tasks, initiation time (IT, in msec) and response time (RT, in msec). IT is defined as the duration of time between the appearance of the target stimulus and the lifting of the participant's finger from the fixation circle and is analogous to saccade latency. RT is defined as the duration of time between the appearance of the target to the time of touch response (used in manual response time analyses). The Pro-point task requires a simple stimulus-driven response (i.e., the time to make a hand-point movement to a visual onset stimulus) and reflects sensorimotor function. In addition to the visual processing of a target and motor response (point to a location), the Anti-point task is considered a goal-directed movement and requires additional cognitive processes often referred to as executive functions.^26,28–30 Namely, the Anti-point task requires subjects to inhibit the pre-potent or stimulus-driven response (pointing to the target) and willfully produce a goal-directed or voluntary hand movement to the location opposite the stimulus. Thus, performance on the Anti-point task also reflects these additional cognitive demands. For additional description of these tasks, please refer to Zhang and colleagues.⁴⁹

SAC

The SAC includes four measures: orientation, immediate memory, concentration, and delayed recall. The participant is required to perform a series of tasks for each measure and every task answered correctly by the participant receives one point. The score for all points acquired on the tasks is then calculated to obtain a total score (SAC), with a maximum of 30 points possible.

KD tablet version

The KD test assesses the participants' speed of rapid number naming, measuring cognitive function with a task that requires intact eye movements and attention. The participant is presented with a demonstration display on a tablet followed by three test trials, with each display containing different arrangements of numbers. On each trial, the participant is required to read aloud the numbers from left to right and top to bottom as quickly and accurately as possible. For each trial, the participant touches the tablet to initiate the display of numbers and touches the tablet again as soon as they have completed reading the numbers aloud. The three test trials become progressively more difficult to read due to removal of lines and decreased spacing between the numbers. As soon as the third test trial is completed, performance is automatically output as the sum of the time taken for the three trials (summary score). Any errors made by the participant also are noted and recorded by the administrator. Participants were consecutively administered two full KD tests (i.e., two sets, each with three trials) and made very few errors on the first test (average 0.83%) and no errors on the second test. The key measure of performance on this task is the lowest of the two summary scores (KD Best). For our analyses, we also used the summary score from the first full test (KD1) and the second full test (KD2) as additional dependent variables to see whether a different selection of performance measure affected our reported findings.

Design, procedure, and counterbalancing

The dependent variables in this study were as follows: For the Pro-point task, IT and RT; for the Anti-point task, IT and RT; for the SAC test, total SAC score; and, for the KD test, each KD test score (KD1, KD2), as well as the best of the two scores (KD Best).

After obtaining consent, each subject was administered the four tasks consecutively. Overall order of administration between the pointing tasks, KD, and SAC tests was randomized to minimize possible practice or order effects. In initial data collection, four M participants and one O participant completed only the questionnaire and two pointing tasks; all additional participants then completed all four tasks (two pointing tasks, KD, SAC). Additionally, there was one O participant who, due to limited or interrupted time, was missing the exact recording of testing time since injury, but met the general inclusion criterion of less than 16 h post-injury.

Statistical analysis

In the Pro-Point and Anti-Point task analysis, all trials containing an error were excluded (3.6%), leaving 48 correct trials per task. ITs and RTs that were greater than 2.75 standard deviations away from the conditional mean (conditional means were calculated by task for each subject) were excluded, removing an additional 3.82% and 5.14% of trials (for Pro-Point and Anti-Point trials, respectively). The remaining trimmed observations for each pointing task were compared among groups used in the mixed effects model analyses for ITs and RTs, where the fixed effect is the group (M, O, N) and the random effect is the study subject. For significant group effects, we followed with mixed effects model post hoc planned comparisons among the three groups (M vs. N; M vs. O; and O vs. N) to evaluate group differences. For some subjects, KD and SAC scores were not recorded; therefore, we also conducted the mixed model analyses for the pointing tasks without those subjects. Our analyses showed no difference in the findings when these subjects were included and when they were removed; therefore, we only report the pointing task analyses when all the subjects were included. One-way analysis of variance was used to compare performance between groups for each KD test and SAC score. For significant group effects, we followed with post hoc planned comparisons among the three groups (M vs. N; M vs. O; and O vs. N) to evaluate group differences. For all statistical tests, a p value less than 0.05 is considered to be statistically significant.

A receiver operating characteristic (ROC) analysis also was performed to determine the sensitivity and specificity of each measure in detecting concussions. For this analysis, data from the O and N groups were pooled together to form the un-concussed group. The concussed group was the same as the M group. The ROC curve, sensitivity (true positive rate, y-axis) versus 1 - specificity (1 - true negative rate = false-positive rate, x-axis) for each test was plotted and the area under the ROC (AUROC) curves was estimated by logistic regression using a maximum likelihood method. A perfect test for discriminating “concussed” would have the ROC curve passing through the coordinates (0, 0), (0, 1) and (1, 1), demonstrating 100% sensitivity and 100% specificity with an AUROC value of 1. A value of 0.5 AUROC, with a line passing through (0, 0) and (1, 1), indicates completely random guessing, with 50% false positives and 50% false negatives. The AUROC criteria were: 0.40–0.59 failed; 0.60–0.69 poor; 0.70–0.79 fair; 0.80–0.89 good, and 0.90–1.00 excellent. Given that the goal of this analysis was to directly compare the sensitivity and specificity of the different tests, we included only those participants that completed all the tests (N = 6, O = 6, M = 7).

Results

Performance

Pointing tasks

Pro-point task

As illustrated in Figure 2, the M group showed significant slowing on the Pro-point task, compared with the N group, as well as the O group, for both IT (83 msec, t[27] = 5.96, p < 0.001 and 47 msec, t[27] = 2.95, p = 0.006, respectively) and RT (164 msec, t[27] = 9.50, p < 0.001 and 90.5 msec, t[27] = 4.55, p = 0.001; respectively). Additionally, the two control groups, O and N, showed significant differences in both IT (35 msec, t[27] = 2.26, p = 0.032) and RT (73 msec, t[27] = 3.77, p = 0.0008). In summary, results for Pro-point performance indicate: 1) significant slowing of simple sensorimotor responses after both a mild traumatic brain injury as well as an orthopedic injury in patients admitted into the ED, compared with uninjured controls and 2) significantly slower responses after mTBI, compared with orthopedic injury.

FIG. 2.

Pro-point performance. Initiation time (IT) and response time (RT) data on the Pro-point task. IT and RT for each group is represented as the mean time (msec) of each hand-point movement across all filtered trials for all subjects (normal controls [N] = 12, orthopedic controls [O] = 7, mild traumatic brain injury group [M] = 11). The Pro-point findings did not differ when removing the subjects that did not have King-Devick^® and Standard Assessment of Concussion scores (subjects remaining, N = 6, O = 6, M = 7). Error bars represent standard error of the mean. *p < 0.05. Color image is available online at www.liebertpub.com/neu

Anti-point task

As illustrated in Figure 3, the M group again showed significant slowing in performance, compared with the N group, as well as the O group on IT (124 msec, t[27] = 9.14, p < 0.001 and 91 msec, t[27] = 5.77, p < 0.001; respectively) and RT (190 msec, t[27] = 16.57, p < 0.001 and 124.57 msec, t[27] = 9.27, p < 0.001; respectively). Similarly to Pro-point performance, there also was a significant difference between the O and N control groups on IT performance (32 msec, t[27] = 2.08, p = 0.047) and RT performance (65 msec, t[27] = 4.98, p < 0.001). In summary, results for Anti-point performance indicate: 1) significant deficits in cognitive performance after mTBI and orthopedic injury, compared with uninjured controls; and 2) significantly greater deficits after mTBI, compared with orthopedic injury.

FIG. 3.

Anti-point performance. Initiation time (IT) and response time (RT) data on the Anti-point task. IT and RT for each group is represented as the mean time (msec) of each hand-point movement across all filtered trials for all subjects (normal controls [N] = 12, orthopedic controls [O] = 7, mild traumatic brain injury group [M] = 11). The Anti-point findings did not differ when removing the subjects that did not have King-Devick^® and Standard Assessment of Concussion scores (subjects remaining, N = 6, O = 6, M = 7). Error bars represent standard error of the mean. *p < 0.05. Color image is available online at www.liebertpub.com/neu

Additionally, to control for Pro-point differences affecting the Anti-point task, the mixed effects analysis was performed again using the conditional means of Pro-point variables as covariate to determine whether Anti-point differences between groups in the Anti-point task could be accounted for simply by sensorimotor differences or whether there remained additional specific cognitive deficits. Anti-point differences between groups remained mostly significant when controlling for sensorimotor deficits. The M group continued to show significant slowing in performance, compared with the N group, as well as the O group, for both IT (94 ms, t[27] = 8.49, p < 0.001 and 70 msec, t[27] = 5.66, p < 0.001, respectively) and RT (139 msec, t[27] = 13.28, p < 0.001 and 91 msec, t[27] = 8.27, p < 0.001, respectively). Additionally, Anti-point differences between the O and N groups in RT (48 msec, t[27] = 4.51, p = 0.001) remained unchanged; however, the difference in IT (23 msec, t[27] = 1.97, p = 0.058) did change to a marginally non-significant trend.

SAC

The total score on the SAC test for each group is illustrated in Figure 4A. Performance on the SAC test revealed a significant decrease in SAC score in the M group, compared with the N group (t[16] = −2.61, p = 0.019) and the O group (t[16] = −2.21, p = 0.041), indicating worse performance on the SAC in the M group. There were no significant differences between the O and N groups (p = 0.70). In summary, results on the SAC test indicate deficits after mTBI in the four functional measures: orientation, immediate memory, concentration, and delayed recall, compared with uninjured normal controls and orthopedic injury patients.

FIG. 4.

Performance on the Standard Assessment of Concussion (SAC) and King Devick^® (KD) tablet version. (A) Total points on the SAC represented in total score (max = 30; normal controls [N] = 6, orthopedic controls [O] = 6, mild traumatic brain injury group [M] = 7). (B) Total minimum time taken to complete all three cards in the KD task (KD Best) represented by seconds (N = 6, O = 6, M = 7). Error bars represent standard error of the mean. *p < 0.05. Color image is available online at www.liebertpub.com/neu

KD tablet version

The KD task was administered twice consecutively, in which the best (lowest) score out of the two was used for analysis. As illustrated in Figure 4B, performance on the KD task revealed no significant differences among the three groups (p = 0.72). Results indicate that mTBI patients admitted into the ED show no significant slowing in rapid number naming on the KD task within hours after an mTBI. There also were no significant differences between groups using either the first or second total score (KD1 and KD2, data not shown).

Sensitivity and specificity

In the ROC analyses (Fig. 5 and Fig. 6), we found that Anti-point RT yielded the greatest AUROC (0.98; 95% CI = [0.96, 1.00]). These results indicate that Anti-point RT appears to be highly sensitive and specific in determining concussed versus non-concussed patients within hours after injury. Pro-point RT showed an AUROC of 0.93 (95% CI = [0.84, 1.00]), also demonstrating “excellent” ability for identifying mild head injury. The SAC total score also demonstrated “good” ability in its sensitivity and specificity of mild head injury with an AUROC of 0.84 (95% CI = [0.65, 1.00]). Conversely, the KD Best achieved only an AUROC of 0.53 (95% CI = [0.26, 0.80]). Neither KD1 nor KD2 fared much better (Fig. 6), suggesting nearly complete random chance in the ability of the KD measures to determine which patients were concussed. These results indicate that in this small sample, RTs in the pointing tasks have excellent sensitivity and specificity to accurately detect sensorimotor and cognitive impairments within hours after injury.

FIG. 5.

Receiver operating characteristic (ROC) analysis. ROC curve and corresponding area under the curve (AUROC) statistics for categorizing “concussed” and “nonconcussed” using Anti-point and Pro-point RT, Standard Assessment of Concussion (SAC) score, and King Devick^{® (}KD) Best scores. Only subjects that had completed all tests (normal controls [N] = 6, orthopedic controls [O] = 6, mild traumatic brain injury group [M] = 7) were included in this analysis. Color image is available online at www.liebertpub.com/neu

FIG. 6.

Area under the receiver operating characteristic curve (AUROC) scores for all tasks and measures. AUROC statistics and 95% confidence intervals for categorizing subjects as “concussed” and “nonconcussed” using Anti-point and Pro-point response time (RT) and initiation time (IT); Standard Assessment of Concussion (SAC) score; and King Devick^{® (}KD)1, KD2, and KD Best scores. Color image is available online at www.liebertpub.com/neu

Discussion

The purpose of this study was to determine the validity of a novel tablet-based approach for measuring subtle cognitive deficits using eye tracking techniques within hours after mTBI. Results show that the Anti-point task did indeed detect deficits of executive function within hours after a mild traumatic brain injury as measured by slowing in voluntary or goal-driven responses. In addition, we found greater slowing of stimulus-driven responses in the Pro-point task in mTBI patients, compared with both control groups, indicating sensorimotor deficits in patients who are admitted into the ED with head injury. Further, ROC analyses indicated that RTs in pointing tasks reached an excellent level of accuracy in detecting concussion and fared better than current standardized tools. These findings suggest that these novel pointing measures are able to detect measurable impairment in executive function, as well as sensorimotor function, within hours after mTBI.

Separately, we also found significant slowing in both sensorimotor and cognitive RT measures in the orthopedic group, compared with normal controls. These findings suggest that these tablet-based measures show a sensitivity to detect subtle changes in sensorimotor and executive function deficits that may occur with orthopedic injury, such as occurrence of a trauma or pain. Previous research has demonstrated that experience of pain is associated with sensorimotor impairment, as well as cognitive impairment.^57,58 However, it is important to note that there were no significant differences in ISS assessment between the mTBI group and the orthopedic group (Table 1).

To improve clinical assessment and management, a more sensitive and clinically useful method of concussion detection is critical for immediate diagnosis after injury. In this study, we propose a novel method in concussion diagnosis to detect cognitive changes in mTBI using a tablet-based task. Our results are consistent with previous research demonstrating oculomotor deficits as an effective measure for detection of mild traumatic brain injury with measurable deficits in saccade and anti-saccade eye movements after 1 week post-injury.^37–43 Additionally, our reaction time analysis is consistent with previous research investigating reaction time changes using various stimulus detection tasks after TBI. These studies have shown deficits in attentional capacity and information processing speeds, including visuomotor processing, although at longer time-points post-injury than the current study.^59–62 The ability for these tasks to accurately and objectively measure deficits based on RT measures provides a promising new technique in detection of functional cognitive impairment within hours after brain injury. The high sensitivity and specificity of the RT tasks indicate that they may be useful in detecting small changes that could predict post-concussion symptoms. Additionally, further research is required to determine how soon after injury these measures are able to detect changes, whether these changes relate to or predict outcome, and the reliability and validity of these measured impairments in different populations for future usefulness in the detection or diagnosis of mTBI.

In this study, we also measured cognitive performance using a brief cognitive screening tool for concussion assessment, the SAC, presently used in sports to provide clinicians with a more objective and standardized method of immediately assessing an injured athlete's mental status on the sport sideline.⁵² Despite relatively small groups, we found a significant cognitive impairment as measured by the SAC in the mTBI group, compared with control groups. These findings are consistent with previous research investigating both ED assessment and sports-related assessment of concussion, providing further independent validation of mild traumatic brain injury in our mTBI group.^52–54,63 Additionally, ROC analysis of SAC scores from mTBI subjects and controls showed a “good” ability to categorize subjects as concussed versus non-concussed. It should be noted that “concussed” and “non-concussed” are defined using the conventional gold standard clinical diagnosis, including measures involving history, clinical exam, and post-concussion symptoms. Together, these results, from an assessment currently used in the field, indicate that orientation, immediate memory, concentration, and delayed recall measured by the SAC test may be useful measures for detecting deficits within 16 h after injury in an ED setting. However, it is important to note that the sensitivity and specificity of the SAC test as determined by the AUROC score was still lower than those of pointing tests.

The KD test, a tablet-based sideline assessment of sports concussion, showed no evidence of cognitive slowing in mTBI patients admitted to the ED, compared with controls. These KD results support previous findings of no significant differences in mTBI patients versus normal controls when assessing KD performance in an ED sample without within-subject baseline analysis.⁶⁴ It is important to recognize that the KD is primarily used for concussion assessment in sports where testing is performed immediately after injury and includes an initial pre-injury baseline assessment for within-subject analyses. Given that accurate baselines may not be available in many traumatic brain injuries, judging performance without an accurate baseline is an important factor that nevertheless must be addressed.

Further, any measures that require an accurate baseline also must take into account potentially interfering variables, such as populations with ongoing developmental brain changes, effects of adrenaline, and/or susceptibility to practice effects or malingering. Nonetheless, our results show that KD assessment itself, without a prior baseline and at longer time intervals post-injury, does not have the capability of detecting cognitive changes after concussion in emergency settings. Further, ROC analysis using any KD measure (KD1, KD2, or KD Best) also demonstrated a failure to accurately classify individuals as concussed or not-concussed. Thus, the KD assessment using rapid number naming as a measure of cognitive function is not able to detect or correctly classify cognitive changes in mTBI patients who were admitted into the ED within hours post-injury.

In this small sample set, the tablet-based pointing tasks appear to have greater sensitivity and specificity to accurately detect functional sensorimotor and cognitive deficits within hours after injury than the current standard tools for concussion. These simple measures of sensorimotor and executive function appear to provide quick and effective assessments of mTBI within hours after injury. The Pro-point and Anti-point tasks provide objective and quantitative measures, are rapid, less expensive, and do not require trained personnel to administer and collect data, rendering these tasks as good candidates for clinical use. Additionally, these tasks may have the potential to determine severity of physiological dysfunction in the brain within hours after injury that could improve clinical management and treatment. However, there are several limitations of the study. First, the study has relatively small sample sizes and larger studies would be needed to bolster generalization of findings. Second, although there were no differences in demographic factors or risk factors in our small study, additional studies manipulating such variables, including previous concussion and age, would be important to better understand whether or how these factors influence performance.

This study is only a first step in simplifying concussion assessment, and future investigations are necessary to determine the extent to which these tasks may benefit clinical diagnosis and management. Such a tool suggests many different avenues for future investigation. First, how soon after injury can these changes be detected? Second, whether, or to what extent, do these changes in sensorimotor and/or executive function after injury predict increased risk of persistent post-concussive symptoms. Additionally, can one detect similar changes in other populations, such as sport injury, immediately after mTBI, where exercise and adrenaline may interfere with measures at the time of injury? These questions will be critical in future investigations. A tool that has potential predictive value (which patients might be at risk for persistent post-concussive symptoms) or usefulness in measuring and evaluating the efficacy of interventions is clinically consequential. In conclusion, these findings demonstrate that these quick tablet-based measures are able to reliably detect sensorimotor and cognitive impairments within hours after a mild traumatic brain injury and in the future may prove useful in evaluating interventions or predicting persistent post-concussion symptoms at the time of injury onset.

Acknowledgments

We thank Alex Hacopian, Angela H. Lin, Namrataa Mehta, and Michelle Won for help in data collection, and Pramod K. Dash and Sylvain Laroche for manuscript review.

This work was supported in part by grants from Mission Connect, a program of TIRR Foundation, and NIH P30-EY010608.

Author Disclosure Statement

No competing financial interests exist.

Co-authors A.B.S., S.S.P., and S.D.R. are named inventors in a patent application for a “Touch sensitive system and method for cognitive and behavioral testing and evaluation” (US 2014/0249447 A1).