Skip to main content
PMC home page
International Journal of Sports Physical Therapy logoLink to International Journal of Sports Physical Therapy
. 2021 Aug 1;16(4):991–1000. doi: 10.26603/001c.24924

Accuracy of the SWAY Mobile Cognitive Assessment Application

Heidi A VanRavenhorst-Bell 1,✉, Melissa A Muzeau 2, Lindsay Luinstra 1, Jared Goering 3, Ryan Z Amick 4
PMCID: PMC8329324  PMID: 34386278

Abstract

Background

Mobile electronic devices have become integral tools in addressing the need for portable assessment of cognitive function following neurocognitive/motor injury. SWAY Medical, Inc., has employed mobile device motion-based technology in the SWAY Cognitive Assessment (SWAY CA) application to assess cognitive function.

Purpose

The purpose of this study was to assess whether the SWAY CA application (reaction time, impulse control and inspective time) was able to reliably operate on different mobile devices and operating systems (iOS, Android). The study further sought to assess the validity of the SWAY CA application against the FDA approved ImPACT QT mobile device application.

Study Design

Original Research, observational study of validity.

Methods

88 healthy, young adults, 18 to 48 years (mean= 22.09 ± sd=4.47 years) completed four, randomized and counter-balanced, reaction time tests (2- SWAY RT, 2- ImPACT QT) using different operating systems (iOS, Android) of 4 randomly assigned mobile devices.

Results

ANOVAs reported the SWAY CA application (reaction time, impulse control, inspection time) operated reliably with iPhone 6S, Samsung Galaxy S9, and iPad Pro 5 mobile devices (p > 0.05), respectively. Google Pixel 3 reliability with SWAY CA application remains undetermined. SWAY CA simple reaction motion measures were in agreement (r = -0.46 to 0.22, p ≤ 0.05) with several ImPACT QT reaction time measures. SWAY CA impulse control and inspection time measures are weakly correlated (r = -0.25 to -0.46, p ≤ 0.05) with five ImPACT QT reaction time measures.

Conclusion

The motion-based SWAY CA mobile device application appears to reliably operate when being administered on different mobile devices and software operating systems. Furthermore, the SWAY CA application appears to be comparable to the ImPACT QT and serve as a valid tool for assessing reaction time measures.

Level of Evidence

Level 2b (observational study of validity).

Keywords: impact qt, mobile device, reaction time, cognitive assessment, sway app

INTRODUCTION

Reaction time is known as an individual’s rate of response (or amount of time lapsed) following the introduction of a known or unknown stimulus.1 It is an important indicator of one’s neurocognitive and functional health,1–3 as well as being a key factor in many daily activities such as participating in sport, driving a car, and even in emergency situations.3 The assessment of reaction time has long been used to evaluate an individual’s cognitive, neurological, and motor (dys)function,4 and more recently has become a respected measure for return-to-play in sport(s) following mild-traumatic brain injury (MTBI).3,5,6 For example, immediately following a sport-related concussion, it is widely accepted that an individual will present with a prolonged reaction time due to an insult on the brain.3,5,7 With time, concussion-induced disruptions in neurocognitive and functional performance are often shown to gradually dampen, and an improvement in reaction time returns.1,8 Furthermore, routine follow-up reaction time assessments are commonly performed and compared to an individual’s pre-concussion (baseline) reaction time measures to determine post-concussion improvements in neurocognitive and functional health prior to an athlete being released for a return-to-play.5,7

The assessment of reaction time has many benefits which include but are not limited to serving as a parallel indicator of one’s central processing speed and cognitive function.5 Traditionally, qualitative evaluations of neurocognitive function and reaction time measures down to the millisecond have involved some form of computerized testing (e.g., software on a desktop computer with a keyboard and mouse).1,5,7,9 Computerized testing is known for its accuracy and reliability5,10; however, are generally administered in a clinical setting and commonly criticized for their lack of portable practical application.5,9 Even laptops, which are viewed as a portable computerized device, require some set-up, along with an appropriate setting to successfully administer a cognitive and reaction time assessment. Such requirements complicate the feasibility of a portable on-field (i.e., athletic venues, athletic training room, military field hospital) assessment application. This is of concern because timely administration of cognitive and reaction time assessments are critical when assessing a potential on-field neuromotor injury.11 A delay in assessment may allow for misdiagnosis, which could result in harm or death of the patient or athlete. Therein, supports the need for a portable practical application to assess reaction time.

Mobile electronic devices such as smartphones and tablets are portable and user-friendly in most any setting (e.g., clinical, medical, and on-field). Most mobile electronic devices are also capable of operating mobile application software as well as administering various health and sport related assessment measures due to an inertial measurement unit (IMU) system built into the mobile device.12–14 IMUs measure specific force, angular velocity and sometimes the orientation of the body or movement of the device, using a combination of magnetometers, gyroscopes and triaxial accelerometers.12,15 In addition, mobile device applications can provide rapid biofeedback (e.g., neurocognitive measures, neuromotor measures, reaction time measures) based on the device IMU measures.12,16 Due to the portability and cognitive assessment application capabilities of a mobile device, use in assessing neurocognitive and neuromotor injuries has become of interest.7,9

One such mobile device application is the Immediate Post-Concussion Assessment and Cognitive Test Quick Test (ImPACT QT). The ImPACT QT is an FDA cleared mobile device application developed to assess neurocognitive function following a suspected concussion.17 Due to its mobility and ease of use, the ImPACT QT is commonly used for sideline assessments in high school and collegiate athletics, as well as during routine clinical assessments.5,7 The five-minute ImPACT QT test includes a series of neurocognitive modules (symbol matching, three letter memory, reverse number counting, attention tracking) administered on a tablet screen. An individual’s rate of response (e.g., neurocognitive and reaction time measures) is recorded by touching the tablet screen following a visual prompt displayed on the display screen. During the assessment, the tablet may be held with both hands or placed on a flat surface while remaining in a standing posture. Following completion of the test, the ImPACT QT application provides three composite scores that may be compared against a subject’s previously established baseline measures.18,19 A decline in the composite scores is often used as an indication of a potential decline in neurocognitive function and consideration for removal of an individual from activity.6,19,20 Wallace and colleagues,19 however, caution of interpretation based on a single low score without cause of concern of a concussion because healthy non-concuss individuals have been shown to randomly present with an unexplained low score.

The ImPACT QT does present with a few limitations. First, the ImPACT QT is only compatible with an iOS (Apple, Cupertino, CA, USA) touch-screen iPad.6,19,20 An iPad, although well-accepted, is not the universal tablet among all end-users, nor clinical and athletic programs. This greatly marginalizes its accessibility and intended purpose of providing critical and often time sensitive sideline assessments. In addition, due to slower software and processor, the iPad is observed to have screen capacitance latency and test results are susceptible to a wider range of variability compared to a traditional desktop or laptop computer assessment. As screen latency can range from 50 to 200 milliseconds, latency induced variability may have an indirect effect on an individual’s true reaction time scores, potentially impacting clinical decisions.1,5,9,21–23 For example, if during an athlete’s baseline assessment screen was between 100 to 200 milliseconds, and was between 50 to 100 milliseconds during an on-field concussion assessment; the end result could be a missed or failed interpretation. The on-field assessment indicated a faster, although inaccurate reaction time measure in comparison to the athlete’s baseline measure. Such inconsistencies due to latent variability of the iPad may place a patient-athlete at risk for another traumatic event that could potentially be more detrimental to neurocognitive function or even fatal.24 In addition, healthcare professionals may experience limitations in making an appropriate diagnosis when attempting to evaluate data comparisons between computer aided testing and reaction time assessments administered on an iPad due to this wide range variability.

Recently, a new method for assessing cognitive function using a mobile device was introduced by SWAY Medical, Inc. The SWAY Cognitive Assessment (SWAY CA) mobile device application registers movement of the mobile device, instead of registering an applied touch screen response.9,25,26 The SWAY CA introduces a series of neurocognitive modules that evaluate an individual’s reaction time, inspection time, impulse control, and working memory-delayed recall. While an individual holds the device with both hands, a module prompts a visual cue on the display screen and evokes the individual to engage in or refrain from an active response (moving the device). An active response is recognized as a minimum motion-based threshold detected as the device is moved in any direction. The cognitive function and reaction time measures (time lapsed from the presence of a stimulus to the initiation of an action) for each of the three SWAY CA modules (simple reaction motion, impulse control, and inspection time)(see Methods section) are reported in milliseconds (ms) and a proprietary SWAY score calculated on a 100-point scale.25,26 The closer an individual’s score is to 100 the better one’s cognitive function and reaction time.25,26 The SWAY CA working memory-delayed recall module, however, is a single proprietary SWAY score based off lapsed time to recall, number of correct recall, and number of sequential squares tracked and recorded correctly.26

To assess movement of the mobile device and interpret one’s rate of reaction time, SWAY CA’s proprietary algorithm uses a triaxial accelerometer motion-based system that is housed within the mobile device. Due to the orthogonal (right angles) placement of the three sensors in reference to each other, detection of device movement and vibration in any direction is registered with increased sensitivity compared to a system with less than three sensors. This increased sensitivity to motion has been shown to minimize mobile device latency down to one to two milliseconds.21,23,26,27 This is a pronounced improvement compared to touch-based reaction time mobile device detection with an average latency of 50 to 200 milliseconds.1,5,9,21–23 An additional advantage to SWAY CA is that it can be used on multiple platforms (smartphones and tablet) and is compatible with iOS (Apple, Cupertino, CA, USA) and Android (Samsung Group, Seoul, South Korea; Google, Mountain View, CA, USA) operating systems.25,26,28

While the prospect of using mobile electronic devices as a clinical evaluation tool has many advantages, developers must ensure that their applications provide consistent results across all devices on which they are intended to operate. This is because, among the most popular smartphone and tablet devices, the number of different hardware and software combinations being used is numerous. Such differences may result in minor compatibility issues that impact processing speed, display screen refresh rate, and input latency. Additionally, different manufacturers may use different solutions for analyzing raw data from integrated sensors.1,12,23 Ultimately, for a mobile application to be versatile and provide clinically relevant and reliable assessments, it is essential to account for these differences across a spectrum of mobile devices and operating systems.

The purpose of this study was to assess whether the SWAY CA application (reaction time, impulse control and inspective time) was able to reliably operate on different mobile devices and operating systems (iOS, Android). The study further sought to assess the validity of the SWAY CA application against the FDA approved ImPACT QT mobile device application.

METHODS

Site Selection

This study was completed in the Human Performance Laboratory (HPLab) at Wichita State University, Wichita, Kansas. This site was selected as the HPLab is experienced in the development and evaluation of mobile device applications.

Participants

A total of 90, college-aged individuals with a mean age of 22.09 ± sd= 4.42 years volunteered to participate in the study. An a priori power analysis was conducted using G*Power 3.1 software (Heinrich-Heine-Universitat Dusseldorf, Dusseldorf, Germany) to identify appropriate sample size. To achieve a power of 0.80 with an α error of probability ≤ 0.05 and a medium effect size, a sample size of 84 participants was required. Volunteers were recruited through direct contact, and technology-based communication, as well as through print materials posted in public areas on the university campus. The Wichita State University Institutional Review Board approved the study, and an informed consent form was obtained from all volunteers prior to completing any questionnaire(s) or participating in data collection.

Inclusion and exclusion criteria. Any pre-existing condition that could interfere with successfully completing the assessment was identified based on the 2020 Physical Activity Questionnaire Plus (PAR-Q+).29 A participant was excluded from the study if they were under the age of 18 year, and were excluded if they reported any of the pre-existing conditions presented as follows; any current medical condition or medical history of a 1) musculoskeletal injury affecting functional movement and balance, 2) neurological dysfunction, 3) uncorrected vision, 4) vestibular disorder or condition, and/or 5) current, un-prescribed or prescribed pharmacological intervention affecting functional movement and balance.

Of the initial 90 volunteers, one participant was excluded for meeting one or more of the exclusion criteria. The remaining 89 participants met the intake questionnaire and were included in the study. One additional participant was removed from the study due to a technology error and inability to download the data output from the mobile device. For the remaining 88 participants, Table 1 provides the demographic information (age, sex), as well as anthropometric measures (height, weight) collected.

Table 1. Subject Demographic Information.

Male (n = 32) Female (n = 56) Total (N = 88)
Mean ± SD
Age (years) 22.38 ± 5.68 21.93 ± 3.63 22.09 ± 4.46
Stature (cm) 178.73 ± 8.27 167.25 ± 7.42 171.43 ± 9.49
Weight (kg) 83.25 ± 14.9 76.25 ± 20.26 78.79 ± 18.72
Open in a new tab

n = Sum of sample, N = Sum of total sample, cm = Centimeters, kg = Kilograms

SWAY Mobile Application

The SWAY System (SWAY Medical Inc., Tulsa, OK, USA) is a mobile device application designed to assess balance (SWAY Balance) and cognitive performance (SWAY CA) through the use of different assessment modules. Both segments of the SWAY System rely primarily on the analysis of movement, as measured through the mobile device’s integrated triaxial accelerometer, to determine performance scores.9,25,27,28,30,31 Evaluation of the balance assessment segment of the SWAY System has previously been reported and received FDA Class II approval.28 The cognitive (reaction time) testing segment has also been evaluated and established clinically reliable and valid measures in comparison to the standard Computerized Test of Information Processing (CTIP) assessment.9 However, SWAY CAs capacity to execute on various mobile devices and operating systems, as well as deliver measures consistent in comparison to the standardized ImPACT QT mobile application remain to be validated. The cognitive performance segment of the SWAY System, SWAY CA, administers three sensory and neuromotor based modules to assess stimulus recognition, cognitive processing speed, neuromotor response, working memory and reaction time.

  • Module 1 – Simple Reaction Time

    • Move the device as fast as one can in any direction when the screen of the device turns orange.

  • Module 2 – Impulse Control

    • Move the device as quickly as possible when you see a green check mark.

    • When you see a red X, keep the device still.

  • Module 3 – Inspection Time

    • Two T-shaped lines will be shown on the device. Once the two lines are masked (covered), you will be instructed to move the device to the side with the longer line.

    • Do not move the device if you are unsure which line was longer. An incorrect response will reduce one’s score.

SWAY CA utilizes tri-axial accelerometers built-in to most mobile devices to detect motion21,25,26,28,32 and measure reaction time in reference to a known stimulus.1,2 Overall, SWAY CA is completed in three to five minutes by the participant.

For each SWAY CA module, participates were instructed to follow the SWAY System instructions displayed on the device screen. Research personnel trained on the SWAY System were continually present to offer further clarification to participants if needed on the application or electronic device. For all modules, participants were instructed to hold the device with both hands and maintain a standing position.

Participants were randomly issued one of four mobile devices preloaded with the SWAY CA application. Device selection was based on convenience of accessibility at time of the study. Mobile devices included:

  1. Apple iPhone 6s Plus, Software Version – iOS 12.2 Model: MKTQ2LL/A, Serial: C38QFBM5GRWT (Apple Computer Inc., Cupertino, CA, USA)

  2. Samsung Galaxy S9, Software Version – Android 9 (8.0.0), Serial: R58M217YT7P (Samsung Group, Seoul, South Korea)

  3. Google Pixel 3, Software Version – Android Version 9 (8.0.0), Serial: 89VXOHN87 (Google LLC., Mountain View, CA, USA)

  4. Apple iPad 5 Air, Software Version – 12.1.1 (16C50), Model: MR7F2LL/A, Serial: DMRY26GRJF8J and Serial: DMRY236PJF8J (Apple Computer Inc., Cupertino, CA, USA)

ImPACT Quick Test Mobile Application

The ImPACT QT (ImPACT Applications, Inc., San Diego, CA, USA) is an FDA approved iPad-based neurocognitive test designed for clinical use (e.g., concussion baseline measures, pre- and post-neurocognitive injuries (concussion assessment)).6,17,33 ImPACT QT administers three neurocognitive modules to assess basic output related to neurocognitive functioning, working memory, processing speed, reaction time and symptom recording in a brief five-seven minutes.6,19,33 The three neurocognitive modules are as follows.

  • Module 1 – Symbol Matching

    • Trial 1: Match shapes with numbers using the touch screen as quickly as you can.

    • Trial 2: Remember which shape goes with what number using the touch screen as quickly as you can.

  • Module 2 – Three Letter Memory and Reverse Number Counting

    • Trial 1: Count backwards from 25 to 1 using the touch screen as fast as you can.

    • Trial 2: Remember a set of letters flashed on the touch screen. Then count backwards from 25 to 1 using the touch screen as fast as you can. Immediately following, type the letters you were asked to remember.

  • Module 3 – Attention Tracking

    • Visually track a moving object on the touch screen. When you identify the object change from read to green you will click on the circle as fast as you can.

The ImPACT QT test was administered utilizing an Apple iPad Pro 5, [Software Version – 12.1.1 (16C50), Model: MR7F2LL/A, Serial: DMRY26GRJF8J and Serial: DMRY236PJF8J (Apple Computer Inc., Cupertino, CA, USA)]. For each module, instructions were provided on the device screen as well as each participant was provided verbal instruction from an experienced research administrator. Participants were instructed to lay the device flat on the countertop surface and maintain a standing position while performing each module.

Procedure. Each participant completed a total of four cognitive assessments (two SWAY CA and two impact QT). Following a similar protocol described in detail in a previous study for balance by Amick and colleagues,28 each participant completed one familiarization trial and one experimental (baseline) trial for each application (SWAY CA, ImPACT QT). To control for a learning effect and bias, participants were issued one of the four previously described preloaded SWAY CA application mobile devices, and a preloaded ImPACT QT iPad in a randomized order. In addition, the order of the two application cognitive assessments (SWAY CA, ImPACT QT) was counter-balanced (e.g., SWAY – ImPACT QT – SWAY – ImPACT QT, or ImPACT QT – SWAY – ImPACT QT – SWAY). Each participant was provided a two-three-minute seated rest period between test applications. The research administrator used a stopwatch to maintain consistent rest periods.

Data Analysis

Statistical analysis was conducted using the Statistical Packages for the Social Science (SPSS) version 23.0 with a level of significance set at α ≤ 0.05 and a confidence level of 95%. All test variables were evaluated for normality of distribution.

Three separate one-way analyses of variance (ANOVAs) were conducted to determine group mean difference of Mobile Device (iPhone 6s Plus, Google Pixel 3, Samsung S9, iPad Pro 5) on each of the SWAY CA baseline measures (simple reaction, impulse control, inspection time). The critical alpha level for each ANOVA was set at p ≤ 0.05. A post-hoc test was completed at a p ≤ 0.05 if a significant mean difference was reported.

A Pearson’s Product Moment Correlation Coefficient (r) was conducted to determine the degree of correlation in baseline SWAY CA measures (simple reaction motion, impulse control, inspection time) and the ImPACT Quick Test application battery of modules at a p ≤ 0.05. The Coefficient of Determination (r2) was further calculated to determine the amount of shared variance between the SWAY CA and ImPACT QT scores. A Pearson’s Product Moment Correlation Coefficient Interpretation as follows, weak r= 0.00 to 0.30, moderate r= 0.31 to 0.59, and strong r= 0.60 to 1.00.7,17

RESULTS

All SWAY CA and ImPACT QT measures were inspected and found to fall within an acceptable range and demonstrated a normal distribution. Table 2 provides the means and standard deviations of each SWAY CA measure (simple reaction, impulse control and inspection time) by mobile device (iPad, iPhone 6S, Google Pixel 3 and Samsung Galaxy 9S).

Table 2. Means and Standard Deviations of SWAY CA Simple Reaction, Impulse Control and Inspection Time by Mobile Device.

SWAY CA Modules (Mean ± SD)
Mobile Device Simple Reaction Impulse Control Inspection Time
iPad 75.38 ± 0.00 59.58 ± 0.00 92.50 ± 0.00
iPhone 6S 73.91 ± 7.56 61.85 ± 7.15 81.73 ± 18.27
Google Pixel 3 71.45 ± 8.15 54.34 ± 0.08 97.50 ± 3.54
Samsung Galaxy S9 74.73 ± 7.61 62.76 ± 6.72 86.09 ± 13.24
Open in a new tab

A one-way analysis of variance (ANOVA), as shown in Figure 1, determined SWAY CA Simple Reaction mean difference did not significantly differ between Mobile Devices, F (3, 84) = 0.182, p = 0.91, ηρ2 = 0.01. One-way ANOVA, as shown in Figure 1, determined SWAY CA Impulse Control mean difference did not significantly differ between mobile devices, F (3, 84) = 1.02, p = 0.39, ηρ2 = 0.04. In addition, the one-way ANOVA, as shown in Figure 1, determined SWAY CA Inspection Time mean difference did not significantly differ between mobile devices, F (3, 84) = 1.08, p = 0.36, ηρ2 = 0.04. Post-hoc tests were not administered based on the lack of significant differences found between mobile devices for each of the SWAY CA measures. As shown in Table 3, it is important to address that Google Pixel 3 (n = 2) and the iPad Pro 5 (n =1) each reported a very small group sample and will be further addressed in the discussion.

Figure 1. Mean difference in SWAY CA Simple Reaction Motion, Impulse Control, and Inspection Time Measures between Mobile Devices.

Figure 1.

Open in a new tab

Table 3. Total Number of SWAY CA and ImPACT QT Assessments by Mobile Device.

Assessment
Device SWAY CA ImPACT QT
iPhone 46
Google Pixel 2
Samsung 39
iPad 1 88
Total 88 88
Open in a new tab

As shown in Table 4, a Pearson Product Moment Bivariate Correlation Coefficient (r) determined that SWAY CA simple reaction motion was negatively correlated, weak to moderate, across all seven ImPACT QT reaction time measures (r = -0.08 to -0.46), however three letter time first click was the only measure not found to be significant at p ≤ 0.05. In addition, the three letters counting correct mean score of the ImPACT QT visual motor speed module was found to have a weak positive correlate with the SWAY simple reaction motion (r = 0.22, p < 0.05). SWAY CA impulse control was found to have a significant negative correlation (p ≤ 0.05) of moderate strength with ImPACT QT attention tracker rectangular average time correct (r = -0.46), attention tracker figure eight average time correct (r = -0.36), and attention tracker complex average time correct (r = -0.31), respectively. The remaining ImPACT QT measures, however, were not found to correlate with SWAY CA Impulse Control (p > 0.05), as indicated in Table 4. SWAY CA inspection time was found to have a significant negative correlation of weak strength with ImPACT QT attention tracker figure eight average time correct (r = -0.29), and symbol match incorrect hidden average answer time (r = -0.25). However, SWAY CA inspection time was not found to correlate with the remaining ImPACT QT measures (p > 0.05) (Table 4).

Table 4. Summary of Bivariate Correlations Between SWAY CA and ImPACT QT Measures.

SWAY CA
ImPACT QT Simple Reaction Motion Impulse Control Inspection Time
Visual Motor Speed
Three Letter Count Correct 0.22 0.17 0.06
Reaction Time
Three Letter Time First Click -0.08 -0.01 -0.17
Rectangular Average Time -0.44 -0.46 -0.19
Figure Eight Average Time -0.46 -0.36 -0.29
Complex Average Time -0.32 -0.31 -0.20
Symbol Match Correct Visible -0.27 -0.19 0.04
Symbol Match Correct Hidden -0.35 -0.06 -0.05
Symbol Match Incorrect Hidden -0.32 -0.10 -0.25
Open in a new tab

Bolded values represent p = 0.05

DISCUSSION

This study sought to determine the validity of the SWAY CA application, as well as its reliability across various hardware platforms and operating systems. The results indicated that SWAY CA application appears to be reliable in operating cognitive assessment measures (simple reaction motion, impulse control, inspection time) on various mobile devices (i.e., iPhone 6s Plus, Google Pixel 3, Samsung S9, and iPad Pro 5) and operating systems (e.g., iOS, Android). Such findings are important because this introduces the feasibility of assessing neurocognitive function and reaction time measures regardless of the mobile device available. Although the SWAY CA measures across all mobile devices were found to be in agreement; the iPad Pro 5 and Google Pixel 3 each offered a rather small contribution to the overall analysis. The iPad has been shown to be a valid and compatible mobile device for the SWAY application’s balance segment25,28,30,31 and did not present with any compatibility concerns when in use with the SWAY reaction time segment. The small sample size of the iPad Pro 5 (n = 1) was due to its lack of availability, as it was also being used to administer the ImPACT QT during experimental testing sessions. The Google Pixel 3, however, presented with a login issue that resulted in limited SWAY CA assessments (n = 2) and generally inconclusive findings of its compatibility. Overall, the SWAY CA application introduces the convenience of mobility and mobile device versatility, unlike the ImPACT QT application that requires the adoption of a universal mobile device.6,19,20,31 Furthermore, the lack of significant difference in SWAY CA measures between mobile devices minimizes concern of a difference in an individual’s SWAY CA measures (e.g., comparison of baseline data to data recorded immediately following an insult, and each follow-up assessment) being due to the use of different mobile devices.

The findings of this study further indicated that the SWAY CA segment of the SWAY System is a valid tool for assessing reaction time. Based on correlation values established between the measures using the SWAY system and the ImPACT QT, (0.32 to 0.63, p = 0.05)7,9,17; the Simple Reaction Motion of the SWAY CA application introduced reaction time measures (-0.27 to -0.46, p ≤ 0.05) comparable with reaction time measures of the ImPACT QT reaction time measures, except three letter time click first. The lack of agreement of the SWAY CA simple reaction motion measure with the ImPACT QT three letter time click first measure, as shown in Table 4, may be due to the difference in task(s) administered by each application to assess and calculate the measure as previously described in the methods section. Overall, these findings suggest that the SWAY CA is a comparable mobile neurocognitive and reaction time assessment tool to the FDA approved ImPACT QT.

In addition, several SWAY CA simple reaction motion, impulse control, and inspection time measures reported a negative correlation (-0.25 to -0.46; p ≤ 0.05) in relation to the ImPACT QT reaction time measures. Both SWAY CA and ImPACT QT measure rate of response based on lapse in time (milliseconds) from the moment a stimulus is introduced to the moment a response is recorded.21–23 The negative correlational values introduced in this study indicate that, on average, an individual’s rate of response (milliseconds) following a stimulus was significantly faster (smaller value) with the motion-based system used for SWAY CA in comparison to the slower (greater value) recorded when using the touch-based system for the ImPACT QT. Relatedly, these findings align with previous studies that reported motion-based systems (i.e., SWAY) to be extremely sensitive in recognizing movement as well as minimize mobile device latency down to one to two milliseconds,21,23,26,27 compared to a 50 to 200 millisecond delay when using a touch-based system (i.e., ImPACT QT).1,5,9,21–23 Of additional importance, hardware specifications between the devices used to administer the two applications differ (SWAY, ImPACT QT). The processors for each of the devices ran the respective operating systems at between 1.8 and 2.5 gigahertz.21,26,27,32,34 The screen on the iPhone 6S, Samsung 9s, and the Google Pixel 3, however, which were used to administer all but one of the SWAY application assessments, have a refresh rate of 60hz, compared to double the refresh rate of the iPad Pro 5 screen at 120hz used to administer the ImPACT QT application.21,26,27,32,34 Interestingly, although all SWAY assessments, except the one iPad Pro 5 measure, operated off a device with a slower refresh rate, the SWAY application was shown to recognize and capture a reaction time movement or cognitive response at a faster rate compared to the ImPACT QT based on the negative correlational findings. These findings further support the superior sensitivity of the motion-based SWAY application when seeking to record an individual’s reaction time measures and further assess one’s neurocognitive function and health. This is of particular importance for an individual in sport or other clinical setting where cognitive and reaction time measures may have critical and potentially life-threatening implications.3,25

While measures of agreement between the SWAY CA and ImPACT QT applications were established across several measures; further investigation is needed to determine the fair to low correlation amongst many of the SWAY impulse control and inspection time measures with the ImPACT reaction time and visual motor speed measures, as indicated in Table 4. One consideration for this absence of agreement may be due to distinct differences in measurement design for a particular assessment. Although both applications include assessment of reaction time measures; the SWAY application is a cognitive assessment tool that evaluates an individual’s cognitive and neuromotor measures,9,26 while the ImPACT QT application is known as a post-concussion cognitive test recognized as a neurocognitive and reaction time assessment tool.17,33 Therein, the impulse control and inspection time measures of the SWAY may differ beyond comparison with the ImPACT QT more so due to the measurement approach each uses. An additional consideration may be the notable difference in latency and electrical pulse cycle between the application operating systems.21,26,27,32,34 As shown in Table 4, the faster response rate of the motion-based system of SWAY compared to the slower touch-based system of the ImPACT QT may help explain the lack of associated strength amongst some of the measures and absence of agreement for others. Future test-retest reliability is warranted to further validate; however, the current findings support the use of a motion-based approach and the SWAY application to assess cognitive function and reaction time measures on a mobile device.

Limitations and Future Directions

This study is the first effort to establish concurrent validity of the cognitive assessment modules of the SWAY application as well as its capacity to operate across multiple mobile devices. Overall, the SWAY application was found to deliver reliable and valid cognitive and reaction time measures across all mobile devices; however, the iPad was only used to administer one SWAY assessment and the Google Pixel 3 did present with some concerns. The lack of data recorded from the iPad was due to lack of availability of the device because it was also being used to administer the ImPACT QT. The inclusion of the iPad in future studies is necessary to determine its compatibility with SWAY CA, as well as the potential impact of iPad latency of scores as previously discussed. It is unknown, however, whether the sporadic error message displayed during login and download when using the Google Pixel 3 was a compatibility issue or related to some other unknown. This unknown will require future exploration to determine.

Furthermore, the current findings should be generalized across all mobile device systems (hardware, software) with caution due to known capacity differences across systems21,26,27,32,34 as potentially indicated with the Google Pixel 3. In addition, as mobile device systems, including the devices in this study, frequently introduce updates to the hardware and software, further verification of SWAY compatibility is necessary. In addition, while the findings of this study supported the concurrent validity of the SWAY ’s ability to yield consistent cognitive and reaction time measures comparable to those of the FDA approved ImPACT QT; further test-retest reliability to determine within intrasession reliability and between intersession reliability is necessary.

Conclusion

In conclusion, the results of the current study indicate that the SWAY application is a reliable and valid method for measuring cognitive and reaction time measures across a variety of mobile devices. Furthermore, the faster capture rate technology used by the motion-based SWAY application appears to offer a potentially more reliable assessment of cognitive function and reaction time in comparison to the FDA approved touch-based ImPACT QT measures. Additionally, the SWAY application’s versatility in operating across various mobile device systems may further support its favorability of use in both health and sport.

Conflict of Interest

Authors have no reported conflicts of interest.

References

  1. Woods David L., Wyma John M., Yund E. William, Herron Timothy J., Reed Bruce. Frontiers in Human Neuroscience. 131. Vol. 9. Frontiers Media SA; Factors influencing the latency of simple reaction time. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Sway: A novel approach for simple reaction time measurement. Burghart Mark, Craig Jordan, Radel Jeff, Wu Andy, Huisinga Jessie. Aug 1;2016 American Journal of Occupational Therapy. 70(4 Suppl 1):7011500052p1. doi: 10.5014/ajot.2016.70s1-po5064. doi: 10.5014/ajot.2016.70s1-po5064. [DOI] [Google Scholar]
  3. Sport-related concussion in children and adolescents. Halstead Mark E., Walter Kevin D., Moffatt Kody. Nov 12;2018 Pediatrics. 142(6):e20183074. doi: 10.1542/peds.2018-3074. doi: 10.1542/peds.2018-3074. [DOI] [PubMed] [Google Scholar]
  4. Visual reaction time and its relationship to neuropsychological test performance. Collins L. F., Long C. J. Jan 1;1996 Archives of Clinical Neuropsychology. 11(7):613–623. doi: 10.1093/arclin/11.7.613. doi: 10.1093/arclin/11.7.613. [DOI] [PubMed] [Google Scholar]
  5. Pilot evaluation of a novel clinical test of reaction time in national collegiate athletic association division I football players. Eckner James T., Kutcher Jeffrey S., Richardson James K. Jul 1;2010 J Athl Train. 45(4):327–332. doi: 10.4085/1062-6050-45.4.327. doi: 10.4085/1062-6050-45.4.327. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. How do ImPACT Quick Test scores compare with ImPACT online scores in non-concussed adolescent athletes? Elbin R J, D’Amico Nathan R, McCarthy Matthew, Womble Melissa N, O’Connor Sydne, Schatz Philip. Feb 11;2020 Arch Clin Neuropsychol. 35(3):326–331. doi: 10.1093/arclin/acz072. doi: 10.1093/arclin/acz072. [DOI] [PubMed] [Google Scholar]
  7. Computerized neurocognitive testing in the management of sport-related concussion: an update. Resch Jacob E., McCrea Michael A., Cullum C. Munro. Dec;2013 Neuropsychology Review. 23(4):335–349. doi: 10.1007/s11065-013-9242-5. doi: 10.1007/s11065-013-9242-5. [DOI] [PubMed] [Google Scholar]
  8. Evaluating the recovery curve for clinically assessed reaction time after concussion. del Rossi Gianluca. Aug 1;2017 J Athl Train. 52(8):766–770. doi: 10.4085/1062-6050-52.6.02. doi: 10.4085/1062-6050-52.6.02. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Reliability and validity of a motion-based reaction time assessment using a mobile device. Burghart Mark, Craig Jordan, Radel Jeff, Huisinga Jessie. 2019Applied Neuropsychology: Adult. 26(6):558–563. doi: 10.1080/23279095.2018.1469491. doi: 10.1080/23279095.2018.1469491. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Badau Dana, Baydil Bilgehan, Badau Adela. Sports. 2. Vol. 6. MDPI AG; Differences among three measures of reaction time based on hand laterality in individual sports; p. 45. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Effect of concussion on clinically measured reaction time in 9 NCAA division I collegiate athletes: a preliminary study. Eckner James T., Kutcher Jeffrey S., Richardson James K. Mar;2011 PM. 3(3):212–218. doi: 10.1016/j.pmrj.2010.12.003. doi: 10.1016/j.pmrj.2010.12.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Daponte P., De Vito L., Picariello F., Riccio M. Measurement. 9. Vol. 46. Elsevier BV; State of the art and future developments of measurement applications on smartphones; pp. 3291–3307. [DOI] [Google Scholar]
  13. The use of mobile applications to collect data in sport, health and exercise science: a narrative review. Peart Daniel J., Balsalobre-Fernández Carlos, Shaw Matthew P. Apr;2019 J Strength Cond Res. 33(4):1167–1177. doi: 10.1519/jsc.0000000000002344. doi: 10.1519/jsc.0000000000002344. [DOI] [PubMed] [Google Scholar]
  14. Mobile devices and apps for health care professionals: uses and benefits. Ventola C.L. 2014P T. 39(5):356–364. [PMC free article] [PubMed] [Google Scholar]
  15. Tracking the evolution of smartphone sensing for monitoring human movement. del Rosario Michael B., Redmond Stephen J., Lovell Nigel H. Jul 31;2015 Sensors. 15(8):18901–18933. doi: 10.3390/s150818901. doi: 10.3390/s150818901. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. González-Cañete Francisco Javier, Casilari Eduardo. Sensors. 3. Vol. 20. MDPI AG; Consumption analysis of smartphone based fall detection systems with multiple external wireless sensors; p. 622. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Department of Health and Human Services – Food and Drug Administration ImPACT Quick Test. ImPACT Quick Test – Traditional 510(K), Computerized Cognitive Assessment Aid for Concussion, Section 5 – 510(k) K170551. [2020-12-17]. https://www.accessdata.fda.gov/cdrh_docs/pdf17/K170551.pdf
  18. ImPACT Applications, Inc. ImPACT Applications: Concussion Management & Training website. [2020-9-20]. https://impacttest.com/
  19. Evaluating the prevalence of low factor scores on the ImPACT™ Quick Test in adolescents and adults using multivariate base rates. Wallace J, Covassin T, Schatz P, Iverson G. Jul;2019 Arch Clin Neuropsychol. 34(5):747. doi: 10.1093/arclin/acz026.17. doi: 10.1093/arclin/acz026.17. [DOI] [Google Scholar]
  20. Brief iPad-based assessment of cognitive functioning with ImPACT Quick Test: prevalence of low scores using multivariate base rates. Wallace Jessica, Schatz Philip, Covassin Tracey, Iverson Grant L. Oct 1;2020 Arch Clin Neuropsychol. 35(8):1276–1282. doi: 10.1093/arclin/acaa078. doi: 10.1093/arclin/acaa078. [DOI] [PubMed] [Google Scholar]
  21. Apple Inc. iOS device compatibility reference website. [2020-12-17]. https://developer.apple.com/library/archive/documentation/DeviceInformation/Reference/iOSDeviceCompatibility/DeviceCompatibilityMatrix/DeviceCompatibilityMatrix.html
  22. Deber J., Jota R., Forlines C.., et al. [2020-12-8];How much faster is fast enough? User perception of latency & latency improvements in direct and indirect touch. Understanding & Extending Touch Interfaces. Conference Proceedings of the 33rd Annual CHI Human Factors in Computing Systems. April 2015. Seoul, South Korea. doi: 10.1145/2702123. [DOI]
  23. Plant Richard R., Quinlan Philip T. Cognitive, Affective, & Behavioral Neuroscience. 3. Vol. 13. Springer Science and Business Media LLC; Could millisecond timing errors in commonly used equipment be a cause of replication failure in some neuroscience studies? pp. 598–614. [DOI] [PubMed] [Google Scholar]
  24. Immediate post-concussion assessment and cognitive testing (ImPACT) practices of sports medicine professionals. Covassin Tracey, Elbin Robert J., III, Stiller-Ostrowski Jennifer L., Kontos Anthony P. Nov 1;2009 J Athl Train. 44(6):639–644. doi: 10.4085/1062-6050-44.6.639. doi: 10.4085/1062-6050-44.6.639. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Normative data for the SWAY balance system. Brett B.L., Zuckerman S.L., Terry D.P.., et al. 2020Clin J Sport Med. 30(5):458–464. doi: 10.1097/jsm.0000000000000632. https://doi.org/doi%3A10.1097%2Fjsm.0000000000000632 [DOI] [PubMed] [Google Scholar]
  26. SWAY Medical, LLC SWAY mobile reaction time overview and scoring system of the SWAY reaction time beta website. [2020-11-23]. https://swaymedical.com/resources/videos
  27. Apple Inc. Getting raw accelerometer events website. [2020-12-17]. https://developer.apple.com/documentation/coremotion/getting_raw_accelerometer_events
  28. Test-retest reliability of the SWAY balance mobile application. Amick Ryan Z., Chaparro Alex, Patterson Jeremy A.., et al. Jul 16;2015 Journal of Mobile Technology in Medicine. 4(2):40–47. doi: 10.7309/jmtm.4.2.6. doi: 10.7309/jmtm.4.2.6. [DOI] [Google Scholar]
  29. 2020 PAR-Q+ the physical activity readiness questionnaire for everyone. Warburton D.E., Gledhill N., Jamnik V.., et al. Feb 3;2019 [2020-12-17]; http://eparmedx.com/wp-content/uploads/2013/03/January2020PARQPlusFillable.pdf
  30. Comparison of a mobile technology application with the balance error scoring system. Patterson Jeremy A., Amick Ryan Z., Pandya Priyanka D., Hakansson Nils, Jorgensen Michael J. May;2014 Int J Athl Ther Train. 19(3):4–7. doi: 10.1123/ijatt.2013-0094. doi: 10.1123/ijatt.2013-0094. [DOI] [Google Scholar]
  31. Validation of measures from the smartphone SWAY balance application:a pilot study. Patterson J.A., Amick R.Z., Thummar T.., et al. 2014Int J Sports Phys Ther. 9(2):135–139. [PMC free article] [PubMed] [Google Scholar]
  32. Android Developers Motion sensors website. [2020-12-17]. https://developer.android.com/guide/topics/sensors/sensors_motion
  33. Van Kampen Derk A., Lovell Mark R., Pardini Jamie E., Collins Michael W., Fu Freddie H. The American Journal of Sports Medicine. 10. Vol. 34. SAGE Publications; The “value added” of neurocognitive testing after sports-related concussion; pp. 1630–1635. [DOI] [PubMed] [Google Scholar]
  34. The next battle for smartphone screen supremacy is about speed, not resolution. Horaczek S. May 15;2019 [2020-12-17];Popular Science. https://www.popsci.com/oneplus-7-pro-smartphone-screen-refresh-rate

Articles from International Journal of Sports Physical Therapy are provided here courtesy of North American Sports Medicine Institute

RESOURCES