Mobile Universal Lexicon Evaluation System (MULES) test: A new measure of rapid picture naming for concussion

Abstract

Objective

This study introduces a rapid picture naming test, the Mobile Universal Lexicon Evaluation System (MULES), as a novel, vision-based performance measure for concussion screening. The MULES is a visual-verbal task that includes 54 original photographs of fruits, objects and animals. We piloted MULES in a cohort of volunteers to determine feasibility, ranges of picture naming responses, and the relation of MULES time scores to those of King-Devick (K-D), a rapid number naming test.

Methods

A convenience sample (n = 20, age 34 ± 10) underwent MULES and K-D (spiral bound, iPad versions). Administration order was randomized; MULES tests were audio-recorded to provide objective data on temporal variability and ranges of picture naming responses.

Results

Scores for the best of two trials for all tests were 40–50 s; average times required to name each MULES picture (0.72 ± 0.14 s) was greater than those needed for each K-D number ((spiral: 0.33 ± 0.05 s, iPad: 0.36 ± 0.06 s, 120 numbers), p < 0.0001, paired t-test). MULES scores showed the greatest degree of improvement between trials (9.4 ± 4.8 s, p < 0.0001 for trials 1 vs. 2), compared to K-D (spiral 1.5 ± 3.3 s, iPad 1.8 ± 3.4 s). Shorter MULES times demonstrated moderate and significant correlations with shorter iPad but not spiral K-D times (r = 0.49, p = 0.03).

Conclusion

The MULES test is a rapid picture naming task that may engage more extensive neural systems than more commonly used rapid number naming tasks. Rapid picture naming may require additional processing devoted to color perception, object identification, and categorization. Both tests rely on initiation and sequencing of saccadic eye movements.

1. Introduction

Sports-related concussions are an increasingly recognized cause of visual and neurological impairment, yet they may be undetected or underreported during athletic events [1–4]. Rapid and effective sideline performance measures are to needed to detect concussions and to complement existing removal from play protocols designed to protect injured athletes [5]. Recent studies have shown that a combination of rapid sideline tests that capture vision, balance and cognition offer the best chance of identifying all athletes with concussion among youth and collegiate athletes [6–10].

The addition of a vision-based measure of rapid number naming, the King-Devick (K-D) test, has been shown to improve the sensitivity of concussion screening [6–9]. The K-D test requires saccades and vergence, thus measuring some aspects of frontal, parietal and brainstem eye movement centers [11, 12]. These anatomic areas and inter-connected circuits of the visual system represent approximately 50% of the brain's pathways and are frequently affected by concussion [13]. The K-D test, available in spiral bound and iPad versions, is highly sensitive [10], yet leaves room for a complementary vision-based sideline performance test.

We developed the Mobile Universal Lexicon Evaluation System (MULES), a new test of rapid picture naming, to potentially capture a more extensive vision network, integrating saccades, color perception, and object identification. The purpose of this study was to introduce the MULES test in a cohort of non-concussed adult volunteers to determine the feasibility of administration, the ranges of picture naming responses, and the relation of MULES test time scores to those of the King-Devick (K-D) test.

2. Subjects and methods

2.1. Study participants

A convenience sample of adult participants with no history of ocular or neurologic disease underwent MULES testing as well as K-D testing by the spiral bound and iPad versions. Written informed consent was obtained from each participant; the Institutional Review Board (IRB) at New York University School of Medicine approved all study protocols.

2.2. New test of rapid picture naming: Mobile Universal Lexicon Evaluation System (MULES)

The MULES is a timed rapid picture naming test designed to capture eye movements and other aspects of vision. The test consists of 54 original photographs of fruits, objects and animals on a single 11 × 17 in. laminated card (Fig. 1). Participants are asked to name the pictures out loud from left to right and top to bottom as rapidly as possible without making errors; the score is the time in seconds required to name all pictures. Numbers of errors are also recorded; misspeaks on naming pictures are recorded as errors only if the participant does not immediately correct the mistake before going on to the next picture.

Fig. 1.

The Mobile Universal Lexicon Evaluation System (MULES) test of rapid picture naming, as examined in the present manuscript (MULES Test© New York University. All rights reserved). The test card is 11 × 17 in. and includes 54 original photographs of fruits, objects and animals. The participant names the pictures out loud from left to right as rapidly as possible without making errors; the score is the time in seconds required to name all pictures. (Please see pdf version of Fig. 1 submitted - not embedded due to size.)

2.3. Rapid number naming: King-Devick (K-D) test

The K-D test is a timed rapid number naming test that has been extensively studied in sports-related concussion [6–10, 14]. To perform the K-D test, participants are asked to read numbers on each of three test cards from left to right and top to bottom as quickly as possible, but without making errors. There are two formats: 1) a spiral bound 6 × 8.5 in. book of test cards, and 2) an iPad version with three test screens. The times in seconds required to complete each card or test screen are recorded; the sum of the three card/screen times constitutes the summary measure for the K-D test. Numbers of errors are also recorded; misspeaks on numbers are recorded as errors only if the participant does not immediately correct the mistake before going on to the next number.

2.4. Testing procedures

Participants completed two trials for each of the three tests: MULES, K-D spiral bound format, and K-D iPad platform. Tests were administered by trained study personnel. The order of test administration was randomized to minimize potential effects of learning on differences between test scores; times and numbers of errors were recorded for each test. In addition, MULES tests were audio-recorded to include data on variability and ranges of responses for picture naming.

2.5. Statistical analyses

Data were analyzed using Stata SE 14.1 (StataCorp, College Station, TX). Within-participant differences between tests in terms of time scores and numbers of seconds required to name MULES pictures vs. K-D numbers were analyzed using paired t-tests. The relation of MULES test score to those of the K-D test platforms (spiral and iPad) was determined using Pearson linear correlations. Analyses also examined learning effects as differences between the two test trials using paired t-tests. The relation of age to time scores was also determined.

3. Results

Mean age for study participants (n = 20) was 34 ± 10 years (range 25–59). Average test scores for the MULES and K-D platforms are presented in Table 1. While scores for the best of two trials for all tests were in the range of 40–50 s (Fig. 2A), times required to name each MULES picture (0.72 ± 0.14 s, 54 pictures) were greater than those needed to name each number on the K-D (spiral: 0.33 ± 0.05 s, iPad: 0.36 ± 0.06 s, 120 numbers), p < 0.0001, paired t-test (Fig. 2B).

Table 1.

Scores for the MULES and K-D tests.

Scores in seconds, mean ± SD (range)	MULES test cohort (n = 20)
K-D score, spiral bound, trial 1	41.3 ± 7.1 (30.3–59.5)
K-D score, spiral bound, trial 2	39.8 ± 6.9a (29.0–56.7)
K-D score, spiral bound, best of two trials	39.2 ± 6.4 (29.0–53.8)
K-D score, iPad, trial 1	45.6 ± 7.4 (32.7–59.0)
K-D score, iPad, trial 2	43.8 ± 7.8b (30.3–59.2)
K-D score, iPad, best of two trials	43.3 ± 7.1 (30.1–57.8)
MULES test score, trial 1	48.1 ± 10.4 (34.5–68.0)
MULES test score, trial 2	38.6 ± 7.3c (29.5–53.4)
MULES test score, best of two trials	38.6 ± 7.3 (29.4–53.4)

K-D = King-Devick test; MULES = Mobile Universal Lexicon Evaluation System.

^aComparison of K-D spiral bound trial 1 vs. trial 2, paired t-test, p = 0.07.

^bComparison of K-D tablet (iPad) trial 1 vs. trial 2, paired t-test, p = 0.03.

^cComparison of MULES trial 1 vs. trial 2, paired t-test, p < 0.0001.

Fig. 2.

Box plots demonstrating (A) test times in seconds and (B) average times to name each number or picture for each of the three measures (spiral bound K-D, iPad K-D, MULES). While times were similar to complete each test (average 40–50 s, Panel A), times required to name each picture on the MULES (54 pictures) were greater than those needed to name numbers on the K-D tests (120 numbers on three test cards/screens, Panel B). K-D = King-Devick test; MULES = Mobile Universal Lexicon Evaluation System.

While decreases in naming time were noted for all tests with repeat testing, MULES scores improved by the greatest degree between trials 1 and 2 (9.4 ± 4.8 s, p < 0.0001 for trials 1 vs. 2, paired t-test), as compared to the K-D (spiral 1.5 ± 3.3 s, iPad 1.8 ± 3.4 s between trials). MULES time scores demonstrated moderate and significant [39] linear correlations with the iPad but not the spiral bound K-D scores (r = 0.49, p = 0.03). Shorter MULES test times correlated with shorter K-D test times (iPad-based; Fig. 3). For the spiral bound K-D test, correlations with MULES scores were not significant (r = 0.30, p = 0.20; Fig. 3). There was a high level of association between higher K-D iPad time scores and greater time scores for spiral bound K-D (r = 0.86, p < 0.0001). Within this cohort of 25–59-year-old adult volunteers, age was not a significant predictor of scores for tests of rapid number or picture naming.

Fig. 3.

Scatter plot demonstrating the relation of time scores on the MULES test to those for the K-D spiral bound and iPad versions. Linear correlations were of significant and of greatest magnitude between the MULES and iPad K-D scores, with longer MULES test times predicting higher iPad K-D times in this cohort (r = 0.49, p = 0.03). K-D = King-Devick test; MULES = Mobile Universal Lexicon Evaluation System.

Seven of 20 participants (35%) had at least one error in one of their MULES test trials. All five non-native English speakers made at least one error, making up 71% of the participants who made errors. Native languages represented included Mandarin (n = 2), Spanish (n = 2) and German (n = 1). When examining the MULES trial with the best time score, 17 of 20 participants (85%) had 0 errors while 2 participants had 1 error and 1 participant had 2 errors. The two images associated with the most errors were the limes and wagon (Fig. 1). Analyses of the audio recordings of MULES testing revealed that these errors involved participants misnaming the limes as “lemons” and forgetting the word for “wagon.” Three participants (15%) had 1 error on their best of two trials for the iPad K-D, while 0 participants made errors on the spiral bound K-D.

4. Discussion

A complex rapid picture naming task, the MULES test may depend on a wider neuroanatomical vision network as compared to rapid number naming tasks, as it requires intact processing of saccades, color perception, object identification, and object categorization. Previous studies have highlighted the value of including a vision test in a concussion screening protocol [6, 9]. While balance and cognitive tests are effective diagnostic tools, they do not detect concussions with 100% sensitivity [10, 15, 16]. Vision tests increase screening sensitivity because vision pathways are widely distributed within the brain and are susceptible to even mild trauma [10, 17]. The King-Devick test (K-D) in particular has already been shown to improve concussion screening sensitivity [6–10, 18, 19], and the MULES may enhance it.

A key feature of the K-D and MULES tests is that they both require the use of saccades, which may be particularly impaired in concussion due to the distributed neural systems involved in this type of eye movement [10]. A number of cortical regions contribute to saccadic eye control, including the frontal eye fields, dorsolateral prefrontal cortex (DLPFC), supplementary motor area, posterior parietal cortex, middle temporal area, and striate cortex [10–13]. By testing saccades, the MULES and K-D assess a wide range of cortical structures and their connections.

Both rapid naming tasks require use of the saccade system in order to perform optimally [20]. However, as a rapid naming task that employs complex visual stimuli, the MULES test may recruit a significantly wider neuroanatomical network as compared to rapid number naming tasks, as the MULES stimuli require additional processing to identify visually-presented objects. In a functional imaging study directly contrasting rapid automatic naming of objects versus numbers, object naming resulted in activation of bilateral fusiform gyri [20]. In contrast, rapid number naming appears to engage other areas that may be specific to numeric representation, including left angular gyrus, and bilateral superior temporal gyri [20–22]. While the MULES and the K-D may rely on different neuroanatomical pathways, they may also complement each other synergistically to improve concussion screening.

One finding of particular interest in our study is that each MULES picture took participants twice as much time to identify as each K-D number on average. This may reflect the fact that the MULES stimuli, which consisted of color photographs, may require a broader range of visual and cognitive processing than the digits employed in the K-D. While rapid naming tasks have yet to be definitively characterized in terms of necessary neural networks, additional stimulus-related processing in the domains of attention, learning, memory, visual perception, conceptual representation, phonology, and articulation are likely. It has been demonstrated that for concepts with greater detail [38], learning and recalling may be easier for the study participant. The added semantic complexity of the MULES provides added utility for this test as a measure that samples a broader network than rapid number naming. As such, it is easier to remember stimuli that are more perceptually and semantically rich (such as pictures) than those that are sparse in these characteristics.

With regard to the K-D in our investigation, it was noted that the linear correlations between the MULES and iPad K-D scores were in a modest and significant range (r = 0.49; p = 0.03), while the associations with spiral-bound K-D scores were not significant (Fig. 3) [39]. Why the iPad version of the K-D would yield scores that reflect the MULES time scores in this cohort is of interest for future and larger studies; the role of chance must always be considered along with technical and cognitive aspects of the tests.

The MULES task here, by virtue of drawing on multiple additional domains, likely engages neural systems, ranging from those involved in processing fundamental visual properties of object stimuli, such as color recognition. This requires multiple areas of visual cortex, including V2 and V4, as well inferior temporal projections to the ventral visual stream [23, 24]. Higher-level representation of objects has been demonstrated to involve activity in lateral occipital cortex, inferior and medial temporal cortex, parietal cortices, and the lateral prefrontal cortex [23, 25].

To characterize objects presented in the MULES task, additional activity in dorsolateral prefrontal cortex may be engaged [26]. In the MULES task, which contains 54 images divided into three groups: fruits, animals, and random objects, image categorization may be another mechanism underlying the increase in naming speed from trial 1 to trial 2. The increase in testing speed between the MULES trials 1 and 2 may also reflect differences in learning, attentional, and memory requirements to discern objects vs. the detection of numbers. The perirhinal cortex, which is part of the medial temporal cortex, encodes object recognition memory [27–29]. Animal studies have shown that a damaged perirhinal cortex results in impaired performance on object recognition memory tasks [27, 30, 31]. Intact short-term memory may be crucial for participants to acquire familiarity with MULES stimuli presented in the second trial, and thus to improve their time scores significantly. Learning that occurs during the first MULES trial may also enhance attention toward salient features of the presented objects and inhibit attention toward irrelevant background in order to more efficiently identify objects in the second trial [32]. Fronto-parietal attentional networks may be needed for this aspect of the MULES task [33].

Finally, there are cultural and linguistic issues that must be considered. The MULES learning effect may also be associated with certain “error-prone” images in this pilot version of the test. A few participants misnamed the image of limes and were not able to recall the word for “wagon.” Participants spent extra time during the first trial trying to recall names for these particular images. Replacing these two images may reduce this learning effect, and this has been accomplished in a revised version of the test to be investigated as a pre-season baseline and post-concussion test in youth, collegiate and professional athletes. Also, non-native English speakers were more prone to making errors in the MULES, especially in their first trial. It was important to include non-native English speakers in the pilot study of the MULES to make the results more generalizable to people with concussion across diverse populations. Continued investigation of the MULES across languages and in participants' native tongues will enable broad adaptation of the test; translators and audio recording may be helpful for this phase of testing. Such studies among broad populations of athletes and non-athletes with concussion, as well as patients of diverse backgrounds with other neurological disorders will provide even more perspective and data on the performance of this new and rapid test. In addition, even for native English speakers, the range of acceptable names for pictures is necessarily broader than that required for rapid number naming such as in the K-D test. For example, we did accept “airplane” or “plane” as responses for the seaplane pictured in the MULES. In this pilot study, responses were recorded electronically for this purpose, and will continue to be analyzed as participants of varying ages and linguistic backgrounds engage in the MULES test.

This is a pilot study to determine the feasibility for administration and to define aspects of the MULES test that require revision and further investigation. Important next steps to examine the utility of this new measure will include: 1) revision to replace the limes (with cherries) and wagon (with bridge), two problematic images particularly for our non-native English speaking pilot participants; 2) examine the MULES as a pre-season baseline test (along with K-D, balance and cognition tests) among youth, collegiate and professional athletes 3) determine the relative capacity for MULES to identify concussed vs. non-concussed control athletes who have been practicing or competing 4) further evaluate learning effects of the MULES between administrations that are commonplace for any performance measure; 5) administration and piloting of the MULES in patients with other neurological disorders, such as Alzheimer's and Parkinson's disease—this has been accomplished for the K-D test of rapid number naming, for example [34–37]; 6) targeted testing to individuals with congenital color blindness—we anticipate that object shapes and forms will, as in these participants' daily visual world, enable them to identify picture names. Creating native language- and age-based norms for this test will also be helpful, as will investigations of patients to determine the relation of MULES scores over time to symptom recovery.

In conclusion, the MULES test, which requires the detection of object, may complement the K-D test, which involves the rapid detection of a sequence of numbers. Participants' raw time scores and degrees of learning effect between trials may help monitor concussion recovery. In addition, the MULES test may be a powerful tool to assess a broader neural network that is dedicated to the processing of color as well as object recognition and categorization. In addition, the MULES test may be a useful diagnostic tool in other neurologic conditions such as multiple sclerosis (MS), Alzheimer's and mild cognitive impairment, and Parkinson's disease, which are also associated with slower K-D time scores [34–38]. Future investigations, in addition to identifying the neural systems underlying the MULES, will also investigate its utility as a sideline performance measure for sports-related concussion among athletes from youth to professional levels.