The new Mobile Universal Lexicon Evaluation System (MULES): A test of rapid picture naming for concussion sized for the sidelines

Abstract

Objective

Measures of rapid automatized naming (RAN) have been used for over 50 years to capture vision-based aspects of cognition. The Mobile Universal Lexicon Evaluation System (MULES) is a test of rapid picture naming under investigation for detection of concussion and other neurological disorders. MULES was designed as a series of 54 grouped color photographs (fruits, random objects, animals) that integrates saccades, color perception and contextual object identification. Recent changes to the MULES test have been made to improve ease of use on the athletic sidelines. Originally an 11 × 17-inch single-sided paper, the test has been reduced to a laminated 8.5 × 11-inch double-sided version. We identified performance changes associated with transition to the new, MULES, now sized for the sidelines, and examined MULES on the sideline for sports-related concussion.

Methods

We administered the new laminated MULES to a group of adult office volunteers as well as youth and collegiate athletes during pre-season baseline testing. Athletes with concussion underwent sideline testing after injury. Time scores for the new laminated MULES were compared to those for the larger version (big MULES). Results: Among 501 athletes and office volunteers (age 16 ± 7 years, range 6–59, 29% female), average test times at baseline were 44.4 ± 14.4 s for the new laminated MULES (n = 196) and 46.5 ± 16.3 s for big MULES (n = 248). Both versions were completed by 57 participants, with excellent agreement (p < 0.001, linear regression, accounting for age). Age was a predictor of test times for both MULES versions, with longer times noted for younger participants (p < 0.001). Among 6 athletes with concussion thus far during the fall sports season (median age 15 years, range 11–21) all showed worsening of MULES scores from pre-season baseline (median 4.0 s, range 2.1–16.4).

Conclusion

The MULES test has been converted to an 11 × 8.5-inch laminated version, with excellent agreement between versions across age groups. Feasibly administered at pre-season and in an office setting, the MULES test shows preliminary evidence of capacity to identify athletes with sports-related concussion.

1. Introduction

There are an estimated 1.6 to 3.8 million sports and activity-related concussions every year across all age groups [1]. Sports-related concussion has received increased attention because of neurological dysfunction that may impair cognition, vision and balance [2,3]. Despite the improved recognition of concussion and the potential neurological consequences, many concussions continue to go undiagnosed or unreported [4,5]. For this reason, rapid and effective sideline performance testing is necessary to detect concussion in order to adequately protect the injured athlete [6]. Rapid sideline tests, including those that assess vision, balance, and cognition, may help confirm the presence of a concussion and prevent the risk of recurrent injury [7–12].

The King-Devick (K-D) test, a sensitive sideline concussion-screening test, uses rapid number naming to capture vergence, saccades and other eye movements as well as attention and language function [7–10]. By testing these functions, widely-distributed and interconnected areas of the cortex and brainstem may be evaluated [13–15]. The Mobile Universal Lexicon Evaluation System (MULES) is a new sideline test of rapid naming of photographic images in context, aims to examine the brain’s visual pathways and neural networks by testing color perception, object identification, conceptual representation, phonology and articulation [16,17].

Measures of rapid automatized naming (RAN), including rapid picture naming, have been used for over 50 years to capture vision-based aspects of cognition and language [18–26]. Such tests that included rapid naming of color photos were described in the 1940’s by Reusch and Wells [19] in a guide entitled Mental Examiner’s Handbook. In the late 1960’s, Geschwind used a version of a picture naming test that included 50 color photographs to be identified by the subject as quickly as possible [20,21]. In 1974, the term rapid automatized naming (RAN) was coined in the context of a study of rapid color, letter and number naming by normal children [22]. Since that time, the literature on RAN has demonstrated that individuals both with and without reading deficits (dyslexia) require more time to name objects than symbols [20–26]. In our group’s initial publication on the MULES [16], it was noted that healthy participants named color photographs in an average of 0.72 s, and named numbers on the King-Devick Test in an average of 0.36 s. Given these important differences in the brains of healthy persons and historically in published reports of those with dyslexia [37–43], the study of picture naming among those with concussion and other neurological disorders will be critical to understanding such injuries.

Since the initial publication on the MULES [16], changes have been made to the design to improve the ease of use of this investigational test on the sidelines. Originally an 11″ × 17″ single-sided paper, the MULES test has been reduced in size to a laminated 8.5″ × 11″ double-sided version. The purpose of this study was to identify performance changes associated with the transition from the larger MULES to the newer, MULES that is better sized for the sidelines according to athletic trainer and parent feedback [16]. Total time scores to complete the test, average time to name each MULES stimulus image, and associations of scores with age and other characteristics of the MULES test were analyzed in a group of athletes undergoing pre-season testing and in healthy volunteers at a single baseline testing session.

2. Subjects and methods

2.1. Study participants

A convenience sample of adult office volunteers as well as athletes from regional collegiate and youth athletic organizations underwent pre-season baseline (pre-injury) testing for concussion, which included the MULES. Athletes with concussion underwent sideline testing after injury. Time scores for the new laminated MULES were compared to those for the larger version (big MULES). Participants with a history of ocular or neurologic disease were excluded from the study. Written informed consent was obtained from each participant; the Institutional Review Board (IRB) at New York University School of Medicine approved all study protocols.

2.2. Two formats of the MULES and testing procedures

Each format of the test consists of 54 original photographs of fruits, objects and animals. The original, “big” MULES test is printed on a single side of a 11 × 17-in. sheet of paper. The modified “laminated” MULES includes the same 54 photographs distributed on two sides of a laminated 8.5 × 11-inch paper (Fig. 1). Participants are asked to name the pictures orally from left to right and top to bottom as rapidly as possible without making errors. The score is the time in seconds required to name all pictures on both sides of the laminated sheet of paper (8.5 × 11-inch paper is flipped by the participant during testing with the timer running). In this study, participants completed two trials of the MULES and the trial with the better score (shorter time) was recorded as the subject’s baseline. This practice of two trials is standard for performance measures used in concussion and in other neurological disorders, such as multiple sclerosis (MS Functional Composite) [27,28]. Testing was administered by trained study personnel at all sites for pre-season baseline; athletic trainers or designated parent testers performed evaluations for concussed athletes on the sidelines. Concussed athletes completed a single trial of the MULES, while pre-season testing involved two trials, recording the best score as the baseline.

Fig. 1.

The Mobile Universal Lexicon Evaluation System (MULES) test of rapid picture naming, as examined in the present manuscript (MULES Test © New York University, text and photographs, registration number TXu002026665, all rights reserved). The laminated MULES is printed two-sided on an 8.5 × 11-inch sheet of paper and includes 54 original photographs of fruits, objects and animals. The participant names the pictures orally from left to right as rapidly as possible. The score is the time in seconds required to name all pictures (participant flips the laminated sheet of paper during test timing).

2.3. Statistical analyses

Data were analyzed using Stata SE 15.1 (StataCorp, College Station, TX). Within-participant differences between MULES test trials (learning effects) and MULES test versions in terms of time scores were analyzed using paired t-tests. The capacity for MULES test time scores and for inter-trial differences (degrees of learning effect) to predict MULES test version (big vs. laminated MULES) was examined using logistic regression models, accounting simultaneously for age. The relation of big MULES vs. laminated MULES test version scores was determined using Pearson linear correlations. Linear regression models were used to examine the same associations, accounting simultaneously for participant age. Similar analyses were used to examine the relation of age to MULES test time scores for both versions. To further assess agreement between the two versions of the MULES test, the intra-class correlation coefficient (ICC) was calculated. The ICC indicates the proportion of the variability in a dataset that is attributable to between-participant differences. For analyses of the 6 athletes who developed concussion so far during the fall sports season, non-parametric tests (Wilcoxon signed-rank) were used to compare pre-season baseline vs. post-injury laminated MULES scores within participants.

3. Results

Among 501 total participants (athletes and office volunteers, age 16 ± 7 years, range 6–59, 29% female), average test times at baseline (best of two trials) were 44.4 ± 14.4 s for the new laminated MULES (n = 196) and 46.5 ± 16.3 s for big MULES (n = 248; Table 1). Accounting for participant age, MULES version (big vs. laminated) was not a significant predictor of baseline score (p = 0.52, logistic regression; Fig. 2).

Table 1.

Scores for the big and laminated MULES at baseline testing session.

MULES version Completed at baseline session			Scores in seconds, mean ± SD (range)
All participants	Group	Totals, gender	Age, mean ± SD
n = 501	Office volunteers	N = 32, 81% female	31 ± 10 years
	Youth athletes	N = 246, 11% female	11 ± 3 years
	Collegiate athletes	N = 223, 43% female	19 ± 1 years
Participants completing laminated MULES		Trial 1	53.4 ± 18.0
n = 196		Trial 2	44.6 ± 14.6a
		Best of 2 Trials = Baseline	44.4 ± 14.4
		Inter-trial difference	8.9 ± 9.0
		Age	17 ± 7 years
		Gender	35% female
Participants completing big MULES		Trial 1	54.7 ± 19.0
n = 248		Trial 2	47.5 ± 18.6a
		Best of 2 Trials = Baseline	46.5 ± 16.3
		Inter-trial difference	7.2 ± 10.3
		Age	15 ± 5 years
		Gender	27% female
Participants completing both MULES versions	Laminated MULES	Trial 1	54.8 ± 17.1
n = 57		Trial 2	49.6 ± 17.9a
		Best of 2 Trials = Baseline	48.9 ± 16.4
		Inter-trial difference	5.2 ± 7.2
	Big MULES	Trial 1	63.0 ± 21.5
		Trial 2	57.5 ± 22.6b
		Best of 2 Trials = Baseline	55.5 ± 18.7
		Inter-trial difference	5.6 ± 12.6
		Age	15 ± 10 years
		Gender	18% female

MULES = Mobile Universal Lexicon Evaluation System.

^aComparison of trial 1 vs. trial 2, paired t-test, p < 0.0001.

^bComparison of trial 1 vs. trial 2, paired t-test, p = 0.002.

Fig. 2.

Box plots demonstrating average baseline scores in seconds for each MULES version (big vs. laminated). The average big MULES baseline score (n = 248) was 46.5 ± 16.3 s and the average baseline score for the laminated MULES was (n = 196) 44.4 ± 14.4 s. Accounting for participant age, MULES version (big vs. laminated) was not a significant predictor of baseline score (p = 0.52, logistic regression).

There were significant reductions in time score between trials 1 and 2 for both big and laminated MULES tests, consistent with learning effects that are inherent in performance measures (p < 0.0001 for each MULES version, paired t-test; Fig. 3). MULES version (big vs. laminated), however, was not a significant predictor of inter-trial difference, or degree of learning effect between the two trials (p = 0.31, logistic regression).

Fig. 3.

Box plots demonstrating average scores for trials 1 and 2 for each MULES version (big vs. laminated). Median inter-trial difference for the big MULES (n = 248) was 6.6 s, while the median inter-trial difference for laminated MULES (n = 196) was 6.1 s. Accounting for participant age, MULES version (big vs. laminated) was not a significant predictor of inter-trial difference, or degree of learning effect between the two trials (p = 0.31, logistic regression).

Both versions of the MULES were completed by 57 participants, with excellent agreement of best baseline scores between the big and laminated MULES (p < 0.001, linear regression, accounting for age; Fig. 4). These testing sessions were approximately one year apart due to timing of development of the laminated MULES following testing of the big MULES. In terms of the magnitude of association between scores for the two MULES test versions among the same group of participants, the linear correlation was r = 0.89, p < 0.0001). The intra-class correlation coefficient (ICC) between the two MULES versions within this cohort (n = 57) was 0.90, 95% CI [0.66, 0.96], indicating excellent agreement between the big and laminated MULES scores. Average times required to name each MULES picture were similar for both the big MULES (0.89 ± 0.32 s) and laminated MULES (0.84 ± 0.28 s). Accounting for participant age among those who completed both versions of the test (n = 57), the time required to name big MULES pictures was a significant predictor of the same measure for the laminated MULES (p < 0.001, linear regression).

Fig. 4.

Scatter plot of big vs. laminated MULES baseline scores for participants who completed both test versions (n = 57). Linear correlation demonstrates strong agreement between baseline scores of each version (r = 0.89, p < 0.0001). Accounting for participant age, big MULES scores were significant predictors of time scores for the new laminated version (p < 0.001, linear regression).

Age was a predictor of test times for both MULES versions, with longer times noted for younger participants (p < 0.001, linear regression). Among 6 athletes with concussion thus far during the fall sports season (age 16 ± 4 years) all showed significant worsening of laminated MULES scores from pre-season baseline to post-injury testing (median 4.0 s, range 2.1–16.4, p = 0.003, Wilcoxon signed-rank test; Table 2).

Table 2.

Laminated MULES test time scores for pre-season baseline vs. post-injury for athlete participants with concussion thus far in the present study.

Participant, Age (years), Gender (M/F)	Sport	Baseline (s)	Post-injury (s)	Difference (post-injury minus baseline time, s)
13 M	Ice Hockey	43.0	45.2	2.1
13 M	Ice Hockey	37.9	52.3	14.4
11 M	Football	45.4	47.8	2.4
21 M	Ice Hockey	36.2	39.7	3.5
19 F	Soccer	47.7	64.0	16.4
19 M	Wrestling	49.3	53.8	4.5
Median (range)		44.2 (36.2–49.3)	50.1 (39.7–64.0)	4.0 (2.1–16.4)a

^aComparison of baseline vs. post-injury test times, Wilcoxon signed-rank test, p = 0.003.

4. Discussion

An investigational vision-based timed performance measure, the MULES test of picture naming has been successfully converted to a more compact laminated version. Our data show excellent agreement between the new laminated MULES and the big MULES in both youth and collegiate athletes and among healthy adult volunteers. Similar to previous studies of the big MULES, time scores for laminated MULES decrease with age among youth athletes, supporting the use of pre-season baseline measurements. Feasibly administered at pre-season and in an office setting, the laminated MULES test shows preliminary evidence of capacity to identify athletes with sports-related concussion.

Sensitive performance measures and other markers for concussion are needed to support what remains primarily a clinical diagnosis [17]. Such measures factor into important sideline decisions in athletes with concussion. The MULES is a rapid picture naming task designed to employ widely distributed afferent and efferent visual networks in the brain. Picture naming also involves other cortical regions responsible for object categorization and language [18–35]. A method to capture vision-based aspects of cognition, rapid automatized naming (RAN) tests involve serial naming aloud of line drawings, pictures or, as in the case of the MULES, color photographic images in context [22].

The MULES test has been recently reformatted, from the “big” single-sided 11 × 17-inch paper format to the “laminated” double sided 8.5 × 11-inch page (Fig. 1). Based on feedback from athletic trainers, clinicians and parents involved in the ongoing research studies, these features make the test more accessible in a sideline and clinical setting. For example, the smaller version fits in a briefcase or athletic bag.

In comparing the two versions of the MULES test, it should be kept in mind that our cohorts that completed the big vs. laminated MULES (15 vs. 17 years; Table 1) differ slightly with respect to mean age. This difference in age is attributable to the timing of our implementation of the newer laminated MULES version in the research studies of youth and collegiate athletes. The new laminated version was introduced just as our collegiate athletics baseline testing program began to expand; thus, the laminated MULES cohort (n = 196, Table 1) includes greater numbers of collegiate athletes and also reflects greater numbers of female participants. As such, our analyses comparing MULES versions with regard to baseline time score and learning effects accounted simultaneously for participant age, with no significant differences found.

As may be expected developmentally for a vision-based performance measure, younger age among youth athlete participants was associated with greater baseline time scores for both versions of the MULES. These findings may be attributable to the ongoing development of frontal lobe executive functions and the temporal-parietal regions responsible for semantic and visual categorization [36–39]. The pre-frontal cortex, which serves many executive functions including temporal integration, preparatory setup for saccades, working memory, and inhibitory control, is among the last structures to develop. Within these structures, white matter volume increases into adulthood, and cortico-cortical tracts reach a full state of myelination in the third decade of life [40–42]. Thus, it may be expected that performance on a rapid-picture naming task, which presumably integrates these functions, will be age-dependent in such a way as to reflect developmental trajectory. This peak in baseline scores in late adolescence interestingly corresponds to performance peaks in processing speed, response suppression, and working memory that were documented in one study of adolescent cognitive development [43].

Correlations of the big vs. laminated MULES scores, along with the intra-class correlation (ICC) and linear regression models, demonstrate excellent agreement between the two test versions in our study cohort. Specifically, the ICC value of 0.90 suggests that a majority of the variation seen among baseline MULES scores is due to differences between participants rather than differences between MULES test versions or test administrators. While the between-participant variability is consistent with the MULES tests capturing individual aspects of visual performance, the reassuring linear correlations and ICC indicate that MULES test versions closely parallel each other in their capacity to measure aspects of vision and cognition. While the observed levels of agreement may have been anticipated based on the similarities of format, spacing and picture content, it is reassuring that data generated from the big MULES testing can be applied to groups now receiving laminated MULES testing.

The results of our study to date are promising, showing consistent worsening of laminated MULES scores from pre-season baseline among a small group of athletes who have suffered a concussion. Most of the laminated MULES scores worsened by only 2–4 s (Table 2), while two others had a 14–16-second worsening from pre-season baseline. Since the MULES is a vision-based performance measure, learning effects are typically noted between testing sessions. For this reason, any worsening of the time score from pre-season baseline is currently considered consistent with injury. Since vision comprises approximately 50% of the brain pathways, and prior studies have demonstrated a need for composite measures to identify all athletes with concussion, addition of a classic rapid picture naming task to the test of rapid number naming may augment the capacity for sideline vision-based tests to diagnose concussion.

It will be interesting in future studies to examine the relation of MULES time scores to other aspects of neurological function, including cognition, balance and symptom burden. Task-based functional MRI protocols implementing the MULES may identify patterns of network activation during baseline assessments as well as those that become dysfunctional following concussion. With imaging studies employing rapid automatic naming tasks, we may gain insight into the pathophysiology of concussions especially its impact on the visual system. Since the MULES test integrates widely distributed cortical networks, applicability of the MULES as an assessment tool will likely extend to a variety of neurological and neurodegenerative disorders beyond concussion.