Antonyms: A Performance Metric to Assess Word-Finding Difficulty in Multiple Sclerosis

Abstract

Word-finding difficulty (tip-of-the-tongue phenomenon) is among the most prevalent changes reported by persons with multiple sclerosis (MS), but we lack validated performance metrics to assess this common symptom. Antonyms is a novel task evaluating retrieval of target phonological codes via a brief, clinically-feasible paradigm that controls for other cognitive processes. Test-retest reliability was good (0.85). Among 660 patients, Antonyms demonstrated better criterion validity than traditional language tasks (e.g., animal generation, Boston Naming Test). Demonstrating discriminate validity, Antonyms was specifically related to patient-reported difficulties with expressive language rather than working memory, executive/speed, or episodic memory. The Antonyms task is provided herein.

INTRODUCTION

Word-finding difficulty is among the most prevalent and earliest cognitive changes reported by persons with multiple sclerosis (MS),^1-3 but there is little research on word-finding in MS, in part because we lack validated metrics sensitive to MS-related word-finding difficulty. Traditional neuropsychological language tasks (e.g., Boston Naming Test [BNT], rapid word generation) were developed for aphasia and appear insensitive to subtle but impactful MS-related changes.²

Successful word finding requires rapid retrieval of the semantic content related to a word and the phonological codes representing that target word.^4-5 Persons with MS describe a tip-of-the-tongue phenomenon during which they know what they want to express (semantic meaning), but they cannot retrieve the word (phonological code). Implication of phonological code retrieval aligns with our observation that phonemic processing is weaker in MS and linked to worse patient-reported word-finding difficulty.⁶

We developed a novel paradigm requiring rapid retrieval of specific target words cued by over-learned associations (i.e., common antonyms), which reduces semantic processing to better capture individual differences in phonological code retrieval. A control task further isolates code retrieval from other input and output demands, including articulation. Herein we describe task development and provide data on reliability and validity of this approach.

METHODS

Development of the Antonyms Task

Several words have over-learned opposites (e.g., in–out, up–down); presenting part of the antonym pair (cue word) strongly activates the other.⁷ For example, if asked for the first word that comes to mind when presented with “hot,” “good,” and “true,” the most common responses are “cold,” “bad,” and “false.” We developed a paradigm during which persons are presented with cue words of well-known antonym pairs (e.g., “young – ——”) and instructed to quickly read each cue aloud (“young”) and state its opposite aloud (“old”). The pilot task consisted of 50 items (two columns of 25) to be completed in rapid succession. As a control procedure, persons were subsequently presented with the 50 cues paired with answers (e.g., “young – old”) to be read aloud in rapid succession. The Antonyms score is derived by subtracting control time from antonym retrieval time to isolate phonological code retrieval (word-finding) latency from reading speed. Time to completion was the dependent variable because high accuracy was expected by design. To optimize this approach, the 50-item pilot task was completed by 170 native English speakers with MS (Table S1), from which 10 items with accuracy <90% or multiple meanings (e.g., “soft”) were removed to derive the final 40-item Antonyms task (see Appendix). Test-retest reliability (ICC[95%CI]; two-way mixed, absolute agreement) of the final 40-item Antonyms task was good (0.85[0.79, 0.90]) among 102 patients with MS (Figure S1), which exceeded test-retest reliabilities of traditional verbal fluency tasks assessed concurrently in the same sample (Category Fluency 0.64[0.51, 0.74]; Letter Fluency 0.61[0.48, 0.72]).

Evaluation of Task Validity

Samples

Criterion validity and discriminant validity of Antonyms were assessed in samples of native English speakers with MS aged 18 to 59 years without another neurologic condition or serious mental illness (e.g., schizophrenia) who completed (a) the year-six follow-up of the Reserve against Disability in Early MS (RADIEMS) cohort study (research sample; IRB-approved, participants provided written informed consent) or (b) standard-of-care cognitive screenings at our MS center between November 2023 and June 2026 (clinical sample; Antonyms added to core clinical battery in November 2023; data captured via IRB-approved chart review).

Criterion Validity

Procedures:

All patients reported frequency of “having a word ‘on the tip of your tongue’ but with difficulty getting it out” as never, rarely, sometimes, fairly often, or very often; to derive more robust sample sizes representing each level, ratings were collapsed into “never/rarely,” “sometimes,” and “often.” Both samples completed Antonyms and the traditional verbal fluency task requiring rapid generation of animal names (Category Fluency). The research sample also generated as many words as possible starting with the letter “f” (Letter Fluency); the clinical sample was asked to retrieve names of visually-presented drawings of low frequency objects (BNT). For comparison, both samples completed the Symbol Digits Modalities Test (SDMT, oral) as a high-sensitivity, non-specific screener of overall MS-related cognitive changes.⁸

Statistical Analyses:

Metrics were evaluated for outliers (±3.0*IQR), skewness (≥±0.8), and kurtosis (≥±1.0), and were transformed appropriately. For the research sample, MANCOVAs investigated performance differences on Antonyms, Category Fluency, Letter Fluency, and SDMT across levels of patient-reported word-finding difficulty, first adjusting for age, sex, and education, then additionally adjusting for premorbid verbal ability (Wechsler Test of Adult Reading, WTAR), and then also adjusting for depression symptoms (Beck Depression Inventory, Fast Screen; performed because depression may mediate associations between subjective and objective cognitive functions, especially on timed tasks.⁹) For the clinical sample, the same MANCOVAs investigated differences in Antonyms, Category Fluency, BNT, and SDMT across levels of patient-reported word-finding (depression assessed with Hospital Anxiety and Depression Scale, Depression Subscale). Finally, combining samples for greater statistical power, the same MANCOVAs investigated differences in Antonyms, Category Fluency, and SDMT across levels of patient-reported word-finding difficulty (models additionally adjusted for sample; depression normalized 0 to 1 within each sample to account for different scales).

Discriminant Validity

In both samples, the Multiple Sclerosis Cognitive Scale (MSCS)¹⁰ assessed patient-reported cognitive difficulty across subscales of Expressive Language, Working Memory, Episodic Memory, and Executive/Speed. Pearson partial correlations investigated associations between Antonyms and MSCS subscales, first adjusting for age, sex, education, and sample, then additionally adjusting for premorbid verbal ability, and then also adjusting for depression symptoms. Steiger’s Z-tests (two-tailed) examined if Antonyms was significantly more related to patient-reported difficulty in Expressive Language than to other subscales.

RESULTS

Samples.

See Table 1 for characteristics of the Research (n=152) and Clinical (n=508) samples.

Table 1. Sample Characteristics.

Demographic and clinical characteristics are shown for the research sample, clinical sample, and the combined sample. Premorbid verbal ability was estimated with the Wechsler Test of Adult Reading (WTAR) proxy of WAIS Verbal Comprehension Index (VCI).

Characteristic	Research Sample	Clinical Sample	Combined Sample
Age, years, mean (sd)	41.2 (7.7)	42.3 (10.1)	42.0 (9.6)
Sex, N (%)
Women	104 (68.4)	383 (75.4)	487 (73.8)
Men	48 (31.6)	125 (24.6)	173 (26.2)
Race and Ethnicity, N (%)
Asian	6 (3.9)	17 (3.3)	23 (3.5)
Black, non-Latino/a	25 (16.4)	96 (18.9)	121 (18.3)
Latino/a	38 (25.0)	75 (14.8)	113 (17.1)
White, non-Latino/a	83 (54.6)	320 (63.0)	403 (61.1)
Progressive Disease Course, N (%)
No	152 (100)	465 (91.5)	617 (93.5)
Yes	0 (0)	43 (8.5)	43 (6.5)
Years since diagnosis, median (IQR)	8.6 (7.3, 9.9)	6.0 [2.0, 13.0]	7.7 [3.0, 11.6]
Bachelor’s degree, N (%)	125 (82.2)	398 (78.3)	523 (79.2)
Premorbid Verbal Ability, mean SS (sd)	108.6 (8.6)	107.0 (8.4)	107.3 (8.4)

Criterion Validity

Within the research sample, word-finding difficulty was reported never/rarely (n=71, 46.7%), sometimes (n=43, 28.3%), or often (n=38, 25.0%); word-finding difficulty was more related to poorer performance on Antonyms than on other tasks, especially when adjusting for additional covariates (Table 2). Within the clinical sample, word-finding difficulty was reported as never/rarely (n=155, 30.5%), sometimes (n=172, 33.9%), or often (n=181, 35.6%); word-finding difficulty was most related to Antonyms (Table 2), which was the only task significantly related to patient-reported word-finding difficulty when adjusting for additional covariates. Within the combined sample (n=660), word-finding difficulty was reported as never/rarely (n=226, 34.2%), sometimes (n=215, 32.6%), or often (n=219, 33.2%). Antonyms was the only task that differed across patient-reported word-finding difficulty in the fully-adjusted model (Table 2).

Table 2. Task performance across levels of patient-reported word-finding difficulty.

ANCOVAs investigated performance differences on each task (labeled on the left) and levels of patient-reported word-finding difficulty (never/rare, sometimes, often) adjusting for age, sex, education, and sample (Model A), additionally adjusting for premorbid verbal ability (Model B), and additionally adjusting for depression symptoms (Model C). Analyses were performed for separately for the research, clinical, and combined samples. Antonyms was the only task related to patient-reported word-finding across all models.

		Research Sample					Clinic Sample					Combined Sample
				Raw Score Means [95%CI]					Raw Score Means [95%CI]					Raw Score Means [95%CI]
		P	ηp2	Never/Rare	Sometimes	Often	P	ηp2	Never/Rare	Sometimes	Often	P	ηp2	Never/Rare	Sometimes	Often
Model
Antonyms	A	<0.001	0.106	23.7 [20.5, 26.8]	29.2 [25.1, 33.3]	34.8 [30.5, 39.2]	<0.001	0.033	26.2 [24.1, 28.4]	27.0 [25.0, 29.0]	31.9 [29.9, 33.9]	<0.001	0.046	25.3 [23.4, 27.1]	27.6 [25.7, 29.4]	32.7 [30.8, 34.5]
	B	<0.001	0.092	24.3 [21.3, 27.3]	28.8 [25.0, 32.6]	34.1 [30.1, 38.1]	0.005	0.021	26.8 [24.7. 28.9]	27.3 [25.4, 29.3]	31.1 [29.2, 33.1]	<0.001	0.033	25.8 [24.1, 27.6]	27.8 [26.0, 29.5]	31.9 [30.1, 33.7]
	C	0.001	0.088	24.1 [21.1, 27.2]	28.7 [24.9, 32.6]	34.6 [30.2, 38.9]	0.044	0.012	26.9 [24.7, 29.0]	27.3 [25.4, 29.3]	30.4 [28.4, 32.5]	<0.001	0.024	26.1 [24.2, 27.9]	27.8 [26.1, 29.6]	31.6 [29.7, 33.5]
Category Fluency	A	0.004	0.075	28.2 [26.5, 29.9]	25.6 [23.5, 27.8]	23.3 [21.0, 25.6]	0.056	0.011	26.7 [25.8, 27.6]	26.0 [25.1, 26.9]	25.2 [24.3, 26.0]	<0.001	0.024	27.2 [26.4, 28.0]	25.9 [25.1, 26.7]	24.8 [24.0, 25.7]
	B	0.011	0.060	27.8 [26.3, 29.4]	25.8 [23.8, 27.9]	23.7 [21.6, 25.9]	0.289	0.005	26.5 [25.6, 27.4]	25.9 [25.0, 26.7]	25.5 [24.7, 26.3]	0.006	0.016	27.0 [26.2, 27.8]	25.8 [25.0, 26.6]	25.1 [24.3, 25.9]
	C	0.086	0.033	27.6 [26.0, 29.2]	25.7 [23.7, 27.7]	24.4 [22.1, 26.7]	0.805	0.001	26.2 [25.3, 27.2]	25.8 [25.0, 26.7]	25.9 [25.1, 26.8]	0.110	0.007	26.7 [25.9, 27.5]	25.7 [24.9, 26.5]	25.5 [24.7, 26.4]
Letter Fluency	A	0.214	0.021	18.2 [16.9, 19.4]	16.7 [15.1, 18.3]	16.6 [14.9, 18.2]
	B	0.384	0.013	18.0 [16.8, 19.2]	16.8 [15.3, 18.3]	16.8 [15.1, 18.4]
	C	0.334	0.015	18.0 [16.8, 19.3]	16.8 [15.3, 18.4]	16.6 [14.8, 18.4]
BNT	A						0.271	0.005	55.0 [54.3, 55 6]	54.7 [54.1, 55.2]	54.3 [53.7, 54.8]
	B						0.776	0.005	54.6 [54.1, 55.2]	54.5 [54.0, 55.0]	54.7 [54.2, 55.2]
	C						0.765	0.001	54.8 [54.2, 55.3]	54.5 [54.1, 55.0]	54.8 [54.3, 55.3]
SDMT	A	0.001	0.088	61.2 [58.2, 64.1]	60.6 [56.8, 64.3]	52.2 [48.2, 56.2]	0.011	0.018	58.0 [56.2, 59 8]	58.4 [56.7, 60.1]	55.1 [53.4, 56.7]	<0.001	0.029	58.9 [57.3, 60.4]	59.0 [57.4, 60.5]	54.6 [53.1, 56.2]
	B	0.002	0.081	60.5 [57.8, 63.3]	60.9 [57.5, 64.4]	52.9 [49.2, 56.6]	0.077	0.010	57.6 [55.9, 59.3]	58.2 [56.5, 59.8]	55.6 [54.0, 57.2]	0.001	0.020	58.4 [56.9, 59.9]	58.8 [57.4, 60.3]	55.2 [53.7, 56.7]
	C	0.047	0.042	60.0 [57.2, 62.7]	60.6 [57.2, 64.1]	54.4 [50.4, 58.3]	0.392	0.004	57.3 [55.6, 59.1]	58.1 [56.5, 59.7]	56.5 [54.9, 58.2]	0.059	0.009	57.9 [56.4, 59.4]	58.6 [57.1, 60.1]	56.0 [54.4, 57.6]

Discriminant Validity

Antonyms was significantly more related to MSCS Expressive Language than to Working Memory, Episodic Memory, or Executive/Speed (Table 3). Moreover, multiple regressions identified Expressive Language as the only MSCS subscale independently related to Antonyms performance (Table S2).

Table 3. Discriminant Validity.

Partial correlations are shown between Antonyms performance and MSCS subscales adjusting for age, sex, education, and sample (A), additionally adjusting for premorbid verbal ability (B), and additionally adjusting for depression symptoms (C). Steiger Z tests indicated a stronger relationship between Antonyms performance and patient-reported difficulties in Expressive Language than between Antonyms performance and difficulties in Working Memory, Episodic Memory, or Executive/Speed.

Model	Expressive Language	Working Memory	Episodic Memory	Executive / Speed
A	.266	.193*	.191*	.142**
B	.233	.162*	.150*	.141*
C	.211	.132*	.117*	.101**

For each model (A, B, C), Steiger Z tests assessed whether the correlation between Antonyms and MSCS Expressive Language significantly differed from correlations between Antonyms and the other MSCS subscales; Bonferroni-adjusted P-values: <0.05* <0.01**

DISCUSSION

Antonyms has good reliability and validity as a performance metric of word-finding difficulty in MS. Patient-reported word-finding difficulty was more related to performance on Antonyms than on traditional language tasks (Category Fluency, Letter Fluency, BNT) or general MS-related cognitive difficulty (SDMT). Antonyms also demonstrated discriminant validity as a metric specifically assessing word-finding / expressive language difficulty, rather than other cognitive functions.

Traditional rapid word generation tasks were initially developed to evaluate verbal fluency in the context of aphasia; these tasks appear less sensitive to MS-related word-finding difficulties, which are subtle but nonetheless impactful during early adulthood. Verbal fluency tasks require generation of any words within broad categories, unlike the naturalistic word-finding demand to rapidly retrieve one specific target word. Outside of aphasia, rapid word generation (verbal fluency) tasks are often interpreted as metrics of cognitive speed or executive function rather than language.^11-12 Slowed cognitive speed may explain worse verbal fluency among persons with MS evaluated decades ago,¹³ leading to inclusion of verbal fluency within MS cognitive test batteries.^14-15 Importantly, unlike historical MS cohorts, cognitive speed is normal among current patients diagnosed within the modern diagnostic and treatment era,¹ for whom verbal fluency performance is also normal, but reported word-finding difficulty remains prevalent. Note that Antonyms does require rapid performance; however, subtraction of control task time isolates word-finding latency from general cognitive speed and articulation.

Antonyms was designed to assess phonological code retrieval by reducing semantic processing requirements through the use of strong word-pair associations. Implication of phonological code retrieval aligns with our observation that phonemic processing is below expectations in MS, and linked to worse patient-reported word-finding difficulty.⁶ A focus on phonological code retrieval is also consistent with radiologic evidence from ultra-high field MRI identifying the planum temporale / temporoparietal region as particularly vulnerable to cortical lesion formation in MS;¹⁶ phonological code retrieval is the chief language-related function of this region.¹⁷

Word-finding / phonological code retrieval needs further investigation, including whether MS-related word-finding difficulty stems from failure to activate specific codes and/or failure to inhibit competing codes (i.e., to retrieve “large,” one must inhibit synonyms such as “big,” “large,” and “huge”; see work on language disorders¹⁸). We need collaboration with experts in linguistics, neurophysiology and neuromodulation of language systems, and sub-millimeter neuroimaging of subtle perisylvian pathology. As a validated performance metric of MS-related word-finding difficulty, Antonyms will help to advance understanding and eventual treatment of this prevalent symptom of great importance to persons living with MS.