Digit Symbol Substitution Test

Abstract

Purpose/Background

Development of the Digit Symbol Substitution Test (DSST) was initiated over a century ago as an experimental tool to understand human associative learning. Its clinical utility, owing to its brevity and high discriminant validity, was first recognized in the 1940s, and now the DSST is among the most commonly used tests in clinical neuropsychology.

Methods

Specific studies and articles were reviewed to illustrate what the test measures, to evaluate its sensitivity to change, and to discuss its use in clinical practice.

Results

The DSST is a valid and sensitive measure of cognitive dysfunction impacted by many domains. Performance on the DSST correlates with real-world functional outcomes (eg, the ability to accomplish everyday tasks) and recovery from functional disability in a range of psychiatric conditions including schizophrenia and major depressive disorder. Importantly, the DSST has been demonstrated to be sensitive to changes in cognitive functioning in patients with major depressive disorder and offers promise as a clinical decision-making tool for monitoring treatment effects in this and other disorders affecting cognition.

Implications/Conclusions

The DSST is sensitive to the presence of cognitive dysfunction as well as to change in cognitive function across a wide range of clinical populations but has low specificity to determine exactly which cognitive domain has been affected. However, the DSST offers a practical and effective method to monitor cognitive functions over time in clinical practice.

DESCRIPTION AND HISTORY

The Digit Symbol Substitution Test (DSST) was initially developed as an experimental tool over a century ago by researchers seeking to understand human associative learning.^1,2 The DSST is a paper-and-pencil cognitive test presented on a single sheet of paper that requires a subject to match symbols to numbers according to a key located on the top of the page. The subject copies the symbol into spaces below a row of numbers. The number of correct symbols within the allowed time, usually 90 to 120 seconds, constitutes the score (Fig. 1).³ The DSST is perhaps the most commonly used test in all of neuropsychology, owing to several inherent properties: brevity, reliability, and the minimal impact of language, culture, and education on test performance. Yet, the question of “what it measures” still has no definitive answer, a consequence of the historical context from which it emerged.

FIGURE 1.

The DSST symbol coding sheet. Figure reprinted from Patel and Kurdi.³

The intent of this article is to provide an overview of the DSST, addressing what cognitive operations are required to perform it and what impaired DSST performance may indicate. The aim is to consider its utility in monitoring cognitive functions over time in clinical practice. With this in mind, selected studies have been highlighted to help illustrate the test's sensitivity to change, as well as how performance on the DSST correlates with real-world functioning, so as to provide practitioners with insight into its use for monitoring treatment effects in patients having a range of disorders, with particular attention to its utility for management of major depressive disorder (MDD).

The utility of the DSST as a clinical tool in neuropsychology first became evident when it was shown to reliably distinguish patients with brain damage from healthy patients during the screening of soldiers in World War II. The use of the DSST widened after becoming incorporated in the Wechsler-Bellevue Intelligence Scale (WBIS), developed in 1939²; the WBIS relied heavily on early versions of the DSST dating to at least 1900.¹ Sometimes termed “coding” or “symbol coding,” the test paradigm has survived with minor alterations to the most recent version found in the Wechsler Adult Intelligence Scale (WAIS), which is currently in its fourth edition (WAIS IV).^4,5 As a subtest of the WAIS, the DSST has undergone repeated and rigorous psychometric validation such as test-retest reliability and discriminant validity in a range of patient samples; however, modest differences exist between these versions.

The original WAIS, developed in 1955, has 90 stimulus items (number of blanks into which the symbols are copied), all of which were among the 67 stimulus items in its predecessor, the WBIS.^2,6 The DSST was not substantially different in the subsequent version of the WAIS, the WAIS-Revised. However, in WAIS-III, the test duration was extended from 90 to 120 seconds, the response form filled a full page, and the size and spacing of the stimuli were increased so that hooked left-handers were no longer required to move their hands to see the key items.

Symbol coding paradigms comparable to the DSST (with different symbols) are included as subtests in the Brief Assessment of Cognition in Schizophrenia and the Repeatable Battery for the Assessment of Neuropsychological Status.^7,8 The Symbol Digit Modalities Test uses essentially the same method, except in reverse; instead of drawing symbols that match digits, the subject is required to write the matching digit in the blank.⁹ This version of the coding paradigm has the advantage that it can be easily altered to allow oral responses in disorders that affect motor functioning (eg, multiple sclerosis).¹⁰

Computerized tests offer another option for monitoring change over time in a clinical setting. To be useful, a measure must be validated for sensitivity to change and reliability in the population in which the clinical application is being contemplated. Several commercial efforts have been undertaken along these lines. For example, the Cambridge Neuropsychological Test Automated Battery developed by Cambridge Cognition is being used to track memory decline in elders with a view toward early detection of dementia.^11,12 Similarly, Cognigram, a version of the Cogstate Brief Battery formatted for use in the clinic, consists of a 10-minute battery of computerized tasks requiring attention, processing speed, memory, and executive functioning that have been validated for sensitivity to change in a range of conditions including MDD.^13–16 Both of these tests are commercially available and approved for clinical use. THINC-IT is a freely available computerized test that also includes a questionnaire to capture subjective cognitive complaints; it is designed for monitoring change in MDD and is becoming increasingly used, although it has not yet achieved regulatory approval.¹⁷ An advantage of the DSST over computerized tools is its simplicity, speed (2 minutes, in contrast to 10–15 minutes for computerized options), and ready accessibility. No computer or software subscription is needed; only the test form, a writing implement, and a timer are required. Like the computerized version, the validity and reliability of the paper DSST depend on the test being administered in the manner it was validated. When scoring the DSST, speed and accuracy are both taken into account, allowing the examiner to use norms or refer to research findings for comparison.

As illustrated below, the DSST is clinically useful because it is sensitive to cognitive deficit in a wide range of brain diseases and conditions. However, without further testing of the patient, the DSST is not informative about the nature of the deficit. The widespread clinical use of the DSST has produced large amounts of data that have helped shed some light on what the DSST measures and the significance of deficits and of change over time in its performance.

What Does the DSST Measure?

The DSST measures a range of cognitive operations. Good performance on the DSST requires intact motor speed, attention, and visuoperceptual functions, including scanning and the ability to write or draw (ie, basic manual dexterity). Performance might also be affected by associative learning. For example, if pairings are rapidly learned following the first few trials, performance speed will improve because the subject will not need to refer to the key to check the accuracy of each pairing. The decision to consciously engage in this learning strategy to improve performance speed calls for the executive functions of planning and strategizing. Working memory, another executive function, is likely required to hold in mind the task rules and for the continual updating of required symbol-digit pairs.

According to Lezak's¹⁸ widely used neuropsychological textbook, the DSST is a measure of “Complex Attention,” yet the author indicates that the cognitive aging literature also points to a strong influence of motor speed on performance. For example, in elderly populations, copying speed alone explains 72% of the variance in performance difference between young and elderly groups.¹⁸ This finding supports the observation that the DSST is a polyfactorial test and that impairment in any one of the required domains (ie, motor speed associated with aging) results in performance decline. Darby and Walsh¹⁹ emphasized that performance on the DSST is the final common pathway for expression of quite diverse types of impairment and described (quoting Kinsbourne) DSST responses as “the end product of the integration of visual perceptual, oculomotor, fine manual motor, and mental functions.”

Maxwell²⁰ argued that the DSST should be classified as a verbal rather than a performance subtest in light of a slightly higher correlation with the verbal intelligence quotient than the performance intelligence quotient in the normative sample of Wechsler. Laux and Lane²¹ reviewed the correlations of the DSST with other measures in the literature and also suggested that DSST performance involves verbal ability.

Beres and Baron²² measured the effect of active training on DSST performance. The authors showed that both elderly and young subjects improved not only within each day but also between days. Although the older women performed at lower levels than did the younger ones at all points during training, both age groups of women improved fairly consistently over the 20 administrations given on the same day (likely because they were learning the actual symbol pairs). Further, both groups improved from day to day, even though on each day they were given different forms of the test. This observation suggests that improvement with exposure occurs in part for reasons other than associative learning of the pairs. These findings indicate that the executive function of strategizing or consciously exerting effort to learn the pairings is among the activities that contribute to performance on this test. It is noteworthy that both groups improved with practice, and the difference between the groups remained fairly constant. That the DSST is sensitive to age effects is well known.²³ The Beres and Baron²² findings suggest that age-related effects are attributable to age differences in motor speed and not in higher cognitive functions. The DSST has been a useful tool to demonstrate age differences in the effects of drugs on cognition, for example, owing to reduced clearance resulting in higher plasma concentrations in older than in younger subjects.²⁴

Finally, the possibility that executive functioning contributes to performance on the DSST is supported by findings of Thornton and Carmody.²⁵ The authors used a Symbol Digit test to examine the relationship between quantitative electroencephalography (QEEG) variables and cognitive performance.²⁵ The authors showed that test performance was associated with QEEG indices, including relative power and coherence at several frequency bands, particularly within the frontal region of the brain.²⁵ This correlation of specific QEEG indices with activation in the frontal lobes supports the theory that executive functioning plays a part in DSST performance.

Sensitivity and Sensitivity to Change of the DSST

David Wechsler² first described the DSST's sensitivity to brain damage in 1936. Since then, the DSST was found to be the single most sensitive test, among WAIS subtests, to determine the presence of brain damage and discriminate between patient groups (including healthy comparison groups).²⁶ For example, Glosser et al²⁷ noted the sensitivity of the DSST in alcoholic Korsakoff patients, patients with right hemisphere damage, and patients with chronic alcoholism. While psychomotor retardation contributed to impaired performance in these patient populations, the investigators, using other cognitive measures, determined that visuoperceptive processing impairments were also involved in DSST performance.

The sensitivity of the DSST is owed, in part, to its property as a polyfactoriaI measure.²⁸ As alluded to above, impaired performance on a polyfactorial measure may arise for different reasons in differently impaired populations. For example, age-related impaired performance may be attributed, in part, to a decline in motor speed, often associated with aging (as noted earlier in this article). In alcoholic Korsakoff patients, patients with right hemisphere damage, and patients with chronic alcoholism, performance deficits arise from impairments in visuoperceptive processing.²⁷ In schizophrenic patients, processing speed is known to be a critically impaired cognitive ability. In fact, deficits in processing speed as measured by DSST performance may mediate a wide range of other cognitive deficits and may further reflect a final common pathway of impairment in a range of simple and complex cognitive processes that are likely to include working memory and executive functions.²⁹

The ability of the DSST to detect deterioration in processing speed has also been useful as a complementary cognitive assessment in the screening of patients with diabetes mellitus type 2.³⁰ In both middle-aged and older persons, diabetes mellitus type 2 has been found to be associated with an increased risk of mild cognitive impairment and dementia.³⁰ Therefore, brief cognitive tests such as the DSST may prove to be a practical and effective method for use in clinical practice to detect decrements in cognitive processing over time. Interestingly, reports of the predictive nature of lower DSST scores on the development of clinical and subclinical disorders of cognition and mobility have recently been published.³¹ With additional confirmatory studies, the authors postulate that slower processing speeds as detected by the DSST may serve as a type of biomarker for disorders of cognition, mobility, and possibly even mood. Use of the DSST for studying the pathophysiology of psychomotor slowing with age may also provide insights into the pathology of age-related brain disorders.³¹ Clearly, the utility of the DSST in the evaluation and management of patients across various therapeutic indications is evident.

The DSST's sensitivity to change, both acutely and chronically, is one of its hallmark features. For this reason, the DSST has been used as a standard tool in clinical pharmacology studies from as early as the 1970s. A review of such studies may offer a means for benchmarking the magnitude of change in standardized effect size units or as a function of the DSST's correlation with other biological measures. The clinical meaningfulness of changes of different magnitudes may then be considered from this vantage point.

Roth et al³² evaluated the sedative effects of antihistamines on a battery of tests including the DSST. The authors demonstrated that the DSST could statistically significantly distinguish the cognitive effect of the H1 antagonist diphenhydramine from placebo with an effect size of 0.28 (calculated from the published statistics).

Thapar et al³³ compared the cognitive and sedating effects of several anesthetic agents at doses commonly used in ambulatory surgery. The dependent variable was performance on a modified DSST, timed for 1 minute. As a standard control, alcohol was administered to the subjects to a blood alcohol concentration (BAC) of 0.11%, a dose associated with significant intoxication. Digit Symbol Substitution Test performance was markedly reduced during alcohol intoxication with scores approximately 10 points lower than that of placebo. Although learning or practice effects may occur because the same test is administered multiple times, the randomized design of the study overcomes any resulting confounding factors. Different magnitudes of cognitive effect, as measured with the DSST, can be used to make comparative inferences about the relative effects of various doses of agents, such as alcohol, on cognitive performance (Fig. 2).³³

FIGURE 2.

Alcohol has known effects on DSST performance; this observation can be used as a benchmark to reference the recovery time following administration of sedating agents. Graph shows effects over time following 2 mg/70 kg midazolam (square), 70 mg/70 kg propofol (circle), and 50 μg/70 kg fentanyl (triangle) on the DSST performance relative to placebo (diamond) and relative to the effect of sustained BAC levels of approximately 0.11%. The alcohol effects are denoted in the gray area (mean: center line; outermost lines: ±1 SEM). Closed symbols represent statistically significant differences from placebo at a given time. Figure reprinted from Thapar et al.³³ Promotional and commercial use of the material in print, digital or mobile device format is prohibited without the permission from the publisher Wolters Kluwer. Please contact permissions@lww.com for further information.

Greenblatt et al³⁴ compared the sensitivity to change of the DSST with that of the electroencephalogram (EEG) to the effects of a single dose of 0.375 mg triazolam versus placebo. Benzodiazepines (eg, triazolam) are widely prescribed drugs for the treatment of anxiety, insomnia, seizures, alcohol withdrawal, and many other disorders. Compared with placebo, triazolam significantly impaired performance on the DSST (P < 0.001) and increased beta amplitude on the EEG (P < 0.002).³⁴ Both DSST and EEG changes closely tracked changes in plasma triazolam concentrations over time.³⁴

The DSST is often used as a pharmacodynamic marker to assess pharmacokinetic-pharmacodynamic relationships. For example, Greenblatt et al³⁵ used the DSST to evaluate age and sex effects on the pharmacokinetics and pharmacodynamics of 0.25-mg triazolam in 61 healthy men and women, aged 20 to 75 years. This double-blind crossover study demonstrated a statistically significant effect of age on predose DSST performance, with lower baseline scores in the elderly subjects further reinforcing the effect of age on DSST performance. The percent decrement in DSST scores with age was also significantly greater in men than in women. In a similar study, Greenblatt et al³⁶ evaluated the effect of dose and sex on the pharmacokinetics and pharmacodynamics of the sedative zolpidem in healthy, nonelderly male and female volunteers and included the DSST among the pharmacodynamic measures. In this placebo-controlled, 4-way crossover study, evaluation of the pharmacodynamic effect area under the curve indicated a significant influence of sex, independent of dose, on the DSST such that effects on women were seen to be greater than those on men.³⁶

The DSST is sensitive enough to distinguish changes over time between different sleep groups in sleep studies. Sleep experiments have used the DSST as a tool to study the cognitive effects of chronic sleep deprivation. Figure 3 shows the mean performance on a computerized DSST (tested every 2 hours) in a sample of 35 subjects randomized to receive 4, 6, or 8 hours of sleep each night for 14 days.^37,38 The 8-hour group showed some improvement over the first few days (possibly a learning effect), whereas the 4-hour group differed from the 8-hour group by more than 12 points by the end of the first week and continued to decline over the second week. In the 6-hour group, there was no change from baseline over time. Tucker et al³⁹ also measured the effect of sleep deprivation on DSST performance and yielded similar results. Another example of the DSST's sensitivity to chronic change is illustrated in abstinence testing of chronic alcoholics. Only the DSST, among a battery of neuropsychological tests, was able to measure the benefit from long-term abstinence (nearly 2 years) over short-term abstinence (6 weeks) in chronic alcoholism, suggesting its greater sensitivity to change compared with other measures.²⁸

FIGURE 3.

The chronic sleep restriction experiment involved randomization to 1 of 3 sleep doses (4-, 6-, or 8-hour time in bed per night), which were maintained for 14 consecutive days. Subjects were tested every 2 hours each day; data points represent the daily average (7:30 am–11:30 pm) expressed relative to baseline. The study also involved 3 baseline (predeprivation) days and 3 recovery days.³⁷ Figure republished with permission of the Journal of Sleep Medicine from Banks S, Dinges DF. Behavioral and physiological consequences of sleep restriction. J Clin Sleep Med. 2007;3:519–528; permission conveyed through Copyright Clearance Center, Inc. TIB indicates time in bed.

The wide use of the DSST across so many applications permits unique opportunities for comparison. For example, it is reasonable to argue that using widely understood perturbations, such as sleep deprivation, alcohol, or common pharmacologic agents as benchmarks, offers a reasonable approach to appreciating the clinical meaningfulness of a change in cognition on the DSST. Recent interest in office-based clinical monitoring of cognitive change in patients having MDD and the sensitivity of the DSST to cognitive change in this population suggests true clinical value may be realized by understanding the clinical meaningfulness of change on the DSST.

Clinical Meaningfulness in Patients With MDD

Many studies have examined neuropsychological performance, using a variety of neuropsychological tests, in patients with MDD, and overall, impairments of moderate size have been observed.⁴⁰ Further, these impairments appeared to have been present from the first depressive episode.⁴⁰ A recent meta-analysis on executive dysfunction estimated the magnitude of impairment relative to control subjects on a wide array of neuropsychological tests and concluded that “MDD is reliably associated with impaired performance on neuropsychological measures of executive functioning.”⁴¹ In that report, data from 1904 subjects evaluated using the DSST in a total of 22 studies demonstrated an impairment of moderate magnitude among patients with MDD with a mean effect size (Cohen’s d) relative to control subjects of 0.55 (confidence interval, 0.34–0.75; P < 0.001). The effect size of 0.55 was in line with the effect sizes of other measures examined. In elderly depressed patients relative to elderly control subjects, Nebes et al⁴² reported DSST decrements of about double this magnitude and further found that performance was worse in patients with late compared with early onset of disease (P < 0.04).

Vortioxetine, an antidepressant indicated to treat MDD, was evaluated in 2 placebo-controlled clinical trials, and the DSST was used to evaluate the efficacy of 10 and 20 mg of vortioxetine for improving cognitive function in patients with MDD.^43,44 In the first study, the effect sizes of treatment with 10 or 20 mg of vortioxetine on DSST scores were, respectively, 0.51 and 0.52.⁴³ The second study also showed positive treatment effects on the DSST, although with a smaller effect size of 0.254 for 10 to 20 mg vortioxetine.⁴⁴ In addition, a meta-analysis showed significant effects of vortioxetine versus placebo in improving cognitive function assessed with the DSST in patients with MDD and that these improvements were greater than those observed with other antidepressants.⁴⁵

In an earlier report of MDD patients studied 6 months after discharge from hospital for an MDD episode, the DSST was strongly associated with the magnitude of functional recovery in work, school, and housing, yielding an odds ratio of nearly 20.⁴⁶ Hence, there is evidence that the impairment and change measured by the DSST reflect meaningful differences relevant to the clinical management of MDD patients. Further, there is growing awareness of the prevalence of cognitive symptoms and their impact on life functioning in MDD patients, leading to an increase in the recognition of such symptoms as clinically important targets for therapeutic improvement.⁴⁷ Although the importance of cognition for functioning and quality of life may be regarded as self-evident, the meaningfulness of a given magnitude of impairment or change on a brief measure such as the DSST is often questioned. A more refined, data-driven method for establishing the meaningfulness of a particular magnitude of change would be of enormous interest to clinicians seeking to use this test for clinical decision making.

As discussed previously, the DSST's sensitivity to change is one of its main strengths. The magnitude of change on DSST performance, as well as the effect on DSST of an illness state relative to a state of health, can be represented using a standard effect size statistic. The standard effect size is a metric that allows one to compare results of different studies, for example, those examining pretreatment versus posttreatment or treatment (drug) versus placebo, or those examining the magnitude of deficit (relative to normal) in various patient groups.

To appreciate the clinical meaningfulness of a statistically significant cognitive effect of a new pharmacologic agent, it is useful to review the effects of a range of known compounds for which the clinical meaningfulness of effect in healthy volunteers is fairly well understood. Figure 4 illustrates one way to consider the clinical meaningfulness of effect sizes through direct comparison with familiar benchmarks.^{32,39,43,44,48–53} The figure shows the magnitude of impairment seen with a well-understood exposure to ethanol (ie, BAC of 0.088%) and compares it with that of 2 mg of lorazepam, a sedative for the treatment of anxiety disorders or symptoms, or 150 mg of diphenhydramine, an antihistamine with sedative effects, in healthy volunteers. Additional comparisons include effect sizes and direction of change after caffeine intake, caffeine deprivation, and 1 full night of sleep deprivation. Although benchmarks from acute adverse effects in healthy volunteers are not clinically equivalent to a sustained 8-week benefit of vortioxetine in patients with MDD, these examples can provide some context to understand the meaningfulness of effects. Such benchmarking offers an evidentiary basis for concluding that the observed improvements of DSST scores are of a magnitude likely to be clinically meaningful. Indeed, it may be possible to think of an improvement of this magnitude as comparable to removing the transient effect of a similar magnitude resulting from the BAC and lorazepam dose noted above.

FIGURE 4.

Effect size on DSST performance of antidepressants in patients with MDD is compared with the effect size in healthy volunteers for known compounds such as acute caffeine consumption, caffeine deprivation, and diphenhydramine (an antihistamine). Benchmarking effect sizes to known compounds in clinically healthy volunteers serves as a guide to evaluating the clinical meaningfulness of particular effect sizes in cognition.^{32,39,43,44,48–53} Positive effect sizes indicate improvements in cognitive functioning, and negative effect sizes indicate decline in cognitive functioning. The top part of the graph shows the effects of antidepressant treatments on cognitive functioning, whereas the bottom part illustrates the effects of other agents known to affect cognitive functioning in healthy volunteers. Figure 4 can be viewed online in color at www.psychopharmacology.com.

CONCLUDING COMMENTS

The utility of the DSST for monitoring patients in the clinic derives from several properties, many of which are shared by other neuropsychological tests, including its brevity, its sensitivity to change, and minimal impact from education and culture. The DSST offers high sensitivity to detect impairment, but low specificity to determine exactly which cognitive domain has suffered the impairment. A legitimate question may be raised as to whether a single brief cognitive assessment tool can itself be sufficient to draw clinically meaningful conclusions under any circumstances. Neuropsychologists have become accustomed to the use of extensive batteries of tests, and redundancy of measurement may improve the reliability of any conclusion drawn about cognitive dysfunction. However, where the objective of testing is not diagnosis, but rather to identify impairment regardless of its nature and origin or to detect change within a patient, a brief and simple test such as the DSST may offer important benefits to clinical practice.

The DSST is sensitive to both the presence of cognitive dysfunction and change in cognitive function, across a wide range of clinical populations. In psychiatric patients, the DSST is sensitive to impairments and improvement in processing speed, executive functioning, and working memory. Performance on the DSST correlates with real-world functional outcomes (eg, the ability to accomplish everyday tasks) and recovery from functional disability. In addition, the DSST has been demonstrated to be sensitive to change in cognitive functioning in patients with MDD and may offer an effective means to detect clinically relevant treatment effects. In the context of clinical decision making in a physician's office, this simple cognitive test can, in combination with a psychiatric clinical evaluation, be useful at the level of the individual patient for medication choice and dosing.