Test–retest reliability and stability of N400 effects: Implications for the study of neuropsychiatric and cognitive disorders

The report in the present issue authored by Kiang and colleagues, “Test–retest reliability and stability of N400 effects in a word-pair semantic priming paradigm”, makes a significant contribution to the N400 and event-related potential (ERP) literatures (Kiang et al., 2013). In two sessions approximately 1 week apart, 16 normal participants performed a lexical decision task to visually-presented word pairs. The study results demonstrate high, but imperfect, test–retest reliability on N400 amplitude over a 1 week interval. Between ~62% and ~90% of the variance in the main N400 amplitude measures at the vertex (channel Cz) at time 2 could be accounted for by the N400 amplitude at time 1 (see Table 2, and Supplementary Table S1 in Kiang et al., 2013). In addition, significant effects of time were found on both N400 amplitude and latency. At time 2, smaller priming effects were observed for N400 amplitude, N400 latency and response time (RT) measures. The amplitude of the N400 priming effect was, on average, approximately 1 μV smaller at session 2, due mostly to increased N400 amplitude to the semantically-related target words. The authors attributed the diminished priming effects to lessened motivation at session 2, and noted that modulations of other overlapping components also cannot be ruled out. One plausible explanation is that a decrease in the amount of attentional resources allocated to the stimuli may have produced a decrease in P300 amplitude. Alternatively, changes in attention or executive control processes may have interfered with the automatic spread of activation to semantically related items during the repeat session. This would presumably cause a direct modulation of the N400 component, augmenting the amplitude elicited by related words. It is interesting to note the authors did not find a significant FN400 – like effect, wherein repeated unrelated words would be expected to produce more positive scalp voltages due to familiarity-related memory processes. This is not surprising, in that 1 week had elapsed between sessions, and a relatively short SOA of 750 ms was employed. Unfortunately, no data was collected on the participant’s recollection or familiarity for the experimental stimuli.

Nonetheless, this is an important and timely report which fills a gap in the N400 literature. As the authors note, “The high test–retest reliability … in this study supports further exploration of their utility as a potential biomarker in longitudinal treatment studies of clinical disorders.”

N400 semantic priming effects have been used as a neurophysiological probe of abnormal semantic processing in clinical disorders, and suggested as a possible biomarker for treatment studies. Our prior longitudinal studies in mild cognitive impairment (MCI) and early-stage AD have found that N400 abnormalities typically emerge 1–2 years before a dementia diagnosis in MCI to AD converters (Olichney et al., 2008). Despite potentially important applications of the N400 to disorders such as Alzheimer’s disease (AD), schizophrenia, temporal lobe epilepsy (Olichney et al., 2002), and aphasia (D’Arcy et al., 2003; Hagoort et al., 1996), there has been a general lack of published test–retest reliability studies. One particularly useful implication of this study is it opens the door to using the N400 as a marker of response to short trials of cognitive therapies. Other possible applications include the short-term responses to stress, sleep deprivation, and other behavioral interventions.

While most ERP studies of schizophrenia have found decreased amplitude N400 effects (Grillon et al., 1991; Kiang et al., 2008), one particular study reported increased N400 amplitudes in these patients (Nestor et al., 1997). This study used sentences, of which 50% ended with a non-sensicle, highly improbable final word, designed to elicit large N400s. The authors interpreted their results as possibly representing a failure of the schizophrenia group to adapt their semantic expectancy to these unusual sentence endings, relative to the normal group. Reality often appears strange and disjointed to those suffering from psychotic disorders, and one can only speculate if the thought disordered and less mentally-flexible patients were perturbed by the incoherence of those sentences, rather than ‘getting it’ (i.e. the gestalt, “this is just a psycholinguistic experiment, expect a lot of curveballs”).

Working with linguistic stimuli poses some unique challenges. Not only do words and concepts strongly influence how we process subsequent words (as semantic priming, repetition priming and reading speed studies consistently show), but the structure of each individual’s semantic network is likely to be slightly different. Nonetheless, the authors found comparable or better test–retest reliability than those found in healthy individuals with other putative ERP biomarkers of schizophrenia, such as the amplitude of the mismatch negativity or P300 (Lew et al., 2007). The carry-over effects observed by Kiang et al. (2013) at 1 week, might be reduced by:

1. Increasing the time-interval between ERP sessions. Using annual longitudinal ERP studies of word repetition, our laboratory has not found significant effects of time upon either the N400 or P600 components in MCI patients who remain clinically stable over 3 years or longer (Olichney et al., 2008).

2. Using novel words lists at retest. This would require carefully-matched linguistic stimuli, balanced on word frequency, typicality, length and other orthographic factors. This might reduce variance by eliminating repeated exposures to the same words, but will not eliminate possible practice effects or adaptation to the experimental paradigm.

The human brain is not only remarkable for its semantic organization and its innate ability to extract ‘meaning,’ but also for its capacity to adapt to the global features of an experiment or an experience. This is illustrated by the results of Kiang et al. (2013), and of Nestor et al. (1997) cited above.

To summarize, Kiang et al. should be commended for this work and these empirical data represent a significant step forward. Cognitive ERPs continue to provide novel insights into the mechanisms of semantic and strategic cognitive processes. Follow-up longitudinal N400 studies with cognitively normal older persons would be particularly helpful in the translation of the N400 from the ivory-tower confines of cognitive science labs into clinical realms where a more thorough dissection and understanding of cognitive pathophysiology remains a highly sought stepping stone on the path to optimizing the treatments for cognitive disorders.