Overnight consolidation of speech sounds predicts decoding ability in skilled adult readers

Abstract

Purpose:

Phonological representations are important for reading. In the current work, we examine the relationship between speech-perceptual memory encoding and consolidation to reading ability in skilled adult readers.

Method:

Seventy-three young adults (age 18–24) were first tested in their word and nonword reading ability, and then trained in the late evening to identify an unfamiliar speech sound contrast (Hindi retroflex-dental). Participants were assessed in their ability to perceive the target contrast immediately before training, after training, and 12 hours later.

Results:

While perceptual performance on the target at any time point was unassociated with reading ability, overnight changes to the post-training perceptual ability over the 12-hour delay was significantly associated with nonword reading (i.e. decoding) ability, but not real-word reading.

Conclusion:

These results provide preliminary support for the hypothesis that individual differences in memory processes that update phonological representations following acoustic-phonetic exposure relate to decoding performance, including in adulthood.

Introduction

The quality of phonological representations is often considered a causal predictor of reading ability (Goswami, 2000, Morais, 2003; c.f. Wessling & Reitsma, 2001). Competing frameworks on word reading converge on the idea that efficient reading is facilitated by a division of labor by the mappings between orthographic (i.e. spelling), semantic (i.e. meaning), and phonological (i.e. pronunciation) representations (Coltheart, 2006; Seidenberg, 2005). The orthography-to-phonology mapping allows the reader to “sound out” an unfamiliar letter string, whereas a frequently encountered letter string may access word meanings directly through connections between orthography and lexical-semantic representations. While that does not preclude phonological processing during recognition of frequent words, such models (e.g. dual-route, Coltheart, 2006; triangle, Seidenberg, 2005) suggest that the quality of phonological representations disproportionately affects decoding ability over the reading of familiar words, especially in experienced readers.

If this is the case, it stands to reason that the ability to represent sounds that comprise our phonological categories are essential to reading, and decoding ability in particular. This prediction has been explored traditionally through the relationships between auditory or speech-perceptual acuity and phonological processing (McArthur & Bishop, 2001), on the basis that the quality of the constituent input signals determines representational quality. Here, we propose the additional possibility that differences in memory processes that support the representation of acoustic-phonetic information may lead to differences in representational quality. Domain-general memory has been implicated in reading previously (Bitan & Karni, 2004; Earle et al., 2020), and particularly in the etiology of reading disorders (Nicolson & Fawcett, 2011; Lum, et al., 2013; Ullman, et al., 2020). These accounts are generally presented as a counter to the proposal of phonological processing as a causal deficit in dyslexia (e.g. Ramus et al., 2003). Of course, these perspectives are not mutually exclusive: subtle differences in broader, domain-general memory processes may yield qualitative differences in speech-category representations, leading to differences in phonological processing and reading.

The importance of memory processes to speech-sound representations is highlighted through the emerging literature on the role of offline consolidation in speech-perceptual learning. Memory consolidation broadly refers to the stages of processing that a memory trace undergoes (see Dudai, 2012 for review). ‘Systems’ consolidation, a process by which features from wake-state experience are integrated with long-term knowledge, is thought to be promoted by a period of time spent ‘offline’ (see Diekelmann & Born, 2010, for review). This may include periods of time spent in the absence of conflicting input, or specifically during sleep (Nemeth, et al., 2019).

In the speech domain, prior works collectively suggest that an offline/sleep-containing period following acoustic-phonetic training facilitates enhancement and generalization of perceptual performance (Earle & Myers, 2015a; 2015b; Fenn, et al., 2003; 2013; Xie, et al., 2016; Qin & Zhang, 2019). The magnitude of perceptual gain over the offline period appear to vary widely across individuals (Earle, et al., 2017; 2018). Acoustic-phonetic information that is encoded during the daily use of spoken language may be subject to similar consolidation processes (and thus, the same variability), observed for the laboratory-trained information.

The current study investigates the relationship between the offline consolidation of new (nonnative) acoustic-phonetic information and reading ability in skilled, adult readers. We hypothesized that individual differences in memory processes that support phonological representations, that is, the ability to learn and consolidate speech information, may be associated with reading ability. Specifically, given the link between phonological representations and decoding ability, we predicted that consolidation of speech would be associated with nonword, but not real-word, reading ability.

Materials & Methods

Participants

Participants were recruited via University of Delaware Institutional Review Board (IRB) – approved flyers posted throughout the university community, as part of a larger study on adults across the language spectrum. Participants were monolingual English speakers between ages 18 to 24, with typical hearing and vision. For the present study, data from participants reporting a history of reading, language, neurological, socio-emotional, attentional, or cognitive impairments or disorders, were excluded from the analyses. Participants on medication known to affect sleep were also excluded. Ninety-four individuals provided informed consent according to University of Delaware IRB guidelines and participated in our initial battery of standardized assessments. Of these, eight participants did not meet study criteria, ten chose to participate in a different arm of the study through our laboratory, and two declined to participate further. This resulted in a dataset of seventy-three participants (16 males and 57 females) who met the above criteria and completed the experimental procedures described below.

Materials

Word-level reading ability was assessed using the Word ID and Word Attack subtests of the Woodcock Reading Mastery Test –III (WRMT-III; Woodcock, 2011) and Phonemic Decoding and Sight Word Efficiency subtests of the Test of Word Reading Efficiency-2 (TOWRE-2; Torgesen, et al., 2012). These tests report good-to-excellent reliability (for WRMT-III, .88–.92; for TOWRE-2, .89–.93). Phonological processing was measured using the Elision subtest of the Comprehensive Test of Phonological Processing-2 (CTOPP-2; Wagner et al., 1999).

In order to track changes to acoustic-phonetic memory that can be distinguished from features encountered during normal language use, a non-native contrast was chosen as target. Auditory stimuli included five exemplars each of the retroflex /ɖ/ and dental /d̪/ consonants, naturally spoken in a /CVC/ frame (/ɖug/-/d̪ug/) by a male, native speaker of Hindi. Productions were recorded in a sound-proof audiology booth, cut to the burst onset, and rescaled to match on mean amplitude using Praat software (Boersma & Weenink, 2013). Visual stimuli were obtained from an online repository of novel visual objects (“Fribbles”, Stimulus images courtesy of Michael J. Tarr, Center for the Neural Basis of Cognition and Department of Psychology, Carnegie Mellon University, http://www.tarrlab.org/).

Stimulus presentation and response recording were controlled via E-Prime 3.0 software (Psychology Software Tools, Pittsburgh, PA) operating on Dell Latitude E5570 PC laptops (Dell, Inc., Round Rock, TX). Auditory stimuli were presented at 70dB via ATH-M50x circum-aural headphones (Audio-Technica, Inc, Stow, OH).

Procedures

Participants completed an initial battery of assessments, administered 1:1 in a quiet room. Typical hearing was confirmed through a 500Hz-6000Hz pure tone audiometric screen administered at 25dB. While we assessed a wide range of capacities through this battery, we only report measures of word-level reading abilities and a measure of phonological processing here. Scores were calculated from the test sheets by two independent scorers (>95% reliability).

Following this session, participants were invited for a 2-session experiment. Overnight consolidation of the target retroflex-dental contrast may be susceptible to interference effects from exposure to English /d/ (Earle & Myers, 2015a). To minimize linguistic activity between training and sleep, the first session (Day 1) was scheduled late in the evening (~8PM), and the second was scheduled at 8AM (Day 2).

Before training (Day 1), baseline perception of the target contrast was assessed through a discrimination test. Participants heard two tokens in succession at an inter-stimulus interval of 800ms, and were instructed to indicate whether the two sounds were the same or different. Each test contained 64 trials (32 same/32 different). The “same” trials contained two different exemplars within the same category (e.g. /ɖug₁/ - /ɖug₂/), to encourage discrimination judgements to be made on the basis of category membership rather than through raw acoustic comparison. For training, participants were familiarized first with the target token-label mapping through a serial presentation of the sounds with their corresponding visual token. Participants were then instructed to choose the image corresponding to the “word” that they hear. Participants completed 200 trials with written feedback (“Correct!”/”Incorrect”) provided after every trial. After training, learning was assessed in two ways: an identification test (50 trials of the training task without feedback), and a discrimination test (the same task used at baseline). Participants then went home, and returned at 8AM (Day 2) to complete reassessments of identification and discrimination.

Analyses and Results

Data presented in this paper and the code used for analyses and graph generation are made available through <repository link through https://github.com/fsearle/consolidation-of-speech-and-reading. Phonological processing and reading scores partially overlap with datasets presented in Earle and Del Tufo (2021), Del Tufo and Earle (2020), and Earle and Ullman (2021).

Power analysis

While data collection took place within the context of a larger study, the analyses described here were planned. The hypothesized relationship between perceptual changes and reading was reasoned to yield a medium effect size, based on medium to large effect sizes observed for consolidation effects in previous datasets. G*Power software (version 3.1; Faul, et al., 2007) was used to calculate a target sample size for our analyses with two predictors, assuming a Cohen’s f²=.25, an alpha of .05, and 1-β error probability at .95. This calculation recommended a sample size of at least 65. In order to arrive at this sample, we targeted an enrollment of approximately 95 in order to account for attrition and exclusion of individuals who do not meet criteria.

Data Transformations

To facilitate treating age as a covariate, we scaled the number of correct responses as a proportion of the total number of trials in each subtest instead of using standard scores (Moeller, 2015). These scaled values were then averaged across the Word Identification (WRMT-III; Woodcock, 2011) and Sight Word Recognition (TOWRE-2; Torgesen, et al., 2012) subtests, to arrive at a single real word reading score. The Word Attack (WRMT-III; Woodcock, 2011) and Phonemic Decoding (TOWRE-2; Torgesen, et al., 2012) scores were averaged for a nonword reading score.

Response bias is often considered for measures of perceptual discrimination (e.g. the tendency to call two tokens the same, or different, when uncertain), however the same bias potentially applies to identification performance (e.g. the tendency to identify “dental”, or “retroflex”, when uncertain). Thus, accuracy on the perceptual tasks were both transformed to d’ scores (Macmillan & Creelman, 2005). D’ scores were averaged across tasks at each time point. Internal reliability for this measure ranged from good to acceptable (Day 1: Chronbach’s α = .714; Day 2: Chronbach’s α = .823). Additionally, we calculated separate values for Training and Consolidation. Training was defined as training-induced change to perceptual performance, and Consolidation was defined as the overnight change in performance. Internal reliability for these metrics ranged from good to acceptable (Training: Chronbach’s α = .737; Consolidation: Chronbach’s α = .846) See Table 1 for a descriptive summary of scores.

Table 1.

Descriptive summary of performance

		Means(SD)
Reading
Untimed (raw)	Real	41.93(2.31)
	Nonword	22.92(2.41)
Timed (raw)	Real	93.81(9.83)
	Nonword	57.92(7.15)
Average (scaled)	Real	.89(.05)
	Nonword	.88(.09)
Perception of the nonnative contrast
Discrimination (d’)	baseline	.78(.68)
	Day 1	1.49(.81)
	Day 2	1.68(1.05)
Identification (d’)	Day 1	1.99(1.06)
	Day 2	2.48(1.37)
Average (d’)	Day1	1.57(.87)
	Day 2	2.06(1.14)
Changes to perceptual ability
Training (Day1-baseline d’)		.79(.78)
Consolidation (Day 2 - Training)		1.26(.94)

Means and standard deviations of performance across study measures. Untimed reading is expressed in raw scores obtained on the WRMT-III (Woodcock, 2011) and timed reading in raw scores obtained on the TOWRE-2 (Torgesen, et al., 2012).

Reading and changes to speech-perceptual ability

In order to determine if reading was associated with the initial ability to encode speech information during training, we employed a mixed effects model on reading performance as the outcome measure, with word Type (word/nonword), Training, and an interaction between word Type and Training, as predictors, age as covariate, with participant modeled as a random factor. Models were fitted using the lme4 package (Bates, et al., 2015) in R (R Development Core Team; Version 3.3.1, 2016). A likelihood ratio test (lmtest package; Zeileis & Hothorn, 2002) was used in a backward-fitting procedure in order to determine the model of best fit. This procedure resulted in an intercept-only model, with none of the fixed effects significantly predicting reading performance.

To address whether reading was associated with overnight consolidation of speech, we employed a mixed effects model on reading performance, with word Type (word/nonword), Consolidation, and an interaction between word Type and Consolidation, as predictors, age as a covariate, and participant modeled as a random factor. In following the same backward-fitting procedures described above, the accepted model (AIC=−363.5, BIC =−345.6, LogLikelihood = 187.8, r²_m = .068, r²_c=.504) included a significant interaction between word Type and Consolidation (β=−.02, SE=.01, p=.041), and effect of word Type (β=−.03, SE=.01, p=.018).

In order to examine the source of the interaction, we regressed the word and nonword reading scores by Consolidation separately. After Bonferroni correction, a positive relationship was observed between nonword reading and Consolidation (F(1,71)=7.60, p=.007, r²=.10). Real word reading was not associated with Consolidation (F(1,71)=2.17, p=.145, r²=.03). A follow-up testing of the word Type main effect yielded no significant effect of word Type on its own (F(1,71)=.76, p=.39, r²<.01), which is consistent with what we observed while fitting a linear mixed effects model for Training effects (see above). See Figure 1 for the pattern of results.

Figure 1.

Predictive values of offline Consolidation of speech on Reading by word Type

Graphical depiction of the variance explained in scaled word-level reading ability by Consolidation of speech information, that is, offline changes to post-training performance on the perception of a nonnative contrast, in real words and nonwords. Error bars indicate 95% CI.

Identification training involved mapping visual labels to speech, and thus may recruit similar processes to decoding. Thus, it is possible that the association between consolidation and decoding ability may be attributable to similarities between decoding and the demands of the identification task, rather than changes to perception. In this case, we would find that the association between consolidation and decoding would be driven primarily by changes to identification performance. To test this, we regressed decoding ability with overnight changes in identification and discrimination scores separately. After Bonferroni correction, we found that changes to discrimination (F(1,71)=7.74, p=.007, r²=.10), but not identification (F(1,71)=.22, p=.640, r²<.01), significantly accounted for variance in decoding. This supports the interpretation that decoding ability is associated with overnight changes to speech-perceptual sensitivity.

Reading and speech-perceptual ability

In order to rule out the possibility that the relationship between consolidation and decoding ability reflects differences in the ability to perform speech-perceptual tasks, we conducted correlational analyses to determine if perceptual ability at any time point (baseline, post-tests on Day 1 and Day 2) was associated with word-level ability. There was a relationship between baseline discrimination and nonword reading ability (p=.03), however this association was not significant after Bonferroni correction (see Table 2).

Table 2.

Correlation matrix across reading and perceptual performance (n=73)

Discrimination		Word	Nonword
baseline	Pearson’s r	0.16	0.26
	p-value	0.16	0.03
Day 1	Pearson’s r	0.07	0.06
	p-value	0.19	0.08
Day 2	Pearson’s r	0.15	0.21
	p-value	0.18	0.16
Identification
Day 1	Pearson’s r	0.16	0.17
	p-value	0.43	0.1
Day 2	Pearson’s r	0.09	0.2
	p-value	0.93	0.48

Strengths of correlations are stated in Pearson’s R values. After applying Bonferroni correction for multiple comparisons (p=.005 for an alpha of .05), none of the relationships are statistically significant.

Mediation by pre-existing phonological processing ability

Differences in phonological awareness has previously been observed to inform differences in the timing of overnight consolidation of artificial grammar (Zion et al., 2019). Thus, given the age and skill of the readers, it is possible that the observed relationship between consolidation of speech and decoding skills may be attributable to differences in phonological processing. We therefore conducted a mediation analysis on the observed relationship between Consolidation and Nonword reading ability, with Phonological Processing (scores obtained on the Elision subtest of the CTOPP-2, Wagner et al., 1999) as a mediator. We used the Lavaan package (Rosseel, 2012) in R, and the statistical significance of the regression paths were tested using bootstrapping procedures (10,000 iterations). As expected, the standardized path between Consolidation and Nonword reading was significant (EST=.025, SE = .011, Z=2.276, p=.023, 95%CI =[.003, .046]), as were each of the regressions in the indirect pathway (Phonological Processing~Consolidation: EST=.766, SE = .389, Z=1.970, p=.049, 95%CI =[.004, 1.527]; Nonword reading~Phonological Processing: EST=.016, SE = .003, Z=4.831, p<.001, 95%CI =[.009, .022]; see Figure 2). The indirect pathway was not statistically significant (EST = .012, SE=.007, Z=1.824, p=.068, 95%CI=[−.001, .025]), although these values suggest a marginal mediation effect. Taken together, it appears that Phonological Processing is associated with both Consolidation and decoding ability. Moreover, Phonological Processing appears to only partially mediate the relationship between Consolidation and Nonword Reading. Thus, Consolidation appears to account for variance in Nonword reading that is not wholly attributable to differences in Phonological Processing.

Figure 2.

The relationship between Nonword reading ability and Consolidation, mediated by Phonological Processing ability

Standardized regression coefficients for the relationship between Consolidation of nonnative acoustic-phonetic information and Nonword reading, as mediated by Phonological Processing. Note: the indirect effect is not statistically significant. *indicates statistical significance at .05 level. ***indicates significance at .001 level.

Discussion

We found that offline changes in post-training perceptual ability were associated with reading ability, specifically in decoding. This relationship between consolidation of speech information and decoding ability does not appear to reflect one’s ability to perform speech-perception tasks, nor was this relationship wholly mediated by phonological awareness. Taken together, these findings suggest that individual differences in memory processes that support speech representations are associated with decoding ability in adulthood.

The finding just one night of consolidation of acoustic-phonetic information relates to decoding ability in skilled, adult readers may be surprising to some. However, we note that overnight sleep duration and quality has been suggested to be stable across years within individuals (Hoch et al., 1997; Gains et al., 2015). Thus, our measure of consolidation over one overnight interval may serve as a crude index for differences in the habitual consolidation of acoustic-phonetic information. Speech/acoustic information that we encounter during daily language use may be similarly integrated with our phonological representations via overnight consolidation, and be thus subject to the same variability. This possibility is broadly consistent with claims that speech representations are dynamic entities, subject to updates throughout the lifespan (Kraljic & Samuel, 2005; Norris, et al., 2003).

This interpretation raises additional questions regarding potential problems in domain-general memory consolidation in those with reading disorders such as developmental dyslexia, and its potential effects on their reading skills. An emerging literature suggests that sleep architecture, and also offline memory consolidation of both linguistic and non-linguistic information, are atypical in individuals with reading disorders (Bruni et al., 2009; Sengottuvel et al., 2020; Smith et al., 2018). The current work offers a potential bridge between these findings with the phonological deficit consistently documented in this population. This work may also have implications for the relationship between socio-economic factors and habitual sleep quality (Patel et al., 2010), which over time could have effects on phonological processing, reading, and other linguistic skills.

An important limitation to acknowledge in the current study is that we lack information regarding the particular contribution of sleep, and sleep quality, to this variability in offline consolidation. Also, our findings do not examine the implied relationship between consolidation of speech information and the quality of phonological representations directly, although the association between consolidation and phonological processing ability is consistent with this interpretation (see Earle & Arthur, 2017, for a similar argument). We also note the gender imbalance in our sample as a potential limitation, and the need for future replication with a balanced sample. Finally, as we examined these relationships in skilled adult readers, the current findings do not inform how these processes occur in disordered populations.

In summary, this study identified a relationship between consolidation of speech information and reading ability in skilled, adult readers. This raises important questions surrounding the potential contributions of habitual memory processes to disordered reading, and warrant further investigation.