Paired Associate Learning Tasks and their Contribution to Reading Skills

Abstract

Associative learning has been identified as one of several non-linguistic processes involved in reading acquisition. However, it has not been established whether it is an independent process that contributes to reading performance on its own or whether it is a process that is embedded in other linguistic skills (e.g., phonological awareness or phonological memory) and, therefore, contributing to reading performance indirectly. Research has shown that performance on tasks assessing associative learning, e.g., paired-associate learning (PAL) tasks, is lower in children with specific reading difficulties compared to typical readers. We explored the differential associations of two distinct verbal-visual PAL tasks (the Bala Bbala Graphogame, BBG, and a Foreign Language Learning Task, FLLT) with reading skills (word reading and pseudo-word decoding), controlling for phonological awareness, rapid naming, and letter and digit span in children at risk for reading disabilities and their typically developing peers. Our study sample consisted of 110 children living in rural Zambia, ranging in age from 7 to 18 years old (48.1% female). Multivariate analyses of covariance were used to explore the group differences in reading performance. Repeated-measures ANCOVA was used to examine children’s learning across the PAL tasks. The differential relationships between both PAL tasks and reading performance were explored via structural equation modeling. The main result was that the children at risk for reading difficulties had lower performance on both PAL tasks. The BBG was a significant predictor for both word reading and pseudo-word decoding, whereas the FLLT—only for word reading. Performance on the FLLT partially mediated the association between phonological awareness and word reading. These results illustrate the partial independence of associative learning from other reading-related skills; the specifics of this relationship vary based on the type of PAL task administered.

Phonological awareness, rapid sequential naming and letter knowledge have been acknowledged as the most important predictors of reading skills across different orthographies (Kirby, Parrila, & Pfeiffer, 2003; Lyytinen et al., 2004; Swanson, Trainin, Necoechea, & Hammill, 2003; Ziegler et al., 2010). Yet, along with these skills, cognitive learning mechanisms not specifically related to linguistic processing appear to mediate reading acquisition (Hulme, Goetz, Gooch, Adams, & Snowling, 2007; Windfuhr & Snowling, 2001). Specifically, non-linguistic mechanisms of associative learning may explain how reading is learned both implicitly and explicitly (Nicolson & Fawcett, 1999). Establishing the connections between written (grapheme) and spoken (phoneme) units is in fact the core learning activity of reading acquisition (Richardson & Lyytinen, 2014). In transparent writing systems, such as Chitonga, the language of the children in our sample, these units are learned explicitly in the school context, however they may also be learned implicitly through exposure to written language in the immediate environment. In the present study, we sought to elucidate the role of associative learning in understanding the development of children’s reading-related skills.

In the process of learning to read, two general types of learning are involved— implicit and explicit. Explicit (or acquisition-conscious) learning is what generally occurs in the classroom when children start learning to read, such as when teachers directly match featured letters to their corresponding sounds. However, passive exposure to corresponding sound and letter sequences may also occur, resulting in associated orthographical and phonological representations that have been implicitly acquired and become part of an automatized procedure in the decoding process (Snowling, 1980). Children with reading difficulties exhibit lower performance on certain types of implicit learning tasks (Folia et al., 2008; Laasonen et al., 2014; Vicari et al., 2005). Thus, associative learning has been studied as a predictor of reading skills (Muter, Hulme, Snowling, & Stevenson, 2004).

Specifically, paired associate learning (PAL) tasks have been used to explore the types of learning involved in reading acquisition. PAL tasks involve learning and remembering the associations between stimuli that are artificially associated (e.g., abstract figures with pseudowords). Findings from recent studies suggest that associative learning may predict reading skills independently from other linguistic processes, especially in children with specific reading disabilities (Li, Shu, McBride-Chang, Liu, & Xue, 2009; Warmington & Hulme, 2012). This implies that associative learning supports reading acquisition by building on the associations between symbols and sounds independently from other language skills. However, other studies claim that the poor performance on PAL tasks of children with specific reading disabilities is more related to phonological deficits or the verbal demands of the PAL tasks than associative learning itself (Litt & Nation, 2014; Litt, de Jong, van Bergen, & Nation, 2013).

The stimuli used in PAL tasks can be uni-modal (e.g., visual stimulus–visual paired associate, verbal stimulus–verbal paired associate) or cross-modal (e.g., visual stimulus–verbal paired associate, and vice versa) in nature (Litt et al., 2013). The process of learning to read can be defined as a form of cross-modal associative learning, involving the association of phonemes (verbal stimuli) with graphemes (visual paired associate). In contrast to processes of implicit learning, PAL tasks involve systematically pairing printed letters of the alphabet with verbally expressed sounds (a cross-modal, visual-verbal pair). Learning these cross-modal associations fosters the development of the alphabetic principle (i.e., the systematic correspondences of sounds and letters), which is a strong predictor of reading skills (Hulme et al., 2007; Muter et al., 2004; Schatschneider, Fletcher, Francis, Carlson, & Foorman, 2004).

Several studies have shown that associative learning is significantly correlated with specific reading difficulties (Messbauer & de Jong, 2003; Wimmer, Mayringer, & Landerl, 1998; Windfuhr & Snowling, 2001), particularly when the PAL tasks involve verbal stimuli (Hulme, 1981). Poor performance on verbal PAL tasks by children with reading difficulties has been reported across several languages that vary in their orthographic, phonological and morphological complexity (Li et al., 2009; Litt & Nation, 2014; Mayringer & Wimmer, 2000; Messbauer & de Jong, 2003). In studies utilizing both uni-modal (verbal-verbal; visual –visual) and cross-modal stimuli (verbal-visual; visual-verbal), Hulme and colleagues (2007) and Litt and colleagues (2013) attempted to determine which aspects of associative learning are more related to reading skills. Hulme and colleagues (2007) reported that the correlations between PAL tasks and reading were strongest for visualverbal tasks. Specifically, only the visual-verbal mappings were significant predictors of word reading and irregular word reading; however, when the visual-verbal mappings were abstract figures and non-words, they did not predict non-word reading. Similarly, Litt and colleagues (2013) explored four PAL mapping conditions—visual–verbal, verbal–verbal, visual–visual, and verbal visual—across reading skills. They found that only the tasks requiring verbal output (visual–verbal and verbal–verbal) were significantly correlated with reading skills. Several studies have used a variety of stimuli, for example, animal pictures with nonsense words (Wimmer et al., 1998), and complex names and pseudo-names with pictures of children (Mayringer & Wimmer, 2000). Results have been consistent with the view that learning visual (orthography) to phonological mappings is important for developing word recognition skills in reading, and that individual differences in this ability can be tapped experimentally by a PAL task (Windfuhr & Snowling, 2001). These results have also indicated that different stimuli may modulate the relationships between these tasks and reading skills, and account for the differences in performance between children with and without reading difficulties.

Using PAL tasks in rural Zambia

Children growing up in rural Zambia generally learn to speak one or more native languages, depending upon their home region (Sousa, Greenop, & Fry, 2010). Once they reach school, however, they become English language learners through the Zambian school system. Children generally begin school in Zambia when they are around seven years of age. However, they may start school at a younger or older age and/or experience grade repetition or time away from their studies (e.g., because of chores at home, care of younger siblings). Also, many children will not have experienced preschool education as it is not required by law (in 2005, 19.2% of Zambian first-graders in Southern Province had preschool experience, and this included children from urban and sub-urban areas; Republic of Zambia Ministry of Education (2006)). Thus, children in rural Zambia show large variability in their language skills in both their mother tongue and in English.

Understanding children’s low performance on reading tasks is challenging due to the various ages at which children may start school, the continuity of their time in school, and the varied quality of Zambia’s public schools. In such a context, PAL tasks may help to identify some of the sources of underperformance on reading-related tasks. Moreover, PAL tasks may differentiate children who could be at risk for specific reading disabilities (henceforth “+SRD”) as identified by reading assessments, from those who may simply be experiencing poor learning environments (i.e., indicated by average performance on PAL tasks).

The use of distinct PAL tasks may provide different types of information on the nature of children’s learning processes. The two PAL tasks used in this study differed in their stimuli: one presented Braille letters and graphemes, the other familiar objects and foreign words (essentially pseudowords). Given the nature of the two PAL tasks (abstract vs. concrete visuals, phonemes vs. pseudowords) we expected the tasks to show different associations with the reading outcomes in this study (word reading and pseudoword decoding). Specifically, four aspects were explored in this study: 1) differences in performance on the PAL tasks between children identified as being at risk and children not at risk for SRD; 2) the learning process as captured by children’s change in performance across the two PAL tasks (BBG and FLLT); 3) the role of the two PAL tasks in predicting skills in word reading (WR) and pseudo-word decoding (PW); and 4) the mediating role of the PAL tasks in the relationships between PA and PW and WR. Given previously conducted studies, we expected to find individual differences in performance across the BBG and the FLLT. We also expected to find group differences between the children who were at risk for SRD and typically developing children in their performance on WR and PW.

Method

Participants

The participants of this study were drawn from a larger research endeavor, the Bala Bbala Project, a large-scale epidemiological study of the risk factors for SRD conducted in a rural farming community of Southern Province, Zambia (Reich, Tan, Hart, Thuma, & Grigorenko, 2013; Tan, Reich, Hart, Thuma, & Grigorenko, 2014). Participating schools were located in a selected district and at each school, a random sample of students in grades 3–7 was chosen to be screened for SRD using measures of alphabet knowledge/reading recognition (RR; Stemler et al., 2009) and phonological awareness (PA; Reich et al., 2013). Subsequently, the children were identified as being without risk for specific reading disabilities (−SRD) when their performance on each task was at or above the 75^th percentile. Children at risk for specific reading disabilities (+SRD) were identified by performance at or below the 25^th percentile. Cutoff percentiles were determined following those used in others studies that classified children according to their performance in reading skills (Catts & Weismer, 2006; Lesaux, Geva, Dressler, & Kamil, 2008). At the time of screening, measures of general cognitive ability, hearing and vision tests, and nutritional status were also collected (cf., Hein, Reich, Thuma, & Grigorenko, in press).

The overall sample (all groups, i.e., −SRD, +SRD and students with medium levels of performance) was used to investigate individual differences in learning patterns across the two PAL tasks. Inclusion in the study required that students had completed both PAL tasks. A total of 480 students were assessed using the BBG. These students represent only a subsample of the larger research project, partially because the data from the BBG relied on the availability of electricity, a scarce resource in this rural context. The exclusion criteria for this subsample were: a) health issues such as malnutrition, visual impairments, or hearing impairments (these were considered distal factors that could affect performance on the assessments, n = 14); and b) evidence of technical problems in the implementation of the BBG. Some children had trouble understanding the operation of the mouse or the game itself. Due to these (and other) factors, the children did not progress through enough items to learn the grapheme-phoneme associations. That is, children were excluded from this study if they experienced fewer than three encounters per target letter (i.e., less than nine trials in a block) (n = 129); and c) 227 children were excluded as their average number of responses was at the level of chance (i.e., a mean less than or equal to 0.33, given that there are three choices offered in each item of the BBG). Responses by chance, which appeared to occur due to a number of factors, will be discussed later, but the mechanism by which the stimulus was presented and the mechanism of learning involved in this task might have influenced the guessing and random responses, especially because the children were often not familiar with computers. The final sample consisted of 110 children from grades 3 to 7 (59 male, mean age = 13.08 years, SD = 2.25). Based on the cutoffs of the 75^th and 25^th percentiles, 30 children (26.1%) were classified as −SRD (14 male, 16 female) and 18 (15.7%) children as +SRD (9 male, 9 female). The children in the −SRD and +SRD groups were similar in age. Table 1 shows the demographic information of the sample by gender, age, and grade.

Table 1.

Descriptives for age, gender and grade of the final sample of the study.

		Age				Gender		Grade
	N	M	SD	Min	Max	Female	Male	M	SD	Range
Total Sample	110	13.1	2.3	8	18.5	51	59	4.9	1.5	3–7
−SRD	30	13.4	1.6	10	16.7	16	14	5.4	0.9	3–7
+SRD	18	14.3	1.9	10	18.5	9	9	4.8	1.1	3–7

Note. −SRD = group without risk for reading disabilities, +SRD = group at risk for reading disabilities.

Measures

Phonological Awareness (PA)

This assessment presents a total of 61 items from eight subtests. In Initial Sound Matching, the participant is given a word and three choices from which the child is asked to select the word that begins with the same sound as the given word (Cronbach’s α = .809). In Final Sound Matching, the child is given a word and three choices, from which the child is asked to select the word that ends with the same sound as the given word. (Cronbach’s α = .792). In Rhyming, three words are given and the child is asked to indicate which two words rhyme (Cronbach’s α = .655). In Blending Syllables, the child is given a word segmented into syllables and asked to say it as one single word (Cronbach’s α = .871). In Segmenting into Syllables, the child is given a single word and asked to separate it into syllables (Cronbach’s α = .928). In Elision, the child is given a word and asked to produce the same word with one sound missing (Cronbach’s α = .887). All stimuli are verbal. For all of these subtests, each correct response was scored as 1, and incorrect responses as 0. The sum of all correct items was used as a total score for further analyses. All estimates of internal consistency (Cronbach’s α) reported in this section were obtained using the sample in this study.

Rapid Automatized Naming (RAN)

In this task, children are asked to name as fast as possible series of repeating familiar stimuli presented in an array consisting of four rows and 9 columns (Denckla & Rudel, 1976). There were two different charts for each stimulus: letters, digits, pictures of familiar objects, and colors. Children’s responses were timed using a stopwatch.

Letter-Digit Span (LD-Span)

This task was adapted from the Wechsler Intelligence Scales for Children, Fourth Edition, WISC-IV (Wechsler, 2003) for use in Chitonga. Each of the four subtests (Letters Forward, Letters Backward, Numbers Forward, and Numbers Backward) includes 16 items. For each item, children are asked to repeat sets of numbers or letters either forward or backward, depending on the subtest. All stimuli and responses are verbal. Each subtest employs a stop rule of four consecutive incorrect responses (Cronbach’s α = .897).

Zambian Achievement Test (ZAT)

The ZAT assesses reading-related and mathematics skills based on the Zambian school curriculum. It is administered individually. This set of assessments was originally developed in Chinyanja (Stemler et al., 2009). The Chitonga translations were piloted with small groups of students to determine item appropriateness and difficulty, and the subtests were accordingly adjusted. Reading Recognition (RR). The version used in this study included 39 multiple-choice items. The objective was to assess children’s pre-reading skills, alphabet knowledge and phonological awareness. All items are presented on paper and the instructions read out loud by the data collector. For example, in one item, children are shown a letter and then asked to find the same letter among a set of four letters. Assessment is discontinued after eight consecutive incorrect responses. The total score is the sum of correct responses (Cronbach’s α = .892). Word Reading (WR). This task consists of 44 words that the child has to read aloud. The words increase in difficulty, i.e., in word/syllable length as well as complexity of sound combinations. There is no time limit. After four consecutive incorrect responses the test is discontinued (Cronbach’s α = .995). Pseudoword Decoding (PW). This test consists of 38 pseudowords with phonetically regular construction. The first item consists of simple vowel–consonant combinations (e.g., ig, ak); subsequent items become progressively more challenging in their length and phonetic construction. The student must read the pseudowords aloud (Cronbach’s α = .998).

Foreign Language Learning Task (FLLT)

The FLLT task assesses how well a child can learn the correspondences between common pictures and words in a foreign language (Spanish, which is very different from Chitonga and completely unknown to our sample population). Our version of the FLLT was adapted from previous work (Baddeley et al., 1995; Jukes et al., 2002) to present pictures that would be familiar to children in rural Zambia. The objective of the FLLT is to assess children’s ability to learn associations between pictures of objects and their corresponding names in Spanish. The task requires the child to remember the Spanish names of objects, given by the examiner, and to show this by pointing to the corresponding object when the examiner says the Spanish word. The task is comprised of four charts presenting pictures of common objects. Each chart represents a level of learning. In each new level, four additional pictures are added to the previous set of pictures. The first level has 4 pictures. The second level has 8 pictures, those from the first level and four new pictures. The third level has a total of 12 pictures, eight from the previous level and four new pictures. Finally the fourth level has a total of 16 pictures; 12 from the previous level and four new pictures. In each level, the examiner points to and names the pictures in a preset order; this learning phase is presented only once in each level. If the child gives an incorrect response, the examiner points to the correct response without naming it again, and if the child gives a correct response the examiner gives positive feedback. Children are given 8 trials to complete the task. The total score is the average number of correct answers across the eight trials. See Krivulskaya et al. (revision submitted) for more details about the task and other ways of modeling trial-by-trial learning trajectories (Cronbach’s α = .946).

The Bala Bbala GraphoGame (BBG)

A different approach to measuring PAL was developed by modifying the so-called ‘GraphoGame,’ a computerized game (Lyytinen et al., 2006) whose objective is to train children in letter–sound correspondence as a main skill to prevent future reading problems (Hintikka, Aro, & Lyytinen, 2005; Lyytinen, Erskine, Kujala, Ojanen, & Richardson, 2009). While the GraphoGame was originally intended to teach literacy skills using the alphabet, it was adapted into a PAL task as a form of dynamic assessment (Reini, 2010). In the assessment version designed for this study, the amount of time spent by each child was on average 2.13 minutes for each block. The objective of the BBG is to assess the ability of the child to learn associations between visual and auditory stimuli, analogous to the learning of phoneme-grapheme correspondences. The BBG uses the basic GraphoGame learning task: several moving balls with visual symbols (letters) are shown at the top of the screen and the player is asked to choose the one that matches the auditory sound (phoneme) before the balls fall to the bottom of the screen. The task is administered in two phases: training and main assessment. A screen shot of the game is shown in Figure 1. Training Phase. This phase has two main objectives: to familiarize the children with how to use the computer mouse and to teach them how to respond to the stimuli by choosing the target object on the screen using the mouse. In the practice task, the sound /a/ is heard via headphones and the player sees a single falling ball with the letter A on the screen. After the player has successfully selected the “A” ball in three trials, a distractor ball with letter X is also shown on the screen. After successfully choosing the A ball three times, from between the A and X balls, the final practice stage is choosing the A ball when there are two distractor X balls on the screen. Only after this training has been completed may the children begin the main assessment. Main assessment. The main assessment is comprised of two stimuli sets with unfamiliar visual symbols (from the Braille alphabet) and unfamiliar speech sounds (from Finnish and Arabic). The task has six blocks. The first three blocks present Finnish vowels (Ä [ae], Ö [oe] and Y [yː]) as auditory stimuli with corresponding letters in Braille. The second three blocks present Arabic consonants (nūn [n], dāl [d] and rāʾ[r]) as auditory stimuli and corresponding Braille symbols. The symbol-sound correspondences for the Braille letters are reported in Table 2. In each block, children completed a variable number of trials. The number of trials is based on the performance of the child in the task. Each trial is composed of three Braille letters as visual stimuli and one auditory stimulus. Only one of the Braille letters is the target (matching the auditory stimulus) and the other two Braille letters are distractors. The trial starts with a sound being presented to the child (via headphones) and three balls appearing on the screen; each ball contains a Braille letter. The balls begin to descend from the top of the screening, indicating to the child that there is a limited time to respond. The children respond by clicking or touching (if using a touchscreen) one of the falling balls. If the child does not respond before the ball touches the ground, the trial is considered missing and a new trial is presented. If the child gives an incorrect response, visual feedback indicating the correct response is provided and a new trial is presented. Positive feedback is given by the software when an answer is correct, and a new trial is presented. The progression of the six blocks is regulated by three rules: a) time; the child may play only for three minutes on each block, after that a new block was started; b) correct response; if the child gets three correct responses in a row for each of the target grapheme-phonemes, the child progresses to the next block; and c) quantity of trials; if the child does not meet criteria (b) within three minutes or after 100 trials, the block stops and a new block begins. Less than 8% of the children met rule (b), and none met the rule for the three blocks (c). The score for this game was computed as an average of the ratio of correct responses to the number of times the target letter was shown in each block (e.g., correct responses for the pair Ä [ae] and L (three dots) divided by the number of times that this pair was presented)(Cronbach’s α = .615).

Figure 1.

Example screen of a BBG variation. In the screen the three falling balls are shown with the pattern of dots inside (from the Braille alphabet).

Table 2.

Sound and symbol stimuli used in the BBG

Sound	Dots Symbols
Ö [oe]
Ä [ae]
Y [yː]
Nūn [n]
Dād [dˁ]
Rāʼ [r]

Note. Sounds as expressed using the International Phonetic Alphabet are reported in brackets. In the second column, alphabetic letters and their corresponding braille symbols are shown.

Universal Non-verbal Intelligence Test: Symbolic Memory (UNIT-SM) (Bracken & McCallum, 1998)

The Symbolic Memory subtest of the UNIT is an assessment of memory with 30 items. For each item, participants first look at an array of line drawings of human figures; the figures represent different ages (i.e., baby, child, adult), genders, and colors (black, green). The array of figures is then covered and the participant is asked to use tile versions of the same images to recreate the array from memory. UNIT-SM is discontinued after seven consecutive incorrect responses or instances of non-response (Cronbach’s α = .801).

Kaufman Assessment Battery for Children, Second Edition (KABC-II-T)

The Triangles subtest of the KABC-II-T (Kaufman, 2004) assesses simultaneous visual processing and has 27 items. For each item, the participants are asked to use physical shapes, mostly triangles, to re-create given pictures. Some of the harder items have time limits. The assessment is discontinued after five consecutive incorrect or non-responses (Cronbach’s α = .830).

Procedure

The Institutional Review Boards of the University of Zambia and Yale University approved the consenting and data collection procedures used in this study. The data collectors were trained to administer the assessments to individual children. An initial verbal consent from the parents was obtained for the screening. After that, individual informed consent for all participating children was obtained from parents or guardians and, in addition, children themselves assented to participate in the study. The physical screening information (i.e., assessment of hearing, vision, height, and weight) was collected using portable audiometers, two tumbling ‘E’ charts, meter sticks, and weight scales. The paper and pencil tasks were administered individually. All measures were translated and adapted into Chitonga, and were administered in Chitonga by data collectors, over the course of several sessions at the school or close to the communities where the children lived. The computerized BBG was administered under the supervision of at least two data collectors.

Data Analyses

First, given that the BBG is a computerized PAL task being used for the first time in this particular context, a preliminary analysis of the data was conducted to explore possible differences between the final sample and the excluded sample based on the criteria of correct responses or quantity of trials. The final sample (n = 110) was comprised of children who initially had more than 33% correct responses in the set of three blocks presenting vowels, as well as the set of three blocks presenting consonants. This was done to secure a sample that performed significantly above chance on both target types. In the preliminary analyses, the final and excluded samples were compared to ascertain whether they differed in age and performance on the reading measures (using an analysis of variance), in gender and grade (using chi-square tests), and in their performance on the BBG (using independent samples t-tests) when increasing the threshold stepwise (increments of 1%) from 33% of correct responses. A threshold above 35% would have resulted in significant differences in performance on the BBG between the final and the excluded sample, thus this threshold was retained as the criterion to select the final sample.

Second, two main analyses were conducted to address the objectives of the study. The first objective was to examine the differences between the children identified as +SRD and −SRD on the PAL tasks. These differences were looked at using multivariate analyses of covariance (MANCOVA). The second objective was to explore the learning process across the blocks and trials. Repeated-measures ANCOVA (controlling for age) were conducted separately for the BBG and FLLT. For the BBG, complementary repeated-measures ANCOVA were conducted to explore the change in performance in the blocks in which vowels were the target separately from those in which consonants were the target.

The third objective was to examine the relationships between both PAL tasks and WR and PW, controlling for PA, RAN and LD-Span. Two models were specified and the direct effects of PA, LD-Span, RAN, the BBG and FLLT on WR (model 1) and PW (model 2) were estimated separately. Next, to look at the mediating role of the PAL tasks, two more models were constructed. PA, RAN and LD-Span were used as independent variables in a model, predicting performance on the BBG and the FLLT, which in turn were modeled to predict WR (model 3) and PW (model 4). The scores of all tasks were regressed on age and standardized residuals were used as the observed variables in the model. A composite score for IQ was computed using principal component analysis based on the KABC-II-T and UNIT scores. The factor scores were used as observed variables in the model to regress all of the reading measure on IQ. In all analyses, except for the analyses of learning across blocks, we used blocks 2 and 3 (vowels), and 5 and 6 (consonants) to compute the total score of the BBG. This decision was made based on the high levels of guessing and chance responses in the first block of vowels (block 1) and the first block of consonants (block 4). Path analysis utilizes full-information maximum likelihood parameter estimation to account appropriately for missing data (Nevitt & Hancock, 2001); there was less than 1% missing data from the total sample for some of the reading measures. Multiple fit indices were used to evaluate the goodness-of-fit of the model: the χ² and Bollen-Stine bootstrap p values of .05 or greater; a ratio of chi-square divided by degree of freedom values less than 3; a Standardized Root Mean Square Residual (SRMR) ≤ .80; a Comparative Fit Index (CFI) ≥ .95; and a Root Mean Square Error of Approximation (RMSEA) ≤ .06 indicate a good fit of the model to the data (Byrne, 2006). The bootstrapping method (with 1000 replications) was used to estimate the parameters and associated standard errors. The significance of the parameters was estimated using the critical ratio test and the bias-corrected bootstrap confidence intervals were used to test the significance of total, direct and indirect effects (Cheung & Lau, 2008). The mediation analysis assessed the following associations: (1) the contributions of the independent variables PA, RAN and LD-Span to WR and PW (total effect); (2) the contributions of the independent variables to the mediators, the BBG and FLLT, and the contributions of the mediators to WR and PW (indirect effect); and (3) the contributions of the independent variables to WR and PW in the presence of the BBG and FLLT (direct effect) (Hayes, 2009).

Results

The results are organized into three parts. First, we present the analyses of individual differences in children’s performance on the BBG and FLLT for the final sample (n = 110). Second, the differences between +SRD and −SRD children are explored, along with their trajectories of learning in both PAL tasks. Third, we display the results of our analyses of the role of the PAL tasks as independent predictive variables of reading skills and as mediators for the effect of PA on reading skills.

Preliminary analysis

Given the number of children excluded, we examined whether the final sample and the excluded subsample differed in any measure that might bias further analyses. With respect to the reading measures, the final sample and the excluded sample were similar in their performance on all assessments except for LD-Span [selected sample: M = 20.40, SD = 5.84; exclude sample M = 18.03, SD = 6.56; F(1,354) = 1.35, p = .005, Cohen’s d = .32], indicating that children in the final sample showed better verbal memory skills. Age, grade and gender distributions were similar in both samples. Thus, both the included and excluded samples differed almost exclusively in the consistency of their patterns of response on the BBG.

Differences in performance on the PAL tasks between +SRD and −SRD groups

The performance of +SRD children (n = 18) was low on all of the measures compared to −SRD children (n = 30). Table 3 shows the descriptive statistics by group and task. The greatest differences and largest effect sizes between the groups were found for PA (η²_ρ = .904) and RR (η²_ρ = .923), which was expected given that the children were classified according to their performance on these tasks. We also found a large effect size in the differences between both groups in WR (η²_p = .579), PW (η²_p= .550), RAN (η²_p= .417), LD-Span (η²_p= .435), UNIT-SM (η²_p = .414), and KABC-II-T (η²_p = .311); large effect sizes were also observed for the associative learning tasks, BBG (η²_p = .221) and FLLT (η²_p = .311).

Table 3.

Bivariate correlations for the final sample and descriptive statistics for the +SRD, −SRD and final sample.

		PA	RR	RAN	LD Span	PW	WR	KABC-II-T	UNIT-SM	FLLT	BBG
	PA	1
	RR	.784**	1
	RAN	−.638**	−.593**	1
	LD-Span	.658**	.582**	−.538**	1
	PW	.728**	.651**	−.534**	.600**	1
	WR	.807**	.668**	−.580**	.543**	.859**	1
	KABC-II-T	.405**	.432**	−.297**	.456**	.338**	..356**	1
	UNIT-SM	.363**	.571**	−.460**	.328**	.366**	.424**	.382**	1
	FLLT	.572**	.530**	−.362**	.393**	.468**	.598**	.217**	.308**	1
	BBG	.436**	.413**	−.288*	.363**	.489**	.495**	.377**	.271**	.360**	1
		PA	RR	RAN	LD-Span	PW	WR	KABC-II-T	UNIT-SM	FLLT	BBG
+SRD	M	19.7	20.4	46.9	15.4	1.0	1.1	11.6	5.8	3.4	.44
	SD	7.9	4.6	12.1	4.9	1.1	4.0	4.8	3.3	1.2	.08
−SRD	M	51.0	37.2	30.4	24.3	20.4	31.8	19.0	10.5	7.2	.55
	SD	5.2	1.5	6.3	4.7	11.0	6.8	5.1	3.9	3.2	.13
Final sample	M	35.5	29.4	38.3	20.4	8.9	12.8	14.6	8.0	5.3	.78
	SD	14.0	7.4	10.9	5.8	11.8	18.3	5.6	3.9	2.7	1.12

Note. The correlations in the top part of the table are partial correlations controlled by age. PA = Phonological Awareness; PW = Pseudoword Decoding; WR = Word Reading; RR = Reading Recognition; RAN = Rapid Automatized Naming; LD-Span = Letter-Digit Spans; FLLT = Foreign Language Learning Task; BBG = Bala Bbala GraphoGame; UNIT-SM = Universal Non-verbal Intelligence Test, Symbolic Memory. KABC-II-T = Kaufman Assessment Battery for Children, Triangles.

^**p <.001,

^*p <.05.

The learning process mapped by the PAL tasks for +SRD and −SRD groups

The PAL tasks required the children to learn associations between visual and verbal stimuli, which is the reason we expected to see improvement in children’s performance across the blocks (in the BBG) and trials (in the FLLT). Given that this analysis was carried out to describe the children’s performance across the blocks, the six blocks of the BBG were considered, however in all other analyses only blocks 2 and 3 and 5 and 6 were considered. Using the six blocks of the BBG, results indicated that students, on average, performed similarly across the blocks [Wilks’s λ= .918, F (5,104) = 1,853, p = .109, η²_p = .082]. Figure 2 shows the average percentage of correct responses for each block. In block four, a non-significant decline in the trend was observed; this is coincident with the change in auditory stimuli from Finnish to Arabic. Table 4 shows the descriptive statistics and the item correlations between the blocks.

Figure 2.

Percentage of correct responses on the BBG by block for the total sample, −SRD and +SRD groups. Note: The bars correspond to the standard errors.

Table 4.

Correlations between blocks and descriptive statistics for the BBG.

	Block 1	Block 2	Block 3	Block 4	Block 5	Block 6
Block 1	1	.463**	.467**	−.120	.007	.082
Block 2	.461**	1	.599**	−.029	.074	.065
Block 3	.457**	.608**	1	.062	.187*	.229**
Block 4	−.118	−.027	.073	1	.232**	.222**
Block 5	−.016	.066	.143	.243**	1	.390**
Block 6	.070	.059	.205**	.228**	.376**	1
	M	SD	Min	Max	Skewness	Kurtosis
Block 1	.46	.15	.15	.83	.470	−.191
Block 2	.50	.17	.19	.94	.752	.258
Block3	.49	.17	.09	.94	.426	−.299
Block 4	.41	.13	.11	.74	.225	−.120
Block 5	.45	.15	.13	.90	.521	.414
Block 6	.46	.17	.11	.94	.770	.366

Note. Correlations above the diagonal represent zero-order correlations. Correlations below the diagonal represent partial correlations controlling for age.

^**p <.001,

^*p <. 005.

For the FLLT, the repeated-measures ANCOVAs again showed no differences in performance across trials [Wilks’s λ= .959, F (2,102) = .622, p = .736, η²_p = .041]. Figure 3 shows the average number of correct responses on each trial. Tables 5a and 5b show the descriptive statistics for each trial. It is important to note that differences were not possible to detect due to the low power associated with the small sample size, and also because, due to the variability of the sample on age, these analyses were controlled for age.

Figure 3.

Mean of correct responses on the FLLT by trial for the total sample, −SRD and +SRD groups. Note: The bars correspond to the standard errors.

Table 5a.

Descriptives for the FLLT by trial.

	M	SD	Min	Max	Skewness	Kurtosis
Trial 1	2.50	1.20	0	4	−.535	− 579
Trial 2	3.41	1.88	1	8	1.064	.768
Trial 3	4.43	2.70	0	12	1.058	.766
Trial 4	5.25	2.94	0	16	1.107	1.223
Trial 5	5.98	3.48	1	16	.773	.033
Trial 6	6.53	3.89	0	16	.632	−.230
Trial 7	6.90	4.17	1	16	.700	−.203
Trial 8	7.50	4.28	1	16	.569	−.485

Table 5b.

Correlations between trials for the FLLT.

	Trial 1	Trial 2	Trial 3	Trial 4	Trial 5	Trial 6	Trial 7	Trial 8
Trial 1	1	.530	.487	.472	.506	.533	.481	.483
Trial 2	.514	1	.834	.765	.805	.717	.672	.643
Trial 3	.470	829	1	.798	.851	.752	.719	.691
Trial 4	.452	.758	.792	1	.880	.853	.792	.774
Trial 5	.483	.799	.847	.876	1	.871	.819	.779
Trial 6	.508	.708	.745	.848	.865	1	.911	.891
Trial 7	.453	.662	.710	.784	.809	.904	1	.912
Trial 8	.456	.632	.681	.766	.767	.885	.906	1

Note: Correlations above the diagonal represent zero-order correlations. Correlations below the diagonal represent partial correlations controlling for age. All correlations were significant at p <.001.

The contribution of the PAL tasks to reading skills

Two models were specified to determine the contributions of PA, RAN, LD-Span, and the BBG and FLLT to the variation in WR and PW. The two models presented adequate fit indices; the regression coefficients and fit indices are reported in Figure 4. Model 1 showed that PA, the BBG, and the FLLT were significant predictors for WR. In model 2, PA, LD-Span and the BBG were the only significant predictors of PW. The correlations between the independent variables in both models were significant for all of the measures in the range of r = .22 to r = .55 (ps < .05), except between RAN and the BBG (r = −.12). Medium correlations were observed between PA and LD-Span (r = .55, p = .002), and between PA and RAN (r = .53, p = .002). In these models, RAN was not significantly related to WR and PW over and above the contributions of PA, LD-Span, the BBG and FLLT. Also, the BBG accounted for a significant amount of variance in WR and PW (for WR r² = .17, p = .003, and for PW r² = .18, p = .003) while the FLLT accounted only for WR (r² = .085, p = .002). These models suggest that the PAL tasks explained part of the variance in reading performance when the other variables were included in the model. However, differences between the PAL tasks were observed: the BBG was a significant predictor of PW and WR, while the FLLT was a significant predictor only for WR.

Figure 4.

Models 1 and 2 regress WR and PW on PA, RAN and LD-Span, and the two PAL tasks respectively. All the variables were regressed by age and IQ was included in the model as control variable but is not represented in these diagrams. The fit indices for the initial and final model were reported. The bold path coefficients correspond to parameter estimates for the final model. The initial model include all the significant and non-significant paths. In the final model the non-significant path were removed. The dash lines indicate non-significant paths. In the paths of the models the standardized regression coefficients were reported. ** p <.001, *p<.05.

Models 3 and 4 explored the hypothesis that the PAL tasks mediate the relationships between PA, RAN and LD-Span, and PW and WR. The models and fit indices are reported in Figure 5. In model 3, only PA was a significant predictor of WR (β = .739, p < .001) without the PAL tasks in the model. When the PAL task were included, the direct effect of PA on WR was still significant (β = .585, p = .002), the direct effect of PA on the BBG and FLLT also were significant (β = .310, p = .037; and β = .551, p = .003, respectively), as was the direct effect of the BBG and FLLT on WR (β = .165, p = .007; and β = .196, p = .007, respectively). Finally, the indirect effects of PA on WR were tested separately (for the BBG: β = .036, p = ns; for the FLLT: β = .092, p = .024 respectively) and both PAL tasks (for the BBG and the FLLT β = .235, p = .004). These results show that the FLLT partially mediated the relationship between PA and WR, while the indirect effect of PA on WR via the BBG was non-significant. Also, a partial mediation was observed when both PAL tasks were considered together. The initial and final models with standardized parameter estimates are shown in Figures 4 and 5.

Figure 5.

Models 3 and 4 regress WR and PW on PA, RAN and LD-Span, and the two PAL tasks respectively. All the variables were regressed by age and IQ was included in the model as control variable but is not represented in these diagrams. The fit indices for the initial and final model were reported. The bold coefficients correspond to values for the final model. The initial model include all the significant and non-significant paths. In the final model the non-significant path were removed. The dash lines indicate non-significant paths. In the paths of the models the standardized regression coefficients were reported. ** p <.001, *p<.05.

The indirect effect was then explored for PW (model 4). The significant direct effects of PA and LD-Span on PW without the mediators were significant (β = .548, p < .001; and β = .197, p =.022, respectively). When the two PAL tasks were included as mediators, the direct effects of PA (β = .475, p < .001) and LD-Span (β = .117, p = .042) on PW were still significant. The direct effect of the BBG on PW was also significant (β = .197, p = .005), but the effect of the FLLT on PW was not (β = .094, p = ns). Given that only the BBG had a significant direct effect on PW, the mediation relationship was tested only for the BBG. The indirect effect of PA and LD-Span via the BBG on PW was not significant (β = .044, p = ns; and β = .012, p = ns, respectively). In this model, then, no mediating relationships were observed for the BBG. The final model with the standardized parameter estimates is shown in Figure 4.

Discussion

Overall, our results showed that tasks of associative learning may differentiate children with and without risk for specific reading disabilities, with those at risk performing lower than typically developing children in both of our verbal-visual PAL tasks, the BBG and FLLT. In addition, both tasks exhibited differential relationships with key components of reading—word reading and pseudoword decoding—suggesting that the types of stimuli used by PAL tasks are differentially related to phonological awareness. The discussion that follows is organized into three components: 1) the relationship between PAL tasks and reading skills; 2) individual differences in performance on both PAL tasks; and 3) the limitations of the study and future directions.

The relationship between PAL and reading skills

The low performance of the children classified as +SRD on the PAL tasks is consistent with what has been reported in the literature regarding studies in different orthographies (Hulme et al., 2007; Mayringer & Wimmer, 2000) and logographic systems (Li et al., 2009). Two different perspectives to explain the association between PAL tasks and reading skills have been discussed in the literature. The first perspective, from the framework of the phonological deficit hypothesis, suggests that children’s low performance on the PAL task might be related primarily to their phonological awareness skills, rather than cross-modal associate learning (Litt & Nation, 2014; Litt et al., 2013). It has been shown that the children with +SRD obtain lower scores the higher the verbal complexity of the stimulus presented; their performance tends to be similar to children not at risk for SRD when the stimuli are visual (Litt & Nation, 2014; Litt et al., 2013). Thus, from this perspective performance on the PAL task may be a manifestation of a deficit in phonological awareness rather than associative learning per se. From the second perspective, PAL tasks account for variation in reading above and beyond phonological awareness in +SRD children and typically developing readers as well (Hulme et al., 2007; Warmington & Hulme, 2012; Windfuhr & Snowling, 2001), thus positing that the cross-modal nature of paired associate learning (visual-verbal) is more closely related to the learning mechanisms of reading than phonological processing (Chow, 2014). The results from this study support the notion that the associative learning involved in PAL tasks is a distinct process from those skills, such as PA, that have traditionally been considered predictors of reading. This is the case for the BBG, which predicted the performance on WR and PW above and beyond the contributions of phonological awareness. In contrast, the FLLT predicted only WR when PA was part of the model.

Considering that teaching phonemic awareness is less common in Zambian schools, and that some Zambian teachers prefer to teach whole words than specific phonemes (Lupele, 2013), the BBG might have been an unusual and difficult task for these children, inasmuch as the children had to learn an artificial alphabet. In contrast, the stimuli presented in the FLLT task were more closely related to the children’s experiences of learning English as a second language, in which the child learns complete words and their meaning, but not necessarily the phonemic composition of the words. In this regard, teaching to read may involve more holistic strategies than phonemic strategies.

The usefulness of the BBG as a PAL task

A significant number of children performed at the level of chance on the BBG. This might be explained by two unanticipated factors: (1) the nature of learning in the BBG, and (2) the environmental conditions. Regarding the first point, we noted that in both the first block of vowels and the first block of consonants, the children seemed to learn by trial and error, as may be expected in the context of an assessment that looks like a game. In this process, children guess, actively remember what they learn, then inhibit any information that may not be relevant to learning the correct associations between the stimuli. In this case, working memory could be an important skill for success in associative learning in the presence of interference (Kast, Baschera, Gross, Jäncke, & Meyer, 2011). However, in this study, the association between the BBG and LD-Span, as a measure of working memory, was unexpectedly low. Also, although differences in LD-Span were observed between the final sample and the excluded sample, the children with low performance on LD-Span (excluded sample) were more likely to give answers at chance level in the BBG compared to the children with higher performance on LD-Span (final sample). Another explanation for children’s performance at the level of chance may be that the children simply needed more time and exposure to the stimuli to increase their performance. Moreover, most of the assessments in this rural setting face challenges such as the lack of electricity and factors in the environment that may have distracted the children from the computerized assessment.

Limitations and future directions

Limitations of this study may be listed in three categories: a) those limitations related to features of the sample; b) those related to the environment in which the assessment was administered; and c) the features of the BBG.

Regarding the first point, the selection of a subsample from an original sample may generate bias in the results that are not possible to estimate due to the limited number of measurements, however at least in the reading measured this does not seem to be the case. The sample of this study was broad in terms of the range of and the relationship between the PAL tasks and the reading skills, was obtained controlling for the age variable. This approach allowed to explore the relationship between these skills, however it is not possible to explore how this relationship will be presented across ages or different stages of the development.

In the second category, some limitations resulted from the challenges of collecting data from computerized tasks in a rural context (e.g., lack of electricity, effects of the heat and dust on the computers, and the need to conduct the assessments oftentimes outdoors).

Finally regarding the computerized features of the BBG, such tasks may require preliminary investigations and subsequent adaptation in order to better capture the process of learning that pertains both to the task per se as well as to learning about/on computers. As reported above, part of the sample was excluded because they performed at the level of chance. Many reasons could explain this: the novelty of the task, the difficulty of the task in comparison to the FLLT due to the type of stimuli, children’s lack of experience with the computers or the rules that were used in this version of the assessment. The BBG was one of the first attempts to design the GraphoGame as an assessment with dynamic and adaptive features. The BBG could be considered a dynamic assessment because of the feedback given on correct and incorrect responses, and the fact that it is possible to monitor the learning associated with the feedback. The BBG is adaptive in that the length of the blocks and the selection of the stimuli are based on the previous performance of the child. However, some aspects could be revised to improve this assessment. For example, it might be important to re-examine the possibility of teaching the target in the initial phase of the assessment, the role of random presentation of the target and the time spent on each block. These two features—restriction of playing time and the random presentation of the target—resulted in children not being exposed equally to the stimuli, making comparisons between children’s performance difficult. For future PAL studies, length of the training task, the number of stimuli and the stop rules need to be re-considered. Exploring these elements may enhance the ability of the BBG to assess associative learning processes related to learning reading skills. Considering these aspects, the BBG could be used as a tool for differentiating children with learning difficulties in reading from children who do not have enough experience in reading yet also present low performance in other reading skills. Because the BBG is in essence a dynamic assessment, it may differentiate children with SDR from those who show low performance on reading tasks due to poor school experience. Children who perform well on the BBG (i.e., exhibit associative learning) yet exhibit low reading skills may indicate a possible environmental effect, that is, poor schooling. Children who do not perform well on the BBG and exhibit low reading skills may be identified as having SRD. In addition, the BBG may provide a good approximation of the intensity of intervention needed for the remediation of these difficulties based on the quantity of items that a child needs to solve before establishing a specific association.

Highlights.

• The role of associative learning in reading performance was explored using two paired-associate tasks.

• Associative learning predicts reading performance above and beyond other reading-related processes.

• The type of stimuli used in the paired-associate tasks affects its association with reading performance.