Towards a theory of individual differences in statistical learning

Abstract

In recent years, statistical learning (SL) research has seen a growing interest in tracking individual performance in SL tasks, mainly as a predictor of linguistic abilities. We review studies from this line of research and outline three presuppositions underlying the experimental approach they employ: (i) that SL is a unified theoretical construct; (ii) that current SL tasks are interchangeable, and equally valid for assessing SL ability; and (iii) that performance in the standard forced-choice test in the task is a good proxy of SL ability. We argue that these three critical presuppositions are subject to a number of theoretical and empirical issues. First, SL shows patterns of modality- and informational-specificity, suggesting that SL cannot be treated as a unified construct. Second, different SL tasks may tap into separate sub-components of SL that are not necessarily interchangeable. Third, the commonly used forced-choice tests in most SL tasks are subject to inherent limitations and confounds. As a first step, we offer a methodological approach that explicitly spells out a potential set of different SL dimensions, allowing for better transparency in choosing a specific SL task as a predictor of a given linguistic outcome. We then offer possible methodological solutions for better tracking and measuring SL ability. Taken together, these discussions provide a novel theoretical and methodological approach for assessing individual differences in SL, with clear testable predictions.

This article is part of the themed issue ‘New frontiers for statistical learning in the cognitive sciences’.

1. Introduction

Over the past two decades, extensive research has focused on statistical learning (SL), demonstrating sensitivity to complex distributional properties in the input. Starting from the seminal work of Saffran and co-workers [1], numerous studies have shown that humans display remarkable sensitivity to distributional regularities in the auditory [2], visual [3], and tactile [4] modalities, with verbal [5] or non-verbal [6] stimuli, comprising adjacent or non-adjacent [7] dependencies, over both time and space [8], even without overt attention [9] and from a very young age [10]. Sensitivity to the input's statistical structure has become an important theoretical construct in explaining a wide range of human capacities such as language learning, perception, categorization, segmentation, transfer and generalization (see [11] for discussion).

Although all of the above studies focused on demonstrating that a given sample of participants shows evidence of learning the distributional properties of a sensory input, recent years have seen a growing interest in tracking individual performance in SL tasks. This line of study is relatively new. Its initial motivation was to confirm the theoretical link between SL and language acquisition. However, more generally, the study of individual differences holds the promise of providing critical insights regarding the mechanisms of SL and could enable more powerful studies ([11–13]; see also [14]). Note that ‘individual differences’ in the context of SL can in principle refer to any quantitative or qualitative differences between individual learners (i.e. differences in both the extent and the speed/trajectory of learning, individual variation in the sensitivity to multiple statistics within the same input, etc.). Nevertheless, individual differences other than overall performance differences have to date rarely been investigated. We return to this issue further on, when considering the limitations of the currently used offline learning measures. For now, the important point is that these recent SL studies that tracked individual performance aimed to show that language learning relies, at least in part, on being sensitive to the statistical properties of a linguistic environment, and that individual variation in sensitivity to such regularities predicts linguistic abilities. Within this research programme, SL and artificial grammar learning (AGL) tasks were shown to correlate with literacy skills in a first language (L1) [15,16], literacy acquisition in a second language (L2) [17], comprehension of syntax [18], sentence processing [13,19,20], semantic and phonological lexical access [21], vocabulary development [22,23] and speech perception [24,25]. Conversely, other studies aimed to show that participants with language deficits, such as children with specific language impairment ([21,26], but see [27]), dyslexic readers [28,29] and agrammatic aphasia patients [30] display poor SL abilities.

This research is characterized by a prototypical experimental approach. First, a SL or AGL task that has been shown to produce above chance performance at the group level is selected, and imported into the study as is or with minor modifications. Typically, the tasks involve a visual or an auditory familiarization stream (representing an artificial grammar or a stream comprising set of transitional probabilities (TPs)), which is followed by a test phase. Second, individual performance in the task is registered for each participant (often the number of correct two-alternative forced-choice (2AFC) decisions in distinguishing presented visual or auditory sequences from foils at the test phase). Third, given the aim of the study (e.g. reading, syntactic processing, speech recognition, etc.), participants' capability in the respective linguistic domain is independently measured through well-established relevant language tests. Fourth, the participants' SL scores are used as predictors of their linguistic test performance. Table 1 presents a set of recent studies that followed this approach, including our own, along with the correlations they obtained.

Table 1.

Summary of recent individual differences studies predicting linguistic abilities from SL performance. RT, reaction time; SLI, specific language impairment; SRT, serial reaction time.

	SL task(s)	operational SL measure	linguistic measure	studied population	number of participants	obtained correlation
Arciuli & Simpson [15]	visual SL	success in 64 2AFC trials	reading skills (reading sub-test WRAT-4)	adults	37	r = 0.34
				6–12 year-old children	38	r = 0.33
Conway et al. [25]	visual AGL	difference in span between grammatical and ungrammatical sequences in test	speech perception in noise	adults	23	r = 0.46
	auditory AGL				22	r = 0.42
	visual AGL				64	r = 0.31
Frost et al. [17]	visual SL	success in 32 2AFC trials	learning scores in non-word decoding, word reading and morphological priming	adult L2 learners	27	r = 0.44 to 0.57
Kidd and Arciuli [18]	visual SL	success in 64 2AFC trials	auditory syntax comprehension task	6–8 year-old children	68	Pearson's r not reported. SL predicts comprehension of passives and relative clauses (logistic mixed-effect models)
Mainela-Arnold & Evans [21]	auditory SL	success in 2AFC test	gating task (lexical-phonological skills), word-definition task (lexical-semantic)	8–12 year-old children with SLI	20	r = 0.2 for both linguistic tasks
				8–12 year-old typically developing children	20	r = 0.28 (phonological); r = 0.1 (semantic)
Misyak & Christiansen [13]	two auditory AGL tasks: adjacent and non-adjacent	success in 2AFC test	comprehension of different types of grammatically complex sentences	adults	30	r = 0.11 to r = 0.49
Misyak et al. [31]	auditory non-adjacent AGL, combined with SRT	differences in the ability to predict the final non-adjacent dependent element after training	self-paced reading of sentences involving object relative clauses	adults	20	r = 0.59
Shafto et al. [22]	visual SL	RT difference of eye movements towards predictable stimuli between learning and test	early receptive vocabulary skills	7.5 month-old infants	58	r = 0.28
Spencer et al. [16]	auditory SL and visual AGL	success in four 2AFC test trials for SL; difference in span between grammatical and ungrammatical for AGL	a series of 10 tasks related to early literacy skills	4–10 year-old children	553	r ranging from 0 to 0.2

Although never explicitly specified, individual differences studies of this kind typically involve three critical preliminary presuppositions, which underlie the logic of this experimental strategy. First, as there is no agreed taxonomy of possible types of SL, it is treated by default as a unified theoretical construct, a general capacity for picking up regularities (with the exception of [13,31]; see, e.g. [32] for discussion). Second, and relatedly, the tasks that are selected for the study from the arsenal of tasks employed in this domain, are naturally assumed to equally represent a good operational proxy of this unified theoretical construct, so that the selection of one specific task for the study is not a matter of deep theoretical concerns (though see [13,31,33]).¹ Third, the performance score of the test phase in the task is naturally assumed to be a valid and reliable measure of the operational proxy, and therefore, a valid and reliable measure of the postulated ability for picking up regularities.

In the following, we will argue that these three critical presuppositions are subject to a number of both theoretical and empirical issues. Although previous studies of individual differences in SL have yielded important initial insights into how SL might be involved in various aspects of cognition, to get a deeper understanding of the extent and precise nature of these relationships we need to address these issues head on.

2. Is statistical learning a general unified capacity?

Most studies of SL do not provide an explicit computational account of learning but, rather, tend to adopt a more abstract notion of the underlying computations in the form of domain-general learning. Typically, the underlying computational system is assumed to be a ‘unified capacity’ instantiated by a unitary learning system that is applied across different modalities and domains. This may be a reasonable first approximation, given that the ability to extract statistical structure from the input is found across a wide range of stimuli as well as different domains, as reviewed above. Indeed, in the simple and abstract sense, there is something common to all these behavioural phenomena: registering regularities in the environment. However, advances in cognitive science require moving from abstract verbal theorizing to refined mechanistic computational theories. From this perspective, it seems that current empirical evidence suggests that the differences in computations across different SL phenomena largely outweigh their superficial abstract similarity.

(a). Modality specificity

Although SL has been demonstrated in all sensory and sensory-motor areas, current evidence systematically suggests qualitatively different patterns of performance in different modalities (see [11] for review). Importantly, tracking individual abilities in different SL tasks reveals significant reliability of capacity within modality, but zero correlation in performance across modalities [34]. Admittedly, one should be cautious drawing firm conclusions from a lack of correlations in a single study, especially given the relatively low reliability of some of the studied SL tasks (which limits the extent of expected correlations between SL measures, see [12,34]). Importantly, however, this result concurs with other findings showing qualitative differences in SL ability in the auditory, visual and tactile modalities [4,35], opposite effects of presentation parameters on visual versus auditory SL performance [36], lack of learning transfer across modalities (e.g. [37]) and interference in learning two artificial grammars within modality, but no interference across modalities [38]. This large body of evidence suggests that individual capacity of learning regularities differs across domains. This state of affairs should not come as a surprise. Recent imaging data suggest that in spite of the suggested role of the medial temporal lobe (MTL) memory system in SL (e.g. [39,40]), substantial SL computations occur already in the early visual and auditory cortices (e.g. [41,42]). The visual and auditory cortices involve different representations, and the set of computations characterizing these cortical areas is naturally constrained by the specific characteristics of the processed input. Thus, both the neurobiological and the behavioural evidence are inconsistent with the presupposition that SL is a unified capacity.

(b). Informational specificity

Although SL can be abstractly defined as ‘learning the statistical properties of the continuous sensory input’, from an informational perspective there are different kinds of ‘statistical properties’ that are the object of learning (see [43] for discussion; see also [44]). First, there is ample evidence that humans are sensitive to transitional statistics in continuous input, allowing them to detect even small changes in TPs [45].² Second, there is evidence that humans also aggregate information about the relative frequency of events (e.g. [50]), as well as their variance in the stream (e.g. [51]), showing sensitivity to distributional statistics. Cue-based statistics as revealed in spatial contextual cuing (e.g. [52]), or temporal cuing (e.g. [53]), are yet another form of learned regularities. In some cases, multiple cues either within or across modalities are needed to learn more complex probabilistic patterns [54]. As Thiessen et al. discuss in their expansive review [43], different kinds of statistical information do not necessarily implicate different sets of computations. Nevertheless, they argue that a complete account of SL must explain not only the learning of distributional statistics (i.e. the frequency and variance of exemplars) but also transitional statistics (i.e. learning the co-occurrences of elements in the stream).

Whether one or more kinds of computations are needed to cover the range of SL behaviours requires additional investigation, mainly through computational modelling, but also through correlational designs. For example, it has been suggested that learning non-adjacent contingencies follows specific constraints that do not exist while learning adjacent contingencies [7]. Indeed, supporting findings show that individual SL ability to learn adjacent contingencies is uncorrelated with their ability to learn non-adjacent contingencies even within modality [13,34,55].³

In sum, current empirical evidence is largely inconsistent with SL being a unified capacity involving a single set of computations. This has immediate implications for any correlational study aiming to tie specific cognitive abilities to SL. We suggest that such studies need to consider SL as a componential ability, requiring researchers to explicitly specify the theoretical link between the specific cognitive construct they investigate and its relation to the specific relevant SL computations.

3. Are all statistical learning tasks equally valid for assessing SL ability?

To date there are no agreed-upon constraints on which tasks should be selected as proxies for SL capacity. This is exemplified by the different tasks employed in correlational studies tying SL to other cognitive capacities, with often very little discussion regarding the theoretical logic governing the specific task selection (but see, e.g. [13], for such discussion). The problem with this state of affairs is twofold. First, without a clear understanding of the specific SL components that are being tapped by a given task, well-defined empirical predictions regarding its predictive validity cannot be generated. Second, understanding the relation between specific SL components and the proxies selected to tap them is necessary for integrating different findings, so as to make sense of the wide range of obtained results. In order to develop such integrative theory of the relations between SL computational components and linguistic capacities (as well as other cognitive capacities), we must first explicitly spell out the different components of SL capacity that, according to current evidence, is a multi-faceted construct.

One promising way to develop a theory regarding the inner structure of a complex construct is to define it in the form of a mapping sentence in line with Facet Theory, a systematic approach to theory development and data collection (e.g. [57,58]). In Facet Theory, the first and most important step in investigating a complex theoretical construct (in our case, SL) is to formulate a mapping sentence, which defines the full domain of the studied phenomena given existing data. A mapping sentence includes content facets that represent the different dimensions of the construct. It further outlines for each content facet a set of possible values (categorical or continuous) that could be relevant to the specific facet. This divides the full range of behavioural phenomena into theoretically distinct sub-types [58]. Importantly, one of the unique characteristics of Facet Theory is that it is taken to be a continuous effort of trial and error, where constructing a mapping sentence that outlines the various facets of a theoretical construct resembles an ongoing process of hypothesis testing and updating. An initial sentence is typically offered as a starting hypothesis (see [34]), and it is subsequently modified given novel empirical data regarding the inter-correlations between the suggested facets and their postulated values. Following this strategy, we define a preliminary mapping sentence below that concurs with a wide range of SL phenomena already reported in the literature, and outlines a potential set of different dimensions:

This suggested that mapping sentence offers then six preliminary content facets to account for SL phenomena.⁴ The first three facets, the type of statistics extracted (transitional versus distributional), the input modality (visual versus auditory)⁵ and the type of contingencies (adjacent versus non-adjacent), were included in the light of empirical evidence (reviewed in the previous section), and which have been suggested to involve non-overlapping sets of computations. Facets (4) and (5) are additional hypothetical dimensions that we offer to account for SL capacity, because they reflect ecologically separable phenomena: SL studies show that it occurs for both verbal and non-verbal material (e.g. [6]), and that statistical contingencies are extracted across both time and space (e.g. [8], though with different biases; see [35]). Admittedly, to date there is little unequivocal evidence showing that these phenomena are governed by non-overlapping computations and necessarily result in different learning constraints. Nevertheless, our recent investigation of SL capacities demonstrates no correlation in performance with verbal versus non-verbal stimuli within modality [34]. Similarly, no interference was found in learning two different sets of regularities at the same time, when they comprised verbal and non-verbal materials (non-words versus tones [38]).⁶ Indeed, recent neurobiological findings suggest that neural temporal coding is independent of the spatial dimension, and that specific time cells represent the flow of time (see [61]).

Importantly, from a theoretical perspective, including facets (4) and (5) in the mapping sentence has the advantage of shaping future investigation, so as to examine empirically the extent of their relative overlap and interaction. Facet number (6)—motor involvement—is yet another dimension that requires further investigation. Statistics of an input can be extracted without any motor involvement (such as in the case of most SL or AGL tasks). However, some SL tasks specifically involve active motor responses to stimuli (such as in the case of motor sequence SL, best exemplified by the serial reaction time (SRT) task, e.g. [62]). Whether such motor activity results in non-overlapping sets of computations in extracting statistical structure is then another open question awaiting future research (see for example [63] for a discussion).

Mapping sentences typically start small and grow bigger as empirical investigation progresses. Our initial proposed mapping sentence, therefore, does not preclude the possibility that other dimensions may be relevant for understanding SL ability. Possible additional candidate facets could be, for example, basic perceptual dimensions (colour, line orientation, etc.; e.g. [38]), full versus quasi-regularity (see [45] for discussion), implicit versus explicit learning settings (e.g. [64]), or, relatedly, unsupervised versus supervised learning settings (see [65] for a discussion of the role of feedback in perceptual category learning). An additional factor that was shown to affect SL performance is the rate of presentation—with opposite effects of both the inter stimulus interval and the actual stimulus duration on SL performance in the visual versus auditory modality ([35,36]; but see [66]). Whether rate of presentation constitutes a separate facet, or simply affects peripheral aspects to SL such as the encoding of individual elements, with different constraints in different modalities (see [11]), deserves further investigation.

Defining a mapping sentence as a working hypothesis for studying individual differences in SL enables theoretical discussions regarding how and why specific SL components modulate specific sub-components of other cognitive abilities, given their overlapping hypothesized computations. This makes the logic of choosing specific SL tasks for a given study more transparent, and allows a clear interpretation of the findings. For example, different components of linguistic phenomena most likely involve more than one type of underlying SL computations. Acquiring phonotactic constraints of a language requires registering both transitional and distributional statistics⁷ of phonemes in the speech stream via the auditory modality [67], while learning to read in L1 or L2 involves assimilating transitional statistics of letter sequences in the visual modality, but also aggregating systematic correlations between letters and sound, and between letter sequences and meaning through morphological form (see [68] for discussion). The mapping sentence above thus allows for more refined discussions of the components involved in each linguistic capacity and its relation to SL.

Importantly, a mapping sentence for SL not only dissects the outcome cognitive phenomena in terms of their different statistical computations, but also points to tasks that could (or should) be used to measure SL as predictors of a specific ability. To date, the arsenal of tasks tapping SL capacity is impressively varied: in addition to those reviewed in table 1, tasks, such as the SRT (e.g. [69]), contextual cuing (e.g. [70]), tone detection (e.g. [71]), or Hebb repetition task (e.g. [72]), are all considered to be proxies of SL, because they all involve learning statistical regularities. The advantage of a mapping sentence is that it provides a priori criteria for selecting one of the many available tasks for a given study, specifying the inter-relations between them. For example, in contrast with tasks such as visual SL or SRT that tap the extraction of transitional statistics, tasks such as contextual cuing require registering the distribution of stimuli to learn the repeated patterns, whereas tasks such as AGL involve both learning of units defined by transitional statistics (see, e.g. [73]), as well as their distributional statistics [43].

So far, we have advocated a research strategy that requires researchers to be very explicit about what specific computations are involved in a given SL task and their predicted outcomes. However, if the target of research is to assess the overall SL capacity of an individual as defined by the mapping sentence, as well as its predictive validity, the proposed mapping sentence provides specific guidelines for developing novel SL tasks to cover a wide range of SL components. Here, we propose that if SL is indeed a multi-faceted construct involving different types of computations with substantial non-overlapping variance, then this capacity should be measured and assessed by a variety of different tasks. Much like in the measurement of other complex constructs (e.g. the g factor measured by WAIS [74]), accurate estimation of multi-faceted constructs involves a large battery of tasks, each covering different parts of the variance. But note that in contrast with general intelligence, which has been mapped through decades of extensive research, the dimensions of SL as an individual ability are yet to be empirically established. Our mapping sentence attempts to offer a preliminary approximation of the possible facets of SL, serving as springboard for such research. At this point, we argue that current evidence points to SL as a multi-faceted individual ability. Selecting tasks as proxies for this ability thus requires an integrative approach with explicit discussions of the specific components that are being tapped.

4. Are standard task test scores a good proxy of statistical learning ability?

The vast majority of studies tracking individual differences in SL employ the same tasks that were originally designed for group-level studies. Here, the underlying assumption is that the outcome measure of performance in the task would serve as a good proxy or indicator of the theoretical construct: individual SL ability. We see two problems with this assumption. First, from a methodological perspective, although the typical SL tasks can reliably estimate the mean performance of the sample as a whole, they are often not sensitive enough to estimate a given individual's SL ability. Second, as we outline below, from a theoretical perspective, the structure of the tasks often intermixes outputs of different SL computations. This practice is likely to confound cognitive capacities that are orthogonal to SL, while also potentially lead to interference effects that mask the true capacity of SL.

(a). Psychometric weakness

A task that is suitable for measuring individual capacity must show substantial between-individual variance and this variance must be highly reliable. If not, the task cannot differentiate between good and bad learners, and cannot reliably predict other cognitive capacities. As we have recently argued [12], most SL tasks that have been used for group-level studies do not withstand psychometric scrutiny. This is owing to a number of shortcomings, such as insufficient number of test trials or the difficulty of the task, which result in a large part of the sample performing at chance, and the lack of variability in test item difficulty. Together, these psychometric weaknesses lead to tasks tapping mainly error variance rather than variance related to SL capacity (see [12] for extended discussion and possible solutions). Although this state of affairs did not hinder demonstrations of learning across a full sample of participants, it constitutes a formidable obstacle to individual differences studies.

(b). Structural confounds

At present, most SL tasks are based on a passive familiarization phase, in which stimuli representing a set of regularities are presented to participants (e.g. a continuous stream of shapes or syllables organized in pairs or triplets in visual and auditory SL, a sequence of ‘grammatical’ sequences in AGL, etc.). Once the familiarization phase is over, it is followed by a test phase that estimates participants' learning of the statistical properties of the previously presented stream, typically through a series of 2AFC responses. We will refer to these measures as offline measures of performance, as they do not track the discovery of regularities from the stream while it unfolds, but attempt to assess the extent of learning once it is over.

The theoretical challenges that offline measures implicate are presented in figure 1, which outlines the components of individual performance in the classical visual SL (VSL) task (e.g. [3,17,36,75,76]; see [77] for a related approach).

Figure 1.

The factors contributing to SL task performance, as measured by standard offline measures. (Online version in colour.)

As an example, consider a common variant of the VSL task, in which 24 abstract shapes are organized into eight triplets. During a familiarization phase, these triplets are repeatedly presented in a continuous stream. The only source of information regarding the composition of the triplets in the stream lies in the TPs between the shapes in the sequence: TPs between shapes within a triplet is 1, whereas TPs of shapes between triplets is 1/7, for 8 triplets without immediate repetition of a triplet. Following familiarization, the test phase begins. It consists of a series of 2AFC trials, each contrasting one of the triplets presented during learning with a ‘foil’, a group of three shapes that never appeared together in the familiarization phase (TPs = 0). In each trial of the test, one foil and one triplet are presented, and participants are asked to decide which group of shapes appears more familiar, given the stream they have seen. The final score that represents SL individual ability is the number of correct responses in the test phase.

Figure 1 depicts a coarse-grained account of possible factors and processes underlying the final observed performance in the task. On the left side of the figure (in blue), we describe the processes involved in the familiarization phase, while on the right side (in red) we list several additional factors affecting the outcome of the test phase. Considering the computation of regularities, we note that participants first have to perceive and encode the individual elements of the stream (factor A). As individuals substantially differ in the resolution of their perceptual system, their differential ability to generate perceptual representations under specific exposure constraints, would inevitably contribute to the variance of performance in the subsequent test phase (see also [11,45]). While encoding the individual events, participants further have to discover the statistical regularities in the stream (factor B), which is, in our present context, the most central factor for SL research. These two factors in combination (A and B) may result in sufficient sensitivity to the statistical regularities to perform above chance in the test phase. In addition, some experiments include instructions or other demand characteristics that may lead participants to try to detect repeated patterns (here, triplets) and memorize them for later recollection. Although this type of strategy is not required for successful SL, such additional explicit memorization efforts will add yet another task component (factor C) with considerable individual differences (e.g. [78]; see [79] for discussion).

Critically, the underlying implicit assumption behind the use of offline measures is that the accurate signature of learning can be retroactively traced, so that the test score would reflect the two, possibly three factors (A, B and C) contributing to SL abilities, and these only. However, as we will argue, similar to many offline tests in other fields in cognitive science (e.g. [80]), the testing phase in SL tasks inevitably interferes with what was learnt during familiarization, obscuring the ability to accurately measure the net SL ability. Here, we label this the interference component (factor i). This leads us to the following operationalization of measuring performance in the VSL task, using offline measures:

By this formula, performance in the VSL task would be some function of the multiplication of the ability to encode shapes (A), the ability to encode their co-occurrences (B), and, in some cases, the ability to store the extracted triplets (or patterns) in memory (C), minus (i), the degree of test interference. We opted for multiplication of the first two/three factors A, B, C, rather than simple additivity, because zero ability in any of the components (inability to encode shapes, inability to extract regularities, or inability to store items in memory) would inevitably result in zero learning.

Note that this operationalization applies not only to the VSL task, but can be generalized (with some obvious modifications given the exact task design) to other SL tasks involving offline measurements. It enables us, however, to explicate the critical shortcomings of this method to assess SL capacity.

5. Shortcomings of current offline measures

The first problem that arises is that performance in offline tests intermixes encoding efficiency, learning statistical regularities and possibly individual memory constraints. As the offline test is administered only after the learning phase in which those processes occur, it cannot differentiate between the relative contribution of these factors to the final learning score. Naturally, one could dismiss this caveat arguing that SL capacity inherently reflects the joint contribution of these components. However, in terms of predictive validity, in order to theoretically tie SL performance to specific cognitive abilities, knowing where exactly a potential weakness lies (encoding, learning or explicit memorization) is crucial. This is especially critical for an explanatory theory regarding how SL results in specific cognitive impairments, such as SLI, dyslexia, etc.

The second problem is that offline measures, being set at the end of learning, do not provide any information regarding the learning dynamics across time. As no data are collected during the learning phase, offline measures simply miss a large part of the action (this is a key part of the motivation for the AGL-SRT task developed by [20,31]; see also [81] for an individual-differences SRT study). Learning dynamics are important for two reasons. First, they provide insights regarding the speed of learning (i.e. how fast a given individual is in picking-up the statistical properties of the input), in addition to the extent of his/her learning (i.e. how much of the underlying structure did he/she learn in a pre-defined time period?) From a theoretical perspective, both speed and degree of learning are useful markers of a participant's SL ability. In addition, learning dynamics can provide valuable insights regarding the shape of the learning trajectory—for example, it can be used to examine whether knowledge is acquired gradually (reflected by a linear/logarithmic learning trajectory), or whether learning is characterized by a sudden burst in performance (i.e. step function).

The third problem is that the post hoc nature of the offline test inevitably introduces testing interference and confounds. For example, to allow for sufficient test items and to improve the resolution of performance scores, patterns and foils are typically repeated throughout the test phase several times. These repetitions effects interfere with learning, thereby blurring the methodological separation between intended learning during familiarization and unintended learning that occurs during the test phase.⁸ It is impossible to know whether responses reflect information acquired during learning or of overriding information presented by the repeated test items (see [12] for discussion).

6. The promise of online measures

The main motivation for using online measures is to track learning throughout the familiarization phase as it unfolds, which alleviates most of the caveats introduced by offline measures. As such, online measures of SL carry the promise of better resolution on multiple levels. First, from a theoretical perspective, they can differentiate cognitive processes that relate to the perceptual encoding of input elements and the learning of their distributional properties, from processes that use this information during a subsequent test. This makes it possible to identify the contribution of each of these components to SL performance. Second, online measures provide information regarding learning dynamics, reflecting how fast each individual learns the statistical properties of a stream, as well as indicating his/her learning trajectory. Third, by gathering a maximal amount of information (by tapping the full learning session) and by avoiding the interference introduced by the test phase, online measures have the promise of higher ‘psychometric resolution’, resulting in more reliable measurements.

Operationally, we define online measures as examining participants' responses throughout the learning process. A typical example is the classic SRT task, where implicit learning of a repeated sequence of digits is monitored. The online measure, the time taken to press a given key corresponding to a given digit, reflects the underlying assumption that faster motor responses are expected for predicted sequences compared with random ones. As predicted events result in faster responses, the trajectory of learning can be traced in this task.

These principles, however, can be easily applied to classical SL tasks. Consider for example the above VSL task. A simple modification can be introduced into the task to yield useful online information (see [83] for an action-sequence version, and [84] for visual AGL). Rather than asking participants to passively watch a stream of visual shapes, which appeared on the screen at a fixed rate of presentation, they are asked to advance the stream of shapes by themselves, at their own pace, by pressing the spacebar (much like in the self-paced reading paradigm [85]). The assumption is that learning the TPs between shapes in the triplets will result in faster bar pressing for predicted shapes (second and third shapes of the triplet) relative to unpredicted shapes (the first shape of each triplet). This makes it possible to track the detailed time-course of learning. RT differences between predicted and unpredicted stimuli have also been demonstrated in other tasks in auditory [86] and audio-visual SL [20,31,32,56]. Importantly, online measures of SL have been found to correlate with sentence processing in L1 [20,31], providing preliminary evidence regarding its predictive validity.

But note that the development of online measures of SL still requires extensive research. First, it is yet to be shown whether the existing online measures of learning provide reliable measures of individual performance, because no studies to date have examined the reliability of such measures (see by contrast, the reliability coefficients of offline measures recently reported in [34]). Second, existing studies present mixed reports regarding the correlations between online and the standard offline measures of SL (high correlations reported in [84], but zero correlations reported in [20,86,87]). Low correlations between offline and online measures in the same task could reflect theoretical issues (e.g. tapping explicit versus implicit knowledge [88], or tapping different components of SL variance [20]). However, such a state of affairs might also be owing to an inherent low reliability of online measures, either because they are unstable or inaccurate. A third issue in the development of online measures is that some online tasks may actually contaminate learning—for example, it was shown that in the SL click-detection paradigm (first proposed in [86]), the mere presence of clicks in the familiarization stream hinders learning owing to its taxation on attentional resources [87]. These issues need to be resolved by further research if the promise of the higher-resolution online measures is to be realized in future SL studies.

7. Concluding remarks

The theoretical interest in SL originally emerged as a potential domain-general alternative to domain-specific approaches to language. Rather than assuming an innate and modular human capacity for processing linguistic information, SL was offered as a general mechanism for learning and processing any type of sensory input. In line with this view, individual performance in SL tasks was systematically shown to correlate with an array of linguistic abilities. Here, we have suggested that further advances in this research enterprise require a deep mechanistic understanding of the precise interrelationship(s) between linguistic performance and SL ability, where SL as a theoretical construct is unpacked, no longer treated as a unified ‘black-box’ entity. On this view, empirical and modelling work should provide a priori hypotheses regarding the set of computations that underlie the learning of specific statistical regularities, within different types of input, in different modalities, taking into account their neurobiological constraints. This will allow for clear and testable fine-grained predictions that tie particular linguistic (and potentially other cognitive) abilities to specific SL computations. In the same vein, different experimental tasks impose different constraints on learning, thereby implicating different learning mechanisms. Transparent discussions regarding the specific computations involved in each SL task, its relations to other SL paradigms and the strategies that learners might use to learn a given statistical structure are necessary for establishing the theoretical link between performance in the task and the cognitive function it is supposed to predict. On the methodological level, such finer-grained hypotheses would call for more refined measures of SL that track SL performance more directly, providing a richer set of data regarding the processes involved in SL. In line with these aims, this paper offers a preliminary taxonomy of SL phenomena and outlines methodological guidelines that can serve such future research.

Authors' contributions

All authors contributed to the writing of this paper.

Competing interests

We declare we have no competing interests.

Funding

This article was supported by the Israel Science Foundation (Grant No. 217/14, awarded to R.F.), by the National Institute of Child Health and Human Development (Grant Nos. RO1 HD 067364, awarded to Ken Pugh and R.F., and PO1-HD 01994, awarded to Haskins Laboratories), and by the ERC (project 692502, awarded to R.F.). L.B. is a research fellow of the Fyssen Foundation.