Acquiring and Processing Verb Argument Structure: Distributional Learning in a Miniature Language

Elizabeth Wonnacott; Elissa L Newport; Michael K Tanenhaus

doi:10.1016/j.cogpsych.2007.04.002

. Author manuscript; available in PMC: 2008 May 29.

Published in final edited form as: Cogn Psychol. 2007 Jul 27;56(3):165–209. doi: 10.1016/j.cogpsych.2007.04.002

Acquiring and Processing Verb Argument Structure: Distributional Learning in a Miniature Language

Elizabeth Wonnacott ¹, Elissa L Newport ², Michael K Tanenhaus ²

PMCID: PMC2405816 NIHMSID: NIHMS45703 PMID: 17662707

Abstract

Adult knowledge of a language involves correctly balancing lexically-based and more language-general patterns. For example, verb-argument structures may sometimes readily generalize to new verbs, yet with particular verbs may resist generalization. From the perspective of acquisition, this creates significant learnability problems (Baker 1979), with some researchers claiming a crucial role for verb semantics in the determination of when generalization may and may not occur (Pinker, 1989). Similarly, there has been debate regarding how verb-specific and more generalized constraints interact in sentence processing (Trueswell et al 1993; Mitchell 1987) and on the role of semantics in this process (Hare et al 2003). The current work explores these issues using artificial language learning. In three experiments using languages without semantic cues to verb distribution, we demonstrate that learners can acquire both verb-specific and verb-general patterns, based on distributional information in the linguistic input regarding each of the verbs as well as across the language as a whole. As with natural languages, these factors are shown to affect production, judgments and real-time processing. We demonstrate that learners apply a rational procedure in determining their usage of these different input-statistics and conclude by suggesting that a Bayesian perspective on statistical learning may be an appropriate framework for capturing our findings.

Keywords: Language Acquisition, Sentence Processing, Verb Argument Structures, Eye-tracking, Artificial Language Learning

Introduction

Adult language incorporates both regular, abstract operations and patterns that are idiosyncratic or specific to particular lexical items. The complex interplay between these two types of process is particularly clear in the relationship between verbs and the argument structure constructions in which they may occur. For example, consider the use of the ditransitive structure in English. For many verbs this construction provides an alternative to the use of a prepositional form, as in (1):

(1) Jack gave/brought/threw/the ball to Henry.

Jack gave/brought/threw Henry the ball.

In addition, the construction may be spontaneously applied to new verbs. For example, Gropen, Pinker, Hollander, Goldberg, and Wilson (1989) demonstrated that children who were taught the new verb pilk with the meaning ‘transfer by car’ would produce such sentences as “he is pilking him the horse”. Yet despite this apparent productivity, certain verbs are unexpectedly ungrammatical in the ditransitive, as in *Jack donated/carried/pushed Henry the ball. This illustrates a phenomenon known as subcategorization: particular verbs are constrained (or ‘subcategorized’) as to the set of constructions with which they may occur (Chomsky, 1965). This combination of generalization and lexical restriction turns out to be common across many constructions in different languages (see Pinker, 1989, for a review of the Dative, Causative, Active-Passive and Locative alternations in English), yet poses a puzzle from the perspective of acquisition. If learners are able to extend verbs to new constructions, how do they learn that some new verb-construction combinations are ungrammatical, rather than simply absent from the particular sample of speech they have heard thus far? This constitutes a significant learning problem, sometimes known as “Baker’s Paradox,” which has received a great deal of attention in the language acquisition literature (e.g. Baker, 1979; Bowerman, 1988; Braine, 1971; Braine & Brooks, 1995; Brooks & Tomasello, 1999; Pinker, 1989; Theakston, 2004). Although it has been proposed that very young children may avoid the issue by adhering to extreme lexical conservatism (see Tomasello 2000; Fisher 2002a; Tomasello & Abbot-Smith 2002; Gertner, Fisher, & Eisengart, 2006 for review and discussion of the evidence), all researchers agree that, at least from around 3 years of age, children do generalize. When this occurs we also see over-generalization, where children use constructions with verbs for which they are ungrammatical (e.g. *Jay said me no, Gropen et al. 1989). In this article our focus is on how the retreat from overgeneralization can be achieved i.e. how learners who are able to generalize balance this ability with their knowledge of verb-specific constraints. One obvious source of potential evidence, direct correction from caregivers, has been shown to be very rare in the input to young children (Braine, 1971; Brown & Hanlon, 1970; Newport, Gleitman & Gleitman, 1977). Debate has therefore focused on two alternate sources of evidence: verb semantics and distributional information.

The semantics approach

One approach to a range of difficult problems in syntax acquisition has been to look for semantic or perceptual characteristics that correlate with the syntactic distributions and to propose that these play a significant role in acquisition (Grimshaw, 1981; Morgan & Demuth, 1996; Morgan & Newport, 1981; Pinker, 1984, 1989). This has sometimes been referred to as the ‘bootstrapping’ approach, implying that the semantic or perceptual properties are inherently more accessible than the distributional patterns themselves, and therefore might serve as a vehicle by which the distribution can be indirectly acquired. For the current problem, semantic factors would seem to be particularly promising, since there is a strong correlation between a verb’s meaning and the set of structures in which it may occur (Fisher, Gleitman, & Gleitman, 1991; Gleitman, 1990). Most fundamentally, the use of a particular argument structure imposes constraints on the number of arguments that must be associated with the verb. For example, causal events generally require structures with two NP slots, and transfer events require structures with three NP slots. Furthermore, the use of a particular structure may also have more subtle semantic connotations¹, for example the use of the ditransitive implies not only transfer but also transfer of possession (Green 1974; Goldberg, 1995; Jackendoff, 1972; Levin, 1993; Pinker, 1984, 1989).

Young children show an awareness of these correlations between meaning and structure in their usage of new verbs (Gropen, Pinker, Hollander & Goldberg, 1991a,b; Gropen, Pinker, Holander, Goldberg & Wilson, 1989), and many researchers agree that these semantic properties play some constraining role in the acquisition of verb distribution (e.g. Bowerman 1988; Braine & Brooks, 1995; Pinker, 1984,1989). Pinker (1989) takes the bootstrapping approach further, proposing that children acquire a complex system of semantically and morpho-phonologically defined ‘narrow’ verb classes which determine syntactic subcategorization. (For example, carry, push and lift belong to a class of verbs which do not occur in the di-transitive construction and which share the meaning ‘continuous parting of force in some manner causing accompanied motion’.) However, several researchers have pointed out that some of the postulated class criteria are inconsistent, so that they do not capture the full pattern of verb-structure co-occurrences (Bowerman 1988; Braine & Brooks, 1995; Goldberg 1995, chapter 5). The general conclusion is that, although there are strong correlations between the two, verb distribution cannot be reduced to verb semantics (or a combination of semantic and perceptual cues). Moreover, experimental evidence suggests that, even when the generalizations captured by Pinker’s classes are real and productive in the adult grammar, young children may only acquire this semantic knowledge relatively late in development (Ambridge, Pine, Rowland and Young, in press; Brooks & Tomasello, 1999).

More generally, a large body of research highlights the complexities involved in acquiring verb semantics at any developmental stage – specifically, the difficulty in extracting verb meanings from purely environmental contingencies (Gleitman, 1990; Gleitman, Cassidy, Nappa, Papafragou, & Trueswell, 2005; Gillette, Gleitman, Gleitman, & Lederer, 1999; Snedeker, 2000). In fact, Gleitman and colleagues have argued that the problem is so hard as to be intractable for many verbs. Instead, they propose that the acquisition of verb semantics itself relies on a process of distributional learning, i.e. that learners make use of information about the linguistic structures in which a verb has occurred to make inferences about its meaning – a process sometimes called ‘syntactic bootstrapping’ (Gleitman, 1990). One piece of evidence for this process is that young children are able to use the syntactic frame in which a new verb occurs to make inferences about its meaning. (Naigles, 1990; Fisher, 1996, 2002b).

This line of research clearly challenges any account in which prior learning of verb-semantics provides the crucial ‘bootstrap’ for acquiring verb-syntax. In addition, the syntactic bootstrapping literature provides a further motivation for the current work. Gleitman and colleagues have claimed that “the set of frames associated with single verbs provides convergent evidence as to their full expressive range” (Gleitman et al., 2005; see also Fisher et al., 1991). Thus an important component of that theory is that learners are able to acquire verb-structure co-occurrences independent of verb meaning. The current work investigates this learning process.

The statistical approach

An alternative to using semantic correlates to acquire verb distribution is to extract that information directly from the set of verb-structure combinations occurring in the input. Such a theory was first proposed by Braine (1971) and has recently gained in popularity (e.g. Braine & Brooks, 1995; Goldberg, 1995; 2007; Theakston, 2004; Tomasello 2000). Two potential learning processes have been discussed: pre-emption and the entrenchment.

Pre-emption refers to the evidence provided by encountering a verb in construction A when construction B would have provided the same communicative function. The hypothesis is that the pre-emption of B provides evidence that it is ungrammatical with that verb. (Related hypotheses have been propounded for many aspects of language acquisition, e.g. Markman (1989)’s principle of mutual exclusivity for word learning, Pinker (1984)’s uniqueness principle for morphology). Goldberg (1995) suggests that this process is aided by the fact that the form which is actually encountered will often be less felicitous than the form it preempts (given that no two constructions are entirely synonymous, Givon, 1985). For example, since the periphrastic causative is not generally associated with direct causation, the sentence ‘He made the rabbit disappear’ should be less felicitous than ‘*He disappeared the rabbit’. Thus encountering the former sentence in place of the latter could provide the child with evidence that disappear is ungrammatical in the transitive. There is evidence for the use of this information, but so far only in older children (4.5 year olds, Brooks & Tomasello, 1999; 6 year olds, Brooks & Zizak, 2002).

In contrast, entrenchment has been construed as blind to the semantic or pragmatic properties of verbs or constructions. The notion is that encountering a verb frequently in the input ‘entrenches’ its use with the particular constructions with which it has occurred, making it less likely to be generalized for use with a new construction (Braine & Brooks, 1995). A number of studies have found evidence for entrenchment in child language. Tomasello, Dodson and Lewis (1999) found that three year olds were more likely to produce over-generalizations with low frequency English verbs (*he arrived me to school) than high frequency equivalents (*he came me to school). Similarly, Theakston (2004) and Ambridge et al. (in press) found that five year olds were likely to give such over-generalizations higher grammaticality ratings when they involved low frequency verbs. Mathews, Lieven, Theakston and Tomasello (2005) explored the ability of 2;6 year olds to use known English verbs with a new construction introduced during the experiment (SOV word order), finding that they were more likely to produce such generalizations with low frequency than high frequency verbs.

Although there is some question as to whether entrenchment and pre-emption are really distinct (see Goldberg, 2005, for the argument that frequency effects reflect the verb’s more frequent occurrence in specific pre-empting constructions), these phenomena provide evidence that learners are sensitive to the frequencies of various combinations of verbs and structures occurring in the input and that they use this information to make inferences as to the status of ‘missing’ verb-structure pairs. This account concurs with a growing body of research demonstrating that learners come equipped with powerful statistical learning mechanisms. For example, Saffran, Newport and Aslin (1996; Saffran, Aslin & Newport, 1996) showed that adults and young infants could track the frequencies and conditionalized probabilities of syllable co-occurrence patterns in a speech stream and apply these statistics to the process of word segmentation; Mintz (2002) showed that adult learners could abstract syntactic categories on the basis of word co-occurrences. The current work explores whether statistical learning can be extended to the problem of verb argument structure acquisition. We also explore whether this same learning process can account for related phenomena in real-time sentence processing.

Verb argument structures in sentence processing

Verb subcategorization has also featured in a parallel literature on on-line sentence processing. Here the focus has largely been on statistical rather than absolute constraints (sometimes known as a verb’s subcategorization ‘profile’). For example, many verbs in English may be followed by a choice of complement structures but are constrained as to how readily they occur with these structures (for instance, find may occur with either a direct object or a sentential complement, as in Arthur found Trillian and Arthur found Trillian was in the car, but is more likely to occur with a direct object). In this literature too, debate has focused on role of this verb-specific knowledge versus more generalized patterns. One influential approach has proposed that real-time comprehension is primarily influenced by biases which operate above the level of individual verbs (Frazier, 1987; Frazier & Fodor, 1978; Frazier & Rayner, 1982). According to this theory, these biases arise from an inherent preference for syntactic simplicity (embodied in various parsing principles such as Minimal Attachment), which influences the structures assigned during real time parsing, irrespective of the particular lexical items involved. Although early experimental work appeared to support this hypothesis (Clifton, Frazier & Randall, 1983, Ferreira & Henderson, 1990; Mitchell, 1987), a large body of work now indicates that verb-specific biases have a strong and immediate influence in real-time processing (Garnsey, Pearlmutter, Meyers, & Lotocky, 1997; Snedeker & Trueswell, 2004; Trueswell & Kim, 1998; Trueswell, Tanenhaus, & Kello, 1993; but cf. Kennison 2001). For instance, Trueswell, Tanenhaus, & Kello (1993) investigated whether readers were sensitive to the likelihood of particular English verbs being followed by a either a direct object or a sentential complement. Participants’ eye-movements were monitored as they read pairs of sentences such as: ‘The chef found/claimed (that) the recipe would require using fresh basil’. Reading times suggested a bias to interpret the post-verbal NP (the recipe) as a direct object with direct object biased verbs like find, but not with sentential complement biased verbs like claim, indicating that subcategorization information was accessed and used to determine upcoming structure as soon as the verb was processed. Similar lexical effects have also been found in spoken language comprehension, with both adults and children (Snedeker & Trueswell, 2004; Trueswell, Sekerina, Hill & Logrip, 1999).

Although the influence of lexically-based, verb-specific biases is now well established, there is evidence that more abstract, verb-general biases also play a role in processing. In particular, it has been shown that post-verbal nouns are occasionally interpreted as direct objects, even with verbs which have never occurred with that type of complement (Mitchell, 1987; Juliano & Tanenhaus, 1993). This phenomenon suggests a verb-independent structural bias, which has been attributed to the inherent preferences of the parsing architecture (Frazier, 1987; Mitchell, 1987). However, Juliano and Tanenhaus (1993, 1994) argued that these effects occur primarily with low frequency verbs. This effect of frequency, akin to the process of entrenchment discussed in the developmental literature, would again appear to signature a statistical process. Juliano and Tanenhaus suggest that the bias may arise from the general preponderance of that type of complement structure across the verbs of the language. This hypothesis has been explored in a number of computational models (Juliano & Tanenhaus, 1994; Tabor, Juliano & Tanenhaus, 1997; Kim, Bangalore & Trueswell, 2002). Critically, these models track both verb-specific statistics (the likelihood of particular verb occurring in a particular structure) and verb-general statistics (the occurrence of different argument structures, across verbs in the language).

Despite the apparent success of these statistical accounts, it is not possible to conclude that even verb-specific biases are actually distributional in nature. A recent series of studies suggests an important role for verb semantics (Hare, McRae, & Elman, 2003, 2004). This work demonstrates that the subcategorization preferences which play a role in online processing are sense contingent. For example, for the verb find, its locate sense is subcategorized to occur only with a direct object, and processing is sensitive to this information (Hare et al., 2003). One interpretation of these results is that distributional analyses are performed not over particular lexical forms, but over particular senses of those forms. However, these findings at least raise the possibility that structural preferences may be entirely driven by verb-semantics. The strong correlations between verb distribution and verb semantics, which hold in each natural language, make it impossible to determine whether verb biases are a result of the verb’s own distributional history or of its membership in some more general semantic classes.

One method of avoiding the confounds inherent in natural language is to explore the learning of artificial languages, in which these factors can be disentangled. One previous study found evidence that learners could acquire and use probabilistic subcategorization patterns which were entirely distributional in nature. Wonnacott and Newport (2005) exposed learners to an artificial language in which all verbs could occur with either of two constructions, but occurred with one construction twice as often as the other. In contrast to natural language input, there were no semantic or structural reasons to prefer the use of any construction with any verb. Participants were then asked to produce their own sentences in the language. The central finding of this study was that the tendency to use each construction with any verb matched the probabilities of the input: participants used the dominant construction twice as often with each verb. In the experiments reported below, we extend this methodology to ask whether learners can acquire both verb-specific and verb-general distributional information, and how the distributional nature of the input influences learners’ usage of these different input statistics.

The current work

We explore the hypothesis that the subcategorization phenomena reported in the acquisition and processing literatures can be accounted for by statistical learning processes (cf. Saffran et al., 1996; Mintz, Newport & Bever, 2002; Thompson & Newport, 2007) -- that is, that learners track the occurrences and co-occurrences of verbs and structures in the input, and can use that information in a sophisticated way to make inferences about the underlying language system. Our approach suggests that the problem of learning when to restrict constructions to specific verbs, versus generalize their use, is part of a larger process of balancing verb-specific and verb-general statistical information.

The benefits of using artificial languages as a means of obtaining precise control over the input to learning is now well established (Aslin, Saffran & Newport, 1998; Braine, 1963; Hudson Kam & Newport, 2005; Gerken, 2006; Gomez, 2002; Mintz, 2002; Moeser & Bregman, 1972; Morgan & Newport, 1981; Morgan, Meier & Newport, 1987; Saffran et al., 1996; Wonnacott & Newport, 2005). In addition, there is emerging evidence that artificial languages exhibit many of the same signature results in processing as those obtained with natural language stimuli (e.g., Magnuson, Tanenhaus, Aslin & Dahan, 2003). Here the methodology allows us to create languages in which the relationship between verbs and potential argument structures is entirely distributional in nature (there are no structural reasons to prefer the use of any argument structure and no semantic or phonological correlates to verb behavior), and then to manipulate verb-structure co-occurrences across different artificial languages, to observe how different distributional patterns affect learning.

In order to explore not only whether learners acquire different distributional relationships, but also how they use that information, all of the experiments involved three different language tests: grammaticality judgment, production, and online comprehension. Although most artificial language experiments have relied on grammaticality judgments (Braine 1990; Gomez, 2002; Moeser & Bregman, 1972; Morgan et al., 1987; Morgan & Newport, 1981; Saffran et al., 1996), some studies have also included tests of production and online comprehension (production: Hudson-Kam & Newport, 2005; Wonnacott & Newport, 2005; online comprehension: Magnuson et al., 2003). Including all three measures in the same experiments will allow us to compare different modes of learning. As in Wonnacott and Newport 2005, the production test was set up to ascertain the tendency to produce each of the possible constructions with each of the verbs in the language. The function of the online comprehension test was to test the tendency to predict each of the constructions’ likelihood with different verbs. To that end, this test employed eye-tracking in the Visual World Paradigm (Altmann & Kamide, 1999; Cooper, 1974; Tanenhaus et al., 1995).

Each of the four languages used in these experiments had the same basic vocabulary of nouns and verbs, and involved the same two argument structures (Verb Agent Patient and Verb Patient Agent Particle). What was manipulated across the different languages was whether and how often different verbs occurred in each of the two structures, with no semantic or other cues to verb distribution in any language.

In Experiment 1 we investigate whether participants can acquire verb-specific constraints (i.e. learn that certain verbs can occur in only one of two competing constructions), even in the presence of a class of unconstrained ‘alternating’ verbs that occur in both constructions. We also ask whether, as in natural language learning and processing, these results are modulated by verb frequency. In Experiment 2 we ask whether we can tip the balance between acquiring verb-specific constraints and generalizing by manipulating the distribution of verb types across the language as a whole. We expose learners to a language containing a larger and more varied class of alternating verbs than in Experiment 1, asking whether this makes them more likely to generalize alternating frames to the constrained verbs. In Experiment 3, we expose learners to languages in which verb-specific and verb-general patterns are probabilistic and examine the influence of these statistics on the different language behaviors. We ask what happens when these different statistics are in conflict, and whether this conflict is affected by the overall distributional properties of the language.

Taken together, these experiments will allow us to determine whether distributional learning mechanisms are able to acquire the types of lexical constraints discussed in the acquisition and processing literatures, and also how the distributional details of the input influence the balance between applying lexical patterns and generalizing.

Experiment 1

The aim of Experiment 1 was to investigate whether participants could learn verb-specific construction constraints -- that is, to avoid over-generalizing with verbs that only occurred in one of two constructions when the language also contained a class of verbs that occurred in both constructions. To this end, we exposed learners to a language with three verb classes: a class of alternating verbs (occurring equally often in each of the two possible constructions) and two classes of one-construction verbs (each occurring only in one of the two constructions). This set-up was intended to resemble the pattern seen with alternating constructions in natural languages. However, there were no semantic or phonological cues to verb class. Thus in this experiment participants could only acquire verb subcategorization restrictions by paying attention to the actual verb-structure co-occurrence statistics. Although all three classes of verbs were equal in size and frequency, we also manipulated individual verb frequency within the one-construction verb classes. We predicted that we would see frequency effects mimicking those seen in natural languages, i.e. that participants would be less likely to produce, accept and predict over-generalizations with high frequency verbs than with low frequency verbs.

Method

Participants

Fourteen adult native English speakers participated in the experiment. Most were undergraduate students at the University of Rochester; the remainder were former students or age mates who attended another college. Participants were run and tested individually. They were paid daily for their participation and received a bonus upon completion of the entire experiment.

Description of the Language

The language had the following features:

Vocabulary

Nouns

There were five two-syllable nouns, each beginning with a different initial consonant and referring to 5 puppet animals [flugat (BEE), blergen (LION), slagum (LADYBUG), nagid (ELEPHANT), tombat (GIRAFFE)]. (The vocabulary set is adapted from that used in Hudson Kam & Newport, 2005). In contrast to verbs, referents were fixed across participants. There were no lexical restrictions on nouns; each occurred freely with all verbs and in each of the verbal structures.

Verbs

There were 12 one-syllable verbs [glim, norg, frag, flern, semz, gund, loom, mer, rov, shen, gofe] denoting 12 transitive actions [PUSH, STROKE, TICKLE, ROCK, KISS, HUG, RAM, BOP, JUMP-ON, DRAG, SIT-ON]. The assignment of verbs to actions was randomized across participants (as was the assignment of verbs to the verb classes – see Verb Classes, below).

Particles

There was one particle word ‘ka’. This had no referent but occurred in one of the two constructions and could be regarded as a linguistic marker of that construction.

Grammar

The language contained two possible sentence constructions:

Verb Noun1 Noun2	(e.g., glim tombat blergen)
Verb Noun2 Noun1 Particle	(e.g., glim blergen tombat ka)

Open in a new tab

Both of the two constructions corresponded to a simple transitive event structure in which Noun1 was the agent and Noun2 the patient of the act denoted by the verb. (Henceforth we refer to the two constructions as the VAP (verb agent patient) and VPA_ka (verb patient agent ‘ka’) constructions). For example, if glim means HIT, tombat means GIRAFFE, and blergen means LION, then:

glim tombat blergen	means	THE GIRAFFE HIT THE LION
glim blergen tombat ka	also means	THE GIRAFFE HIT THE LION

Open in a new tab

These forms were designed to have the following crucial properties:

Synonymy

Although it would be unusual, perhaps impossible, to find two entirely synonymous forms in any natural language (Givon, 1985), our aim in this work was to test whether syntactic constraints can be learned in the absence of semantic or pragmatic cues. Therefore in these experiments VN₁N₂ and VN₂N₁ka were synonymous, so there could be no functional reason for preferring either construction for any specific verb.

Temporary ambiguity

Sentences in this language were potentially ambiguous until the final particle. That is, if a learner hears Verb Noun Noun …, she will not know if this was a Verb Agent Patient or Verb Patient Agent sequence until the final ka particle is (or isn’t) heard. Crucially, however, this ambiguity holds only if the language learner believes that the verb can occur in both constructions. Thus, if we are able to access participants’ expectations about the structure they are hearing before they hear the disambiguating ka particle, we may learn: (a) whether they have acquired verb subcategorization and (b) whether they are able to use this knowledge in real-time sentence comprehension.

For example, imagine a participant is hearing the sentence:

glim	flugat	blergen	ka
RAM	BEE	LION	PARTICLE

Open in a new tab

as it unfolds. If the learner has acquired the constraint that glim can occur only in the VPA_ka construction, and if he or she is able to apply this constraint online, then as soon as the sentence begins and the verb is identified, the VPA_ka construction (and not the VAP construction) should be activated. Each noun begins with a distinct consonant. Thus as soon as the first noun is identified, the learner will expect this to be the patient. In the present example, this will be at the point where she has heard glim f …. At this point the learner should predict a sentence semantics in which BEE is the patient of the event RAM. On the other hand, if the learner thinks that glim can occur in both constructions, then the VAP and VPA_ka constructions will both be activated. In this case, the participant will not assume that BEE is the patient of RAM action until the final ka particle has been heard.

Verb Classes

The 12 verbs were divided into three classes:

4 VAP-only verbs (verbs that always occurred with the VAP structure)
4 VPA_ka-only verbs (verbs that always occurred with the VPA_ka structure)
4 alternating verbs (verbs that occurred with each structure 50% of the time)

Henceforth we will refer to VAP-only and VPA_ka-only verbs as one-construction verbs.

The two one-construction verb classes were then further sub-divided into low frequency (2 verbs) and high frequency (2 verbs). Low frequency verbs occurred in the input set less often than the alternating verbs, and the high frequency verbs occurred more often than the alternating verbs (see Input Sentence Sample).

To make sure there was no accidental preferred association between any verbs and their construction classes, a different random assignment was performed for each participant. This ensures that there are no unnoticed regularities between verb class and semantic or phonological features.

Input Sentence Sample

The input set in this experiment had the following distributional properties:

VAP-only, VPA_ka-only and alternating verb classes occurred equally often.
VAP and VPA_ka constructions occurred equally often.
Within the verb classes, verbs occurred equally often, except that high frequency verbs occurred three times as often as low frequency verbs and 1.5 times as often
as alternating verbs.

Materials

Sentences were built from a set of pre-recorded sound files for each of the individual words. Each of the 12 verbs, five nouns, and the particle were recorded with an intonation pattern appropriate to each of the possible sentence positions in which it could occur, and the resulting sound files were edited so that words of each type were of a standard duration (verb 470 ms, noun in first position 570 ms, noun in second position 625 ms, particle 470 ms). The words were concatenated into sentences in real-time during the experiment. This methodology ensured that word boundaries occurred at the same point across sentences, and also that two sentences such as glim tombat blergen and glim tombat blergen ka were identical up until the final ka particle. Participants questioned after the experiment seemed unaware that sentences were not produced as whole units. These audio stimuli were played along with video clips which provided the semantics. The clips involved scenes in which pairs of puppets engaged in simple transitive events (e.g. GIRAFFE HIT LION). The experiment ran using ExBuilder, a custom software package. The software was responsible for concatenating sentences from the word sound files and performed the participant-by-participant vocabulary randomization and the assignment of verbs to verb classes.

Procedure

A 5-day procedure was used for this experiment. One procedure was followed on days 1–3 (which were primarily for sentence exposure) and another on days 4 and 5 (which were primarily for testing).

Procedure Days 1 to 3

Part 1 - Vocabulary Learning

Participants viewed static pictures of each of the five puppet animals while hearing their names (4 exposures per noun), and then took a short vocabulary test in which they saw each picture and produced the appropriate noun orally. They then received feedback and help from the experimenter. This process was then repeated. Most participants could name every animal correctly by the second vocabulary test on day 1, and all participants could do this on day 2.

Part 2 - Sentence Exposure

Participants viewed a set of 144 transitive scene sentence pairs. The input set had the structure described above, and sentences were presented in random order. Participants were instructed to repeat the sentence out loud to help themselves learn.

Part 3 - Short Tests

These tests were mainly included to give participants some experience of the test procedures and were not scored. In order to assess whether test exposure would dilute the frequency manipulation, half of the participants were not given this part of the experiment on days 1 and 3. However, there were no differences between these groups, so the participants are all included in all analyses. Participants took the three tests in the order described below. Test items were presented in random order in each test.

Production Test

In each test trial, participants saw a scene and heard the verb corresponding to the action in the scene, followed by a pause. Their task was to complete the sentence. This method, which follows the procedure established by Hudson Kam and Newport (2005), allows participants to demonstrate knowledge of sentence structure and verb class without having explicitly memorized each of the twelve verbs. There were 12 test items, one for each verb.

Online Comprehension Test

In each trial participants viewed two video scenes showing the same pair of animals and the same event, but with reversed roles taken by the agent and patient (e.g. video 1: BEE RAM LION, video 2: LION RAM BEE). They then heard a sentence in the language (e.g. glim flugat blergen ka) and had to choose which scene depicted the correct meaning of the sentence. There were 12 test items, one for each verb. In order to minimize interference with the input statistics, all of the sentences we used obeyed the verb subcategorization restrictions that appeared in the input set: VAP-only verbs occurred with the VAP construction, and VPA_ka-only verbs occurred with the VPA_ka construction. Two of the four alternating verbs occurred with the VAP construction, and two occurred with the VPA_ka construction.

On days 1–3 we did not record eye movements during the Online Comprehension Test. However, the purpose of these tests was to familiarize participants with the test format that would be used on days 4 and 5. Thus trials included specific procedures for re-calibration and centralizing eye-fixations. The structure of each trial was as follows:

A white cross appeared in the center of the screen, which participants clicked on to start the trial. (When the test was eye-tracked, an automatic drift correction of the calibration was performed at this point). The two videos then appeared side by side on the screen, frozen on the first frame (see figure 1). Video 1 played, followed by Video 2, and the two pictures remained frozen on the screen in final frame. Another white cross appeared between the two pictures, and participants clicked on this cross to play the sentence (this centralized fixations, ensuring that participants were not looking at either picture before the sentence was heard). The sentence was heard. Participants then chose a picture by using the mouse to click on a red button at the bottom of that picture. They could only do this after the sound file had finished playing. Choosing the picture ended the trial.

Screen shot from Online Comprehension Test. At this point in the trial, the two videos have each played in turn and now remain on the screen, frozen in the final frame. The participant will now use the mouse to click on the white cross between the two.

Grammaticality Judgment Test

Participants viewed a video scene and heard a sentence with either a VAP or a VPA_ka structure. There were 12 test items, one for each verb. For one-construction verbs, half of these sentences broke the subcategorization restrictions of the input sentences, i.e. two VAP-only verbs occurred with VPA_ka, and two VPA_ka-only verbs occurred with VAP. Two of the four alternating verbs occurred with the VAP construction and two occurred with the VPA_ka construction. Participants were asked to make two responses, one after the other:

- Is this a correct sentence in the language? [Participants click ‘Yes’ or ‘No’]
- Give the sentence a score out of 5, where 5 means ‘definitely a correct sentence in the language’ and 1 means ‘definitely not a correct sentence in the language’. [Participants click on a number between 1 and 5].

Measures of both binary judgment and ratings were included on the assumption that they would show differences in sensitivity. However, no such differences were found. Since the two measures yielded an identical pattern of results in all of the experiments, the ratings data are not included in this paper.

Procedure on Days 4 and 5

Part 1 - Vocabulary Test

Participant saw pictures of each animal and said the appropriate noun. All participants scored 100% on both days.

Part 2 - Intermittent Short Exposure and Longer Tests

Short Sentence Exposure 1

Participants viewed a subset of 72 sentences from the exposure set used on the previous days. Again they were instructed to repeat the sentence out loud to help themselves learn.

Production Test

This was exactly like the production test given on days 1–3, except that there were 24 test items, 2 per verb, and all the scenes used in this test were entirely novel, i.e. had not occurred in the sentence exposure or the previous production test. (This ensured that participants could not base their productions on specific memorized sentences.) We scored the percentage of productions which correctly applied one of the two possible sentence constructions, and the proportion of these which corresponded to each construction for each verb.

Short Sentence Exposure 2

Participants viewed a subset of 24 sentences from the exposure set. (This was included between tests to refresh participants’ knowledge of the language and to minimize test effects on this knowledge).

Online Comprehension Test

This was like the Online Comprehension Test given on days 1–3, except that there were now 24 test items, 2 per verb (sentences with one-construction verbs obeyed the verb’s subcategorization restrictions, and each alternating verb occurred once with each construction); all of the sentences used were entirely novel, i.e. had not occurred in sentence exposure or the previous Online Comprehension Test; and participants were now eye-tracked while listening to the sentences and viewing the two scenes.

Eye-movements were monitored using an Eyelink II head mounted eye-tracker. At the beginning of the Online Comprehension Test, the system was mounted and the eye-tracker calibrated using the standard nine-point Eyelink calibration procedure. As described above, the trials included components which allowed for drift correction and centralized fixations. Eye-movements were monitored from the point at which the sentence began playing until the participant finished the trial by clicking on the red button at the bottom of one of the pictures (only possible after the sentence had finished). The Eyelink software automatically parses the eye track into three categories: saccades, fixations, and blinks. A look to one of the two pictures was defined from saccade onset to the end of the fixation. Because participants tended to look slightly below the picture when they clicked on the red button, the bottom border was extended by 15 pixels to capture more fixations. We recorded the average proportions of looks to each of the two pictures every 4ms, as the sentence unfolded in time.

Short Sentence Exposure 3

Participants viewed another subset of 24 sentences. (This was included between tests to refresh participants’ knowledge of the language and minimize tests effects on this knowledge).

Grammaticality Judgment Test

This was exactly like the test on days 1–3 except that there were now 24 test sentences, two per verb; and all of the sentences used in the test were novel. Each verb occurred with each of the two constructions, so that half of the sentences with one-construction verbs broke the subcategorization restrictions. We recorded whether participants judged each verb to be grammatical in each of the two constructions.

Results

Overall, 98% of productions were found to have word orders conforming to one of the two possible constructions (verb agent patient or verb patient agent particle), showing that participants had indeed mastered the basic language structures². The small minority of incorrect sentences had one of two other word orders: verb agent patient particle or verb patient agent. These were excluded from the analyses that follow.

Figure 2 shows, for each verb class, the proportion of correct productions which used the VAP construction (since incorrect productions are excluded, all remaining productions used the VPA_ka construction). In these and all subsequent experiments, there were no reliable effects of Practice Test Exposure (whether participants had taken the extra practice tests on days 1 and 2) or Day of Testing (whether the test was taken on day 4 or day 5), and these factors did not interact with the effects of either Verb Class or Verb Frequency. These factors are therefore collapsed.

Experiment 1 Production Test: Proportion of correct productions with *VAP* construction for each verb type.

Overall, 58.8% of productions were produced with the VAP construction, a marginally significant bias towards that construction (t=1.8, p<0.1, df=13). However, in spite of this bias, participants demonstrated clear knowledge of the subcategorization restrictions on the one-construction verbs. The majority of productions with VAP-only verbs used the VAP construction, and the majority of productions with VPA_ka-only verbs used the VPA_ka construction. Alternating verbs used each of the two constructions approximately equally often (the slight bias towards the VAP construction was not significant within this class, t=0.75, df=13, p>0.05). Overall, the main effect of verb class was significant (F(2,24)=16.31, p<0.01), and planned comparisons showed significant differences between all three classes (VAP-only versus VPA_ka-only: t=4.9, p<0.01, df=13, VAP-only versus alternating: t=3.76, p<0.01, df=13, and VPA_ka-only versus alternating: t=2.49, p<0.05, df=13).

As predicted, this learning of verb restrictions was modulated by verb frequency: participants used the correct construction more often with high frequency verbs than with low frequency verbs, F(1,12)=5.12, p<0.05. This effect mirrors so-called entrenchment effects in natural languages: participants are less likely to use verbs in new structures (i.e., to produce overgeneralizations) when those verbs have occurred more frequently in the input.

Grammaticality Judgment Test

Figure 3 shows, for each of the different verb types, the difference between the grammaticality judgments given to sentences using the VAP construction and those using the VPA_ka construction. (These scores are calculated by subtracting the judgment scores for each construction type. These raw data are shown in Table 1.) Note that the scores are with respect to the VAP construction, so that a score of 100% would mean always accepting VAP sentences and rejecting VPA_ka construction sentences, -100% would mean always accepting VPA_ka sentences and rejecting VAP sentences, and 0% would mean judging the two constructions to be equally grammatical.

Table 1.

Experiment 1 Grammaticality Judgment Test: Proportion of sentence types accepted as grammatical for each verb type.

	Sentence Type
Verb Class	VAP	VPA_ka	Difference
VAP-only low frequency	98.2%	67.9%	30.4%
VAP-only High frequency	100.0%	39.4%	60.7%
VPA_ka-only low frequency	53.6%	96.4%	−42.9%
VPA_ka-only High frequency	46.4%	100.0%	−53.6%
Alternating medium frequency	88.4%	94.6%	−6.3

Open in a new tab

The difference scores show a significant main effect of verb class (F(2,24)=16.27, p<0.01). Participants were more likely to accept the VAP construction with VAP-only verbs and more likely to accept the VPA_ka construction with VPA_ka-only verbs, whereas with alternating verbs they accepted the two structures approximately equally often. Planned comparisons showed significant differences between all three classes (VAP-only versus VPA_ka-only: t=4.39, p<0.01, df=13, VAP-only versus alternating: t=3.84, p<0.01, df=13, and VPA_ka-only versus alternating: t=3.3, p<0.01, df=13).

For the two one-construction classes, we also observed effects of verb frequency (F(1,12)=6.81, p<0.05). Turning from the difference scores to the scores for the different construction types (see Table 1), we can see that this is largely due to the greater tendency to reject incorrect overgeneralizations: for VAP-only verbs: 67.9% of low frequency and 39.4% of high frequency verbs were accepted in sentences with the VPA_ka construction; for VAP-only verbs: 46.4% of low frequency and 55.4% of high frequency verbs were accepted in sentences with the VAP construction. Again, this mirrors the entrenchment effects which have been described in the developmental literature.

In contrast to the results of the other tests, we see no bias towards the VAP construction in these data (in fact there were 2% more VPA_ka productions overall, t=−0.76, df=13, p>0.05).

Online comprehension test

Overall participants chose the correct picture on 95.7% of trials, showing again that they had mastered the two possible word orders in the language. However, the purpose of this test was to ascertain whether participants were able to use their knowledge of verb subcategorization to form expectations in real time processing. That is, we asked whether looking preference would reveal different points of disambiguation for the one-construction and alternating verbs. Figure 4 shows the proportion of looks to each of the two pictures as participants heard sentences using each of the two constructions.

The results are particularly clear when we compare looking patterns as participants hear VPA_ka sentences that begin with VPA_ka-only versus VPA_ka sentences that begin with alternating verbs. When the sentence begins with a VPA_ka-only verb, participants begin to look more often at the picture corresponding to the VPA_ka sentence at around 916ms, i.e. during the first noun. In contrast, when the sentence begins with a alternating verb, looks to the VPA_ka picture do not begin to rise until around 1900, i.e. after the participant has heard the disambiguating ka particle (before that time, in the critical region there is a bias towards the VAP construction). This demonstrates that participants show verb-specific biases online, and that, as in natural languages, this information is used at the earliest possible moment in processing.

For VAP sentences the comparison is less clear. In both cases, there are more looks to the VAP picture in the critical region. In the case of the alternating verb, this again demonstrates the inherent bias towards the VAP construction. However, in both cases there are, by chance, more looks to the VPA_ka picture in the initial period, causing looks to the VAP picture to rise even before the first noun has been heard (at which point there is no reason to identify either picture with either of the two constructions). Thus it is not possible in either case to identify the true point of disambiguation.

In order to factor out the effects of previous looking behavior, and to allow easy comparison across the different verb types, we also counted the total proportion of looks to each picture which began during the critical region from the beginning of the first noun (when they first potentially have enough information to identify the sentence structure), up to the beginning of the ka particle (the final disambiguating information). Allowing the standard 200ms for programming an eye-movement (Hallet, 1986), this period was taken to be the time period 669–1850ms. These data are shown in Figure 5.

Experiment 1 Online Comprehension: Percentage of looks to *VAP* picture during critical region of sentence (i.e. after beginning of first noun and before ‘ka’).

There is a clear influence of verb type on looking behavior within this critical period. For the subcategorized verbs, the majority of looks were in the direction of the appropriate picture. There was a significant main effect of verb class, (F(2,24)=6.11, p<0.01), and planned comparisons showed significant differences between the VAP-only and VPA_ka–only classes (t=2.81, df=13, p<0.05) and VPA_ka-only and alternating classes (t=2.56, df=13, p<0.05). However, alternating verbs also showed a VAP bias and the difference between alternating and one-construction verbs was not significant (t=1.14, df=13, p>0.05), due to the strong bias towards the VAP construction for the alternating class, which was itself significant (t=2.68, p<0.05, df=13).

We also examined the data from one-construction verbs for effects of frequency. Although these effects were in the expected direction, they were not significant (frequency effect for VAP class: F(1,12)=1.39, p>0.05, frequency effect for VPA_ka class: F(1,12)=0.26, p>0.05, overall frequency effect: F(1,12)=3.56, p>0.05).

Discussion

The central finding of this experiment is that participants were able to acquire verb-specific subcategorization constraints from the input distribution. That is, they successfully learned the permissible and non-permissible constructions for each verb. Since there are no semantic or phonological cues to verb class and no pre-empting structures indicating missing ungrammatical forms, we conclude that this is accomplished through entirely statistical processes: tracking the frequency of the various verb-structure co-occurrences and using this as evidence as to the status of both the occurring and the missing verb-structure combinations. It is important to emphasize that these constraints were successfully learned even though the language showed heterogeneity in the way constructions were used with different classes of verbs. Although the majority of verbs in the miniature language were subcategorized to occur with only one construction, a substantial number could occur with both constructions (four verbs in each of the three classes). In addition, participants’ ability to avoid over-generalization is particularly notable given that all test items were novel sentences (novel combinations of verbs and nouns), so the learning could not have been accomplished by memorizing specific sentences or verb-noun combinations from the input exposure set.

Results from the three different tests were highly consistent: learners avoided overgeneralizations in their own productions, judged over-generalized forms to be less grammatical, and used their knowledge of verb subcategorization in real-time parsing. This last result was particularly striking in that we saw an influence of verb-class on eye-movements at the earliest possible moment, suggesting that participants were forming expectations about upcoming structure as soon as the verb was processed. This provides strong evidence that our artificial language stimuli are being parsed in a manner akin to natural language and establishes that purely distributional verb-biases can play a role in this process. The only difference between the three tests was that, whereas participants were clearly biased to predict the VAP construction and show a marginal preference for that construction in production, this bias did not affect judgments of grammaticality. (We return to this point in the Discussion of experiment 2).

In addition, we saw frequency effects within the one-construction verb classes: participants were less likely to produce or accept new structures with high frequency verbs. This result is important for two reasons. First, it mimics the entrenchment effects reported in the acquisition and processing literatures, suggesting that our learners are following a similar development path to learners of natural languages. (We acknowledge that, as our two constructions have the same meaning, our effects might also be described as statistical pre-emption (Goldberg, 2005) Whether the same effects would have been obtained with two semantically distinct constructions is an interesting question for future research.) Secondly, observing learners’ behavior with verbs for which they have had different amounts of exposure has provided us with a window into the learning process. It shows us the process by which our learners are able to retreat from over-generalization: the more often they hear a verb used in only one of the constructions, the more certain they are that it shouldn’t be generalized. We expect that increased exposure to our languages would make learners increasingly less likely to over-generalize.

Experiment 2

The central finding of Experiment 1 was that learners were able to learn that certain verbs were (arbitrarily) subcategorized to occur only with one construction, even in the presence of a class of verbs which could occur in both. This constrained behavior can be described as lexical conservatism – the treatment of each verb was dependent on that verb’s own behavior in the past. However, though we have emphasized this conservative behavior, which shows successful acquisition of the verb classes, our learners did sometimes generalize, as we saw whenever they made an ‘error’ with the one-construction verbs. In this case, the treatment of these verbs must be affected by the behavior of the other verbs in the input. Thus, as in real acquisition, learners of these languages must balance what they know about verbs in general against their verb-specific knowledge. Our hypothesis is that this balancing process depends on the distributional details on the input (though we acknowledge that other semantic and phonological factors will also play a role in natural languages). We have already seen one distributional factor which affects the tendency to generalize: individual verb frequency. In Experiment 2, we investigate whether the tendency to generalize a construction to a new verb might depend on distributional factors outside of that verb’s own history. In particular, we explore whether changing aspects of the class of alternating verbs might make learners more likely to extend that class and treat all verbs as alternating.

In Experiment 1, the presence of a class of unrestricted, alternating verbs did not prevent learners from acquiring subcategorization restrictions on other verbs. However, two aspects of the language may have helped to make the limits of that class minimally confusing. First, there were only half as many alternating verbs as one-construction verbs, so that learners observed mostly lexically conservative behavior. This may have encouraged them to attend to lexically-based distributional information. Second, although we included frequency distinctions within the one-construction verb classes, all verbs in the alternating class occurred equally frequently. This may have provided the learner with additional evidence that there was a distinct set of verbs which could occur with both constructions. If, in contrast, verbs within the alternating class occurred with varying frequencies, a rational learner might be less sure whether all verbs in the language might likewise fall in this class, since the absence of sentences in which some verbs occurred in certain structures might simply be due to the low frequency of those combinations.

In Experiment 2, we manipulated these factors. Learners were exposed to a language in which verb-structure co-occurrence patterns were identical to those in Experiment 1 (one-construction verbs occurred 100% of the time in one structure, alternating verbs occurred 50% of the time in each structure), but the class of alternating verbs had two new properties: It was twice as large (in terms of number of verbs and overall frequency) as the two one-construction classes together, and it contained verbs which occurred with varying degrees of frequency. In addition, the frequency distinction within the two one-construction verb classes was increased from 3:1 to 5:1. This was done in an effort to increase the likelihood of finding these effects, particularly in the eye-tracking data, where they did not appear in Experiment 1.