Sentactics®: Computer-Automated Treatment of Underlying Forms

Abstract

Background

Treatment of Underlying Forms (TUF) is a linguistically-based treatment for improving agrammatic sentence deficits, which enjoys a substantial database attesting to its efficacy for improving both sentence comprehension and production in agrammatic aphasia. However, TUF requires considerable linguistic background to administer and administration time can exceed the number of treatment sessions allotted in toto for reimbursement by third party payors in the United States. Thus, Sentactics®, an interactive computer system that enables delivery of TUF by a virtual clinician was developed.

Aims

This study tested the effects of Sentactics® on the acquisition and generalized production and comprehension of complex sentences. Additionally, a direct comparison of the results of computer-delivered Sentactics® and clinician-delivered TUF was undertaken.

Methods & Procedures

Twelve agrammatic aphasic speakers participated in the study, with six receiving Sentactics® and six serving as experimental controls, who received no treatment. All participants were administered pre- and post-treatment sentence comprehension and production tests and other measures to evaluate the effects of Sentactics®. Performance of the Sentactics® group also was compared to eight agrammatic patients who previously received clinician-delivered TUF treatment, identical to that delivered via Sentactics®, but with a human clinician.

Outcomes & Results

Sentactics® significantly improved all six aphasic speakers’ ability to comprehend and produce both trained and untrained, linguistically related, complex sentences as compared to six agrammatic control participants who did not receive Sentactics®. In addition, comparing the results of the Sentactics® to clinician-delivered TUF revealed no significant differences between approaches with regard to acquisition or generalization patterns.

Conclusions

These data provide further support for the efficacy of TUF and demonstrate the viability of computer-delivered therapies in the field of aphasia treatment.

Introduction

Individuals with agrammatic Broca’s aphasia exhibit difficulty comprehending and producing sentences. Output is often limited to simple, canonical (S-V-O in English) sentences or ungrammatical word strings consisting of mainly nouns, with few verbs or closed class elements (Berndt, Haendiges, Mitchum, & Sandson, 1997; Berndt, Mitchum, Haendiges, & Sandson, 1997; Kim & Thompson, 2000; Marshall, Pring, & Chiat, 1998; Miceli, Silveri, Villa, & Caramazza, 1984; Zingeser & Berndt, 1990). Sentence comprehension deficits also are prevalent, affecting primarily non-canonical forms such as passives and object relatives, particularly in semantically reversible structures (Friedmann & Shapiro, 2003; Grodzinsky & Finkel, 1998; Thompson & Shapiro, 2005). Several treatments for agrammatic sentence deficits have been tested, including those that focus on practicing the surface form of target structures (Doyle, Goldstein, & Bourgeois, 1987; Helm-Estabrooks 1981; Helm-Estabrooks & Nicholas, 2000; C. K. Thompson & McReynolds, 1986; Wambaugh & Thompson, 1989) and those which focus on training linguistic constructs and computations that are essential for sentence comprehension and production, including Mapping Therapy (MT) (Davis & Tan, 1987; Haendiges, Berndt, & Mitchum, 1996; Rochon, Laird, Bose, & Scofield, 2005; Schwartz, Saffran, Fink, Myers, & Martin, 1994; Stadie et al., 2008) and Treatment of Underlying Forms (TUF) (Ballard & Thompson, 1999; Jacobs & Thompson, 2000; Thompson, Ballard, & Shapiro, 1998; Thompson & Shapiro, 1995, 1997; Thompson, Shapiro, Kiran, & Sobecks, 2003). Central to both MT and TUF is training verbs, with emphasis on the thematic roles they assign, because without verbs sentences are ungrammatical. TUF also focuses on the syntactic properties of sentences that operate to derive noncanonical structures.

Research has shown that all of these methods can be useful for improving production and comprehension of trained sentences, however, the extent to which generalization to untrained sentence types results from treatment varies considerably across approaches. In general, treatments that focus on the surface form of target structures do not foster generalization to untrained forms. Similarly, MT shows limited improvement of structures beyond those trained. For example, a study by Schwartz et al. (1994) trained eight individuals with chronic nonfluent aphasia to comprehend active sentences, which resulted in improved comprehension (and production) of active forms, but little generalization was noted to untrained passive sentences. In contrast, research examining the effects of TUF has shown that this approach engenders robust generalization from trained to untrained sentence types. Importantly, rather than broad generalization across sentence types, generalization is constrained to structures that are linguistically related to one another. That is, improved comprehension and production of untrained sentences with syntactic properties similar to those of the trained form results from treatment. For example, training sentences involving wh-movement such as object relatives results in improved production and comprehension of other structures involving wh-movement such as object cleft structures and object extracted wh-questions. However, training wh-movement structures does not improve linguistically unrelated structures such as passive sentences, which involve NP-movement.

Additionally, studies testing the effects of TUF have shown that when more complex sentences are trained, generalization to less complex sentences occurs, but not vice-versa. The Complexity Account of Treatment Efficacy (CATE; Thompson et al., 2003) emphasizes that training complex structures results in generalization to less complex sentences only when untreated structures encompass processes relevant to treated ones. This account explains a generalization pattern from trained object relative structures to untrained object clefts and object wh-questions, as seen in TUF, because object relatives are more complex than the other two forms.

Despite its demonstrated efficacy, TUF is not widely utilized by clinicians at least in the United States. One reason for this is that it requires considerable linguistic knowledge as well as a substantial amount of training to administer. Additionally, the treatment requires up to twenty sessions per structure to train, which often exceeds the number of sessions allotted in toto for therapy reimbursed by third party payors in the United States. We, therefore, developed a computer-automated version of TUF – Sentactics® – in attempt to overcome these limitations and make it possible for individuals with aphasia to practice the protocols with only minimal guidance from clinicians.

Sentactics® combines TUF protocols with a well-developed and tested interactive computer system that enables face-to-face communication with a virtual agent, developed at the Center for Spoken Language Research (CSLR) at the University of Colorado. Several projects have used CSLR’s virtual agents to teach various language skills such as reading (Cole, Wise, & Van Vuuren, 2007) and voice training through Lee Silverman Voice Treatment (LSVT) to patients with Parkinson’s disease (Cole, Halpern et al., 2007). CSLR’s virtual agents also have been developed to train conversational scripts in people with aphasia (Cherney, Halper, Holland, & Cole, 2008; Manheim, Halper, & Cherney, 2009).

Application of computer technology to aphasia intervention is not a novel concept and has resulted in the development of various computerized aphasia intervention approaches over the past years. Computer-assisted or computerized aphasia therapy procedures have been developed for clinical intervention in aphasia for training word-finding (Fink, Bartlett, Lowery, Linebarger, & Schwartz, 2008; Fink, Brecher, Schwartz, & Robey, 2002; Ramsberger & Marie, 2007), reading (Katz & Wertz, 1997; Laganaro & Overton Venet, 2001), speech perception (Manheim et al., 2009), speech sound production (Reeves, Jefferies, Cunningham, & Harris, 2007), and production of client-selected sentences (Linebarger, Schwartz, Romania, Kohn, & Stephens, 2000; Linebarger, Schwartz, & Kohn, 2001; Linebarger, McCall, & Berndt, 2004). However, comparatively few of these have been tested for treatment efficacy. Furthermore, to our knowledge no studies comparing computerized and clinician-delivered treatment have been undertaken.

The purpose of this study was twofold. First, we experimentally tested the efficacy of Sentactics®, examining both the treatment and generalization effects of this computer-delivered treatment. That is, we asked whether training with Sentactics® improves comprehension and production of trained sentence types, as well as untrained, linguistically related, less complex sentences. Secondly, we examined the relative effectiveness of computer-delivered Sentactics® and clinician-delivered TUF by comparing the results for participants trained with Sentactics® to those for previously clinician-trained TUF participants. This study then sought specifically to validate the use of computerized TUF protocols and more generally to provide data supporting the use of computerized treatments for aphasia.

Method

Participants

Twelve individuals with agrammatic Broca’s aphasia (10 males) participated in the study, with six receiving Sentactics® and six serving as controls. All were selected from the participant pool in the Aphasia and Neurolinguistics Research Laboratory using explicit selection criteria (see below). The first six consecutively identified candidates who met these criteria received Sentactics®; the following six served as experimental controls and received no treatment. After completion of the study, the control participants were entered into other experimental treatments in the Aphasia and Neurolinguistics Research Laboratory. For all participants, aphasia resulted from a single, thromboembolic stroke, which occurred at least one year prior to the study. They ranged in age from 35 to 68 (mean=49.5), had at least a high school education, were native speakers of English, and all but one were right handed (see Table 1). All participants passed screening tests for apraxia and dysarthria, hearing (at 500 Hz, 1000Hz, 2000 Hz and 4000 Hz at 40dB), and vision, and all demonstrated ability to read single words and to perform computer mouse movements. None of the participants reported a prior (pre-morbid) history of speech-language, learning or neurological disorders. All provided signed informed consent prior to participation.

Table 1.

Demographic data for participants who received Sentactics® (S1-S6), control participants (C1-C6), and those selected from previous studies who received clinician-delivered TUF (TUF1-TUF8).

	Age	Education (years)	Handedness	Months Post Stroke
S1	35	16	R	13
S2	44	16	R	19
S3	59	20	L	196
S4	68	18	R	98
S5	59	12	R	75
S6	38	18	R	42
M	50.5	16.7		73.8
C1	58	15	R	14
C2	52	22	R	59
C3	37	13	R	106
C4	41	12	R	49
C5	59	18	R	32
C6	44	14	R	15
M	48.5	15.6		45.8
TUF1	51	20	R	6
TUF2	59	18	R	24
TUF3	51	12	R	8
TUF4	38	19	R	51
TUF5	58	19	R	33
TUF6	56	16	R	49
TUF7	52	12	R	139
TUF8	54	12	R	59
M	55	16.0		46.1

Language Testing

The diagnosis of aphasia was made based on performance on the Western Aphasia Battery (WAB; Kertesz, 1982) with Aphasia Quotients (AQs) ranging from 46.5 to 89.1 (mean=73.9) (see Table 2). Fluency scores were 4 or 5 and auditory-verbal comprehension scores ranged from 7.6 to 10.0. To further test sentence comprehension and production the Sentence Production Priming Test (SPT) and Sentence Comprehension Test (SCT) of the Northwestern Assessment of Verbs and Sentences (NAVS; Thompson, experimental version) were administered, with results showing better comprehension compared to production and better performance on canonical sentences (subject extracted wh-questions and subject relatives) compared to noncanonical structures (object extracted wh-questions and object relatives) (see Table 3)¹.

Table 2.

Western Aphasia Battery (WAB) scores for the aphasic participants. S1–S6 received Sentactics®; C1–C6 served as control participants and did not receive Sentactics®. TUF1-TUF8 received clinician-delivered TUF.

	Aphasia Quotient (AQ)	Fluency	Comprehension	Repetition	Naming
S1	46.5	4	7.6	5.4	6.3
S2	79.9	4	9.15	6.8	10
S3	78.1	4	9.45	6.8	9.8
S4	87.6	5	10	9.7	9.1
S5	78.1	4	9.35	6.5	7.9
S6	73.7	4	7.95	8.2	7.7
M	74.0	4.16	8.9	7.3	8.2
C1	50.5	4	8.1	5.8	6.4
C2	74.3	4	9.0	6.3	9.8
C3	77.2	4	9.2	6.5	9.8
C4	89.1	5	10	9.9	9.5
C5	79.4	4	9.4	6.8	7.8
C6	72.6	4	7.7	8.1	7.5
M	73.85	4.16	8.9	7.2	8.5
TUF1	79.6	4	9	9.2	9.6
TUF2	72.4	5	9.3	7	7.9
TUF3	66.8	5	7.2	5.6	7.6
TUF4	79	4	9.3	8.6	8.6
TUF5	71.2	5	8.7	7.6	7.3
TUF6	66.8	2	8.5	9.4	7.5
TUF7	62.4	4	7.2	4.8	7.1
TUF8	69.6	5	8.2	5.4	7.2
M	73.85	4.3	8.4	7.2	7.9

Table 3.

Pre- and post-treatment scores on the Northwestern Assessment of Verbs and Sentences (NAVS) for participants trained with Sentactics® (S1-S6) and control participants (C1-C6). (3a) Sentence Production Priming Test (SPPT) scores; (3b) Sentence Comprehension Test (SCT) scores.

(a)
	Subject wh-questions		Subject relatives		Object wh-questions		Object Relatives
	Pre-Test	Post-Test	Pre-test	Post-test	Pre-test	Post-test	Pre-test	Post-test
S1	0	100	0	20	0	0	0	100
S2	100	0	0	40	60	100	0	100
S3	60	20	80	80	20	100	0	20
S5	0	0	0	0	0	0	0	80
S6	0	80	0	40	0	100	0	20
M	32	40	16	36	16	60	0	64
C1	20	20	0	0	0	0	0	0
C2	60	60	0	0	20	20	0	0
C3	100	80	60	0	0	0	0	0
C4	0	0	0	20	20	40	0	0
C5	0	0	0	0	0	0	0	0
C6	0	40	0	0	0	0	0	0
M	30	33.3	10	3.3	6.6	10	0	0
(b)
	Subject wh-questions		Subject relatives		Object wh-questions		Object Relatives
	Pre-Test	Post-Test	Pre-test	Post-test	Pre-test	Post-test	Pre-test	Post-test
S1	0	100	80	80	0	60	40	60
S2	100	80	40	0	80	100	80	100
S3	100	80	60	80	100	100	60	100
S5	100	100	100	100	80	100	40	100
S6	100	60	100	80	100	100	60	100
M	80	84	76	68	72	92	56	92
C1	80	70	70	70	40	60	20	20
C2	100	100	80	80	60	40	40	40
C3	80	60	80	60	80	60	40	60
C4	100	100	60	80	100	80	40	40
C5	100	100	80	80	80	80	60	60
C6	100	100	80	60	80	80	40	20
M	93.3	88.3	75	71.7	73.3	66.6	40	40

Narrative language samples, collected by asking participants to tell the story of Cinderella after viewing a wordless picture book, also were analyzed to determine production ability. Using a method described by Thompson et al. (1995), all samples were transcribed, segmented and analyzed for utterance length, words per minute, and aspects of grammaticality, including noun:verb ratio, open:closed class ratio, and the proportion of verbs produced with correct arguments and grammatical morphology. Results showed an agrammatic production pattern for all participants (see Table 4).

Table 4.

Pre- and post-treatment data derived from narrative analysis for participants trained with Sentactics® (S1-S6) and control participants (C1-C6).

	S1		S2		S3		S5		S6		Mean
	Pre	Post	Pre	Post	Pre	Post	Pre	Post	Pre	Post	Pre	Post
MLU	2.29	2.72	4.65	5.13	5.25	7.7	7.15	7.45	4.6	7	4.79	6
WPM	10.71	19.61	63.33	77.17	55.53	49.38	82.57	86.86	25.71	74.47	47.6	52
Noun:Verb Ratio	1.88	1.58	1.44	1.63	1.01	1.57	0.91	1.49	0.76	0.67	1.2	1.39
Open: Closed Class Ratio	1.86	1.84	2.77	1.92	1.48	1.36	0.80	1.02	1.03	0.84	1.59	1.40
% Grammatical sentences	29	21	23	27	64	48	41	51	76	67	46.7	42
Complex:Simple Sentence Ratio	0	0.15	0.77	1.36	0.27	0.28	0.48	0.5	0.30	0.91	0.36	0.64
% Verbs with correct arguments	62	25	66	59	78	91	66	86	82	88	71	70
	C1		C2		C3		C4		C5		C6		Mean
	Pre	Post	Pre	Post	Pre	Post	Pre	Post	Pre	Post	Pre	Post	Pre	Post
MLU	4.9	5.2	4.44	4.18	6.34	5.99	7.05	7.01	3.30	3.31	6.29	6.88	5.39	5.47
WPM	28.95	30.08	44.33	46.5	59.9	50.09	77.50	80.21	16.4	14.04	44.97	46.02	45.34	44.49
Noun:Verb Ratio	1.20	1.34	1.49	1.44	2.10	2.3	0.99	0.92	2.01	1.98	1.12	1.13	1.46	1.52
Open: Closed Class Ratio	1.13	1.02	1.69	1.82	1.36	1.36	0.92	1.02	2.06	2.22	1.34	1.02	1.42	1.41
% Grammatical sentences	75	72	36	33	49	48	47	44	22	19	44	46	45.5	43.66
Complex:Simple Sentence Ratio	0.40	0.28	0.65	0.59	0.45	0.42	0.50	0.48	0.18	0.15	.67	.65	.48	.43
% Verbs with correct arguments	39	41	62	61	77	76	84	82	61	61	55	49	63	61.66

Clinician-trained TUF participants

Agrammatic, Broca’s aphasic individuals who participated in previous studies were selected for comparison to the Sentactics®-trained patients. This included only (and all) patients who received TUF focused on object relatives (with generalization tested to object cleft and object wh-question structures) (n=8; 7 males)². All were at least 1 year post-onset of a single, thromboembolic stroke and ranged in age from 38 to 59 years (mean= 55), with WAB AQs from 62.4 to 79.6 (mean=73.85) at the time of testing (see Tables 1 and 2). All showed agrammatic sentence production patterns as assessed by their Cinderella narratives and impaired production and comprehension of noncanonical sentences as assessed by the NAVS. They were right handed, native English speakers and demonstrated visual and hearing acuity within normal limits. All reported no prior (pre-morbid) history of speech-language, learning or neurological disorders.

Experimental stimuli

Twenty-four active (NP-V-NP) sentences were developed using a set of 24 transitive verbs. All were semantically reversible, with two animate nouns. All nouns and verbs were controlled for frequency, imageability, and length, with none exceeding two syllables. For each sentence, two black and white line drawings were developed, one depicting the target sentence and the other depicting its semantically reversed counterpart. Fourteen of the items were used for training, whereas 10 were used to test comprehension and production of untrained sentences.

For the 14 training items, object relative structures were developed as in 1a below to comprise the sentences targeted in treatment. For the 10 test items, three sentence types were developed: an object relative (1a), object cleft (1b) and object-extracted wh-question (1c) to comprise a total of 30 items. These sentences are linguistically related in that they all are noncanonical and involve wh-movement. However, they differ in syntactic complexity: object relatives comprise the most complex and object-extracted wh-questions comprise the least complex form. (See Thompson & Shapiro, 2007, for discussion of wh-movement as well as complexity metrics in the syntactic domain.)

1. 1. Pete saw the man who the woman saved. (object relative)
   2. It was the man who the woman saved. (object cleft)
   3. Who did the woman save? (object-extracted wh-question)

All target object relative sentences as well as the prompts and feedback used in training were recorded by a female native speaker of English. These recordings and pictures were integrated into the Sentactics® program, which directly followed previously established TUF protocols, taking participants through a series of steps to derive the surface form of complex sentences from its underlying active (i.e., subject-verb-object) form. The 30 test items also were recorded by the same female speaker and entered into the program, with appropriate prompts, for pre- and post-treatment testing.

Procedures

Sentactics® was administered by a virtual clinician (Sabrina) using a Dell laptop computer, taking participants through all pre-treatment testing, treatment and probe sessions. Participants were instructed to follow Sabrina’s directions and to respond to them verbally or, where appropriate, using the computer mouse. During all Sentactics® sessions, a human examiner was present in the room (seated behind and to the side of the participant) to initiate administration of the protocols, ensure that the computer ran smoothly, and to monitor participants’ verbal responses using a Targus wireless keypad. Entries of 0 or 1 were entered on the keypad for incorrect and correct responses, respectively, occurring during pre- and post-treatment production tests as well as production probes administered daily, prior to each treatment session (see below). During treatment these entries were made when the Sentactics® treatment protocol required production responses, triggering computer-generated feedback (correction procedures or verbal praise) necessary for training. Both verbal and mouse-generated responses were automatically tracked by the computer for data analysis purposes. In addition, Sentactics® recorded all verbal responses, which were transcribed following each session for reliability purposes.

Pre-treatment Test

Prior to beginning Sentactics®, participants’ ability to produce and comprehend each sentence type was tested using the 30 items developed for this purpose, with item presentation pseudorandomized such that the same sentence type did not occur more than three times consecutively. Response contingent feedback was not provided, however, Sabrina gave intermittent encouragement on both production and comprehension tests. Production was tested prior to comprehension and percent correct performance on these tests served as the primary outcome measure for the study.

The pre-treatment production test used a sentence-production priming procedure to elicit target sentences. The computer randomly generated a picture pair (see Figure 1) and Sabrina explained “Here are two pictures, both show a man and a woman” as the computer drew a circle around the man and the woman, respectively. The computer then highlighted the picture on the left side of the screen as Sabrina modeled the target sentence (e.g., “For this picture, I could say, ‘It was the woman who the man saved’.”) and then prompted the participant to produce a similar sentence to describe the picture on the right side of the screen (i.e., “For this picture you could say …”). Participants were given 15 seconds to respond, with only syntactically well-formed representations of the target sentence considered correct. Crucially, production of object relative and object cleft structures required overt production of “who” or “that” in the embedded clause, even though reduced relatives are often considered grammatically correct (e.g., Pete saw the woman ___ the man saved). However, responses with incorrect verb inflection, missing determiners or semantic paraphasias were considered correct.

Figure 1.

Sample Sentactics® sentence production test screen. A similar screen was used for testing comprehension, although for comprehension testing, neither picture in the pair was highlighted.

A sentence-picture matching task was used in the pre-treatment comprehension test. On each trial a randomly selected stimulus pair was presented (such as in Figure 1), Sabrina produced a sentence, and instructed the participant to click on the corresponding picture. A 5 second response time was allowed, following which the next randomly selected picture pair was presented for testing.

Treatment

For all participants object relative structures were trained using the 14 sentences designated for Sentactics®. At the beginning of each Sentactics® session all items were probed for production accuracy using procedures identical to those used in pre-treatment testing, with performance on these probes reflecting daily treatment progress. Once these probes were completed, object relative treatment trials were begun.

On each trial, the computer randomly selected a target sentence for practice. First, Sabrina presented the sentence-picture matching task, followed by the sentence production-priming task. Computer-generated feedback was provided for participant responses to each. Next, Sabrina proceeded through the computerized-TUF training protocol to provide additional practice comprehending and producing the target sentence. For this practice, only the picture corresponding with the target sentence was presented together with written words displaying the active sentence form of the matrix and embedded clauses in card-like boxes on the computer. Slots for entry of word cards to form the target sentence were also provided at the bottom of the computer screen. Sabrina first asked participants to read the active sentences aloud and to identify and produce the verb, Agent, and Theme in each, with feedback provided. She then demonstrated how to combine the two active sentences to form the target object relative by placing and moving the word cards into appropriate slots. Then the participant was instructed to read the target sentence and identify the verb, Agent, and Theme in the derived, complex structure. Feedback and corresponding correction procedures were provided for all responses. Following this, the word cards were returned to their original position (i.e. in active sentence form) and Sabrina instructed the participant to construct the complex sentence by moving the word cards into their proper slots using the computer mouse. Finally, Sabrina repeated the comprehension and production tasks (i.e., sentence-picture matching and sentence-production-priming) with feedback provided. (See Thompson, 2008, for a step-by-step description of the TUF object relative training protocol; also see www.communication.northwestern.edu/videos/aphasia/ORtreatment.html for a sample of the beta version of Sentactics®.) The computer then randomly selected a new target sentence, generated the corresponding picture stimuli, and proceeded through the training steps. Participants received four one-hour Sentactics® sessions per week, occurring on two separate days. Sessions continued for a maximum of 20 or until 80% correct performance on the daily production probe was noted for 4 consecutive days, whichever occurred first.

Post-Treatment Testing

Following completion of Sentactics®, subjects were administered post-treatment comprehension and production probes, identical to those used for pre-treatment testing. The NAVS was also administered and Cinderella narratives were collected and analyzed. Control participants were tested on all measures following a six to eight week period, during which they received no treatment. All controls were offered either clinician- or computer-delivered TUF, free of charge, following this period.

Clinician-delivered TUF procedures

The procedures for clinician-delivered TUF were identical to those used for computer-delivered Sentactics®, except for the treatment-delivery method. As with Sentactics®, all sentence structures were tested for comprehension and production prior to and following treatment, and daily probes were administered prior to each one-hour treatment session throughout the treatment phase to evaluate performance. One difference between Sentactics® and clinician-delivered TUF was that in the former only object relatives were probed daily; the latter tested all three structures at the beginning of each treatment session.

Data analysis

All responses derived from pre- and post-treatment production tests as well as daily production probes were transcribed and checked for intra-rater reliability by the examiner, with results showing 98% agreement. The examiner also transcribed all verbal responses occurring during Sentactics® and checked these transcriptions for reliability by replaying the voice recordings stored in Sentactics®. These latter checks resulted in 99.8% agreement.

For both production and comprehension pre- and post-treatment tests, computer-generated means were derived for both Sentactics® and control participants and group means were calculated. Pre- and post-treatment scores were compared within each participant group and analyzed statistically using the Wilcoxon signed ranks test. Between-group comparisons were conducted by analyzing differences between pre- and post-treatment mean scores for each group using the Mann-Whitney test. A significance level of p<.05 was set for all comparisons.

Results

Response to treatment

Sentactics® improved all participants’ ability to produce target object relative sentences. On treatment probes conducted prior to each Sentactics® session, all participants showed 0% correct production of object relatives; by the end all had reached criteria, except for participant S1 who performed between 60 and 80% correct on the final four treatment probes.

Pre- to post-treatment performance

Production

Figures 2 and 3 present data derived from pre- and post-treatment production tests for Sentactics® and control participants, respectively, for object relatives, object clefts and object wh-questions. These data show that Sentactics®-trained participants improved in the production of the trained object relatives from pre- to post-treatment testing, with a significant difference found between the two test points (Z=2.207, p=.027). In contrast, the untreated control participants showed little change in performance, with no significant difference across test points found (Z=0, p=1.0). Analysis of the group data showed a significant difference between the treated and untreated control groups’ performance (Z=3.083, p=.002).

Figure 2.

Proportion of correct responses on pre- and post-treatment production tests for Sentactics®-trained participants for object relative, object cleft and object wh-question structures. Error bars, which represent standard errors (SEs), have been inserted for the group mean scores.

Figure 3.

Proportion of correct responses on pre- and post-treatment production tests for object relative, object cleft and object wh-question structures for the control participants. Error bars, which represent standard errors (SEs), have been inserted for the group mean scores.

Sentactics® also resulted in generalization to untrained object cleft structures and object-wh questions, with five of the six participants showing improvement in one or both structures. Only participant S1 showed no generalization to either structure. Analysis of the group data indicated that improvement from pre- to post-testing was significant for object wh-questions (Z= 2.203, p=.043), but not for object clefts (Z=1.826, p=.067). In contrast, the control groups’ performance did not differ significantly from pre- to post-testing for either structure (object cleft: Z=1.00, p=.317; object wh-question: Z=1.289, p=.197) (See Figure 3). Further, comparison of the two group’s performance indicated a significant difference for object clefts (Z=2.321, p=.041), but not for object wh-questions (Z=.323, p=.747).

Comprehension

Figure 4 depicts the results of pre- and post-treatment comprehension tests for participants trained with Sentactics®. As can be seen, production of object relatives was improved for all participants, from a mean of 50% (range: 30-86%) pre-treatment to a mean of 82% (range 57-100%) post-treatment. Pre treatment scores were at chance (t(5)=-0.013, p=.99), whereas, post treatment scores were above chance (t(5)=3.901, p=.011) and the difference in object relative comprehension scores before and after treatment was significant (Z=2.201, p=.028).

Figure 4.

Proportion of correct responses on pre- and post-treatment comprehension tests for Sentactics®-trained participants for object relative, object cleft and object wh-question structures. Error bars, which represent standard errors (SEs), have been inserted for the group mean scores.

Improvement on untrained object cleft and object wh-questions was less consistent, with pre-treatment means of 61% (range: 15-100%) and 77% (range: 40-100%), respectively, and post-treatment means of 76% (range: 47-100%) for object clefts and 86% (range: 60-100%) for object wh-questions. Comparison of scores before and after treatment yielded no significant difference for either (object clefts: Z=1.483, p=.138; object wh-questions: Z=1.461, p=.144). However, a comparison of the pre and post comprehension scores to chance showed that object clefts improved from chance-level at pre-treatment (t(5)=.838, p=.44) to above chance at post-treatment (t(5)=3.18, p=.025); whereas comprehension of wh-questions was above chance at both pre- (t(5)=2.608, p=.048) and post-treatment (t(5)=6.177, p=.002).

The control groups’ pre- to post-test comprehension performance is shown in Figure 5. Notably, performance was largely stable across test points for all participants and no significant differences were found for any structure pre- to post- treatment (object relative: Z=.447, p=.655; object cleft: Z=.816, p=.414; object wh-question: Z=.272, p=.785). Analysis of Sentactics® compared to the control participants’ performance revealed a significant difference between the treated and untreated groups for object relatives (Z= 2.908, p=.004) but not for object clefts (Z=1.386, p=.166) or object wh-questions (Z=1.328, p=.184).

Figure 5.

Proportion of correct responses on pre- and post-treatment comprehension tests for object relative, object cleft, and object wh-question structures for control participants. Error bars, which represent standard errors (SEs), have been inserted for the group mean scores.

Effects of Sentactics® on the NAVS and narrative language samples

The NAVS was administered and Cinderella narratives were collected and analyzed at pre- and post-treatment to determine whether Sentactics® resulted in improved performance on other language measures. On the NAVS, subject- and object-extracted wh-questions and subject and object relatives were tested for production (using the SPPT) and comprehension (using the SCT). Table 3 presents the results for both the Sentactics® and control groups. On the SPPT, mean scores improved from pre- to post-treatment for all participants trained with Sentactics® for all sentence types. However, pre- to post-treatment changes in subject wh-questions and subject relatives were not significant (subject wh-question: Z=.185, p=.854; subject relative: Z=1.633, p=.102). In addition, no significant differences were found between the two groups from pre- to post-treatment for either subject-extracted wh-questions (Z=-.942, p=.346) or subject relatives (Z=-1.702, p=.089). For the noncanonical wh-movement structures, the Sentactics® group showed numerically improved scores for both object-extracted wh-questions and object relatives. Object wh-questions improved from a mean of 16% correct at pre-treatment to 60% at post-treatment and object relatives improved from a mean of 0% at pre-treatment to 64% at post-treatment, however, only the improvement on object relatives was significant (object relatives: Z=2.041, p=.041; object extracted wh-questions: Z=1.414 p=.157). In contrast, the control group showed no changes from pre-to post-treatment (6.6% accuracy at pre-treatment and 10% accuracy at post-treatment for object wh-questions; 0% accuracy at both pre-treatment and post-treatment for object relatives). When tested statistically, a significant difference was found between groups for object relative (Z=-3.003, p=.003), but not for object-wh questions (Z=-942, p=.346).

On the SCT, there was little change from pre- to post- treatment for either subject wh-question (Z=.368, p=.713) or subject relative (Z=.816, p=.414) for the Sentactics® participant group. In addition, there was no significant difference between the control and Sentactics® groups for either sentence type (wh-questions: Z=-.774, p=.439; subject relatives: Z=-.173, p=.863). For the two structures involving wh-movement, object extracted wh-questions improved from 72% pre-treatment to 92% post-treatment and object relatives improving from 56% pre-treatment to 92% post-treatment. However, only improvement in object relatives was significant (object relatives: Z=2.041, p=.041; object extracted wh-questions: Z=1.633, p=.102). For the controls, no statistically significant differences across test points were found for comprehension of either wh-movement structure (object wh-question: Z=.447, p=.655; object relative: Z=.447, p=.655). Comparing performance between groups significant effects were found for object relatives (Z=-2.459, p=.014), but not for object wh-questions (Z=-1.905, p=.057).

Analysis of the Cinderella narrative data also compared pre- to post- treatment scores for five of the six Sentactics®-trained participants (see Table 4) and compared performance of this group to the untrained control participants’ narrative data. First considering the Sentactics®-trained group, improvement from pre- to post-treatment was noted on three of seven variables examined: mean length of utterance (MLU) increased from 4.79 to 6.0 words; words per minute (WPM) increased from 47.6 to 52.0; and the complex to simple sentence ratio improved from .36 to .64. These difference in MLU and complex to simple sentence ratio were significant (MLU: Z=2.023, p=.043; complex to simple sentence ratio: Z=2.023, p=.043). For the control group, all measures were stable across the two test points, with none of the measures showing a statistical difference between the two test points. However, comparing changes on narrative measures between the Sentactics® and control groups resulted in no significant differences between groups for any of the variables measured.

Clinician-delivered TUF versus Sentactics®

Figure 6 presents the results of clinician-delivered TUF on production of object relative, object cleft, and object-extracted wh-questions for eight agrammatic aphasic speakers, selected from previous studies. These data show that all participants improved on all three structures. Object relatives improved from a mean of 4% pre-treatment to 75% correct at post-treatment (range: 0-100%, sd=9.76%). Production of object cleft structures improved from 5% (range: 0-20%) to 62% (range: 20-85%) and production of object wh-questions improved from 28% (range: 0-90%) to 84% (70-100%). Statistical comparisons of pre- to post-treatment scores were significant for all structures (object relatives: Z=2.521, p=.012; object clefts: Z=2.524, p=.012; object wh-questions Z=2.521, p=.012).

Figure 6.

Proportion of correct responses for object relatives, object clefts, and object wh-questions on production tests administered prior to and following clinician-delivered TUF. Data are from Dickey & Thompson (2007), Thompson et al. (2003), and Thompson et al. (2008). Error bars, which represent standard errors (SEs), have been inserted for the group mean scores.

In order to compare the results of TUF and Sentactics®, differences in pre- and post-treatment scores were calculated and compared for each sentence type between the two treatments (see Figure 7). This comparison showed a numerically larger benefit of Sentactics® on the trained form (object relatives), in the face of a numerically larger benefit of clinician-delivered TUF on generalization to untrained forms (both object clefts and object wh-questions). However, none of these differences between the two delivery methods was statistically significant (object relatives: Z=1.753, p=.081; object clefts: Z=1.553, p=.142; object wh-questions: Z=1.807, p=.081).

Figure 7.

Percent change from pre- to post-treatment on production of trained object relative (OR) and untrained object cleft (OC) and object wh-question (OWH) structures derived from clinician-delivered TUF and Sentactics®. Error bars, which represent standard errors (SEs), have been inserted for the group mean scores.

The pre- to post-treatment comprehension data for clinician-delivered TUF are shown in Figure 8. For object relative structures, comprehension improved from a mean of 62% (range: 20-100%) pre-treatment to a mean of 89% (range 78-100%) post-treatment. Object clefts improved from a pre-treatment mean of 44% (range: 10-90%) to a mean of 78% (range: 65-100%). In contrast, comprehension of object wh-questions was little improved with treatment, changing from a mean of 79% (60%-100%) pre-treatment to 88% (range: 40-100%) post-treatment. Statistical analyses of these data indicated a significant effect for both object relatives and object clefts, but not for object wh-questions (object relatives: Z= 2.201, p=.028; object clefts: Z=2.371, p=. 018; object wh-questions: Z=1.1595, p=.246).

Figure 8.

Proportion of correct responses for object relatives, object clefts, and object wh-questions on comprehension tests administered prior to and following clinician-delivered TUF. Data are from Dickey & Thompson (2007), Thompson et al. (2003), and Thompson et al. (2008). Error bars, which represent standard errors (SEs), have been inserted for the group mean scores.

To compare the results of clinician-delivered TUF and Sentactics®, differences in pre- and post-treatment scores were calculated and compared for each sentence type (see Figure 9). This comparison revealed no significant comprehension differences between TUF and Sentactics® for any of the sentence types (object relatives: Z=.215, p=.836; object clefts: Z=1.285, p=.234; object wh-questions: Z=.1438, p=.945).

Figure 9.

Percent change from pre- to post-treatment on comprehension of trained object relative (OR) and untrained object cleft (OC) and object wh-question (OWH) structures derived from clinician-delivered TUF and Sentactics®. Error bars, which represent standard errors (SEs), have been inserted for the group mean scores.

Comparison of number of sessions

The difference between clinician-delivered TUF and Sentactics® with regard to the number of sessions required to reach criterion also was examined. On average, participants using Sentactics® required 13.67 sessions to reach criterion (range: 6-24, sd=2.89) and those receiving clinician-delivered TUF required and average of 14.25 sessions (range: 6-24, sd=2.28). Statistical analysis indicated no significant differences between groups (Mann Whitney Z=.1302, p=.949)

Discussion

This study examined the effects of Sentactics®, a computer automated version of TUF, on production and comprehension of complex sentences with wh-movement. Results showed that training with Sentactics® improved performance, with acquisition and generalization patterns similar to those derived when TUF is delivered by human clinicians. That is, training of complex object relatives resulted in significantly improved comprehension and production of both trained and untrained object relative structures and the participants trained with Sentactics® improved significantly more than control participants, who did not receive treatment of any kind. In addition, Sentactics® resulted in generalized gains in comprehension and production of linguistically related structures (i.e., object clefts and/or object-extracted wh-questions). All, but one participant (S1), showed this pattern. Notably, S1 presented with a WAB AQ of 46.5, which is lower than most participants who respond favorably to TUF. Indeed, previous studies show that participants with AQs below 50 do not show the same treatment effects as those with AQs above 50: more severely impaired patients show similar acquisition patterns, but little or no generalization results from treatment (Ballard & Thompson, 1999). Considered collectively, these findings add to the body of literature that supports the Complexity Account of Treatment Efficacy (CATE) (Thompson et al., 2003). That is, training complex sentence structures results in generalization to less complex, linguistically related structures, even when training is computer-delivered.

To further examine the generalization effects of Sentactics®, performance on other language measures was evaluated following treatment. One measure, the NAVS, tested comprehension and production of noncanonical structures. The NAVS used the same sentence-picture matching and sentence production priming procedures for testing comprehension and production, respectively. However, the NAVS was human-clinician administered and all Sentactics® tests were computer-delivered. The noncanonical (wh-movement) forms tested by the NAVS also differed slightly from those tested with Sentactics®. Sentactics® tested the object relative structure: “Pete saw the man who the woman saved”; whereas, the NAVS tested the object relative structure: “There is a man who the woman is saving”. The NAVS also included canonical structures not included in Sentactics® probes (i.e., subject-extracted wh-questions, e.g., “Who is the man saving?”, and subject relatives, e.g., “There is a woman saving the man”). Results showed that Sentactics® improved performance on both production and comprehension tests of the NAVS. Although the main improvements were noted on object relatives and object wh-questions, improvement also was observed on subject wh-questions and subject clefts. These data indicate that improvements noted secondary to Sentactics® generalized across tasks and different sentence types.

Generalization to narrative language also resulted from Sentactics®, with significant increases noted from pre- to post-treatment in mean length of utterance and the proportion of complex sentences produced. In contrast, no significant changes were noted across test points for the control participants on these or other narrative language variables. These data suggest that TUF not only improves production and comprehension of trained sentences types (and their linguistically related counterparts), but also that it improves the quality of spoken discourse. However, because differences between pre-and post-treatment changes did not differ significantly for the Sentactics® compared to control participants, further research is required. That is, to completely understand the extent of generalization to narrative discourse studying a greater number of aphasic participants is needed.

This study also compared the results from Sentactics® to those obtained from studies using clinician-delivered TUF. In comparing computer-delivered Sentactics® and clinician- delivered TUF, we found no significant differences between any of the production or comprehension measures, indicating that improvement is not impacted by the mode of treatment delivery. Nonetheless, there was a numerical difference between the trained and untrained forms: participants trained with Sentactics® showed larger gains on the trained form (i.e., object relatives); whereas, those trained with TUF showed larger gains on the untrained forms (i.e., object clefts and object wh-questions). One possible explanation for this discrepancy may relate to differences in the daily treatment probe task that was used for Sentactics® compared to that used for clinician-delivered TUF. The former probed only one structure – object relatives – each day prior to treatment; the latter probed all three structures daily. Thus, it is possible that the generalization effects found with TUF reflected a practice effect. This possibility, however, is unlikely because the number of object relative probes did not differ between Sentactics® and TUF, yet lesser improvement was noted with TUF. Rather, it is more likely that presenting a variety of structures during probing, as in the clinician-delivered TUF probes, may have enhanced participants’ ability to discriminate and attend to differences in target sentences. Further research is needed to fully understand the differences, if any, between Sentactics® and clinician-delivered TUF with regard to acquisition of trained and untrained sentence types.

Another interesting finding derived from the present study was that there was no difference in the amount of treatment required to reach criterion for computer-delivered and clinician-delivered TUF. That is, there was no statistical advantage for one or the other approach with regard to the number of one-hour of training session required.

More broadly, the present data have implications for computerized approaches to aphasia intervention, in general. That is, the present findings suggest that computer adaptations of other well-researched aphasia treatments also may result in treatment gains similar to those derived from human clinicians and highlight the need for research examining the relative effectiveness of the two approaches. We suggest, however, that the full benefit of computer-delivered treatment remains to be demonstrated and we caution clinicians against prescription of such treatments that are without efficacy data to support their use.

Conclusion

Results of this study show that computer-delivered TUF results in effects that do not differ significantly from those derived from clinician-delivered TUF and, therefore, suggest that computer-delivery is indeed a viable option for delivery of TUF. These data also suggest that computerized aphasia treatment, in general, may be equally effective to more traditional human clinician approaches. We, however, emphasize the importance of and need for further research examining the effects of the two.

Figure 10.

Number of sessions required to reach criterion for Sentactics® and clinician-delivered TUF. Error bars, which represent standard errors (SEs), have been inserted for the group mean scores.