Self-Monitoring During Spelling Practice: Effects on Spelling Accuracy and On-Task Behavior of Three Students Diagnosed With Attention Deficit Hyperactivity Disorder

Abstract

We used a multiple baseline across subjects design to investigate the effects of self-monitoring on the on-task behavior and spelling accuracy of three fifth-grade students diagnosed with attention deficit hyperactivity disorder (ADHD) during an independent spelling practice period in the general education environment. We also compared their on-task levels to that of peers without disabilities in the same classroom context before and after the self-monitoring intervention. Our results showed that time on-task and spelling accuracy increased for the students diagnosed with ADHD during independent spelling practice after the teacher taught them how to self-monitor. These same students' on-task behavior also increased to levels comparable to that of their peers. Suggestions for educators interested in teaching their students self-monitoring strategies are included following a discussion of the results of the analysis.

Researchers have estimated that between 2 and 18% of children currently exhibit behaviors associated with attention deficit hyperactivity disorder, commonly known as ADHD (Rowland, Lesesne, & Abramowitz, 2002). Behaviors characteristic of ADHD are commonly identified during childhood, but the adverse symptoms often continue through adolescence and adulthood (National Institute of Mental Health, 2008). Individuals diagnosed with ADHD engage in behaviors often characterized as inattentive, hyperactive, or impulsive.

An estimated 80% of school-aged children with a label of ADHD also have a variety of learning and behavioral challenges (Reid, Maag, Vasa, & Wright, 1994). A number of treatments have been proven effective for treating problem behaviors associated with ADHD (NIMH, 2008). Medication is the most common type of intervention used to treat the associated behavior problems, but behavior modification interventions are also effective when students diagnosed with ADHD are taught how to use them (Pfiffner, Barkley, & DuPaul, 2006; Reid, Trout, & Schartz, 2005). In a number of studies, students taught to use behavior modification methods to remediate behaviors associated with ADHD outperformed those who were given medication alone (Pfiffner et al.; Reid et al., 2005).

Self-monitoring is one of the more frequently used behavior modification methods in education (Reid et al., 2005). When educators teach students to self-monitor, they first teach them how to observe a target behavior and then record whether they engaged in the behavior or not (Nelson & Hayes, 1981).

Self-monitoring interventions can typically be classified into two broad categories: self-monitoring of on-task behavior or of a more specific performance (e.g., number of math problems accurately completed; Harris, Friedlander, Saddler, Frizzelle, & Graham, 2005; Rafferty & Raimondi, 2009; Reid, 1996). Although research indicates that students with ADHD benefit from both types of self-monitoring interventions (see Reid et al., 2005, for a review), results from Harris et al. suggest that students with ADHD may perform better academically when self-monitoring on-task behavior when compared to monitoring more specific performance during independent spelling practice activities.

In their study, Harris et al. (2005) evaluated the differential effects of teaching students with ADHD to monitor their on-task behavior versus monitoring the number of words they accurately practiced from their weekly spelling list during each independent practice period. Results from the study suggest that both interventions had positive effects on students' on-task behavior; however, a majority of the students accurately practiced more words using the self-monitoring of on-task behavior intervention. These researchers did not report spelling test performance data due to possible ceiling effects and ethical concerns related to testing students above their instructional levels. Nevertheless, the weekly spelling test data would have been useful for validating the improvement in on-task behavior for these students.

Harris et al. (2005) implemented their study in a general education environment, which is commendable because a majority of students with disabilities are being educated in this classroom setting at least 80% of the school day (U.S. Department of Education, 2007); however, these authors did not collect or report any social comparison data on their participants' on-task behaviors. Investigating the effects that interventions have upon the behaviors of students with disabilities when compared to general education students in the same classroom environment can be important to determine if the changes in behavior are socially valid (Alessi, 1980; Gureasko-Moore, DuPaul, & White, 2006; Wolf, 1978).

We examined the effects of a self-monitoring intervention of on-task behavior during independent spelling practice with three, elementary-aged students diagnosed with ADHD. We measured the effects of the intervention on levels of on-task behavior and spelling accuracy. On-task levels were also compared to those of peers without disabilities in the same general education classroom context.

Methods

Participants and Setting

Prior to recruiting participants for this study, we obtained project approval from the Human Subjects Institutional Review Board. Six students from a rural elementary school in the northeastern part of the United States who received academic intervention services for English Language Arts, including spelling instruction, participated. The students ranged in age from 10 to 11 years. All sessions took place at the end of the school day when students received 90 min of enrichment instruction.

Target students. Three of the students served as the target students in this study, and the teacher taught each target student how to use the self-monitoring intervention. Each target student met the following criteria prior to being included in the study: (a) a diagnosis of ADHD and an educational disability as defined by IDEA (2004); (b) age between 10 and 11 years; (c) spelling scores at least one year below grade level; and (d) a history of off-task behavior during independent spelling practice. Table 1 includes detailed information about the participants.

Table 1.

Characteristics of Target Students and Comparison Peers

	Gender	Age	Diagnosisa	Grade Level	Spelling Grade Levelb	Race	Medicationc (yes/no)
Target Student Craig	Male	10 yrs	ADHD/LD	5	3.8	Caucasian	yes
Comparison John	Male	10 yrs	–	5	3.1	Caucasian	no
Target Student Jenny	Female	10 yrs	ADHD/OHI	5	3.6	Caucasian	no
Comparison Kim	Female	10 yrs	–	5	3.6	Caucasian	no
Target Student Dan	Male	10 yrs	ADHD/OHI	5	3.8	Caucasian	yes
Comparison Allen	Male	11 yrs	–	5	3.6	Caucasian	no

Note. ^a The acronym LD stands for learning disability and OHI for other health impairment.

^b The “spelling grade level” indicates the grade level at which the participant scored on the Woodcock Johnson III (WJ-III) spelling achievement test.

^c The target students who took medication for symptoms related to ADHD did not receive any changes to their medication during the course of the study. According to their parents, these individuals took their medication as prescribed each day.

Comparison peers. The other three students were included for comparison reasons; the teacher did not teach these students how to use the self-monitoring intervention. The classroom teacher identified potential comparison peers based upon the following criteria: (a) age between 10 and 11 years; (b) a history of high levels of on-task behavior during independent spelling practice; (c) not receiving special education services; and (d) spelling scores at least one year below grade level. Comparison peers were arbitrarily matched with target students with the exception that gender was considered.

Spelling Study Task

The teacher implemented all experimental procedures. Monday through Thursday, for the first 15 min of the 90 min instructional period, the students spent time studying their weekly spelling words. Each student received 20 spelling words a week from the spelling lists provided in the Harcourt Trophies Reading Series (2005).

The teacher taught a six-step spelling strategy to all students in the classroom—including the participants—one month prior to the beginning of data collection. She told all participants in the study to use this strategy during the independent spelling practice sessions. Prior to the collection of any baseline data, the teacher assessed each student to make sure he/she was able to implement the strategy with fidelity. In this study, we used the same modified steps of a spelling study strategy used by Harris et al. (2005). The steps were as follows: “(a) look at the word, (b) close your eyes and spell the word aloud, (c) study the word again, (d) cover the word, (e) write the word three times, and (f) check to see if the word is spelled correctly” (Harris et al., p. 149). The teacher listed the study strategy steps on the board. This list was visible to all students in the classroom, and it was visible every day that the students practiced their spelling words.

Measurement

During each 15-min observational period, the teacher, who served as the primary observer, and the classroom aide, who served as the secondary observer, used a momentary time sampling procedure to measure the occurrence of on-task behavior for each of the target students and comparison peers. They observed participants every 3 sec alternating between the target students and comparison peers (Alberto & Troutman, 2003). For example, they observed target student one, then comparison peer one. Next, they observed target student two, followed by comparison peer two, and continued that pattern throughout the spelling practice period. This system allowed for on-task behavior to be measured for each child 50 times per 15 min observation. The teacher and the classroom aide relied on an audio player that emitted brief tones via split headphones to signal them to observe and record the occurrence or nonoc-currence of on-task behavior for a single participant at the end of each interval. We converted those data into percentages by dividing the number of intervals each participant was observed on-task by the total number of intervals and multiplied by 100.

On-task behavior was defined as any of the following: (a) writing on the spelling practice list or the self-monitoring card, (b) looking at the spelling practice list or the self-monitoring card, (c) implementing any step of the spelling study procedure, or (d) asking the teacher or adult for directions by raising his or her hand and waiting quietly to be addressed by the teacher or adult. On-task behavior was not recorded if the student was observed: (a) doodling/sketching on the spelling practice list or self-monitoring card, (b) looking at any surfaces or objects other than the spelling practice list or self-monitoring card, (c) talking to their peers, (d) asking for help from their peers unless directed to do so, or (e) inappropriately asking for help from the teacher or adult (i.e., not raising his or her hand nor waiting quietly to be addressed by the teacher or adult).

Spelling accuracy is reported as the percentage of words the student spelled correctly on weekly spelling tests, which occurred each Friday. The accuracy percentages were obtained by dividing the number of words the target student spelled correctly by the total number of spelling words on the test, which was always 20 words.

Interobserver Agreement

The classroom teacher, who was also a co-author of the study, trained the classroom aide to collect data for agreement purposes. After discussing the behavioral definitions and reviewing the data sheet and recording procedures, the teacher and the aide practiced observing and recording the participants' on-task behaviors. The momentary time sampling recording sheets were then compared using an interval-by-interval reliability check, and any discrepancies between the data on the teacher's and the aide's sheets were discussed. The teacher and the aide repeated the steps until both the observers reached 100% agreement for two consecutive observations, then formal baseline observations were begun. It took approximately 30 min each day over a period of two days to reach criterion.

Agreement percentages were obtained by dividing the number of intervals in which both observers agreed on the on-task status of the student by the total number of intervals and multiplying the quotient by 100. The aide completed reliability checks for approximately 22% of the observational periods in this study. She completed at least one reliability check during the baseline and intervention conditions, for each participant. The mean inter-rater agreement for on-task behavior was 96% with a range of 93 to 100%.

Social Validity Measures

The primary measure of social validity in this study was the comparison of on-task data between those students with ADHD to children without ADHD and who were reported to engage in satisfactory levels of on-task during the independent spelling instructional period. Comparison students' data are graphically depicted with those of the target students.

A second measure of social validity was obtained after the study was completed from the teacher and the target students. The teacher asked each student the following questions in a one-on-one interview: (a) What did you think about the self-monitoring intervention? (b) What do you think other kids would think about the self-monitoring intervention and why? and (c) Is there anything else about this method you would like to tell me or other kids? A fellow researcher asked the teacher the following questions: (a) Tell me your impressions of the self-monitoring intervention? (b) Would you recommend this intervention to other teachers? Why or why not? and (c) Is there anything else about this intervention you would like to tell me or other teachers?

Experimental Design and Procedures

We used a multiple-baseline design across subjects to evaluate the effects of self-monitoring on the on-task and spelling performance of the students diagnosed with ADHD.

Baseline condition. Prior to beginning the independent spelling practice session, the teacher gave the participants their spelling lists and any other necessary materials, such as paper to practice their words and a writing utensil. Before students were prompted to begin practicing their spelling words, the classroom teacher reviewed the classroom rules and the spelling study strategy. No further direction was provided. Typically during this time, the teacher and classroom aide would spend time preparing for the remainder of the instructional day. There was no interaction between the teacher and the students during this time; therefore, there were no differential consequences provided by the teachers for on- or off-task behavior.

Target student training and intervention condition. We created and copied a number of self-monitoring cards on half-sheets of paper. The question, “Am I on-task?” was centered at the top of the card, and a t-chart was included below the question. We labeled the left side of the T-chart “yes” and the right side “no.” The chart itself was left blank for the students to fill in with tally marks. We also created a tape that randomly emitted short tones (average interval of 40 sec and range of 20 to 60 sec). Randomization was achieved by writing the multiples of five from 20 to 60 (e.g., 20, 25, 30, etc.) on separate pieces of paper and placing them in a hat. Numbers were drawn and written on a draft piece of paper. These numbers indicated the intervals between the short tones. The range of tones was chosen based upon the teacher's analysis of the target students' baseline data, which showed that the students' did not remain on-task for more than 3 consecutive intervals during baseline (i.e., for never more than 60 s).

The three students who were taught the self-monitoring procedure could hear the tones via headphones connected to the tape player. These tones were to serve as cues for the students to assess and record their on-task behavior. When target students heard a randomly emitted tone while practicing their spelling words, the teacher taught them to assess their on-task behavior by asking themselves, “Am I on-task?” They would then use their writing utensil to create a tally mark in either the “yes” or “no” column of the self-monitoring card based upon their self-observation. The teacher gave each student a new self-monitoring card every day. The teacher trained each target student individually on the self-monitoring strategy, and the first session in which the student used the self-monitoring intervention during the after-school program was the following day. We modified the training procedures described by Hallahan, Lloyd, Kosiewicz, Kauffman, and Graves (1979).

During the training session, the teacher used a procedural checklist to ensure each step of the training remained consistent across students. The steps of the checklist were as follows:

• (a) the teacher explicitly defined and modeled examples of staying on-task (e.g., looking at the list, looking at the self-monitoring card, tallying the self-monitoring card, asking a teacher for assistance by raising a hand);

• (b) the teacher defined and modeled examples of what on-task behaviors are not (e.g., doodling on the spelling list or self-monitoring card, looking anywhere other than the list or self-monitoring card, and talking to other students in the class);

• (c) the teacher demonstrated how to use the self-monitoring card;

• (d) the teacher demonstrated how to self-monitor and tally when the tone occurred; and

• (e) the student verbally described and physically performed each step of the intervention to the teacher with 100% accuracy for three consecutive trials.

After the training, the teacher instructed that target student to use the self-monitoring intervention in the general educational classroom during spelling practice. During practice time, neither the teacher nor the aide interacted with the students. Although the teacher monitored to make sure that the students continued to implement the steps of the spelling strategy and self-monitoring procedures correctly, none of the students were observed implementing either of the procedures incorrectly. Therefore, there were no differential consequences provided by the teacher for accurate self-monitoring or for on-task or spelling behaviors.

Weekly spelling tests. Students took a written spelling test each Friday. The teacher followed the same routine when implementing the class spelling test across both phases of the study; during the routine: (a) dividers were put into place between students; (b) students were told to take out a lined piece of paper and pencil; (c) students were given time to set up their papers with their name, date, and numbering down the page (1 through 10 on the front side and 11 through 20 on the back); (d) the teacher began the test by stating the first word, using it in a sentence, and repeating the word (this step was repeated for all 20 words); (e) students were given an opportunity at the end of the test to check over their words and ask to have any missed words repeated; and (f) the teacher collected the spelling tests.

Results

On-Task Behavior

The on-task behavior for each target student increased after the implementation of the self-monitoring intervention (see Figure). During the baseline condition, the mean percentage of on-task behavior for Craig, Jenny, and Dan was 47%, 52%, and 38%, respectively. After the intervention was implemented, the mean percentage of on-task behavior was 85%, 88%, and 80%, respectively.

Figure.

Percentages of on-task behavior for each target student and comparison peer.

On-task behavior was relatively high in baseline for the three comparison peers: 86% for John, 96% for Kim, and 94% for Allen. The mean on-task behavior for the comparison peers remained high, and increased slightly following the intervention for the target peers: 93% for John, 99% for Kim, and 97% for Allen. The Figure also shows that the gap between the target students' on-task levels and those of the comparison peers decreased after the intervention was implemented, with two of the three target children's on-task behavior being commensurate with that of their same-aged peer with an ADHD diagnosis. However, there were differences in on-task levels between target and comparison peers' on-task levels at the close of the evaluation.

Spelling Accuracy

As can be seen in the Figure, the number of words each target student accurately spelled increased on the weekly spelling tests after being taught how to self-monitor his/her on-task behavior. Although there were a limited number of assessments per child, all spelling accuracy was higher during the intervention period than during baseline. Craig's spelling accuracy increased by 33%. He went from failing grades to passing grades. During baseline, Jenny's grades were passing in the lower C range. After the intervention, her average accuracy increased by 15%, which is in the high B range. Dan also went from failing grades during baseline to passing grades following intervention; his accuracy average increased by 28%. (Permission to use the comparison peers' spelling test data was not obtained and are thus unavailable.)

Teacher Reflection

The teacher stated that she liked the intervention because she was able to implement it with resources that were easily available to her, and it was not time consuming for the students. She appreciated that the students were able to remain on-task for a longer period of time, and she was particularly pleased with the increases in their spelling performance after being taught the self-monitoring strategy. The teacher went on to reflect that she would continue to use the intervention in her classroom.

Student Interviews

All of the target students indicated that the intervention helped them pay more attention to their work. These students told the classroom teacher they felt more prepared for their spelling tests and scored better on them after starting the intervention. The target students agreed that the tones used to cue them to self-monitor helped them. However, one target student, Dan, mentioned that he was a little embarrassed about wearing headphones and that sometimes the tones distracted him and made him lose focus; however, he did see the value in the intervention and indicated that he would like to continue to use it in the future.

Discussion

The self-monitoring intervention improved the on-task behavior and spelling accuracy of three children diagnosed with ADHD. In addition, levels of on-task behavior following intervention for two of the three children were generally consistent with those of same-aged peers without disabilities who were observed in the same classroom. Each of the students commented that the intervention helped them concentrate on their studying more, and the teacher commented favorably about the intervention and the effects.

Results from the present study provide additional support that students diagnosed with ADHD can benefit from self-monitoring on-task behavior in general education settings (e.g., Davies & Witte, 2000; Edwards, Salant, Howard, Brougher, & McLaughlin, 1995), particularly during independent spelling practice (Harris et al., 2005). This is important because students with a diagnosis of ADHD typically exhibit behaviors described as inattentive, hyperactive, or impulsive, which often pose challenges for these students in the general education classroom where teachers expect students to be able to regulate their behaviors independent of adult supervision (Bussing, Gary, Leon, Wilson, & Reid, 2002). Interventions, such as the self-monitoring one described in the current study, that help students succeed in the classroom and are not labor intensive or require many resources to implement are more likely to be adopted by teachers of children diagnosed with ADHD. In addition, finding interventions that help students with behavior deficits perform at levels similar to their peers is important in their quest to make academic gains and be socially accepted in the classroom (Bussing et al., 2002; Reid et al., 2005). The results from this study show that self-monitoring may be an effective component of treatments aimed at developing classroom repertoires similar to those of children of typical development.

The target students reported an overall satisfaction with the self-monitoring intervention and its effects on their spelling performance. The one negative bit of feedback had to do with the wearing of the headphones, and this feature could be modified in future implementation. Because having students buy into an intervention is important (Horner et al., 2005), a tactile prompting device could also be used to cue the students to self-monitor as an alternative to an auditory cuing mechanism. One such device is the MotivAider (http://habitchange.com/), which vibrates at set intervals to cue students to self-monitor. Preliminary research results suggest that this might be an effective alternative to auditory cuing mechanisms (e.g., Amato-Zech, Hoff, & Doepke, 2006; Farrell & McDougall, 2008).

Limitations to Consider

The reader should interpret the findings of this study in light of some notable limitations. First, although students' spelling test data were included in this study, these data should be interpreted with caution due to standardization and reliability issues. Although the weekly words selected for the students were a part of the curriculum and selected according to the students' instructional skill level, the difficulty of the words from week to week may have differed. Additionally, the words were scored as either right or wrong, no partial credit was given. A better form of academic spelling measure might be to use curriculum-based measurement (CBM) probes for spelling. CBM assessments include standardized implementation and scoring procedures, and previous research results of Deno (2003) indicate that these measures are reliable and valid.

Second, it is not possible to determine which variables were responsible for the efficacy of the self-monitoring intervention. For example, the use of the headphones was not kept constant between both conditions of the study. Therefore, it is possible that the use of the headphones may have minimized distractions for the students while studying, which could have affected their time on-task independent of the other components of the self-monitoring intervention. Also, explicitly training students to discriminate on-task and off-task behavior, regardless of providing them with the self-monitoring materials, could have resulted in some behavior change. Nevertheless, the self-monitoring package still represents an effective and practical tool for teachers to promote on-task behavior during study periods.

Third, the teacher was only able to implement the intervention condition of this study for a short duration of time. Questions, therefore, remain about the durability of the intervention, whether the intervention would eventually result in on-task levels consistent with those of their same-aged peers, and whether or not additional intervention components would need to be added to achieve sustained and socially valid outcomes.

Fourth, time was also not available in this study to evaluate procedures for systematically fading the external supports for this intervention (i.e., cuing mechanism, self-monitoring card). To do so, one might consider, for instance, lengthening the time interval between cues gradually until the cues are no longer used.

Implications for the Classroom

An individual's ability to self-monitor his or her own behavior can improve his or her academic and social outcomes (Lan, 2005). Therefore, educators should strongly consider teaching their students how to self-monitor. Teachers are encouraged to choose the type of self-monitoring intervention after carefully considering a student's individual strengths, needs, and goals (Harris, Graham, Reid, McElroy, & Hamby, 1994; Harris et al., 2005).

Table 2 includes an abbreviated list of steps that teachers should consider following when developing a self-monitoring intervention (modified from Cooper, Heron, & Heward, 2007; Hallahan et al., 1979; Harris et al., 1994; Maag, Reid, DiGangi, 1993; Rafferty, 2010; Rafferty & Raimondi, 2009; Rankin & Reid, 1995). Although the current study was conducted in a general education classroom, any teacher, in any classroom context, can use these steps.

Table 2.

Steps to Creating and Implementing a Self-Monitoring Strategy in the Classroom.

Step	Description	Questions to Ask
1	Identify the behavior in need of remediation.	✓ What behavior am I trying to remediate?
2	Operationally define the target behavior.	✓ Is my description of the behavior written in measureable and observable terms? ✓ Would someone else be able to use my definition without having any questions?
3	Determine how you will collect data.a	✓ Can I count the number of times this behavior occurs or am I more interested in measuring duration (i.e., how long a behavior occurs) or latency (how long it takes for a student to engage in a behavior after a directive is given)?
4	Collect baseline dataa	✓ Did I collect data on at least three to five different occasions? ✓ Are the data I collected relatively stable?
5	Determine if it is appropriate to remediate the behavior.	✓ Does the student already possess the skills to engage in the behavior I would like him/her to do (i.e., is this a performance deficit)? ✓Does this behavior occur frequently enough to remediate using this type of intervention?
6	Design procedures and all materials.b	✓ Will I have the student self-monitor during the activity or after? ✓Will I need to create a cuing mechanism, and if so, how often should he/she be cued to self-observe? ✓ Will I need to create answer keys if I am having the student self-monitor some form of his/her academic accuracy? ✓ Will I teach the student how to graph his/her performance over time, and if so, what type of graph will I make?
7	Teach student how to self-monitor.	✓ Do my lesson plans include teacher modeling, guided practice, and independent practice components?
8	Monitor student's progress.	✓ How often will I monitor the student's progress to make informed instructional decisions?
9	Fade use of inter-vention and monitor maintenanceb	✓ What are the steps that I will take to systematically fade the use of the self-monitoring materials the student is using? ✓ How often will I monitor the student's behavior to make sure that he/she is still engaging in appropriate levels of behavior even after the intervention has been faded?

Notes. ^a See Cooper, Heron, and Heward (2007) text for more information about data collection methods and procedures.

^b See Hallahan, Lloyd and Stroller (1982) text for more information about materials and fading procedures.