Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2019 Feb 15.
Published in final edited form as: J Early Interv. 2018 Apr 27;40(2):177–191. doi: 10.1177/1053815118771391

Special Education Teachers’ Perceptions and Intentions Toward Data Collection

Lisa A Ruble 1, John H McGrew 2, Wing Hang Wong 1, Kristen N Missall 3
PMCID: PMC6377170  NIHMSID: NIHMS984466  PMID: 30774283

Abstract

Although data-based decision making is an evidence-based practice, many special educators have difficulty applying the practice within daily routines. We applied the Theory of Planned Behavior (TPB) to understand the influences that promote or hinder early childhood special educators’ intentions to collect data. We assessed three influences on behavioral intention to collect data derived from the TPB: (a) attitude toward collecting data, (b) social norms for collecting data, and (c) perceived behavioral control for collecting data. All three influences correlated positively with teachers’ reported intention to collect data; however, only perceived behavioral control of barriers correlated positively with actual data collection. Additional measures of teacher self-efficacy and administrative support correlated positively with intention to collect data, but not with actual data collection behaviors. Perceived behavior control accounted for the most variance in actual data collection behavior. Implications of the findings for data collection practices in educational settings are discussed.

Keywords: theory of planned behavior, data collection, self-efficacy, special education

Introduction

Data collection is a critical feature of evidence-based educational practice. Educational reform efforts such as the Every Student Succeeds Act (U.S. Department of Education, 2009) and federal legislation such as the Individuals with Disabilities Education Improvement Act (2004), commonly referred to as IDEA, incorporate accountability and quality at their base. Data collection forms the basis for evaluating and informing student learning outcomes and the effectiveness of teaching practices (Brawley & Stormont, 2013), and provides the foundation of teaching quality. Despite its importance, however, data collection remains a challenge for many teachers (Mandinach, Gummer, & Northwest, 2012).

Researchers have observed a disconnection between teachers’ beliefs about the importance of data collection and data collection practices (Brawley & Stormont, 2013; Sandall, Schwartz, & LaCroix, 2004). Sandall and colleagues (2004) examined early childhood special educators’ attitudes toward the use of data for making instructional decisions and collaborating with families. Although teachers agreed that data collection was critical, they reported inconsistent data collection practices in their classrooms. Similarly, in a survey of early childhood special educators and Head Start teachers on their data collection perceptions and practices, about 90% of teachers reported data collection as “very important,” but only 66% reported using data for progress monitoring, and less than 33% reported analyzing and summarizing data for children in early childhood settings (Brawley & Stormont, 2013). The results from these two studies identify a mismatch between perceptions and data collection practices.

Understanding teacher perceptions of the barriers related to data collection is important to be better able to support the widespread practice in everyday classrooms. Barriers identified by teachers include negative teacher attitudes and beliefs (Ingram, Louis, & Schroeder, 2004), frustration with paperwork requirements (Billingsley, 2004; Fore, Martin, & Bender, 2002; Zabel & Zabel, 2001), and general lack of knowledge and expertise (Martinez & Young, 2011) associated with data collection. Other barriers include a lack of time, collecting data during instruction, and managing data (Brawley & Stormont, 2013).

For special education teachers, data collection may be particularly challenging as efforts to document progress on individualized education program (IEP) goals are not necessarily uniform, identical, or standardized across students and settings. Moreover, because IEP goals are unique for each student, an idiographic assessment approach is necessary. That is, standardized assessment and curriculum-based assessment approaches are not always appropriate or readily available for measuring progress on some skills (i.e., social skills, communication skills, learning readiness skills; Ruble, McGrew, & Toland, 2012). Therefore, measuring outcomes of instructional plans for these types of skills often places the burden of identification of a data collection approach directly on the special educator. Accordingly, special educators need a broad array of data collection and analysis skills to monitor student progress and to make critical, child-specific, data-based decisions about the effectiveness of interventions and educational programs (Gischlar, Hojnoski, & Missall, 2009; Yell, Drasgow, & Lowrey, 2005).

We became interested in teachers’ views of data collection for monitoring IEP goals after observing wide variability in teachers’ data collection efforts as part of a study that tested a parent-teacher consultation intervention with follow-up coaching referred to as the Collaborative Model for Promoting Competence and Success (COMPASS) for young students with autism spectrum disorder (ASD) (Ruble, Dalrymple, & McGrew, 2012). In our first randomized controlled trial (RCT) of COMPASS (Ruble, Dalrymple, & McGrew, 2010), we observed that most teachers were not collecting data (i.e., only 21% were collecting data at the second consultation session following goal development) and as a result we introduced discussion and planning for data collection as an additional prerequisite element of IEP implementation into our standard coaching sessions. After the addition of this element to consultation, data collection improved to 40% of IEP goals by consultation session 3, 67% of goals for session 4, and 53% of goals for session 5; see Ruble, Dalrymple, & McGrew (2010) for a full description. Thus, over the entire school year, data collection for IEP goals increased, but then decreased and remained variable and low overall. In planning our second RCT (Ruble, McGrew, Toland, Dalrymple, & Jung, 2013), we wanted to investigate teachers’ attitudes toward data collection for IEP goals and factors associated with the likelihood of some teachers collecting data versus others. Using the TPB, we developed a measure to assess influences on intentions to collect data for IEP goals and prospectively applied it. In the following section, we describe our rationale for selecting the TPB.

The Theory of Planned Behavior

The TPB (Ajzen, 1991) is a conceptual framework for understanding social and intrapersonal influences on intention to perform specific behaviors (Francis et al., 2004) and can serve as a useful platform for theory-driven research in education (Mercer, Idler, & Bartfai, 2014; Volpe & Suldo, 2014). The TPB is a social-cognitive theory that has been used extensively in health services research to understand the knowledge transfer and uptake of evidence-based practice within healthcare settings (Francis et al., 2004). More recently, however, the TPB has been applied to educational problems such as understanding principals’ intentions toward inclusive education (Yan & Sin, 2015), peers’ intentions regarding classmates with ASD (Freitag & Dunsmuir, 2015), and adolescents’ intentions toward a leisure-time physical activity within a school-based intervention program (Chatzisarantis, Kamarova, Kawabata, Wang, & Hagger, 2015).

Within TPB, behavioral intention is modeled as a function of three variables: (a) the person’s attitude toward the specific behavior (Attitude; AB); (b) subjective normative beliefs about the specific behavior (Subjective Norm; SN), and (c) perceived behavioral control (PBC) of the specific behavior (see Figure 1). As shown in the figure, both AB and SN are theorized to impact intention directly and behavior indirectly, through intention. In contrast, PBC is theorized to have a direct impact (i.e., not mediated by intention) on both intention and actual behaviors. According to the TPB, these factors, both individually and collectively, predict behavioral intention and behavior. Armitage and Conner (2001) examined 185 independent studies using the TPB and found that it explained 39% of the variance in intention and 27% of the variance in behavior.

Figure 1.

Figure 1

Open in a new tab

Theory of Planned Behavior.

By investigating the factors outlined in the TPB as potentially related to data collection for IEP goals, we propose that researchers may be able to develop ways to increase teachers’ intention for use of data collection practices in the classroom. Specifically, as illustrated in Figure 1, a teacher may be more likely to collect data if s/he believes that data collection is a necessary process for obtaining optimal student learning outcomes (AB); believes that colleagues value data collection (SN); and believes it is possible for her/him to collect data (PBC).

In the current study we applied the TPB to develop a deeper understanding of the relation between the variables that facilitate and hinder teachers’ intention to collect data for IEP goals and data collection behavior. Additionally, we evaluated the variables of self-efficacy and administrative support in relation to intention and behavior.

Self-efficacy is the belief that one can achieve what one intends (Bandura, 1997). It is aligned closely with perceived behavioral control and is often included as an additional variable in studies using TPB. According to social-cognitive theory (Bandura, 1989), people who feel they are able to meet specific challenges exert more effort, are more willing to persist and persevere, and are better equipped to cope with setbacks than those who do not (Bandura, 1991). Self-efficacy is not a general trait, meaning that it varies across different domains and activities. In the teaching profession, self-efficacy encompasses beliefs about one’s capabilities to deliver content, manage classroom environments, and engage students. Moreover, teacher self-efficacy is assumed to influence student outcomes indirectly through specific teacher behaviors (Bandura, 1997; Klassen, Tze, Gordon, & Betts, 2011).

Administrative support may play a positive role in helping offset some of the barriers to data collection faced by special education teachers. Ruble, Usher, and McGrew (2011), for example, surveyed 35 special education teachers of children with autism and found an inverse correlation (r = -.44, p < .01) between perceived administrative support and teacher emotional exhaustion (i.e., burnout). Administrative support also has been related positively to commitment and job satisfaction among special education teachers (Billingsley & Cross, 1992) and to teachers’ decisions to use new instructional methodologies (Corrigan, Steiner, McCracken, Blaser, & Barr, 2001). Thus, administrative support may be helpful in encouraging teachers to make organizational and procedural changes, such as replacing untested or invalidated practices with evidence-based data collection practices (Rosenholtz, 1989) and developing an atmosphere supporting positive attitude and creating social norms for data collection.

Current Study

The purpose of the current study was to apply the TPB to investigate external and internal factors associated with special education teachers’ views of data collection for IEP goals and data collection behavior. We had a total of six research questions. Three research questions addressed intention to collect data (questions 1–3), two addressed data collection behavior (questions 4–5), and one addressed reasons for collecting data. First, we asked whether teachers’ attitudes toward collecting data (AB), social norms of collecting data (SN), and perceived control of barriers that might hinder collecting data (PBC) were related to teacher intention to collect data for IEP goals. Consistent with the TPB, we expected AB, SN and PBC to be related positively with intention to collect data for IEP goals. Second, we asked whether teachers’ perceptions of administrative support and self-efficacy for data collection were related to intention to collect data. We expected administrative support and self-efficacy to be related positively to intention to collect data. Third, we examined how much variance in intention to collect data could be explained by the study variables (AB, SN, PBC, self-efficacy, and administrative support). We expected all variables to account for intention to collect data. Fourth, we asked to what extent are intention to collect data, perceived behavior control, self-efficacy, and administrative support related to data collection behavior. We hypothesized that each of the variables would be related positively to data collection behavior. Fifth, we asked how much variance in data collection behavior can be explained by the variables significantly related to data collection behavior. We expected results to be similar to those reported by Armitage and Conner (2001), which averaged 27% of variance. Lastly, we asked about teachers’ reasons for collecting data. This last question provided additional context for understanding the results.

Method

Participants

Data for the study were collected as part of a randomized controlled trial of a teacher consultation intervention. Public school special education teachers responsible for the IEPs of students with autism aged 3 to 8 years old were recruited from one Midwestern state and one Southern state. School administrators provided permission for researchers to contact teachers directly. A total of 44 special education classroom teachers who were the case managers overseeing the IEPs of students with autism were recruited and were randomly assigned either to the experimental or control group (see Ruble, McGrew et al., 2013 for consort diagram). All but one teacher was female. Each of the teachers was state certified; about 45% of teachers reported their highest degree was a bachelor’s degree and 47% a master’s degree (8% did not report degree earned). Thirty-one percent of the teachers taught inclusive preschool (67% of these teachers also taught in inclusive elementary settings), 14% taught in a resource room for students with disabilities to access intermittently, and 52% taught in a special education classroom for students with disabilities only (3% did not report classroom type). None of the classrooms was limited to the instruction of children with autism only. Overall, about 30% of teachers taught in inclusive classrooms and about 66% of teachers taught in seclusionary classrooms. The average class or caseload size was 12.4 (SD = 5.3). Participants averaged 5.70 years (range = < 1 to 20; SD = 5.68) of teaching experience with students with ASD; however, experience was skewed positively due to two highly experienced teachers, who had 17 and 20 years teaching experience respectfully.

After recruitment, all participants received a baseline assessment (Time 1) prior to group assignment (see next section for description of measures). After this initial assessment, teachers were allocated randomly to one of three groups: (a) placebo control where they were asked to complete online training in three evidence-based autism interventions; (b) COMPASS consultation with follow-up face-to-face teacher coaching; and (c) COMPASS consultation with follow-up web-based coaching (see Ruble, McGrew et al., 2013). At the end of the school year, all participants received a final assessment (Time 2).

Measures and Procedures

Measures used for the current sub study were administered at the end of the school year (Time 2), except where noted otherwise.

Teacher Intention toward Data Collection Efforts (TIDE) is a 17-item questionnaire developed for the current study. Instructions for creating scales to assess the elements of the TPB were used as a guide (http://people.umass.edu/aizen/pdf/tpb.measurement.pdf). Teachers used a 6-point Likert scale to rate their subjective norms (9 items), attitudes (3 items), and perceived behavioral norms (4 items) toward data collection and behavioral intention to collect data. Table 1 lists the items and participants’ ratings means and standard deviations. In the current sample, the internal consistency (α) was .82 for AB, .90 for SN, and .88 for PBC. For PBC, teachers answered four questions on the barriers to data collection. Using a 6-point Likert scale (1 = strongly disagree, 6 = strongly agree), teachers rated whether the following acted as barriers to data collection: too little time, too much to do, unclear measurement systems, and too many students. Lastly, teachers completed two open-ended questions. The first question asked what plans they had for keeping data and the second question asked who looks at data and for what reasons.

Table 1.

Teachers’ Intention Toward Data Collection Efforts (TIDE) Items.

Attitude (AB) items Response format M SD
 1. For me to collect data during the next two weeks is 1–6; not important to extremely important 4.86 0.93
 2. Keeping data will help the child meet his/her IEP goal 1–6; completely false to completely true 4.77 1.12
 3. Children who obtain their IEP goals are more successful 1–6; completely false to completely true 4.91 1.03
Subjective norm (SN) items
 4. My administration believes that keeping data over the next 2 weeks is 1–6; not important to extremely important 5.00 0.94
 5. My co-workers believe keeping data over the next 2 weeks is 1–6; not important to extremely important 4.40 1.18
 6. Parents of my students believe keeping data over the next 2 weeks is 1–6; not important to extremely important 4.34 1.28
 7. My administration would support me in trying to keep data over the next 2 weeks 1–6; not at all likely to extremely likely 4.86 1.32
 8. My coworkers would support me in trying to keep data over the next 2 weeks 1–6; not at all likely to extremely likely 4.84 1.18
 9. Parents of my students would support me in trying to keep data over the next 2 weeks 1–6; not at all likely to extremely likely 4.59 1.35
 10. Parents pay very close attention to the data for each child 1–6; strongly disagree to strongly agree 3.48 1.11
 11. Co-workers pay very close attention to the data for each child 1–6; strongly disagree to strongly agree 3.66 1.36
 12. Upper administration pays very close attention to the data for each child 1–6; strongly disagree to strongly agree 3.80 1.44
Perceived behavioral control (PBC) items
 13. Too little time interferes with ability to keep timely data 1–6; strongly disagree to strongly agree 2.52 1.41
 14. Too much to do interferes with ability to keep timely data 1–6; strongly disagree to strongly agree 2.59 1.45
 15. Measurement systems that are unclear interfere with ability to keep timely data 1–6; strongly disagree to strongly agree 4.34 1.11
 16. Too many students interfere with ability to keep timely data 1–6; strongly disagree to strongly agree 3.14 1.29
Behavioral intention item
 17. I intend to keep data over the next 2 weeks 1–6; not at all likely to extremely likely 5.16 0.81
Open in a new tab

Self-efficacy was measured with two items extracted from the Autism Self-Efficacy Scale for Teachers (ASSET; Ruble, Toland, Birdwhistell, McGrew & Usher, 2013). The original ASSET is a 30-item self-report measure that assesses special education teachers’ beliefs in their ability to carry out professional tasks related to instructing students with autism. Items are rated using a 100-point rating scale (0 = cannot do at all; 100 = certain can do), as suggested by Bandura’s (2006) instructions for creating self-efficacy scales. A mean score across items is calculated, with higher scores reflecting higher self-efficacy. The internal consistency of the original, 30-item ASSET using Cronbach’s α was .96. For this study, we extracted two items that were pertinent to data collection: (a) collect data to monitor student’s progress toward objectives, and (b) make use of data to re-evaluate student’s goals or objectives. The mean of the two items was 74.52 (SD = 12.09). The internal consistency of the two items using Cronbach’s α was .93 for the current sample.

Administrative support was assessed with an 11-item Administrative Support questionnaire (Ruble & McGrew, 2013). Teachers were asked to rate the administrative support they received using a 4-point Likert scale (1 = not much support; 4 = very much support) in 11 areas (e.g., participating in training/continuing education; having flexibility to be creative or develop innovative programs; having adequate classroom staffing; having adequate planning time). The mean item score was used to assess overall support. The scale mean and standard deviation were 3.04 and .64, respectively. Internal consistency measured using Cronbach’s α was .89 for the current sample.

Data collection behavior (COMPASS group only)

The teachers (n = 29) receiving COMPASS participated in a series of four consultation sessions following an initial goal setting and planning meeting throughout the school year. During the consultation sessions, teachers were asked whether they were collecting data on each of the three skills targeted with COMPASS and if so, to provide a sample of their data as well as videotape observation of student progress. Teachers were assigned a score of 0 (no data collected) or 1 (data collected) for each of the three skills at each consultation session. The overall mean score obtained across the four consultation sessions was used in the analysis (M = .84, SD = .19).

Data Analysis

Quantitative analysis

For data analysis, we separated data based on group assignment, Sample 1, which included the entire sample of control and experimental group teachers (n = 44) and Sample 2, which included only the experimental group teachers (n = 29). Note that when analyses were conducted within Sample 1, because IEP goal quality and attainment were significantly higher within the experimental group, group assignment was controlled to adjust for any potential impact on the study variables. Thus, to answer the first two research questions, which examined the relationships between intention to collect data for IEP goals and elements of the TBP in question 1, and administrative support and self-efficacy in question 2, we used Pearson partial correlation controlling for group assignment as applied to Sample 1. To answer the third research question, which examined the unique impact of the predictor variables (i.e., AB, SN, PBC, administrator support, self-efficacy) on intention to collect data, we used multiple regression again applied within Sample 1. Variables significantly correlated with intention to collect data using partial Pearson correlations were entered into a standard regression after first entering group assignment as a control variable, and using “intention to collect data” as the dependent variable.

To answer question 4, which examined the potential relationships between data collection behavior and the TPB elements of intention PBC, self-efficacy, and administrative support, we used simple Pearson correlations as applied to Sample 2, because these data were available only for the experimental group. To answer question 5, which focused on determining the unique impact of the predictor variables (i.e., PBC, intention, administrative support, self-efficacy) on actual data collection, variables significantly correlated bivariately were entered into a standard regression with Data Collection Behavior as the dependent variable.

Qualitative analysis

Lastly, to examine the reasons teachers collect data, the relevant open-ended question was analyzed qualitatively (i.e., for what reasons do you collect data?) using content analysis (Graneheim & Lundman, 2004). Initially, one coder read a sample of the transcripts using open coding and identified three recurring concepts (i.e., fulfillment of requirements, progress monitoring, and educational decisions), which were developed into a codebook. The codebook author, together with another coder, applied the codebook independently to see whether both of them captured the same recurring concepts from the sample of transcripts. To check reliability, 20% of the transcripts were coded independently and compared by the first author and a trained graduate student. Reliability was established using percent agreement. The number of codes with agreement was divided by the total number of codes (those with agreement plus those without agreement) resulting in 92% agreement.

Results

Attitude

Overall, teachers reported positive attitudes toward collecting data. They reported that collecting data was important (M = 4.9 out of 6) and would help the child achieve IEP goals and be successful (M = 4.8).

Subjective Norm

Using a scale from 1–6 (not important to extremely important), teachers reported that administration (M = 5.0), coworkers (M = 4.4), and parents (M = 4.3) view data collection as important. Using another 1–6 scale (not at all likely to extremely likely), teachers reported that administration (M = 4.9), coworkers (M = 4.8), and parents (M = 4.6) would support them in collecting data. Teachers gave lower ratings for who pays attention to data. Using a scale from 1–6 (strongly disagree to strongly agree), teachers were slightly more likely to agree than disagree that parents (M = 3.5), coworkers (M = 3.7), and administration (M = 3.8) pay some attention to data.

Perceived Behavioral Control

Using a scale from 1–6 (strongly disagree to strongly agree), teachers generally disagreed that too little time (M = 2.5), too much to do (M = 2.6), and too many students (M = 3.1) interfere with timely data collection. Instead, they agreed somewhat that unclear measurement systems are a barrier to timely data collection (M = 4.3).

Behavioral Intention

Using a scale from 1–6 (not at all likely to extremely likely), teachers reported that they were likely to extremely likely to collect data over the next 2 weeks (M = 5.2).

What is the relationship between AB, SN, and PBC and Teacher Intention to Collect Data for IEP Goals?

Table 2 shows the partial correlation matrix, controlling for group assignment, between self-reported intention to collect data and the study variables: the three TPB predictors of behavioral intention (i.e., SN, AB, PBC), self-efficacy for data collection, and administrative support. The hypothesis that teachers’ SN, AB, and PBC toward data collection would correlate positively with intention to collect data was supported for all three TPB predictors (r = .59, p <.001 for SN; r = .46, p <.001 for AB; r = .32, p < .05 for PBC).

Table 2.

Sample 1: Partial Correlation Matrix Controlling for Group Assignment (n = 44).

1 2 3 4 5 6
1. Intention --
2. Subjective Norm (SN) .59*** --
3. Attitude (AB) .46** .67*** --
4. Perceived Behavioral Control (PBC) .32* .50** .26 --
5. Self-Efficacy .55*** .53*** .54*** .42** ---
6. Administrative Support .35* .51** .16 .47** .39** --
Open in a new tab

Note. one-tailed tests:

*

p < .05;

**

p < .01;

***

p < .001.

To What Extent are Teachers’ Perceptions of Administrative Support and Self-Efficacy for Data Collection Related to Intention to Collect Data?

The second hypothesis that self-efficacy with data collection and administrative support would correlate with intention to collect data was also supported (r = .55, p < .001; r = .35, p <.05; respectively). Further, two of the three predictive elements of the TPB correlated with administrative support: SN (r = .51, p < .01) and PBC (r = .47, p < .01), and all three predictive elements of the TPB correlated with self-efficacy (correlations ranged from .42 [PBC] to .54 [AB]). Administrative support correlated positively with self-efficacy (r = .39, p < .01).

How Much Variance in Intention to Collect Data Can Be Explained by the Variables?

To better understand the unique contributions of the predictor variables, all variables that correlated significantly with intention to collect data were included as independent measures in a standard regression analysis (i.e., SN, AB, PBC, self-efficacy, administrative support), controlling for group assignment. All assumptions for multicollinearity, normality, and linearity were met. Tolerance test values ranged between .37 and .67, the Normal Probability Plot displayed a straight diagonal line from left to right, and no outliers were identified based on examination of Mahalanobis distances (Tabachnik & Fidell, 2007). Using all the predictors simultaneously, the multiple correlation coefficient R was .64 (R2 = .41), and the adjusted R2 = .30 (Table 3). That is, 30% of the variance in intention to keep data could be predicted using SN, AB, PBC, self-efficacy, and administrative support, F (6,32) = 3.69, p = .007. However, only one variable, self-efficacy, uniquely and significantly contributed to the model beyond the contribution of the other variables (beta = .37; t = 2.08, p = .04).

Table 3.

Regression Analysis of Predictors of Data Collection Intention and Behavior.

Predictors of data collection intention β p R R2 Adj R2 Test of overall model
T x Group .13 .38 .64 .41 .30 F (6,32) = 3.69, p = .007
SN .13 .57
AB .13 .53
PBC .10 .56
Self-efficacy .37 .04
Admin support .07 .71
Predictors of data collection behavior
PBC .29 .19 .45 .20 .13 F (2,23) = 2.84, p = .08
Admin support .22 .33
Open in a new tab

To What Extent are Intention to Collect Data, PBC, Self-Efficacy, and Administrative Support Related to Data Collection Behavior?

Table 4 shows the bivariate correlation matrix between data collection behaviors and intention to collect data, PBC, self-efficacy for data collection, and administrative support. Unexpectedly, the correlation with intention to collect data was not significant, although the result was consistent with a trend (r = .25, p = .10). However, two of the remaining four variables, PBC (r = .41, p < .05) and administrative support (r = .37, p < .05) correlated significantly with data collection behavior.

Table 4.

Bivariate Correlation Matrix (n = 29).

Data collection behavior p value
1. Data collection behavior --
2. Intention to collect data .25 .10
3. Behavioral control (PBC) .41* .02
4. Self-efficacy .15 .44
5. Administrative support .37* .03
Open in a new tab

Note. PCB = perceived behavioral control.

one-tailed test;

*

p < .05.

How Much Variance in Data Collection Behavior Can Be Explained by the Variables Significantly Related to Data Collection Behavior?

To better understand the unique contributions of the predictor variables, the two variables that correlated significantly with data collection behavior were included as independent measures in a standard regression analysis (i.e., PBC, administrative support). All assumptions of multicollinearity, normality, and linearity were met. Tolerance test values ranged between .58 and .71, the Normal Probability Plot displayed a diagonal line from left to right, and no outliers were identified based on examination of Mahalanobis distances (Tabachnik & Fidell, 2007). Using both of the predictors simultaneously, the multiple correlation failed to reach significance, F (2,23) = 2.84, p = .08. Moreover, neither variable, PBC or administrative support, had a unique and significant contribution to the model (Table 3).

What were Teachers’ Reasons for Collecting Data?

Qualitative analyses identified that the majority of the teachers (48%) reported that the reason to collect data is for progress monitoring of IEP goals. For instance, teachers reported that data are used to “determine what a child knows and what to work on next” and to “track progress.” Other than progress monitoring, 39% of the teachers reported that they use data to make educational decisions. For instance, teachers use data to “plan the IEP,” “make adjustments in instruction or other areas,” “determine needs for extended school year,” and “change goals.” Lastly, 14% of the teachers reported that the reason for data collection is to meet the legal requirements for educating students with disabilities. For instance, teachers mentioned that collected data needed to be submitted to IEP team meetings and “filed in the due process folder.”

Discussion

Measuring change in learning is essential for determining whether students are benefiting from their educational program; thus, data collection is a necessary prerequisite for the accurate implementation of evidence-based practices in the classroom. The current study applied the TPB and examined relationships between special education teachers’ reports of their intention to collect data and data collection behavior for IEP goals.

First, teachers reported positive attitudes overall for collecting data. They reported data collection helped students to achieve IEP goals and benefit from the educational plan. They also reported that their supervisors, colleagues, and parents viewed data collection positively, but that these individuals do not necessarily pay attention to data. Their report of barriers to collecting data focused on unclear measurement systems rather than on caseload sizes, having too little time, or too much to do. Lastly, they rated themselves as likely or extremely likely to collect data. Overall, these findings suggest that beliefs and attitudes support data collection, but that data are not necessarily valued by others. Further, of the potential barriers, measurement systems that are perceived as unclear and complex interfere with data collection.

As hypothesized, all three elements of the TPB correlated with teachers’ reports of intention to collect data for IEP goals and were associated generally with the other hypothesized variables in expected ways. These results provide initial confirmation of the potential utility of the TPB for understanding the influence of teacher attitudes, social norms, and perceived control related to data collection. Furthermore, the results suggest that the TPB may provide a useful framework for developing interventions to help understand, support, and modify special education teachers’ data collection behavior.

The regression analysis revealed that teachers’ data collection self-efficacy was the strongest predictor of intention to collect data for IEP goals. This suggests that self-efficacy may be critical in understanding special education teachers’ beliefs about their own abilities to collect data, a finding consistent with studies that have examined teacher self-efficacy associated with other types of skills (e.g., ability to respond to bullying; ability to teach students with ASD; Boulton, 2014; Ruble, Toland et al., 2013).

The importance of general support from administration was reflected by its significant correlation with two of the factors of the TPB: subjective norms and perceived behavioral control. Moreover, higher perceived administrative support was related to higher behavioral intention to collect data for IEP goals. This suggests that administrative support may be one controllable, external factor that could offset factors negatively related to intention to collect data (i.e., negative personal and social attitudes/norms and low self-efficacy). Administrative support also correlated with several other variables in our parent RCT study (i.e., student goal attainment outcome, teacher self-efficacy, teacher emotional exhaustion, teacher personal accomplishment, teacher stress; Ruble & McGrew, 2013), suggesting the potential importance of administrative support generally for teaching and teachers. One implication of these findings is that administrators may employ specialized strategies designed to support teachers in data collection and in other areas affecting teaching quality. Several administrative strategies have come out of studies of schoolwide implementation of multi-tiered systems of support (MTSS) to support student development of academic and behavioral skills. Studies have shown convincingly that administrator buy-in and related school climate is necessary for teacher buy-in, training on assessment materials and support for fidelity must be provided for educators, time must be allowed consistently and systematically across classrooms and grades for data collection, and time must be made schoolwide for data interpretation, with some schools using professional learning community or grade-level meeting time for data review (see McIntosh & Goodman, 2016; Shapiro, Zigmond, Wallace, & Marston, 2011). Consistent with our findings, studies have demonstrated the importance of administrative support in improving a variety of different behavioral outcomes for teachers and other professionals (e.g., Rosenthal, McCormick, Guzman, Villamayor, & Orellano, 2003; Santoli, Sachs, Romey, McClurg, & Trussville, 2008).

These findings also have implications for possible school-based practices and preservice training. For example, school administrators may wish to ask for data more often and to encourage teachers to use data in daily educational decision making, especially when collaborating on special education teams. The quantitative data suggest administration lacks involvement in reviewing data. This is unfortunate given data were described as necessary for making decisions about extended school year and reporting general progress. Another potential application of these results is to help enhance teacher self-efficacy for data collection. For example, it may be useful to implement training or practices based on models developed for increasing self-efficacy that have been applied across a variety of other behaviors, including strategies at the preservice level (e.g., Bandura, 1982; Marks & Allegrante, 2005; Stajkovic & Luthans, 1998).

Data Collection Behaviors

The results for data collection behaviors presented a somewhat different picture, however. As hypothesized, perceived behavioral control correlated positively with behavior as did general administrative support. However, contrary to our hypothesis, the correlation between intention to collect data for IEP goals and behavior, although positive at trend level, failed to reach statistical significance. These preliminary findings suggest that, as a predictor, behavioral control may be more important than intention in this sample using the TPB model. However, these findings are inconsistent with a previous meta-analysis of studies that used the TPB, which reported a strong relationship between intentions and behavior (Armitage & Conncer, 2001). There are possible methodological explanations for the low correlation between intention and behavior. First, the sample was small, limiting our power to detect a correlation. Second, our measure of intention was focused broadly and was open to any data collection conducted within the previous 2 weeks (i.e., Do you intend to collect data over the next 2 weeks?), whereas our measure of data collection was focused narrowly on data collection for the three primary COMPASS goals targeting social, communication, and learning/readiness skills and asked only about whether data were collected, ignoring frequency of data collection. It is possible that the correlation would be stronger if both measures focused on a similar range of data collection activities and also considered frequency.

Directions for Future Research and Limitations

Several potential implications of our findings for administrators and for designing interventions to increase and improve data collection for IEP goals were mentioned above. Additionally, the TIDE may be helpful for future implementation science research (Ruble & McGrew, 2015). The TIDE was helpful in identifying potential predictors of data collection intention and behavior for future research. Most importantly, the findings suggest specific barriers that may interfere with data collection for IEP goals. Teachers reported that unclear measurement systems were the biggest barrier to data collection. These findings suggest practical areas that can be addressed. Professional development activities that focus on ways to address these issues may help overcome these barriers. Remaining to be addressed in the future is what impact the process of data collection has on child educational outcomes. Certainly, research is ample in investigating the role that frequent, skills-based assessment can play in intervention planning and child outcomes across broad literatures from applied behavior analysis (see Fisher, Piazza, & Roane, 2013) and response to intervention (see Jimerson, Burns, & VanDerHeyden, 2016); however, the focus here is a clearer understanding of the multi-dimensional components that influence data collection processes and systems.

This study also had some limitations. One limitation is the lack of information on teachers’ experiences with data collection for IEP goals. For example, it would have been informative to have a measure of teachers’ data collection knowledge, experience, and use. Moreover, our measure of data use was strictly limited to data collection for the three primary COMPASS goals. Because the goals targeted by COMPASS are often the most difficult for teachers to assess (social skills, communication skills, and learning/readiness skills), the findings may not generalize to other areas such as data collection on academic skills and thus may overestimate the degree to which our predictors relate to data collection generally. Second, data collection was limited to teacher report. In order to show stronger connections between data collection and data-based decision making efforts, alternative assessment approaches (i.e., review of permanent products, direct observation) are likely necessary to illuminate perceptions versus behaviors. Third, data collection occurred at the beginning and end of the school year. Because data-based decision making is a fluid and ongoing progress throughout the academic year, future research might consider development of a broader measure that encompasses the full range of data collection activities of special education teachers in order to understand teachers’ data collection behaviors and ways to support these efforts.

Conclusion

This study of teacher data collection behaviors for monitoring IEP goals expanded the literature by identifying and applying the TPB framework. Directions for future research are offered by informing and extending factors that hinder and facilitate data collection for IEP goals. Potential facilitators worth further exploration include a culture of peers who value data collection, administrators who ask about data collection, and parents who are informed about progress based on data collection. Additionally, studies that consider teachers’ report of self-efficacy for data collection may be able to identify low-efficacy teachers who would benefit from professional development plus coaching and training in the use of clear and feasible data collection methods.

Acknowledgments

This work was supported by Grant Numbers R34 MH073071 and 1RC1MH089760 from the National Institute of Mental Health. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Mental Health or the National Institutes of Health.

We are grateful to the teachers, families, and children, who generously donated their time and effort. We wish to thank special education directors and principals for allowing their teachers to participate. We also want to acknowledge our research team members Ryan Johnson, Rachel Aiello, Jessica Birdwhistell, Jennifer Hoffman, and Lauren Feltner for their efforts as well as Nancy Dalrymple and Thomas Guskey.

References

  1. Ajzen I. The theory of planned behavior. Organizational Behavior and Human Decision Processes. 1991;50:179–211. doi: 10.1016/0749-5978(91)90020-T. [DOI] [Google Scholar]
  2. Armitage CJ, Conner M. Efficacy of the theory of planned behaviour: A meta-analytic review. British Journal of Social Psychology. 2001;40:471–499. doi: 10.1348/014466601164939. [DOI] [PubMed] [Google Scholar]
  3. Bandura A. Self-efficacy mechanism in human agency. American Psychologist. 1982;37:122–147. doi: 10.1037/0003-066X.37.2. [DOI] [Google Scholar]
  4. Bandura A. Human agency in social cognitive theory. American Psychologist. 1989;44:1175–1184. doi: 10.1037//0003-066X.44.9.1175. [DOI] [PubMed] [Google Scholar]
  5. Bandura A. Social cognitive theory of self-regulation. Organizational Behavior and Human Decision Processes. 1991;50:248–287. doi: 10.1016/0749-5978(91)90022-L. [DOI] [Google Scholar]
  6. Bandura A. Self-efficacy: The exercise of control. New York, NY: Freeman; 1997. [DOI] [Google Scholar]
  7. Bandura A. Guide for constructing self-efficacy scales. In: Pajares F, Urdan T, editors. Self-efficacy beliefs of adolescents. Greenwich, CT: Information Age Publishing; 2006. pp. 307–337. [Google Scholar]
  8. Billingsley BS. Promoting teacher quality and retention in special education. Journal of Learning Disabilities. 2004;37:370–376. doi: 10.1177/00222194040370050101. [DOI] [PubMed] [Google Scholar]
  9. Billingsley BS, Cross LH. Predictors of commitment, job satisfaction, and intent to stay in teaching: A comparison of general and special educators. The Journal of Special Education. 1992;25:453–471. doi: 10.1177/002246699202500404. [DOI] [Google Scholar]
  10. Boulton MJ, Hardcastle K, Down J, Fowles J, Simmonds JA. A comparison of preservice teachers’ responses to cyber versus traditional bullying scenarios: Similarities and differences and implications for practice. Journal of Teacher Education. 2014;65(2):145–155. doi: 10.1177/0022487113511496. [DOI] [Google Scholar]
  11. Brawley S, Stormont MA. Investigating reported data practices in early childhood: An exploratory study. Journal of Positive Behavior Interventions. 2013;14:1–10. doi: 10.1177/1098300713480838. [DOI] [Google Scholar]
  12. Chatzisarantis N, Kamarova S, Kawabata M, Wang JC, Hagger MS. Developing and evaluating utility of school-based intervention programs in promoting leisure-time physical activity: an application of the theory of planned behavior. 2015 doi: 10.7352/IJSP2015.46.095. [DOI] [Google Scholar]
  13. Corrigan PW, Steiner L, McCracken SG, Blaser B, Barr M. Strategies for disseminating evidence-based practices to staff who treat people with serious mental illness. Psychiatric Services. 2001;52:1598–1606. doi: 10.1176/appi.ps.52.12.1598. [DOI] [PubMed] [Google Scholar]
  14. Fisher WW, Piazza CC, Roane HS, editors. Handbook of applied behavior analysis. New York, NY: Guilford; 2013. [Google Scholar]
  15. Fore C, III, Martin C, Bender WN. Teacher burnout in special education: The causes and the recommended solutions. High School Journal. 2002;86:36–44. doi: 10.1353/hsj.2002.0017. [DOI] [Google Scholar]
  16. Francis JJ, Eccles MP, Johnston M, Walker A, Grimshaw J, Foy R, Bonetti T. Constructing questionnaires based on the theory of planned behavior: A manual for health services researchers. 2004 May; Retrieved from http://www.rebeqi.org/ViewFile.aspx?itemID=212.
  17. Freitag S, Dunsmuir S. The inclusion of children with ASD: Using the Theory of Planned Behaviour as a theoretical framework to explore peer attitudes. International Journal of Disability, Development and Education. 2015;62:405–421. doi: 10.1080/1034912X.2015.1046818. [DOI] [Google Scholar]
  18. Gischlar K, Hojnoski R, Missall K. Data-based decision making to improve child outcomes: Interpreting and using data. Young Exceptional Children. 2009;13(1):2–18. [Google Scholar]
  19. Graneheim UH, Lundman B. Qualitative content analysis in nursing research: Concepts, procedures and measures to achieve trustworthiness. Nurse Education Today. 2004;24:105–112. doi: 10.1016/j.nedt.2003.10.001. [DOI] [PubMed] [Google Scholar]
  20. Individuals with Disabilities Education Act. Building the legacy: IDEA 2004. 2004 Retrieved from http://idea.ed.gov/
  21. Ingram D, Louis SK, Schroeder R. Accountability policies and teacher decision making: Barriers to the use of data to improve practice. The Teachers College Record. 2004;106:1258–1287. doi: 10.1111/j.1467-9620.2004.00379.x. [DOI] [Google Scholar]
  22. Jimerson SR, Burns MK, VanDerHeyden AM, editors. Handbook of response to intervention: The science and practice of multi-tiered systems of support. 2. New York, NY: Springer; 2016. [Google Scholar]
  23. Klassen RM, Tze V, Gordon K, Betts S. Teacher efficacy research 1998–2009: Signs of progress or unfulfilled promise? Educational Psychology Review. 2011;23:21–43. doi: 10.1007/s10648-010-9141-8. [DOI] [Google Scholar]
  24. Mandinach E, Gummer E, Northwest E. Navigating the landscape of data literacy: It is complex. 2012 May; Retrieved from: https://www.wested.org/online_pubs/resource1304.pdf.
  25. Marks R, Allegrante JP. A review and synthesis of research evidence for self-efficacy-enhancing interventions for reducing chronic disability: Implications for health education practice (part II) Health Promotion Practice. 2005;6:148–156. doi: 10.1177/1524839904266792. [DOI] [PubMed] [Google Scholar]
  26. Martinez R, Young A. Response to intervention: How is it practiced and perceived. International Journal of Special Education. 2011;26:44–52. [Google Scholar]
  27. McIntosh K, Goodman S. Integrating multi-tiered systems of support: Blending RTI and PBIS. New York, NY: Guilford; 2016. [Google Scholar]
  28. Mercer SH, Idler AM, Bartfai JM. Theory-driven evaluation in school psychology intervention research: 2007–2012. Vol. 43. Bethesda, MD: National Association of School Psychologists; 2014. [Google Scholar]
  29. Rosenholtz SJ. Workplace conditions that affect teacher quality and commitment: Implications for teacher induction programs. Elementary School Journal. 1989;89:421–39. [Google Scholar]
  30. Rosenthal VD, McCormick RD, Guzman S, Villamayor C, Orellano PW. Effect of education and performance feedback on handwashing: The benefit of administrative support in Argentinean hospitals. American Journal of Infection Control. 2003;31:85–92. doi: 10.1067/mic.2003.63. [DOI] [PubMed] [Google Scholar]
  31. Ruble L, Dalrymple N, McGrew J. The effects of consultation on Individualized Education Program outcomes for young children with autism: The collaborative model for promoting competence and success. Journal of Early Intervention. 2010;32(4):286–301. doi: 10.1177/1053815110382973. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Ruble L, Dalrymple N, McGrew J. The collaborative model for promoting competence and success for students with ASD. New York, NY: Springer; 2012. [Google Scholar]
  33. Ruble L, McGrew JH. Teacher and child predictors of achieving IEP goals of children with Autism. Journal of Autism and Developmental Disorders. 2013;43(12):2748–2763. doi: 10.1007/s10803-013-1884-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Ruble L, McGrew J. COMPASS and implementation science: Improving educational outcomes of children with ASD. New York, NY: Springer; 2015. [Google Scholar]
  35. Ruble L, McGrew J, Toland M. Goal attainment scaling as outcome measurement for randomized controlled trials. Journal of Autism and Developmental Disorders. 2012;42(9):1974–1983. doi: 10.1007/s10803-012-1446-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Ruble LA, McGrew JH, Toland MD, Dalrymple NJ, Jung LA. A randomized controlled trial of COMPASS web-based and face-to-face teacher coaching in autism. Journal of Consulting and Clinical Psychology. 2013;81(3):566–572. doi: 10.1037/a0032003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Ruble L, Toland M, Birdwhistell J, McGrew J, Usher E. Preliminary study of the Autism Self-Efficacy Scale for Teachers (ASSET) Research in Autism Spectrum Disorders. 2013;7:1151–1159. doi: 10.1016/j.rasd.2013.06.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Ruble LA, Usher EL, McGrew JH. Preliminary investigation of the sources of self-efficacy among teachers of students with autism. Focus on autism and other developmental disabilities. 2011;26(2):67–74. doi: 10.1177/1088357610397345. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Sandall SR, Schwartz IS, LaCroix B. Interventionists’ perspectives about data collection in integrated early childhood classrooms. Journal of Early Intervention. 2004;26:161–174. doi: 10.1177/105381510402600301. [DOI] [Google Scholar]
  40. Santoli SP, Sachs J, Romey EA, McClurg S, Trussville AL. A successful formula for middle school inclusion: Collaboration, time, and administrative support. Research in Middle Level Education. 2008;32:1–13. doi: 10.1080/19404476.2008.11462055. [DOI] [Google Scholar]
  41. Shapiro ES, Zigmond N, Wallace T, Marston D, editors. Models for implementing response to intervention: Tools, outcomes, and implications. New York, NY: Guilford; 2011. [Google Scholar]
  42. Stajkovic AD, Luthans F. Self-efficacy and work-related performance: A meta-analysis. Psychological Bulletin. 1998;124:240–261. doi: 10.1037/0033-2909.124.2.240. [DOI] [Google Scholar]
  43. Tabachnik B, Fidell L. Using multivariate statistics. Boston, MA: Pearson; 2007. [Google Scholar]
  44. U.S. Department of Education. Race to the top program executive summary. 2009 Retrieved from www2.ed.gov/programs/racetothetop/executive-summary.pdf.
  45. Volpe RJ, Suldo SM. Introduction to the special issue on theoretical frameworks in school psychological intervention research: Interdisciplinary perspectives and future directions. Vol. 43. Bethesda, MD: National Association of School Psychologists; 2014. [Google Scholar]
  46. Yan Z, Sin K. Exploring the intentions and practices of principals regarding inclusive education: An application of the Theory of Planned Behaviour. Cambridge Journal of Education. 2015;45:205–221. doi: 10.1080/0305764X.2014.934203. [DOI] [Google Scholar]
  47. Yell ML, Drasgow E, Lowrey KA. No child left behind and students with autism spectrum disorders. Focus on Autism and Other Developmental Disabilities. 2005;20:130–139. doi: 10.1177/10883576050200030101. [DOI] [Google Scholar]
  48. Zabel RH, Zabel K. Revisiting burnout among special education teachers: Do age, experience, and preparation still matter? Teacher Education and Special Education. 2001;24:128–139. doi: 10.1177/088840640102400207. [DOI] [Google Scholar]

RESOURCES