Impact of the Self-Determined Learning Model of Instruction on Self-Determination: A Randomized-Trial Control Group Study

Abstract

Promoting self-determination has become best practice in special education. There remains, however, a paucity of causal evidence for interventions to promote self-determination. We conducted a group-randomized, modified equivalent control group design study of the efficacy of the Self-Determined Learning Model of Instruction to promote self-determination. Data on self-determination using multiple measures was collected with 312 high school students with cognitive disabilities in both a control and treatment group. We examined the relationship between the SDLMI and self-determination using structural equation modeling. After determining strong measurement invariance for each latent construct, we found significant differences in latent means across measurement occasions and differential effects attributable to the SDLMI. This was true across disability category, though there was variance across disability populations.

There has been considerable progress in research and intervention to promote self-determination since the construct was first introduced to the field of special education (Cobb, Lehmann, Newman-Gonchar, & Alwell, 2009). Such advances are important because research has linked student self-determination status to the attainment of more positive academic (Fowler, Konrad, Walker, Test, & Wood, 2007; Konrad, Fowler, Walker, Test, & Wood, 2007; Lee, Wehmeyer, Soukup, & Palmer, 2010) and transition outcomes, including more positive employment, recreation, and independent living outcomes (McGuire & McDonnell, 2008; Wehmeyer & Palmer, 2003; Wehmeyer & Schwartz, 1997), and more positive quality of life and life satisfaction (McDougall, Evans, & Baldwin, 2010; Shogren, Lopez, Wehmeyer, Little, & Pressgrove, 2006; Wehmeyer & Schwartz, 1998).

There have been numerous curricular and instructional models introduced to enable educators to provide this instructional focus (Test, Karvonen, Wood, Browder, & Algozzine, 2000) and research has documented, to some degree, the efficacy of many of these interventions. In a meta-analysis of single subject and group design studies, Algozzine, Browder, Karvonen, Test, and Wood (2001) found evidence for the efficacy of instruction to promote component elements of self-determined behavior. Cobb et al. (2009) conducted a narrative metasynthesis—a narrative synthesis of multiple meta-analytic studies—covering seven meta-analyses examining self-determination and concluded that there is sufficient evidence to support the promotion of self-determination as effective. Also, there are several norm-referenced, validated assessments of self-determination that are widely used (Shogren et al., 2008).

Recently, Wehmeyer, Palmer, Shogren, Williams-Diehm, and Soukup (in press) conducted a randomized trial control group study of the effect of interventions to promote the self-determination of high school students with cognitive disabilities. Students in the treatment group (n=235) received instruction using a variety of instructional methods to promote self-determination and student involvement in educational planning meetings over three years; students in the control group (n=132) received no such intervention. Self-determination was measured using two instruments across three measurement intervals. Using latent growth curve analyses, Wehmeyer and colleagues determined that students with cognitive disabilities who participated in intervention to promote self-determination over a three-year period showed significantly more positive patterns of growth in their self-determination scores than did students not exposed to interventions to promote self-determination during the same time period.

The Wehmeyer et al. (in press) study did not provide data on the effect of any single intervention, instead providing evidence that efforts to promote self-determination using multiple interventions resulted in enhanced self-determination. As Cobb and colleagues (2009) noted in their metasynthesis, self-determination is a multifaceted construct and interventions that achieve the best outcomes are multi-component interventions. One such multi-component intervention that was implemented with all students in the treatment group for the Wehmeyer et al. study and has been linked to goal attainment and more positive adult outcomes for youth with disabilities through multiple single-subject and quasi-experimental design studies is the Self-Determined Learning Model of Instruction (SDLMI; Wehmeyer, Palmer, Agran, Mithaug, & Martin, 2000).

The SDLMI is a model of teaching (e.g., intended for teachers as end-users to guide and direct instruction) that supports teachers to enable students to self-regulate and self-direct the learning process and, as a result, engage in self-determined learning. Students who self-determine learning set educational goals based upon their own interests, abilities, and needs; meaningfully participate in decisions pertaining to the design of interventions to achieve this goal; implement strategies that enable them to modify and regulate their own behavior; and utilize strategies that support them to track their progress toward the goal and to modify either the goal or the action plan, as needed. The intent of any model of teaching is to promote student learning and the efficacy of such models, as such, must include evaluations of the effects of instruction on learning. Wehmeyer et al. (2000) also proposed that instruction using the SDLMI would have the additional benefit of promoting student self-determination. Numerous studies have examined the impact of the SDLMI on student educational goal attainment (Lee, Wehmeyer, Palmer, Soukup, & Little, 2008; Palmer & Wehmeyer, 2003; Palmer, Wehmeyer, Gibson, & Agran, 2004; Wehmeyer et al., 2000). Recently, Shogren, Palmer, Wehmeyer, Williams-Diehm, and Little (2010) used a group-randomized trial control group study to evaluate the impact of the SDLMI on academic and transition goal achievement and access to the general education curriculum. Students in the treatment group received instruction using the SDLMI for one academic year. Analysis by multilevel modeling for pre-and post-intervention data found that instruction using the SDLMI resulted in significant changes in goal attainment and access to the general education curriculum.

To date, however, there have been no studies that provide causal evidence testing the hypothesis that instruction with the SDLMI will also promote student self-determination. Only the Wehmeyer et al. (2000) study actually collected data on self-determination using a norm-referenced measure, and although there were promising results from that study, its design precluded making causal attributions about the SDLMI’s effects. The purpose of the current study was to evaluate if students with cognitive disabilities who received instruction using the SDLMI showed enhanced self-determination. We had two research questions. Do students in the treatment group show higher levels of self-determination after exposure to the SDLMI compared to the control group? And, when treatment is introduced to the control group in Year 2, does the control group demonstrate the same pattern of change in self-determination outcomes as the treatment group? We hypothesized that students exposed to the SDLMI in the treatment group would show larger increases in self-determination than students in the control group. Further, we hypothesized that when the SDLMI was introduced to the control group in Year 2, that group would show the same pattern of change in self-determination as the treatment group in Year 1.

Method

Participants

Power analysis

Based upon previous research with the SDLMI, we determined an effect size in the moderate impact range (Cohen’s d = .38) and, along with an intra-class correlation (ρ) of .05 and baseline levels of self-determination as measured by The Arc’s Self-Determination Scale, conducted a power analysis using formulas specified by Murray (1998) and software, called Optimal Design, developed by Spybrook, Raudenbush, Liu, and Congdon (2006). Power analysis determined that we would need at least 26 schools with 12 students at each school to detect our effect in a completely balanced group randomized trial design.

Sample

Participants were 312 high school students with intellectual disability (n=94; 30%) or learning disabilities (n=218; 70%) recruited (next section) from 20 school districts located in three states: Kansas (n=85; 27%), Missouri (n=97; 31%), and Texas (n=130; 42%). At the start of the study, participants ranged in age from 13.5 to 21.3 years (M = 16.5; SD = 1.40). The sample was 44% female (n=137; M Age=16.4, SD = 1.25) and 56% male (n=175; M Age=16.5, SD = 1.50). The majority of participants were Caucasian (n=173; 55%), although other race/ethnicities were represented: Hispanic (n=68; 22%), African American (n=61; 20%); Native American/Alaskan Native (n=2; 1%); Asian or Pacific Islander (n=2; 1%); and Other (n=6; 2%). By teacher report, 43% (n=133) of students were eligible for free or reduced lunch.

Design and Procedures

We implemented a group-randomized, modified equivalent control group time series design (Murray, 1998) over two years to examine the impact of the SDLMI on self-determination. Project personnel contacted school districts, and districts that agreed to participate (n = 20) identified high school campuses (n = 39) to participate. Next, the primary district contact person (e.g., transition coordinator, special education administrator) worked with project staff to identify students who met the project criteria, which included a) students receiving special education services under the categorical label of intellectual or learning disability that b) were expected to be receiving services for two years after project implementation, and c) could reliably complete self-report measures. Informed consent was obtained for each teacher participating, parents or guardians, as well as assent from the student.

Campuses that agreed to participate were assigned to be a “treatment” or “control” campus. Random assignment occurred at the campus level (rather than the teacher or student level) because special educators often work collaboratively and students typically interact with multiple teachers, introducing the potential that students might receive instruction from teachers in both the control and treatment group (if random assignment was done at the teacher level) or that teachers would teach students in both the control and treatment group (if random assignment was done at the student level). During Year 1 of the project, teachers at treatment campuses were trained to implement the SDLMI and, subsequently, implemented it with participating students. Teachers at control campuses continued with typical instruction. Year 1 provided a pretest-posttest control group comparison study of the impact of the SDLMI. During Year 2, teachers on control campuses were trained in the SDLMI in the same fashion as the occurrence on treatment campuses in Year 1 and implemented the model with participating students. Teachers at treatment campuses continued implementing the SDLMI with participating students. Thus, all students received intervention in Year 2. We chose to provide intervention for all participants in Year 2 to ensure that all students and campuses could benefit from involvement with the study and to minimize, to the extent possible, attrition of teachers/students at control campuses.

After consent and assent to participate were obtained, baseline data were collected, including demographic information and two measures of self-determination. Self-determination data were collected at the end of Year 1 and of Year 2. As expected, there was attrition in the sample over time. At the end of Year 1, 103 control-group students had completed baseline and Year 1 posttest scores from The Arc’s Self-Determination Scale (SDS) and 94 treatment-group students had completed pre and post scores from this measure. A total of 111 control-group students had the completed set at the end of the year for the AIR Self-Determination Scale (AIR), with 94 treatment-group students having likewise. At the end of Year 3, 94 control group students and 88 treatment group students had data from all three points from the SDS and 96 control group and 88 treatment group students had likewise. Preliminary analyses indicated no significant differences between completers and non-completers on key variables; therefore all participants with data from at least one wave of assessment were included in the analyses. To account for missing data, all statistical models were estimated using full-information maximum likelihood (FIML) estimation. FIML is a “state-of-the-art” approach to unbiased estimation with missing data that outperforms other deletion or single imputation methods under the missing at random assumption (Schafer & Graham, 2002).

Intervention

The SDLMI is a model of teaching based on the component elements of self-determination, the process of self-regulated problem solving, and research on student-directed learning. Teachers were instructed to support students, using the SDLMI, to set two goals, one academic goal and one transition goal. Instruction on the model occurred in the educational context in which goals were being addressed, including the student’s general education classroom, special education settings, and, in some cases, in community-based settings. Students were continuously engaged with the model in relation to the specific goals set throughout the first year (for the treatment group) and second year (for both groups). Once a goal is reached, the student sets another goal and the process continues. Implementation of the model consists of a three-phase instructional process. Each phase presents a problem to be solved by the student. The student solves each problem by posing and answering a series of four Student Questions per phase that students learn, modify to make their own, and apply to reach self-selected goals. Each question is linked to a set of Teacher Objectives and each phase includes a list of Educational Supports that can be used to enable students to self-direct learning.

The Student Questions in the model are constructed to direct the student through a problem-solving sequence in each instructional phase. Each instructional phase poses a problem the student must solve (What is my goal? What is my plan? What have I learned?). The solutions to the problems in each phase lead to the problem-solving sequence in the next phase. Teachers teach students to solve a sequence of problems to construct a means-ends chain - a causal sequence - that moves them from where they are (an actual state of not having their needs and interests satisfied) to where they want to be (a goal state of having those needs and interests satisfied). To answer the questions in this sequence, students must regulate their own problem solving by setting goals to meet needs, constructing plans to meet goals, and adjusting actions to complete plans. The four questions differ from phase to phase, but represent identical steps in the problem-solving sequence. That is, students answering the questions must: (1) identify the problem, (2) identify potential solutions to the problem, (3) identify barriers to solving the problem, and (4) identify consequences of each solution. These steps form the means-end problem-solving sequence represented by the Student Questions in each phase.

The first time a teacher uses the model with a student, the initial step in the implementation process is to read the question with or to the student, discuss what the question means and then, if necessary, change the wording to enable that student to better understand the intent of the question. Such wording changes must be made such that the problem-solving intent of the question remains intact. The Teacher Objectives within the model are just that -- the objectives a teacher will be trying to accomplish by implementing the model. In each instructional phase, the objectives are linked directly to the Student Questions. These objectives can be met by utilizing strategies provided in the Educational Supports section of the model. [Detailed information about the model and its implementation is in Wehmeyer et al. (2000).]

Fidelity to Treatment

Fidelity to treatment for implementation of the SDLMI was monitored by three types of fidelity measurement (Fixsen, Naoom, Blasé, Friedman, & Wallace, 2005): (a) a context fidelity measure that describes the necessary precursors to high-level performance (e.g., completion of training), (b) a compliance fidelity measure that provides an outline of the core intervention components and their use by practitioner, and (c) a competence fidelity measure that illustrates how well the practitioner is performing the core intervention components of an evidence-based program or practice. For the context fidelity indicator, all special education teachers received training from the same group of trainers on the SDLMI. Compliance fidelity was monitored through on-going support and communication to facilitate teachers to implement the interventions. For this, regular notices to announce important agendas and schedules of implementation were sent via email. All teachers and students followed the same procedures regarding implementation of the respective materials or intervention. In addition, teachers were contacted by phone by project staff and provided assistance via telephone support as needed. Finally, competence fidelity was evaluated, as feasible, by reviewing worksheets and written materials completed by the participating students in relation to each of the interventions.

Instrumentation

Measures of Self-Determination

The Arc’s Self-Determination Scale. The Arc’s Self-Determination Scale (SDS, Wehmeyer & Kelchner, 1995) is a 72-item self-report measure based on the functional theory of self-determination. A total of 148 points are available on the scale, with higher scores indicating higher levels of self-determination. An overall self-determination score, as well as subscale scores for each of the four essential characteristics of self-determined behavior: autonomy, self-regulation, psychological empowerment, and self-realization (Wehmeyer, 1996a) can be calculated. The SDS was developed and normed with 500 adolescents with cognitive disabilities (Wehmeyer, 1996b) Subsequent research (Shogren, Lopez, Wehmeyer, Little, & Pressgrove, 2006; Shogren et al., 2006) has verified the proposed theoretical structure of The Arc’s Self-Determination Scale (i.e., four related, but distinct subscales [autonomy, self-regulation, psychological empowerment, and self-realization] that contribute to a higher-order self-determination construct).

The AIR Self-Determination Scale (AIR, Wolman, Campeau, Dubois, Mithaug, & Stolarski, 1994) assesses student capacity and opportunity for self-determination. The AIR has a Student, Educator, and Parent version and the Student (AIR-S) self-report version was utilized in the present study. The AIR-S version has 24 questions in four different sections. There are two sections related to things students do related to self-determination (“Things I Do” subscale) and how students feel about performing these self-determined behaviors (“How I Feel” subscale). There are also two sections related to students opportunity for self-determination at home (“What Happens at Home” subscale) and at school (“What Happens at School” subscale).

The AIR was developed and normed with 450 students with and without disabilities in California and New York (Wolman et al., 1994). The AIR was demonstrated to have adequate reliability and validity in the measurement of capacity and opportunity for self-determination (Wolman et al., 1994). Recent research (Shogren, Wehmeyer et al., 2006) has confirmed the theoretical structure of the AIR (i.e., two related subscales – capacity and opportunity – that contribute to a higher-order self-determination construct). This research also confirmed that while SDS and the AIR-S are related (r = .50), they are measuring distinct aspects of the self-determination construct. Shogren, Wehmeyer, et al. found that combining these two measures into one global, higher-order self-determination construct was not justified by data.

Analytic Plan

Structural equation modeling (SEM) was used to examine the relationship between the SDLMI and student self-determination outcomes. SEM has many advantages over traditional manifest variable techniques (ANOVA or regression), particularly the ability to represent latent constructs without measurement error which reduces bias in estimates of latent means and effect sizes. SEM procedures involve the integration of measurement models, which specify the relationships among latent and observed variables, with structural models, which specify the relationship between latent factors (Kline, 2005). The measurement equivalence of the constructs across groups can be assessed and direct statistical comparisons of the similarities and differences in the means, variances, correlations, and regression relationships among the constructs examined (Little, 1997).

We created two groups (treatment and control) and estimated a series of models (described below) to determine if exposure to the SDLMI impacts self-determination outcomes. All models were estimated in Mplus, version 6.0 (Muthén & Muthén, 2007). Six latent constructs were included in the measurement model: self-determination as measured by The Arc’s Self-Determination Scale at Time 1(SDS-1), Time 2 (SDS-2), and Time 3 (SDS-3) and self-determination as measured by the AIR Self-Determination Scale at Time 1(AIR-1), Time 2 (AIR-2), and Time 3 (AIR-3). The structure of the model is shown in Figure 1. For each latent construct, facet-representative parcels (or subscale parcels) were created to form manifest indicators. Parceling is an aggregation technique that combines several scale items into a single parcel, using the mean of the items. Parcels are more reliable, less likely to violate distributional assumptions, and more precise than scale items (Little, Cunningham, Shahar, & Widaman, 2002). At each time point, the AIR was measured by four parcels (subscales) that reflect student perceptions of their capacity for self-determination (“Things I do”; “How I feel”) and the opportunities they have to demonstrate self-determined behaviors (“What happens at school”; “What happens at home”). Similarly, the SDS was indicated by four parcels (subscales) representing the essential characteristics of self-determined behavior: Autonomy, Self-Regulation, Psychological Empowerment, and Self-Realization.

Figure 1. Model Structure.

Our self-determination outcome data had a hierarchical structure (i.e. students nested within schools). It is well known in the social sciences that clustered data can lead to biased estimates and incorrect statistical inference (Raudenbush & Byrk, 2002). To address this hierarchical structure, all models were estimated with a robust maximum likelihood estimator that provides corrected standard errors and a corrected chi-square value (Satorra, 2000). Because, the present study was concerned with inferences only at the student-level and not at the campus-level, we accounted for the variance between campuses in our models and used group-mean centering of predictors (Enders & Tofighi, 2007).

Model Testing

A necessary first step in comparing the treatment and control group is exploring the factorial invariance of the constructs in the treatment and control group. Testing factorial invariance ensures that the same constructs are being measured in the treatment and control group and that comparisons in the latent space (e.g., latent mean level differences) are justified (Meredith, 1993). Statistically, factorial invariance involves estimating a succession of models in which each new model contains additional constraints on measurement parameters. The eight invariance models tested to evaluate factorial invariance are found in Table 3. (For detailed information regarding the specification of each model and the constraints that are tested, see Little, Preacher, Selig, & Card, 2007). The first four models evaluated measurement invariance and the subsequent four structural invariance. Measurement invariance constraints are considered acceptable if there is a change of less than .01 in the Comparative Fit Index (CFI, Bentler, 1990). The χ² Difference Test (i.e., is there a non-significant change in χ² when the constraint is added to the model) was used to evaluate structural invariance (see Little, 1997).

Table 3.

Latent Means

	Time 1		Time 2		Time 3
Group	M	{95% C.I.}	M	{95% C.I.}	M	{95% C.I.}
AIR
Self-Determination Scale
Intervention Group	.00*	{.00 – .00}	.07	{−.17 – .31}	.30	{.08 – .52}
Control Group	.16	{‒.10 – .42}	.11	{−.15 – .37}	.17	{−.10 – .44}
Latent d	−.20		−.05		.14
SDS
The Arc’s Self-Determination Scale
Intervention Group	.00*	{.00– .00}	−.06	{−.21 – .10}	.24	{.06 – .42}
Control Group	−.01	{−.27 – .25}	−.06	{−.32 – .21}	.03	{−.26 – .33}
Latent d	.01		.00		.23

^*Latent mean estimate fixed for identification

Model Testing - Latent Mean Comparisons

To examine if students in the treatment group showed increased self-determination scores after exposure to the SDLMI compared to students in the control group, we evaluated within- and between-group differences in self-determination scores over time. Latent mean level differences were evaluated using chi-square difference tests. Effect sizes for between-group differences were calculated as follows:

$$\text{Latent}d=\frac{{\alpha }_I-{\alpha }_C}{\left(\frac{n_I{\psi }_I+n_C{\psi }_C}{n_I+n_C}\right)}$$

This statistic, which we refer to as Latent d, is based on Cohen’s d (Hancock, 2001) where α_I and α_C represent the intervention and control group latent means, respectively; n_I and n_C are the intervention and control group sample sizes; and ψ_I and ψ_C represent the variance of the latent variable for the intervention and control group. The latent means for the AIR and SDS at the first measurement occasion were fixed to zero in the intervention group. As the latent variances are also scaled in relation to the first measurement occasion (fixed to 1 in the intervention group), the means can be interpreted in the standard deviation metric.

Model Testing - Structural Model

To evaluate if students in the control group demonstrated the same increase in self-determination scores as the treatment group after the SDLMI was introduced in Year 2, we specified a cross-lagged panel model. The structure of the cross-lagged panel model is shown in Figure 2. We compared the relationships between the measures of self-determination from Time 1 to Time 2 in the treatment group and Time 2 to Time 3 in the control group. An unconstrained model was first estimated to establish model fit and check model parameters. The model was equivalent to the weak-invariance measurement model in all regards except that the latent autoregressive paths, within-time correlations, and cross-lagged paths were estimated. Autoregressive paths estimate the relationship between the same construct at different time points (e.g., the relationship of the SDS from Time 1 to Time 2). Within-time correlations estimate the relationships between the constructs at a given time point (e.g., the correlation between the SDS and AIR at Time 1). Cross-lagged paths estimate the relationship between different constructs at different time points (e.g., the relationship between the SDS at Time 1 and the AIR at Time 2). Chi-square difference tests were used to test whether corresponding paths were equivalent across the treatment and control group.

Figure 2. Cross-Lagged Panel Model.

Note: Intervention group estimates are shown on top.

Although not a primary research question, we also conducted preliminary analyses to determine if there was an effect of disability type on the impact of the SDLMI on self-determination outcomes. A constrained version of the cross-lagged panel model was estimated with the second and third measurements of the AIR and SDS regressed onto a dummy variable indicating type of disability (0 = learning disability; 1 = intellectual disability). The dummy variable was group-mean centered in order to remove all between-campus variance and thus estimate the true within-group effect (i.e. student level) of disability type.

Results

Factorial Invariance

The initial, freely estimated, measurement model demonstrated acceptable model fit. Following standard procedures to evaluate measurement invariance, we equated the loadings and the intercepts (in sequential steps) and, as shown in Table 1, found no significant changes in fit. This non-significant change in model fit indicates that the constructs included in the model and depicted in Figure 1 are invariant (i.e., the loading and intercept values could be equated for each indicator) when measured across the treatment and control groups. Table 2 provides parameter estimates for the strong-invariant model. We also evaluated the structural invariance, using the χ² Difference Test. As shown in Table 1, the test revealed significant differences in the variances and covariances of the latent constructs across the treatment and control group, as well as in the latent means. These differences were expected as a result of the intervention being implemented at different times in the treatment and control groups. The differences in the latent variances were accounted for in the structural models, following standard practices in multi-group models of allowing each group to have their own estimate of latent variances (Little, 1997).

Table 1.

Invariance Testing Across Time & Group

Model	χ2	df	p	Δχ2	Δdf	p	RMSEA	90% C.I.	CFI	TLI	SRMR	Constraint Tenable?
Measurement Invariance
Independence (Null)	3375.604	608	<.001	---	---	---	---	---		---
Configural Invariance	725.552	426	<.001	---	---	---	.067	.059–.076	0.892	0.846	0.087	Yes
Weak Invariance	758.928	456	<.001	---	---	---	.065	.057 – .074	0.891	0.854	0.097	Yes
Strong Invariance	813.088	486	<.001	---	---	---	.066	.058 – .074	0.882	0.852	0.102	Yes
Structural Invariance
Omnibus Test of Vars/Covs*	823.188	487	<.001	64.260	31	<.001	.067	.059 – .075	0.879	0.848	0.134	No
Equal Latent Variances*	782.904	466	<.001	23.976	10	<.001	.066	.058 – .074	0.885	0.851	0.127	No
Equal Latent Means†	829.869	496	<.001	536.067	14	<.001	.066	.058 – .074	0.965	0.955	0.108	No

^*For the chi-square difference test, the weak invariant model served as the parent model

^†For the chi-square difference test, the strong invariant model served as the parent model

χ² = chi-square; df = degrees of freedom; p = probability value; Δχ² = chi-square difference; Δdf = chi-square difference in degrees of freedom; RMSEA = root mean square error of approximation; 90% C.I. = 90% RMSEA confidence interval; CFI = Comparative Fit Index; TLI = Tucker-Lewis Index; SRMR = standardized root mean squared residual

Table 2.

Parameter Estimates for the Strong-Invariant Model

					Completely Standardized Solution
	Unstandardized Solution				Intervention Group						Control Group
Indicator	λ	(S.E.)	τ	(S.E.)	λ	(S.E.)	τ	(S.E.)	θ	R2	λ	(S.E.)	τ	(S.E.)	θ	R2
AIR: Time 1
DO	.63	(.05)	3.67	(.06)	.85	(.02)	4.99	(.31)	.27	.73	.85	(.04)	5.30	(.31)	.28	.73
FE	.66	(.05)	3.79	(.06)	.91	(.03)	5.22	(.30)	.17	.83	.88	(.02)	5.34	(.35)	.23	.77
SC	.57	(.04)	3.73	(.06)	.68	(.04)	4.45	(.26)	.54	.46	.64	(.04)	4.47	(.37)	.59	.41
HO	.53	(.05)	3.80	(.06)	.62	(.05)	4.61	(.32)	.62	.38	.57	(.06)	4.29	(.27)	.68	.32
AIR: Time 2
DO	.63	(.05)	3.67	(.06)	.85	(.02)	3.81	(.73)	.28	.72	.88	(.03)	4.96	(.28)	.22	.78
FE	.66	(.05)	3.79	(.06)	.93	(.03)	4.09	(.73)	.13	.87	.86	(.03)	4.73	(.30)	.26	.74
SC	.57	(.04)	3.73	(.06)	.74	(.10)	3.76	(.47)	.45	.55	.71	(.07)	4.47	(.26)	.50	.50
HO	.53	(.05)	3.80	(.06)	.67	(.09)	3.69	(.35)	.55	.45	.62	(.08)	4.29	(.27)	.62	.38
AIR: Time 3
DO	.63	(.05)	3.67	(.06)	.87	(.03)	5.31	(.38)	.25	.75	.82	(.02)	5.11	(.32)	.33	.67
FE	.66	(.05)	3.79	(.06)	.91	(.02)	5.42	(.46)	.18	.82	.94	(.04)	5.73	(.36)	.12	.88
SC	.57	(.04)	3.73	(.06)	.74	(.06)	5.09	(.38)	.45	.55	.66	(.06)	4.64	(.34)	.56	.44
HO	.53	(.05)	3.80	(.06)	.62	(.07)	4.61	(.32)	.62	.38	.54	(.07)	4.10	(.32)	.71	.29
SDS: Time 1
AUT	.30	(.05)	1.97	(.06)	.56	(.08)	3.70	(.26)	.69	.32	.53	(.07)	4.29	(.22)	.72	.28
SRE	.20	(.03)	1.23	(.06)	.47	(.09)	2.93	(.32)	.78	.22	.33	(.06)	2.57	(.23)	.89	.11
PSY	.13	(.02)	.83	(.02)	.81	(.06)	5.18	(.51)	.35	.66	.69	(.05)	5.44	(.40)	.53	.47
REA	.12	(.02)	.76	(.02)	.70	(.05)	4.61	(.52)	.50	.50	.69	(.07)	5.60	(.44)	.52	.48
SDS: Time 2
AUT	.30	(.05)	1.97	(.06)	.54	(.09)	3.62	(.33)	.71	.29	.49	(.07)	4.17	(.41)	.76	.24
SRE	.20	(.03)	1.23	(.06)	.45	(.08)	2.82	(.24)	.80	.20	.35	(.07)	2.84	(.25)	.88	.12
PSY	.13	(.02)	.83	(.02)	.76	(.07)	4.97	(.49)	.41	.59	.53	(.12)	4.43	(.25)	.72	.28
REA	.12	(.02)	.76	(.02)	.67	(.06)	4.44	(.45)	.55	.45	.53	(.13)	4.49	(.30)	.72	.28
ASDSRC: Time 3
AUT	.30	(.05)	1.97	(.06)	.54	(.09)	3.52	(.28)	.71	.29	.52	(.09)	3.76	(.40)	.73	.27
SRE	.20	(.03)	1.23	(.06)	.43	(.08)	2.68	(.28)	.81	.19	.39	(.09)	2.68	(.21)	.85	.16
PSY	.13	(.02)	.83	(.02)	.80	(.05)	5.09	(.39)	.36	.64	.69	(.08)	4.79	(.45)	.53	.47
REA	.12	(.02)	.76	(.02)	.65	(.04)	4.20	(.59)	.58	.42	.60	(.07)	4.31	(.35)	.64	.36

λ = factor loading; τ = intercept; θ = residual variance; R² = communality; S.E. = standard error

Latent Mean Comparisons

The test of the equality of the latent means in Table 1 shows that the latent means could not be equated across the treatment and control group, indicating that there were significant differences in the latent means across measurement occasions for the treatment and control group. Table 3 presents the latent means and associated 95% confidence intervals for the two scales. We conducted within- and between-group comparisons of the latent means using chi-square difference tests to evaluate for significance, and calculated effect sizes for between-group differences.

Within-Group Comparisons

The intervention group showed significant improvements on both the AIR and SDS from baseline to the final measurement point (i.e., end of Year 2). The intervention group improved from .00 to .30 units on the AIR (d = .31) and from .00 to .24 units on the SDS (d = .24). Both of these increases were significantly different from zero according to chi-square difference tests. Specifically, the strong metric invariance model demonstrated a significant loss of model fit for both the AIR (Δχ² (1) = 7.00, p < .01) and the SDS (Δχ² (1) = 7.36, p < .01) scales when the initial and final latent means were constrained to be equal. The control group, on the other hand, showed only slight increases in self-determination. The control group improved from .16 to .17 units on the AIR (d = .01) and from −.01 to .03 units (d = .05) on the SDS. The changes for the AIR (Δχ² (1) = 0.01, p = .92) and the SDS (Δχ² (1) = 0.11, p = .74) were not significant. The control group actually decreased in self-determination scores between the first and second measurement occasions, but then subsequently increased between the second and third measurement occasions. This pattern suggests that after receiving the intervention at the beginning of Year 2, the students’ negative developmental trajectory for self-determination reversed direction as a result of intervention.

Between-Group Comparisons

Chi-square difference tests revealed no between-group differences in self-determination as measured by the AIR and SDS at the three time points. These findings are not unexpected given the small group sample sizes. As such, between-group effect sizes were calculated and reported in Table 3. For the AIR scale, the control group had a higher initial level of self-determination as compared to the intervention group (d = −.20). However, the intervention groups’ self-determination scores improved at a faster rate compared to the control group as the between-group effect size at time 3 was d = .14. Regarding the SDS scale, the between-group effect size at baseline was d = .01. At the final time point, the effect size was d = .23. Both the AIR and SDS effect sizes are considered small according to conventional guidelines (Cohen, 1977), but expected in educational research. Had the control group not received the intervention at the second time point, one would expect the third time point differences to be larger. Regardless, the effect sizes demonstrate that students that received the SDLMI intervention at baseline had larger increases in self-determination than those students’ receiving the intervention at the second time point.

Structural Models

An unconstrained cross-lagged panel model was first estimated. This initial model demonstrated acceptable fit according to the RMSEA [χ² (464) = 809.919, p = <.01, RMSEA = .069 (90% C.I. = .061—.077, CFI = 0.867]. Next, chi-square difference tests were used to test whether the autoregressive, within-time correlations, and cross-lagged paths were equivalent. For both the AIR and SDS, the autoregressive paths from baseline to the second time point for the intervention group were found to be equivalent to the autoregressive paths from the second time point to the final time point for the control group [Δχ² (1) = 0.388, p = .53; and Δχ² (1) = 0.739, p = .39, respectively], indicating a similar effect of the introduction of the SDLMI in the control and treatment group. A constraint equating the within-time point correlations at baseline in both groups did not result in a significant loss of model fit [Δχ² (1) = 3.461, p = .06]. Similarly, the within-time point constraint equating the second time point for the intervention group and the third time point for the control group (i.e. the first time point after receiving the intervention) did not result in a significant model fit decrease [Δχ² (1) = 0.151, p = .70]. Finally, the cross-lagged regression of the AIR at the second time point onto the ARC at baseline for the intervention group was not significantly different from the cross-lagged regression of the AIR at the final time point onto the ARC at the second time point for the control group [Δχ² (1) = 0.151, p = .70]. The difference test for the other cross-lagged relationship (i.e. time 2 ARC onto time 3 AIR) produced a negative chi-square value and thus was not evaluated. However, as all of the other hypothesized constraints held, it appears that the impact of the SDLMI on students’ self-determination was similar for the intervention and control groups. Standardized estimates of all latent relationships that were significantly different from zero according to the chi-square different tests are presented in Figure 2.

Finally, the effect of disability type on intervention response was evaluated. Estimates, associated confidence intervals, and model fit information can be found in Table 4. Since the variable was dichotomous, the regression coefficients in Table 4 can be interpreted as the difference in means between students with intellectual or learning disability. The means are essentially the residual self-determination information, or in other words the mean amount of change between time points. The model demonstrated acceptable fit [χ² (500) = 714.452, p = <.01, RMSEA = .053 (90% C.I. = .044— .061, CFI = 0.904]. As can be seen in Table 4, those with learning disabilities in the intervention group had larger increases in self-determination by the third time point as opposed to those with intellectual disability. This difference was true for both the AIR (−.14) and the SDS (−.19), although only the AIR mean difference reached significance (i.e. the confidence interval did not contain zero). For the control group, those with intellectual disability reported slightly larger mean increases in self-determination by the final time point for both the AIR (.07) and the SDS (.10), although due to the wide confidence intervals and our limited sample size, these results must be interpreted with caution. These results provide a preliminary indication that students with learning disabilities experienced larger increases in self-determination over time as opposed to students with intellectual disability.

Table 4.

Standardized Effect of Disability on Changes in Self-Determination

	Time 2		Time 3
Group	M	{95% C.I.}	M	{95% C.I.}
AIR
Self-Determination Scale
Intervention Group	.21	{.02 – .21}	−.14	{−.24 – (−.04)}
Control Group	−.05	{−.13 – .02}	.07	{−.05 – .19}
SDS
The Arc’s Self-Determination Scale
Intervention Group	−.17	{−.25– .01}	−.19	{−.35 – .03}
Control Group	−.02	{−.17 – .14}	.10	{−.10 – .29}
Model Fit
χ2(506)	726.58, p < .01
RMSEA	.05 (90% C.I.: .04 – .06)
CFI:	.90

Discussion

Summary of Findings

Measurement Equivalence

An important first step in examining latent constructs across groups is ensuring that the same constructs are being measured in each group (Little, 1997). Establishing strong measurement invariance of each of the latent constructs across our treatment and control group (see Table 1) provided a basis to assume that since constructs were defined in the same operational manner in each group, we could compare outcomes across both groups. Further, by establishing measurement invariance, we were able to account for measurement error and assume that differences in latent means reflect true score differences and not contaminated by differences due to the measurement process (Hancock, 2001). This is a significant advantage of SEM over methods of comparing group differences (Kline, 2005; Little, 1997).

Latent Mean Comparisons

With regard to the first research question, we found that there were significant differences in the latent means across measurement occasions for both measures of self-determination for the treatment and control group. Within the treatment group, we found significant increases in self-determination scores on both the AIR and the SDS between baseline and the end of Year 2; increases not found for the control group. This suggests that the increases in self-determination were a function of the intervention. In fact, rather unexpectedly, when looking at the pattern of latent means presented in Table 3, the control group actually experienced a reduction in self-determination on both measures from Time 1 to Time 2, when they were not receiving intervention.

In terms of comparing changes between groups, our sample size limited our ability to conduct statistical comparisons of differences. However, we did calculate between group effect sizes. When looking at the effect size of the SDLMI at Time 3, we found that the impact of the SDLMI fell in the small range according to Cohen’s (1977) criteria, although effect sizes in this range are viewed as meaningful in education research because of the multiple factors that affect student outcomes. And, this finding in combination with the within-group differences, suggests a significant impact of the SDLMI on student self-determination outcomes.

Structural Models

Our second research question asked whether or not the impact of the SDLMI would be found when the intervention was implemented with different groups at different times. Because of our design we were able to compare the impact of the SDLMI from Time 1 to Time 2 in our treatment group and from Time 2 to Time 3 in our comparison group to determine if there efficacy of the SDLMI was similar in the two groups. The results suggest that the impact of the SDLMI was the same for the intervention and control group even though their exposure to the SDLMI occurred in different years. This strengthens our interpretation that the SDLMI impacts student self-determination outcomes, as the same pattern was seen across two different, randomly assigned groups receiving intervention at different time points.

We also conducted an additional analysis to preliminarily examine differences in the impact of the SDLMI for students with intellectual and learning disability. While our power to detect differences based on disability group was limited because of our sampling plan, the analysis does provide preliminary information to consider in future research. The analysis indicated that there was a differential impact of the SDLMI on the two groups. Students with learning disabilities who received intervention had greater increases in their self-determination scores as compared to students with intellectual disability. However, these differences were only significant on the AIR and not the SDS. Further research is needed to explore the degree to which these differences can be explained by student cognitive capacity, or other factors.

Limitations of the Study

There were several limitations that must be considered in interpreting the results of this study. First, we were not able collect data on students’ intelligence and achievement and relied on teacher estimates of both. Second, we did not explore the multiple individual and ecological factors that may have mediated and moderated the impact of the SDLMI. A limitation with regard to our analysis of the impact of the model by disability label was having a sufficient sample size to conduct such analyses. The primary purpose of our study was to evaluate the impact of the SDLMI on students with cognitive disabilities. Future research is needed that utilizes a sampling plan that will allow direct comparisons between students with diverse disability labels. Relatedly, we experienced attrition from Year 1 to Year 2, as is expected in multi-year studies. There were no significant differences between completer and non-completer characteristics, and we used state of the art procedures to analyze our data; nonetheless, this threat to internal validity must be considered in interpreting the findings.

Another limitation is that the measures were strictly student self-report assessments. We did collect data on a teacher-version of the AIR scale, but because teachers have limited knowledge about a student’s opportunity to engage in self-determination-related activities at home, those scores could not be used. Finally, we were limited in our ability to collect treatment fidelity data. We maximized context and compliance fidelity through our training and ongoing support, and evaluated compliance data by exploring the degree to which teachers completed worksheets and materials with students. However, because the SDLMI is a model of instruction that teachers overlay on the activities they are doing in the classroom, it is difficult to have a standardized assessment of fidelity. Thus, there is a possibility that other factors contributed to the changes in student self-determination, independent of the SDLMI. However, given the clear changes within our treatment group over time and the similarity in impact of the SDLMI when implemented during different years with the treatment and control group, we would argue that the intervention had an impact on student self-determination outcomes.

Implications for Future Research and Practice

It is important that intervention developers in the area of self-determination provide causal evidence of the impact of their intervention on student performance and self-determination. This study provides the second study providing such evidence for the SDLMI. Shogren et al. (2010) provided evidence that after one year of intervention with the SDLMI, high school students with cognitive disabilities receiving instruction with the model had significantly better academic and transition goal attainment outcomes and had greater access to the general education curriculum. The present study provided evidence that instruction with the SDLMI over two years significantly improves student self-determination. It is worth pointing out that the results speak to the importance of sustained efforts to promote self-determination instead of time-limited (one semester or one year) interventions.

The implications for practice are straightforward. The SDLMI provides a model of instruction that teachers can implement to improve student goal attainment and self-determination. Teachers who implement the SDLMI have indicated that their opinion of the capacity of their students to self-direct learning has changed, and almost without exception, teachers believe such efforts are valuable (Lee et al., 2008; Wehmeyer et al., 2000). The SDLMI can be used with any type of goal, across multiple academic and other educational needs or content areas. With minor modifications, it has been shown to be useful in promoting goal attainment for early elementary-age students with disabilities, students in secondary education, and students with disabilities in postsecondary education. Further, the SDLMI addresses skills—including self-regulated problem solving, goal setting, self-monitoring and self-evaluation, among others—that are as important for students without disabilities as for students with disabilities and the model has potential to be implemented schoolwide. Shogren, et al (2010) and Lee and colleagues (2010) provided evidence that the SDLMI can be implemented in the context of general education classrooms across content areas and that such intervention impacts goal attainment, including goals linked to the general education curriculum, and access to the general education curriculum for students with disabilities. The benefits to students without disabilities remains untested, but seems promising.

As to implications for research, there are a number of paths that need exploration beyond specific studies to replicate the current finding. For one, there is a need to evaluate the SDLMI with students without disabilities. At this point, there are no validated measures of self-determination for students without disabilities, but certainly goal attainment can be evaluated and other measures related to self-determination could be evaluated. Research needs to examine specific factors—classroom, student, and teacher variables—that impact the efficacy of the SDLMI. Further, research on the longer-term outcomes for students who receive instruction using the SDLMI, including adult and post-secondary education outcomes, would be beneficial. Finally, randomized trial studies of the model’s impact with students of different ages, including elementary school and post-secondary education students, would provide further evidence of the model’s utility and efficacy.