Skip to main content
NCBI home page
Wiley Open Access Collection logoLink to Wiley Open Access Collection
. 2019 Jan 8;32(3):575–590. doi: 10.1111/jar.12551

Evaluating the effects of multisystemic therapy for adolescents with intellectual disabilities and antisocial or delinquent behaviour and their parents

Annemarieke Blankestein 1,2,✉, Rachel van der Rijken 2,3, Hester V Eeren 2,4, Aurelie Lange 2,4, Ron Scholte 1,2,5, Xavier Moonen 6, Katrien De Vuyst 7, Jo Leunissen 8, Robert Didden 1
PMCID: PMC6850470  PMID: 30620111

Abstract

Background

An adaptation of multisystemic therapy (MST) was piloted to find out whether it would yield better outcomes than standard MST in families where the adolescent not only shows antisocial or delinquent behaviour, but also has an intellectual disability.

Method

To establish the comparative effectiveness of MST‐ID (n = 55) versus standard MST (n = 73), treatment outcomes were compared at the end of treatment and at 6‐month follow‐up. Pre‐treatment differences were controlled for using the propensity score method.

Results

Multisystemic therapy‐ID resulted in reduced police contact and reduced rule breaking behaviour that lasted up to 6 months post‐treatment. Compared to standard MST, MST‐ID more frequently resulted in improvements in parenting skills, family relations, social support, involvement with pro‐social peers and sustained positive behavioural changes. At follow‐up, more adolescents who had received MST‐ID were still living at home.

Conclusions

These results support further development of and research into the MST‐ID adaptation.

Keywords: delinquency, intellectual disability, multisystemic therapy, out‐of‐home placement, treatment effects

1. INTRODUCTION

Adolescents with intellectual disabilities1 and their families are predisposed to a variety of problems. Severe behaviour problems are seen three times as often in adolescents with borderline intellectual functioning or mild intellectual disabilities as in individuals without intellectual disabilities (De Ruiter, Dekker, Verhulst, & Koot, 2007; Emerson, Einfeld, & Stancliffe, 2011; Wallander, Dekker, & Koot, 2003). Adolescents with intellectual disabilities are at increased risk of engaging in offending behaviour, of re‐offending and of becoming involved with the juvenile justice system (McReynolds, Schwalbe, & Wasserman, 2010; Thompson & Morris, 2016). More specifically, research has shown that 10%–30% of youths in detention have intellectual disabilities (Kaal, Overvest, & Boertjes, 2014; Thompson & Morris, 2016). Without intervention, the behaviour problems of adolescents with intellectual disabilities often persist (Emerson et al., 2011).

Parents of adolescents with intellectual disabilities often report higher levels of parenting stress than parents of typically developing adolescents (Patton, Ware, McPherson, Emerson, & Lennox, 2016). High levels of parenting stress can lead to negative child outcomes such as insecure attachment, neglect and abuse in children and are associated with negative parenting styles (Meppelder, Hodes, Kef, & Schuengel, 2015; Neece & Lima, 2016; Powell & Parish, 2017). A combination of academic‐related disability or intellectual disability, abuse and co‐occurring mental health problems substantially increases the risk of youth delinquency. As a result, some adolescents get stuck in an offending recidivism cycle which places them at risk of incarceration (Mallett, 2014; Thompson & Morris, 2016).

In some cases, both the adolescents and their parent(s) have intellectual disabilities. Such families often experience multiple problems, such as financial problems or mental health problems (Schuiringa, Van Nieuwenhuijzen, Orobio de Castro, & Matthys, 2015), and frequently lack problem‐solving skills, which may, for instance, lead to care re‐entry. Moreover, transgenerational transmission of psychosocial and socioeconomic problems has been observed in these families (Tausendfreund, Knot‐Dickscheit, Schulze, Knorth, & Grietens, 2016). One of the biggest challenges for these families is that they have a limited social network. This may be worrisome because a (larger) social network can serve as a buffering mechanism to parenting stress (Llewellyn & Hindmarsh, 2015; Meppelder et al., 2015).

Because of the accumulation of risk factors for adolescents with intellectual disabilities and their families, these families are often involved with youth care. Research has shown that adolescents from families involved with youth care are twice more likely to be placed out of home than are adolescents from families not involved with youth care (Lightfoot, Hill, & LaLiberte, 2011). Though out‐of‐home placement is sometimes inevitable and necessary to avoid further escalation of problems or to guarantee child safety, it leads to high emotional and societal financial costs (Allen, Lowe, Moore, & Brophy, 2007; Lee et al., 2014; Vermeulen, Jansen, Knorth, Buskens, & Reijneveld, 2017). Research suggests that families experiencing a multitude of difficulties, such as families with members who have an intellectual disability, are best treated with home‐based, flexible, integrated and multicomponent services (Tausendfreund et al., 2016). Through home‐based treatment, out‐of‐home placement may be prevented or postponed.

A home‐ and community‐based intervention known to reduce the number of out‐of‐home placements, and recidivism amongst juveniles with antisocial or delinquent behaviour is multisystemic therapy (MST; Henggeler, Schoenwald, Borduin, Rowland, & Cunningham, 2009). MST targets 12‐ to 18‐year‐old adolescents at risk of out‐of‐home placement due to their severe problem behaviour. Based on Bronfenbrenner's (1979) social‐ecological model, MST assumes that the adolescent's antisocial behaviour is driven by the interplay of risk factors in the systems surrounding the adolescent, such as family, friends, school and neighbourhood. Because of its multisystemic nature, MST seems a promising intervention for the prevention of impending out‐of‐home placement and incarceration of adolescents with intellectual disabilities and antisocial or delinquent behaviour. To our knowledge, however, the effectiveness of MST has not been evaluated in a sample consisting of only adolescents with intellectual disabilities. In addition, although one of the MST treatment principles states that interventions should be appropriate to the youth's age and developmental needs (Henggeler et al., 2009), the treatment manual does not include any specific guidance on how to deliver MST to family members with intellectual disabilities. In fact, it seems that MST therapists have some difficulty treating adolescents with intellectual disabilities, since a previous pilot study showed that, after standard MST, adolescents with intellectual disabilities were placed out of home more frequently than adolescents without intellectual disabilities. In addition, keeping or getting adolescents with intellectual disabilities at school or work seemed more difficult (Lange & Van der Rijken, 2012). As a consequence, standard MST was hypothesized to not optimally suit the needs and characteristics of adolescents with intellectual disabilities and their families and an adaptation of standard MST, MST‐ID2, was piloted.

The present study's aim was to evaluate the effects of MST‐ID in a sample of adolescents with intellectual disabilities and antisocial or delinquent behaviour, and their parents. The present authors hypothesized that (a) MST‐ID would show positive treatment outcomes and sustain these up to 6‐month follow‐up and that (b) treatment outcomes would be better for MST‐ID compared to standard MST.

2. METHOD

2.1. Participants and procedure

Table 1 displays the baseline characteristics of the 128 families included in the study. It shows that 43.6% and 35.6% of the adolescents receiving MST‐ID and standard MST, respectively, were female, that the average ages were 15.2 and 14.9 years, respectively, and that 94.5% and 95.9% of the adolescents were born in the Netherlands.

Table 1.

Baseline differences between MST‐ID and standard MST and standardized bias in full sample (N = 128)

Variable MST‐ID (N = 55) Standard MST (N = 73) Test Statistic Standardised Bias
Mean SD Mean SD t test Before PS application After PS application
Age 15.20 1.73 14.90 1.38 −0.963 0.158 0.138
CBCL
Internalizing problems 61.10 9.49 61.30 8.22 0.148 −0.025 −0.156
Externalizing problems 65.40 8.85 68.80 7.78 2.321* −0.388 0.017
Total behavioural problems 64.20 9.74 67.00 6.46 1.826 −0.285 −0.061
YSR
Internalizing problems 52.70 9.01 52.80 9.52 0.033 −0.006 −0.226
Externalizing problems 57.20 11.22 60.50 8.42 1.76 −0.297 −0.006
Total behavioural problems 54.30 10.91 57.10 8.01 1.478 −0.248 −0.077
OBVL
Total parenting stress 66.70 11.14 69.90 8.60 1.834 −0.287 0.068
SCIL
SCIL score primary caregiver 17.50 5.37 21.20 4.40 4.139*** −0.685 −0.149
WISC/WAIS
TIQ score youth 73.90 6.70 75.10 7.21 0.936 −0.177 −0.021
% % Chi‐Square
Gender
Female 43.6 35.6 0.848 0.160 −0.075
Country of birth
The Netherlands 94.5 95.9 1.351 −0.029 0.000
Western country 0.0 1.4 −0.030 0.000
Non‐Western country 5.5 2.7 0.059 0.000
Living situation adolescent
Together with one parent 56.4 61.6 2.824 −0.092 0.127
Together with multiple parents 40.0 38.4 0.029 −0.127
Other 3.6 0.0 0.063 0.000
Living situation adolescent
Lived at home 96.4 100.0 2.697 −0.192 0.000
Level of education
None/primary/special/polytechnic education 74.5 50.0 7.870** 0.558 −0.124
Lower secondary education (vmbo/mavo/mbo) 25.5 50.0 −0.558 0.124
Higher secondary education (havo/vwo) 0.0 0.0 0.000 0.000
Previous treatment
Present 90.7 93.2 0.249 −0.082 −0.030
Engagement in school or work
Present 70.4 56.9 2.378 0.291 0.005
Court order
No 32.7 53.4 5.524 −0.270 0.065
Civil 41.8 27.4 0.188 0.101
Criminal 25.5 19.2 0.082 −0.166
Police contacts up to 6 months prior to treatment
Absent 49.1 54.2 0.322 −0.101 0.189
Relation father
Present 80.0 93.2 4.960* −0.326 −0.011
Relation mother
Present 98.2 98.6 0.041 −0.033 0.000
Relation siblings
Present 90.9 95.9 1.328 −0.172 −0.140
Relation peers
Present 100.0 98.6 0.759 0.000 0.000
Country of birth primary caregiver
The Netherlands 76.4 76.4 0.305 0.000 0.020
Western country 3.6 5.6 −0.024 −0.160
Non‐Western country 20.0 18.1 0.024 0.140
Level of education primary caregiver
None/primary/special/polytechnic education 34.5 12.3 9.935** 0.328 −0.025
Lower secondary education (vmbo/mavo/mbo) 50.9 60.3 −0.138 −0.005
Higher secondary education (havo/vwo) 14.5 27.4 −0.190 0.030
Employment primary caregiver
Employed 41.8 43.1 0.020 −0.025 −0.189
Partner primary caregiver
Present 78.8 70.0 1.207 0.215 0.055
Open in a new tab

CBCL: Child Behaviour Checklist; MST: multisystemic therapy; OBVL: Opvoedingsbelasting Vragenlijst; PS: propensity score; SCIL: Screener for Intelligence and Learning Disabilities; WAIS: Wechsler Adult Intelligence Scale; WISC: Wechsler Intelligence Scale for Children; YSR: Youth Self Report; TIQ: total IQ.

* p < 0.05, ** p < 0.01,*** p < 0.001.

Multisystemic therapy‐ID was provided by two teams from one organization (specializing in care for people with intellectual disabilities) in the Netherlands. Standard MST was offered by 24 teams from seven Dutch organizations (offering clinical inpatient and outpatient care). Participants were not randomly assigned to the treatment conditions. Randomization was not used because the teams offering standard MST were not allowed to change their inclusion criteria to only treat adolescents with intellectual disabilities. Therefore, MST‐ID and standard MST were studied in their everyday clinical practice settings. Dutch referral agencies referring families to standard MST and MST‐ID include primary healthcare providers, the Child Protection Council, juvenile judges and referral institutions at the council level. Additionally, as the organization offering MST‐ID specializes in intellectual disability care, families are also referred to this treatment by other intellectual disability care agencies.

All MST therapists had completed higher education in social sciences. They also completed the 5‐day MST training, participated in weekly supervision and expert consultation meetings, and attended quarterly booster sessions (Henggeler et al., 2009). Between March 2014 and October 2015, all teams were asked to refer adolescents with a known IQ score between 50 and 85 (i.e., intellectual disabilities) and their primary caregivers (from here on referred to as parents) to the research team. If IQ scores were unknown, therapists could refer adolescents based on an educational level of vmbo‐t (the Dutch equivalent of vocational education) or lower, because adolescents with this level of education are much more likely to have intellectual disabilities than adolescents with higher educational levels (Kaal, Nijman, & Moonen, 2015). To verify the presence of intellectual disabilities in adolescents who were referred based on their educational level, their IQ was tested using the Dutch Wechsler Intelligence Scale for Children III—Short Form (Wechsler, 2005) or the Dutch Wechsler Adult Intelligence Scale III—Short Form (Wechsler, 2000) depending on their age. To participate in the research, adolescents and their parents had to have sufficient proficiency in the Dutch language. That is, an interpreter did not need to be present in order for parents to be able to answer the questionnaires.

Families referred to the research team were asked to sign consent in order to take part in the study. The study was approved by the Committee Scientific Research Participation of one of the participating mental health care agencies and complied to the American Psychological Association's ethical principles regarding research with human participants. Of the 247 families who were referred to the research team, 33 families were excluded for one of the following reasons: The adolescent was too young (aged <12 years) (n = 1), families had insufficient knowledge of the Dutch language (n = 2), the adolescent did not have intellectual disabilities according to the results of the IQ test (n = 15), the presence of intellectual disabilities could not be assessed because the adolescent refused testing (n = 5), or the adolescent received other treatments simultaneously with MST (n = 10). Of the 214 families who met the inclusion criteria, 128 families (60%) gave written informed consent. The final sample consisted of 55 families receiving MST‐ID and 73 families receiving standard MST. When families did not give consent, baseline data were not collected. Therefore, analyses comparing families giving and not giving consent were not conducted.

A set of questionnaires was filled in by therapists and by parents at the start of the treatment, at the end of the treatment and 6 months after finishing the treatment (follow‐up). Home visits were conducted by the research team at the start and at the end of the treatment to administer the questionnaires. Six months after the treatment, the parents were contacted by the independent call centre “Kwestion” for a telephone interview entailing a set of follow‐up questionnaires. Six months after treatment, 11 families could not be reached (MST‐ID n = 4, standard MST n = 7). Of the 117 families (91%) that could be contacted at follow‐up, 87 families gave consent for the interview (74%). Eight families did not want to take part (7%), 20 families were unavailable at the time (17%), and two families could not take part for other reasons (2%).

2.2. Interventions

Multisystemic therapy is aimed at adolescents aged 12–18 years who display antisocial or delinquent behaviour and are at risk of out‐of‐home placement. It is a multisystemic intervention with a duration of three to 5 months (Henggeler et al., 2009). In MST, caregivers are key to achieving and sustaining long‐term outcomes in the reduction in juvenile externalizing behaviour. Therefore, the development of parental skills and empowerment of parents are main components of MST. Ultimately, MST aims to create a supportive context that encourages adaptive behaviour in adolescents and parents, while mobilizing or strengthening support systems for the family (Henggeler & Schaeffer, 2016).

Studies evaluating the effectiveness of MST compared to treatment as usual (TAU), and including follow‐up data, show a reduction in out‐of‐home placements up until 2 years after treatment for adolescents receiving MST in the United States of America and Norway (Ogden & Hagen, 2006; Vidal, Steeger, Caron, Lasher, & Connell, 2017). Different results were found in England, where Butler, Baruch, Hickey, and Fonagy (2011) and Fonagy et al. (2018) reported that at 18‐month follow‐up, no differences existed between the number of out‐of‐home placements. In Canada, Cunningham (2002) concluded that MST showed no distinguishable treatment outcomes, which McIntosh (2015) later refuted, showing clinically significant treatment improvements for families receiving MST treatment. Thus, research suggests that results of MST vary across contexts (Van der Stouwe, Asscher, Stams, Deković, & Van der Laan, 2014).

In the Dutch context, MST has been shown to lead to a reduction in externalizing problem behaviour and higher parenting competence lasting until 3 years after treatment (Asscher et al., 2014; Asscher, Deković, Manders, Van der Laan, Prins, & Dutch MST Cost‐Effectiveness Study Group, 2013).

Over the years, adaptations of standard MST have been developed and scientifically evaluated to suit the needs and characteristics of a number of different target populations (for an overview, see https://mstservices.com/target-populations/target-populations). Adaptations of MST follow a standardized procedure of development as described in detail in Schoenwald (2014).

In the current study, a new adaptation of MST, MST‐ID, was piloted. Research has shown that the needs of families with intellectual disabilities are different from families whose members do not have intellectual disabilities (Neece & Lima, 2016; Schuiringa, Van Nieuwenhuijzen, Orobio de Castro, Lochman, & Matthys, 2017; Soenen, Van Berckelaer‐Onnes, & Scholte, 2016). Therefore, the Dutch Knowledge Centre on MID has provided guidelines on how to adapt interventions to the strengths and needs of individuals with intellectual disabilities (De Wit, Moonen, & Douma, 2012). For MST‐ID, incorporating these guidelines has resulted in training of therapists in the identification of an intellectual disability, the identification of parental stress and how this is affected by the intellectual disabilities of the adolescent, techniques to motivate families to enter the treatment and engage them in the treatment, promoting active involvement of the social network and paying special attention to generalization of acquired knowledge or skills. Furthermore, it has led to a specific focus on adaptations made to the use of language (i.e., using easier language), adding visual cues and simplification of content of treatment sessions by focussing on one assignment.

As in any MST treatment, therapist adherence to the treatment principles was independently monitored using monthly telephone interviews with parents. Parents scored the 28 items of the Therapy Adherence Measure—Revised (TAM‐R; Henggeler, Borduin, Schoenwald, Huey, & Chapman, 2006) on a scale of 1–5 with a score of 1 meaning “not at all” and a score of 5 “very much.” The average therapist adherence scores were 4.35 (SD = 0.56) for MST‐ID and 4.38 (SD = 0.62) for standard MST. These scores are similar to TAM‐R scores seen in American research on standard MST (M = 4.41; SD = 0.49, Letourneau, Sheidow, & Schoenwald, 2002) as well as in a Dutch RCT that evaluated the effectiveness of standard MST in individuals without intellectual disabilities (M = 4.36; SD = 0.51, Manders, Deković, Asscher, Van der Laan, & Prins, 2011). In the present study, the level of therapist adherence did not differ between MST‐ID and standard MST (t(125) = 0.304, p = 0.76). Standard MST and MST‐ID therapists thus adhered to the treatment principles equally well. MST‐ID mean treatment duration was 5.1 months (range: 2–8 months) and the mean duration of standard MST was 4.4 months (range: 2–7 months).

2.3. Instruments

2.3.1. SDI

A set of background variables was measured at the start of the treatment using the SDI questionnaire (Sociodemographic Information; MST‐NL, 2012). Therapists reported a variety of family demographics detailed in Table 1.

2.3.2. Wechsler IQ tests

IQ was assessed using a short form of the Dutch Wechsler Intelligence Scale for Children (WISC‐III‐NL; Wechsler, 2005) in adolescents up until the age of seventeen. For adolescents aged 17–18, the Wechsler Adult Intelligence Scale—Short Form (WAIS‐III‐NL; Wechsler, 2000) was used. The short form of the WISC‐III‐NL included the subtests picture completion, information, block design, symbol search and vocabulary. For the WAIS‐III‐NL, the subtests included were vocabulary, similarities, block design and matrix reasoning. The short form of the WISC‐III‐NL has been validated for use in individuals with intellectual disabilities with a high internal consistency (r = 0.96; De Ruiter, Dekker, Douma, Verhulst, & Koot, 2008). The short form of the WAIS‐III‐NL has been shown to have a high correlation (r = 0.89) with the total IQ score within a Dutch population of individuals with intellectual disabilities (Van Duijvenbode, Didden, Van den Hazel, & Engels, 2016).

2.3.3. SCIL

Parents were asked to complete the Dutch Screener for Intelligence and Learning Disabilities 18+ (SCIL 18+; Nijman, Kaal, Van Scheppingen, & Moonen, 2016) to screen for the presence or absence of intellectual disabilities. The screener consists of 14 questions that result in a total SCIL score that can range from 2 to 28. A total SCIL score of 20 and above indicates the absence of intellectual disabilities. A total SCIL score of 19 and below indicates the presence of intellectual disabilities. The screener gives a valid indication of whether or not a person's IQ is below 85 and shows a good test–retest reliability of r = 0.92 (Nijman et al., 2016).

2.3.4. CBCL and YSR

Adolescents’ problem behaviour was measured using the Child Behaviour Checklist (CBCL 6–18; Achenbach & Rescorla, 2001) as completed by the parents and the Youth Self Report (YSR; Achenbach & Rescorla, 2001) as completed by the adolescents. The subscales internalizing, externalizing and rule‐breaking behaviour were measured as well as the total problem behaviour scale. Answers were given on a three‐point scale ranging from 0 “Never” to 2 “Often.” T scores were computed and used for analyses. Higher T scores indicate that adolescents experienced more problems or were believed to experience more problems by the parents. The test–retest reliability of the CBCL (sub)scales (r = 0.91 for internalizing behaviour; r = 0.92 for externalizing behaviour; r = 0.94 for total problem behaviour; r = 0.91 for rule‐breaking behaviour) and the YSR (sub)scales (r = 0.80 for internalizing behaviour; r = 0.89 for externalizing behaviour; r = 0.87 for total problem behaviour) used in this study is good. Research has shown that Cronbach's alphas for the CBCL 6–18 were higher for parents of children with intellectual disabilities than for parents of children without intellectual disabilities (Dekker, Koot, Van der Ende, & Verhulst, 2002).

2.3.5. OBVL

Parenting stress was assessed using the Opvoedingsbelasting Vragenlijst (OBVL, Burden of Parenting Questionnaire; Vermulst, Kroes, De Meyer, Nguyen, & Veerman, 2012). Parents completed this self‐report instrument which consists of 34 items. Answers range from 1 “Not at all true” to 4 “Completely true.” Scores on all items were summed up to compute a T score for total parenting stress. A higher T score indicates a higher level of parenting stress. The reliability of total parenting stress measured by the OBVL is good, with a Cronbach's alpha of 0.89 (Vermulst et al., 2012).

2.3.6. Primary treatment outcomes

The three main outcomes of the MST quality assurance system were measured at the end of treatment and at 6‐month follow‐up: (a) The adolescent is living at home (yes/no); (b) the adolescent attends school or works for at least 20 hours a week (yes/no); and (c) the adolescent has not been involved with the police since the start of treatment (measured at the end of treatment)/the adolescent has not been involved with the police in the previous 6 months (measured at follow‐up) (yes/no). At the end of treatment, therapists reported the outcomes using the SDI questionnaire (MST‐NL, 2012). These reports are discussed with the team supervisor and the MST consultant from MST‐Netherlands. This means that the treatment outcomes are monitored by multiple parties. At follow‐up, parents reported on the aforementioned primary outcomes in the telephone interview.

2.3.7. Secondary treatment outcomes

In addition to the primary treatment outcomes, MST's “instrumental outcomes” were assessed. These instrumental outcomes include six items that identify skills which are “instrumental” to achieving positive treatment outcomes and are reported by therapists. The instrumental outcomes measure whether or not families show (a) improved parenting skills, (b) improved family relations and (c) improved social support, and whether or not the adolescent (d) obtained success in an educational or vocational setting, (e) is involved with pro‐social peers and (f) obtained changes in problem behaviour that sustained for 3–4 weeks (MST‐NL, 2012).

In addition to the instrumental outcomes, the subscales externalizing problem behaviour and rule‐breaking behaviour from the CBCL and total parenting stress measured with the OBVL were used as secondary treatment outcomes at the end of treatment. At follow‐up, only the CBCL subscale rule‐breaking behaviour was used and the OBVL was not re‐administered to minimize the number of questions parents had to answer. The CBCL subscale rule‐breaking behaviour was considered the most relevant to our target population.

2.4. Statistical analyses

2.4.1. Analyses of MST‐ID treatment outcomes

In order to evaluate the results of MST‐ID up to 6 months post‐treatment, pre‐test–post‐test‐follow‐up differences were analysed within the MST‐ID group. Two‐sided Friedman ANOVAs and resulting chi‐squares were used for dichotomous variables and repeated measures ANOVAs for continuous variables. Analyses were performed in IBM SPSS Statistics version 23.

2.4.2. Comparative treatment effects

Because families were not randomly assigned to one of the treatments, adolescents assigned to either MST‐ID or standard MST could differ on pre‐treatment variables. If differences existed, the propensity score (PS) method would be used to adjust for this allocation bias. The PS is a balancing score which can be used to achieve a balanced distribution of the observed covariates of the intervention and the control group, while also balancing the missingness on these variables. The PS represents the probability for a given adolescent of being allocated to MST‐ID or standard MST, based on all pre‐treatment variables. Adolescents with a similar PS are assumed to be comparable on the distribution of the pre‐treatment variables. After estimation of the PS, this score can be used to balance the two treatment conditions in order to allow for a comparison on the treatment outcomes (Austin, 2011; Rubin, 2001). It was assumed that balance was achieved when standardized biases did not exceed 0.25 (Harder, Stuart, & Anthony, 2010; West et al., 2014). The PS was estimated in a univariate logistic regression function with the treatment groups (MST‐ID or standard MST) as the dependent variable. All observed pre‐treatment variables, as well as missing indicators for all pre‐treatment variables with missing data, were included as predictors in the PS model (Ali et al., 2014; Austin, 2011; Brookhart et al., 2006; Stuart, 2010). The inclusion of missing indicators enabled us to also include families with missing data in the PS estimation, as well as include the missing data patterns in the PS estimation (Cham & West, 2016; Harder et al., 2010).

Application of the PS by weighting

The PS was applied by weighting the groups by the odds of their estimated PS scores (Stuart, 2010). With this procedure, individuals in standard MST best matching individuals in MST‐ID are “upweighted,” whereas individuals whose covariate values are dissimilar from treated individuals are “downweigthed.” As a result of the weighting procedure, the average treatment effect of the treated (ATT) was estimated (Stuart, 2010). This is the effect that would be found if all families treated with MST‐ID had been treated with standard MST.

Analysis of treatment effect

To estimate treatment effect estimates in the weighted sample for all outcome measures, regression analysis was used. The post‐treatment effect on dichotomous outcomes and the effect at 6‐month follow‐up were estimated using logistic regression. The results were used to estimate average risk ratios (RRs; Austin & Small, 2014). The treatment effects on the continuous outcome measures were assessed using OLS regression. Thereafter, simple bootstrapping was used to calculate 90% confidence intervals for all outcome measures. In total, 5,000 bootstrap samples were drawn from the weighted sample, and in each bootstrapped sample, treatment effects were estimated as described (Austin & Small, 2014). Analyses were performed in IBM SPSS Statistics version 23 and Stata version 12. Because treatment effects might be different when not only the adolescent, but also the parent has intellectual disabilities, the present authors also explored the differential treatment effects in a subgroup of adolescents and parents with intellectual disabilities.

3. RESULTS

3.1. Participant characteristics

Figure 1 shows a flow chart detailing the number of families included at various points in time. Table 1 displays the demographic characteristics of the 128 families included in the study. The adolescents receiving MST‐ID had significantly lower educational levels and less often a father figure was present. The adolescents’ externalizing problems also differed significantly; parents of adolescents receiving MST‐ID reported significantly lower levels of externalizing problems than did parents of adolescents receiving standard MST. Furthermore, the parents of adolescents receiving MST‐ID had significantly lower educational levels and had lower SCIL scores.

Figure 1.

Figure 1

Open in a new tab

Flow chart detailing number of families included at various points in time [Colour figure can be viewed at http://wileyonlinelibrary.com]

3.2. MST‐ID treatment outcomes

The present authors tested treatment effects for MST‐ID from pre‐treatment to 6‐month follow‐up using repeated measures analyses for dichotomous variables (Friedman test). Table 2 shows the results of these analyses. The percentage of adolescents with police contact after treatment reduced significantly (χ 2(2) = 15.91, p < 0.01). Post hoc analyses (see Table 2) revealed that the presence of police contact was reduced between the start of the treatment and the end of the treatment and that this effect was maintained at follow‐up. No significant differences between pre‐ and post‐tests were found for engagement in school or work (χ 2(2) = 3.65, p = 0.16) or adolescents living at home (χ 2(2) = 1.00, p = 0.61). Therefore, post hoc results were not applicable.

Table 2.

Treatment outcomes for MST‐ID (N = 55)

Outcome variable Pre‐test Post‐test Follow‐up Pre‐post Z‐score Pre‐follow‐up Z‐score Post‐follow‐up Z‐score
% % %
No police contact 49.1 78.2 80.0 − 2 . 968 ** − 3 . 500 *** −0.302
Engagement in school or work 70.4 85.5 72.2 N/a N/a N/a
Living at home 96.4 96.4 100.0 N/a N/a N/a
M (SD) M (SD) M (SD) Mdiff (SE) pre‐post Mdiff (SE) pre‐follow‐up Mdiff (SE) post‐follow‐up
Rule breaking behaviour 66.00 (8.19) 62.46 (7.33) 62.19 (8.65) 4 . 00 (0.94) ** 4 . 77 (1.29) ** 0.77 (1.30)
Open in a new tab

Significant results are marked in italics.

MST: multisystemic therapy.

** p < 0.01, *** p  < 0.001.

A repeated measures ANOVA showed that there was an effect on rule‐breaking behaviour (F(1, 33) = 13.59, p < 0.01). Post hoc results (see Table 2) revealed that there was a significant reduction in rule‐breaking behaviour between the start and the end of the treatment and between the start and 6‐month follow‐up. This means that rule‐breaking behaviour decreased during treatment and that this effect maintained until 6 months after treatment.

3.3. Comparative treatment effects

3.3.1. Balance assessment

To analyse the comparative effects of MST‐ID and standard MST, the present authors first evaluated whether balance between the two treatment groups could be achieved using the PS method. For this purpose, the standardized biases were assessed before and after PS application (see Table 1). The standardized bias of all pre‐treatment variables as well as the missing indicators included in the PS estimation was lower than 0.25, which means that balance was achieved after removing families with non‐overlapping PS scores (i.e., a PS score that did not fall in the range of PS scores that was observed in the other treatment group). Though this restricts the generalizability of the results to the cases for which overlap was present, removing those cases allows for balancing the treatment conditions more precisely (Harder et al., 2010). Excluding families with a non‐overlapping PS resulted in a balanced sample of 30 families who received MST‐ID and 33 families who received standard MST (25 families who received MST‐ID and 40 families who received standard MST were excluded).

Families with a non‐overlapping PS who received MST‐ID differed too much from the families who received standard MST to allow for comparison. Therefore, the present authors looked into the differences between the overlapping and non‐overlapping groups within MST‐ID (Table 3). Compared to the families who received MST‐ID and who were included in the analyses, the excluded MST‐ID families reported significantly lower levels of adolescents’ externalizing problems, lower levels of total behavioural problems, fewer family situations in which a father figure was present, lower educational levels of parents and lower SCIL scores of parents.

Table 3.

Baseline differences within MST‐ID between overlapping group and non‐overlapping PS group

Variable Non‐overlapping group (N = 25) Overlapping group (N = 30) Test statistic
Mean SD Mean SD t test
Age 14.92 2.00 15.37 1.47 0.927
CBCL
Internalizing problems 59.04 10.96 62.77 7.84 1.423
Externalizing problems 61.44 9.14 68.60 7.24 3.242**
Total behavioural problems 60.48 10.75 67.30 7.69 2.737**
YSR
Internalizing problems 50.29 9.46 54.92 8.14 1.86
Externalizing problems 53.42 9.33 60.65 11.86 2.385*
Total behavioural problems 50.88 10.70 57.54 10.29 2.245*
OBVL
Total parenting stress 63.64 11.59 69.23 10.26 1.899
SCIL
SCIL score primary caregiver 15.80 5.27 18.97 5.10 2.258*
WISC/WAIS
TIQ score youth 73.83 6.99 74.00 6.60 0.093
% % Chi‐Square
Gender
Female 48.0 40.0 0.355
Country of birth
The Netherlands 88.0 100.0 3.808
Western country 0.0 0.0
Non‐Western country 12.0 0.0
Living situation adolescent
Together with one parent 60.0 53.3 3.241
Together with multiple parents 32.0 46.7
Other 8.0 0.0
Living situation adolescent
Lived at home 92.0 100.0 2.491
Level of education
None/primary/special/polytechnic education 80.0 70.0 0.719
Lower secondary education (vmbo/mavo/mbo) 20.0 30.0
Higher secondary education (havo/vwo) 0.0 0.0
Previous treatment
Present 91.7 90.0 0.044
Engagement in school or work
Present 72.0 69.0 0.059
Court order
No 28.0 36.7 1.985
Civil 52.0 33.3
Criminal 20.0 30.0
Police contacts up to 6 months prior to treatment
Absent 44.0 53.3 0.475
Relation father
Present 60.0 96.7 11.458**
Relation mother
Present 96.0 100.0 1.222
Relation siblings
Present 88.0 93.3 0.469
Relation peers
Present 100.0 100.0 N/a
Country of birth primary caregiver
The Netherlands 68.0 83.3 3.187
Western country 8.0 0.0
Non‐Western country 24.0 16.7
Level of education primary caregiver
None/primary/special/polytechnic education 56.0 16.7 9.458**
Lower secondary education (vmbo/mavo/mbo) 36.0 63.3
Higher secondary education (havo/vwo) 8.0 20.0
Employment primary caregiver
Employed 36.0 46.7 0.638
Partner primary caregiver
Present 75.0 82.1 0.395
Open in a new tab

CBCL: Child Behaviour Checklist; MST: multisystemic therapy; OBVL: Opvoedingsbelasting Vragenlijst; PS: propensity score; SCIL: Screener for Intelligence and Learning Disabilities; WAIS: Wechsler Adult Intelligence Scale; WISC: Wechsler Intelligence Scale for Children; YSR: Youth Self Report; TIQ: total IQ.

* p <0.05, ** p <0.01.

3.3.2. Analysis of treatment effect

Based on the analyses of data from the subsample of 63 families retained following the PS, Table 4 shows that there were no significant between‐group differences on the primary outcome measures at the end of the treatment. At 6‐month follow‐up, however, significantly more adolescents lived at home after MST‐ID than did adolescents after having received standard MST (see Table 4).

Table 4.

Comparative treatment effect of MST‐ID and standard MST post‐treatment and at 6‐month follow‐up

Post‐treatment outcomes RR 90% CI
MST‐ID (N = 30) Standard MST (N = 33)
% %
Primary outcomes
No police contact 76.7 66.7 0.700 0.311–1.901
Engagement in school or work 80.0 81.8 0.978 0.790–1.279
Living at home 93.3 93.9 0.994 0.909–1.075
Secondary outcomes
Improved parenting skills 93 . 3 75 . 8 1 . 232 1 . 031–1.587
Improved family relations 100 . 0 75 . 8 1 . 280 1 . 078–1.618
Improved social support 96 . 7 81 . 8 1 . 181 1 . 049–1.473
Success in educational setting 83.3 78.8 1.026 0.834–1.312
Involved with pro‐social peers 93 . 3 78 . 8 1 . 185 1 . 022–1.519
Changes in problem behaviour lasting a minimum of 3–4 weeks 93 . 3 78 . 8 1 . 149 1 . 001–1.449
M (SD) M (SD) B 90% CI
Externalizing problems 63.15 (6.97) 67.14 (8.74) −3.991 −8.107 − 0.384
Total parenting stress 63.65 (10.99) 63.93 (12.44) −0.274 −6.005 − 6.006
Treatment outcomes at follow‐up RR 90% CI
MST‐ID (N = 20) Standard MST (N = 17)
% %
Primary outcomes
No police contact 78.9 70.6 0.716 0.198–2.295
Engagement in school or work 70.0 76.5 0.915 0.655–1.265
Living at home 100 . 0 76 . 5 1 . 308 1 . 084–1.693
M (SD) M (SD) B 90% CI
Secondary outcome
Rule breaking behaviour 64.25 (7.38) 63.75 (9.92) −0.496 −4.632 − 5.439
Open in a new tab

Significant results are marked in italics.

MST: multisystemic therapy.

On the secondary outcomes, five out of six “instrumental outcomes” differed significantly between MST‐ID and standard MST. Families who had received MST‐ID showed significantly higher percentages of improved parenting skills, improved family relations, improved social support, involvement with pro‐social peers and changes in problem behaviours in contrast to families who had received a standard MST treatment.

The differential treatment effect in the subgroup where both the adolescents and the parents had intellectual disabilities (n = 48) could not be established, because within this subsample, balance between MST‐ID and standard MST could not be achieved using the PS. This meant that the subgroup treatment samples were too different to compare.

4. DISCUSSION

The current study evaluated the effects of MST‐ID, therewith piloting this adaptation of standard MST. MST‐ID targets adolescents with intellectual disabilities in combination with antisocial or delinquent behavioural problems and their parents. Following our first hypothesis, the present authors found that MST‐ID significantly reduced adolescents’ rule‐breaking behaviour, which dropped from a subclinical mean score at the start of treatment to an average range mean score post‐treatment and at 6‐month follow‐up. The percentage of adolescents with police contact was also significantly reduced after MST‐ID, dropping from 51% to 20% at follow‐up. Thus, as hypothesized, MST‐ID showed positive treatment outcomes which were sustained up to 6 months after treatment. Because a previous pilot study showed that adolescents with intellectual disabilities were placed out of home more frequently than adolescents without intellectual disabilities following standard MST (Lange & Van der Rijken, 2012), the current study also aimed to compare the effects of MST‐ID and standard MST in a population of adolescents with intellectual disabilities. It was hypothesized that treatment outcomes would be better for MST‐ID compared to standard MST.

Regarding this second hypothesis, no differences were found on the primary outcomes (living at home, police contact and engagement in school or work) at the end of treatment. Six months after treatment, however, the percentage of adolescents living at home was higher in MST‐ID than in standard MST (100% in MST‐ID vs. 77% in standard MST). In addition, the present authors found that MST‐ID obtained better treatment outcomes than standard MST on several of the secondary outcome measures: MST‐ID more frequently resulted in improvements in parenting skills, family relations, social support, involvement with pro‐social peers and lasting behavioural changes than did standard MST. Although MST‐ID did not obtain significantly better results on all outcome variables, the present authors would argue that the differences the present authors did find support the adaptation of MST for adolescents with intellectual disabilities and their parents. Our results suggest that the instrumental outcomes of MST may be underlying to treatment outcome retention up to 6‐month follow‐up. The improved parenting skills, family relations, social support, contact with pro‐social peers and lasting behavioural changes may explain why the percentage of adolescents living at home 6 months post‐treatment is higher in the MST‐ID group than in the standard MST group. Though further research is needed, it seems advisable for standard MST therapists treating families with adolescents with intellectual disabilities to pay increased attention to the instrumental outcomes to ensure the retention of positive change in parenting skills and prevent the out‐of‐home placement of adolescents at follow‐up. The additional training received by MST‐ID therapists, in which specific attention is paid to the identification of parenting stress and an intellectual disability, techniques to motivate families to enter treatment, creating alliance between the family and the therapist, generalization of acquired skills, simplification of treatment content and focussing on one assignment while using visual cues, may explain why MST‐ID leads to better results in some areas.

Maintenance of treatment results is difficult in families with adolescents with intellectual disabilities and has largely been ignored in the intervention literature focusing on youths with intellectual disabilities. Researchers argue that studies should more often assess long‐term outcomes as well as focus on increasing initial family engagement to maximize the chances of maintaining treatment results (Crnic, Neece, McIntyre, Blacher, & Baker, 2017). It has been stated that long‐term home care interventions and the construction of lasting (professional) networks are needed to maintain results in families with a multitude of problems (Tausendfreund et al., 2016). With effects of MST‐ID still present 6 months after treatment, families who received MST‐ID seem to have succeeded in learning to generalize newly acquired skills to different situations, even after having received a relatively short intervention.

Unfortunately, the effects of MST‐ID could not be established in families where both adolescents and parents had intellectual disabilities, because this group was too different from the families receiving standard MST. In fact, almost half of the families treated with MST‐ID were excluded from the analyses because they differed too much from the families treated with standard MST. One of the differences found was that the parents in the MST‐ID group more often had an intellectual disability than the parents in the standard MST group. This baseline difference between families receiving MST‐ID and families receiving standard MST may in part be explained by how families are referred to the interventions. Families known to have intellectual disabilities and related problems usually are referred to organizations specializing in intellectual disability care. Consequently, MST‐ID, provided by an organization specialized in care for people with intellectual disabilities, may have had more referrals of families in which the parent was known to have an intellectual disability than standard MST. Thus, different referral paths may have led to the baseline differences found.

In addition to differences in parental intellectual disabilities, the excluded MST‐ID families differed significantly from the included families on reported behavioural problems, the presence of a father figure and parental educational level. Parents with intellectual disabilities seemed to report less problem behaviour of their children. Though research has suggested that measures such as the CBCL can be answered by parents (of adolescents) with intellectual disabilities (Dekker et al., 2002), instruments developed for use in general populations often employ language that is not easily understood by persons with limited vocabularies or limited information processing. Therefore, the use of instruments such as the SCIL, developed specifically for people with intellectual disabilities, or instruments thoroughly validated for use in this population should be encouraged.

While other evidence‐based systemic treatments such as multidimensional family therapy (Liddle et al., 2018) and family flexible assertive community treatment (Family FACT) have started developing modules for adolescents or families with intellectual disabilities (see e.g., Rijkaart & Neijmeijer, 2011; Youth Interventions Foundation, 2018), to our knowledge no research has been published evaluating their effects in a population of adolescents or parents with intellectual disabilities. Moreover, most interventions that target adolescents with intellectual disabilities and antisocial or delinquent behaviour focus on the individual (without involving or with a much less involvement of the systems surrounding the adolescent) or are aimed at adolescents who are placed out of home. MST‐ID aims to prevent out‐of‐home placement by involving the adolescent and all systems around him or her. Therefore, MST‐ID seems to add to the existing treatments for adolescents with intellectual disabilities and antisocial or delinquent behaviour.

4.1. Limitations

Although our study showed that MST‐ID generated more positive outcomes than standard MST in adolescents with intellectual disabilities and their parents, results only apply to 55% of the research sample. This is due to the fact that 45% of the families treated with MST‐ID were too different from the families treated with standard MST to allow for comparison of their treatment results. Although the exclusion of families with non‐overlapping PS scores restricts the generalizability of the results, overall, removing cases without overlapping PS scores allows for more precisely balancing the treatment arms (Harder et al., 2010).

The PS method was used to control for the non‐random assignment of families to MST‐ID or standard MST as prior studies on and using the PS (Vidal et al., 2017; West et al., 2014) have shown that this method can be used to equate non‐randomized groups through balancing differences in pre‐treatment characteristics, thereby mimicking balance achieved by random assignment on those covariates (West et al., 2014). While selection bias and bias in baseline characteristics can be reduced using the PS (Vidal et al., 2017), a critical issue in PS analysis is the selection of baseline variables or covariates (West et al., 2014). Although a wide range of initial differences between families receiving MST‐ID and standard MST were controlled (i.e., a total of 27 clinically relevant variables were included into our estimation of the PS), there could still be baseline differences that were not measured and, thus, were not controlled. This may have lead to hidden biases in the results. Nevertheless, the use of the PS method is a viable alternative to an RCT and even enhances external validity when treatment selection is thoroughly controlled (Stuart, Cole, Bradshaw, & Leaf, 2011). Careful application of the PS, therefore, can be used to demonstrate that a treatment is effective even without randomization.

Furthermore, it is unknown whether all youths with intellectual disabilities and receiving standard MST were referred to the research team. During the inclusion period of this study, 1,301 families were referred to standard MST. Of these families, 164 (13%) were referred to the research team because of a (suspected) adolescent’ intellectual disabilities. With intellectual disability prevalence estimated at approximately 15% of the Dutch population (Dutch Knowledge Centre for Child & Adolescent Psychiatry, 2017), the percentage of adolescents referred to the research team approximates the percentage in the general Dutch population.

Data management in this study was not in its entirety independent. Researchers were not blind to the treatment conditions, because they carried out home visits and, for safety reasons, received the contact information of the therapist delivering MST(‐ID) to the families. Since the researchers knew which therapists worked for which organizations, it was impossible to achieve masked assessment. Also, researchers carrying out the data collection were involved in data processing and data analyses. Thus, independent data management could not be realized. To reduce the chance of bias, the researchers who handled the data were supervised by two independent researchers, who were neither involved in the development of the assessed programmes nor in data collection.

Lastly, the present study did not take the duration of the treatment into account, because the present authors intended to establish the comparative effect of MST‐ID and standard MST as provided in daily clinical practice. De Wit et al. (2012) advise that intellectual disability adaptations of existing interventions should reserve more time, because persons with intellectual disabilities often have a slow information processing speed and experience difficulty concentrating for a longer period of time. MST generally treats families for 3–5 months. This seems a short duration for families with intellectual disabilities. In MST‐ID, treatment sessions have to be shorter to suit the needs and abilities of family members with intellectual disabilities. Therefore, more sessions may be needed to reach the treatment goals. Indeed, the mean treatment duration of MST‐ID was longer than the duration of standard MST.

5. CONCLUSION

There is a need for evidence‐based interventions that consider the strengths and abilities of families with intellectual disabilities. Interventions should do whatever it takes to realize lasting results in families with intellectual disabilities. Unnecessary care re‐entry and high societal, personal and emotional costs as a result of incarceration should be avoided. To achieve this, interventions for individuals with intellectual disabilities yielding positive post‐treatment outcomes which are maintained over (longer periods of) time are needed.

Multisystemic therapy‐ID has shown to achieve lasting favourable outcomes in families with adolescents with intellectual disabilities who are generally difficult to engage in treatment. More research is needed to establish the effects of MST‐ID when both the adolescent and the parent(s) have intellectual disabilities.

ACKNOWLEDGEMENTS

This study was funded by the Youth Justice Department, Dutch Ministry of Security and Justice. The present authors thank Marina Boonstoppel‐Boender, Puck Coenen and our former interns for their support in the data collection.

Blankestein A, van der Rijken R, Eeren HV, et al. Evaluating the effects of multisystemic therapy for adolescents with intellectual disabilities and antisocial or delinquent behaviour and their parents. J Appl Res Intellect Disabil. 2019;32:575–590. 10.1111/jar.12551

Notes

1

The definition of intellectual disabilities varies across countries. In the Netherlands, intellectual disability generally encompasses intelligence quotient (IQ) scores of 50–70 (mild intellectual disability) and IQ scores of 70–85 (borderline intellectual functioning in the Diagnostic Statistic Manual IV‐TR, American Psychiatric Association, 2000) with co‐occurring deficits in adaptive functioning. Symptoms of intellectual disabilities must have begun during the developmental period (American Psychiatric Association, 2013).

2

Per the MST Services publication, "Multisystemic Therapy ® (MST®) Adaptations: Pilot Studies to Large‐Scale Dissemination," the work presented in this manuscript would be classified as "Model/Adaptation Development Research."

REFERENCES

  1. Achenbach, T. M. , & Rescorla, L. A. (2001). Manual for the ASEBA school‐age forms and profiles. Burlington, NJ: University of Vermont, Research Center for Children, Youth & Families. [Google Scholar]
  2. Ali, M. S. , Groenwold, R. H. H. , Belitser, S. V. , Pestman, W. R. , Hoes, A. W. , Roes, K. C. B. , … Klungel, O. H. (2014). Reporting of covariate selection and balance assessment in propensity score analysis is suboptimal: A systematic review. Journal of Clinical Epidemiology, 16, 112–121. 10.1016/j.jclinepi.2014.08.011. [DOI] [PubMed] [Google Scholar]
  3. Allen, D. G. , Lowe, K. , Moore, K. , & Brophy, S. (2007). Predictors, costs, and characteristics of out of area placement for people with intellectual disability and challenging behaviour. Journal of Intellectual Disability Research, 51, 409–416. 10.1111/j.1365-2788.2006.00877.x. [DOI] [PubMed] [Google Scholar]
  4. American Psychiatric Association . (2000). Diagnostic and statistical manual of mental disorders, 4th ed. Washington, DC: American Psychiatric Publishing. [Google Scholar]
  5. American Psychiatric Association . (2013). Diagnostic and statistical manual of mental disorders, 5th ed. Washington, DC: American Psychiatric Association. [Google Scholar]
  6. Asscher, J. J. , Deković, M. , Manders, W. A. , Van der Laan, P. H. , Prins, P. J. M. , & Dutch MST Cost‐Effectiveness Study Group . (2013). A randomized controlled trial of the effectiveness of multisystemic therapy in the Netherlands: Post‐treatment changes and moderator effects. Journal of Experimental Criminology, 9, 169–187. 10.1007/s11292-012-9165-9. [DOI] [Google Scholar]
  7. Asscher, J. J. , Deković, M. , Manders, W. A. , Van der Laan, P. H. , Prins, P. J. M. , Van Arum, S. , & Dutch MST Cost‐Effectiveness Study Group . (2014). Sustainability of the effects of multisystemic therapy for juvenile delinquents in The Netherlands: Effects on delinquency and recidivism. Journal of Experimental Criminology, 10, 227–243. 10.1007/s11292-013-9198-8. [DOI] [Google Scholar]
  8. Austin, P. C. (2011). An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate Behavioral Research, 46, 399–424. 10.1080/00273171.2011.568786. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Austin, P. C. , & Small, D. S. (2014). The use of bootstrapping when using propensity‐score matching without replacement: A simulation study. Statistics in Medicine, 33, 4306–4319. 10.1002/sim.6276. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Bronfenbrenner, U. (1979). The ecology of human development: Experiments by nature and design. London, UK: Harvard University Press. [Google Scholar]
  11. Brookhart, M. A. , Schneeweiss, S. , Rothman, K. J. , Glynn, R. J. , Avorn, J. , & Stürmer, T. (2006). Variable selection for propensity score models. American Journal of Epidemiology, 163, 1149–1156. 10.1093/aje/kwj149. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Butler, S. , Baruch, G. , Hickey, N. , & Fonagy, P. (2011). A randomized randomised controlled trial of multisystemic therapy and statutory therapeutic intervention for young offenders. Journal of the American Academy of Child & Adolescent Psychiatry, 50, 1220–1235. 10.1016/j.jaac.2011.09.017. [DOI] [PubMed] [Google Scholar]
  13. Cham, H. , & West, S. G. (2016). Propensity score analysis with missing data. Psychological Methods, 21, 427–445. 10.1037/met0000076. [DOI] [PubMed] [Google Scholar]
  14. Crnic, K. A. , Neece, C. L. , McIntyre, L. L. , Blacher, J. , & Baker, B. L. (2017). Intellectual disability and developmental risk: Promoting intervention to improve child and family well‐being. Child Development, 88, 436–445. 10.1111/cdev.12740. [DOI] [PubMed] [Google Scholar]
  15. Cunningham, A. (2002). Lessons learned from a randomized study of multisystemic therapy in Canada. Ontario: Centre for Children and Families in the Justice System. [Google Scholar]
  16. De Ruiter, K. P. , Dekker, M. C. , Douma, J. C. H. , Verhulst, F. C. , & Koot, H. M. (2008). Development of parent‐ and teacher‐reported emotional and behavioural problems in young people with intellectual disabilities: Does level of intellectual disability matter? Journal of Applied Research in Intellectual Disabilities, 21, 70–80. 10.1111/j.1468-3148.2007.00370.x. [DOI] [Google Scholar]
  17. De Ruiter, K. P. , Dekker, M. C. , Verhulst, F. C. , & Koot, H. M. (2007). Developmental course of psychopathology in youths with and without intellectual disabilities. Journal of Child Psychology and Psychiatry, 48, 498–507. 10.1111/j.1469-7610.2006.01712.x. [DOI] [PubMed] [Google Scholar]
  18. De Wit, M. , Moonen, X. , & Douma, J. (2012). Guideline effective interventions for youngsters with MID: Recommendations for the development and adaptation of behavioural change interventions for youngsters with Mild Intellectual Disabilities. Utrecht, The Netherlands: Dutch Knowledge Centre on MID. [Google Scholar]
  19. Dekker, M. C. , Koot, H. M. , Van der Ende, J. , & Verhulst, F. C. (2002). Emotional and behavioral problems in children and adolescents with and without intellectual disability. Journal of Child Psychology and Psychiatry, 43, 1087–1098. 10.1111/1469-7610.00235. [DOI] [PubMed] [Google Scholar]
  20. Dutch Knowledge Centre for Child and Adolescent Psychiatry . (2017). Prevalence of mild intellectual disability. Retrieved from https://www.kenniscentrum-kjp.nl/en/professionals/Mild-intellectual-disability/Introduction-11/Prevalence-7
  21. Emerson, E. , Einfeld, S. , & Stancliffe, R. J. (2011). Predictors of the persistence of conduct difficulties in children with cognitive delay. Journal of Child Psychology and Psychiatry, 52, 1184–1194. 10.1111/j.1469-7610.2011.02413.x. [DOI] [PubMed] [Google Scholar]
  22. Fonagy, P. , Butler, S. , Cottrell, D. , Scott, S. , Pilling, S. , Eisler, I. , … Goodyer, I. M. (2018). Multisystemic therapy versus management as usual in the treatment of adolescent antisocial behaviour (start): A pragmatic, randomised controlled, superiority trial. Lancet Psychiatry, 5, 119–133. 10.1016/S2215-0366(18)30001-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Harder, V. S. , Stuart, E. A. , & Anthony, J. C. (2010). Propensity score techniques and the assessment of measured covariate balance to test causal associations in psychological research. Psychological Methods, 15, 234–249. https://doi.org/10.1037%2Fa0019623. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Henggeler, S. W. , Borduin, C. M. , Schoenwald, S. K. , Huey, S. J. , & Chapman, J. E. . (2006). Multisystemic therapy adherence scale‐revised (TAM‐R) . Unpublished instrument. Charleston: Department of Psychiatry and Behavioral Sciences, Medical University of South Carolina.
  25. Henggeler, S. W. , & Schaeffer, C. M. (2016). Multisystemic therapy: Clinical overview, outcomes, and implementation research. Family Process, 55, 514–528. 10.1111/famp.12232. [DOI] [PubMed] [Google Scholar]
  26. Henggeler, S. W. , Schoenwald, S. K. , Borduin, C. M. , Rowland, M. D. , & Cunningham, P. B. (2009). Multisystemic therapy for antisocial behavior in children and adolescents, 2nd ed. New York, NY: The Guilford Press. [Google Scholar]
  27. Kaal, H. L. , Nijman, H. L. I. , & Moonen, X. M. H. (2015). SCIL Screener voor intelligentie en licht verstandelijke beperking: Voor volwassenen (SCIL 18+) & voor jongeren van veertien tot en met zeventien jaar (SCIL 14–17) [Screener for intelligence and mild intellectual disability: For adults (SCIL 18+) and for adolescents aged 14 to 17 (SCIL14‐17)]. Amsterdam, The Netherlands: Hogrefe. [Google Scholar]
  28. Kaal, H. L. , Overvest, N. , & Boertjes, M. J. (Eds.) (2014). Beperkt in de keten: Mensen met een licht verstandelijke beperking in de strafrechtsketen [People with mild intellectual disability in the justice system]. Amsterdam, The Netherlands: Boom Lemma. [Google Scholar]
  29. Lange, A. M. C. , & Van der Rijken, R. E. A. (2012). Regular MST for youth with a mild intellectual disability: Summary results. Unpublished manuscript.
  30. Lee, B. R. , Ebesutani, C. , Kolivoski, K. M. , Becker, K. D. , Lindsey, M. A. , Brandt, N. E. , … Barth, R. P. (2014). Program and practice elements for placement prevention: A review of interventions and their effectiveness in promoting home‐based care. American Journal of Orthopsychiatry, 84, 244–256. 10.1037/h0099811. [DOI] [PubMed] [Google Scholar]
  31. Letourneau, E. J. , Sheidow, A. J. , & Schoenwald, S. K. (2002). Structure and reliability of the MST therapist adherence scale in a large community sample(Tech. Rep.). Charleston: Family Services Research Center, Medical University of South Carolina.
  32. Liddle, H. A. , Dakof, G. A. , Rowe, C. L. , Henderson, C. , Greenbaum, P. , Wang, W. , & Alberga, L. . (2018). Multidimensional family therapy as a community‐based alternative to residential treatment for adolescents with substance use and co‐occurring mental health disorders. Journal of Substance Abuse Treatment, 90, 47–56. 10.1016/j.jsat.2018.04.011 [DOI] [PubMed] [Google Scholar]
  33. Lightfoot, E. , Hill, K. , & LaLiberte, T. (2011). Prevalence of children with disabilities in the child welfare system and out‐of‐home placement: An examination of administrative records. Children and Youth Services Review, 33, 2069–2075. 10.1016/j.childyouth.2011.02.019. [DOI] [Google Scholar]
  34. Llewellyn, G. , & Hindmarsh, G. (2015). Parents with intellectual disability in a population context. Current Developmental Disorders Reports, 2, 119–126. 10.1007/s40474-015-0042-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Mallett, C. A. (2014). Youthful offending and delinquency: The comorbid impact of maltreatment, mental health problems, and learning disabilities. Child and Adolescent Social Work Journal, 31, 369–392. 10.1007/s10560-013-0323-3. [DOI] [Google Scholar]
  36. Manders, W. A. , Deković, M. , Asscher, J. J. , Van der Laan, P. H. , & Prins, P. J. M. (2011). De implementatie van multisysteem therapie in Nederland: De invloed van behandelintegriteit en nonspecifieke factoren op behandeluitkomsten [The implementation of multisystemic therapy in the Netherlands: The influence of treatment integrity and nonspecific factors on treatment outcomes]. Gedragstherapie, 44, 327–340. [Google Scholar]
  37. McIntosh, C. (2015). Final Evaluation Summary of the Multisystemic Therapy Program (Research Report No. 2015–R015). Retrieved from Public Safety Canada website: https://www.publicsafety.gc.ca/lbrr/archives/cn92799255-eng.pdf.
  38. McReynolds, L. S. , Schwalbe, C. S. , & Wasserman, G. A. (2010). The contribution of psychiatric disorder to juvenile recidivism. Criminal Justice and Behavior, 37, 204–216. 10.1177/0093854809354961. [DOI] [Google Scholar]
  39. Meppelder, M. , Hodes, M. , Kef, S. , & Schuengel, C. (2015). Parenting stress and child behaviour problems among parents with intellectual disabilities: The buffering role of resources. Journal of Intellectual Disability Research, 59, 664–677. 10.1111/jir.12170. [DOI] [PubMed] [Google Scholar]
  40. MST‐NL . (2012). Sociaal demografische informatie (SDI) 2.0 [Social demographic information (SDI) 2.0] . Nijmegen, The Netherlands: Praktikon. [Google Scholar]
  41. Neece, C. L. , & Lima, E. J. (2016). Interventions for parents of people with intellectual disabilities. Current Developmental Disorders Reports, 3, 124–128. 10.1007/s40474-016-0088-4. [DOI] [Google Scholar]
  42. Nijman, H. , Kaal, H. , Van Scheppingen, L. , & Moonen, X. (2016). Development and testing of a screener for intelligence and learning disabilities (SCIL). Journal of Applied Research in Intellectual Disabilities, 29, 575–9. 10.1111/jar.12310. [DOI] [PubMed] [Google Scholar]
  43. Ogden, T. , & Hagen, K. A. (2006). Multisystemic treatment of serious behaviour problems in youth: Sustainability of effectiveness two years after intake. Child and Adolescent Mental Health, 11, 142–149. 10.1111/j.1475-3588.2006.00396.x. [DOI] [PubMed] [Google Scholar]
  44. Patton, K. A. , Ware, R. , McPherson, L. , Emerson, E. , & Lennox, N. (2016). Parent‐related stress of male and female carers of adolescents with intellectual disabilities and carers of children within the general population: A cross‐sectional comparison. Journal of Applied Research in Intellectual Disabilities, 31, 51–61. 10.1111/jar.12292. [DOI] [PubMed] [Google Scholar]
  45. Powell, R. M. , & Parish, S. L. (2017). Behavioural and cognitive outcomes in young children of mothers with intellectual impairments. Journal of Intellectual Disability Research, 61, 50–61. 10.1111/jir.12308. [DOI] [PubMed] [Google Scholar]
  46. Rijkaart, A. , & Neijmeijer, L. (2011). Modelbeschrijving ACT LVB met complexe problematiek [Model description ACT‐ID with complex problems]. Utrecht: Netherlands Institute of Mental Health and Addiction. [Google Scholar]
  47. Rubin, D. B. (2001). Using propensity scores to help design observational studies: Application to the tobacco litigation. Health Services & Outcomes Research Methodology, 2, 169–188. 10.1023/A:1020363010465. [DOI] [Google Scholar]
  48. Schoenwald, S. (2014). Multisystemic therapy In Ehrenreich‐May J., & Chu B. C. (Eds.), Transdiagnostic treatments for children and adolescents: Principles and practice (pp. 313–338). New York: The Guilford Press. [Google Scholar]
  49. Schuiringa, H. , Van Nieuwenhuijzen, M. , Orobio de Castro, B. , Lochman, J. E. , & Matthys, W. (2017). Effectiveness of an intervention for children with externalizing behavior and mild to borderline intellectual disabilities: A randomized trial. Cognitive Therapy and Research, 41, 237–251. 10.1007/s10608-016-9815-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Schuiringa, H. , Van Nieuwenhuijzen, M. , Orobio de Castro, B. , & Matthys, W. (2015). Parenting and the parent‐child relationship in families of children with mild to borderline intellectual disabilities and externalizing behavior. Research in Developmental Disabilities, 36, 575–12. 10.1016/j.ridd.2014.08.018. [DOI] [PubMed] [Google Scholar]
  51. Soenen, S. , Van Berckelaer‐Onnes, I. , & Scholte, E. (2016). A comparison of support for two groups of young adults with mild intellectual disability. British Journal of Learning Disabilities, 44, 146–158. 10.1111/bld.12127. [DOI] [Google Scholar]
  52. Stuart, E. A. (2010). Matching methods for causal inference: A review and a look forward. Statistical Science, 25, 575–21. 10.1214/09-sts313. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Stuart, E. A. , Cole, S. R. , Bradshaw, C. P. , & Leaf, P. J. (2011). The use of propensity scores to assess the generalizability of results from randomized trials. Journal of the Royal Statistical Society: Series A (Statistics in Society), 174, 369–386. 10.1111/j.1467-985x.2010.00673.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Tausendfreund, T. , Knot‐Dickscheidt, J. , Schulze, G. C. , Knorth, E. J. , & Grietens, H. (2016). Families in multi‐problem situations: Backgrounds, characteristics, and care services. Child & Youth Services, 37, 4–22. 10.1080/0145935X.2015.1052133. [DOI] [Google Scholar]
  55. Thompson, K. C. , & Morris, R. J. (2016). Juvenile delinquency and disability. Geneva: Springer International Publishing. [Google Scholar]
  56. Van der Stouwe, T. , Asscher, J. J. , Stams, G. J. J. M. , Deković, M. , & Van der Laan, P. H. (2014). The effectiveness of multisystemic therapy (MST): A meta‐analysis. Clinical Psychology Review, 34, 468–481. 10.1016/j.cpr.2014.06.006. [DOI] [PubMed] [Google Scholar]
  57. Van Duijvenbode, N. , Didden, R. , Van den Hazel, T. , & Engels, R. C. M. E. (2016). Psychometric qualities of a tetrad WAIS‐III short form for use in individuals with mild to borderline intellectual disability. Developmental Neurorehabilitation, 19, 26–30. 10.3109/17518423.2014.893265. [DOI] [PubMed] [Google Scholar]
  58. Vermeulen, K. M. , Jansen, D. M. C. , Knorth, E. J. , Buskens, E. , & Reijneveld, S. A. (2017). Cost‐effectiveness of multisystemic therapy versus usual treatment for young people with antisocial problems. Criminal Behaviour and Mental Health, 27, 89–102. 10.1002/cbm.1988. [DOI] [PubMed] [Google Scholar]
  59. Vermulst, A. , Kroes, G. , De Meyer, R. , Nguyen, L. , & Veerman, J. W. (2012). Opvoedingsbelastingvragenlijst (OBVL). Handleiding [Burden of Parenting Questionnaire]. Nijmegen, The Netherlands: Praktikon. [Google Scholar]
  60. Vidal, S. , Steeger, C. M. , Caron, C. , Lasher, L. , & Connell, C. M. (2017). Placement and delinquency outcomes among system‐involved youth referred to multisystemic therapy: A propensity score matching analysis. Administration and Policy in Mental Health and Mental Health Services Research, 44, 853–866. 10.1007/s10488-017-0797-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  61. Wallander, J. L. , Dekker, M. C. , & Koot, H. M. (2003). Psychopathology in children and adolescents with intellectual disability: Measurement, prevalence, course, and risk. International Review of Research in Mental Retardation, 26, 93–134. 10.1016/S0074-7750(03)01003-6. [DOI] [Google Scholar]
  62. Wechsler, D. (2000). WAIS‐III. Nederlandstalige bewerking: Afname en scoringshandleiding [WAIS‐III. Dutch edition: Administration and scoring manual]. Lisse: The Psychological Corporation. [Google Scholar]
  63. Wechsler, D. (2005). Wechsler Intelligence Scale for Children (WISC‐III‐NL): Handleiding en verantwoording, Derde Editie NL [Wechsler Intelligence Scale for Children (WISC‐III‐NL): Instruction manual]. London: Harcourt Assessment. [Google Scholar]
  64. West, S. G. , Cham, H. , Thoemmes, F. J. , Renneberg, B. , Schulze, J. , & Weiler, M. (2014). Propensity scores as a basis for equating groups: Basic principles and application in clinical treatment outcome research. Journal of Consulting & Clinical Psychology, 82, 906–919. 10.1037/a0036387. [DOI] [PubMed] [Google Scholar]
  65. Youth Interventions Foundation (2018). MDFT voor jongeren met een licht verstandelijke beperking [MDFT for adolescents with an intellectual disability]. Retrieved from https://www.stichtingjeugdinterventies.nl/opleidingen/jeugdzorg/mdft-jongeren-lichtverstandelijke-beperking. Accessed on 02–05‐2018. [Google Scholar]

Articles from Journal of Applied Research in Intellectual Disabilities are provided here courtesy of Wiley

RESOURCES