Skip to main content
PMC home page
UKPMC Funders Author Manuscripts logoLink to UKPMC Funders Author Manuscripts
. Author manuscript; available in PMC: 2019 Nov 1.
Published in final edited form as: Psychother Res. 2018 Apr 22;29(8):1010–1019. doi: 10.1080/10503307.2018.1464681

A systematic review exploring therapist competence, adherence and therapy outcomes in individual CBT for children and young people

Hannah A Rapley 1,2, Maria E Loades 1,3
PMCID: PMC6763333  EMSID: EMS84377  PMID: 29683046

Abstract

Objective

Whilst the evidence base for Cognitive Behavioural Therapy (CBT) with children and young people is growing, the mechanisms through which these beneficial effects occur is still unclear. This systematic review seeks to appraise the relationship between therapeutic outcomes in CBT and therapist adherence and competence, within the child and adolescent literature.

Method

A systematic review was carried out, with five studies identified as meeting the inclusion criteria.

Results

The literature is currently small and inconclusive. Amongst the studies reviewed there were inconsistent findings, with minimal-to-no effect sizes found between adherence, competence, and outcomes.

Conclusion

The current paucity of research in this area means that conclusions are currently limited. The role and impact of adherence and competence on therapeutic outcomes remains unclear within individual CBT in a child population. This is comparable with the current adult literature, where findings also remain inconclusive. Further research avenues are discussed.

Keywords: Cognitive Behaviour Therapy, CBT, adherence, competence, children, young people

Introduction

Although increasing evidence continues to emerge in support of the efficacy of Cognitive Behavioural Therapy (CBT), the mechanisms through which it exerts its beneficial effects are generally not well understood (Kazdin, Whitley, & Marciano, 2006; Webb, DeRubeis, & Barber, 2010). Effectively assessing and understanding these underlying mechanisms is crucial to the success of expanding CBT effectiveness, training, and service provision, in order to disseminate further evidence-based practice into routine care.

Researchers have long hypothesized about a number of different ‘active ingredients’ that may be responsible for the therapeutic improvements observed in CBT. Orlinsky and Howard defined process research as “everything that can be observed to occur between and within the patient and therapist during their work together” (1986, p. 311-312). Research investigating these process variables aims to understand which elements and processes in an intervention contribute to positive treatment outcomes. By understanding these treatment factors, researchers and clinicians can modify interventions, thus providing optimum doses of active ingredients and minimizing inert elements, with the ultimate goal of improving treatment efficacy (Kazdin et al., 2006). Without rigorous assessment, and understanding of adherence to treatment protocols and the competency with which they are delivered, the internal validity of interventions can be compromised. This therefore limits clinicians’ and researchers’ ability to attribute client outcomes to the treatment they received. Unfortunately, adequate assessment of treatment integrity is not the norm in outcome trials, particularly in youth psychotherapy. Perepletchikova, Treat and Kazdin (2007) found that only 3.5% of studies of youth and adult psychotherapy reviewed, met their criteria for adequate implementation and reporting of treatment integrity procedures.

Therapist adherence refers to the extent to which a therapist delivers the intervention and techniques as prescribed by the treatment manual or model (Perepletchikova & Kazdin, 2005; Waltz, Addis, Koerner, & Jacobson, 1993). Measures of therapist adherence usually assess how frequently or how thoroughly therapists employ particular techniques. It constitutes a major component of treatment integrity, and is essential to establish experimental validity. Whilst the theoretical importance of therapist adherence in psychotherapy has been discussed at length, the level of impact on treatment outcome remains unclear (Emmelkamp et al., 2014).

Therapist competence is defined as the extent to which a therapist implements an intervention skilfully and appropriately for the patient in question (Barber et al., 2006). Although therapist adherence and competence are related constructs, they are conceptually distinct. Put simply; therapist adherence refers to ‘how much’ of the therapy the therapist employs, and therapist competence refers to ‘how well’ the interventions are executed (Shaw & Dobson, 1988). Muse and McManus (2013) identified that in practice there can be much overlap between the two constructs. Moderate to high correlations (ranging from r=0.5 to r=0.85) have been found between therapist adherence and competence (e.g. Barber, Krakauer, Calvo, Badgio, & Faude, 1997; Barber, Liese, & Abrams, 2003; Carroll et al., 2000), which indicates the difficulty of disentangling these two constructs. Some authors have stated that in order for therapists to be competent in a therapeutic modality, they must be adherent to the intervention protocol; such that they argue therapist adherence is a prerequisite for the competent delivery of an intervention, whilst adherence provides no guarantee of competence (Waltz et al., 1993), thus meaning that adherence may be necessary but not sufficient for attaining competence.

Research examining therapist adherence and/or competence has almost exclusively employed observational rather than experimental methods. Hogue, Liddle and Rowe (1996) identified three general methods for assessing adherence in therapy: Quality control measures can be employed prior to conducting therapy, e.g. through training and regular supervision. Secondly, notes can be reviewed for treatment elements, or through therapist self-report. Thirdly, and indicated as the best means of capturing therapist adherence, is observational review of therapists’ in-session behaviour. Most commonly, trained raters code one-or-more video or audio-taped therapy sessions using measures of adherence. Similarly, there are several methods for evaluating competency, including observer, supervisor and therapist ratings.

The most commonly used measure that encompasses both adherence and competence in CBT is the Cognitive Therapy Scale and it’s revised version, the Cognitive Therapy Scale – Revised (CTS-R (Blackburn et al., 2001)). The CTS-R is a 12-item scale, and the rating for each item incorporates a Dreyfus scale of competence, ranging incompetent to expert. Each item is rated on a Likert scale ranging from 0 to 6, where 0 indicates that the skill was not demonstrated at all (incompetent), and 6 indicates that the skills was proficiently demonstrated, even when faced with challenges to its demonstration (expert). Five items relate to general therapeutic skills (agenda-setting, feedback, collaboration, pacing/use of time, and interpersonal effectiveness) and 7 items relate to CBT specific skills (facilitation of emotional expression, identifying key cognitions, application of behavioural techniques, guided discovery, conceptualisation, application of change methods, and homework-setting). The CTS-R was designed for use when doing CBT with adults, and therefore, may not be sensitive to the developmental adaptations required to deliver CBT competently to children and young people. An alternative measure of competence is the Cognitive Behaviour Therapy Scale – Children and Young People (CBTS-CYP, Stallard, Myles & Branson, 2014).

A number of factors need to be considered when rating adherence and competence. Waltz and colleagues (1993), recommend that the stage of therapy, session number, and progress-made-thus-far should be considered when rating therapist competence. These methods are labour intensive and thus expensive (Horvath, Del Re, Flückiger, & Symonds, 2011), which may in part explain the lack of regular monitoring of these variables in outcome research. However, it is difficult to draw conclusions about the efficacy of CBT protocols unless it can be ensured that protocols are adequately and competently delivered (Waltz et al., 1993; Weck, Grikscheit, Jakob, Höfling, & Stangier, 2015). Thus, accurate and rigorous measurement of these variables is imperative. Without this, successful dissemination of treatment provision and therapist training is not viable (Muse & McManus, 2013).

Despite the difficulties of conducting research into process variables, a body of research exists examining process variables for CBT within an adult population. A study of 1,247 adult patients and 43 therapists within primary care found little support for an association between competence in CBT and client outcome (Branson, Shafran, & Myles, 2015). However, significantly more patients of the most competent therapists demonstrated reliable improvements in their symptoms of anxiety than would be expected by chance alone. Within a study of adults with anxiety and panic disorder, therapist competence and adherence in the early stages of therapy was associated with better outcomes amongst those with panic disorder (Haug et al., 2016). Amongst those with social anxiety, lower therapist competence and adherence was associated with higher drop-outs from therapy. The authors conclude that therapist competence, adherence and alliance may have independent contributions to the outcome of CBT for anxiety disorders, but in different phases of treatment.

A comprehensive meta-analysis of 36 studies in the adult literature found huge variability in the adherence-outcome and competence-outcome relationships, with aggregate estimates of effect sizes being very close to zero (Webb et al., 2010). Neither the adherence-outcome (r=0.02) nor the competence-outcome (r=0.07) effect size estimates were found to be significantly different from zero, thus suggesting that neither therapist adherence nor competence were significant predictors of treatment outcomes. Additional analyses showed that when only interventions for depression were considered, a significant correlation between competence and outcome emerged. No significant effect was found for therapist adherence. Another meta-analysis (Zarafonitis-Muller, Kuhr, & Bechdolf, 2014) found a small but significant effect of therapist competence on therapeutic improvement (r=0.24) when looking across a range of disorders, and a moderate effect (r=0.38) when depression interventions were considered on their own. No significant results were found for the influence of therapist adherence to protocol on treatment outcome. Thus, these meta-analyses find minimal support for a role of therapist adherence or competence on therapy outcomes. However, findings from the second study suggest that a competent delivery of cognitive-behavioural techniques may contribute to therapeutic improvements, whereas solely manual guided adherent implementation of CBT does not appear to have a significant impact on therapeutic outcomes.

As indicated above, there is significant inconsistency in the literature. It has been hypothesised that this may be due to study limitations and measurement difficulties associated with process variables (Feeley, DuRubeis, & Gelfand, 1999; Webb, Auerbach, & DeRubeis, 2012). These have included; non-optimal research designs, use of audio not video recordings, use of undergraduate not professional raters, and small sample sizes.

CBT process research in children and young people is significantly smaller and has lagged behind the adult literature. Only a handful of studies have examined therapist adherence-outcome or competence-outcome relationships in CBT for young people. A review by Webb and colleagues (2012) summarized the evidence for outcome and process variables in CBT for adolescent depression. They concluded that whilst a growing body of research supports the efficacy of CBT for adolescent depression, the mechanisms through which it is beneficial remain unclear. It is important that process research within CBT focuses independently on children and young people, due to likely differences in developmental factors such as cognitive functioning, social development and emotional skills (Kingery et al., 2006), as well as the frequent inclusion of parents or carers in the therapeutic process. Assumptions cannot therefore be made that results from adult process research is directly transferable to a child and adolescent population.

The present review

The evidence for the role of therapist adherence and competence on therapy outcomes is mixed in adults (Webb et al., 2010), and much less attention has been paid to these mechanisms within a child population (under 18 years old). This systematic review seeks to identify what is understood about these mechanisms in CBT with children. Does the literature suggest a similar picture to that currently seen with adults? Or do these mechanisms work differently in this population? Is there currently enough information for conclusions to be drawn?

This systematic review aims to summarise and scrutinize the literature available on therapist adherence and competence process research in relation to therapy outcomes in individual CBT for children and young people, in order to synthesise the state of the evidence base, and illuminate areas for further research. Put simply, this review aims to answer the question; is therapist adherence and therapist competence linked to therapy outcomes in CBT for children and young people?

Method

Search strategy

The procedures were informed by accepted systematic reviewing guidelines (Khan, Ter Riet, Glanville, Sowden, & Kleijnen, 2001; Moher, Liberati, Tetzlaff, & Altman, 2009). The following databases were searched from the earliest available listing to 28th July 2016: PsycINFO, Embase and PubMed. The first 50 pages of Google Scholar were screened for additional articles. Key-word search terms included; 1) cognitive-behavioural therapy 2) children and young people 3) therapist adherence, and 4) therapist competence. Full search terms can be found in Table 1.

Table 1. Search terms.

CBT OR “Cognitive Behavio?r* Therapy”
AND Child* OR “young people” OR “young person” OR adolesc* OR p?ediatric
AND Therapist AND competen* OR adherence
Open in a new tab

Eligibility criteria and study selection

Studies were included if participants were under the age of 18, the intervention consisted of individual CBT, and contained psychometrically validated measures of therapist adherence and/or competence, and a measure of therapeutic outcome. Studies had to explicitly examine the link between therapist adherence or competence and therapy outcomes to be included. They were excluded if participants were over the age of 18, the intervention was less than three sessions of CBT and if the intervention was group-based. This was to ensure some standardisation of intervention. The searches were restricted to English-language publications. Full inclusion and exclusion criteria can be found in Table 2.

Table 2. Inclusion and exclusion criteria.

Inclusion criteria
Studies involving: 		Exclusion criteria
Studies involving:
Participants under the age of 18 Adult participants
CBT intervention (minimum three session) Alternative models of intervention.
CBT interventions with less than three sessions.
Purpose was to treat a mental health disorder Purpose was not to treat a mental health disorder.
Individual CBT Group CBT
Measure of symptom change on a psychometrically validated scale or standardised interview, pre- and post-intervention.
A psychometrically validated measure of therapist adherence and/or therapist competence.
Explicit examination of the link between therapist adherence and therapy outcome and/or therapist competence and therapy outcome
Articles published in English. Articles not published in English.
Open in a new tab

The initial search returned 237 articles. Once duplicates were removed 212 articles remained. Abstracts and titles were screened for relevance by the lead author. From this, 47 articles were taken through to the full-text screening phase. Here, full texts were screened by the lead author. A random sample of 20% of these articles were additionally screened by a 2nd reviewer. This inter-rater reliability returned 100% agreement for inclusion/exclusion. Any discrepancies during this process were resolved through discussion with supervisor. Following this, five articles were found to meet the inclusion criteria and were included in this systematic review. See Figure 1 for PRISMA diagram of search strategy (Moher, Liberati, Tetzlaff, Altman, & The PRISMA Group, 2009).

Figure 1.

Figure 1

Open in a new tab

PRISMA diagram of search strategy (Moher, Liberati, Tetzlaff, Altman, et al., 2009).

Data extraction and analysis plan

Data regarding the sample characteristics, composition of the interventions, measures of therapist adherence and competence, outcome measures, and indicators of efficacy, such as statistically and non-statistically significant outcomes were extracted from the five articles.

Results

Description of the included studies

The final review included five studies. Of these five studies, the data from four studies was from RCTs, with the other being a controlled observational design. Participants ranged from 5-17 years old, with three studies containing adolescents, and the remaining involving younger children. All studies containing a mix of genders. Ethnicity was reported in all studies, with Caucasian and African American participants in the majority. A range of socio-economic statuses were indicated. Four studies were from the USA, and one from Holland (Liber et al., 2010). Three of the interventions were designed to treat anxiety disorders (Ginsburg, Becker, Drazdowski, & Tein, 2012; Liber et al., 2010; Podell, 2011), and two substance abuse (Chinchilla, 2007; Hogue et al., 2008). Some of the interventions used named manualised treatment protocols, e.g. Coping Cat (Podell, 2011), and the FRIENDS programme (Liber et al., 2010). Table 3 summaries characteristics of the included studies.

Table 3. Study characteristics.

Author (Year) Design Number of participants for CBT Mean age, (% male) Primary target of intervention Intervention No. of sessions
Chinchilla, (2007) RCT 64 15, range 13-17, (80%) Substance abuse CBT Mean = 12.12
Ginsburg et al. (2012) RCT 17 11.12, (30%) Anxiety disorders CBT Mean = 7.29
Hogue et al. (2008) Controlled 62 15.5, (81%) Substance abuse CBT Mean = 12.3
Liber et al. (2010) RCT 24 10.72, range 8-12, (58%) Anxiety disorders FRIENDS programme 14 (10 with child, 4 with parent)
Podell, (2011) RCT 139 10.74, range 7-17, (48.3%) Anxiety disorders Coping Cat 14
Open in a new tab

Qualities of studies

Study quality was assessed using the Critical Appraisal Skills Programme guidelines (Critical Appraisal Skills Programme, 2017), which were chosen to provide a descriptive framework to assess study quality and rigour. Study quality varied. Three of the studies were published in peer-reviewed journals, however two were from unpublished doctoral theses (Chinchilla, 2007; Podell, 2011), and therefore had not been subject to peer scrutiny.

Varying attention and detail was paid to the measurement of therapist adherence and competence. Four of the studies measured both therapist adherence and competence (Chinchilla, 2007; Ginsburg et al., 2012; Hogue et al., 2008; Podell, 2011), and one measured just adherence (Liber et al., 2010). This variability was further compounded as all studies used different measures of therapist adherence and competence, except two studies where different versions of the same measure were used (Chinchilla, 2007; Hogue et al., 2008). Therapist adherence measures rated the presence or absence of CBT treatment components to assess if the intervention was delivered as prescribed, these were rated using checklists or Likert scales. Competence measures included assessments of therapist characteristics, qualities and how well clinical skills were employed. Observer-rated Likert measures were used. However, whilst measurement approaches were similar across studies, there was little consistency in the outcome measures used.

Psychometric properties of therapist adherence and competence measures were reported sporadically and often insufficiently. Often the measures had been created for the study at hand, or previous associated studies (Chinchilla, 2007;; Ginsburg et al., 2012). The most robust reporting of psychometric properties came from Hogue et al., (2008), where intra-class correlation coefficients (ICCs) are discussed, with the measure showing ‘good-to-excellent’ interrater reliability for therapist adherence, and ‘fair-to-poor’ for competency, based on Cicchetti’s (1994) criteria. Otherwise reporting of psychometric properties was insufficient.

In all studies ratings of adherence and/or competence were completed by observational raters. In most studies, multiple ratings of therapist adherence and competence were completed. Two studies, (Chinchilla, 2007; Hogue et al., 2008) aimed to review five sessions of an intervention (two from the beginning, and three sessions from later in therapy). Two other studies collected adherence and/or competence measures on two occasions, in the earlier and then later stages of therapy (Ginsburg et al., 2012; Liber et al., 2010). The remaining study used less rigorous collection of measures, details of which can be found in Table 4.

Table 4. Study outcomes.

Author (Year) Outcome measures Measure of adherence Measure of competence Coding No. of sessions coded Conclusions
Chinchilla, (2007) R-CBCL*
YSR		 Contextual Assessment of Therapist Competence (CATC-CBT) Observer rated 5 sessions No significant relationships.
Ginsburg et al. (2012) ADIS-IV
CGI-I,S		 Treatment Adherence and Therapist Competence (TATC) Observer rated 2 sessions Greater session structure and competence implementing this component was associated with better treatment outcomes.
Hogue et al. (2008) PEI
CBCL
YSR		 Therapist Behaviour Rating Scale – Competence Observer rated 5 sessions Main effect for adherence on marijuana use (d=.44) and on parent reported externalizing (d=.37). Curvilinear main effect of adherence on parent-reported internalizing symptoms (d=.40).
Liber et al. (2010) ADIS-IV
CBCL
MASC		 The Australian Treatment Adherence Protocol for the FRIENDS programme None Observer rated 2 sessions No sig relationship between adherence and child or parent reported outcomes.
Podell, (2011) CGI – I,S
CBCL
MASC
CGAS		 CBT Checklist Supervisor Rating Form Supervisor rated 1 measure of competence. Adherence unclear. Therapist treatment integrity was a significant predictor of outcome on parent-rated measures. Therapist competence was sig predictor of child-reported outcomes.
Open in a new tab

Note: R-CBCL = Revised Behaviour Checklist; YSR = Youth Self Report; ADIS-IV = Anxiety Disorders Interview Schedule for Children/Parents; CGI-I,S = Clinical Global Impression – Improvement subscale, Severity subscale; PEI = Personal Experience Inventory; MASC = Multidimensional Anxiety Scale for Children; CGAS = Global Assessment Scale for Children.

All studies, expect one (Ginsburg et al., 2012), detailed the training of both therapists and independent raters. Quality control measures; employed to ensure that both therapists and independent raters are trained to a significant level of reliability, are discussed.

In studies where randomisation was present, details of these processes were documented, and raters were blinded to conditions.

Outcomes of studies

The overall picture of findings from the remaining five studies is limited and inconclusive. A well-designed and thorough study by Hogue et al., (2008) found a small-to-medium effect (d=0.44) of therapist adherence on reduction of marijuana usage,. Hogue and colleagues also analysed curvilinear effects within their data. They found a curvilinear small-to-medium effect (d=0.40) of therapist adherence on parent-reported internalizing symptoms, thus suggesting that moderate levels of therapist adherence predicted the lowest internalizing scores, whereas low and high levels of therapist adherence predicted relatively worse internalizing scores. However, this analysis was conducted across both the CBT and multidimensional family therapy MDFT conditions, so the effects cannot be attributed to CBT alone.

Podell (2011) found that therapist adherence was a significant predictor of outcome on parent-rated outcome measures only, whilst therapist competence was shown to be a significant predictor of child-reported outcomes. However, limitations in terms of measurement rigour should be considered: Therapist competence ratings were rated once per patient by a supervisor at the conclusion of the intervention, and the frequency of measurement of therapist adherence is unclear.

Ginsburg and colleagues (2012) reported that greater session structure and greater competence implementing these components was associated with better treatment outcomes. However, no effect sizes are provided and therefore limited conclusions can be drawn.

Two studies found no significant relationships between therapist adherence and/or competence and therapeutic outcomes (Chinchilla, 2007; Liber et al., 2010).

Discussion

This systematic review analysed findings from five studies in which therapist adherence and/or competence was examined in relation to therapy outcomes, amongst clinical samples of children receiving individual CBT. Support for a role for therapist adherence or competence on therapeutic outcomes was limited and inconsistent. Study quality and the lack of literature mean that currently limited conclusions can be drawn.

The included studies present a mixed and inconclusive picture of the relationship between therapist adherence and outcome, and therapist competence and outcome in CBT for children and young people. Whilst two studies identified no significant relationships between therapist adherence, competence and outcome, some significant effects were found across the others. Inconsistent effects were found on some measures, but not others, e.g. parent versus child measures, and internalising versus externalising symptoms (Hogue et al., 2008; Podell, 2011). It is unclear whether this variability is detecting true differences in effects, or instead reflects limitations of measurement. Quality of the included studies was markedly varied. Studies varied in terms of measures of therapist competence and adherence used, and the robustness and frequency with which they were used. Additionally, the included studies were focused on two different clinical disorders, thus often studies were seeking to adhere to a different intervention programme or manual. This is a significant confound, and is likely to complicate identifying true effects. Thus, conclusions drawn must be tentative, as effects may vary across disorders and protocols. Future studies should seek to control for this variability.

Overall the findings from this small collection of studies are mixed and inconclusive. Due to the small number of studies, the varied outcomes and effect sizes, and variability in methodology, it is not possible to draw valid and reliable conclusions about the relationship between adherence and competence and outcomes, in youth CBT.

Whilst the adult literature is more developed with a greater number of studies, similar inconsistent effects have been found. A substantial meta-analysis by Webb and colleagues found large variability between therapist adherence-outcome and competence-outcome effects, with aggregate effect sizes not being significantly different from zero (Webb et al., 2010). Thus, suggesting an inconclusive picture across both adult and child literature. It has yet to be explored if the effects of therapist adherence and competence on therapeutic outcomes vary with the developmental trajectory of childhood.

Interpretations about the lack of a significant relationship between outcome and therapist adherence and competence, both within this review, and within the adult literature, are limited due to substantial methodological variability in the assessment of adherence and competence (Huppert, Barlow, Gorman, Shear, & Woods, 2006). Firstly, measures of therapist adherence and competence are often newly created for the purposes of a specific trial, with limited reliability and validity (Perepletchikova & Kazdin, 2005; Webb et al., 2010). Adherence and competence are frequently conceptualised as stable characteristics within a therapist and within a treatment. This is reflected by these variables commonly being assessed at a single time-point in the majority of studies (Horvath et al 2001; Webb et al 201). However, therapist adherence and competence likely vary between sessions for the same patient, as well as across different patients of the same therapist (Boswell et al., 2013). It is therefore crucial that multiple sessions, across the span of the intervention should be coded in order to obtain reliable ratings of adherence and competence (Webb et al., 2010). In this review attempts were made in all but one study to obtain at least two or more ratings of therapist adherence and competence. However, often ratings are only based on assessments of single sessions (Webb et al., 2010).

Secondly, another methodological limitation may be due to the therapists selected to take part in research studies. In RCTs therapists are usually selected, trained and monitored to high levels of competence and adherence. It is therefore likely that this may restrict the range of scores, thus creating a ceiling effect, which may make it difficult to determine the true relationship between therapist competence, adherence and outcome. Only one study reviewed (Chinchilla, 2007) reported that therapist competence and adherence scores were not skewed by a ceiling effect.

Thirdly, when considering adherence to protocols, the importance and utility of treatment manuals can be controversial, particularly in respect to how rigidly, or adherently, they should be implemented. There is little consensus regarding what constitutes a sufficient versus suboptimal level of adherence necessary to influence or even promote desired outcomes (Haug et al., 2016). Some studies suggest that high levels of therapist adherence indicate therapist rigidity, which may undermine the development of an effective therapeutic relationship (Castonguay, Goldfried, Wiser, Raue, & Hayes, 1996), and prevent the competent delivery of an intervention. More recently some researchers have investigated whether a curvilinear relationship between therapist adherence and outcome exists; in that low and very high therapist adherence may predict worse treatment outcomes than a moderately adherent therapist. Barber et al., (2006) found a curvilinear adherence-outcome effect in adults, where moderate therapist adherence predicted greatest improvement in drug use and depressive symptoms, when compared to high and low adherence. A similar curvilinear effect of therapist adherence was found on one outcome measure in the included study by Hogue and colleagues (2008). Thus moderate therapist adherence may represent a balance between treatment protocol and clinical flexibility, which therefore may be related to the concept of therapist competence (Stratton, 2011). The concept of therapist responsiveness has been offered as a possible explanation of the variations in findings in process research (Stiles, 2009). This refers to the idea that therapists adapt their behaviour to the unfolding context of treatment, including patient behaviours and characteristics. It infers that therapists, on the whole, do not deliver predetermined levels of an intervention, but instead are responsive to the emerging context in therapy (Webb et al., 2012).

Therapist process variables, such as therapist adherence and competence, do not operate in a vacuum. It is likely that the relationship of process variables to outcomes may be moderated by a range of other therapist and patient variables. Webb et al. (2010) indicated that therapeutic alliance could be a moderating variable, which should therefore be considered in future studies. Therapeutic alliance is the quality and nature of the bond and relationship between the therapist and the client. The adult literature has found it to be a relatively consistent predictor of outcome across a variety of treatment modalities (Martin, Garske, & Davis, 2000). However, whilst a minority of studies have investigated such process variables together, the variables have mostly been examined in isolation from one another. Thus the relationship remains unclear. In one of the few studies to investigate alliance alongside other process variables, Weck et al., (2015) found a moderating effect of adherence with alliance on outcome, thus indicating that the better the therapeutic alliance, the stronger the effect of therapist adherence on treatment outcome. They also found that alliance mediated the relationship between therapist competence and outcome. No studies have yet looked at this within a youth population.

This review provides a systematic appraisal of the literature focused on the relationship between therapy outcomes and therapist adherence and/or competence. Its strengths lie in the rigorous and transparent procedures followed, but the utility of the review is limited by the current small evidence base identified. However, this therefore highlights the paucity of research in this area and hopefully can encourage future areas of development.

Future process-outcome studies within youth CBT should consider the possible curvilinear effect of therapist adherence and ensure this is investigated in future studies. If further marginal or inconsistent effects are found, it will be important to acknowledge and measure multiple sources of variability which may affect process variables and their relationships to change (Perepletchikova & Kazdin, 2005). It is likely that therapist adherence and competence are complex constructs that are influenced by a range of interacting variables (Boswell et al., 2013). Future research should focus on identifying factors that both facilitate and hinder treatment integrity, particularly therapeutic alliance and responsiveness.

Conclusions

With an increasing emphasis on dissemination of evidence-based practice, and value for money, it is important that researchers seek to understand the effects of process variables on therapeutic outcomes to maximise the effectiveness and efficiency of evidence-based therapy in routine clinical practice. This review highlights that the literature on process variables, specifically therapist adherence and competence as components of treatment integrity, in individual CBT for children and young people is currently sparse, with few significant findings or agreement. It is therefore difficult to make substantial recommendations for CBT practice, based on the available youth process literature. There is a need for future well-designed process studies in youth CBT, in other to understand the ‘active ingredients’ of this approach, to be able to refine protocols, and maximise treatment effectiveness and training; ultimately improving clinical practice and outcomes. The prediction of therapy outcome from therapeutic processes is most likely to be a complex relationship, influenced by a multitude of factors. By identifying such variables, more “effective and efficient CBT interventions” (Webb et al., 2012, p. 663) can be delivered for children

Public health significance.

Cognitive Behavioural Therapy has a growing evidence base for children. However, it is still unclear which mechanisms enable these beneficial effects. How well a therapist adheres to the model, and the competence with which they deliver the therapy, are two possible variables that may contribute to the outcome of therapy. This article reviews the state of the current literature.

References

  1. Barber JP, Gallop R, Crits-Christoph P, Frank A, Thase M, Weiss R, Gibbons M. The role of therapist adherence, therapist competence, and alliance in predicting outcome of individual drug counseling: Results from the National Institute Drug Abuse Collaborative Cocaine Treatment Study. Psychotherapy Research. 2006;16(2):229–240. [Google Scholar]
  2. Barber JP, Krakauer I, Calvo N, Badgio PC, Faude J. Measuring adherence and competence of dynamic therapists in the treatment of cocaine dependence. The Journal of Psychotherapy Practice and Research. 1997;6(1):12. [PMC free article] [PubMed] [Google Scholar]
  3. Barber JP, Liese BS, Abrams MJ. Development of the Cognitive Therapy Adherence and Competence Scale. Psychotherapy Research. 2003;13(2):205–221. [Google Scholar]
  4. Blackburn I-M, James IA, Milne DL, Baker C, Standart S, Garland A, Reichelt FK. The Revised Cognitive Therapy Scale (CTS-R): Psychometric properties. Behav Cogn Psychother. 2001;29(04):431–446. doi: 10.1017/S1352465801004040. [DOI] [Google Scholar]
  5. Boswell JF, Gallagher MW, Sauer-Zavala SE, Bullis J, Gorman JM, Shear MK, et al. Barlow DH. Patient Characteristics and Variability in Adherence and Competence in Cognitive- Behavioral Therapy for Panic Disorder. Journal of Consulting and Clinical Psychology. 2013;81(3):443–454. doi: 10.1037/a0031437. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Branson A, Shafran R, Myles P. Investigating the relationship between competence and patient outcome with CBT. Behaviour Research and Therapy. 2015;68:19–26. doi: 10.1016/j.brat.2015.03.002. [DOI] [PubMed] [Google Scholar]
  7. Carroll KM, Nich C, Ball SA, McCance E, Frankforter TL, Rounsaville BJ. One- year follow- up of disulfiram and psychotherapy for cocaine-alcohol users: sustained effects of treatment. Addiction. 2000;95(9):1335. doi: 10.1046/j.1360-0443.2000.95913355.x. [DOI] [PubMed] [Google Scholar]
  8. Castonguay LG, Goldfried MR, Wiser S, Raue PJ, Hayes AM. Predicting the Effect of Cognitive Therapy for Depression: A Study of Unique and Common Factors. Journal of Consulting and Clinical Psychology. 1996;64(3):497–504. [PubMed] [Google Scholar]
  9. Chinchilla P. Comorbidity as a moderator of process-outcome relations in individual and family therapy for adolescent substance abuse. Fordham University; 2007. Retrieved from https://search.proquest.com/docview/304874700. [Google Scholar]
  10. Cicchetti DV. Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment. 1994;6:284–290. [Google Scholar]
  11. Critical Appraisal Skills Programme. CASP Case Control Checklist. 2017 Retrieved from http://www.casp-uk.net/checklists.
  12. Emmelkamp PMG, David D, Beckers T, Muris P, Cuijpers P, Lutz W, et al. Vervliet B. Advancing psychotherapy and evidence- based psychological interventions. International Journal of Methods in Psychiatric Research. 2014;23(1):58–91. doi: 10.1002/mpr.1411. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Feeley M, DuRubeis RJ, Gelfand LA. The temporal relation of adherence and alliance to symptom change in cognitive therapy for depression. Journal of Consulting and Clinical Psychology. 1999;67:578–582. doi: 10.1037//0022-006x.67.4.578. [DOI] [PubMed] [Google Scholar]
  14. Ginsburg G, Becker K, Drazdowski T, Tein J. Treating Anxiety Disorders in Inner City Schools: Results from a Pilot Randomized Controlled Trial Comparing CBT and Usual Care. Child Youth Care Forum. 2012;41(1):1–19. doi: 10.1007/s10566-011-9156-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Haug T, Nordgreen T, Öst L, Tangen T, Kvale G, Hovland O, et al. Havik OE. Working alliance and competence as predictors of outcome in cognitive behavioral therapy for social anxiety and panic disorder in adults. Behaviour Research and Therapy. 2016;77:40–51. doi: 10.1016/j.brat.2015.12.004. [DOI] [PubMed] [Google Scholar]
  16. Hogue A, Henderson CE, Dauber S, Barajas PC, Fried A, Liddle HA. Treatment adherence, competence, and outcome in individual and family therapy for adolescent behavior problems. Journal of Consulting and Clinical Psychology. 2008;76(4):544–555. doi: 10.1037/0022-006X.76.4.544. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Hogue A, Liddle HA, Rowe C. Treatment adherence process research in family therapy: a rationale and some practical guidelines. Psychotherapy: Theory, Research, Practice, Training. 1996;33(2):332–345. [Google Scholar]
  18. Horvath AO, Del Re AC, Flückiger C, Symonds D. Alliance in Individual Psychotherapy. Psychotherapy. 2011;48(1):9–16. doi: 10.1037/a0022186. [DOI] [PubMed] [Google Scholar]
  19. Huppert JD, Barlow DH, Gorman JM, Shear MK, Woods SW. The interaction of motivation and therapist adherence predicts outcome in cognitive behavioural therapy for panic disorder. Cogn Behav Pract. 2006;13(3):198–204. [Google Scholar]
  20. Kazdin AE, Whitley M, Marciano PL. Child- Therapist and Parent- Therapist Alliance and Therapeutic Change in the Treatment of Children Referred for Oppositional, Aggressive, and Antisocial Behavior. Journal of Child Psychology and Psychiatry. 2006;47(5):436–445. doi: 10.1111/j.1469-7610.2005.01475.x. [DOI] [PubMed] [Google Scholar]
  21. Khan KS, Ter Riet G, Glanville J, Sowden AJ, Kleijnen J. Undertaking systematic reviews of research on effectiveness: CRD's guidance for carrying out or commissioning reviews. NHS Centre for Review and Dissemination; 2001. [Google Scholar]
  22. Kingery JN, Roblek TL, Suveg C, Grover RL, Sherrill JT, Bergman RL. They're not just little adults: Developmental considerations for implementing Cognitive-Behavioral Therapy with anxious youth. Journal of Cognitive Psychotherapy. 2006;20(3):263–273. doi: 10.1891/088983906780644037. [DOI] [Google Scholar]
  23. Liber JM, McLeod BD, Van Widenfelt BM, Goedhart AW, van Der Leeden AJM, Utens EMWJ, Treffers PDA. Examining the relation between the therapeutic alliance, treatment adherence, and outcome of Cognitive Behavioral Therapy for children with anxiety disorders. Behaviour Therapry. 2010;41(2):172–186. doi: 10.1016/j.beth.2009.02.003. [DOI] [PubMed] [Google Scholar]
  24. Martin DJ, Garske JP, Davis MK. Relation of the Therapeutic Alliance With Outcome and Other Variables: A Meta-Analytic Review. Journal of Consulting and Clinical Psychology. 2000;68(3):438–450. [PubMed] [Google Scholar]
  25. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta- analyses: the PRISMA statement. British Medical Journal. 2009;339 [PMC free article] [PubMed] [Google Scholar]
  26. Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group Preferred Reporting Item for Systematic Review and Meta-Analyses: the PRISMA Statement. PLoS Med. 2009;6(7) [PMC free article] [PubMed] [Google Scholar]
  27. Muse K, McManus F. A systematic review of methods for assessing competence in cognitive–behavioural therapy. Clinical Psychology Review. 2013;33(3):484–499. doi: 10.1016/j.cpr.2013.01.010. [DOI] [PubMed] [Google Scholar]
  28. Orlinsky DE, Howard KI. Process and outcome in psychotherapy. In: Garfield SL, Bergin AE, editors. Handbook of psychotherapy and behaviour change. New York: Wiley and Sons; 1986. pp. 311–385. [Google Scholar]
  29. Perepletchikova F, Kazdin AE. Treatment Integrity and Therapeutic Change: Issues and Research Recommendations. Clinical Psychology: Science and Practice. 2005;12(4):365–383. [Google Scholar]
  30. Perepletchikova F, Treat TA, Kazdin AE. Treatment Integrity in Psychotherapy Research: Analysis of the Studies and Examination of the Associated Factors. Journal of Consulting and Clinical Psychology. 2007;75(6):829–841. doi: 10.1037/0022-006X.75.6.829. [DOI] [PubMed] [Google Scholar]
  31. Podell JL. Cognitive behavioural therapy for anxious youth: Therapist variables and child treatment outcome. Temple University; 2011. [Google Scholar]
  32. Shaw BF, Dobson K. Competency Judgments in the Training and Evaluation of Psychotherapists. Journal of Consulting and Clinical Psychology. 1988;56(5):666–672. [PubMed] [Google Scholar]
  33. Stallard P, Myles P, Branson A. The Cognitive Behaviour Therapy Scale for Children and Young People (CBTS-CYP): Development and Psychometric Properties. Behav Cogn Psychother. 2014;42(3):269–282. doi: 10.1017/S135246581300115X. [DOI] [PubMed] [Google Scholar]
  34. Stiles WB. Responsiveness as an Obstacle for Psychotherapy Outcome Research: It's Worse Than You Think. Clinical Psychology: Science and Practice. 2009;16(1):86–91. [Google Scholar]
  35. Stratton KJ. Intervention and impact: An examination of treatment adherence, therapeutic alliance, and outcome in cognitive therapy. Doctoral dissertation, The New School University; 2011. Retrieved from https://search.proquest.com/openview/b5df484a48c536501f53578d030e9fa4/1.pdf?pq-origsite=gscholar&cbl=18750&diss=y. [Google Scholar]
  36. Waltz J, Addis ME, Koerner K, Jacobson NS. Testing the Integrity of a Psychotherapy Protocol: Assessment of Adherence and Competence. Journal of Consulting and Clinical Psychology. 1993;61(4):620–630. doi: 10.1037//0022-006x.61.4.620. [DOI] [PubMed] [Google Scholar]
  37. Webb CA, Auerbach RP, DeRubeis RJ. Processes of Change in CBT of Adolescent Depression: Review and Recommendations. Journal of Clinical Child and Adolescent Psychology. 2012;41(5):654–665. doi: 10.1080/15374416.2012.704842. [DOI] [PubMed] [Google Scholar]
  38. Webb CA, DeRubeis RJ, Barber JP. Therapist adherence/competence and treatment outcome: A meta-analytic review. Journal of Consulting and Clinical Psychology. 2010;78(2):200. doi: 10.1037/a0018912. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Weck F, Grikscheit F, Jakob M, Höfling V, Stangier U. Treatment failure in cognitive- behavioural therapy: Therapeutic alliance as a precondition for an adherent and competent implementation of techniques. British Journal of Clinical Psychology. 2015;54(1):91–108. doi: 10.1111/bjc.12063. [DOI] [PubMed] [Google Scholar]
  40. Zarafonitis-Muller S, Kuhr K, Bechdolf A. [The relationship between therapist's competence and adherence to outcome in cognitive-behavioural therapy-results of a metaanalysis] Fortschritte der Neurologie-Psychiatrie. 2014;82(9):502–510. doi: 10.1055/s-0034-1366798. [DOI] [PubMed] [Google Scholar]

RESOURCES