Client feedback in psychological therapy for children and adolescents with mental health problems

Abstract

Background

Childhood and adolescent mental health problems are a serious and growing concern worldwide. Research suggests that psychotherapy can have a significant and positive impact on children and adolescents with mental health problems, such as anxiety disorders, depression and conduct disorders. Client feedback tools serve as a method of monitoring clients' progress and providing feedback from clients to therapists during the therapeutic process. These tools may help to enhance clinicians' decision‐making by allowing them to adapt their treatment plans as the therapy progresses, resulting in a reduction of treatment failures. Research has shown that client feedback tools have a positive effect on adults' psychotherapy. This review addresses whether feedback tools in child and adolescent therapy could help therapists to better treat their young clients.

Objectives

To assess the effects of client feedback in psychological therapy on child and adolescent mental health outcomes.

Search methods

We searched the Cochrane Common Mental Disorders Controlled Trials Register (CCMDCTR, Studies and References), the Cochrane Central Register of Controlled Trials (CENTRAL), Ovid MEDLINE (1946‐), Embase (1974‐) and PsycINFO (1967‐) to 3 April 2018. We did not apply any restriction on date, language or publication status to the search.

Selection criteria

We included randomised controlled trials (RCTs) that compared client feedback to no client feedback in psychological therapies for children and adolescents.

Data collection and analysis

Two review authors independently assessed references for inclusion eligibility and extracted outcome, risk of bias and study characteristics data into customised forms. We contacted study authors to obtain missing data. We analysed dichotomous data using risk ratios (RRs) and calculated their 95% confidence intervals (CIs). For continuous data, we calculated mean differences (MDs), or standardised mean differences (SMDs) if different scales were used to measure the same outcome. We used a random‐effects model for all analyses.

Main results

We included six published RCTs, conducted in the USA (5 RCTs) and Israel (1 RCT), with 1097 children and adolescents (11 to 18 years old), in the review.

We are very uncertain about the effect of client feedback on improvement of symptoms, as reported by youth in the short term because we considered evidence to be of very low‐certainty due to high risk of bias and very serious inconsistency in the effect estimates from the different studies. Similarly, we are very uncertain about the effect of client feedback on treatment acceptability, due to high risk of bias, imprecision in the results, and indirectness of measuring the outcome (RR 1.08, 95% CI 0.73 to 1.61; 2 studies, 237 participants; very low‐certainty).

Overall, most studies reported and carried out randomisation and allocation concealment adequately. None of the studies were blinded or attempted to blind participants and personnel and were at high risk of performance bias, and only one study had blind outcome assessors. All of the studies were at high or unclear risk of attrition bias mainly due to poor, non‐transparent reporting of participants' flow through the studies.

Authors' conclusions

Due to the paucity of high‐quality data and considerable inconsistency in results from different studies, there is currently insufficient evidence to reach any firm conclusions regarding the role of client feedback in psychological therapies for children and adolescents with mental health problems, and further research on this important topic is needed.

Future studies should avoid risks of performance, detection and attrition biases, as seen in the studies included in this review. Studies from countries other than the USA are needed, as well as studies including children younger than 10 years.

Plain language summary

Client feedback in psychological therapy for children and adolescents with mental health problems

Why is this review important?

Systematic feedback from clients to psychotherapists may improve outcomes of psychological therapy. Typically, client feedback is provided regularly throughout the course of therapy. Clients fill in a questionnaire about how they feel about the therapy and how they are feeling in general. The questionnaire is then scored and reviewed by the therapist. The idea is that the therapist will use the feedback to adjust the therapy process, aiming to improve the client‐therapist and client‐therapy fit and, ultimately, client involvement, treatment response and outcomes. There is empirical evidence for the use of client feedback in adult psychotherapy to improve outcomes and response, but there is limited knowledge about this practice in children and adolescents. This review can provide a better understanding of the current role of client feedback in psychotherapy for children and adolescents, as well as future directions that promote evidence‐based practice in child and adolescent psychotherapy.

Who will be interested in this review?

Psychotherapists working with children and adolescents, policy makers, children and adolescents with mental health problems, and their caregivers and relatives.

What questions does this review aim to answer?

Some research suggests that client feedback has a positive impact on outcomes of psychotherapy with adults. We do not know whether client feedback works with children and adolescents.

Which studies were included in the review?

To be included in the review, studies had to be randomised controlled trials, where children and adolescents with mental health problems were allocated at random (by chance alone) to receive either psychotherapy with client feedback or standard psychotherapy. We searched electronic databases to find all such trials published up until 3 April 2018, and found six (with a total of 1097 children and adolescents) that met our inclusion criteria.

What does the evidence from the review tell us?

There have been few investigations of client feedback in psychological therapies for children and adolescents with mental health problems. Most of them were carried out in the USA with older children and adolescents (11 to 18 years old).

There was no clear evidence supporting the effectiveness of client feedback in psychological therapy for children and adolescents.

What should happen next?

It cannot be ruled out that client feedback has positive effects on psychotherapy outcomes in children and adolescents. High‐quality studies are needed to provide sufficient evidence. Future studies should also include younger children, and should take place in countries other than the USA.

Summary of findings

Summary of findings for the main comparison. Client feedback compared to no client feedback in psychological therapy for children and adolescents with mental health problems.

Client feedback compared to no client feedback in psychological therapy for children and adolescents with mental health problems
Patient or population: children and adolescents, 11 to 18 years old, with mental health problems Setting: community (5 studies) and inpatient (1 study), USA (5 studies) and Israel (1 study) Intervention: client feedback in psychological therapy Comparison: no client feedback in psychological therapy
Outcomes	Anticipated absolute effects* (95% CI)		Relative effect (95% CI)	№ of participants (studies)	Certainty of the evidence (GRADE)	Comments
	Risk with NO CLIENT FEEDBACK	Risk with CLIENT FEEDBACK
Improvement: as reported by youth at postintervention assessed with: OS‐severity (0‐100 points; lower = better), ORS (0‐40 points; lower = worse)**, and Y‐OQ (lower = better) follow‐up: postintervention	See comment	See comment	‐	359 (3 RCTs)	⊕⊝⊝⊝ Very low a, b	We could not pool results due to substantial heterogeneity. In addition, one study found little or no difference between the two groups in reliable clinical change (RR 0.91, 95% CI 0.63 to 1.30, 163 participants)
Treatment acceptability: dropouts from treatment follow‐up: postintervention	Study population		RR 1.08 (0.73 to 1.61)	237 (2 RCTs)	⊕⊝⊝⊝ Very low a, c, d	Another study reported on this outcome as planned sessions not held: MD ‐0.13 (95% CI ‐1.02 to 0.76, 257 participants), and we could not pool this with the other two studies
	280 per 1000	302 per 1000 (204 to 450)
Therapeutic alliance: as reported by youth at postintervention assessed with: SRS (0‐40 points; higher = better), WAI (higher = better) follow‐up: postintervention	See comment	See comment	‐	309 (2 RCTs)	⊕⊝⊝⊝ Very low b, e	We could not pool results due to substantial heterogeneity.
Clients' psychosocial functioning	None of the included studies reported on this outcome
Duration of treatment assessed with weeks follow‐up: postintervention	The mean was 1 to 18 weeks**	The mean in the intervention group was 0.28 weeks shorter (1.57 shorter to 1.01 longer)	‐	593 (3 RCTs)	⊕⊕⊕⊝ Moderate f	**Based on mean scores for included studies (median: 17 weeks)
*The risk in the intervention group (and its 95% confidence interval) is based on the assumed risk in the comparison group and the relative effect of the intervention (and its 95% CI) **Scale reversed to enable pooling CI: confidence interval; OR: odds ratio; OS: Ohio Scales; RR: risk ratio; SFSS: Symptoms and Functioning Severity Scale; SRS: Session Rating Scale; TA: Therapeutic alliance; WAI: Working Alliance Inventory; Y‐OQ: Youth Outcome Questionnaire
GRADE Working Group grades of evidence High certainty: we are very confident that the true effect lies close to that of the estimate of the effect Moderate certainty: we are moderately confident in the effect estimate; the true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different Low certainty: our confidence in the effect estimate is limited; the true effect may be substantially different from the estimate of the effect Very low certainty: we have very little confidence in the effect estimate; the true effect is likely to be substantially different from the estimate of effect

^aDowngraded one step for risk of bias: two studies were of high risk of performance bias, two studies did not adequately describe randomisation and allocation concealment procedures, and two studies were of high risk of attrition bias.
 ^bDowngraded two steps for serious inconsistency: heterogeneity was substantial (I² ≥ 90%) and results could not be pooled.
 ^cDowngraded one step for indirectness: comparing dropouts from treatment can give an indication, but is not a direct measurement of treatment acceptability.
 ^dDowngraded one step for imprecision: CIs were wide and included both no effect and appreciable harm.
 ^eDowngraded one step for risk of bias: the included studies did not adequately describe randomisation and allocation concealment procedures and were of high risk of attrition bias.
 ^fDowngraded one step for risk of bias: all studies were of high risk of performance bias, two studies did not adequately describe randomisation and allocation concealment procedures, two studies were of high risk of detection bias, and two studies were of high risk of attrition bias.

Background

Description of the condition

Childhood mental health problems, such as anxiety disorders, mood disorders and conduct disorders, are a serious and growing concern worldwide (Perou 2013). They are characterised as "serious deviations from expected cognitive, social, and emotional development" (US Department of Health 1999). Prevalence rates for such mental health problems are high. Research suggests, for example, that up to 20% of the paediatric population of the USA warrants a mental health‐related diagnosis each year (Angold 2002), and the World Health Organization (WHO) reports that neuropsychiatric disorders are among the leading causes of worldwide disability in young people (WHO 2011).

The impact of childhood and adolescent mental health disorders is significant. Mental health problems certainly cause notable stress, not just to the child with mental illness but also to that child's entire family (Jewell 2009; Pratt 2007). Child and adolescent mental health disorders are associated with substance use (Hermens 2013), and other risk‐taking behaviours (Lehrer 2006). Additionally, suicidality is strongly correlated with the presence of a psychiatric disorder (Beghi 2013), and suicide is one of the leading causes of death among young people (CDC 2011; Pelkonen 2003). Lastly, mental health problems among children and adolescents are strongly correlated with adult criminal activity (Copeland 2007), and adult mental health problems (NRCIM 2009).

Among adults, rates of mental health disorders have been found to be as high as 33% population‐wide (Kessler 2005). Approximately one‐half of all mental disorders begin prior to the age of 14 years (WHO 2011), therefore effective early intervention is crucial. Therapy can serve as such a critical intervention. Research suggests that psychotherapy can have a significant and positive impact on children and adolescents with mental health problems, such as anxiety disorders, depression and conduct disorders (Manassis 2010; Weisz 2017; Weisz 2013).

Description of the intervention

Several psychological therapies have been evaluated as treatments for mental health problems in children and adolescents. The most widely studied have been different types of cognitive behaviour therapy (CBT), which have been found to be effective for children and adolescents with anxiety (including obsessive compulsive disorder (OCD)) and depression (Arnberg 2014; Compton 2004; James 2015; McGuire 2015). Psychodynamic psychotherapy may also be effective (Abbass 2013), as well as interpersonal therapy (Mufson 2006). However, there is also evidence to suggest that non‐CBT interventions for children and adolescents with anxiety are not significantly beneficial (Reynolds 2012).

Client feedback tools serve as a method of monitoring clients' progress and providing feedback from clients to therapists during the therapeutic process. These tools may help to enhance clinicians' decision‐making by allowing them to adapt their treatment plans as the therapy progresses, resulting in a reduction of treatment failures (Kelley 2009).

The use of feedback tools has evolved in recent years. A series of instruments have been developed to provide feedback to clinicians, allowing them to better evaluate treatment outcomes (McAleavey 2012a). For a feedback measurement system to be effective it should ideally be psychometrically sound, short and useful in everyday practice by clinicians (Kelley 2009). Furthermore, such tools should be able to measure short‐term change, to measure well‐being and psychopathology, and to pair feedback on outcomes with feedback on practices in order to allow for improvement in clinical decision‐making (Seidman 2010).

Examples of client feedback tools used in adults and young people include the following.

• The Contextualized Feedback Systems (CFS), in which various measures are rated by the youth, their caregivers and clinicians (Bickman 2008).

• The Outcome Questionnaire (OQ) system, which includes the use of the OQ‐45, a self‐report scale that measures weekly change in feedback to therapists. It gives a total score and also three subscale scores on symptom distress, interpersonal relations and social role. The OQ also tracks treatment response during and following treatment, which classifies clients into improvers, no‐changers and deteriorators, and includes a signal‐alarm system to notify clinicians if a client is at risk of deteriorating (Lambert 2003).

• The Partners for Change Outcome Management System (PCOMS) is a psychotherapy assurance system that uses two scales: the Outcome Rating Scale (ORS), which measures mental health functioning, and the Session Rating Scale (SRS) (Miller 2005), which assesses therapeutic alliance. Most recently, research on feedback derived from the ORS and SRS system has been adapted for children and adolescents, with the development of the Child Outcomes Rating Scale (CORS) (Duncan 2006), the Child Session Rating Scale (CSRS) (Cooper 2013), and the Young Person's Clinical Outcomes in Routine Evaluation (YP‐CORE) (Twigg 2009). CORS is a self‐rated scale measuring psychological distress in the child and it is reviewed by the therapist and child at the beginning of each session; CSRS is a measure of alliance between the child and the therapist; YP‐CORE is self‐rated tool that provides therapists with a relatively quick method for gathering information on the overall functioning of their client and can be administered at any time between screening and follow‐up.

How the intervention might work

In the course of usual clinical treatment, information on clients' progress is often based solely on the judgement of the treating clinician. However, such assessments can be problematic because there are often critical flaws in clinicians' intuition and observations of the therapeutic process (Kelley 2009), and clinicians are not always able to recognise treatment failure as it is happening (Lambert 2003). The use of feedback tools in psychotherapy serves to counteract this difficulty by providing therapists with information regarding their clients' progress in treatment.

Typically, client feedback is provided regularly throughout the course of therapy. Clients fill in a questionnaire about how they feel about the therapy and how they are feeling in general. The questionnaire is then scored and reviewed by the therapist. The idea is that the therapist will use the feedback to adjust the therapy process, aiming to improve the client‐therapist and client‐therapy fit and, ultimately, client involvement, treatment response and outcomes.

Research suggests that client feedback tools have a positive effect on adults' psychotherapy treatment. Compared with clinical treatment lacking formal feedback methods, clinical treatments that utilise client feedback tools seem to result in a reduction in the number of clients deteriorating during therapy, and an increase in the number of clients who show a clinical improvement (Lambert 2003; Lambert 2011).

Whether the use of client feedback in the treatment of children and adolescents would show the same positive effects is currently being investigated. Recent research suggests that feedback in clinical treatment with young people does in fact have a positive impact on treatment outcomes (Bickman 2011; Shimokawa 2010).

Certain challenges exist relative to the utilisation of feedback tools in the therapy process. Therapists are sometimes reluctant to use such tools, believing that they are the best judges of their clients' progress (Lambert 2003). Additionally, even when such measures are in place, a lack of adherence to the completion of said measures and a lack of utilisation of the feedback provided in the measures can render the feedback moot (Kelley 2009).

Why it is important to do this review

To date, there has been no research to synthesise the current evidence available regarding the use of feedback tools in child and adolescent psychotherapy. Whilst the majority of research available on client feedback pertains to an adult population, there is a limited amount of research available on the role of feedback in clinical work with children and adolescents. As Duncan et al point out, there is a notable lack of formal feedback mechanisms employed in child therapy, giving children and adolescents "little voice in the services they receive" (Duncan 2006). As a result, successes or failures in treatment are generally judged by treating clinicians and not by the parents or clients themselves. Should they prove effective, feedback tools in child therapy could help therapists to better treat their young clients. This review aims to address this issue, examining the current literature available on feedback tools in psychotherapy with children and adolescents and the efficacy of employing such tools.

Objectives

To assess the effects of client feedback in psychological therapy on child and adolescent mental health outcomes.

Methods

Criteria for considering studies for this review

Types of studies

We included randomised or quasi‐randomised controlled trials (RCTs), including cluster‐RCTs.

Types of participants

Participant characteristics

We included studies on children and adolescents up to 18 years of age, of any gender or ethnicity.

Diagnosis

We included studies on children and adolescents with diagnosed mental disorders, such as anxiety disorders, conduct disorders and mood disorders. Diagnosis is usually made by a mental health professional applying the Diagnostic and Statistical Manual of Mental Disorders (APA 2013), or the International Classification of Diseases diagnostic criteria (WHO 1994), during standardised or non‐standardised clinical interview. Specific mental health symptoms and problems can also be assessed using instruments such as the State Trait Anxiety Inventory for Children (Spielberger 1976), Children's Depression Inventory‐2 (Kovacs 1981), and the Minnesota Multiphasic Personality Inventory‐Adolescents (Butcher 1992). We also included children and adolescents presenting with subclinical symptoms of mental disorders, such as symptoms of anxiety, conduct problems or symptoms of depression. We excluded diagnoses of severe mental health disorders, such as psychoses. We planned to investigate any potential impact of the type of mental health problem (see Subgroup analysis and investigation of heterogeneity).

Comorbidities

We included studies regardless of potential physical or mental comorbidity. However, we excluded participants with a known organic source of mental disorder, i.e. triggered by a physical condition of the brain (such as epilepsy or brain tumours), by a systemic condition of the body indirectly affecting the brain (such as sleep deprivation or sensory deprivation), or by alcohol or drug use.

Setting

Primary and secondary care psychiatric or psychotherapeutic settings (community, inpatient or outpatient mental health services).

Types of interventions

Experimental intervention

We included studies evaluating systematic client feedback from clients or caregivers, or both, to the therapist or counsellor as a component of any kind of psychological therapy, as laid out below. We excluded studies evaluating pharmacological treatments.

Client feedback is defined, in this systematic review, as subjective ratings by clients or clients' caregivers, or both, on any aspect of the therapy or counselling they are receiving, or on any aspect of their mental health, or both. The ratings are based on standardised measures and are used by the mental health professional to inform further treatment planning. Examples of standardised client feedback tools for children and adolescents include the Child Outcomes Rating Scale (CORS) (Duncan 2006), the Child Session Rating Scale (CSRS) (Cooper 2013), and the Young Person's Clinical Outcomes in Routine Evaluation (YP‐CORE) (Twigg 2009). We planned to investigate any potential impact due to whether the client or the caregiver provided the feedback (see Subgroup analysis and investigation of heterogeneity).

Types of psychological therapy

We planned to group psychological therapies into four overall categories based on their theoretical underpinning, similar to the Cox 2014 Cochrane Review.

• Cognitive behaviour therapy (CBT), which uses cognitive restructuring training and teaches behavioural changes, including cognitive therapy, behavioural therapy, problem‐solving therapies and 'third wave' psychotherapies such as mindfulness.

• Integrative therapies, including interpersonal therapy where the relationship between mood and relationship problems is explored and the focus is on improving relationship skills, and counselling as it typically incorporates a range of different approaches and techniques.

• Humanistic therapies, which offer an empathic, non‐directive and non‐judgemental approach, based on client‐centred principles, including interventions described as 'supportive therapies'.

• Psychodynamic therapies, where the therapeutic relationship is used to explore and resolve unconscious conflict through the use of interpretation and transference, including play therapy.

However, only one of the Included studies prescribed to a specific type or types of psychological therapy.

Comparator intervention

Psychological therapy (as described above) without systematic client feedback.

Types of outcome measures

We included studies that met the above inclusion criteria regardless of whether they reported on the outcomes below.

Primary outcomes

• Improvement rates or numbers (e.g. percentage of clients who are clinically recovered), or level or change of continuous scores on relevant validated scales (e.g. Youth Self‐Report (YSR) (Achenbach 2001), Child Behavior Checklist (CBCL) (Achenbach 2001), Strengths and Difficulties Questionnaire (SDQ) (Goodman 1997)).

• Treatment acceptability, as measured by number of dropouts from therapy. Planned, early termination was not included in this outcome. We attempted to collect information from authors if it was unclear whether dropouts might have included planned, early terminations.

Secondary outcomes

• Therapeutic alliance. Examples of relevant scales include the Therapeutic Alliance Rating Scale (Marziali 1981), and the Working Alliance Inventory (Horvath 1989).

• Client's psychosocial functioning (e.g. Children's Global Assessment Scale (Shaffer 1983)).

• Duration of treatment. Time in treatment or number of sessions.

• Client satisfaction (e.g. the Multidimensional Adolescent Satisfaction Scale (Garland 2000a; Garland 2000b)).

Timing of outcome assessment

We collected outcome measures for different follow‐up time points, if available, categorised as follows: short‐term (less than 2 months); medium‐term (2 to 12 months); and long‐term (more than 12 months). Our primary time point was medium‐term. See Appendix 1 for further details on how we planned to deal with multiple time points.

Hierarchy of outcome measures

We prioritised child‐reported measures over caregiver‐, teacher‐ or therapist‐reported measures. See Appendix 1 for details on selecting one scale among many as outcome measures.

Search methods for identification of studies

Cochrane Common Mental Disorders Controlled Trials Register (CCMDCTR)

The Cochrane Common Mental Disorders Group (CCMD) maintains two archived clinical trials registers at its editorial base in York, UK: a references register and a studies‐based register. The CCMDCTR‐References Register contains over 40,000 reports of RCTs in depression, anxiety and neurosis. Approximately 50% of these references have been tagged to individual, coded trials. The coded trials are held in the CCMDCTR‐Studies Register and records are linked between the two registers through the use of unique Study ID tags. Coding of trials is based on the EU‐Psi coding manual, using a controlled vocabulary; (please contact the CCMD Information Specialists for further details). Reports of trials for inclusion in the Group's registers are collated from routine (weekly), generic searches of MEDLINE (1950 to 2016), Embase (1974 to 2016) and PsycINFO (1967 to 2016); quarterly searches of the Cochrane Central Register of Controlled Trials (CENTRAL) and review‐specific searches of additional databases. Reports of trials are also sourced from international trial registers via the World Health Organization's trials portal (the International Clinical Trials Registry Platform (ICTRP)), pharmaceutical companies, the handsearching of key journals, conference proceedings and other (non‐Cochrane) systematic reviews and meta‐analyses.

Details of CCMD's generic search strategies (used to identify RCTs) can be found on the Group's website, with an example of the core MEDLINE search used to inform the register displayed in Appendix 2.

The Group's Specialised Register had fallen out of date with the Editorial Group's move from Bristol to York in June 2016.

Electronic searches

The Cochrane Common Mental Disorders Information Specialist conducted searches on the following databases.

• CCMDCTR (Studies and References Register) (all years to 28 June 2016).

• Cochrane Central Register of Controlled Trials (CENTRAL) (all years to Issue 4, 2018).

• Ovid MEDLINE (2016 to 3 April 2018).

• Ovid Embase (2016 to 3 April 2018).

• Ovid PsycINFO (all years to 3 April 2018).

Searches for this review have been through a number of iterations. In light of relevant studies identified from earlier searches conducted by the author team (Appendix 3), and the search methodologies reported by Kendrick 2016, we searched the CCMDCTR in July 2015 and April 2018 using the following search strategy (note: the register was only current to 28 June 2016 at this time).

CCMDCTR (Studies and References Register)

#1. ((physician* or psychiatri* or psychotherapist* or therapist* or "primary care" or "general practi*") and (client* or patient* or oupatient*) near (feedback or feed‐back)):ti,ab,kw,ky,mh,mc,emt
 #2. (("psychotherapeutic outcome*" or "treatment outcome*") and (feedback or feed‐back)):ti,ab,kw,ky,mh,mc,emt
 #3. ("patient reported" near (assessment* or feedback or feed‐back or information or outcome* or progress*)):ti,ab,kw,ky,mh,mc,emt
 #4. (#1 or #2 or #3)
 #5. ((active or routine* or regular*) NEAR2 (evaluation or feedback or feed‐back or measurement* or monitor* or outcome*)):ti,ab
 #6. (CORS or CSRS or OQ or PCOMS or SRS):ab
 #7. ((outcome* or goal*) NEAR (manag* or monitor*)):ti
 #8. (feedback or feed‐back):ti
 #9. (#5 or #6 or #7 or #8)
 #10. (CAMHS or child* or boy* or girl* or infant* or juvenil* or minors or paediatric* or pediatric* or *school* or kindergarten or nursery or *adolesc* or *pubert* or pubescen* or pupil* or *teen* or young or youth* or student* or undergrad* or graduate or graduates or college or campus or university):ti,ab
 #11.(adolesc* or child* or paediatric* or pediatric*):so
 #12. (#10 or #11)
 #13. ((#4 or #9) and #12)

Key to Cochrane Register of Studies (CRS) search fields:
 ab:abstract; ti:title; kw:keywords; ky:other keywords; emt:EMTREE headings; mh: MeSH headings; mc:MeSH checkwords; so:source/journal name

We carried out additional searches in July 2015 on the Cochrane Central Register of Controlled Trials (CENTRAL; 2015, Issue 7), OVID PsycINFO (all years to 7 July 2015) (Appendix 4), and in April 2018 on the Cochrane Central Register of Controlled Trials (CENTRAL; 2018, Issue 4,) and OVID MEDLINE, Embase and PsycINFO (2016 to 3 April 2018) (Appendix 5).

We also conducted a cited reference search for reports of included studies on the Web of Science (7 July 2015).

We did not apply any restriction on date, language or publication status to the search.

Searching other resources

Grey literature

We searched sources of grey literature, including dissertations and theses, clinical guidelines and reports from regulatory agencies, at Open Grey (www.opengrey.eu).

Handsearching

We did not carry out any additional handsearching to that already undertaken for the CCMDCTR.

Reference lists

We checked the reference lists of all included studies and relevant systematic reviews to identify additional studies missed from the original electronic searches (for example, unpublished or in‐press citations). We also conducted a cited reference search on the Web of Science.

Correspondence

We contacted trialists and subject experts for information on unpublished or ongoing studies or to request additional trial data.

Data collection and analysis

Methods from the protocol that were not used can be found in Appendix 1. These methods were not used because the evidence we found did not allow it.

Selection of studies

Two review authors (HB, KSW) independently screened all citations and abstracts identified in the search. We obtained full reports for potentially eligible studies and two review authors (HK, KHB) independently screened these. We resolved any disagreements through discussion. If an agreement could not be reached, a senior review author (AB) made a final decision. We documented the justifications for excluding studies from the review. We outlined the process of study identification and its results as flow diagrams according to the PRISMA statement (Moher 2009).

Data extraction and management

We developed data extraction forms in an electronic format using web‐based systematic review software (DistillerSR). We pilot tested the data extraction forms to ensure ease of use and ability to capture all relevant data. Two review authors (HB, KSW) independently carried out the data extraction. We documented any disagreements about data extraction and resolved these by consensus. If an agreement could not be reached, a senior review author (AB) made a final decision.

We collected the following study characteristics for each included study: study design, study duration, setting, recruitment, number randomised/analysed per group, participant age, participant gender, type of mental health problem, description of interventions including type of psychological therapy, therapist qualifications, intervention duration, outcomes, follow‐up time point of outcome, for outcome scales: person completing the scale.

Where outcomes were reported with insufficient detail to include in a meta‐analysis, for instance, mean effects without confidence intervals (CIs) or standard deviations (SDs), we contacted the study authors to request more information.

If we identified more than one publication for the same included study, we considered the main publication to be the one with more information or with long‐term outcomes; we considered all others companion publications and we only collected data if they had not been provided in the main publication.

Comparisons

Client feedback in psychological therapy compared to no client feedback in psychological therapy.

See Appendix 1 on how we planned to stratify studies within analyses.

Scale‐derived data

We included continuous data from rating scales only if the psychometric properties of the measuring instrument had been described in a peer‐reviewed journal (Marshall 2000). Ideally the measuring instrument should either be (i) a self‐report or (ii) completed by an independent rater or caregiver (not the therapist). We realise that this is not often reported clearly and noted it in the table of Characteristics of included studies. We prioritised endpoint data over change from baseline data as endpoint data are easier to interpret from a clinical point of view. If endpoint data were not available, we used change data and analysed these in a separate subgroup (Higgins 2011).

Skewed data

Continuous data on clinical and social outcomes are often not normally distributed. To avoid the pitfall of applying tests that assume normally distributed data to non‐normally distributed data, we aimed to apply the following standards to all data before inclusion: a) SDs and means are reported in the paper or obtainable from the authors; b) when a scale starts from the finite number zero, the SD, when multiplied by two, is less than the mean (as otherwise the mean is unlikely to be an appropriate measure of the centre of the distribution (Altman 1996).

Assessment of risk of bias in included studies

We classified studies as at 'low', 'unclear' or 'high' risk of bias, based on the domain‐specific assessments of risk of bias of Cochrane's 'Risk of bias' tool criteria (Higgins 2011). We assessed the following domains for risk of bias.

• Random sequence generation.

• Allocation concealment.

• Blinding of participants and personnel.

• Blinding of outcome assessment.

• Incomplete outcome data.

• Selective outcome reporting.

• Other bias, in particular (i) outcomes that are assessed by the same measure used for client feedback, and (ii) therapist qualification, but also other biases as described in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011).

If we could not make an assessment due to lack of information, we contacted the authors of the study to request more information.

Two review authors (HK and KHB or HB) independently evaluated risk of bias and resolved differences by discussions with a senior review author (AB).

When considering treatment effects in the summary of findings (see 'Summary of findings' tables below), we took into account the risk of bias for the studies that contributed to that outcome using the GRADE approach (Schünemann 2008). In addition, we planned to carry out sensitivity analyses excluding studies with a high risk of bias (see Sensitivity analysis).

Measures of treatment effect

For binary data we extracted the numbers in each treatment group and the numbers experiencing the outcome of interest, and calculated a standard estimation of the risk ratio (RR) and its 95% CI.

For continuous outcomes we collected the endpoint mean, the SD of the mean and the number of participants for each treatment group at each assessment. Where endpoint mean was not reported, we extracted changes from baseline and SD at each time point. We calculated effect size measures using the mean difference (MD) and its 95% CI. We combined data from different scales that measured the same outcome by using the standardised mean difference (SMD) and its 95% CI. We separated means and SDs of endpoint and change from baseline data into subgroups in the analyses and we combined the subgroups, if appropriate, using MD and its 95% CI (Higgins 2011).

Unit of analysis issues

Cluster‐randomised trials

For cluster‐randomised studies (such as randomisation by clinician or practice), we adjusted for the clustering effect by dividing the clusters by a 'design effect'. We calculated this using the mean number of participants per cluster (m) and the intraclass correlation coefficient (ICC) [Design effect = 1+(m‐1)*ICC] (Donner 2002). If the ICC was not reported, we derived it from a similar study. We synthesised appropriately analysed cluster‐RCTs, taking into account ICCs and other relevant data extracted from study reports, with other studies using the generic inverse variance technique.

See also Appendix 1 on how we planned to deal with cross‐over studies and studies with multiple treatment groups.

Dealing with missing data

We tried to contact study authors for any unreported data. For any particular outcome, if more than 50% of data were unaccounted for, we did not reproduce these data or use them within analyses. We extracted data to allow an intention‐to‐treat analysis in which all randomised participants were analysed in the groups to which they were originally assigned. For continuous outcomes, we calculated missing SDs from other available data such as CIs, standard errors, P, T or F values, as detailed in the Cochrane Handbook for Systematic Reviews of Interventions, section 7.7.3 (Higgins 2011).

Assessment of heterogeneity

Clinical heterogeneity

We inspected all studies for clearly outlying people or situations that we had not predicted would arise and fully discussed these.

Statistical heterogeneity

We quantified inconsistency in findings across studies using the I² statistic, which measures the percentage of observed variation that can be attributed to true differences between the studies (Higgins 2003). We also examined the strength of evidence for heterogeneity using the Chi² test P value. We interpreted the I² estimate, accompanied by a statistically significant Chi² statistic, as suggested in the Cochrane Handbook for Systematic Reviews of Interventions, section 9.5.2 (Higgins 2011):

• 0% to 40%: might not be important;

• 30% to 60%: may represent moderate heterogeneity;

• 50% to 90%: may represent substantial heterogeneity;

• 75% to 100%: considerable heterogeneity.

Assessment of reporting biases

We planned to assess publication bias by inspecting a funnel plot for asymmetry, should more than 10 studies be combined for an outcome.

Data synthesis

We used a random‐effects model for data synthesis as there were a distribution of different, but related, estimations of intervention effects, particularly since different conditions and psychotherapies were included.

Studies reporting skewed data were transformed on a log scale to reduce skew. In order to combine such results with data from other studies, we log‐transformed their reported data as well, and we analysed them using the SMD (section 9.4.5.3 Meta‐analysis of skewed data in Higgins 2011).

We summarised narratively the results of outcomes not considered appropriate for meta‐analysis.

Subgroup analysis and investigation of heterogeneity

Client feedback may have different effects in different subgroups of children and adolescents, and with different treatment conditions. Therefore, we planned to carry out the following subgroup analyses for the primary outcomes.

• Type of mental health problem (e.g. anxiety, depression). We expect children and adolescents with behaviour disorders to be more outspoken in their feedback to therapists, and perhaps exaggerate negative experiences, while anxious/depressed children and adolescents may be more inhibited in their feedback.

• Type of psychological therapy (cognitive behaviour therapy (CBT), integrative, humanistic, psychodynamic). The evidence base for CBT is quite extensive, while the impact of the other three types of therapy is more unclear. We cannot exclude the possibility that the different types of therapies will have different impacts on outcome, with or without the use of client feedback.

• Number of treatment sessions (fewer than 4 sessions, 4 to 12 sessions or more than 12 sessions). There are likely to be differences in the numbers of therapy sessions received and this is expected to affect treatment outcome. For instance, it is likely that short‐term therapies will increase adherence and cooperation in this age group.

• Client feedback provided by the client or by the caregivers. Children and adolescents will have different experiences of therapy sessions from their caregivers and they might use different types of language to describe their experiences. These differences may have an impact on how the therapist interprets and adapts to the feedback. In addition, small children may have difficulties in giving feedback themselves, so feedback from parents would be relied on.

• Frequency of feedback (every session, 50% of sessions or more, less than 50% of sessions). High‐frequency feedback will give the therapist more opportunity to change the line of therapy and adjust to the needs of the client; we expect this to affect the impact on outcome.

Sensitivity analysis

We aimed to conduct sensitivity analyses for the primary outcomes, excluding:

• quasi‐randomised studies, because inadequate randomisation could lead to allocation of particular subgroups of participants to particular intervention groups and opportunities for manipulation of the randomisation sequence;

• studies that are not cluster‐randomised trials, because therapists would learn from feedback from other participants when randomisation was carried out at individual participant level;

• studies that measure outcomes with the same instrument used for client feedback; and

• studies rated as high risk of bias for allocation concealment, because group allocation could be predicted in studies where this is not concealed adequately.

'Summary of findings' table

We used the GRADE approach to summarise and interpret findings (Schünemann 2008), and the GRADE profiler to import data from Review Manager 2014 to create 'Summary of findings' tables. These tables provide outcome‐specific information concerning the overall quality of evidence (risk of bias, inconsistency, indirectness, imprecision and publication bias) from each included study in the comparison, the magnitude of effect of the interventions examined, and the sum of available data on all outcomes rated as important to client care and decision‐making. We included the following outcomes in Table 1.

• Improvement rates or numbers (e.g. percentage of participants who are clinically recovered).

• Improvement scores on relevant validated scales (e.g. Youth Self‐Report (YSR), Child Behavior Checklist (CBCL) or Strengths and Difficulties Questionnaire (SDQ)).

• Treatment acceptability, as measured by number of dropouts from therapy.

• Therapeutic alliance (e.g. Therapeutic Alliance Rating Scale or Working Alliance Inventory).

• Client's psychosocial functioning (e.g. Children's Global Assessment Scale).

• Duration of treatment (time in treatment or number of sessions).

Results

Description of studies

Results of the search

The first search in November 2013 identified 5657 deduplicated references, the second search in July 2015 identified 388 deduplicated references, the third search in July 2016 identified 99 deduplicated references, and the fourth search in April 2018 identified 159 deduplicated references. We screened 6301 references, and excluded 6256 references in title and abstract screening because of irrelevance. We retrieved 45 full‐text papers and assessed their eligibility. We excluded 32 references, included six studies (9 references), and identified four ongoing studies. Please see Figure 1 for PRISMA flow diagram.

1.

Study flow diagram.

We contacted all study authors for additional information and received replies from Drs Bickman, Lester and Melendez.

Included studies

We included six studies with 1097 participants in this review, with characteristics as follows (see also Characteristics of included studies).

Design

The review included five randomised controlled trials (RCTs) and one cluster‐RCT of client feedback versus no client feedback in psychological therapy for children and adolescents with mental health problems. The cluster trial used mental health treatment site as the unit of randomisation (Bickman 2011‐cluster). Bickman 2015‐rural and Bickman 2015‐urban were in fact one study, and results were separated by site in this review as they were reported in the publication. Treatment was provided for a few days in Lester 2013 to five months in Bickman 2015‐urban. We did not find any cross‐over trials in the study search. Each study was published in English.

Sample sizes

The sample size for studies ranged from 69 in Melendez 2002 to 340 in Bickman 2011‐cluster.

Setting

Five studies were conducted in the USA and one study in Israel (Shechtman 2016). One study was conducted in an inpatient setting (Lester 2013), and the remaining studies in the community.

Participants

All studies included children and adolescents from 11 to 12 up to 17 to 18 years old, although one study did not report the age range of participants (Ogles 2006). The mean age of participants ranged from 13 to 15 years old in the included studies.

The gender balance in most of the included studies was equal. One of the included studies had 80% male participants (Melendez 2002).

Lester 2013 included participants from an acute inpatient department who posed a serious and imminent risk of harm to self or others due to a psychiatric illness (depressive disorder, bipolar disorder, mood disorder). Melendez 2002 included participants with mood disorder, pervasive developmental disorder (PDD), conduct disorder, oppositional defiant disorder (ODD), and attention deficit hyperactivity disorder (ADHD). The average youth in the Ogles 2006 study was exhibiting problems within the clinical range and typical of youth receiving mental health services, with many with a history of previous hospitalisation, suspension in school, arrest, placement in a foster home, group home or other placement, or leaving home without permission over night. Shechtman 2016 included school children with attention deficit disorder (ADD), ADHD, behaviour problem diagnosis, or undiagnosed problem behaviours, such as aggression, antisocial behaviour, loneliness, shyness or withdrawal. Bickman 2011‐cluster and Bickman 2015‐rural/Bickman 2015‐urban did not report what type of mental health problems the youths exhibited.

Interventions

Feedback condition

Bickman 2011‐cluster and Bickman 2015‐rural/Bickman 2015‐urban used Contextualized Feedback Systems (CFS) as the client feedback tool (Bickman 2012). CFS is a web‐based application that provides computerised feedback reports to agency personnel (e.g. directors, supervisors, clinicians) based on the Peabody Treatment Progress Battery (PTPB; Bickman 2010), a psychometrically sound and clinically useful battery of measures that promotes overall practice improvement through frequent and comprehensive assessments. These very brief measures were completed at the close of a treatment session by the youth, caregiver, and clinician. The battery assesses symptom severity, therapeutic alliance, life satisfaction, motivation for treatment, hope, treatment expectations, caregiver strain, and service satisfaction. Examples of feedback included mean scores and alerts if the youth’s symptoms ranked in the top 25th percentile in severity, indicators of whether change from one measurement instance to the next met criteria for reliable change, and trend graphs for change over multiple measurement points.

Lester 2013 used Partners for Change Outcome Management System (PCOMS) as the client feedback tool (Miller 2005). PCOMS is the joint use of two measures. The first is the Outcome Rating Scale (ORS; Miller 2003), which tracks the outcome of therapy from the perspective of the client. It is an ultra‐brief, easy to administer measure that assesses personal well‐being or symptom distress, how well the client is getting along within the family system, satisfaction with school and relationships outside of the house, and a subjective overall sense of well‐being. The ORS is of moderate validity and solid reliability (Duncan 2006), and has been adapted to use with children and adolescents (Duncan 2003a; Duncan 2006). The second is the Session Rating Scale (SRS; Duncan 2003b), which tracks the therapeutic alliance with clients at every session. The entire SRS is based on encouraging clients to identify therapeutic alliance problems and to elicit client concerns about the therapeutic process so that the clinician may change to better fit client expectations. The SRS is of solid reliability and adequate validity (Duncan 2003b), and has been adapted to use with children and adolescents (Cooper 2013). The combination of the ORS and SRS were administered, and discussed with the experimental group participants during individual therapy sessions.

Melendez 2002 used the Ohio Scales‐Short version‐Problems Severity Scale (unpublished; derived from the Ohio Scales (Ogles 2001)), and the Therapeutic Alliance Scales for Children (TASC) as the client feedback tool (Shirk 1992). The Ohio Scales are intended as a practical and rigorous device for the assessment of outcomes for youth with severe emotional disturbance. The TASC measures the child's positive and negative affective response to the therapist, and the child's collaboration with the therapist. The feedback report to the therapist contained scores from youth, teachers, and the therapist, and indicated the quality of change in each reporting source's score from the initial session to the most recent session.

Ogles 2006 provided feedback about outcome progress, but did not specify which measures were used for the feedback. Outcome measures for this study included the Ohio Scales (Ogles 2001), Target Complaints (Battle 1966), Goal Attainment Scaling (GAS) (Kiresuk 1994), Family Adaptability and Cohesion Scale (FACES III) (Olson 1985); presumably all or some of these scales were used for the feedback. We requested clarification from study investigators, but no further information was available at the time this review was prepared. A brief feedback report was distributed to the wraparound service team, including the parent(s), after each of four data collection points.

Shechtman 2016 used the Youth Outcome Questionnaire (Y‐OQ‐30) as the client feedback tool (Burlingame 2004). The Y‐OQ‐30 is a 30‐item instrument for the assessment of adolescents' well‐being containing six subscales: somatic symptoms, loneliness, behaviour problems, aggression, anxiety and depression. Participants completed the questionnaire at the end of each session and therapists received weekly reports.

Frequency of feedback

Therapists in Bickman 2015‐rural/Bickman 2015‐urban, and Lester 2013 received client feedback at every session. In Bickman 2011‐cluster, Melendez 2002, and Shechtman 2016 therapists received feedback weekly, and in Ogles 2006 after two, four, eight and 12 weeks.

No feedback condition

Only three of the included studies compared client feedback to strictly no client feedback (Lester 2013; Ogles 2006; Shechtman 2016). Bickman 2011‐cluster compared weekly feedback to feedback every 90 days. Youths in this study remained in treatment for about four months, mean = 3.8±3.1, median = 3.3. Bickman 2015‐rural/Bickman 2015‐urban compared session‐by‐session feedback to feedback every six months. Participants of this study stayed in treatment for an average four to five months. Melendez 2002 compared full feedback from youths, parents and therapists to limited feedback only from the therapist. Although three of the studies did not strictly compare feedback to no feedback, we included these studies because 1) the average participant in Bickman 2011‐cluster and Bickman 2015‐rural/Bickman 2015‐urban would have left treatment by the time the therapist in the control condition viewed the feedback report; and 2) the therapists in the control condition of Melendez 2002 did not receive feedback from the client, only therapist scores.

Type of therapy

Only in Shechtman 2007 was a specific type of therapy used: supportive‐expressive group counselling, which involved process‐oriented groups, in which children and adolescents were encouraged to express their concerns or difficulties, go through cognitive and affective exploration, develop insight into their behaviour, and make a commitment to change. The therapeutic process for each individual was based on the three‐stage model of change suggested by Hill 2005. The major therapeutic factors proposed by Yalom 2005, such as interpersonal learning, catharsis, and group cohesion, were also present in these groups. Counsellors used structured activities, such as therapeutic games, bibliotherapy, phototherapy, art therapy, and therapeutic cards, to help the children and adolescents express their emotions and to aid in establishing a climate for effective group work.

In none of the other included studies was a specific type or types of psychological therapy prescribed. In Bickman 2011‐cluster, clinicians reported using various therapeutic approaches, including cognitive‐behavioural, integrative‐eclectic, behavioural, family systems, and play therapy. Evidence‐based mental health services were used in Bickman 2015‐rural/Bickman 2015‐urban; a third (33%) of clinicians reported subscribing to a cognitive‐behavioural therapeutic orientation, however, many (48%) reported some other unspecified orientation or that they had no particular therapeutic orientation. In Lester 2013, adolescents admitted for acute care received routine interventions by a child psychiatrist and intensive co‐ordinated treatment by a physician‐led team of mental health professionals including psychologists, social workers, and nurses. The programme also provided weekly individual and family therapy, daily process groups, skill building activities, and educational instruction. In Melendez 2002, participants received weekly individual and family clinical services provided by a licensed mental health worker. The services were focused on crisis stabilisation, although participants could receive planned individual and group therapy as well. Ogles 2006 included participants of wraparound services, which are family‐centred and child‐focused individualised packages of support and services for children and adolescents with serious emotional disturbances, and which are a popular alternative to out‐of‐home placement (in the USA). Although psychological therapy is very likely to be included in wraparound services, there was no special mention in the study report.

Duration of treatment

Duration of treatment ranged from (means) one week with 1.8 sessions in Lester 2013, to 19 weeks with 12 sessions in Bickman 2015‐urban.

Therapists' training and experience

Nearly all therapists in Bickman 2011‐cluster, Bickman 2015‐rural/Bickman 2015‐urban and Melendez 2002 were trained to the level of a Master's degree. Therapists in Lester 2013 were licensed Professional Counsellors, licensed Psychological Examiners, and licensed Clinical Social Workers. Counsellors in Shechtman 2016 were graduate students of a counselling programme that had received training in group counselling and the use of feedback information, as well as weekly supervision. Most therapists in Bickman 2015‐rural/Bickman 2015‐urban had less than five years' experience with providing services for children or youth. Counsellors in Shechtman 2016 had at least two years' experience, and therapists in Lester 2013 had four to five years' experience. Bickman 2011‐cluster and Melendez 2002 did not report on therapists' experience. Ogles 2006 did not report on training or experience of therapists.

Outcomes

Primary outcomes

Only one study reported dichotomous results on improvement level or change: Shechtman 2016 reported on the number of participants achieving a reliable clinical change. All studies reported on improvement level or change on various scales. Lester 2013 reported on the ORS (Duncan 2006; Miller 2003), as reported by adolescents and parents at discharge (typically 3 to 14 days) and also on the Y‐OQ (Burlingame 2004), but we chose to use the ORS data in the analysis because the ORS is more widely applied in clinical practice. Melendez 2002 reported on the Problem severity scale of the Ohio Scales (Dowell 2008; Ogles 2001; Turchik 2007), as reported by adolescents, therapists and teachers at six weeks, 12 weeks and at six to nine months. Shechtman 2016 reported on the Y‐OQ as reported by adolescents. Bickman 2011‐cluster and Bickman 2015‐rural/Bickman 2015‐urban reported analysis of variances (ANOVAs) for the Symptoms and Functioning Severity Scale (SFSS) (Athay 2012; Lambert 2015), which is a part of the Peabody Treatment Progress Battery (PTPB). For the Ohio Scales, Ogles 2006 reported data for a post hoc, non‐randomised sample (participants with clinical versus subclinical problems). We requested clarification from study investigators, but no further information was available at the time this review was prepared.

The scores on these scales were also measures for the client feedback in the studies.

Treatment acceptability, as measured by number of dropouts from therapy was reported by Melendez 2002. Bickman 2015‐rural/Bickman 2015‐urban reported on planned sessions not held. The remaining studies did not report complete results on dropouts. We requested clarification from study investigators, but no further information was available at the time this review was prepared.

Secondary outcomes

Lester 2013 used the SRS (Miller 2005), as reported by adolescents to measure therapeutic alliance at discharge (typically 3 to 14 days). Shechtman 2016 used the Working Alliance Inventory (WAI) (Horvath 1989), as reported by adolescents at the end of treatment. Melendez 2002 used the Therapeutic Alliance Scales for Children (TASC; Shirk 1992), but we could not use the results in the review as no means and SDs (or equivalent) were reported. We requested clarification from study investigators, but no further information was available at the time this review was prepared. The scores on these scales were also measures for the client feedback in the studies.

Duration of treatment (both in weeks or days and number of sessions held) was reported by Bickman 2011‐cluster, Bickman 2015‐rural/Bickman 2015‐urban and Lester 2013.

Ogles 2006 reported on clients' satisfaction (parent report) using the Goal Attainment Scaling (GAS) (Kiresuk 1994). In addition, Lester 2013 and Melendez 2002 reported on therapists' attitudes or satisfaction with using the feedback system, but we could not use the results in the review as no means and SDs (or equivalent) were reported.

No studies reported on participants' psychosocial functioning.

Excluded studies

We excluded 32 studies, with the following reasons: seven studies were not RCTs, 15 studies included only adults, five studies did not compare systematic client feedback, three studies did not compare feedback in a therapy context, one study compared two different types of client feedback, and in one study client feedback was only a minor component in a complex intervention. See Characteristics of excluded studies for further details.

Ongoing studies

We identified four ongoing studies.

• NCT01873742 is a RCT comparing client feedback with no client feedback in children, adolescents and adults ≥ 12 years (planned sample size = 200) in Norway.

• NCT02023736 is a RCT comparing client feedback with no client feedback in children, adolescents and adults ≥ 12 years (planned sample size = 1000) in the USA.

• NCT02567266 is a RCT comparing client feedback with no client feedback in children and adolescents aged 12 to 18 years (planned sample size = 222) in the USA.

• van Sonsbeek 2014 is a RCT recruiting four to 17‐year olds (planned sample size = 432) at outpatient mental health care departments in eastern Netherlands. Children and adolescents with all kinds of mental health problems and all kinds of treatment will be included in the study. The trial has three arms: 1) feedback as usual/control: clinicians receive basic feedback regarding symptoms and quality of life of the client; 2) the feedback of condition 1 is extended with feedback about the results of the additional questionnaire and with practical suggestions to improve treatment; 3) the feedback of condition 2 is discussed with colleagues on the basis of a standardised format for case consultation.

See Characteristics of ongoing studies for more details.

Risk of bias in included studies

For details of the risk of bias judgements for each study, see Characteristics of included studies. A graphical representation of the overall risk of bias in included studies is presented in Figure 2 and Figure 3.

2.

Risk of bias graph: review authors' judgements about each risk of bias item presented as percentages across all included studies.

3.

Risk of bias summary: review authors' judgements about each risk of bias item for each included study.

Allocation

Three studies described an adequate method for generating a sequence for randomisation and allocation concealment (Bickman 2015‐rural/Bickman 2015‐urban; Melendez 2002; Ogles 2006), and we judged them to be at low risk of selection bias. We judged three studies to be at unclear risk of selection bias (Bickman 2011‐cluster; Lester 2013; Shechtman 2016). They reported that the participants were randomly assigned, but no details on random sequence generation or allocation concealment were provided in the published reports. We requested clarification from study investigators, but no further information was available at the time this review was prepared.

Blinding

In all included studies except Shechtman 2016, therapists were aware of feedback condition, therefore, we judged these studies to be at high risk of performance bias. In Shechtman 2016, therapists were aware of which intervention they gave, but this may not have represented any bias, as knowledge of of the intervention was actually the experimental condition itself. Participants may or may not have been aware of which intervention they received. Therefore, we judged the risk of performance bias in Shechtman 2016 to be unclear.

All self‐report (client or carer assessed) outcomes (various improvement scales, therapeutic alliance, psychosocial functioning, client satisfaction) were at high risk of detection bias.

For outcomes assessed by researchers or therapists (various improvement scales), we judged Bickman 2011‐cluster to be of low risk of detection bias as blinding of outcome assessment was adequately described. Three studies did not report on blinding of outcome assessors and we judged them to be at unclear risk of detection bias (Bickman 2015‐rural/Bickman 2015‐urban; Ogles 2006;Shechtman 2016). The remaining studies did not have researcher or therapist assessed outcomes.

Objective outcomes, including dropouts from treatment, duration of treatment and planned sessions not held were at low risk of detection bias.

Incomplete outcome data

We judged two studies to be of unclear risk of attrition bias. In Bickman 2011‐cluster there was significant attrition from the initial random assignments of sites that could have biased the samples. Attrition rates at follow‐up were high and unequal between groups in Melendez 2002; subjects were more likely to drop out from the limited feedback group.

We judged four studies to be of high risk of attrition bias. In Bickman 2015‐rural/Bickman 2015‐urban only randomised participants that had reported on a symptom scale were included in analyses, the study did not report initial numbers randomised. In Lester 2013, two participants in the intervention group were identified as outliers and excluded from further analyses. Ogles 2006 reported that dropout rates were not significantly different between the two groups, but reasons for dropping out, number of dropouts per group, and number of participants analysed per group were not reported. In addition, the study reports that there were "relatively large number of dropouts between 3 and 9 months" with no further details reported on these dropouts (Shechtman 2016).

Selective reporting

Results were reported for all outcomes stated in the methods section for four of the studies, and we judged them to be at low risk of selective reporting bias. Ogles 2006 reported on outcomes in post hoc subgroups and most data could not be used, therefore, we judged this study to be at high risk of selective reporting bias.

Other potential sources of bias

Bickman 2015‐rural/Bickman 2015‐urban, Melendez 2002 and Ogles 2006 all seemed to be free of other biases and we judged them to be at low risk. They all reported baseline characteristics that were similar between groups. Bickman 2011‐cluster was a cluster‐RCT and the study authors clearly reported the intracluster correlation coefficient (ICC) and adjusted for clustering in analyses; we subsequently judged this study at low risk as we could not detect any other potential biases. We judged Lester 2013 at unclear risk of other bias as there were differences in baseline characteristics.

Effects of interventions

See: Table 1

Client feedback versus no client feedback

Five studies including 858 participants contributed data to this comparison. See also: Table 1.

Primary outcomes

1.1 Improvement

1.1.1 Improvement, dichotomous data

One study reported on the number of participants achieving a reliable clinical change and found that there was little or no difference between the group where therapists received client feedback and those that did not (risk ratio (RR) 0.91, 95% confidence interval (CI) 0.63 to 1.30; 163 participants, very low‐certainty evidence; Analysis 1.1).

1.1. Analysis.

Comparison 1 Client feedback versus no client feedback, Outcome 1 Improvement, dichotomous: number of participants with clinical improvement.

1.1.2 Improvement, data from various scales

See Table 2 for details of scale data extracted for improvement.

1. Details of scale data extracted for primary outcome 'improvement'.

Study	Scale	Intervention: mean (SD); N	Control: mean (SD); N	Comments
Lester 2013	ORS (low = worse) at discharge (typically 3‐14 days), youth report	34.73 (6.49); N = 58	30.59 (7.38); N = 60	Entered to Analysis 1.2 reversed scale and log transformed to enable meta‐analysis with Melendez 2002 and Shechtman 2016 skewed data
Lester 2013	ORS (low = worse) at discharge (typically 3‐14 days hospitalisation), parent report	25.17 (7.04); N = 58	23.23 (8.39); N = 60	Entered to Analysis 1.2 reversed scale and log transformed to enable meta‐analysis with Melendez 2002 and Shechtman 2016 skewed data
Melendez 2002	Ohio Scales ‐ Problem Severity Scores (high = worse), postintervention, youth report	7.08 (6.17); N = 25	17.48 (16.58); N = 25	Entered to Analysis 1.2 log transformed skewed data to enable meta‐analysis
Melendez 2002	Ohio Scales ‐ Problem Severity Scores (high = worse), postintervention, therapist report	11.2 (8.16); N = 25	12.76 (9.67); N = 25	Entered to Analysis 1.2 log transformed skewed data to enable meta‐analysis
Melendez 2002	Ohio Scales ‐ Problem Severity Scores (high = worse), postintervention, teacher report	9.04 (7.51); N = 25	10.56 (7.89); N = 25	Entered to Analysis 1.2 log transformed skewed data to enable meta‐analysis
Melendez 2002	Ohio Scales ‐ Problem Severity Scores (high = worse), medium‐term (6 to 9 months), youth report	14.5 (11.43); N = 16	18.54 (17.88); N = 13	Entered to Analysis 1.4 log transformed skewed data to enable meta‐analysis
Melendez 2002	Ohio Scales ‐ Problem Severity Scores (high = worse), medium‐term (6 to 9 months), therapist report	21.47 (11.99); N = 17	18.33 (13.33); N = 15	Entered to Analysis 1.4 log transformed skewed data to enable meta‐analysis
Melendez 2002	Ohio Scales ‐ Problem Severity Scores (high = worse), medium‐term (6 to 9 months), teacher report	17.43 (13.75); N = 14	14.27 (12.09); N = 11	Entered to Analysis 1.4 log transformed skewed data to enable meta‐analysis
Shechtman 2016	Y‐OQ‐30 ‐ total symptoms (high = worse), short‐term (at last session), youth report	20.44 (18.76); N = 107	15.4 (15.12); N = 84	Entered to Analysis 1.2 log transformed skewed data to enable meta‐analysis

N: number
 ORS: Outcome Rating Scale
 SD: standard deviation
 Y‐OQ: Youth Outcome Questionnaire

Due to high heterogeneity (I² = 91%) we were unable to pool results from youth reports in the short term (postintervention or at discharge); three studies reported on this outcome, see Analysis 1.2 and Figure 4. Two studies found that client feedback resulted in improvement compared to no client feedback: mean difference (MD) ‐0.13 (95% CI ‐0.21 to ‐0.05; 118 participants) and MD ‐0.90 (95% CI ‐1.43 to ‐0.38; 50 participants) respectively, whereas one study found the opposite effect: MD 0.28 (95% CI 0.01 to 0.56; 191 participants). We considered risk of bias to be high for this outcome. At medium‐term follow‐up, one study reported little or no difference between the two conditions: MD ‐0.25 (95% CI ‐0.96 to 0.46; 29 participants; Analysis 1.4).

1.2. Analysis.

Comparison 1 Client feedback versus no client feedback, Outcome 2 Improvement, various scales: short‐term (youth report).

4.

Forest plot of comparison: 1 Client feedback versus no client feedback, outcome: 1.2 Improvement, various scales: short‐term (youth report).

1.4. Analysis.

Comparison 1 Client feedback versus no client feedback, Outcome 4 Improvement: Ohio Scales‐Problem Severity, medium‐term.

There was little or no difference in improvement between client feedback and no client feedback from therapist reports (Analysis 1.3; Figure 4): MD 0.11 (95% CI ‐0.34 to 0.56; 1 study, 50 participants), parent reports: MD ‐1.94 (95% CI ‐4.73 to 0.85; 1 study, 118 participants), and teacher reports: MD 0.44 (95% CI ‐0.05 to 0.93; 1 study, 50 participants) in the short term. There was general consistency in study results (I² = 0%). At medium‐term follow‐up, one study reported little or no difference between the two conditions for therapist reports: MD 0.16 (95% CI ‐0.31 to 0.63; 32 participants) and for teacher reports: MD 0.20 (95% CI ‐0.50 to 0.90; 25 participants; Analysis 1.4).

1.3. Analysis.

Comparison 1 Client feedback versus no client feedback, Outcome 3 Improvement, various scales: short‐term (parent, therapist, teacher report).

We were unable to use data from two studies of 597 participants, which reported no statistically significant differences between the groups using repeated measures hierarchical longitudinal model (HLM) analysis, see Table 3.

2. Data not added to analyses.

Outcome	Study	Participants (youths)	Results	Analysis methods
Improvement‐SFSS, youth	Bickman 2011‐cluster	Not reported	Accumulated from several sessions: MD 0.02, SE 0.10; N (completed forms) = 1341	HLM, repeated measures analysis, adjusted for clustering and race
Improvement‐SFSS, clinician	Bickman 2011‐cluster	Not reported	Accumulated from several sessions: MD 0.10, SE 0.10; N (completed forms) = 1291
Improvement‐SFSS, carer	Bickman 2011‐cluster	Not reported	Accumulated from several sessions: MD 0.01, SE 0.13; N (completed forms) = 935
Improvement‐SFSS, youth	Bickman 2015‐rural	141	Accumulated from several sessions: MD ‐0.05, SE 0.09	HLM, repeated measures analysis
Improvement‐SFSS, carer	Bickman 2015‐rural	141	Accumulated from several sessions: MD 0.06, SE 0.11
Improvement‐SFSS, clinician	Bickman 2015‐rural	138	Accumulated from several sessions: MD 0.10, SE 0.06
Improvement‐SFSS, youth	Bickman 2015‐urban	115	Accumulated from several sessions: MD 0.11, SE 0.10
Improvement‐SFSS, carer	Bickman 2015‐urban	113	Accumulated from several sessions: MD 0.06, SE 0.12
Improvement‐SFSS, clinician	Bickman 2015‐urban	94	Accumulated from several sessions: MD ‐0.00, SE 0.10
Therapeutic alliance: TASC, youth report	Melendez 2002	47	The difference at week 12 remains minimal, M = 37.88 for high versus M = 35.57 for low. After nine weeks of treatment, the high feedback youth reported an average increase of 1.5 points, while the youth in the low feedback group reported a slight decrease in alliance of less than one point; this on a scale of 12 to 48	Repeated measures ANOVA
Therapeutic alliance: TASC, therapist report	Melendez 2002	48	Alliance scores for the high feedback group increased by an average 1.9 points, while scores for the low feedback group increased by an average of only 0.10 points	Repeated measures ANOVA

ANOVA: analysis of variance
 HLM: hierarchical longitudinal model

M: mean
 MD: mean difference
 SE: standard error
 SFSS: Symptoms and Functioning Severity Scales
 TASC: Therapeutic Alliance Scale for Children

1.2 Treatment acceptability

There was very low‐certainty evidence of little or no difference in treatment acceptability between client feedback and no client feedback, as measured by participants dropping out from treatment: RR 1.08 (95% CI 0.73 to 1.61; 2 studies, 237 participants; Analysis 1.5; Figure 5) and by planned treatment sessions not held MD ‐0.13 (95% CI ‐1.02 to 0.76; 1 study, 257 participants; Analysis 1.6; Figure 6). There was general consistency in study results (I² = 0%). We considered risk of bias to be high for this outcome.

1.5. Analysis.

Comparison 1 Client feedback versus no client feedback, Outcome 5 Treatment acceptability: dropouts from treatment.

5.

Forest plot of comparison: 1 Client feedback versus no client feedback, outcome: 1.5 Treatment acceptability: dropouts from treatment.

1.6. Analysis.

Comparison 1 Client feedback versus no client feedback, Outcome 6 Treatment acceptability: planned sessions not held.

6.

Forest plot of comparison: 1 Client feedback versus no client feedback, outcome: 1.6 Treatment acceptability: planned sessions not held.

Secondary outcomes

1.3 Therapeutic alliance

Due to high heterogeneity (I² = 90%) we were unable to pool results from youth reports in the short term (postintervention or at discharge), see Analysis 1.7. One study using the Session Rating Scale (SRS) to measure therapeutic alliance found that client feedback was more effective than no client feedback: MD 1.59 (95% CI 0.50 to 2.68; 1 study, 118 participants, very low‐certainty evidence). This difference represents a clinically important difference from 37.25 points (35 to 38 points = fair therapeutic alliance) in the no‐feedback group to 38.84 points (39 to 40 = good therapeutic alliance) in the feedback group. Another study using the Working Alliance Inventory (WAI) to measure therapeutic alliance found little or no difference between client feedback and no client feedback: MD ‐0.21 (95% CI ‐0.47 to 0.05; 191 participants, very low‐certainty evidence). We were unable to use data from another study of 48 participants, which described a minimal difference in therapeutic alliance favouring client feedback as reported by youth and therapists, as SDs were not reported, see Table 3. We considered risk of bias to be high for this outcome.

1.7. Analysis.

Comparison 1 Client feedback versus no client feedback, Outcome 7 Therapeutic alliance: youth‐rated, short‐term.

1.4 Clients' psychosocial functioning

None of the included studies reported on this outcome.

1.5 Duration of treatment

There was moderate‐certainty evidence of little or no difference between client feedback and no client feedback: MD ‐0.28 (95% CI ‐1.57 to 1.01; 3 studies, 593 participants; Analysis 1.8). There was general consistency in study results (I² = 18%). We considered risk of bias to be high for this outcome.

1.8. Analysis.

Comparison 1 Client feedback versus no client feedback, Outcome 8 Duration of treatment (weeks).

1.6 Client satisfaction

One study found that parents of children and adolescents receiving client feedback were more satisfied with treatment than those receiving no client feedback: RR 2.24 (95% CI 1.31 to 3.81; 1 study, 72 participants; Analysis 1.9). We considered risk of bias to be high for this outcome.

1.9. Analysis.

Comparison 1 Client feedback versus no client feedback, Outcome 9 Client satisfaction: Goal Attainment Scaling (GAS), parent report, medium‐term.

Subgroup analyses

Type of mental health problem

All studies included participants with different types of problems or did not describe types of mental health problems of the participants.

Type of psychological therapy

Only in one study was a specific type of psychological therapy prescribed: supportive‐expressive group counselling (see Included studies for more details). This study found a reduction in improvement with client feedback compared to no client feedback (standardised mean difference (SMD) 0.29, 95% CI 0.01 to 0.58; 191 participants) whereas the two studies that did not prescribe to any particular type of therapy found increased improvement with client feedback compared to no client feedback (SMD ‐0.69, 95% CI ‐1.00 to ‐0.37; 168 participants); see Analysis 1.10.

1.10. Analysis.

Comparison 1 Client feedback versus no client feedback, Outcome 10 Subgroup analysis: type of therapy ‐ improvement, various scales: short‐term (youth report).

Number of treatment sessions

One study provided less than four sessions on average and found client feedback to be more effective than no client feedback in symptom improvement at postintervention (MD ‐0.13, 95% CI ‐0.21 to ‐0.05; 118 participants; Analysis 1.11), but little or no difference in treatment acceptability (RR 1.08, 95% CI 0.68 to 1.72; 118 participants; Analysis 1.12). One study provided 12 sessions and found a reduction in symptom improvement with client feedback compared to no client feedback (MD 0.28, 95% CI 0.01 to 0.56; 191 participants; Analysis 1.11).

1.11. Analysis.

Comparison 1 Client feedback versus no client feedback, Outcome 11 Subgroup analysis: number of sessions ‐ improvement: various scales (youth report), short‐term.

1.12. Analysis.

Comparison 1 Client feedback versus no client feedback, Outcome 12 Subgroup analysis: number of sessions ‐ treatment acceptability: dropouts from treatment.

One study provided four to 12 sessions on average and found little or no difference between client feedback and no client feedback in treatment acceptability (MD 0.48, 95% CI ‐0.75 to 1.71; 116 participants; Analysis 1.13).

1.13. Analysis.

Comparison 1 Client feedback versus no client feedback, Outcome 13 Subgroup analysis: number of sessions ‐ treatment acceptability: planned sessions not held.

Another study provided over 12 sessions on average and also found little or no difference between client feedback and no client feedback in treatment acceptability (MD ‐0.47, 95% CI ‐1.16 to 0.22; 141 participants; Analysis 1.13).

The other study that reported on Improvement did not report on number of sessions and we excluded it from this subgroup analysis (Melendez 2002).

Client feedback provided by the client or by the caregivers

Client feedback was provided by the client (child or adolescent) in all three studies that reported on improvement. Again, we could not pool these results due to high heterogeneity (I² = 91%). Again, two studies found that client feedback resulted in improvement compared to no client feedback: SMD ‐0.59 (95% CI ‐0.96 to ‐0.22; 118 participants) and SMD ‐0.93 (95% CI ‐1.52 to ‐0.33; 50 participants), whereas one study found the opposite effect: SMD 0.29 (95% CI 0.01 to 0.58; 191 participants; Analysis 1.2).

The two studies where client feedback was provided by the clients, found little or no difference between client feedback and no client feedback in treatment acceptability (RR 1.08, 95% CI 0.73 to 1.61; 237 participants; 2 studies; Analysis 1.14).

1.14. Analysis.

Comparison 1 Client feedback versus no client feedback, Outcome 14 Subgroup analysis: feedback by client or parent ‐ treatment acceptability: dropouts from treatment.

In two studies both parents and clients provided client feedback. These studies also found little or no difference between client feedback and no client feedback in treatment acceptability (MD ‐0.13, 95% CI ‐1.02 to 0.76; 257 participants, 2 studies; Analysis 1.15).

1.15. Analysis.

Comparison 1 Client feedback versus no client feedback, Outcome 15 Subgroup analysis: feedback by client or parent ‐ treatment acceptability: planned sessions not held.

Frequency of feedback

Feedback was on a session‐by‐session basis for one of the included studies that reported on improvement; this study found that client feedback may result in improvement compared to no client feedback (MD ‐0.13, 95% CI ‐0.21 to ‐0.05; 118 participants). The other studies that reported on improvement provided feedback in more than 50% of the sessions, results from these studies could not be pooled due to high heterogeneity (I² = 93%); one of the studies found that client feedback may result in improvement compared to no client feedback (MD ‐0.90, 95% CI ‐1.43 to ‐0.38; 50 participants) whereas the other study found the opposite effect (MD 0.28, 95% CI 0.01 to 0.56; 118 participants; Analysis 1.16).

1.16. Analysis.

Comparison 1 Client feedback versus no client feedback, Outcome 16 Subgroup analysis: frequency of feedback ‐ improvement: various scales (youth report), short‐term.

Three studies that reported on treatment acceptability found little or no difference between client feedback and no client feedback for dropouts: (RR 1.08, 95% CI 0.68 to 1.72; 168 participants, 1 study; Analysis 1.17); or planned sessions not held: (MD ‐0.13, 95% CI ‐1.02 to 0.76; 257 participants, 2 studies; Analysis 1.18). One study that reported on treatment acceptability found little or no difference between client feedback and no client feedback for dropouts: (RR 1.08, 95% CI 0.50 to 2.32; 69 participants; Analysis 1.17).

1.17. Analysis.

Comparison 1 Client feedback versus no client feedback, Outcome 17 Subgroup analysis: frequency of feedback ‐ treatment acceptability: dropouts from treatment.

1.18. Analysis.

Comparison 1 Client feedback versus no client feedback, Outcome 18 Subgroup analysis: frequency of feedback ‐ treatment acceptability: planned sessions not held.

We did not make any comparisons on subgroups because there were too few studies per subgroup (< 10).

Sensitivity analyses

Quasi‐randomised studies

None of the studies were quasi‐randomised.

Studies that are not cluster‐randomised trials

The only included cluster‐randomised study did not provide usable data for any of the primary outcomes.

Studies that measure outcomes with the same instrument used for client feedback

All included studies measured improvement with the same instrument that was used in client feedback.

Studies rated at high risk of bias for allocation concealment

We did not rate any of the studies at high risk of bias for allocation concealment.

Reporting bias

We planned to assess publication bias by inspecting a funnel plot for asymmetry, should more than 10 studies be combined for an outcome. This review included only six studies, therefore, we did not assess publication bias.

Discussion

Summary of main results

We aimed to assess the effects of client feedback in psychological therapy on child and adolescent mental health outcomes. Six studies with 1097 children and adolescents (11 to 18 years old) with mental health problems were included in the review. The studies were all randomised controlled trials (RCTs) with one of them being cluster‐randomised. Five studies were carried out in the USA and one in Israel, one on acutely ill inpatients and the remaining in the community. Participants received mixed types of psychological therapy, and in three of the studies the therapy was part of a wider package of care, also including support measures, such as crisis stabilisation, skills building, and educational instruction.

Please see Table 1 for details of our findings at the postintervention time point. For the first primary outcome, improvement (measured by different scales reported by adolescents), we could not pool results from the different studies due to very high inconsistency and we considered the evidence to be of very low certainty. For the second primary outcome, treatment acceptability (measured by unplanned dropouts from treatment), we are very uncertain about the effect of client feedback; only two studies measured this outcome and the certainty of evidence was again very low.

Two studies reported on therapeutic alliance, and again we considered the evidence to be of very low certainty, and again we could not pool effect estimates due to very serious inconsistency. No studies reported on psychosocial functioning. Finally, there was moderate‐certainty evidence from threee studies that client feedback probably makes little or no difference to the duration of treatment.

Overall completeness and applicability of evidence

The studies included in this review all address the study question. They all applied systematic client feedback to therapists throughout the course of standard psychotherapy.

However, only three of the six included studies compared client feedback to strictly no client feedback (Lester 2013; Ogles 2006; Shechtman 2016). We included the remaining three studies although they compared client feedback to limited client feedback (Bickman 2011‐cluster; Bickman 2015‐rural/Bickman 2015‐urban; Melendez 2002), and thus did not strictly meet the review inclusion criteria. The limited client feedback was either delivered six months later in the control condition than in the intervention, or it was reported as therapist ratings of clients' problem severity. In our view, the delayed feedback was practically equal to no feedback, as most clients would have left therapy by the time the therapist viewed the feedback. Further, we judged the therapist ratings to be sufficiently different from clients' self‐reports to be included as a control condition.

We were unable to include three studies in the analyses of the primary outcome (improvement level or change) because of failure to provide data for the randomised sample (Ogles 2006), or report only of accumulated scores (Bickman 2011‐cluster; Bickman 2015‐rural/Bickman 2015‐urban). We contacted the authors but did not obtain any further data.

The scales used to measure improvement or change were the same as those used for client feedback. This may introduce a bias, as clients in the intervention groups would have been aware that their therapists would view their scores in the next session, while clients in the control condition would have known that their therapists would not view their scores. The two study groups' response sets could have been influenced by this. Preferably, client feedback scales and study outcome measures should be different in order to avoid bias.

Further, none of the outcome scales that were predefined in the protocol as measures of improvement (Achenbach scales Youth Self‐Report (YSR) and Child Behavior Checklist (CBCL), and Strengths and Difficulties Questionnaire (SDQ)) were used by study authors. Instead, we included the following validated scales that were used by study authors to report on improvement: Outcome Rating Scale (ORS), Ohio Scales‐Short version‐Problems Severity Scale, the Symptoms and Functioning Severity Scale (SFSS), and the Youth Outcome Questionnaire (Y‐OQ). These scales measure general psychological well‐being, and problem severity, or function, or both. They seem conceptually similar to the Achenbach scales and SDQ, although the similarity has not been statistically tested, as far as we know.

Only three of the included studies provided data for our other primary outcome, treatment acceptability (Bickman 2015‐rural/Bickman 2015‐urban; Lester 2013; Melendez 2002). We contacted the authors of the other studies but did not obtain any further data.

We had wished to ascertain whether client feedback is more effective for some therapy forms than others by stratifying studies by type of therapy. However, most included studies did not specify type(s) of psychological therapy received by the adolescents, and when they did they were mixed.

Most of the included studies were conducted in the USA with 11 to 18‐year old adolescents of unknown ethnicity. The participating adolescents had various mental health problems, often not specified, both within and across studies. It is unclear whether the results of this review are generalisable to younger samples or to non‐American adolescents.

Certainty of the evidence

We rated the certainty of the evidence for the primary outcome improvement as very low due to risk of bias and serious inconsistency. Our risk of bias assessments concluded with high risk of performance, detection or attrition bias, or both, and some effect estimates were in favour of client feedback and some were in favour of no client feedback for the different studies. There were too few included studies to establish whether these differences in direction of the effect estimates were due to any specific subgroup differences. It is uncertain whether client feedback has any effect on symptom improvement.

We rated the certainty of the evidence for our other primary outcome, treatment acceptability, as very low due to risk of bias, indirectness and imprecision. There were data from three studies available for this outcome, however we could only analyse two together (see Table 1). We assessed their risk of performance and attrition bias to be high. We downgraded for indirectness because the studies compared dropouts from treatment, which is not a direct measure of treatment acceptability. We also downgraded for imprecision, as the confidence intervals (CIs) were wide and included both no effect and appreciable harm. It is uncertain whether client feedback has any effect on treatment acceptability.

Potential biases in the review process

We are fairly confident that our literature searches and screening process enabled us to identify all published and unpublished RCTs and quasi‐RCTs on the effectiveness of client feedback in psychological therapy for children and adolescents with mental health problems, as defined by our protocol and retrieved up to our most recent search date (3 April 2018). We performed systematic searches in the former Cochrane Depression, Anxiety and Neurosis Group's Specialised Register (CCDANTR), including two trial registries, to identify both completed and ongoing trials. We also searched sources of grey literature, as well as reference lists for additional published or unpublished literature. Two review authors independently screened potentially eligible studies for inclusion and assessed risk of bias in included studies. None of the review authors had any conflicts of interest.

Agreements and disagreements with other studies or reviews

To our knowledge, there are no previous research syntheses of the effects of client feedback in child and adolescent psychotherapy. There are research reports suggesting that client feedback improves outcomes in adult psychotherapy, but these are not based on systematic review processes (Lambert 2003; Lambert 2011).

Authors' conclusions

Implications for practice.

Due to the paucity of studies with low risk of bias and inconsistency in results, there is currently insufficient evidence to reach any firm conclusions regarding the role of client feedback in psychological therapies for children and adolescents with mental health problems, and further research on this important topic is needed. Practicing clinicians, clients and parents should know that there is a lack of clear evidence for using client feedback in psychological therapy, and should also consider that the weak evidence is very inconsistent regarding the effect of client feedback on improvement of symptoms and therapeutic alliance.

Implications for research.

There is a need for more well‐designed, large‐scale trials to obtain conclusive evidence regarding the effects of client feedback in child and adolescent psychotherapy. Future studies should avoid risks of performance, detection and attrition biases, as seen in the studies included in this review. Specifically, outcomes should be measured with different instruments than the scales used to provide client feedback to the therapists. Studies from countries other than the USA are needed, as well as studies including children younger than 10 years of age.

Appendices

Appendix 1. Additional methods table

Methods section in review	Additional methods from protocol
Criteria for considering studies for this review – Types of studies	We will also include cross‐over trials. For cross‐over trials, we will only use data from the first active treatment phase.
Criteria for considering studies for this review – Types of participants	We will also include studies that include only a subset of relevant participants (e.g. those that include both adult and child populations) only if the study includes more than 80% eligible participants.
Criteria for considering studies for this review – Types of outcome measures – Timing of outcome assessment	If data are available at more than one time point within one of the defined time categories (e.g. at 3 and 6 months for the medium‐term time point), we will select the latest within each time category.
Criteria for considering studies for this review – Types of outcome measures – Hierarchy of outcome measures	Where there are several possible measures for one outcome, we will select the measures or scales in the order laid out for each outcome, as above, and any other validated scales after those.
Data collection and analysis – Data extraction and management ‐ Comparisons	We planned to stratify studies according to the following type of psychological therapy. Cognitive behaviour therapy (CBT) Integrative therapies Humanistic therapies Psychodynamic therapies
Data collection and analysis – Unit of analysis issues – Cross‐over trials	A major concern of cross‐over trials is the carry‐over effect. It occurs if an effect of the treatment in the first phase is carried over to the second phase. Therefore, for identified cross‐over studies, we would only use data from the first phase (Elbourne 2002).
Data collection and analysis – Unit of analysis issues – Studies with multiple treatment groups	For studies involving more than two treatment arms or subgroups, if relevant, we summed both the sample sizes and the numbers of people with events for dichotomous data across groups. Similarly, for continuous data, we combined numbers into a single sample size, mean and standard deviation using the formulae in section 7.7.3.8 (Combining groups) of the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011).

Appendix 2. CCMDCTR Core MEDLINE Search

Core search strategy used to inform the Cochrane Common Mental Disorders Group's specialised register: OVID MEDLINE (to June 2016)
 A weekly search alert based on condition + RCT filter only
 1. [MeSH Headings]:
 eating disorders/ or anorexia nervosa/ or binge‐eating disorder/ or bulimia nervosa/ or female athlete triad syndrome/ or pica/ or hyperphagia/ or bulimia/ or self‐injurious behavior/ or self mutilation/ or suicide/ or suicidal ideation/ or suicide, attempted/ or mood disorders/ or affective disorders, psychotic/ or bipolar disorder/ or cyclothymic disorder/ or depressive disorder/ or depression, postpartum/ or depressive disorder, major/ or depressive disorder, treatment‐resistant/ or dysthymic disorder/ or seasonal affective disorder/ or neurotic disorders/ or depression/ or adjustment disorders/ or exp antidepressive agents/ or anxiety disorders/ or agoraphobia/ or neurocirculatory asthenia/ or obsessive‐compulsive disorder/ or obsessive hoarding/ or panic disorder/ or phobic disorders/ or stress disorders, traumatic/ or combat disorders/ or stress disorders, post‐traumatic/ or stress disorders, traumatic, acute/ or anxiety/ or anxiety, castration/ or koro/ or anxiety, separation/ or panic/ or exp anti‐anxiety agents/ or somatoform disorders/ or body dysmorphic disorders/ or conversion disorder/ or hypochondriasis/ or neurasthenia/ or hysteria/ or munchausen syndrome by proxy/ or munchausen syndrome/ or fatigue syndrome, chronic/ or obsessive behavior/ or compulsive behavior/ or behavior, addictive/ or impulse control disorders/ or firesetting behavior/ or gambling/ or trichotillomania/ or stress, psychological/ or burnout, professional/ or sexual dysfunctions, psychological/ or vaginismus/ or Anhedonia/ or Affective Symptoms/ or *Mental Disorders/

2. [Title/ Author Keywords]:
 (eating disorder* or anorexia nervosa or bulimi* or binge eat* or (self adj (injur* or mutilat*)) or suicide* or suicidal or parasuicid* or mood disorder* or affective disorder* or bipolar i or bipolar ii or (bipolar and (affective or disorder*)) or mania or manic or cyclothymic* or depression or depressive or dysthymi* or neurotic or neurosis or adjustment disorder* or antidepress* or anxiety disorder* or agoraphobia or obsess* or compulsi* or panic or phobi* or ptsd or posttrauma* or post trauma* or combat or somatoform or somati#ation or medical* unexplained or body dysmorphi* or conversion disorder or hypochondria* or neurastheni* or hysteria or munchausen or chronic fatigue* or gambling or trichotillomania or vaginismus or anhedoni* or affective symptoms or mental disorder* or mental health).ti,kf.

3. [RCT filter]:
 (controlled clinical trial.pt. or randomized controlled trial.pt. or (randomi#ed or randomi#ation).ab,ti. or randomly.ab. or (random* adj3 (administ* or allocat* or assign* or class* or control* or determine* or divide* or distribut* or expose* or fashion or number* or place* or recruit* or subsitut* or treat*)).ab. or placebo*.ab,ti. or drug therapy.fs. or trial.ab,ti. or groups.ab. or (control* adj3 (trial* or study or studies)).ab,ti. or ((singl* or doubl* or tripl* or trebl*) adj3 (blind* or mask* or dummy*)).mp. or clinical trial, phase ii/ or clinical trial, phase iii/ or clinical trial, phase iv/ or randomized controlled trial/ or pragmatic clinical trial/ or (quasi adj (experimental or random*)).ti,ab. or ((waitlist* or wait* list* or treatment as usual or TAU) adj3 (control or group)).ab.)

4. (1 and 2 and 3)

Records are screened for reports of RCTs within the scope of the Cochrane Common Mental Disorders Group. Secondary reports of RCTs are tagged to the appropriate study record.
 Similar weekly search alerts are also conducted on OVID Embase and PsycINFO, using relevant subject headings (controlled vocabularies) and search syntax, appropriate to each resource.

Appendix 3. Searches conducted by the Author team (to Nov 2013)

Embase via Ovid SP was searched (1974 to 2013 November 25) using the following terms:

1. feedback/ or (((client$ or patient$) adj (response or respond or progress$)) or "client? guid$ assess$" or "client? satisfaction questionnaire?" or "clinical outcomes in routine evaluation" or "core‐om" or feedback$ or "feeding back" or feed‐back? or "fed back" or "goal? attainment scal$" or "helpful aspects of therapy" or "help$ alliance questionnaire?" or "idiographic assess$" or "important events questionnaire" or "monitor$ alliance" or "therapy alliance" or "treatment alliance" or "outcome? questionnaire$" or "outcome? monitor$" or "outcome? rating scal$" or patient‐focused or patient‐progress$ or "progress monitor$" or "progress track$" or "session? rating scal$" or "session? report$ form?" or "partner? for chang$" or "pcoms" or "treatment outcome? pack$" or "treatment progress$" or "treatment response" or "treatment respond").ti,ab.
 2. exp psychotherapy/ or exp counseling/ or (Psychotherap$ or counsel$ or ((cogniti$ or behavio?r$) adj (therap$ or treat$))).ti,ab.
 3. (adolescen$ or boys or boy or girl or girls or child$ or juvenile? or kid or kids or minor or minors or kindergar$ or parent? or mother? or father? or p?ediatric$ or pe?diatric$ or pre?school or school$ or high?school or student? or pre?teen$ or teen$ or mid?teen$ or late?teen$ or puber$ or pubescen$ or pre?pubescen$ or pupil? or student? or school or young$ or youth?).ti,ab. or (child$ or adolescen* or p?ediatr* or pe?diatr*).jw.
 4. 1 and 2 and 3
 5. 1 and 2
 6. limit 5 to (child or preschool child <1 to 6 years> or school child <7 to 12 years> or adolescent <13 to 17 years>)
 7. 4 or 6
 8. limit 7 to (human and embase)

MEDLINE via Ovid SP was searched (1946 to 25 Nov 2013) using the following terms:

1. feedback/ or (((client$ or patient$) adj (response or respond or progress$)) or "client? guid$ assess$" or "client? satisfaction questionnaire?" or "clinical outcomes in routine evaluation" or "core‐om" or feedback$ or "feeding back" or feed‐back? or "fed back" or "goal? attainment scal$" or "helpful aspects of therapy" or "help$ alliance questionnaire?" or "idiographic assess$" or "important events questionnaire" or "monitor$ alliance" or "therapy alliance" or "treatment alliance" or "outcome? questionnaire$" or "outcome? monitor$" or "outcome? rating scal$" or patient‐focused or patient‐progress$ or "progress monitor$" or "progress track$" or "session? rating scal$" or "session? report$ form?" or "partner? for chang$" or "pcoms" or "treatment outcome? pack$" or "treatment progress$" or "treatment response" or "treatment respond").ti,ab.
 2. exp psychotherapy/ or exp counseling/ or (Psychotherap$ or counsel$ or ((cogniti$ or behavio?r$) adj (therap$ or treat$))).ti,ab.
 3. (adolescen$ or boys or boy or girl or girls or child$ or juvenile? or kid or kids or minor or minors or kindergar$ or parent? or mother? or father? or p?ediatric$ or pe?diatric$ or pre?school or school$ or high?school or student? or pre?teen$ or teen$ or mid?teen$ or late?teen$ or puber$ or pubescen$ or pre?pubescen$ or pupil? or student? or school or young$ or youth?).ti,ab. or (child$ or adolescen* or p?ediatr* or pe?diatr*).jw.
 4. 1 and 2 and 3
 5. 1 and 2
 6. limit 5 to ("all child (0 to 18 years)" or "preschool child (2 to 5 years)" or "child (6 to 12 years)" or "adolescent (13 to 18 years)")
 7. 4 or 6
 8. limit 7 to humans

PsycINFO via Ovid SP was searched (1806 to November Week 3 2013) using the following terms:

1. feedback/ or (((client$ or patient$) adj (response or respond or progress$)) or "client? guid$ assess$" or "client? satisfaction questionnaire?" or "clinical outcomes in routine evaluation" or "core‐om" or feedback$ or "feeding back" or feed‐back? or "fed back" or "goal? attainment scal$" or "helpful aspects of therapy" or "help$ alliance questionnaire?" or "idiographic assess$" or "important events questionnaire" or "monitor$ alliance" or "therapy alliance" or "treatment alliance" or "outcome? questionnaire$" or "outcome? monitor$" or "outcome? rating scal$" or patient‐focused or patient‐progress$ or "progress monitor$" or "progress track$" or "session? rating scal$" or "session? report$ form?" or "partner? for chang$" or "pcoms" or "treatment outcome? pack$" or "treatment progress$" or "treatment response" or "treatment respond").ti,ab.
 2. exp psychotherapy/ or exp counseling/ or (Psychotherap$ or counsel$ or ((cogniti$ or behavio?r$) adj (therap$ or treat$))).ti,ab.
 3. (adolescen$ or boys or boy or girl or girls or child$ or juvenile? or kid or kids or minor or minors or kindergar$ or parent? or mother? or father? or p?ediatric$ or pe?diatric$ or pre?school or school$ or high?school or student? or pre?teen$ or teen$ or mid?teen$ or late?teen$ or puber$ or pubescen$ or pre?pubescen$ or pupil? or student? or school or young$ or youth?).ti,ab. or (child$ or adolescen* or p?ediatr* or pe?diatr*).jw. or (childhood age birth 12 or preschool age 2 5 yrs or school age 6 12 yrs or adolescence 13 17 yrs).ag.
 4. 1 and 2 and 3
 5. limit 4 to human

PubMed (Excluding MEDLINE) was searched (26 November, 2013) using the following terms:

#1 "Feedback"[Mesh:NoExp] OR (((client*[tiab] OR patient*[tiab]) AND (response[tiab] OR respond[tiab] OR progress*[tiab])) OR "client satisfaction questionnaire"[tiab] OR "clinical outcomes in routine evaluation"[tiab] OR "core‐om"[tiab] OR feedback*[tiab] OR "feeding back"[tiab] OR feed‐back*[tiab] OR "fed back"[tiab] OR "goal attainment scaling"[tiab] OR "helpful aspects of therapy"[tiab] OR "helping alliance questionnaire"[tiab] OR "idiographic assessment"[tiab] OR "important events questionnaire"[tiab] OR "monitoring alliance"[tiab] OR "therapy alliance"[tiab] OR "treatment alliance"[tiab] OR "outcome questionnaire"[tiab] OR "outcome monitoring"[tiab] OR "outcome rating scale"[tiab] OR patient‐focused[tiab] OR patient‐progress*[tiab] OR "progress monitoring"[tiab] OR "progress tracking"[tiab] OR "session rating scale"[tiab] OR "session report form"[tiab] OR "partners for change"[tiab] OR "pcoms"[tiab] OR "treatment outcome package"[tiab] OR "treatment progress"[tiab] OR "treatment response"[tiab] OR "treatment respond"[tiab])

#2 "Psychotherapy"[Mesh] OR exp "Counseling"[Mesh:NoExp] OR (Psychotherap*[tiab] OR counsel*[tiab] OR ((cogniti*[tiab] OR behavior*[tiab] OR behaviour*[tiab]) AND (therap*[tiab] OR treat*[tiab])))

#3 (adolescen*[tiab] OR boys[tiab] OR boy[tiab] OR girl[tiab] OR girls[tiab] OR child*[tiab] OR juvenile*[tiab] OR kid[tiab] OR kids[tiab] OR minor[tiab] OR minors[tiab] OR kindergar*[tiab] OR parent*[tiab] OR mother*[tiab] OR father*[tiab] OR pediatric*[tiab] OR paediatric*[tiab] OR peadiatric*[tiab] OR preschool[tiab] OR pre‐school[tiab] OR school*[tiab] OR highschool[tiab] OR high‐school[tiab] OR student*[tiab] OR preteen*[tiab] OR pre‐teen*[tiab] OR teen*[tiab] OR midteen*[tiab] OR mid‐teen* OR lateteen*[tiab] OR late‐teen*[tiab] OR puber*[tiab] OR pubescen*[tiab] OR prepubescen*[tiab] OR pre‐pubescent*[tiab] OR pupil*[tiab] OR student*[tiab] OR school[tiab] OR young*[tiab] OR youth*[tiab] OR "Adolescent"[Mesh] OR "Child"[Mesh]

#4 #1 AND #2 AND #3

#5 Medline[sb]

#6 #4 NOT #5

Appendix 4. Search strategies (conducted by CCMD July 2015)

Search strategies: CENTRAL and PsycINFO (2015‐07‐07)

The Cochrane Central Register of Controlled Trials (CENTRAL) was searched (all years to 7‐July‐2015) using the following terms:
 #1 ((psychiatri* or psychotherapist* or therapist*) and ((client* or patient* or oupatient*) near (feedback or feed‐back))):ti,ab,kw
 #2 ((psychiatri* or psychotherapist* or therapist*) and (patient‐reported near (outcome* or progress))):ti,ab,kw
 #3 (psychotherapeutic outcome* and (feedback or feed‐back or (patient‐reported and (information or outcome* or progress)))):ti,ab,kw
 #4 ((physician or "primary care" or "general practi*" or clinician*) and ((client* or patient* or oupatient*) near (feedback or feed‐back or progress))):ti,ab,kw
 #5 MeSH descriptor: [Physician‐Patient Relations] this term only
 #6 MeSH descriptor: [Mental Disorders] explode all trees
 #7 MeSH descriptor: [Mental Health] explode all trees
 #8 MeSH descriptor: [Psychological Phenomena and Processes] explode all trees
 #9 ((#1 or #2 or #3) or ((#4 or #5) and (#6 or #7 or #8)))
 #10 (adolesc* or child* or paediatric* or pediatric*):so (Word variations have been searched)
 #11 CAMHS or child* or boy* or girl* or infant* or juvenil* or minors or paediatric* or pediatric* or school* or kindergarten or nursery or adolesc* or pubert* or pubescen* or pupil* or teen* or young or youth* or student* or undergrad* or graduate or graduates or university or college or campus:ti (Word variations have been searched)
 #12 (#10 or #11)
 #13 (#9 and #12)
 #14 ((feedback or feed‐back) near (CAMHS or child* or boy* or girl* or infant* or juvenil* or minors or paediatric* or pediatric* or school* or kindergarten or nursery or adolesc* or pubert* or pubescen* or pupil* or teen* or young or youth* or student* or undergrad* or graduate or graduates or university or college or campus))
 #15 (#14 and (#6 or #7 or #8))
 #16 (#13 or #15)

OVID PsycINFO was searched (2015‐07‐07) using the following terms:
 1. COUNSELING/
 2. PSYCHOTHERAPY/
 3. exp PSYCHOTHERAPEUTIC OUTCOMES/
 4. TREATMENT OUTCOMES/
 5. THERAPISTS/
 6. "3310".cc.
 7. or/1‐6
 8. FEEDBACK/
 9. (feedback or feed‐back).ti,id.
 10. or/8‐9
 11. 7 and 10
 12. ((physician* or psychiatri* or psychotherapist* or therapist* or primary care or general practi* or clinician*) and ((client* or patient* or oupatient*) adj5 (feedback or feed‐back))).ti,ab,id.
 13. ((physician* or psychiatri* or psychotherapist* or therapist* or primary care or general practi* or clinician*) and (patient reported adj3 (information or outcome*))).ti,ab,id.
 14. (psychotherapeutic outcome* and (feedback or feed‐back or (patient reported adj3 (information or outcome*)))).ti,ab,id.
 15. or/11‐14
 16. TREATMENT EFFECTIVENESS EVALUATION/
 17. CLINICAL TRIALS/
 18. MENTAL HEALTH PROGRAM EVALUATION/
 19. randomly.ab.
 20. randomi#ed.ti,ab,id.
 21. (control* adj3 (trial or study or group*1)).ti,ab,id.
 22. "2000".md.
 23. (waitlist* or (wait* and list* and (control* or group))).ti,ab,id.
 24. (treatment as usual or TAU or usual care or care as usual).ti,ab,id.
 25. or/16‐24
 26. 15 and 25
 27. (3 or 4) and 10
 28. 26 or 27
 29. (CAMHS or child* or boy* or girl* or infant* or juvenil* or minors or paediatric* or pediatric* or school* or kindergarten or nursery or adolesc* or pubert* or pubescen* or pupil* or teen* or young or youth* or student* or undergrad* or college or campus).ti,ab,id.
 30. (adolesc* or child* or paediatric* or pediatric*).jx.
 31. 29 or 30
 32. 28 and 31

Appendix 5. Search strategies (conducted by CCMD April 2018)

Database: Ovid MEDLINE(R) Epub Ahead of Print, In‐Process & Other Non‐Indexed Citations, Ovid MEDLINE(R) Daily and Ovid MEDLINE(R)
 Data Parameters: 1946 to Present (limits applied: 2016 onwards)
 Date Searched: April 3rd 2018
 Searched by: CCStrategy Developed by: SD
 Hits: 45
 Search Strategy:
 1. ((physician* or psychiatri* or psychotherapist* or psychotherapy or therapist* or primary care or general practi* or clinician*) and ((client* or patient* or oupatient*) adj5 (feedback or feed‐back))).ti,ab,kw.
 2. ((psychiatri* or psychotherapist* or psychotherapy or therapist*) and (patient‐reported adj5 (outcome* or progress))).ti,ab,kw.
 3. (psychotherapeutic outcome* and (feedback or feed‐back or (patient‐reported and (information or outcome* or progress)))).ti,ab,kw.
 4. ((physician or "primary care" or "general practi*" or clinician*) and ((client* or patient* or oupatient*) adj5 (feedback or feed‐back or progress))).ti,ab,kw.
 5. ((CORS or CSRS or OQ or PCOMS or SRS) and (feedback or feed‐back or fed back)).ti,ab,kw.
 6. (or/1‐5)
 7. (CAMHS or child* or boy* or girl* or infant* or juvenil* or minor* or paediatric* or pediatric* or school* or kindergarten or nursery or adolesc* or pubert* or pubescen* or pupil* or teen* or young or youth* or student* or undergrad* or graduate or graduates or college or campus or university).ti,ab,kw.
 8. (6 and 7)
 9. ((feedback or feed‐back) adj5 (CAMHS or child* or boy* or girl* or infant* or juvenil* or minors or paediatric* or pediatric* or school* or kindergarten or nursery or adolesc* or pubert* or pubescen* or pupil* or teen* or young or youth* or student* or undergrad* or graduate or graduates or university or college or campus)).ti,ab,kw.
 10. Physician‐Patient Relations/
 11. exp Mental Disorders/
 12. Mental Health/
 13. (or/10‐12)
 14. (9 and 13)
 15. (8 or 14)
 16. randomized controlled trial.pt.
 17. controlled clinical trial.pt.
 18. (randomized or randomised).ab.
 19. placebo.ab.
 20. clinical trials as topic.sh.
 21. randomly.ab.
 22. trial.ti.
 23. (or/16‐22)
 24. retracted publication.pt.
 25. "retraction of publication".pt.
 26. comment.pt.
 27. published erratum.pt.
 28. (retraction or retracted or withdrawn or errata or erratum or correction).ti.
 29. (or/24‐28)
 30. (15 and 23)
 31. (15 and 29)
 32. (30 or 31)
 33. (2016* or 2017* or 2018*).yr,dt,ed,ep.
 34. (32 and 33)
 
 Database: Embase
 Host: Ovid
 Data Parameters: 1974 to 2018 April 02 (limits applied: 2016 onwards)
 Date Searched: April 3rd 2018
 Searched by: CCStrategy Developed by: SD
 Hits: 83
 Search Strategy:
 1. ((physician* or psychiatri* or psychotherapist* or psychotherapy or therapist* or primary care or general practi* or clinician*) and ((client* or patient* or oupatient*) adj5 (feedback or feed‐back))).ti,ab,kw.
 2. ((psychiatri* or psychotherapist* or psychotherapy or therapist*) and (patient‐reported adj5 (outcome* or progress))).ti,ab,kw.
 3. (psychotherapeutic outcome* and (feedback or feed‐back or (patient‐reported and (information or outcome* or progress)))).ti,ab,kw.
 4. ((physician or "primary care" or "general practi*" or clinician*) and ((client* or patient* or oupatient*) adj5 (feedback or feed‐back or progress))).ti,ab,kw.
 5. ((CORS or CSRS or OQ or PCOMS or SRS) and (feedback or feed‐back or fed back)).ti,ab,kw.
 6. (or/1‐5)
 7. (CAMHS or child* or boy* or girl* or infant* or juvenil* or minor* or paediatric* or pediatric* or school* or kindergarten or nursery or adolesc* or pubert* or pubescen* or pupil* or teen* or young or youth* or student* or undergrad* or graduate or graduates or college or campus or university).ti,ab,kw.
 8. (6 and 7)
 9. ((feedback or feed‐back) adj5 (CAMHS or child* or boy* or girl* or infant* or juvenil* or minors or paediatric* or pediatric* or school* or kindergarten or nursery or adolesc* or pubert* or pubescen* or pupil* or teen* or young or youth* or student* or undergrad* or graduate or graduates or university or college or campus)).ti,ab,kw.
 10. doctor patient relation/
 11. exp mental disease/
 12. mental health/
 13. (or/10‐12)
 14. (9 and 13)
 15. (8 or 14)
 16. crossover‐procedure/ or double‐blind procedure/ or randomized controlled trial/ or single‐blind procedure/ or (random* or factorial* or crossover* or cross over* or placebo* or (doubl* adj blind*) or (singl* adj blind*) or assign* or allocat* or volunteer*).tw.
 17. (15 and 16)
 18. retracted article/
 19. erratum/
 20. (retraction or retracted or withdrawn or errata or erratum or correction).ti.
 21. (or/18‐20)
 22. (15 and 21)
 23. (17 or 22)
 24. (2016* or 2017* or 2018*).yr,dc.
 25. (23 and 24)

Database: PsycINFO
 Host: OVID
 Data Parameters: 1806 to March Week 4 2018 (limits applied: 2016 onwards)
 Date Searched: April 3rd 2018
 Searched by: CCStrategy Developed by: SD
 Hits: 52
 Search Strategy:
 1. ((physician* or psychiatri* or psychotherapist* or psychotherapy or therapist* or primary care or general practi* or clinician*) and ((client* or patient* or oupatient*) adj5 (feedback or feed‐back))).ti,ab.
 2. ((psychiatri* or psychotherapist* or psychotherapy or therapist*) and (patient‐reported adj5 (outcome* or progress))).ti,ab.
 3. (psychotherapeutic outcome* and (feedback or feed‐back or (patient‐reported and (information or outcome* or progress)))).ti,ab.
 4. ((physician or "primary care" or "general practi*" or clinician*) and ((client* or patient* or oupatient*) adj5 (feedback or feed‐back or progress))).ti,ab.
 5. ((CORS or CSRS or OQ or PCOMS or SRS) and (feedback or feed‐back or fed back)).ti,ab.
 6. (or/1‐5)
 7. (CAMHS or child* or boy* or girl* or infant* or juvenil* or minor* or paediatric* or pediatric* or school* or kindergarten or nursery or adolesc* or pubert* or pubescen* or pupil* or teen* or young or youth* or student* or undergrad* or graduate or graduates or college or campus or university).ti,ab.
 8. (6 and 7)
 9. exp Therapeutic Processes/
 10. COUNSELING/
 11. PSYCHOTHERAPY/
 12. exp PSYCHOTHERAPEUTIC OUTCOMES/
 13. TREATMENT OUTCOMES/
 14. THERAPISTS/
 15. (or/9‐14)
 16. FEEDBACK/
 17. ((feedback or feed‐back) adj5 (CAMHS or child* or boy* or girl* or infant* or juvenil* or minors or paediatric* or pediatric* or school* or kindergarten or nursery or adolesc* or pubert* or pubescen* or pupil* or teen* or young or youth* or student* or undergrad* or graduate or graduates or university or college or campus)).ti,ab.
 18. (15 and 17)
 19. (16 and 17)
 20. (8 or 18 or 19)
 21. clinical trials.sh.
 22. (randomi#ed or randomi#ation or randomi#ing).ti,ab,id.
 23. (RCT or at random or (random* adj3 (assign* or allocat* or control* or crossover or cross‐over or design* or divide* or division or number))).ti,ab,id.
 24. (control* and (trial or study or group) and (placebo or waitlist* or wait* list* or ((treatment or care) adj2 usual))).ti,ab,id,hw.
 25. ((single or double or triple or treble) adj2 (blind* or mask* or dummy)).ti,ab,id.
 26. trial.ti.
 27. placebo.ti,ab,id,hw.
 28. (or/21‐27)
 29. (20 and 28)
 30. (retraction or retracted or withdrawn or errata or erratum or correction).ti.
 31. (20 and 30)
 32. (29 or 31)
 33. (2016* or 2017* or 2018*).yr,dc,mo.
 34. (32 and 33)
 
 Database: CENTRAL
 Host: Wiley
 Data Parameters: Issue 4 of 12, April 2018
 Date Searched: April 3rd 2018
 Searched by: CCStrategy Developed by: SD
 Hits: 20
 Search Strategy:
 #1. ((psychiatri* or psychotherapist* or therapist*) and ((client* or patient* or oupatient*) near (feedback or feed‐back))):ti,ab,kw
 #2. ((psychiatri* or psychotherapist* or therapist*) and (patient‐reported near (outcome* or progress))):ti,ab,kw
 #3. (psychotherapeutic outcome* and (feedback or feed‐back or (patient‐reported and (information or outcome* or progress)))):ti,ab,kw
 #4. ((physician or "primary care" or "general practi*" or clinician*) and ((client* or patient* or oupatient*) near (feedback or feed‐back or progress))):ti,ab,kw
 #5. MeSH descriptor: [Physician‐Patient Relations] this term only
 #6. MeSH descriptor: [Mental Disorders] explode all trees
 #7. MeSH descriptor: [Mental Health] explode all trees
 #8. ((#1 or #2 or #3) or ((#4 or #5) and (#6 or #7)))
 #9. (adolesc* or child* or paediatric* or pediatric*):so
 #10. (CAMHS or child* or boy* or girl* or infant* or juvenil* or minors or paediatric* or pediatric* or school* or kindergarten or nursery or adolesc* or pubert* or pubescen* or pupil* or teen* or young or youth* or student* or undergrad* or graduate or graduates or university or college or campus):ti
 #11. (#9 or #10)
 #12. (#8 and #11)
 #13. ((feedback or feed‐back) near (CAMHS or child* or boy* or girl* or infant* or juvenil* or minors or paediatric* or pediatric* or school* or kindergarten or nursery or adolesc* or pubert* or pubescen* or pupil* or teen* or young or youth* or student* or undergrad* or graduate or graduates or university or college or campus))
 #14. (#13 and (#6 or #7))
 #15. (#12 or #14)

Data and analyses

Comparison 1. Client feedback versus no client feedback.

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Improvement, dichotomous: number of participants with clinical improvement	1	163	Risk Ratio (M‐H, Random, 95% CI)	0.91 [0.63, 1.30]
2 Improvement, various scales: short‐term (youth report)	3		Mean Difference (IV, Random, 95% CI)	Totals not selected
2.1 Youth report	3		Mean Difference (IV, Random, 95% CI)	0.0 [0.0, 0.0]
3 Improvement, various scales: short‐term (parent, therapist, teacher report)	2		Mean Difference (IV, Random, 95% CI)	Subtotals only
3.1 Parent report	1	118	Mean Difference (IV, Random, 95% CI)	‐1.94 [‐4.73, 0.85]
3.2 Therapist report	1	50	Mean Difference (IV, Random, 95% CI)	‐0.13 [‐0.55, 0.30]
3.3 Teacher report	1	50	Mean Difference (IV, Random, 95% CI)	‐0.16 [‐0.61, 0.30]
4 Improvement: Ohio Scales‐Problem Severity, medium‐term	1		Mean Difference (IV, Random, 95% CI)	Subtotals only
4.1 Youth report	1	29	Mean Difference (IV, Random, 95% CI)	‐0.25 [‐0.96, 0.46]
4.2 Therapist report	1	32	Mean Difference (IV, Random, 95% CI)	0.16 [‐0.31, 0.63]
4.3 Teacher report	1	25	Mean Difference (IV, Random, 95% CI)	0.20 [‐0.50, 0.90]
5 Treatment acceptability: dropouts from treatment	2	237	Risk Ratio (M‐H, Random, 95% CI)	1.08 [0.73, 1.61]
6 Treatment acceptability: planned sessions not held	2	257	Mean Difference (IV, Random, 95% CI)	‐0.13 [‐1.02, 0.76]
7 Therapeutic alliance: youth‐rated, short‐term	2		Mean Difference (IV, Random, 95% CI)	Totals not selected
7.1 Youth report	2		Mean Difference (IV, Random, 95% CI)	0.0 [0.0, 0.0]
8 Duration of treatment (weeks)	4	593	Mean Difference (IV, Random, 95% CI)	‐0.28 [‐1.57, 1.01]
9 Client satisfaction: Goal Attainment Scaling (GAS), parent report, medium‐term	1	72	Risk Ratio (M‐H, Random, 95% CI)	2.24 [1.31, 3.81]
10 Subgroup analysis: type of therapy ‐ improvement, various scales: short‐term (youth report)	3		Std. Mean Difference (IV, Random, 95% CI)	Subtotals only
10.1 Supportive‐expressive group counselling	1	191	Std. Mean Difference (IV, Random, 95% CI)	0.29 [0.01, 0.58]
10.2 No specific type	2	168	Std. Mean Difference (IV, Random, 95% CI)	‐0.69 [1.00, ‐0.37]
11 Subgroup analysis: number of sessions ‐ improvement: various scales (youth report), short‐term	2		Mean Difference (IV, Random, 95% CI)	Totals not selected
11.1 < 4 sessions	1		Mean Difference (IV, Random, 95% CI)	0.0 [0.0, 0.0]
11.2 4‐12 sessions	1		Mean Difference (IV, Random, 95% CI)	0.0 [0.0, 0.0]
12 Subgroup analysis: number of sessions ‐ treatment acceptability: dropouts from treatment	1		Risk Ratio (M‐H, Random, 95% CI)	Subtotals only
12.1 < 4 sessions	1	168	Risk Ratio (M‐H, Random, 95% CI)	1.08 [0.68, 1.72]
13 Subgroup analysis: number of sessions ‐ treatment acceptability: planned sessions not held	2	257	Mean Difference (IV, Random, 95% CI)	‐0.13 [‐1.02, 0.76]
13.1 > 12 sessions	1	141	Mean Difference (IV, Random, 95% CI)	‐0.47 [‐1.16, 0.22]
13.2 4‐12 sessions	1	116	Mean Difference (IV, Random, 95% CI)	0.48 [‐0.75, 1.71]
14 Subgroup analysis: feedback by client or parent ‐ treatment acceptability: dropouts from treatment	2	237	Risk Ratio (M‐H, Random, 95% CI)	1.08 [0.73, 1.61]
14.1 Client	2	237	Risk Ratio (M‐H, Random, 95% CI)	1.08 [0.73, 1.61]
14.2 Parent	0	0	Risk Ratio (M‐H, Random, 95% CI)	0.0 [0.0, 0.0]
15 Subgroup analysis: feedback by client or parent ‐ treatment acceptability: planned sessions not held	2	257	Mean Difference (IV, Random, 95% CI)	‐0.13 [‐1.02, 0.76]
15.1 Client	0	0	Mean Difference (IV, Random, 95% CI)	0.0 [0.0, 0.0]
15.2 Parent	2	257	Mean Difference (IV, Random, 95% CI)	‐0.13 [‐1.02, 0.76]
16 Subgroup analysis: frequency of feedback ‐ improvement: various scales (youth report), short‐term	3		Mean Difference (IV, Random, 95% CI)	Totals not selected
16.1 All sessions	1		Mean Difference (IV, Random, 95% CI)	0.0 [0.0, 0.0]
16.2 > 50% of sessions	2		Mean Difference (IV, Random, 95% CI)	0.0 [0.0, 0.0]
17 Subgroup analysis: frequency of feedback ‐ treatment acceptability: dropouts from treatment	2	237	Risk Ratio (M‐H, Random, 95% CI)	1.08 [0.73, 1.61]
17.1 All sessions	1	168	Risk Ratio (M‐H, Random, 95% CI)	1.08 [0.68, 1.72]
17.2 > 50% of sessions	1	69	Risk Ratio (M‐H, Random, 95% CI)	1.08 [0.50, 2.32]
18 Subgroup analysis: frequency of feedback ‐ treatment acceptability: planned sessions not held	2		Mean Difference (IV, Random, 95% CI)	Subtotals only
18.1 All sessions	2	257	Mean Difference (IV, Random, 95% CI)	‐0.13 [‐1.02, 0.76]

Characteristics of studies

Characteristics of included studies [ordered by study ID]

Bickman 2011‐cluster.

Methods	Cluster‐RCT
Participants	Youths 11‐18 years old entering home‐based services at 49 sites were randomised by site, 21 sites dropped out (11 feedback and 10 control), leaving 28 sites with 340 youths and 120 clinicians Setting: 28 sites in 10 states affiliated with a private, for‐profit, behavioural health organisation providing home‐based mental health treatment Country: USA Mental health problem: not specified. SFSS intake scores (on a scale from 1‐5 where 5 is most severe), youth‐rated (mean (SD)): feedback: 2.38 (0.72); control 2.34 (0.67) Gender: feedback 46% female; control 54% female Age: mean (SD): feedback 14.57 (1.80) years; control 15.03 (1.83) years Ethnicity: feedback: white 45%, American Indian or Alaska Native 1%, Asian 1%, black or African American 33%, more than one 14%, other 6%; control: white 68%, American Indian or Alaska Native 2%, Asian 0%, black or African American 17%, more than one 11%, other 2% Pharmacotherapy during the study: not reported
Interventions	Weekly feedback (plus cumulative feedback every 90 days) using Contextualized Feedback Intervention and Training, which is an earlier version of CFS. N = 173 Feedback every 90 days using Contextualized Feedback Intervention and Training, which is an earlier version of CFS. N = 167* Feedback measures: a psychometrically sound and clinically useful battery of very brief measures that promotes overall practice improvement through frequent and comprehensive assessments (PTPB, Bickman 2010) was completed at the close of a treatment session by the youth, caregiver, and clinician). Examples of feedback included mean scores and alerts if the youth’s symptoms ranked in the top 25th percentile in severity. Indicators of whether change from one measurement instance to the next met criteria for reliable change and trend graphs for change over multiple measurement points were also provided. Frequency of feedback: weekly versus once every 90 days Type of psychological therapy: specific type of treatment was not prescribed. Clinicians reported using various therapeutic approaches, including cognitive‐behavioural, integrative‐eclectic, behavioural, family systems, and play therapy. Number of treatment sessions: mean (SD) feedback: 10.68 (8.54); control: 11.36 (9.78) Treatment duration: mean (SD) feedback: 14.91 (11.23) weeks; control: 18.07 (15.53) weeks Therapists' training and experience: trained to the level of Master Arts (MA) or Master Science (MSc): 68%; Bachelor Arts (BA) or Bachelor Science (BSc) 22.5%
Outcomes	Symptom severity measured by SFSS (Athay 2012), completed by clinician, carer and youth** Duration of treatment (reported both in weeks and number of sessions held)
Notes	Funding sources: National Institute of Mental Health and the Leon Lowenstein Foundation Declarations of interest: Dr Bickman, Dr Kelley, Dr Breda, Dr Reimer, and Vanderbilt University have a financial interest in CFS. Dr de Andrade reports no competing interests. * Because youths remained in CFS about four months (mean = 3.8±3.1; median = 3.3), many would have been discharged before their first 90‐day report became available. The study authors considered the 90‐day group to be a no‐feedback control group. ** Unable to use data, repeated measures analysis results, see Table 3. We requested data from study investigators, but no further information was available at the time this review was prepared.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Quote: "randomly assigned by the research study’s data manager". No further details reported. We requested clarification from study investigators, but no further information was available at the time this review was prepared.
Allocation concealment (selection bias)	Unclear risk	Quote: "randomly assigned by the research study’s data manager". No further details reported. We requested clarification from study investigators, but no further information was available at the time this review was prepared.
Blinding of participants and personnel (performance bias) All outcomes	High risk	As the clinicians were aware of the frequency of feedback in the intervention and control group; blinding of clinicians could not have been achieved. There is no mention of blinding of participants.
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Quote: "The researchers did not collect or enter data and received a limited data set for analyses that included dates of treatment but no other personal identifiers."
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Significant attrition from the initial random assignments of sites could have biased the samples.
Selective reporting (reporting bias)	Low risk	Results were reported for all outcomes stated in methods section.
Other bias	Low risk	This was a cluster‐randomised trial and the study authors clearly reported intracluster correlation coefficient and adjusted for clustering in analyses.

Bickman 2015‐rural.

Methods	Parallel‐group RCT
Participants	141 youth new to services aged 11‐18 years, their carers and eight clinicians Setting: subjects were recruited from a rural outpatient mental health services clinic* Country: USA Mental health problem: not specified. SFSS intake scores (on a scale from 1‐5 where 5 is most severe), youth‐rated (mean (SD)): feedback: 2.42 (0.59); control 2.45 (0.70) Gender: feedback group 30 (43%) female, control group 31 (50%) female Age: mean (SD): feedback group 14.53 (2.30) years, control group 14.6 (2.19) years Ethnicity: feedback group Hispanic 6 (9%), African American 2 (3%), white 56 (82%), Asian 0 (0%), other ethnic 8 (12%); control group Hispanic 1(2%), African American 1 (2%), white 56 (93%), Asian 0 (0%), other ethnic 3 (5%) Pharmacotherapy during the study: Not reported
Interventions	High intensity client feedback: systematic client feedback (CFS) on treatment progress (e.g. symptoms and functioning) and process (e.g. therapeutic alliance) generated session‐by‐session for clinician review. N = 76 Very low intensity client feedback: client feedback from subjects (and caregivers) on progress in mental health symptoms and functioning generated every 6 months for clinician review. N = 65 Feedback measures: 11 measures from the PTPB (Bickman 2010), administered to clinicians, clients, and a caregiver (if present) within the session during the closing 5–10 min, or immediately after a session, including assessment of clinically‐relevant constructs such as symptom severity, therapeutic alliance, life satisfaction, motivation for treatment, hope, treatment expectations, caregiver strain, and service satisfaction Frequency of feedback: session‐by‐session versus once every 6 months Type of psychological therapy: evidence‐based mental health services. A third (33 %) of clinicians reported subscribing to a cognitive‐behavioural therapeutic orientation. However, most (48 %) reported some other unspecified orientation or that they had no particular therapeutic orientation. Number of treatment sessions: mean (SD): feedback group: 9.74 (7.45); control group 10.73 (8.24) Treatment duration: mean (SD): feedback group: 16.76 (14.1) weeks; control group 17 (13.05) weeks Therapists' training and experience: nearly all (90%) clinicians had attained a master’s (80%) or doctoral (10%) degree, typically in social work (47%). Most (47%) had less than 1 year of experience at their current work place. Relatively few clinicians had more than 5 years of experience anywhere — either in their current workplace (10%) or elsewhere (19%). About one‐fifth (19%) of the clinicians had no experience in providing services for children or youth either in their current work place or in any other place before they used CFS.
Outcomes	Symptom severity measured by SFSS (Athay 2012), completed by clinician, carer and youth** Treatment acceptability (planned sessions not held) Duration of treatment (reported both in weeks and number of sessions held)
Notes	Funding sources: Agency on Healthcare Research and Quality Declarations of interest: not reported * The Bickman 2015 study was carried out at two clinics, a rural and an urban; see Bickman 2015‐urban for the other site. ** Unable to use data, repeated measures analysis results; see Table 3. We requested data from study investigators, but no further information was available at the time this review was prepared.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Computer generated sequence. Quote: "Immediately after the initial intake appointment was completed and relevant intake measures were submitted in CFS, an automated process within the CFS application randomly assigned clients to a feedback or a control condition."
Allocation concealment (selection bias)	Low risk	Central allocation, see comment above
Blinding of participants and personnel (performance bias) All outcomes	High risk	As the clinicians were aware of the frequency of feedback in the intervention and control group; blinding of clinicians could not have been achieved. There is no mention of blinding of participants.
Blinding of outcome assessment (detection bias) All outcomes	Unclear risk	There is no mention of blinding of outcome assessors. We requested clarification from study investigators, but no further information was available at the time this review was prepared.
Incomplete outcome data (attrition bias) All outcomes	High risk	Only randomised participants that had reported on a symptom scale were included. Quote: "The sample used in these analyses consisted of youth [... that] had completed at least one SFSS measure." The study did not report initial numbers randomised.
Selective reporting (reporting bias)	Low risk	Results were reported for all outcomes stated in methods section.
Other bias	Low risk	The study seems to be free of other biases. Baseline characteristic of the two groups are similar.

Bickman 2015‐urban.

Methods	Parallel‐group RCT
Participants	116 youth new to services aged 11‐18 years, their carers and 13 clinicians Setting: subjects were recruited from a urban outpatient mental health services clinic* Country: USA Mental health problem: not specified. SFSS intake scores (on a scale from 1‐5 where 5 is most severe), youth‐rated (mean (SD)): feedback: 2.45 (0.71); control 2.30 (0.54) Gender: feedback group 32 (60%) female, control group 34 (59%) female Age: mean (SD): feedback group 14.0 (2.18) years, control group 14.3 (1.94) years Ethnicity: feedback group Hispanic 21(44%), African American 26 (54%), white 1 (2%), Asian 0 (0%), other ethnic 20 (42%); control group Hispanic 24 (43%), African American 28 (52%), white 2 (4%), Asian 1 (2%), other ethnic 23 (43%) Pharmacotherapy during the study: not reported
Interventions	High intensity client feedback: systematic client feedback (CFS) on treatment progress (e.g. symptoms and functioning) and process (e.g. therapeutic alliance) generated session‐by‐session for clinician review. N = 56 Very low intensity client feedback: client feedback from subjects (and caregivers) on progress in mental health symptoms and functioning generated every 6 months for clinician review. N = 60 Feedback measures: 11 measures from the PTPB (Bickman 2010), administered to clinicians, clients, and a caregiver (if present) within the session during the closing 5–10 min, or immediately after a session, including assessment of clinically‐relevant constructs such as symptom severity, therapeutic alliance, life satisfaction, motivation for treatment, hope, treatment expectations, caregiver strain, and service satisfaction Frequency of feedback: session‐by‐session versus once every 6 months Type of psychological therapy: Evidence‐based mental health services. One‐third (33 %) of clinicians reported subscribing to a cognitive‐behavioural therapeutic orientation. However, most (48 %) reported some other unspecified orientation or that they had no particular therapeutic orientation. Number of treatment sessions: mean (SD): feedback group: 12.96 (13.08); control group 10.39 (7.85) Treatment duration: mean (SD): feedback group: 19.82 (17.65) weeks; control group 17.23 (13.49) weeks Therapists' training and experience: nearly all (90%) clinicians had attained a master’s (80%) or doctoral (10%) degree, typically in social work (47%). Most (47%) had less than 1 year of experience at their current work place. Relatively few clinicians had more than 5 years of experience anywhere — either in their current workplace (10%) or elsewhere (19%). About one‐fifth (19 %) of the clinicians had no experience in providing services for children or youth either in their current work place or in any other place before they used CFS.
Outcomes	Symptom severity measured by SFSS (Athay 2012), completed by clinician, carer and youth** Treatment acceptability (planned sessions not held) Duration of treatment (reported both in weeks and number of sessions held)
Notes	Funding sources: Agency on Healthcare Research and Quality Declarations of interest: not reported * The Bickman 2015 study was carried out at two clinics, a rural and an urban; see Bickman 2015‐rural for the other site. ** Unable to use data, repeated measures analysis results; see Table 3. We requested data from study investigators, but no further information was available at the time this review was prepared.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Computer generated sequence. Quote: "Immediately after the initial intake appointment was completed and relevant intake measures were submitted in CFS, an automated process within the CFS application randomly assigned clients to a feedback or a control condition."
Allocation concealment (selection bias)	Low risk	Central allocation, see comment above
Blinding of participants and personnel (performance bias) All outcomes	High risk	As the clinicians were aware of the frequency of feedback in the intervention and control group; blinding of clinicians could not have been achieved. There is no mention of blinding of participants.
Blinding of outcome assessment (detection bias) All outcomes	Unclear risk	There is no mention of blinding of outcome assessors. We requested clarification from study investigators, but no further information was available at the time this review was prepared.
Incomplete outcome data (attrition bias) All outcomes	High risk	Only randomised participants that had reported on a symptom scale were included. Quote: "The sample used in these analyses consisted of youth [... that] had completed at least one SFSS measure." The study did not report initial numbers randomised.
Selective reporting (reporting bias)	Low risk	Results were reported for all outcomes stated in methods section.
Other bias	Low risk	The study seems to be free of other biases. Baseline characteristic of the two groups are similar.

Lester 2013.

Methods	Parallel‐group RCT
Participants	168 adolescents (ages 12‐17) and their parents gave consent for study participation and 120 guardians and adolescents completed all components of the study Setting: recruited from youth admitted for acute stabilisation at an inpatient psychiatric hospital (Methodist Behavioral Hospital) Country: USA Mental health problem: those who pose a serious and imminent risk of harm to self or others due to a psychiatric illness: depressive disorder, bipolar disorder, mood disorder Gender: feedback: 50% male, TAU: 46.67% male Age: mean (SD): feedback: 14.79 (1.47) years; TAU: 14.73 (1.62) years Ethnicity: feedback: white 50/58 (86%), African American 6/58 (10%), Biracial 2/58 (3%), Hispanic 0/58 (0%); TAU: white 43/60 (72%), African American 13/60 (22%), Biracial 2/60 (3%), Hispanic 2/60 (3%) Pharmacotherapy during the study: naturalistic prescribing
Interventions	Systematic client feedback: PCOMS during individual therapy sessions during inpatient psychiatric treatment. N = 60 TAU: individual therapy sessions with no feedback during inpatient psychiatric treatment: N = 60 Feedback measures ORS (Miller 2003) tracks the outcome of therapy from the perspective of the client, it provides immediate feedback to both the client and practitioner. The ORS is an ultra‐brief, easy to administer measure that assesses four dimensions of client functioning. Individual dimension measures personal well‐being or symptom distress. Interpersonal well‐being dimension measures how well the client is getting along within the family system. Social relationships dimension measures satisfaction with school and relationships outside of the house. Overall dimension is a subjective overall sense of well‐being. SRS (Duncan 2003b): tracks the TA with clients at every session. The entire SRS is based on encouraging clients to identify TA problems and to elicit client concerns about the therapeutic process so that the clinician may change to better fit client expectations. The SRS is a brief measure that encourages regular conversations with the client about the TA at the close of every session. The combination of the ORS and SRS were administered, scored, and discussed with the experimental group participants during individual therapy sessions. These participants were asked to complete the ORS at the beginning of each individual therapy session and the SRS at the end of each individual therapy session. The therapist scored the inventories immediately using a provided 10‐centimetre ruler and discussed the feedback on the ORS and SRS with the participants. Frequency of feedback: at each session Type of psychological therapy: adolescents admitted for acute care received routine interventions by a child psychiatrist and intensive co‐ordinated treatment by a physician‐led team of mental health professionals including psychologists, social workers, and nurses. The programme also provided weekly individual and family therapy, daily process groups, skill building activities, and educational instruction; these services were provided by child psychologists, social workers, certified special education teachers, case managers, and behavioural instructors. Number of treatment sessions: mean (SD): feedback: 1.72 (0.55); TAU: 1.91 (0.75) Treatment duration: mean (SD): feedback: 7.71 (2.20) days; TAU: 8.02 (3.02) days. Acute psychiatric hospital level of care typically lasted 3 to 14 days. Therapists' training and experience: the therapists were licensed Professional Counsellors, licensed Psychological Examiners, and a licensed Clinical Social Worker. Three of the therapists had four years of experience, while the other two each had five years of experience. Each therapist underwent training with the primary researcher. During training, the therapists conducted practice assessments and scoring with each other.
Outcomes	Improvement: ORS and Y‐OQ (Burlingame 1995) (adolescent and parent) at admission and discharge; TA; SRS (at discharge); length of stay; dropouts from treatment
Notes	Funding sources: not reported (PhD dissertation) Declarations of interest: not reported
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Quote: "The youth were randomly assigned to the treatment group (PCOMS) or control group (TAU)" Quote: "The therapists were provided with a list of predetermined assignments for participants to be in either the experimental or control group. The list was based on a randomized block design with each therapist and provided to each therapist at the beginning of the research study." No further details reported. We requested clarification from study investigators, but no further information was available at the time this review was prepared.
Allocation concealment (selection bias)	Unclear risk	Allocation concealment method not reported. We requested clarification from study investigators, but no further information was available at the time this review was prepared.
Blinding of participants and personnel (performance bias) All outcomes	High risk	Therapists could not have been blinded; they treated participants in both TAU and experimental group
Blinding of outcome assessment (detection bias) All outcomes	High risk	Outcome measures were self‐reported by youth and their parents/guardians.
Incomplete outcome data (attrition bias) All outcomes	High risk	168 gave consent, 48 dropped out (balanced between groups with reasons provided). Of the 120 cases with complete data, 2 were identified as outliers and excluded from further analyses (both in intervention group).
Selective reporting (reporting bias)	Low risk	Dissertation. All predefined outcomes have been reported.
Other bias	Unclear risk	Baseline characteristics only reported for the 120 participants that did not drop out. One difference between the groups was the diagnoses, specifically the number of depressed disorder and mood disorder diagnoses, which may have been a function of the different psychiatrists interacting with the clients and making the determination. Three of the participants had a length of stay that was greater than 15 days, and all three were assigned to the control group.

Melendez 2002.

Methods	Parallel‐group RCT
Participants	69 children and adolescents participated in the trial Setting: recruited from three therapeutic day schools from two Midwestern cities that serve youth with behavioural disorders Country: USA Mental health problem: mood disorder, PDD, CD, ODD, ADHD Gender: 40 (80%) male Age: Range: 12‐13 years, N = 21; 14‐17 years, N = 24; 18+ years, N = 5 Ethnicity: 11 (22%) black, 11 (22%) Hispanic, 16 (32%) white. 12 (24%) not reported Pharmacotherapy during the study: not reported
Interventions	High feedback (full feedback): weekly report to the therapist from youth, teachers, and the therapist. The report indicating, also, the quality of change in each reporting source's score from the initial session to the most recent session. N = 35 Low feedback (limited feedback). report with only therapist scores. N=34 Feedback measures: Ohio Scales‐Short version, Youth and Worker (teachers and therapists) forms, TA Scales for Children (youths and therapists) (every third visit) Frequency of feedback: weekly Type of psychological therapy: students receive weekly individual and family clinical services from a licensed mental health worker during the school day. Although the services were focused on crisis stabilisation, students could receive planned individual and group therapy as well Number of treatment sessions: not reported Treatment duration: 3 months Therapists' training and experience: all therapists possessed at least a Master’s degree in a field relevant to child and family counselling
Outcomes	Improvement: Ohio Scales‐Problem Severity (Ogles 2001), as reported by therapist, teacher and youth at week 6, 12 and at follow‐up (6 to 9 months; that is, 3 to 6 months after end of intervention); Treatment acceptability: dropouts from intervention Outcomes not added to this review: Total sample per group and SDs (or equivalent) not reported*: TA Scales for Children (Shirk 1992)
Notes	Funding sources: not reported (PhD dissertation) Declarations of interest: not reported We are grateful to the study author for replying to our request of additional data for the assessment of risk of bias. * At the time this review was prepared, we were still awaiting response from study author on outcome data.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Satisfactory generation of random sequence, randomised within therapist. Quote: "An equal number of strips of paper labeled High and Low that together equaled the number of subjects for that therapist were placed in an envelope and then selected one at a time until all subjects were assigned to a group"
Allocation concealment (selection bias)	Low risk	Allocation was concealed; drawn from an envelope, see comment above
Blinding of participants and personnel (performance bias) All outcomes	High risk	Blinding of therapists was not achieved, as communicated by G. Melendez, the study investigator, in an email to HK on 1 Dec 2015
Blinding of outcome assessment (detection bias) All outcomes	High risk	Blinding of outcome assessment was not achieved, as communicated by G. Melendez, the study investigator, in an email to HK on 1 Dec 2015
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Similar number of dropouts in both groups during the intervention (full feedback : 10/35; limited feedback: 9/34). Attrition rates at follow‐up were high and unequal between groups ‐ subjects were more likely to drop out from the limited feedback group
Selective reporting (reporting bias)	Low risk	All outcomes described in the methods of this dissertation were reported (ANOVA), although for two outcomes no means and/or SDs were reported.
Other bias	Low risk	The study seems to be free of other biases. Baseline characteristics of the two groups are similar

Ogles 2006.

Methods	Parallel‐group RCT
Participants	72 youths and their families participated in the trial Setting: recruited from parents and adolescents scheduled for a wraparound service family team meeting Country: USA Mental health problem: most youths had significant levels of problems as indicated by scores on the Ohio Scales‐Problem Severity and Functioning (Ogles 2001), Vanderbilt Functioning Index (Bickman 1998), and Target Complaints (Battle 1966). Scores indicated that the average youth was exhibiting problems within the clinical range and typical of youth receiving mental health services. Many of the youth had a history of previous hospitalisation, suspension in school, arrest, placement in a foster home, group home or other placement, or leaving home without permission over night. Scores also indicated that youth had a history of involvement in an average of 7 of 24 problem behaviours (e.g. fights, illegal behaviours) or critical events (e.g. self‐harm attempt)* Gender: 38 (53%) male Age: mean (SD) 13.31 (3.04) years Ethnicity: not reported Pharmacotherapy during the study: not reported
Interventions	Systematic client feedback to the team members (including parents) of wraparound services. N = 37 No feedback with wraparound services. N = 35 Feedback measures: measures not reported. A brief feedback report regarding outcome progress was distributed to the team including the parent(s) after each of 4 data collection points. Frequency of feedback: after 2 weeks, 4 weeks, 8 weeks, and 12 weeks Type of psychological therapy: family team meetings of wraparound services, which are family‐centred and child‐focused individualised packages of support and services for children and adolescents with serious emotional disturbances, and which are a popular alternative to out‐of‐home placement (in the USA). Although psychological therapy is likely to be included in wraparound services, there was no special mention, except that 11 of the youth had not participated in counselling in the year prior to the team meeting. Number of treatment sessions: not reported Treatment duration: 3 months of feedback within wraparound services Therapists' training and experience: not reported
Outcomes	Client satisfaction: Goal Attainment Scaling (Kiresuk 1994), parent report Outcomes not added to this review: No means and SDs (or equivalent) reported, and total number of participants analysed not reported**. Improvement levels at post‐treatment and 6‐month follow‐up as indicated by the Ohio Scales (Ogles 2001), Target Complaints (Battle 1966), and Family Adaptability and Cohesion Scale (FACES III) (Olson 1985)
Notes	Funding sources: Ohio Department of Mental Health Declarations of interest: not reported * The study authors reported that some of the youth with below‐clinical problem scores had been referred by the local child welfare agency as part of a reunification plan. In these circumstances, the wraparound team met at the end of the youth’s out‐of‐home placement in order to co‐ordinate services as the child returned home. Parent ratings of the youths at that point in time (after an extended period of time away from the child) were lower. And these youths also were rated as having increased problems and poorer functioning as the study continued. It should be noted that the problems increased on average only slightly and into the mild range. As a result, the study authors write that wraparound services may have served the purpose of easing the transition and preventing more serious increases in problems. ** We requested clarification from study investigators, but no further information was available at the time this review was prepared.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	The first two participants were not randomised. Quote: "The first two youth were deliberately assigned to the non‐feedback group in order to work out the details of the assessment process. From that point on, youth were assigned to the feedback condition based on a matched randomization procedure where an equal number of feedback and non‐feedback slips of paper were placed in a hat and selected one at a time to make the group assignment for families that enrolled in the study."
Allocation concealment (selection bias)	Low risk	Allocation of the first two participants was not concealed
Blinding of participants and personnel (performance bias) All outcomes	High risk	No blinding mentioned. Parents and personnel in the feedback condition could not have been blinded as they received a copy of the feedback report.
Blinding of outcome assessment (detection bias) All outcomes	Unclear risk	Blinding of outcome assessors not reported
Incomplete outcome data (attrition bias) All outcomes	High risk	Although study authors report that dropout rates at 3 months were not significantly different between the two groups, reasons for dropping out, number of dropouts per group, and number of participants analysed per group were not reported. In addition, the study reports that there were "relatively large number of dropouts between 3 and 9 months" with no further details reported on these dropouts.
Selective reporting (reporting bias)	High risk	Participants with initial problem scores below‐clinical levels were separated from those with above‐clinical levels in some of the analyses, in what appears to be a post hoc decision. Results were not reported for outcomes at all time points.
Other bias	Low risk	The study seems to be free of other biases. Baseline characteristics of the two groups were similar.

Shechtman 2016.

Methods	Parallel‐group RCT
Participants	220 children and adolescents (ages 10‐18) were randomised and 191 completed the study Setting: pupils in 25 schools had been referred to a school counsellor for internalising or externalising behaviour problems Country: Israel Mental health problem: ADD, ADHD, behaviour problem diagnosis, or undiagnosed problem behaviours, such as aggression, antisocial behaviour, loneliness, shyness or withdrawal Gender: not reported Age: feedback: 34/107 in elementary school (grades 4 to 6), 39/107 in junior high school (grades 7 to 9), 34/107 in high school (grades 10 to 12); no feedback: 36/84 in elementary school (grades 4 to 6), 36/84 in junior high school (grades 7 to 9), 34/84 in high school (grades 10 to 12) Ethnicity: Jewish
Interventions	Systematic client feedback: supportive‐expressive group counselling with counsellors receiving weekly feedback charts for each group participant. N = 105 No feedback: supportive‐expressive group counselling. N = 115 Feedback measures: Y‐OQ‐30 is a 30‐item instrument for the assessment of adolescents’ well‐being (Burlingame 2004). It contains six subscales: somatic symptoms, loneliness, behaviour problems, aggression, anxiety and depression. Frequency of feedback: participants completed the questionnaire at the end of each session Type of psychological therapy: the intervention was based on the supportive‐expressive group counselling modality (Shechtman 2007), which involves process‐oriented groups, in which children and adolescents are encouraged to express their concerns or difficulties, go through cognitive and affective exploration, develop insight into their behaviour, and make a commitment to change. The therapeutic process for each individual is based on the three‐stage model of change suggested by Hill (Hill 2005). The major therapeutic factors proposed by Yalom (Yalom 2005), such as interpersonal learning, catharsis, and group cohesion, were also present in these groups. Cousellors used structured activities, such as therapeutic games, bibliotherapy, phototherapy, art therapy, and therapeutic cards, to help the children and adolescents express their emotions and to aid in establishing a climate for effective group work. Number of treatment sessions: 12 (for both groups) Therapists' training and experience: counsellors were 25 female graduate students of a counselling programme. They had at least two years of practice at the school counselling centre and had received training in group counselling and the use of feedback information, as well as weekly supervision.
Outcomes	Well‐being: Y‐OQ‐30 (Burlingame 2004) weekly; therapeutic bonding: Working Alliance Inventory (WAI) at the end of treatment (Horvath 1989)
Notes	Funding sources: not reported Declarations of interest: not reported
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Sequence generation not described
Allocation concealment (selection bias)	Unclear risk	Allocation concealment not described
Blinding of participants and personnel (performance bias) All outcomes	Unclear risk	Therapists were aware of which intervention they gave, but this may not have represented any bias, as knowledge of the intervention was actually the experimental condition itself. p 346 quote: "...we have no information on how the counselors treated the feedback they received. Although they were trained to use the information provided to them we cannot guarantee that they gave it full attention". Participants may or may not have been aware of which intervention they received. Knowledge of which condition they were assigned to would probably have affected how they completed the questionnaires. p 340 quote: "All the pupils completed the OQ‐30 at the end of each session, but only the counselors in the feedback conditions received the participants’ scores before the next session".
Blinding of outcome assessment (detection bias) All outcomes	Unclear risk	p 340 quote: "The questionnaires were completed anonymously, but a personal code — the three last digits of the pupil’s telephone number — was used to permit comparison of scores for each child". Knowledge of which condition they were assigned to would probably have affected how they completed the questionnaires. p 340 quote: "All the pupils completed the OQ‐30 at the end of each session, but only the counselors in the feedback conditions received the participants’ scores before the next session".
Incomplete outcome data (attrition bias) All outcomes	High risk	p 337 quote: "...191 of them completed the questionnaires". p 342 quote: "In the first session 39 out of 93 children in the group with feedback... , compared with 21 out of 74 (28.4%) in group with no feedback. ... In the last session 30 out of 103 children in the group with feedback ..., compared with 15 out of 84 (17.9%) in group with no feedback". Apparently, outcome data are missing for 12/105 and 31/115 participants in the feedback and the no‐feedback group, respectively, in the first session. In the last session corresponding numbers were 2/105 and 84/115
Selective reporting (reporting bias)	Low risk	No information about publication of the protocol. Outcome reporting consistent with the Methods section of the paper
Other bias	Unclear risk	No information about how counsellors were allocated to the intervention and control conditions

ADHD: attention deficit hyperactivity disorder
 CD: conduct disorder
 CFS: Contextualized Feedback Systems
 N: number
 ODD: oppositional defiant disorder
 ORS: Outcome Rating Scale
 PCOMS: Partners for Change Outcome Management System
 PDD: pervasive developmental disorder
 PTPB: Peabody Treatment Progress Battery
 RCT: randomised controlled trial
 SD: standard deviation
 SFSS: Symptoms and Functioning Severity Scale
 SRS: Session Rating Scale
 TA: therapeutic alliance
 TAU: treatment as usual
 Y‐OQ: Youth Outcome Questionnaire

Characteristics of excluded studies [ordered by study ID]

Study	Reason for exclusion
Albert 2007	Wrong intervention: corrective versus positive feedback from therapist to client
Anderson 2011	Feedback in research, not therapy
Axford 2012	Not a RCT: book review
Barbera 1994	Not a RCT: no control group
Barry 1972	Not a RCT: theoretical
Bickman 2008	Not a RCT: essay
Borntrager 2015	Not a RCT: description of two programmes
De Jong 2014	Wrong population: age ≥ 17 years. Mean > 37 years
De Los Reyes 2010	No feedback
Emener 1984	Wrong population: adults
Falkenstrom 2013	Wrong population: adults
Fischer 2000	Wrong population: adults
Friedberg 2004	Not a RCT: review
Hagborg 1992	No systematic client feedback
Hansen 2015	Not a RCT
Harmon 2007	Wrong population: adults
Hoenders 2014	Wrong population: adults only; mean age 41.3 years
Holowaty 2012	Wrong population: adults
Jefferis 1999	No systematic client feedback
Kearns 2012	No therapy
Lambert 2001	Wrong population: adults
Lambert 2005	Wrong population: adults
McClintock 2017	Wrong population: adults
Mikeal 2016	Wrong population: adults
Moore 1996	No therapy
Moses 2011	No systematic client feedback
Rise 2012	Wrong population: adults aged > 21 years
Rise 2016	Wrong population: adults
Sanci 2015	Wrong intervention: client feedback minor component of complex intervention
Sripada 2011	Wrong population: adults age range 18‐65 years
Yeung 2012	Wrong population: adults aged ≥ 18 years

RCT: randomised controlled trial

Characteristics of ongoing studies [ordered by study ID]

NCT01873742.

Trial name or title	The STIC Online Feedback System in Psychotherapy Process‐outcome Research: A multi‐site, multi‐modality, international RCT‐study
Methods	Open‐label, parallel arm RCT
Participants	Individuals, couples, and families aged 12‐70 years Planned sample size = 200
Interventions	1. STIC feedback system 2. TAU
Outcomes	Beck Depression Inventory II
Starting date	June 2013 Estimate completion date: October 2016
Contact information	Terje Tilden, Modum Bad, Norway
Notes	Listed as completed (no listed publication or data posted on clinicaltrials.gov/ct2/show/NCT01873742, 22/05/2018) Location: Norway Funding: Modum Bad

NCT02023736.

Trial name or title	Assessing psychotherapy outcome in treatment as usual versus treatment as usual with the STIC feedback system
Methods	Open‐label, parallel arm RCT
Participants	Children, adolescents, and adults ≥ 12 years Planned sample size = 1000
Interventions	STIC: computerised measurement and feedback system consisting of weekly questionnaires, completed on the computer, that target symptoms and functioning in a variety of domains of a clients life, the feedback system consists of a web‐portal where therapists may access their clients' STIC responses, for the purposes of planning treatment, assessing progress, and discussing change with clients. + TAU TAU: psychotherapy treatment as planned and implemented by their psychotherapists, exact type of treatment varied by therapist and according to client need
Outcomes	Change in mental health symptoms at end of treatment, 6 months, and 12 months follow‐up
Starting date	December 2013 Estimate completion date: August 2017
Contact information	Contact: Jacob Z Goldsmith, 847 733 4300 ext 650, jgoldsmith2@family‐institute.org, Illinois USA
Notes	Currently ongoing study (recruiting, clinicaltrials.gov/ct2/show/NCT02023736, 22/05/2018) Funding: The Family Institute at Northwestern University Location: Chicago, USA

NCT02567266.

Trial name or title	Community study of outcome monitoring for emotional disorders in teens
Methods	Triple‐blind, parallel RCT
Participants	Children, adolescents 12 to 18 years Planned sample size = 222
Interventions	Unified Protocol for Adolescents: emotion‐focused, transdiagnostic approach for adolescents with a primary emotional disorder + Youth Outcomes Questionnaire: parent‐ and youth‐report measures of symptoms and alliance administered weekly on a tablet computer. The YOQ online system then generates reports to provide clinicians with systematic feedback about client progress. TAU plus: clinicians assigned to the TAU condition will be instructed to use whatever treatment methods and outcome monitoring strategies they typically use with adolescents with internalising disorders + YOQ: parent‐ and youth‐report measures of symptoms and alliance administered weekly on a tablet computer. The YOQ online system then generates reports to provide clinicians with systematic feedback about client progress. TAU: clinicians assigned to the TAU condition will be instructed to use whatever treatment methods and outcome monitoring strategies they typically use with adolescents with internalising disorders
Outcomes	Change over time in CGI‐I, CGI‐S, and in CGAS
Starting date	January 2016 Estimated completion date: July 2019
Contact information	Amanda Jensen‐Doss (305‐284‐8332, ajensendoss@miami.edu); Jill Ehrenreich‐May ((305) 284‐6476, j.ehrenreich@miami.edu)
Notes	Currently ongoing study (recruiting, clinicaltrials.gov/ct2/show/NCT02567266, 22/05/2018) Location: Miami and Connecticut, USA Funding: University of Miami

van Sonsbeek 2014.

Trial name or title	The effective components of feedback from Routine Outcome Monitoring (ROM) in youth mental health care
Methods	Three‐arm parallel‐group RCT
Participants	Children and adolescents between 4 and 17 years Planned sample size = 432 Participants recruited from referrals to all outpatient youth departments (in Arnhem, Ede, Nijmegen and Tiel) of a large mental health care institution in the eastern part of the Netherlands. Children and adolescents with all kinds of mental health problems (e.g. developmental disorders, anxiety disorders and mood disorders) and all kinds of treatment (e.g. individual treatment and group treatment, cognitive‐behavioural treatment and solution‐focused treatment, frequent or irregular treatment) will be included in the study
Interventions	Feedback as usual/control: clinicians receive basic feedback regarding symptoms and quality of life of the client The feedback of condition 1 is extended with feedback about the results of the additional questionnaire and with practical suggestions to improve treatment (youth clinical support tools) The feedback of condition 2 is discussed with colleagues on the basis of a standardised format for case consultation
Outcomes	Symptom severity: measured with the Dutch version of the SDQ Quality of life: measured with the KIDSCREEN‐27 parent version and the KIDSCREEN‐52 child‐adolescent version Satisfaction with treatment: measured with the Jeugdthermometer child version and the Jeugdthermometer parent versions about the treatment of the child and about the parenting skills training Number of sessions: will be counted for each client Length of treatment: will be registered in days for each client Dropout: will be calculated as the percentage of clients that abandon treatment (registered as unilateral decision to end treatment) in each feedback group
Starting date	Registered 4 Nov 2013
Contact information	m.van.sonsbeek@propersona.nl
Notes	Funding source: ZON‐MW, the Netherlands Organization for Health Research and Development Declarations of interest: the authors declare that they have no competing interests Dutch Trial Register NTR4234

CGAS: Children's Global Assessment Scale
 CGI‐I: Clinical Global Impression‐Improvement
 CGI‐S: Clinical Global Impression‐Severity
 RCT: randomised controlled trial
 ROM: routine outcome monitoring
 SDQ: Strengths and Difficulties Questionnaire
 STIC: Systemic Therapy Inventory of Change
 TAU: treatment as usual
 Y‐OQ: Youth Outcomes Questionnaire

Differences between protocol and review

In the Measures of treatment effect section, the published protocol states that we will extract change from baseline data, whereas the Data extraction and management section states that we will extract endpoint data. We extracted endpoint data as they are easier to interpret from a clinical point of view.

The protocol did not state how we would deal with skewed data. Skewed data were log transformed to enable meta‐analysis (Higgins 2011), this has now been included in the Methods section.

Please also see Appendix 1 for methods from the protocol that were not used in this review.

Contributions of authors

Hanna Bergman screened the literature, extracted data, assimilated and interpreted data, carried out GRADE and summary of findings, and wrote the report.

Hege Kornør screened the literature, cross‐checked data extraction and GRADE assessments, assessed risk of bias, and contributed in writing the report.

Adriani Nikolakopoulou analysed data and provided statistical support.

Ketil Hanssen‐Bauer screened the literature and commented on report drafts

Karla Soares‐Weiser wrote the protocol, screened literature and commented on report drafts.

Thomas K Tollefsen screened the literature and commented on report drafts

Arild Bjørndal had the idea for the review, recruited the team, provided funding for the review, screened the literature and commented on every draft.

Sources of support

Internal sources

• Center for Child and Adolescent Mental Health, Eastern and Southern Norway, Norway.

  Staff time and other resources for the review.

External sources

• No sources of support supplied

Declarations of interest

Hanna Bergman worked for Enhance Reviews Ltd. during preparation of this review and was paid for her contribution to this review. Enhance Reviews Ltd. is a private company that performs systematic reviews of literature. Hanna works for Cochrane Response, an evidence consultancy operated by Cochrane.

Hege Kornør is an Associate Editor for the Cochrane Developmental, Psychosocial and Learning Problems Group (CDPLPG).

Adriani Nikolakopoulou has no known conflicts of interest.

Ketil Hanssen‐Bauer has no known conflicts of interest.

Karla Soares‐Weiser was the director of Enhance Reviews Ltd. during preparation of this review. Enhance Reviews Ltd. is a private company that performs systematic reviews of literature. Karla is employed by Cochrane as Deputy Editor in Chief.

Thomas K Tollefsen has no known conflicts of interest.

Arild Bjørndal is CEO of an organisation that teaches one of the methods to practitioners.

New