Interrelations between participant and intervention characteristics, process variables and outcomes in online interventions: A protocol for overarching analyses within and across seven clinical trials in ICare

Abstract

Background

It is well known that web-based interventions can be effective treatments for various conditions. Less is known about predictors, moderators, and mediators of outcome and especially interrelations between participant and interventions characteristics, process variables and outcomes in online interventions. Clinical trials often lack statistical power to detect variables that affect intervention effects and their interrelations. Within ICare, we can investigate the interrelation of potential predictor and process variables in a large sample.

Method

The ICare consortium postulated a model of interrelations between participant and intervention characteristics, process variables and outcomes in online interventions. We will assess general and disorder-specific interrelations between characteristics of the intervention, characteristics of the participants, adherence, working alliance, early response, and intervention outcomes in a sample of over 7500 participants from seven clinical trials evaluating 15 online interventions addressing a range of mental health conditions and disorders, using an individual participant data meta-analyses approach.

Discussion/conclusion

Existing research tends to support the efficacy of online mental health interventions, but the knowledge base regarding factors that affect intervention effects needs to be expanded. The overarching analyses using data from the ICare intervention trials will add considerably to the evidence.

Highlights

• •We will investigate interrelations between participant and interventions characteristics, process variables and outcomes.
• •Common measures for these variables will be used in all ICare trials.
• •We aim to use data from 7 clinical trials in a pre-planned individual client data meta-analyses approach.

1. Introduction

It is well known that web-based interventions can be effective treatments for various conditions (Andersson, 2016). Less is known about predictors, moderators, and mediators of outcome and especially about interrelations between participant and interventions characteristics, process variables and outcomes in online interventions. One of the aims of the ICare consortium is to assess these interactions both within and across a range of clinical trials evaluating Internet-interventions for the prevention and treatment of various mental problems and disorders.

Within ICare, we developed 15 different online interventions promoting resilience or addressing depression, anxiety or disordered eating using the same technological platform (see Table 1). These interventions are comparable regarding their overall structure: they consist of multiple consecutive sessions that can be augmented by diaries, symptom monitoring, and prompts, they are similar regarding usability aspects and their look and feel. Also, for the clinical trials we are conducting to evaluate these interventions, we agreed on common measures for key mental health outcomes, as well as potential moderators and mediators of treatment effects. This will enable us to pool data from multiple studies to address a number of overarching research questions that are usually difficult to answer due to lack of statistical power or different measures for the same construct used by individual studies.

Table 1.

Overview of included clinical studies.

	Study registration	Target population	Targeted disorder/domain	Interventions	Planned sample size	Participant allocation	Primary outcome
Study 1: everyBody plus self-help (Vollert et al., 2018)	ISRCTN12608780	Women with bulimia nervosa, binge eating disorder or OSFED waiting for outpatient treatment	•Eating disorders	•everyBody plus + TAU •TAU only	275	Randomization	Time to clinically relevant improvement of eating disorder symptoms
Study 2: everyBody (Nacke et al., 2018)	ISRCTN13716228	Women in the general population	•Disordered eating	•everyBody basic •everyBody original •everyBody plus •everyBody AN •everyBody fit	4160	Based on screening data	WCS total score
Study 3: CORE (Herrero et al., 2018)	ISRCTN13856522	University students	•Resilience	•CORE •Control group	464	Randomization	RS-14 total score
Study 4: ICare-Prevent (Weisel et al., 2018)	DRKS00011099	University students	•Depression, anxiety	•ICare prevent unguided •ICare prevent guided •Control group	957	Randomization	Disorder specific symptom severity (HAM-D/HAM-A)
Study 5: Healthy Teens @ School (Jones Bell et al., 2018)	ISRCTN51957280	High School students	•Disordered eating, overweight, healthy lifestyle	•Healthy teens @ school (weight management track) •Healthy teens @ school (healthy habits track) •Control group	430	Cluster-randomization (active intervention vs. control group) Based on screening date (type of active intervention)	Intuitive eating scale (adolescents) total score
Study 6: PLUS (Musiat et al., 2018)	ISRCTN15570935	University students	•Depression, anxiety	•PLUS •Control group	1110	Randomization	PHQ 9 and GAD 7 score
Study 7: WE CAN (Spencer et al., 2018)	ISRCTN11399850	Carers of individuals with anorexia nervosa	•Depression, anxiety	•WE CAN unguided •WE CAN peer support •WE CAN professional support	242	Randomization	PHQ 9 score

In order to do this, we devised a model of interrelations between participant and intervention characteristics, process variables and outcomes in online interventions (see Fig. 1). In this model, which is partly based on previous findings on the interrelations between these variables, we assume that 1) adherence affects outcome; 2) intervention and participant characteristics affect adherence; 3) intervention and participant characteristics affect outcome; 4) intervention and participant characteristics affect working alliance; 5) intervention and participant characteristics affect early response; 6) working alliance and adherence are related; 7) early response and adherence are related; 8) working alliance affects outcome and 9) early response affects outcome.

Fig. 1.

A Model of interrelations between participant and intervention characteristics, process variables and outcomes in online interventions.

In what follows, we will briefly summarize previous findings on these interrelations from research on online interventions.

1.1. How do participants adhere to online interventions? How do they perceive working alliance?

Adherence has been defined as “the extent to which individuals experience the content of the […] intervention” (Christensen et al., 2009, p. 2). Adherence and treatment dropout have become a focus of research on online interventions in recent years, yet it is still often poorly described and the measures and terms used are inconsistent (e.g., time spent on the intervention, number of logins, sessions/modules attempted, sessions/modules completed, etc.), thus making comparisons between interventions and setting difficult (Brouwer et al., 2011; Melville et al., 2010; Sieverink et al., 2017). Accordingly, our first aim is to describe adherence within and across interventions evaluated by the ICare consortium, using consistent and comparable measures.

1.2. How do participants perceive working alliance?

Working alliance has been investigated in a number of studies on different forms of online interventions and in various conditions (Anderson et al., 2012; Bergman Nordgren et al., 2013; Cook and Doyle, 2002; Hadjistavropoulos et al., 2017; Hanley, 2009; Herbst et al., 2016; Jasper et al., 2014; Knaevelsrud and Maercker, 2006; Kraemer et al., 2001; Preschl et al., 2011). Results show that independent of the treatment format and diagnostic groups, participant-rated alliance scores were high, roughly equivalent to alliance ratings found in studies on face-to-face therapy (Berger, 2017). We aim to expand this knowledge base by examining working alliance within and across interventions evaluated by the ICare consortium, including prevention programs and unguided interventions. In the latter, we will focus on aspects of working alliance regarding agreement on therapeutic goals and tasks between participants and the self-help programs.

1.3. Does adherence predict outcome?

The relationship between intervention adherence and outcomes has been investigated in a number of studies. Their findings indicate that better adherence is linked to better outcomes (Carrard et al., 2011; Christensen et al., 2002; Cobb et al., 2005; Couper et al., 2010; Cugelman et al., 2011; El Alaoui et al., 2016; Manwaring et al., 2008; Richardson et al., 2013; Troop et al., 1996). However, knowledge about the sufficient or optimal intervention dose is scarce. Previous research has shown that the adherence measure that is chosen plays an important role and that some measures of adherence are related to outcomes while others are not (Donkin et al., 2013). Hence, using multiple measures of adherence is crucial to enable the identification of patterns of use and their interrelations with outcomes. Within the ICare consortium, we will investigate how different aspects of adherence and use patterns are related to outcomes within and across interventions.

1.4. Do intervention and participant characteristics predict adherence?

Identifying variables that predict adherence is important in order to identify engaging features of interventions, to improve the content and design of our interventions and to identify participants who need a higher level of support to enable them to adhere to an intervention sufficiently. Intervention characteristics that have previously been linked to adherence include treatment credibility (Alfonsson et al., 2017; Alfonsson et al., 2016; El Alaoui et al., 2015b; Melville et al., 2010), the provision of guidance (Brouwer et al., 2011; Geraghty et al., 2010; Robinson et al., 2010; Wangberg et al., 2008; Zarski et al., 2016), peer support (Graham et al., 2017), regular updates of the intervention (Brouwer et al., 2011), the duration of the intervention (Cugelman et al., 2011), tailoring (Couper et al., 2010), the use of persuasive system design elements (Kelders et al., 2012), and periodic prompts (Beintner and Jacobi, 2017; Fry and Neff, 2009).

Participant characteristics that have previously been linked to adherence include baseline symptom severity (Christensen et al., 2009; Melville et al., 2010), age (Buller et al., 2008; Christensen et al., 2009; Melville et al., 2010; Wangberg et al., 2008), gender (Christensen et al., 2009; El Alaoui et al., 2015b; Kelders et al., 2013; Melville et al., 2010; Wangberg et al., 2008), marital status (Christensen et al., 2009; Melville et al., 2010), level of education (Alfonsson et al., 2016; Melville et al., 2010; Wangberg et al., 2008), outcome expectancy (Crutzen et al., 2011; Geraghty et al., 2010), locus of control (Geraghty et al., 2010), intrinsic motivation (Alfonsson et al., 2017; Alfonsson et al., 2016), and amount of internet use (Kelders et al., 2013). We aim to investigate the impact of a variety of interventions and participants characteristics on multiple aspects of adherence within and across interventions.

1.5. Do intervention and participant characteristics predict outcome?

Identifying variables that predict outcome is important in order to identify effective features of interventions, to improve the content and design of our interventions and to identify participants who are likely to benefit from an intervention and those who are not. Intervention characteristics that have previously been linked to outcomes include a strong theoretical foundation (Webb et al., 2010), the incorporation of behaviour change techniques (Webb et al., 2010), treatment credibility (Alfonsson et al., 2016; El Alaoui et al., 2016), and the provision of guidance (e.g., Berger et al., 2011; Furmark et al., 2009; Murray et al., 2007; Sanchez-Ortiz et al., 2011; Titov et al., 2008). Participant characteristics that have previously been linked to outcomes include age (Anderson et al., 2012), gender (Spek et al., 2008), level of education (Ebert et al., 2013), marital status (Button et al., 2012), treatment and outcome expectations (Bergman Nordgren et al., 2013; Ebert et al., 2013; Lutz et al., 2017), baseline symptoms and severity (Button et al., 2012; Ebert et al., 2013; El Alaoui et al., 2015a; El Alaoui et al., 2016; Spek et al., 2008), mental health self-efficacy (Clarke et al., 2014). We aim to investigate the impact of a variety of interventions and participants characteristics on key mental health outcomes within and across interventions.

1.6. Do intervention and participant characteristics predict working alliance?

Factors that influence working alliance have rarely been investigated in online interventions. While the frequency of therapist feedback and therapist qualification impacted working alliance (Hadjistavropoulos et al., 2017), baseline symptom severity did not (Herbst et al., 2016). We aim to expand the knowledge base by examining how intervention and participant characteristics affect working alliance within and across interventions.

1.7. Do intervention and participant characteristics predict early response?

Factors that affect early response have hardly been investigated in online interventions. In one study, higher treatment credibility was associated with a faster rate of improvement, while higher overall functioning at baseline was related to a slower rate of improvement (El Alaoui et al., 2015b). In the other study, participants who were early responders tended to have lower baseline physical and mental impairment (Lutz et al., 2017). We aim to expand the knowledge base by examining how intervention and participant characteristics affect early response within and across interventions.

1.8. Are working alliance and adherence related?

To our knowledge, interrelations of working alliance and adherence to online interventions have been investigated only in one study which showed a significant relation between working alliance and adherence (Anderson et al., 2012). We aim to expand the knowledge base by examining how working alliance and adherence are interrelated.

1.9. Are early response and adherence related?

It is likely that there is a bidirectional interrelation between early response to an intervention and adherence: Participants who adhere to the intervention may be more likely to experience improvement early on. Participants who experience improvement early during the intervention may either be more motivated go on with the treatment to achieve further improvement, or they may be more likely to drop out when they are satisfied with their gains. So far, the interrelation between early response and adherence in online interventions has hardly been investigated. In the two studies that addressed this, higher adherence was associated with faster symptom improvement (El Alaoui et al., 2015b) and early response (Lutz et al., 2017), but we can only speculate on how early responders adhere to the intervention once they have improved. In order to expand our knowledge on the interrelation between early response and adherence, we will examine whether adherence early during the intervention predicts early response and whether early response affects adherence later during the intervention.

1.10. Does working alliance predict outcome?

The impact of working alliance on outcomes of online interventions has been investigated in a number of studies (Berger, 2017; Sucala et al., 2012). Overall, when the association between alliance scores and outcome was reported, correlations were in a positive direction but not always statistically significant (Alfonsson et al., 2016; Anderson et al., 2012; Bergman Nordgren et al., 2013; Hadjistavropoulos et al., 2017; Herbst et al., 2016; Jasper et al., 2014; Knaevelsrud and Maercker, 2006; Preschl et al., 2011). We aim to expand the knowledge base by examining how working alliance affects outcome in a range of online interventions including prevention.

1.11. Does early response predict outcome?

Early response has been shown to be a strong predictor of treatment outcome in psychotherapy in several studies (e.g., Delgadillo et al., 2014; Hilbert et al., 2015; Raykos et al., 2013), yet has hardly been investigated in online interventions. The two studies who addressed the relationship between early response and outcome in internet-based cognitive behaviour therapy revealed that early response predicted outcome (Lutz et al., 2017; Schibbye et al., 2014) and that change at four weeks into treatment seems to be the best predictor of end-of-treatment outcome compared with change at other assessment points during treatment (Schibbye et al., 2014). Little is known about the role of early response in prevention interventions. We aim to expand the knowledge base by examining how early response affects key mental health outcomes in a range of online interventions including prevention.

2. Method

Within each clinical study, data collected at baseline, at mid-intervention, at post intervention and at 6- and 12-month follow-up, as well as automatically recorded log-data of intervention participants will be used to answer the questions outline above that will help us understand the complex interactions of participant and intervention characteristics, working alliance, adherence, early response, and outcome as illustrated in Fig. 1.

Analyses will be performed as appropriate separately for each active intervention condition, for each clinical study including all active intervention and control conditions of the clinical study, and across active intervention conditions, using individual participant data meta-analysis techniques. Analysis strategies are described in detail in Section 2.3 of this protocol.

2.1. Analysis populations

Analyses will be performed using the full analysis sets for each study, which contains all participants who were included into the clinical study unless specified otherwise in Section 2.3.

Before the analyses of the data can be performed, data from each clinical study will be reviewed during a blind data review step. Blinding refers to the omission of the allocated randomization groups. The blinded data review will be performed by DG, who will make sure that blinding is adhered to. To maintain blinding, all information specific for the randomized groups (e.g. about number of sessions/modules which does not exist in the waiting list group) has to be removed from the dataset before the review. Within the blind data review a suitable strategy for imputation of missing data for each clinical study will be determined.

2.2. Measures

The ICare investigators have agreed to employ a number of common measures of common mental health problems across the clinical studies to be included in the analyses as candidate outcome and predictor/moderator variables.

Table 2 summarizes the measurement times of assessed common measures within each clinical study.

Table 2.

Common baseline and outcome measures across clinical studies.

Measure	Screening (T0)/baseline (T1)	4 weeks after start of intervention Tmid	Post-intervention (T2)	6 month follow up (T3)	12 month follow up (T4)
Level of education	1,2,3,4,5,6,7
Gender	1,2,3,4,5,6,7
Household size	1,2,3,4,5,6,7
Household income	1,2,3,4,6,7
Year of birth	1,2,3,4,5,6,7
Age	1,2,3,4,5,6,7
Marital status	1,2,3,4,5,6,7
Employment status	1,2,3,4,5,6,7
Total population of place of residence	1,2,3,4,5,6,7
Any diagnosed mental disorder	1,2,3,5,6
Type of diagnosed mental disorder	1,2,3,5,6
Any prior psychotherapy	1,2,3,5,6
Helpfulness of prior psychotherapy	1,2,3,5,6
BFI-10	1,2,3,4,6,7
PHQ-9	1,2,3,4,6,7	1,2,3,4,6,7	1,2,3,4,6,7	1,2,3,4,6,7	1,2,3,4,6,7
GAD-7	1,2,3,4,6,7	1,2,3,4,6,7	1,2,3,4,6,7	1,2,3,4,6,7	1,2,3,4,6,7
AUDIT-C	1,2,3,4,5,6a,7	1,2,3,4,5,6,7	1,2,3,4,5,6,7	1,2,4,5,6,7	1,2,4,5,6,7
EDE-Q	1,2,5	1,2	1,2,5	1,2,5	1,2,5
CEQ	1,2,3,4,5,6,7	1,2,7
RSES	1,2,3,4,5,6,7	1,2,3,4,5,6,7	1,2,3,5,6	1,2,5,6	1,2,5,6
CD-RISC-10	1,2,3b,4,6,7	4	3b,4,6,7	3b,4,6,7	3b,4,6,7
SSRQ	1,2,4
Adapted WAI-SR		1,2,3,4,5,6,7	3

BFI10: 10-Item Big Five Inventory; PHQ-9: Patient Health Questionnaire depression module; GAD-7: 7-item Generalized Anxiety Disorder Scale; AUDIT-C: Alcohol Use Disorders Identification Test; EDE-Q: Eating Disorder Examination Questionnaire; CEQ: credibility/expectancy questionnaire; RSES: Rosenberg Self-Esteem Scale; CD-RISC-10: Connor-Davidson Resilience Scale; SSRQ: Short Self-Regulation Questionnaire; WAI-SR: Working Alliance Inventory. See below for details on measures.

^aClinical study 6 uses the 10-items version of the AUDIT, which contains all items of AUDIT-C.

^bClinical study 3 uses CD-RISC-25, which contains all items of CD-RISC-10.

2.2.1. Process variables

2.2.1.1. Adherence

In order to examine interrelations between adherence, working alliance, and early response, a number of different adherence measures (see Table 3 and Table 4) will be calculated for the whole intervention period and also separately for each of the two intervention phases before (phase one) and after (phase two) the mid-intervention assessment. In addition, for each clinical study, principal investigators will provide a list of all the actions they undertook to facilitate adherence, e.g., email or telephone prompts, and push notifications.

Table 3.

Universal measures (for all clinical studies).

Measure	Definition
Study dropout:	Study dropout is defined as binary variable (0/1) per participant. A participant has dropped from the study if the primary endpoint is not provided.
Time from baseline assessment to first intervention use:	The time to first login is defined as minutes between completion of the baseline questionnaire as documented in the database and the first session is opened as documented by the server logs.
Overall participation:	The proportion of complete assignments is defined as relative frequency (ratio) between completed assignments and all assignments in the intervention. Assignments are defined for each intervention in (see Appendix, “Description of the Interventions”). •Reading assignments are considered completed when the participant has opened the respective pages. •Surveys and diaries are considered completed when an entry has been made.
Proportion of completed assignments per session:	In each study the intervention is structured into x sessions. The proportion of complete assignment per session is defined as relative frequency (ratio) between completed assignments and all assignments in a session.
Number of opened sessions:	A session is considered opened by a participant when at least one assignment within this session has been completed.
Last opened session before post intervention assessment:	The last opened session before post intervention assessment will be determined from the data to measure intervention (non-usage attrition) dropout. A session is considered to be opened by a participant when at least one assignment within this session has been completed.
Proportion of participants who completed the intervention:	The percentage of intervention completers is defined as proportion of participants who opened all sessions.

Table 4.

Specific measures (may vary between clinical studies and intervention arms).

Measure	Definition
Number and/or proportion (e.g. 5 out of 10 possible entries) of entries in a specific diary:	Diary entries are counted (absolute frequency) on a per participant basis. If a maximum number of entries is defined the proportion (relative frequency) of provided entries will be calculated.
Number and/or proportion (e.g. 5 out of 10 possible entries) of entries in a specific task:	Task entries are counted (absolute frequency) on a per participant basis. If a maximum number of entries is defined the proportion (relative frequency) of provided entries will be calculated.
Number of messages written in group discussions:	Messages are counted (absolute frequency) on a per participant basis.
Number of personal messages written to guide:	Messages are counted (absolute frequency) on a per participant basis.

2.2.1.2. Working alliance

Working alliance will be assessed at mid-intervention using the Working Alliance Inventory (WAI-SR) (Munder et al., 2010) adapted for online interventions. It measures the therapeutic alliance by assessing three key aspects of the therapeutic alliance: (a) agreement on the tasks of the intervention, (b) agreement on the goals of the intervention, and (c) development of an affective bond (this aspect will be measured in guided interventions only).

2.2.1.3. Early response

A participant will be classified as an early responder if there is an improvement on at least one of four paramount outcome measures (PHQ-9 Total Score (Kroenke et al., 2001), GAD-7 Total score (Spitzer et al., 2006), AUDIT-C Total Score (Bush et al., 1998), EDE-Q Total Score (Fairburn and Beglin, 2008)) and no deterioration on any of these outcomes at mid intervention compared with baseline. Improvement and deterioration will be determined by calculating the Reliable Change Criterion for each measure in each individual study sample, using the baseline standard deviation in the respective sample and Cronbach's alpha of the measure. In samples that involve subsamples from multiple countries with different languages, the reliable change criterion will be calculated separately for each subsample.

2.2.2. Predictors and moderators of process variables and outcomes

We will examine a number of intervention characteristics as candidate predictor/moderator variables for adherence, working alliance, early response, and outcomes (see Table 5). Across studies, investigators have agreed to collect a number of common baseline variables to be included in the analyses as candidate predictor/moderator variables (see Table 6). These include socio-demographic variables, psychopathological symptoms, treatment expectations and a number of risk and protective factors.

Table 5.

Intervention characteristics.

Active intervention	Active intervention vs. waitlist control group
Targeted disorder/domain	Depression/anxiety vs. eating disorder vs. other
Number of sessions
Group discussion	Moderated synchronous or asynchronous group discussion vs. no group discussion
Guidance by a coach	Guided vs. unguided intervention
Guidance content	Messages to enhance motivation promote adherence vs. individualized feedback on session entries
Automated feedback	Automated feedback based on survey entries vs. no automated feedback
Symptom monitoring	Weekly symptom monitoring vs. no symptom monitoring
Self-monitoring of automatic irrational thoughts	Self-monitoring of automatic irrational thoughts vs. no self-monitoring of automatic irrational thoughts
Anonymity	Anonymous vs. non-anonymous participation
Incentives for participation	Incentive vs. no incentive
Multimedia content (audio, video)	Number of videos and audios in the intervention

Table 6.

Participant characteristics.

Group	Variable	Specification
Socio-demographic variables	Level of education	European Qualifications Framework (EQF) Level 1–8, 99
	Gender	Male (1), female (2), other (3)
	Household size
	Children under 18 years in household
	Household Income	% of gross domestic product (at purchasing power parity) per capita
	Year of birth
	Marital status	Single; with partner, but living apart; married, or living with partner; divorced without new partner; divorced with new partner; widowed without new partner; widowed with new partner
	Employment status	Student; unemployed, stay at home parent our spouse; self-employed; part time employee (less than 35 h); full time employee (35 h or more); pensioner
	Size of place of residence (no. of inhabitants)	Less than 5000; 5000–10,000; 10,000–20,000; 20,000–50,000; 50,000–100,000; 100,000–500,000; more than 500,000
Psychopathology	Depression	PHQ 9 (adult populations only)
	Anxiety	GAD 7 (adult populations only)
	Substance disorders	AUDIT-C
	Any diagnosed mental disorder (lifetime)	Self-report, yes/no
	Lifetime depression	Self-report, yes/no
	Lifetime anxiety disorder, obsessive compulsive disorder, or PTSD	Self-report, yes/no
	Lifetime substance related disorder	Self-report, yes/no
	Lifetime eating disorder	Self-report, yes/no
	Lifetime bipolar disorder or psychosis	Self-report, yes/no
	Lifetime ADHD	Self-report, yes/no
	Lifetime other mental disorder	Self-report, yes/no
Expectations	Prior experience with psychotherapy	Self-report, yes/no
	Helpfulness of previous psychotherapy	Self-report, not at all; somewhat; very much
	Participant expectations	CEQ
Risk and protective factors	Self-esteem	RSES
	Resilience	CD-RISC-10
	Self-regulation	SSRQ
	Personality	BFI-10

The PHQ-9 is the depression module of the self-administered version of the PRIME-MD diagnostic instrument for common mental disorders. It scores each of the 9 DSM-5 diagnostic criteria as 0 (not at all) to 3 (nearly every day). The PHQ-9 is also a reliable and valid measure of depression severity (Kroenke et al., 2001).

The GAD-7 is a one-dimensional instrument designed to detect symptoms of generalized anxiety disorder as it is defined in the DSM-5. The item scores range from 0 (not at all) to 3 (nearly every day). The GAD-7 is a valid and efficient tool for screening for GAD and assessing its severity in clinical practice and research (Spitzer et al., 2006).

The AUDIT-C is a brief and valid primary care screening test for heavy drinking and/or active alcohol abuse or dependence. The response options for the three items are scored 0–4 points, and possible AUDIT-C scores range 0–12 points (Bush et al., 1998).

The CEQ credibility/expectancy questionnaire is a quick and easy-to-administer scale for measuring treatment expectancy and rationale credibility for use in clinical outcome studies (Devilly and Borkovec, 2000).

The Rosenberg self-esteem scale (RSES), is a self-esteem measure widely used in social-science research. The scale measures state self-esteem by asking the respondents to reflect on their current feelings. It is considered a reliable and valid quantitative tool for self-esteem assessment (Rosenberg, 1965).

The CD-RISC-10 is a resilience measure. The items reflect the ability to bounce back from the variety of challenges that can arise in life. It measures a characteristic that differentiates individuals who are functioning well after adversity from those who are not (Campbell-Sills and Stein, 2007).

The Short Self-Regulation Questionnaire (SSRQ) is a 31-item questionnaire that was designed to assess self-regulation capacity across the seven processes of self-regulation. Items are scored on a 1–5 scale (strongly disagree–strongly agree), and can be summed to create a total score. Items include “I doubt I could change even if I wanted to,” “I am able to accomplish goals I set for myself,” “It's hard for me to notice when I've had enough (alcohol, food, sweets),” and “I am able to resist temptation” (Carey et al., 2004).

The BFI-10 is a brief personality measure. It allows assessing the Big Five by only two items per dimension. Previous research has shown that the BFI-10 possesses psychometric properties that are comparable in size and structure to those of the full-scale BFI (Rammstedt and John, 2007).

2.2.3. Outcomes

In addition to the primary outcome of each study, four paramount outcome measures (PHQ-9, GAD-7, AUDIT-C, EDE-Q) will be assessed within and across clinical studies as appropriate. PHQ-9 and GAD-7 will only be assessed in adult populations. The EDE-Q provides a comprehensive assessment of the specific psychopathology of eating-disordered behaviour in a relatively brief self-report format (Fairburn and Beglin, 2008). It will only be assessed in interventions targeting disordered eating.

2.3. Planned analyses

The following section contains overall descriptions of the statistical analyses aimed at answering the proposed research questions that have been planned ahead of recruitment for each clinical study. Since the nature of the study is exploratory, the statistical procedures described below are intended as a first step in the analyses. Subsequent analyses using different procedures or statistical models may follow and will be defined based on the results of a blinded data review (see Section 2.1). Analyses across interventions will be performed using individual participant data in a one-stage approach. Differences between the interventions, e.g., regarding participant populations, study procedures and interventions, will be accounted for by including random study and random treatment effects in the model.

2.3.1. How do participants adhere to each intervention? How do they perceive working alliance?

2.3.1.1. Adherence

For each active intervention, intervention dropout (i.e., nonusage attrition) by session will be depicted in a diagram (see Fig. 2).

Fig. 2.

Intervention dropout (example).

Means, medians, and standard deviations or Ns will be calculated for each applicable adherence measure and each active intervention.

We will calculate descriptive measures separately with the full sample for each intervention and also with the sample excluding participants who have never logged on to the platform for each intervention. Accordingly, we will calculate the average utilization of several features of the programs.

2.3.1.2. Working alliance

Only participants who completed the mid-intervention assessment will be included in the analyses. Means, medians, and standard deviations will be calculated for each applicable aspect of the therapeutic alliance (WAI-SR) and each active intervention.

2.3.2. Do different adherence measures predict outcome?

In a second step, we will determine how different adherence measures (Number of opened sessions, number of entries in self-monitoring diaries, number of messages written in group discussion, number of completed surveys, number of personal messages written to guide) are related to intervention outcomes at post-intervention and follow up.

For each active intervention, the primary study outcome as well as up to four paramount outcome measures (PHQ-9 Total Score, GAD-7 Total score, AUDIT-C Total Score, EDE-Q Total Score) will be regressed on each of the adherence measures relevant for the intervention in a (generalized) linear model. We will perform completer and intention-to-treat analyses separately for each outcome and each assessment point (post-intervention, FU), including all participants for whom the outcome is available. If an outcome is predicted by more than one adherence measure, the predictors will be entered in a stepwise forward regression analysis to determine the final prediction model. The same will be repeated across all active interventions for the four paramount outcome measures (PHQ-9 Total Score, GAD-7 Total score, AUDIT-C Total Score, EDE-Q Total Score).

2.3.3. Do intervention and participant characteristics predict adherence?

In the third step of the project, adherence predictors will be identified. For this purpose, we will examine how characteristics of the interventions and characteristics of the participants are related to different adherence measures. We will employ an empirical approach for the identification of predictors of different adherence measures as proposed by the MacArthur Foundation (Kraemer et al., 2001; Kraemer et al., 2002). A prediction model for each active intervention and each relevant adherence measure will be determined in four steps:

• 1.Correlations between the adherence measure and all available baseline variables will be determined and baseline variables that do not significantly (i.e., p ≥ .05) correlate with the adherence measure will be excluded from further analyses.
• 2.Correlations between baseline variables with significant correlations with the adherence measure will be calculated and proxies will be removed. A variable is considered a proxy for another variable if all of the following conditions are met: 1) both variables are significantly correlated with the adherence measure, 2) both variables are significantly correlated with each other, 3) both variables are measured at the same time and 4) both variables reflect interrelated behaviours (Kraemer et al., 2005; Kraemer et al., 2001).
• 3.The adherence measure will be regressed on each of the remaining baseline variables. To facilitate interpretation of findings, baseline variables will centred around their mean or median as appropriate (Kraemer and Blasey, 2004).
• 4.If an adherence measure is predicted by more than one variable, the predictors will be entered in a stepwise forward regression analysis to determine the final prediction model.

Across all active interventions of all clinical studies, a prediction model for four paramount adherence measures (study dropout, time from baseline assessment to first login, overall participation, number of opened sessions) will be determined in the steps described above.

In addition, to assess the impact of intervention characteristics across all active interventions of all clinical studies, a prediction model for four paramount adherence measures (study dropout, time from baseline assessment to first login, overall participation, number of opened sessions) will be determined in the steps described above.

2.3.4. Do intervention and participant characteristics predict outcome?

In the fourth step of the project, outcome predictors will be identified. For this purpose, we will examine how characteristics of the interventions and characteristics of the participants are related to different outcome measures at post intervention and follow-up. In the primary analyses, outcome predictors will be analysed in completer samples (i.e., all participants who provided the outcome). Sensitivity analyses will be performed in intention-to-treat samples (i.e., all participants who were allocated to the intervention). Strategies for imputing missing data will be decided upon following a blinded data review separately for each clinical study (see Section 2.1). We will include all variables used in the analysis model (including the outcome variable and any interactions or non-linear terms determined in the primary analyses) in the imputation model to warrant consistency between the imputation and analysis models. We will also include predictors of study dropout (as identified in analyses described above) as auxiliary variables in the imputation model to improve the accuracy of the imputed values (Hayati Rezvan et al., 2015). We will employ an empirical approach for the identification of predictors of different outcome measures as proposed by the MacArthur Foundation (Kraemer et al., 2001; Kraemer et al., 2002). A prediction model for each active intervention and the primary study outcome as well as up to four paramount outcome measures (PHQ-9 Total Score, GAD-7 Total score, AUDIT-C Total Score, EDE-Q Total Score) will be determined in four steps equivalent to the procedure described under Section 2.3.3.

Sensitivity analyses will be performed adjusting for adherence, using overall participation and the adherence measure that has been shown to be most closely related to the outcome. The analyses described above will be performed separately within each active intervention condition and between intervention conditions within clinical studies. For the latter, analyses will be performed as moderator analyses, where candidate moderator variable will be entered into mixed effects models instead of regression analyses.

Across all active interventions of all clinical studies, a prediction model for four paramount outcome measures (PHQ-9 Total Score, GAD-7 Total score, AUDIT-C Total Score, EDE-Q Total Score) will be determined in the steps described above.

In addition, to assess the impact of intervention characteristics across all active interventions of all clinical studies, a prediction model for four paramount outcome measures (PHQ-9 Total Score, GAD-7 Total score, AUDIT-C Total Score, EDE-Q Total Score) will be determined in the steps described above.

2.3.5. Do intervention and participant characteristics predict working alliance?

In the fifth step of the project, predictors for the working alliance as measured by the WAI-SR adapted for online interventions will be identified. For this purpose, we will examine how characteristics of the interventions and characteristics of the participants are related to the three dimensions (task, goal, bond) of the working alliance; the bond dimension will only be examined for guided interventions. We will employ an empirical approach for the identification of predictors of each dimension of the working alliance as proposed by the MacArthur Foundation (Kraemer et al., 2001; Kraemer et al., 2002). A prediction model for each active intervention and each working alliance dimension will be determined in four steps equivalent to the procedure described under Section 2.3.3.

Across all active interventions of all clinical studies, a prediction model for each working alliance dimension will be determined in the steps described above.

In addition, to assess the impact of intervention characteristics across all active interventions of all clinical studies, a prediction model for each working alliance dimension will be determined in the steps described above.

2.3.6. Do intervention and participant characteristics predict early response?

In the sixth step of the project, predictors of early response will be identified. For this purpose, we will examine how characteristics of the interventions and characteristics of the participants are related to early response. Only participants who completed the mid-intervention assessment will be included in the analyses.

We will employ an empirical approach for the identification of predictors of early response as proposed by the MacArthur Foundation (Kraemer et al., 2001; Kraemer et al., 2002). A prediction model for each active intervention will be determined in four steps equivalent to the procedure described under Section 2.3.3.

Across all active interventions of all clinical studies, a prediction model for early response will be determined in the four steps described above.

In addition, to assess the impact of intervention characteristics across all active interventions of all clinical studies, a prediction model for early response will be determined in the four steps described above.

2.3.7. Are working alliance and adherence related?

In the seventh step of the project, we will examine relations between the three dimensions of working alliance and different adherence measures. Only participants who completed the mid-intervention assessment will be included in the analyses. The adherence measures described above will be calculated separately for each of the two intervention phases before (phase one) and after (phase two) the mid-intervention assessment.

2.3.7.1. Does phase one adherence predict working alliance?

For each active intervention, each working alliance dimension will be regressed on each of the phase one adherence measures relevant for the intervention in a linear model. We will perform completer analyses separately for each working alliance dimension. If a working alliance dimension is predicted by more than one adherence measure, the predictors will be entered in a stepwise forward regression analysis to determine the final prediction model. Across all active interventions, each working alliance dimension will be regressed on each of the phase one adherence measures relevant for the intervention in a linear model. We will perform completer analyses separately for each working alliance dimension. If a working alliance dimension is predicted by more than one adherence measure, the predictors will be entered in a stepwise forward regression analysis to determine the final prediction model.

2.3.7.2. Does working alliance predict phase two adherence?

For each active intervention, each of the phase two adherence measures relevant for the intervention will be regressed on each working alliance dimension in a linear model. We will perform completer analyses separately for each adherence measure. If an adherence measure is predicted by more than one working alliance dimension, the predictors will be entered in a stepwise forward regression analysis to determine the final prediction model. Across all active interventions, each of the phase two adherence measures relevant for the intervention will be regressed on each working alliance dimension in a linear model. We will perform completer analyses separately for each adherence measure. If an adherence measure is predicted by more than one working alliance dimension, the predictors will be entered in a stepwise forward regression analysis to determine the final prediction model.

2.3.8. Are early response and adherence related?

In the eighth step of the project, we will examine relations between early response and adherence. Only participants who completed the mid-intervention assessment will be included in the analyses. The adherence measures described above will be calculated separately for each of the two intervention phases before (phase one) and after (phase two) the mid-intervention assessment.

2.3.8.1. Does phase one adherence predict early response?

For each active intervention, early response will be regressed on each of the phase one adherence measures relevant for the intervention in a linear model. We will perform completer analyses. If early response is predicted by more than one adherence measure, the predictors will be entered in a stepwise forward regression analysis to determine the final prediction model. Across all active interventions, early response will be regressed on each of the phase one adherence measures relevant for the intervention in a linear model. We will perform completer analyses. If early response is predicted by more than one adherence measure, the predictors will be entered in a stepwise forward regression analysis to determine the final prediction model.

2.3.8.2. Does early response predict phase two adherence?

For each active intervention, each of the phase two adherence measures relevant for the intervention will be regressed on early response in a linear model. We will perform completer analyses separately for each adherence measure. Across all active interventions, each of the phase two adherence measures relevant for the intervention will be regressed on early response in a linear model. We will perform completer analyses.

2.3.9. Does working alliance predict outcome?

In a ninth step, it will be determined how the three dimensions of working alliance are related to intervention outcomes at post-intervention and follow up. Only participants who completed the mid-intervention assessment will be included in the analyses.

For each active intervention, the primary study outcome as well as up to four paramount outcome measures (PHQ-9 Total Score, GAD-7 Total score, AUDIT-C Total Score, EDE-Q Total Score) will be regressed on each of the working alliance dimensions relevant for the intervention in a linear model. We will perform completer analyses separately for each outcome and each assessment point (post-intervention, FU), including all participants for whom the outcome is available. Sensitivity analyses will be performed in intention-to-treat samples (i.e., all participants who were allocated to the intervention). Strategies for imputing missing data will be decided upon following a blinded data review separately for each clinical study (see Section 2.1). If an outcome is predicted by more than one working alliance dimension, the predictors will be entered in a stepwise forward regression analysis to determine the final prediction model.

Across all active interventions, four paramount outcome measures (PHQ-9 Total Score, GAD-7 Total score, AUDIT-C Total Score, EDE-Q Total Score) will be regressed on of the working alliance dimensions in a linear model. We will perform completer analyses separately for each outcome and each assessment point (post-intervention, FU), including all participants for whom the outcome is available. Sensitivity analyses will be performed in intention-to-treat samples (i.e., all participants who were allocated to the intervention). Strategies for imputing missing data will be decided upon following a blinded data review separately for each clinical study (see Section 2.1). If an outcome is predicted by more than one working alliance dimension, the predictors will be entered in a stepwise forward regression analysis to determine the final prediction model.

2.3.10. Does early response predict outcome?

In a final step, it will be determined how early response is related to intervention outcomes at post-intervention and follow up. Only participants who completed the mid-intervention assessment will be included in the analyses.

For each active intervention, the primary study outcome as well as up to four paramount outcome measures (PHQ-9 Total Score, GAD-7 Total score, AUDIT-C Total Score, EDE-Q Total Score) will be regressed on early response in a linear model. We will perform completer analyses separately for each outcome and each assessment point (post-intervention, FU), including all participants for whom the outcome is available. Sensitivity analyses will be performed in intention-to-treat samples (i.e., all participants who were allocated to the intervention). Strategies for imputing missing data will be decided upon following a blinded data review separately for each clinical study (see Section 2.1).

Across all active interventions, four paramount outcome measures (PHQ-9 Total Score, GAD-7 Total score, AUDIT-C Total Score, EDE-Q Total Score) will be regressed on early response in a linear model. We will perform completer analyses separately for each outcome and each assessment point (post-intervention, FU), including all participants for whom the outcome is available. Sensitivity analyses will be performed in intention-to-treat samples (i.e., all participants who were allocated to the intervention). Strategies for imputing missing data will be decided upon following a blinded data review separately for each clinical study (see Section 2.1).

3. Discussion and conclusion

To our knowledge, this is the first pre-planned study that aims to pool data from over 7500 participants from seven clinical trials on online interventions addressing a range of Internet-based interventions for the prevention and self-help/treatment of different mental health conditions and disorders and implemented in different setting (schools, universities, health care system). The study design was developed in close cooperation with investigators of each of these studies and common measures regarding characteristics of the interventions, characteristics of the participants, adherence, and intervention outcomes were agreed upon before each study commenced. It will allow us to compare adherence, working alliance and early response across similarly structured interventions addressing different mental health conditions in different settings. It will also contribute to the identification of general and disorder-specific interrelations between characteristics of the intervention, characteristics of the participants, adherence, working alliance, early response, and intervention outcomes.

The findings of these analyses will shed light on research questions including: For whom are online interventions suitable? Under what conditions? Who engages in online interventions? Who benefits from them? How can we design online interventions in order to ensure that users receive the necessary intervention dose? What amount of guidance is necessary in online interventions in different groups of users, respective to both adherence and outcomes? What intervention dose is necessary to achieve clinically meaningful outcomes? While many of these questions have been addressed in psychotherapy research, findings from studies with internet-based interventions are rather scarce. Overall, this project will help to answer questions specifically related to the complex interrelations between participant, intervention and process characteristics and general outcomes of internet-based interventions.

Competing interests

The authors declare that they have no competing interests.

Funding

This project has received funding from the European Union's Horizon 2020 research and innovation program under grant agreement no 634757.

Author contributions

IB and CJ designed the study in the first place. TB, DG, DE, MZ, KW and RH contributed significantly to the design of the planned study. All co-authors supported the common measures for use in individual trials and countries. IB wrote the manuscript. TB, MZ, KW and CJ contributed revisions to the manuscript. All authors read and approved the final manuscript.