RELEASE: a protocol for a systematic review based, individual participant data, meta- and network meta-analysis, of complex speech-language therapy interventions for stroke-related aphasia

Abstract

Background

Speech and language therapy (SLT) benefits people with aphasia following stroke. Group level summary statistics from randomised controlled trials hinder exploration of highly complex SLT interventions and a clinically relevant heterogeneous population. Creating a database of individual participant data (IPD) for people with aphasia aims to allow exploration of individual and therapy-related predictors of recovery and prognosis.

Aim

To explore the contribution that individual participant characteristics (including stroke and aphasia profiles) and SLT intervention components make to language recovery following stroke.

Methods and procedures

We will identify eligible IPD datasets (including randomised controlled trials, non-randomised comparison studies, observational studies and registries) and invite their contribution to the database. Where possible, we will use meta- and network meta-analysis to explore language performance after stroke and predictors of recovery as it relates to participants who had no SLT, historical SLT or SLT in the primary research study. We will also examine the components of effective SLT interventions.

Outcomes and results

Outcomes include changes in measures of functional communication, overall severity of language impairment, auditory comprehension, spoken language (including naming), reading and writing from baseline. Data captured on assessment tools will be collated and transformed to a standardised measure for each of the outcome domains.

Conclusion

Our planned systematic-review-based IPD meta- and network meta-analysis is a large scale, international, multidisciplinary and methodologically complex endeavour. It will enable hypotheses to be generated and tested to optimise and inform development of interventions for people with aphasia after stroke.

Systematic review registration

The protocol has been registered at the International Prospective Register of Systematic Reviews (PROSPERO; registration number: CRD42018110947)

Background

The recent Cochrane systematic review of speech and language therapy (SLT) for aphasia after stroke demonstrated the effectiveness of SLT compared to no SLT on measures of functional communication, expressive language, reading and writing (Brady, Kelly, Godwin, Enderby, & Campbell, 2016). However, the meta-analyses were restricted to group summary statistics extracted from randomised controlled trial reports. SLT for aphasia is a highly complex intervention delivered to a heterogeneous population. Interventions may vary by theoretical approach, treatment target or delivery mode (computer, volunteer or professionally facilitated). Therapy regimens may vary in intensity (hours of therapy weekly), duration (weeks or months over which therapy is delivered) and dosage (total number of therapy hours delivered). Therapists draw on a variety of delivery models (in isolation or combination), providers and augmentations (e.g., home-based practice) to develop a tailored intervention to meet an individual’s rehabilitation and communication needs.

While current stroke guidelines acknowledge the benefits of SLT for people with aphasia following stroke (2008) evidence of how to optimise interventions has been lacking (RCP 2016). Better outcomes may be associated with higher intensity interventions (up to 15 hours weekly) than lower intensity interventions (up to 5 hours weekly) (Brady et al., 2016) although this was confounded by significantly higher dropouts in the interventions delivered at a higher intensity. The situation was complicated further by some indication that benefits and dropouts may be related to the time since stroke. Differential dropouts (and benefit) were mainly observed in the context of early recruitment to intervention after stroke. Those recruited years after stroke did not dropout, but evidence of benefit was absent (Brady et al., 2016).

Further exploration of these interacting factors within the Cochrane systematic review methodology was limited by the availability of suitable randomised controlled comparisons, limited overlap in outcomes across trials (Wallace et al., 2018) and the availability of data collected and reported. In conducting the Cochrane review we had access to the individual participant data (IPD) (n = 323/3002) from a small number of included trials; this facilitated calculation of group summary statistics and representation of the trials within the review which would otherwise have been omitted and risked reporting bias.

Conducting the systematic review of relevant records highlighted the long tradition of reporting IPD in published reports of SLT for aphasia after stroke. We were also aware of the significant advantages an IPD analysis would afford, including the possibility of larger sample sizes which would support more reliable statistical analysis and permit adjustment for confounding variables in predictors of recovery. This stimulated our interest in the possibility of pooling data for secondary analysis purposes.

The process of systematically gathering IPD for the purpose of secondary analysis is complex, requiring careful planning to avoid problems and to streamline the research process. A protocol provides a detailed description of the planned research processes and procedures which enables comparisons of pre-specified plans with the completed research; highlights protocol deviations in the reporting of the completed study; supports replication of the research; ensures early documentation of pre-specified decisions and facilitates adherence to such decisions throughout the research; reduces the risk of research waste through unintentional duplication of research activities. Registration of systematic review protocols is supported by the international PROSPERO database (funded by the National Institute for Health Research, UK) which profiles a brief summary of a systematic review protocol.

Brief registrations however can fall short of the increasingly detailed descriptions of complex systematic review protocols required in order to adhere with current reporting systematic review recommendations such as PRISMA (Moher, Liberati, Tetzlaff, Altman, & Group, 2009; David; Moher et al., 2015) and relevant extensions for protocols (PRIMSA-P (Shamseer et al., 2015)), individual participant data (PRISMA-IPD (Stewart, Clarke, Rovers, Riley, Simmonds, Stewart, Tierney and the PRISMA-IPD Development Group 2015)); complex interventions (PRISMA-CI Guise, Butler, Chang, Viswanathan, Pigott, Tugwell, and the Complex Interventions Workgroup 2017); and network meta-analysis (PRISMA-NMA Hutton, Salanti, Caldwell, Chaimani, Schmid, Cameron, Ioannidis, Straus, Thorlund, Jansen, Mulrow, Catalá-López, Gøtzsche, Dickersin, Boutron, Altman, Moher 2015). Increasingly, such complex protocols are expanded upon in a journal article which supplements reporting of any findings. Our protocol describes the planned RELEASE systematic-review-based, IPD meta- and network meta-analysis of a large scale, international, multidisciplinary aphasia research dataset. Changes to this protocol will be documented in any subsequent publication.

Aim

We aim to develop a database of IPD to explore the contribution that individual characteristics (including stroke and aphasia profiles) and SLT intervention components make to language recovery (reflected in improved performance on measures of language) of people with aphasia following stroke. Our proposed analyses will use this database to explore the following research questions:

• (1)What is the pattern of language recovery (functional communication, overall aphasia severity, spoken language production, auditory comprehension, reading and writing) for people with aphasia after stroke?
• (2)What are the predictors of language recovery following aphasia?
• (3)What are the components of effective aphasia rehabilitation interventions?
• (4)Are some interventions (or intervention components) more beneficial for some participant subgroups (individual, stroke or aphasia characteristics) than others?

Methods

Eligibility criteria

All research study designs with IPD on people with aphasia after stroke are eligible for inclusion. While data from randomised controlled trial (RCT) designs are preferable in meta-analysis of IPD which looks at intervention effectiveness, the IPD generated in the context of other study designs may also contribute to covariate-adjusted analyses of recovery profiles and predictors of prognosis after stroke (Abo-Zaid, Sauerbrei and Riley, 2012). Thus, we will employ no study design restrictions to IPD contributions. We will also have no language restrictions.

Inclusion criteria

We will accept IPD datasets that:

• are collected in the context of a primary research study or clinical register with relevant ethical approvals in place which are published or unpublished;

• include data on a minimum of 10 people with aphasia after stroke (reflecting the considerable time required for data preparation and actual IPD contribution);

• include IPD on aphasia severity at a minimum of one time-point (baseline);

• include IPD information on time since stroke (or time since aphasia onset) at first assessment;

• include IPD on functional language use, overall severity of aphasia, language expression, auditory comprehension, reading or writing.

Participant populations of relevance to our analysis are (a) participants who had no SLT (b) participants who may have had historical SLT prior to the primary research and (c) participants that received SLT in the context of the primary research.

Exclusion criteria

Datasets will be excluded if they include only:

• qualitative IPD;

• non-language data (e.g., response to a stimulus measured in time);

• data at group summary statistics level.

Where a dataset includes participants with aphasia of mixed aetiologies, we will extract the stroke-specific data only. That stroke-specific IPD dataset will be included if all other RELEASE eligibility criteria are met.

Definitions

Speech and language therapy interventions

SLT will be “any targeted practice or rehabilitation tasks that aimed to improve language or communication abilities, activities, or participation” (Brady et al., 2016), which are often (but not always) delivered by a speech and language therapist. SLT provided by others is also eligible for inclusion and we will record the provider for further analysis.

Social support and stimulation

Interventions which provide informal support and stimulation of language in a functional situation, but do not include therapeutic interventions that aim to improve the participant’s language impairments, will be considered social support and stimulation interventions.

Conventional SLT

Interventions which are only described as “conventional”, “typical” or “usual” SLT, and where further intervention details (permitting further categorisation of the SLT approach) are unreported will be referred to as “conventional SLT”. Equivalent terms used in the literature may include traditional SLT, standard SLT, typical SLT or “as directed by the therapist”. We acknowledge that what is considered conventional in one context may not be directly comparable to conventional SLT in another.

SLT co-interventions

In some cases, a co-intervention may be administrated before, during or after the SLT intervention. These may include, for example, pharmacological interventions (e.g., Levodopa in Breitenstein et al., 2015) or neurostimulation (e.g., transcranial direct current stimulation in Abo, Kakuda, Watanabe, Morooka, Kawakami, Senoo, 2012). Such cointerventions are not typically within the remit of routine clinical SLT and examining their contribution to language recovery is beyond the scope and resources of this study, but their presence will be noted as will their possible contribution to the analysis.

Information sources

A range of electronic databases will be searched from their inception, including the Cochrane Stroke Group Trials Register, CENTRAL and other Cochrane Library Databases (CDSR, DARE, HTA), MEDLINE, EMBASE, CINAHL, AMED, LLBA, and SpeechBITE with a comprehensive RCT optimised search strategy as used in the relevant Cochrane Review (Brady et al., 2016). We will also review all studies included and excluded from the systematic review (Brady et al., 2016). We will also search major trials registers including ClinicalTrials.gov (www.clinicaltrials.gov/), the Stroke Trials Registry (www.strokecenter.org/trials/), Current Controlled Trials (www.controlled-trials.com), and WHO ICTRP (www.who.int/ictrp/search/en/). As an example, our MEDLINE search strategy is presented in Appendix 1.

Study records

We will systematically screen all records identified by our search for eligibility and inclusion in RELEASE. Where eligible datasets are identified, we will invite the primary research teams to contribute their dataset to the RELEASE database. We will also extend an invitation to contribute datasets to members of the Collaboration of Aphasia Trialists (CATs, www.aphasiatrials.org). Initial project development work involving this network generated several commitments to contribute to the database in preparation for our funding application.

Selection process

Record titles and abstracts will be screened for eligibility using the criteria listed earlier. Full-text publications of relevant records will be retrieved where possible and reviewed. Where published reports are unavailable (for recently completed studies, clinical registries or similar), we will clarify eligibility in discussion with the primary research team. Disagreements will be resolved through discussion, where necessary involving an additional reviewer. Where eligible IPD are available in the public domain we will extract the data relevant to RELEASE, creating an electronic dataset for use in the study.

Data collection

Where we identify a dataset that is eligible for RELEASE but unavailable in the public domain, we will approach the relevant data gatekeepers for that dataset and invite them to contribute the anonymised data to the RELEASE study. We will use a systematic approach to record all communication attempts (in all formats including telephone calls, emails and other communication formats) in relation to these primary research datasets and associated teams including queries around eligibility, invitations to contribute and other correspondence. Communication with all external researchers will be consistent; for example, all will receive a similar number of invitations to contribute data and follow-up invitations.

Where the primary research team expresses interest, we will request that they contribute a copy of their anonymised electronic dataset in an encrypted format. We will invite submission of all relevant supporting documents such as a data dictionary, ethical approval for the primary research, a funder’s report, or other reporting of that dataset and findings. We will also request evidence of gatekeeper (data controller) approval to share the dataset with the RELEASE collaborators. Where necessary, if the primary research team require additional permissions to share the dataset for the purposes of secondary data analysis, we will request a copy of this.

Data extraction

Using best practices in reporting complex interventions (Hoffmann, Glasziou, Boutron, Milne, Perera, Moher, Altman, Barbour, Macdonald, Johnston, Dixson-Woods, McCulloch, Wyatt, Chan, & Michie, S., 2014), we will develop and pilot a data extraction table to support the collection of relevant data across multiple datasets. Data items extracted are listed within four main groups in Table 1. All available sources of information on the primary dataset will be used to populate the table such as published papers and through direct communication with the primary research team to gather data items unavailable within the contributed materials. Once data extraction has been completed on a dataset, the primary research team will be asked to review the data extraction for accuracy and completeness.

Table 1. Data extraction items.

Participant characteristics Demographic information (e.g., age, sex, handedness, ethnicity) Environmental descriptors (e.g., living environment, social support) Stroke characteristics (e.g., type, time since stroke, severity, cognition)

Language measure Functional communication Aphasia ability/severity Auditory comprehension Spoken language production Reading Writing

Primary dataset level information Design Inclusion/exclusion criteria (e.g., dysarthria, prior stroke) Recruitment dates (or publication) Numbers of participants Country and language Data collection time-point(s)

SLT Intervention (where relevant) Provider Delivery mechanism(s) Context of intervention Duration (total number of days during which therapy was delivered) Intensity (hours of therapy provided on a weekly basis) Frequency (how many sessions provided weekly) Dosage (total number of hours of therapy provided) Tailoring (by difficulty, by functional relevance) Adherence (data capture and actual adherence rates) Theoretical approach Treatment target

For public domain datasets, a second researcher will rigorously double check the data. Any data items unavailable from the sources described earlier will be considered either “not applicable” (e.g., details of SLT intervention within a study that does not have an intervention) or “unreported”.

Outcomes

Our primary outcome will be change in language recovery profiles according to overall language ability, auditory comprehension, spoken language production, reading, writing, and functional communication (Table 1). As RELEASE will undertake secondary analysis (data synthesis) of data originally gathered in the context of primary research studies, it is important that we do not pre-specify the language assessment tools eligible for inclusion. Given the nature of our international, multidisciplinary, multilingual database, we will need to be responsive to emerging datasets. However, measurement tools included in our analysis will (a) capture the outcomes of relevance to RELEASE, (b) be published and accessible in the public domain, and (c) be approved by the RELEASE Collaborators. Screening tools will be excluded, given their typical lack of sensitivity due to ceiling effects, questionable psychometric properties and the impact this would have on analysis.

Outcomes of primary importance to people with aphasia and their families typically include communicative participation and activity (e.g., Enderby & John, 2015; Wallace, Worrall, Rose, & Le Dorze, 2014). However, these outcomes have historically been captured infrequently in aphasia research (Brady et al., 2016), though this will change for future aphasia research with relevant measures included in the recently published core outcome set for aphasia research (Wallace, Worrall, Rose, Le Dorze, Breitenstein, Hilari, Babbitt, Bose, Brady Cherney, Copland, Cruice, Enderby, Hersh, Howe, Kelly, Kiran, Laska, Marshall, Nicholas, Patterson, Pearl, Rochon, Rose, Sage, Small, & Webster 2018).

Many of the commonly used tools also capture a mix of activity and participation constructs which would prevent consideration of activity and participation outcomes in isolation. For this reason, in the context of this secondary analysis research, we will use the umbrella term “Functional Communication” to accommodate outcome measures within primary research studies which capture the functional use of language. This will also facilitate comparisons with the relevant evidence synthesis from the Cochrane review (Brady et al., 2016). As more researchers adopt the core outcomes set for aphasia (Wallace et al., 2018), the prevalence of outcome measures reflecting activity and participation will change.

Methodologically, functional communication measures capture information using performance-based measures, self- or proxy report, Likert scales, categorical scales, observational profiles and counts of discourse features. Consequently, data synthesis may be challenging. If availability and synthesis of this outcome is problematic (less than 20% of IPD data can be synthesised) then we will synthesise as much data as possible, but we will consider measures of overall language ability (a global measure of language performance across spoken and written language domains) as an important outcome.

We will extract and use raw scores wherever possible. Percentage scores will be used to calculate the raw score if possible. Correct or incorrect scoring systems will be aligned on the same tool and with the same direction of scoring, i.e., all data will be reported in the same format, all reported as positive or all reported as negative.

Risk of bias of individual studies

We will extract information on the methodological quality of each primary research dataset included in RELEASE (Aromataris et al., 2015). We will consider the following potential risks of bias:

• Selection bias: Choice and allocation of individual participants to a specific group. Within the context of a randomised controlled trial, for example, we will consider whether the randomisation sequence generated was truly random and whether the sequence allocation was concealed up to the time of allocation of the individual to a group.

• Performance bias: Differences in co-interventions between groups (in group comparison study designs) that were unaccounted for within the intervention comparisons. Blinding participants to the delivery of SLT (or not) is unlikely to be achieved though may be possible in the context of pharmacological interventions or electrical stimulation co-interventions.

• Detection bias: We will document blinding of outcome assessors.

• Attrition bias: We will examine whether there is evidence of systematic between group differences in the numbers of drop-outs (withdrawals for any reason) or non-adherence (those that declined to continue study participation during the intervention).

For each potential risk of bias, we will code the studies as low, unclear, or high risk. We will consider the impact of any potential biases on our findings narratively or using sensitivity analyses as appropriate.

Data handling and synthesis

Each contributing primary research dataset will be given a unique identifier while each constituent participant will have a unique RELEASE ID which will facilitate identification of each dataset and participant to specific analyses. Datasets will be checked for duplicate participant datasets through careful review of IPD demographics across all datasets and reports from the same primary research team and where possible clarify with the primary researchers. We will avoid the risk of double counting by excluding any duplicate data from the analysis and reporting.

The included research datasets will be classified by study design and reflecting the contribution each dataset will make to the proposed analyses as either (a) randomised controlled trial (RCT); (b) a non-randomised comparative study with two or more groups but where no randomisation was applied; (c) cohort/case series or (d) registries. We will carry out various checks on the data, discussing and clarifying discrepancies with the primary research teams. We will, for example, check the version of an outcome measurement tool used and the range values reported to ensure they are reasonable before combining the data to create a new master dataset. While some large registries may have sufficient data to examine some of the RELEASE research questions in isolation, a minimum of two datasets will be included within any RELEASE meta-analysis. As our eligibility criteria require a minimum number of 10 IPD in each dataset, the minimum number of IPD in any meta-analysis will be 20.

Where we identify cross-over datasets, we will extract the data as standard, noting the cross-over point. Data up to the point of cross-over will likely be included in the analysis as planned. We will not use data beyond the cross-over point.

For outcomes of relevance to RELEASE, we anticipate that the contributing datasets may capture relevant outcomes using a range of measurement instruments, including those in or adapted for any language. In synthesising these data, we will first identify the measurement used most commonly across datasets (“anchor measure”) and then profile all the remaining assessment tools (“minority measures”) used to capture that outcome, the number of studies that used it and the available IPD, the median score and interquartile range (IQR). Language or version variations will be treated as different tools. Minority measures will then be transformed to match the format and range of the anchor instrument of that language outcome through a method previously used by the Early Breast Cancer Trialists Collaborative Group (Peto et al., 2012). This process will be repeated for all outcomes.

In order to maintain a semblance of the anchor measure’s distribution, each anchor measure and related minority measure will be divided into quartiles. A linear transformation from the minority to the anchor will be applied within each quartile. We considered but rejected three other major approaches to transformation (normalising, internal, and direct linear) as they would be difficult to interpret given that the aim of RELEASE is to examine clinically important changes in score rather than statistically significant differences. We rejected normalising transformations, such as the Van der Waerden (1953), on the basis that (i) pooled ranks across the measures would be impossible due to the varying widths of the scales, and (ii) the “true” value of the score would be lost. Internal normalising (i.e., standardising within each study) would also result in the loss of the true value of the scales and suffer the same problems of interpretability. Direct linear transformation, while maintaining the value of the anchor measure and thus some interpretability, would result in the distribution of the minority measure being mapped on to the anchor measure, and not retaining the distribution of the anchor measure itself. As scales are often skewed in different directions, this would impact on the interpretability of the results.

All variables of interest will be synthesised (Table 1) across contributing primary research datasets. A master data dictionary and decision tree will be developed and maintained to document all synthesis decisions to ensure consistency in the treatment of data items and adherence to Collaboration-wide consensus decisions. For example, stroke lesion types will be recorded as either ischaemic (reflecting the terms infarct, lacuna, thrombosis or embolism) or haemorrhage (hematoma, intracranial haemorrhage or mixed stroke) where possible. The decision tree will also function to describe in detail all contributing language assessment tools thus allowing us to use the overall aphasia severity assessment summary score and to consider the contribution specific language subtests (e.g., naming) may make to other aspects of our planned analyses.

Attempts will be made to extract any missing data which is unavailable within a dataset from alternative sources including all associated published papers or reports, unpublished materials or (where possible) requesting the data from the primary research team. If this data remains unavailable, we will record as appropriate that it is unreported, or missing, and we will examine all missing data for patterns of loss or whether the data is missing completely at random. Variables with data not missing completely at random will be excluded from our analysis. Where more than 20% of data is missing from a primary research dataset’s variable, that dataset’s variable will be excluded from the meta-analyses. We anticipate that few contributing datasets will include all variables of interest to our analyses. In such situations, our analyses will be restricted to the subset of datasets with data on the variable. Where there is less than 20% missing data, patterns of missingness will be examined to ensure data is missing completely at random. This will be achieved by coding the variable with missing data as “missing” or “not missing” and then comparing this variable to demographic variables that may be expected to have some impact on recording (e.g., participant’s age, sex, type of stroke).

We will test for the randomness of missing data (in order to exclude the possibility of bias) by coding the variable with missing data as “missing” or “not missing”. We will then compare this variable to demographic variables that may be expected to have some impact on recording (e.g., participant’s age, sex, type of stroke) and categorical variables will be compared with the X² test. Depending on the normality of the data distribution, we will compare continuous variables using the t-test or Mann–Whitney U test.

Sensitivity and subgroup analysis

The impact of any data synthesis decisions on the findings will be considered by conducting a series of sensitivity analyses. Amongst the planned sensitivity analyses will be an exploration of the impact of the choice of assessment tool included in the data syntheses. If a dataset has employed two or more assessment tools eligible for synthesis within a language outcome (in the absence of any other deciding criteria, such as number of records or presence of follow-up data), the assessment most commonly used across the RELEASE database will go forward to the data synthesis and the impact of that choice will be investigated in a sensitivity analysis. We will also consider the findings based on the anchor measures alone, and what impact exclusion of the minority measures would have had on the findings.

A key decision in any meta-analysis is whether to use a random or fixed effect model in the meta-analysis. Standard errors for random effects models are more efficient, and thus random effects models should be used in preference to fixed effects models (Wu, 1973). The Wu–Hausman test will be used to review the two possible approaches to the meta-synthesis of the data using either a random effect or fixed effect model, to assess whether the effects are consistent and do not preclude the use of random effects.

Possible bias due to the availability of historic datasets or changes in clinical practices will be assessed by excluding older datasets (for example, with a last participant recruitment date of 1999 or earlier) from the analysis. As it is possible, or even likely, that response to treatment will depend on time since stroke and stroke severity, subgroup analyses of these variables will also be undertaken in addition to being included as covariates in the modelling process. Further subgroups may be identified as the analysis is conducted and will be explored appropriately.

We anticipate that SLT will typically have been described at the primary research study or group level and only rarely (if ever) at IPD level within an electronic dataset. SLT intervention data will be extracted from narrative descriptions in the primary research protocol, report, or in communication with the primary research team. We have worked closely with RELEASE collaborators to define and categorise therapy approaches in preparation for meaningful synthesis and analysis (Rose et al., 2018; Table 2) and all SLT interventions will be combined into these (and if required, additional) clusters for analysis.

Table 2. Therapy categories.

Therapy approach defined by treatment target Mixed SLT: SLT targets both auditory comprehension and spoken language production impairments. Auditory Comprehension SLT: SLT targets rehabilitation of auditory comprehension. Word Finding SLT: SLT targets rehabilitation of word retrieval or naming. Reading comprehension SLT: SLT targets rehabilitation of reading comprehension. Writing SLT: SLT targets rehabilitation of written language expression.

Therapy approach defined by theoretical approach Functional or Pragmatic SLT: Therapy targets improvement in communication activities and tasks considered to be useful in day-to-day functioning, and often involves targeted practice of real-world communication situations. Phonological SLT: Therapy uses phonological approaches. It seeks to improve the sound structure of language by targeting improvements in the phonological input and output routes. Semantic SLT: Therapy uses semantic approaches which focus on interpretation of language with the aim of improving semantic processing. Semantic and Phonological SLT: Employs treatment programme which uses both semantic and phonological approaches. Constraint Induced Aphasia Therapy: Participants are required to use spoken communication alone. Other communicative methods such as gesture are not encouraged or permitted. Multimodal Therapy (to improve verbal communication): Participants are encouraged to use one or more non-verbal modality (such as gestures) to facilitate improvements in their spoken language abilities. Multimodal Therapy (to improve total communication): Participants are supported to use non-verbal channels of communication alongside or as an alternative to spoken language production or writing in communication. Melodic Intonation Therapy: Employs rhythm and formulaic language to support recovery of language and exaggerated melodic sentence patterns to elicit spontaneous speech. Conversational Partner Training SLT: Targets communication interaction between the person with aphasia and their conversation partner(s). Conversational partners may be spouse, family member, friends or healthcare professionals.

Meta-biases

In the section above, Risk of bias of individual studies, we detailed how we would consider the risk of bias within the individual studies contributing to RELEASE. In this section, we consider all potential sources of bias in how we conduct our proposed IPD meta-analysis and the steps we will take to reduce risks of bias in our planned meta-analyses.

• Publication bias: Our rigorous approach to the identification and selection of eligible IPD datasets will identify datasets that were unpublished, reported in the grey literature, and conducted or reported in any language, thus reducing the risk of publication bias. Where there are enough datasets making the same comparisons, we will explore the potential risk of publication bias using funnel plots. Where a funnel plot is not possible due to limitations on the availability of similar datasets, we will explore the risk of publication bias by tabulating the individual datasets and examining the distribution between the ranges of sample size and the proportion of studies reporting significant and non-significant findings. Exploration of publication bias will not be possible in the context of registry datasets.

• Selection bias: We will incorporate a systematic review component as the first stage to building our IPD database, actively invite dataset contributions from primary research teams that were not previous collaborators, and extract IPD reported in the public domain and the grey literature.

• Availability bias: While we will be as inclusive as possible, some current or historic datasets may remain unavailable where the primary research teams cannot be contacted, no longer have data access, or may still be reporting their data.

• Other biases: We will consider other possible sources of bias including comparison choice bias or potential carryover of treatment effects within cross-over datasets. We will review the relevance of the primary research study’s objectives as eligible datasets may not have been gathered in the context of a research study with a focus on language recovery or rehabilitation.

Statistical analyses

The data will be analysed using the SAS v9.4 PROC MIXED, with the outcome as change from baseline (absolute numerical or percentual depending on the planned analysis) with study as a random effect. We anticipate that many of the included datasets will be small (10 IPD minimum) and so the inclusion of the random intercepts for individual patients would be at risk of failures of the model to resolve. Thus, we chose not to include them.

Progression of aphasia recovery, stratified by domain of assessment

We will describe the progression of aphasia recovery using two approaches: first, we will examine language performance at a single time point (a snapshot). We will examine the distribution of language domains of interest at the first assessment (baseline or time 0) within each dataset. At this time-point, participants will not have received any study-mandated intervention, having only completed a baseline assessment, and may have been enrolled into the study at any time-point since index stroke. This will allow us to generate an overview of language impairment across domains of interest, at time 0 for all participants, where time 0 can range from index stroke, up until decades after index stroke. Graphs of baseline transformed outcome measure scores over time, and stratified by time since stroke, age, sex and living context, will also be generated to examine the contribution that these potential confounders make to the language scores at each baseline time-point.

Our second approach will examine the trajectory of language impairment progression over all domains of interest by examining the absolute change in each domain score since baseline, and the rate of change of each score over time. We will stratify our observations by SLT allocations: 1) No study-mandated SLT, and no receipt of historical SLT; 2) no study-mandated SLT, but participant may have received historical SLT; and 3) study-mandated SLT.

We will attempt to fit simple linear regressions, with the option of splitting by time period since index stroke if it appears necessary.

Identifying predictors of language recovery and components of effective intervention

IPD on stroke profile, demographic characteristics and living context at baseline and (when available) follow-up time-points will be extracted. We will explore the data graphically and using summary statistics. The effect of time will be considered by (a) the time since stroke as an absolute measure of the transformed scores; and (b) time since baseline, using change from baseline. Change from baseline was selected as the outcome measure due to it being a more meaningful outcome for people with aphasia and clinicians, rather than a standardised measure (The Stroke Association, 2019; Cochrane Handbook for Systematic Reviews of Interventions; Wallace, Worrall, Rose, LeDorze, Cruice, Isaksenm Kong, Simmons-Mackie, Scarinci & Gauvreau 2017). Where residuals are normally distributed (once adjusted for baseline score) they will not fail the basic assumption of a linear (mixed model) regression.

Intervention details for each study that includes an SLT intervention targeting aphasia rehabilitation will be recorded in a data extraction table alongside descriptive information on the study, the participant characteristics and outcome data. Where studies are similar in their participants and the type of intervention delivered, and where suitable language measurement data is reported before and after the intervention, we will pool the data within a meta-analysis.

Demographic and stroke covariates identified as statistically significant will be used to create our planned model for analysis. These statistically significant covariates in the basic model will be treated as a fixed effect along with study as the random effect, then each of the other covariates of interest will be added to the basic model for examination of its effect on the adjusted data.

This will allow us to account for differences in participant characteristics before any treatment variables are examined for influence on the outcome variables. The principal analysis method will be a mixed effects model, with the primary research study as the random effect.

Our preference will be to extract data on all components of therapy regimen (for example, frequency, duration, dosage, duration; Table 1, SLT intervention) wherever possible as an actual numerical variable. We anticipate that in some datasets this data will be recorded as IPD while in other cases it will be recorded at the group level as an intervention protocol. However, while we will initially consider these data on a continuum, we are also prepared to use categorical variables which will allow further exploration of these components in a meaningful way. These further analyses will be informed by the findings of earlier questions and will control for any important factors. For example, timing, intensity, frequency, duration and dosage of an intervention will be analysed as continuous variables or, if the data fall into natural categories, as categorical variables. Other aspects of therapy will be considered as present or not present. These include augmentation of dose with prescribed home-based practice and tailoring of an intervention by difficulty or functional relevance.

Network meta-analysis

Network meta-analysis approaches are a specific approach to meta-analysis which allows for an estimate of the difference between direct and indirect comparisons. For example, one study compares treatment A with treatment B; and another study compares treatment A with treatment C. Using a network meta-analysis approach the data from these studies can be combined to compare A versus B and A versus C (direct comparisons) but can also give an estimate of treatment B versus C (indirect comparison).

We will compare a range of SLT interventions (for example, semantic approaches to SLT compared to phonological approaches). We will also examine impairment-based and activity/participation-based approaches. Based on our previous consensus work (Rose et al., 2018). we will consider three therapy perspectives:

• (a)the role of the intervention within the study design (e.g., usual care or social support as a comparison control versus therapy as the experimental intervention);
• (b)Therapy approach defined by impairment target (e.g., rehabilitation of spoken language production); and
• (c)therapy defined by a theoretical approach (e.g., constraint induced aphasia therapy).

We do not plan to explore the broad groupings of SLT or social support as we have already addressed these questions at group summary statistics level (Brady et al., 2016).

IPD network-meta-analysis

Previous aphasia meta-analysis and network meta-analysis have typically used group-level aggregated summary statistics to provide a helpful overview of the evidence (e.g., Bhogal, Teasell, and Speechley, 2003; Brady et al., 2016). However, this approach also carries a risk of ecological bias and confounding. RELEASE aims to use pre-existing IPD to explore some of the differences in the delivery of therapy components (SLT Intervention in Table 1) in relation to specific participants’ profiles and language recovery. A large aphasia IPD dataset will permit exploration of the highly heterogeneous nature of aphasia after stroke, individual level covariates’ influence on SLT treatment effects across language domains and to control for individualistic predictors. Feasibility will be contingent on the number of trials and IPD available, as well as whether any of the treatment aspects are predictors of improvement.

An IPD network meta-analysis can be undertaken using either a one or two-stage approach. A two-stage approach is similar to a standard meta-analysis approach; first, the IPD is processed centrally, aggregate data is generated for each dataset contributed (instead of using the primary research team’s reported summary statistics) and then the aggregated data is meta-analysed. This approach can, however, lead to bias in effects, greater heterogeneity and lower power to detect associations between language outcomes and continuous variables (Debray et al., 2018).

A one-stage IPD network meta-analysis approach combines all available IPD from across all datasets. In the context of RELEASE, the datasets will have been contributed by the primary research teams or will have been extracted from the public domain. The relevant data items are then selected for each planned analysis and put into a single model. A key methodological benefit of this approach is that confounding can be addressed as the impact of several (participant and language) variables on an intervention effect can be examined at the same time. The RELEASE study will adopt a one-stage network meta-analysis approach.

Confidence in cumulative evidence

We will review the quality of the data contributing to our analyses and consider the impact it may have on the confidence we have in the cumulative results. Where appropriate we will apply grading of recommendations, assessments, development and evaluation (GRADE) approaches (Guyatt, Oxman, Schunemann, Tugwell, & Knottnerus, 2011) or similar tools to reach a judgement about the quality of evidence on our findings.

Discussion

We plan to undertake an IPD meta-analysis and network meta-analysis to explore language recovery and the effects of specific SLT approaches on aphasia and prognostic factors. We anticipate that this project will serve to highlight areas of greater certainty as well as uncertainty in relation to language recovery and components of effective rehabilitation for aphasia after stroke. Continuing gaps in our knowledge will assist in prioritising future aphasia research and the design of those research activities. Our findings will also be useful to clinicians, who need evidence-based guidance to offer and tailor interventions to their clients’ needs. In addition, the project will highlight the importance of high-quality design and reporting of participant demographics, prognostic factors and intervention details in the context of aphasia after stroke. RELEASE will also generate a legacy database which will be supported in the future by the wider Collaboration of Aphasia Trialists who will moderate access to this resource.

Disclosure statement

Marian Brady reports grants from National Institute for Health Research (NIHR), Health Services and Delivery Research Programme (HS&DR–14/04/22), grants from Chief Scientist Office (CSO), Scottish Government Health and Social Care Directorates, grants from EU Cooperation in Science and Technology (COST) funded Collaboration of Aphasia Scientists (IS1208 www.aphasiatrials.org), and grants from The Tavistock Trust for Aphasia during the conduct of the study. Audrey Bowen reports that data from her research is included within the analyses in the RELEASE report. Her post at the University of Manchester is partly funded by research grants and personal awards from NIHR and Stroke Association. Caterina Breitenstein reports grants from the German Federal Ministry of Education and Research (BMBF) during the conduct of the study. Erin Godecke reports Western Australian State Health Research Advisory Council (SHRAC) Research Translation Project Grants RSD-02720; 2008/2009 during the conduct of the study. Neil Hawkins reports grants from National Institute for Health Research during the conduct of the study. Katerina Hilari reports grants from The Stroke Association, grants from European Social Fund and Greek National Strategic Reference Framework, and grants from The Tavistock Trust for Aphasia outside the submitted work. Petra Jaecks reports a PhD grant from Weidmüller Stiftung. Brian MacWhinney reports grants from National Institutes of Health. Rebecca Marshall reports grants from National Institute of Deafness and Other Communication Disorders, NIH, USA during the conduct of the study. Rebecca Palmer reports grants from NIHR senior clinical academic lectureship, grants from NIHR HTA, and grants from Tavistock Trust for Aphasia outside the submitted work. Ilias Papathanasiou reports funding from European Social Fund and Greek National Strategic Reference Framework. Jerzy Szaflarski reports personal fees from SK Life Sciences, LivaNova Inc, Lundbeck, NeuroPace Inc, Upsher-Smith Laboratories, Inc, grants and personal fees from SAGE Pharmaceuticals, UCB Pharma, Biogen, Eisai Inc and GW Pharmaceuticals outside the submitted work. Shirley Thomas reports research grants from NIHR and The Stroke Association outside the submitted work. Ineke van der Meulen reports grants from Stichting Rotterdams Kinderrevalidatiefonds Adriaanstichting, other from Stichting Afasie Nederland, Stichting Coolsingel and Bohn Stafleu van Loghum during the conduct of the study. Linda Worrall reports a grant from the National Health and Medical Research Council of Australia. The remaining authors report no potential conflict of interest.