Comparative safety and effectiveness of cognitive enhancers for Alzheimer's dementia: protocol for a systematic review and individual patient data network meta-analysis

Abstract

Introduction

Alzheimer's dementia (AD) is the most common cause of dementia, and several organisations, such as the National Institute for Health and Care Excellence, suggest that management of patients with AD should be tailored to their needs. To date, little research has been conducted on the treatment effect in different subgroups of patients with AD. The aim of this study is to examine the comparative effectiveness and safety of cognitive enhancers for different patient characteristics.

Methods and analysis

We will update our previous literature search from January 2015 forward, using the same terms and electronic databases (eg, MEDLINE) from our previous review. We will additionally search grey literature and scan the reference lists of the included studies. Randomised clinical trials of any duration conducted at any time comparing cognitive enhancers alone or in any combination against other cognitive enhancers, or placebo in adults with AD will be eligible. The outcomes of interest are cognition according to the Mini-Mental State Examination, and overall serious adverse events. For each outcome and treatment comparison, we will perform a Bayesian hierarchical random-effects meta-analysis combining the individual patient data (IPD) from each eligible study. If the identified treatment comparisons form a connected network diagram, we will perform an IPD network meta-analysis (NMA) to estimate subgroup effects for patients with different characteristics, such as AD severity and sex. We will combine aggregated data from studies that we will not be able to obtain IPD, with the IPD provided by the original authors, in a single model. We will use the PRISMA-IPD and PRISMA-NMA statements to report our findings.

Ethics and dissemination

The findings of this study will be of interest to stakeholders, including decision makers, guideline developers, clinicians, methodologists and patients, and they will help to improve guidelines for the management of patients with AD.

Trial registration number

CRD42015023507.

Strengths and limitations of this study.

• This study will be the first network meta-analysis (NMA) using individual patient data (IPD) evaluating the comparative effectiveness and safety of cognitive enhancers for different patient characteristics, such as Alzheimer's dementia severity and sex.

• The outputs of this study will provide clinicians, patients and caregivers with tailored evidence to inform their decision-making.

• Although our IPD-NMA can be informed by observational studies providing data on adverse drug events, we will restrict to randomised clinical trials as this study design is the gold standard for examining interventions and there are numerous clinical trials available on this topic.

• A potential difficulty in the conduct of our study is that IPD can only be obtained by contacting the original trial authors. To overcome this difficulty and improve the response rate, we will use validated approaches suggested for electronic surveys and provide a cash incentive to each author.

Introduction

Alzheimer's dementia (AD) is the most common cause of dementia, and has an insidious onset with progressive deterioration in cognition (eg, memory, thinking and perception), function, behaviour and mood. To date, 46.8 million people worldwide live with dementia. This number will almost double every 20 years, and it is estimated to reach 131.5 million by 2050.¹ As dementia progresses, it impacts quality of life for the individual and causes a substantial burden on the family, caregivers, healthcare system and society. AD ultimately leads to death with a median survival from diagnosis of only 7 years.² A recent study showed that as age increases, the rates of AD increase overall for both men and women, but it is more prevalent in women (rate/100 years=2.50 (1.85–3.41)) than men (rate/100 years=1.89 (1.22–2.94)).³ Pharmacological treatment consists of cognitive enhancers, including the cholinesterase inhibitors (donepezil, galantamine and rivastigmine), and memantine, a N-methyl-d-aspartic acid receptor antagonist.⁴ It is currently unclear as to whether galantamine, rivastigmine or donepezil should be used by patients with severe AD, and whether memantine is the most optimal treatment for severe AD, which is the patient population in most need of medication.⁵ It has been shown that the use of acetylcholinesterase inhibitors and increased doses of donepezil in patients with dementia increase the risk of bradycardia, as well, cholinesterase inhibitors doubles the risk of hospitalisation for bradycardia in older patients.^{6 7} Also, the use of other medications may increase risk of adverse events. For example, cardiac medications like β-blockers may increase risk of bradycardia, and anti-inflammatories may increase risk for gastrointestinal bleeding.^{6 8–10}

To determine the relative effectiveness of cognitive enhancers for patients with different patient characteristics (eg, mild-moderate AD vs severe AD, females vs males), we aim to conduct a systematic review and individual patient data (IPD) network meta-analysis (NMA). NMA is an extension of pairwise meta-analysis and is the statistical method that combines different sources of evidence from a network of randomised clinical trials (RCTs) comparing different treatments for the same clinical topic within the same model. A NMA model can provide estimated treatment effects even for treatments that have never been directly compared in a head-to-head study. A key assumption in NMA is the transitivity assumption, which requires the balance of the distribution of potential effect modifiers across the treatment comparisons.^11–13 In AD, patients may respond differently to the medication based on severity of AD and sex, and hence severity and sex could be considered treatment effect modifiers. The optimal approach to assess the transitivity assumption is to compare the patient-level characteristics using IPD across treatment comparisons. Under the transitivity assumption, an IPD-NMA may tailor results to the patient characteristics. Tailoring the management of patients with AD is an issue that has been also brought up by several organisations,¹⁴ including the Alzheimer's Society of Ontario¹⁵ and the National Institute for Health and Care Excellence (NICE).⁴ Also, the Alzheimer's Disease International (ADI) federation in their world Alzheimer report 2015 mention that there has been dramatically little research into the treatment effect across people of different age and sex.¹

The use of aggregated data reported in RCTs does not always allow us to reach a definitive conclusion on which medication is the safest or most effective for patients with different severities of AD and for females/males. This is because the covariates of interest (eg, sex, severity of disease) are inconsistently reported in RCTs and a relationship at the aggregated study level is not necessarily true at the individual patient level. Indeed, we previously attempted a systematic review and NMA of aggregated data and we were unable to provide definitive conclusions regarding the influence of patient characteristics on the results.^{16 17}

The NMA results of our previous unpublished study were tailored to age, AD severity, comorbidity and study duration via subgroup analysis. These results were similar to four Cochrane reviews examining cognitive enhancers for AD.^18–21 Specifically, the reviews showed that all cholinesterase inhibitors, donepezil, rivastigmine and galantamine, significantly improved cognition^18–21 against placebo, yet cholinisterase inhibitors overall and donepezil improved behaviour,^{18 19} cholinisterase inhibitors overall and rivastigmine improved function,^{19 20} and rivastigmine improved AD severity.²⁰ These effects were associated with higher doses of rivastigmine,²⁰ suggesting that dose may be a treatment effect modifier. However, a (network) meta-analysis using aggregated data may suffer from relatively low statistical power for detecting a treatment-by-covariate interaction and introduces potential aggregation bias (also known as ecological fallacy).^22–24 This bias may occur if one (incorrectly) assumes that relationships observed at the group level hold at the individual level as well.^25–27 The use of IPD will help explain the relationship between treatment effects and patient-level characteristics, allowing healthcare providers to individualise the management of patients with AD (such as for patients with more severe AD or who are using medications such as β-blockers). In addition, in our previous NMA, we attempted a subgroup analysis for AD severity, but we were unable to infer on the treatment effectiveness for the severe AD subgroup because there were only few RCTs available that reported on patients with severe AD and a NMA was impossible (disconnected network). The advantage of IPD is that we are not restricted to using the information reported in the publication. For example, for the 15 RCTs that did not report severity of disease in patients, we will be able to include them in the IPD-NMA analysis. Also, we will be able to use the information on severe AD from studies that included patients ranging from mild-to-severe and moderate-to-severe disease. This will help increase power in our analysis compared with the aggregated data NMA. However, it should be noted that although IPD may increase power for the identification of treatment-by-covariate interactions, it has been shown that the studies usually included in a meta-analysis are underpowered themselves.²⁸

The aim of this study is to examine the comparative effectiveness and safety of cognitive enhancers versus placebo or best supportive care by patient characteristics, such as AD severity and sex. We will use IPD-NMA to identify potential treatment effect modifiers, and estimate the most effective and safest treatments for patients with different characteristics. We will combine aggregated data from studies that we are not able to obtain IPD, with the IPD obtained from authors who provide these data. Recent simulations have shown that adding IPD to AD studies in a NMA can significantly improve precision, reduce bias and increase information compared with NMA relying on aggregated data alone.²⁹

Methods and analysis

This systematic review and IPD-NMA protocol was prepared according to the preferred reporting items for systematic reviews and meta-analyses protocols (PRISMA-P) guidelines,³⁰ and was registered with the international prospective register of systematic reviews (PROSPERO; Registration #CRD42015023507).

Eligibility criteria

The research question and protocol are based on our previous systematic review and NMA.¹⁷ Therefore, we will update our previous systematic review,¹⁷ and we will use similar population, interventions, comparators, study designs and time period (PICOST) criteria. Eligible studies are RCTs including adults with AD administered a cognitive enhancer compared with each other, best supportive care, or placebo. The specific PICOST criteria are:

• Population: Adults (aged ≥18 years) with AD diagnosed using various criteria (eg, Diagnostic and Statistical Manual of Mental Disorders, Nursing Minimum Data Set criteria) of any duration with either moderate AD, that is, Mini-Mental State Examination (MMSE) of 10–20 or severe AD, that is, MMSE<10.³¹ These criteria have changed over time and we will record how the authors define AD severity for each study.

• Interventions: Cognitive enhancers (donepezil, rivastigmine, galantamine and memantine) alone or in any combination.

• Comparators: Cognitive enhancers, best supportive care alone or in any combination, and placebo.

• Outcomes: The primary outcome of interest is cognition according to the MMSE (efficacy outcome, continuous variable), and the secondary outcome is overall serious adverse events (SAEs; safety outcome, dichotomous variable); both outcomes were reported by many of the included trials previously and for which NMA was possible. In particular, in our previous NMA using aggregated data, 60 RCTs with 15 862 patients contributed to a NMA for the MMSE outcome, and 51 RCTs with 19 329 patients contributed to a NMA for SAEs.

• Study design: We will restrict to RCTs, as this is the gold standard for examining interventions.³² We will exclude quasi-RCTs, that is, quasi-random methods used to allocate patients to groups, such as consecutive allocation. Observational studies may provide data on safety, but these typically rely on administrative data and it is challenging to obtain sufficient information on individual patient characteristics.

• Time: Studies of any duration conducted at any time.

• Other: Unpublished and published studies written in any language will be included.

Search strategy and study selection

We will update our literature search from 5 January 2015 onwards using terms from our previous review¹⁷ in MEDLINE (OVID interface, 5 January 2015 onwards), the Cochrane Central Register of Controlled Trials (CENTRAL; 5 January 2015), Embase (OVID interface, 5 January 2015 onwards). We will use the search strategy and literature search (as created by an experienced librarian, Dr. Laure Perrier, and peer reviewed using Peer Review of Electronic Search Strategies (PRESS)³³ by Ms Becky Skidmore). We present our literature search for MEDLINE in online appendix 1. Briefly, we will search reference lists of included studies and relevant reviews. Grey literature (ie, difficult to locate and unpublished studies) will be searched via trial registry websites (such as Public Health Agency of Canada, Health Canada, FDA, metaRegister of Controlled Trials) and conference abstracts (such as International Pharmaceutical Conference). Non-English articles will be translated to determine their inclusion. In case study publications report data from the same study group (ie, companion reports), we will include the most complete follow-up data and the other study will be used for online supplementary data.

We will use the Synthesi.SR tool³⁴ to screen citations and full-text articles. To ensure reliability, our team has previously conducted a pilot test using our eligibility criteria on a random sample of 50 titles and abstracts from the literature search results. When a high agreement (>90%) was reached, two team members screened each title and abstract for inclusion, independently (level 1 screening). After pilot-testing full-text screening criteria, pairs of reviewers independently reviewed the full text of potentially relevant articles (level 2 screening). Conflicts were resolved by discussion in both levels. In the update of our previous systematic review,¹⁷ we will not conduct a pilot test, but we will follow the same screening process. We will report the overall per cent agreement, as well reasons for study exclusion at both levels. The PRISMA flow diagram will be used to report the study selection.³⁵

Data abstraction

The data we plan to abstract include study characteristics (eg, year of publication), aggregated patient characteristics (eg, number of patients), outcome results (eg, MMSE, SAE) and source of funding (categorised as: funded/authored by an employee of a drug manufacturer or other commercial organisation, government-sponsored/non-profit organisations, including universities and hospitals, no funding, funding unclearly reported, and funding not reported).³⁶ We will also abstract the corresponding authors’ mail and email addresses, as well their phone number. Two reviewers will abstract data independently, and all conflicts will be resolved through discussion.

The corresponding authors’ contact information will be abstracted from the papers. For missing information, we will search authors’ online research profiles (eg, Google Scholar) or PubMed. We will use recommended approaches for electronic surveys to improve response rates.³⁷ Specifically, we will (1) send an email to the corresponding authors explaining the study purpose and requesting their data, enclosing a signed letter on letterhead; (2) send reminder emails at 2, 6, 10 and 14 weeks intervals after the initial email; (3) send a reminder by post in addition to email the 7th week; and (4) contact the corresponding author by phone during the 15th week. A financial incentive will be also offered to the corresponding author in the form of a $100 Amazon gift certificate. We will inform all authors that their article will be appropriately cited and, if they agree, they will be acknowledged in our paper. To ensure that we will be able to conduct this study, we will also contact clinical data sharing sites such as Clinical Study Data Request (CSDR) and Yale University Open Data Access (YODA) to obtain IPD on any of the eligible studies.

We will ask authors to provide IPD on: (1) patients, including age, sex, severity of Alzheimer's disease (eg, baseline MMSE level), presence of behavioural disturbance, comorbid conditions (eg, stroke, cardiovascular conditions, Parkinson's disease), other medications used for each patient (such as β-blockers and other antiarrhythmic drugs, as these can increase risk of adverse events, especially gastrointestinal side effects and bradycardia^{6 7}), drop-outs along with reasons for drop-out, and number of participants; (2) medication, including treatment patient was allocated, dosage; (3) outcomes, including event and date of event and time taken to achieve the event for SAEs, and MMSE values and measurement dates; and (4) date and method of randomisation. All IPD will be saved on a secure server, adhering to Personal Health Information Protection Act (PHIPA).³⁸

Risk of bias and quality appraisal

As with the original review, we will appraise the risk of bias using the Cochrane risk of bias tool.³² Two reviewers will independently assess the risk of bias in each included study after pilot testing on a random sample of five RCTs. Disagreements will be resolved by discussion. To ensure data consistency, as recommended by the PRISMA-IPD guidelines,³⁹ we will (1) compare IPD provided by the investigator with aggregate data reported in the publication; (2) assess whether the eligibility criteria of each study are in agreement with the IPD; (3) check date consistency, for example, date patient randomised versus date trial opened. We will also check whether the randomisation of patients is adequate (ie, intervention and comparison groups are balanced for important patient characteristics), by comparing numbers and types of patients in each arm. We will ask the author for clarifications, if inconsistencies are identified. Our IPD analysis will be based on the intention-to-treat principle including all previously excluded patients.

We will draw a comparison-adjusted funnel plot⁴⁰ for both the MMSE and SAE. This plot allows the examination of heterogeneity and different types of bias, such as selective reporting, publication and funding biases. After ordering the treatments included in the network chronologically regarding their year of availability on the market, we will plot the difference between each observed effect and overall treatment effect against the SE of the observed effect. The comparison-adjusted funnel plot will be used only when RCTs with two treatment arms are included in the analysis, as this method does not account for correlations induced by multi-arm trials and potential asymmetry in the plot can be masked. Whenever an eligible study includes multiple arms, we will construct funnel plots for each treatment comparison and outcome separately. Funnel plots for each treatment comparison will be plotted only when at least 10 RCTs are available. Reasons for funnel plot asymmetry will be explored. Two review authors will also independently assess the quality of evidence in each NMA using the GRADE approach as extended for NMA.⁴¹

Synthesis

The characteristics of the included studies, patients and treatments, as well as risk of bias of studies will be described irrespective of whether IPD is obtained. We will present summary statistics and potential outlier patient values to describe the outcome data in each study.

We will perform a Bayesian hierarchical random-effects meta-analysis for each treatment comparison, as we anticipate clinical and methodological between-study heterogeneity. All IPD from included studies will be combined into a single model using a multilevel model where each study is a different cluster. We will use the odds ratio for SAE⁴² and the mean difference effect size for MMSE.²⁷ In case we are able to obtain IPD for a subset of trials, then we will use a two-part model with the same between-study variance in both parts and accounting for treatment-by-covariate interactions (including, eg, comorbidities such as arrhythmias in the model⁴³). The first part will entail a one-stage model using IPD only, whereas the second part will entail applying a pairwise meta-analysis modelling aggregate data.⁴³

If the treatment comparisons that inform the eligible RCTs form a connected network of trials (see figure 1), the random-effects NMA model will be used in the primary analysis. If possible, we will combine information across a network of trials using only IPD. If we are not successful in obtaining IPD for at least one study, we will combine both IPD and aggregated data in a single model; this will allow the inclusion of all trials in the analysis. Information on patient-level covariates (eg, AD severity, sex, comorbidities, use of non-pharmacological interventions) received from the authors will be included in the model as secondary analyses. We will statistically evaluate whether the transitivity assumption is valid using the design-by-treatment interaction model.^{44 45} If statistical inconsistency is identified, we will perform the loop-specific method^{46 47} using aggregated data to locate the piece of the network responsible for the observed inconsistency. If these approaches suggest network inconsistency, we will check the data for discrepancies, and if none are identified, a subgroup or meta-regression analysis will be considered. The subgroup and meta-regression analyses will consider the potential treatment effect modifiers described in the ‘Data abstraction’ section.

Figure 1.

Network diagrams for (A) Mini-Mental State Examination (MMSE) and (B) serious adverse events outcomes, as published in our previous systematic review and network meta-analysis.¹⁷

We will estimate subgroup effects, including patient characteristics received from authors (eg, age, sex, severity of Alzheimer's disease, previous use of AD medications) using treatment-by-covariate interaction terms within studies and combining these across studies. Other subgroups will include study-level variables, such as intervention characteristics. We will apply three model specifications assuming that: (1) the regression coefficients are different and unrelated across comparisons; (2) the regression coefficients are different but related, sharing the same distribution; and (c) the regression coefficients are identical across comparisons.^{48 49} A common within-network between-study variance will be assumed across comparisons.⁵⁰ We will compare the results of the models by evaluating the statistical significance of the regression coefficients for interactions, monitoring the reduction in the between-study variance, and using the Deviance Information Criterion (DIC)⁵¹ to compare the overall fit and parsimony of the models. The model with the lowest DIC corresponds to the best-fitting model and a difference of three units or more is considered significant.⁵¹ We will use the IPD-NMA model with the best fit for our results and the other model results will be reported in an online appendix. The summary treatment effects will be presented using the odds ratios (ORs) or mean differences along with their corresponding credible intervals and predictive intervals.⁵² We will rank the interventions for each of the MMSE and SAE outcomes using the surface under the cumulative ranking curve.⁵³

We will conduct multiple sensitivity analyses to examine the robustness of our results. First, we will restrict to studies with IPD only. Second, we will use different priors for the between-study variance^54–56 Third, we will restrict to RCTs with a low risk of bias for sequence generation, allocation concealment and blinding components of the Cochrane risk of bias tool. Fourth, we will use imputation techniques for missing outcome data. In particular, for MMSE we will perform the ‘informative missingness difference of means’ method,⁵⁷ and for SAE we will apply the ‘informative missingness odds ratio’ method accounting for the uncertainty due to missing outcome and basing imputations on observed outcomes.⁵⁸

All analyses will be conducted using the Bayesian software OpenBUGS⁵⁹ with Markov Chain Monte Carlo (MCMC) samplers. Two chains will be generated and convergence will be evaluated by their mixing, after discarding the first 10 000 iterations. We will use non-informative priors for all parameters of the models apart from the between-study variance for which we will use the empirical distributions suggested by Turner et al⁵⁵ for dichotomous data and Rhodes et al⁵⁶ for continuous data. We will present our findings in accordance with the PRISMA extension for NMA⁶⁰ and PRISMA extension for IPD.³⁹

Ethics and dissemination

To the best of our knowledge, this study will be the first IPD-NMA examining the comparative effectiveness and safety of cognitive enhancers versus placebo or best supportive care by AD severity and sex. Such an analysis may be more powerful in comparison with the NMA using aggregated data, and will allow healthcare providers to individualise the management of patients with AD. The findings of our study will fill an important knowledge gap in healthcare, and will be used to inform decision-making for patients suffering from this debilitating disease.

The results of this systematic review and IPD-NMA will be of interest to stakeholders, including decision makers, guideline developers, clinicians, methodologists and patients. The dissemination of our findings will be knowledge user-driven and tailored to how and when knowledge users want to receive information. Team members will act as knowledge brokers, using their networks to facilitate dissemination, such as The Cochrane Collaboration, PRISMA-IPD, Drug Safety and Effectiveness Network (DSEN). We will also host a knowledge exchange event with our partners to discuss the results and facilitate dissemination. We will publish our findings in an open access journal, and present them at relevant meetings (Canadian Geriatrics Society; CGS), as well to newsletters of organisations (Alzheimer's Society of Ontario, CGS).

There is a challenge to our study that is worth noting. Our data set relies on the authors’ willingness to share their data.⁶¹ However, we have extensive experience contacting authors, as it is a regular process to ask for additional data on included studies during the systematic review conduct, and we have a good response rate (on average >60%). The additional offer of $100 incentive will help us improve the response rates. We will also contact clinical data sharing sites such as CSDR and YODA for data on any of our included studies. If we are unable to obtain IPD for all studies included in the systematic review, we will combine both IPD and aggregated data (as reported in the study publication) in the analyses. This is because it has been suggested that combining IPD with aggregate data minimises the chances of confounding bias in aggregate data NMA.^{29 62}

The IPD-NMA does not require ethical approval, as it synthesises data from clinical trials (and informed consent was already obtained for the original study). We will only request anonymised data from the authors, and we will link each patient to a specific identifier to prevent the patient from being identified.