Rates of retention of persons with a mental health disorder in outpatient smoking cessation and reduction trials, and associated factors: protocol for a systematic review and meta-analysis

Abstract

Introduction

Smoking among persons with a mental health disorder is associated with inequitable health, social and economic burden. Randomised controlled trials (RCTs) are considered the gold standard design for the assessment of healthcare intervention efficacy/effectiveness. However, many RCTs of smoking interventions for persons with a mental health disorder lack rigour due to low participant retention. No systematic review has pooled retention rates in randomised trials of smoking interventions for persons with a mental health disorder or explored associated factors. The aims of the systematic review will therefore be to: (1) summarise overall rates of participant retention in smoking cessation and reduction trials involving persons with a mental health disorder (including for experimental and control groups separately) and (2) determine if retention rates vary according to participant, environmental, researcher and study factors.

Methods and analysis

PsycINFO, EMBASE, MEDLINE, CENTRAL and The Cochrane Tobacco Addiction Review Group Specialised Register will be searched for reports of RCTs of outpatient smoking cessation or reduction interventions for adults with a mental health disorder. The search terms will include MeSH terms and free text words, and there will be no language or date restrictions. All databases will be searched from inception to present. Data will be analysed using the Mantel-Haenszel fixed-effect model, and where substantial heterogeneity (I² >50%) is detected, DerSimonian & Laird inverse-variance random effects model. Pooled estimates and 95% CIs will be calculated for overall participant retention rates and for intervention and control trial arms separately. Associations between participant retention and participant, environmental, researcher and study factors will be assessed via subgroup analyses and, where sufficient data are obtained, meta-regression.

Ethics and dissemination

This study does not require ethical approval. The findings of this review will be disseminated via publication in a peer-reviewed open access medical journal and presentations at international scientific meetings.

Strengths and limitations of this study.

• This will be the first systematic review to pool retention rates in randomised controlled trials of outpatient smoking interventions for persons with a mental health disorder and explore associated factors.

• Identification of participant, environmental, researcher and study factors associated with retention will be informative when designing future smoking intervention trials for persons with a mental health disorder and may lead to increased retention and, in turn, rigour of research in the field.

• It will employ Cochrane’s methods for the conduct of systematic reviews.

• Trials of smoking interventions delivered exclusively in psychiatric inpatient setting will not be included.

Introduction

Tobacco smoking is a leading cause of preventable morbidity and mortality worldwide.¹ Smoking prevalence has steadily declined to between 13% and 20% over the past 40 years in the general population of high-income countries;^2–4 however, it has remained unchanged among persons with a mental health disorder.^{5 6} Population surveys suggest that 40%, 44% and 36% of community residing persons with a mental health disorder in the USA,⁷ UK⁸ and Australia⁷ smoke, respectively, with prevalence estimates shown to increase as mental illness severity and number of life time disorders increase.^{9 10} These groups are also reported to smoke more heavily and be more nicotine dependent than smokers without a mental health disorder^9–11 and have been estimated to consume up to 45% of all cigarettes sold in high-income countries.^{9 12 13} Smoking among persons with a mental health disorder is consequently associated with inequitable health, social and economic burden.¹⁴ Despite this, many high-quality randomised trials of smoking interventions exclude persons with a mental health disorder^{15 16} and those focused on this population group often lack rigour due to, for example, small sample size and low participant retention.^17–19 More rigorous intervention research is needed to address this international public health issue.^{20 21}

Randomised controlled trials (RCTs) are considered the gold standard design for assessment of healthcare intervention efficacy and effectiveness.²² In any trial, low rates of participant retention for follow-up data collection can compromise the internal and external validity and reduce statistical power.^23–25 Threats to internal validity are particularly salient as retention decreases (<80%²⁵) or is differential between allocation groups,²⁶ increasing the risk of bias and confounding.²⁶ Additionally, statistical methods (eg, multiple imputation) cannot account entirely for the impact of low/differential retention and have been noted to occasionally be used inappropriately.²⁷ Illustrating a consensus regarding the need to consider and address the issue of low retention in clinical and health behaviour trials: the inclusion of items on participant retention in the Consolidated Standards of Reporting Trials (CONSORT) statement,^{28 29} and prioritisation of trial methodology research to develop novel approaches to increase retention in randomised trials.^{30 31} In addition, the emergence of recent systematic reviews examining retention rates in health behaviour studies^32–36 and publication of retention data from high quality randomised trials^37–39 indicates increased interest in the topic of retention and a recognition of its importance.

Participant retention is problematic in trials of smoking cessation interventions,^{40 41} including those involving persons with a mental health disorder.^17–19 For example, review of risk of bias assessments in systematic reviews of smoking cessation intervention trials involving persons with depression¹⁷ and substance use disorders¹⁹ revealed retention rates as low as 27%–33%, at trial end points. In addition, 65 of the trials (34/49;¹⁷ 31/35¹⁹) included in the reviews transparently reported on participant retention, of which 40 (22/34;¹⁷ 18/31¹⁹) achieved retention rates less than 80%. In addition, there was evidence of differential retention (≥10% difference in retention rates between allocation groups) in 13 (5/34;¹⁷ 8/31¹⁹) trials, with this figure likely an underestimate due to non-systematic reporting of retention data according to allocation group. No study has systematically and quantitatively summarised retention rates of persons with a mental health disorder in smoking trials or considered factors that may impact on such rates.

Factors likely to impact participant retention in any trial could be categorised into four types: (1) ‘participant’ (eg, demographic information) (2) ‘environmental’ (eg, recruitment method/setting) (3) ‘researcher’ (eg, staff qualifications) and (4) ‘study’ (eg, trial design).⁴² In terms of participant factors, evidence from single studies in the field of smoking and mental illness suggests younger age, higher income, less severe mental health symptomology and readiness to quit at baseline may be associated with higher retention in trials.37 39 43 44 While no research has reported on environmental and researcher factors associated with retention in smoking trials involving persons with a mental health disorder, findings from the broader field of smoking research suggest that higher retention rates are observed when proactive (compared with reactive) recruitment methods are adopted,⁴⁰ and staff delivering the intervention receive more comprehensive training.²⁹ With regard to study factors, Cochrane systematic review evidence suggests open-label designs increase the relative risk of higher retention rates (risk ratio (RR) 1.37; 95% CI 1.16 to 1.63) in randomised trials of health interventions outside the field of smoking.³³ Further, adoption of strategies with demonstrated effectiveness in improving retention in health research could also be considered a ‘study’ factor. Such strategies were developed based on identified barriers to participant retention³² and include monetary reimbursement for time,⁴⁵ provision of reminders for follow-up assessments⁴⁶ and participant tracking systems.⁴⁶ Identification of participant, environmental, researcher and study factors associated with higher participant retention in smoking trials involving persons with a mental health disorder will be informative for researchers when designing definitive trials and in turn potentially improve the rigour of available intervention research in the field.

No review has systematically examined rates of participant retention or explored associated factors in smoking cessation and/or reduction trials involving persons with a mental health disorder. Therefore, the aims of the proposed systematic review will be to: (1) summarise overall rates of participant retention in such trials (including for experimental and control groups separately) and (2) determine if retention rates vary according to participant, environmental, researcher and study factors.

Methods

This protocol adheres to the preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) statement.⁴⁷

Inclusion criteria

Studies identified by a Cochrane review of outpatient smoking cessation and reduction interventions for persons with a mental health disorder⁴⁸ will be included. Criteria for inclusion:

• Study design: RCTs and cluster RCTs.

• Participants: adult (aged 18 years or above) smokers, who have either been diagnosed with a mental health disorder (determined via medical record or self-report) in the past 12 months or are currently receiving treatment for the same. Participants will not be required to express an intention to quit smoking to be eligible. No limits will be placed on recruitment setting; however, the majority of the intervention component must be delivered in a community setting. Interventions delivered exclusively during an inpatient psychiatric admission will be excluded as they are the focus of another Cochrane review.⁴⁹

• Intervention type: any intervention to aid smoking cessation or reduction, for example, pharmacotherapy, psychoeducation, cognitive and behavioural therapies. Interventions aiming to increase the uptake/utilisation of smoking cessation supports will be considered. However, whole-of-setting interventions (eg, smoke-free policy) will be excluded.

• Intervention delivery: face-to-face, telephone, online, mail, individual and/or group. Interventions can be facilitated by any person (eg, research officer or healthcare clinician) or via unmoderated online/phone-based methods.

• Control: all control conditions will be considered, for example, placebo, no treatment, usual care, other smoking cessation/reduction interventions.

• Follow-up: undertaken at least 6 months postbaseline assessment.⁵⁰

Search strategy

The following databases will be searched for reports of trials of smoking cessation and reduction interventions among smokers with a mental health disorder:

• The Cochrane Tobacco Addiction Review Group Specialised Register.

• Cochrane Central Register of Controlled Trials (CENTRAL).

• MEDLINE (OVID SP 1946 to present) & MEDLINE in-process & other non-indexed citations (OVID SP).

• EMBASE (OVID SP, 1947 to present).

• PsycINFO (OVID SP,1806 to present).

Search terms will include MeSH terms and free text words, and there will be no language or date restrictions. All databases will be searched from inception to present. Reference lists of included studies will be checked for other relevant research. Online supplementary appendix 1 comprises the proposed Medline search strategy. Searches will likely be undertaken in December 2019.

In addition to the above, we will search international clinical trials registers for recently completed trials, including: the Australian and New Zealand Clinical Trials Registry (ANZCTR), UK Clinical Trials Gateway; US Clinical Trials Register and the WHO Portal.

Study screening and data extraction

Two review authors will independently screen the titles and abstracts of search results for relevance, acquire and screen the full texts of potentially eligible articles and extract data from included studies. Disagreements during the screening and data extraction processes will be resolved by referral to a third review author. Covidence (https://www.covidence.org/home) and Microsoft Excel will be used to manage data during screening and extraction phases.

Data to be extracted from included trials are outlined in table 1, with independent variables for consideration identified in the respective column and summarised under relevant headings below. A large number of potential independent variables could be considered, selection will be guided by extant literature, experts in the field and, with exception of ‘participant factors’, focus on factors modifiable by researchers.

Table 1.

Data extraction fields

Study details	Extraction format	Considered as an independent variable
Author	First author last name
Year	Year published
Title	Title of publication
Country	From text
Years of study	Year/s study was conducted
Funding source	From text
Conflict of interests	From text
Methods
Study design	RCT, cluster RCT
Aims of study	From text
Years of recruitment	Years
Method of recruitment	Face-to-face, social media, random-digit dialling;; snowball recruiting; use of social networks; advertisements, media or notices; identification of potential participants through public registries, medication record and so on	Yes
Recruitment setting	Hospital, community healthcare, primary care, non-government organisation (NGO), other	Yes
Recruiter qualifications, training and professional role		Yes
Familiarity of recruiter to participant	Unknown (eg, external researcher), limited previous contact (eg, staff member in hospital), regular clinician, peer, other	Yes
Retention strategy employed	Yes/no; type	Yes
Participants
Age	Mean (standard deviation), median (range)	Yes
Gender	Male, female, other/unspecified.	Yes
Race/ethnicity	From text	Yes
Socio-economic status	Employment status, level of education or similar	Yes
Mental health diagnosis	Type of mental illness/es of participants	Yes
Severity of distress/ mental health disorder	DASS-21, Kessler 10, Global Assessment of Functioning (GAF) or similar measures	Yes
Inclusion/exclusion criteria	From text
Current smoker definition	Daily, weekly, occasional smoker	Yes
Cigarette consumption (baseline)	Mean (standard deviation), median (range)	Yes
Nicotine dependence (baseline)	Fagerstrom Test of Nicotine Dependence (FTND) or similar measures	Yes
Motivation to quit (baseline)	Readiness and Motivation to Quit Smoking Questionnaire or similar measures	Yes
Requirement to set a quit date on recruitment	Yes/no	Yes
Intervention
Treatment type	Pharmacotherapy, psychoeducation, cognitive and behavioural therapies, staff training, other
Dosage (if applicable)
Duration	Weeks
Number of session (where applicable)
Facilitator qualifications and training	From text
Mode of delivery	Face-to-face, telephone, online, mail, individual and/or group, other
Description of comparator	Placebo, no treatment, waitlist, usual care, other smoking cessation/reduction intervention, other
Outcomes
Primary outcomes	Specified and collected
Secondary outcomes	Specified and collected
Definition of abstinence	From text
Definition of smoking reduction	From text
Type of biochemical validation (if applicable)	Carbon monoxide (CO), cotinine, other	Yes
Timing of follow-up assessments	Months	Yes
Mode of follow-up assessment	Face-to-face, telephone interview, postal or electronic survey and so on	Yes
Follow-up setting (if applicable)		Yes
Potential costs to participant associated with follow-up assessments	Absence from work, fuel and parking costs and so on	Yes
Intention to treat and per protocol analyses conducted	Yes/no
Risk of bias
Method of randomised sequence generation
Method of allocation concealment
Blinding of participants	Yes/no	Yes
Blinding of research staff	Yes/no	Yes
Blinding of outcome assessors	Yes/no	Yes
Levels of loss to follow-up
Reporting of loss to follow-up
GRADE	GRADE methodology rating

Participant factors

Participant characteristics for consideration, collected at the baseline assessment of individual studies, may include: age,^{39 40} gender,^{40 43} race/ethnicity,⁴⁰ socioeconomic status,⁴⁰ mental health diagnosis,³⁷ severity of distress,¹³ motivation/readiness to quit smoking,^{40 44} nicotine dependence,^{30 40} daily cigarette consumption^{40 49} and requirement to set a quit date on recruitment.⁴⁰

Environmental factors

Environmental factors comprise those related to the study setting. Such factors that may be considered include the method⁴⁰ and setting⁵¹ of recruitment and outcome data collection.

Researcher factors

Researcher factors include characteristics, skills and roles of the staff that design and implement the study and may include: recruiter qualifications, training and professional role^{51 52} and relationship of recruiter to participants.⁵¹

Study factors

Study factors comprise features of the study design and implications of such for participants. A number of study factors may be considered: blinding/masking of research personnel,³³ adoption of retention strategies,^{32 37} timing of follow-up assessments,^{39 40} biochemical validation of self-reported abstinence and potential cost/s to participants of follow-up assessment completion.^{32 51}

Outcomes

The primary outcome of interest for the review is participant retention rate at the longest follow-up. Within each study, retention will be defined as participation in the final follow-up assessment of a trial (eg, completion of telephone or face-to-face interview or response to postal and electronic questionnaires). Retention rate will be calculated using the formula: number retained/number recruited. Study authors will be approached to obtain data required to calculate outcome measures, if not reported in the published manuscript. Secondary analyses will explore follow-up rates separately at other time points, for example, at 6 months, 12 months and 18 months.

Analysis

Data will be analysed using Review Manager 5.3 and Stata 15.0. Outcome data will be collected in accordance with intention-to-treat principles, where all recruited participants will be included in denominator figure of the retention algorithm. Data that are missing due to participant death will be deducted from the total sample size for all analyses.

According to study aim

Aim 1: the proportion of participants retained will be pooled using the Stata 15.0 prevalence command, ‘metaprop’ using the Freeman-Tukey double arcsine transformation to stabilise variances and prevent exclusion of studies where proportions approached 0 or 1.^{53 54} We will calculate pooled estimates and 95% CIs for overall participant retention rates as well as for intervention and control trial arms separately. Differences between intervention and control arms will be assessed via risk ratios. Data will be analysed using the Mantel-Haenszel fixed-effect model. If substantial heterogeneity (I²>50%)⁵⁰ is detected in the pooled estimates, we will run sensitivity analyses using DerSimonian & Laird inverse-variance random effects meta-analysis.⁵⁵

Aim 2: the association between participant, environmental, researcher and study factors and overall participant retention will be assessed via subgroup analyses and, where sufficient data are obtained (>10 observations),⁵⁰ meta-regression. Subgroup analyses will be undertaken in line with the meta-analysis approach described for Aim 1. Independent variables (table 1) will be categorised and reduced to two or three levels for the purpose of this analysis. In terms of the meta-regression, we will build models using the ‘regress’ and, where relevant, ‘metareg’ commands. Study level weights will be obtained from the fixed effects meta-analysis and will be included in the regression models using the analytical weights option. Categorical and numerical independent variables to be considered in these models. Ratio data will be natural log transformed prior to analyses. Where data are not amenable to meta-analysis, factors reported in individual studies to be associated with retention will be summarised via narrative synthesis.

Statistical heterogeneity and sensitivity analyses

The I² statistic, which describes the percentage of total variation across studies that is due to heterogeneity rather than chance, will be used to assess statistical heterogeneity of pooled data. Heterogeneity in pooled estimates will be described as low (I ²~25%), moderate (I ² ~50%) or high (I ² ~75%).⁵⁶ Sensitivity analyses will be conducted to explore the impact of including studies with an overall high risk of bias.

Patient and public involvement

Patients/public were not involved at any stage in the development of this protocol.

Discussion

This rigorous systematic review will be the first to pool retention rates in RCTs of outpatient smoking interventions for persons with a mental health disorder. Identification of participant, environmental, researcher and study factors associated with retention will be informative when designing future smoking intervention trials for persons with a mental health disorder, and in turn, may lead to increased rigour of research in the field due to higher participant retention. Findings may not generalise to trials of smoking interventions delivered exclusively in psychiatric inpatient settings.