Skip to main content
BMJ Open logoLink to BMJ Open
. 2021 Mar 5;11(3):e042888. doi: 10.1136/bmjopen-2020-042888

Using financial incentives to increase physical activity among employees as a strategy of workplace health promotion: protocol for a systematic review

Thomas Leonhard Heise 1,2,, Jennifer Frense 1, Lara Christianson 3, Till Seuring 4
PMCID: PMC7938995  PMID: 33674371

Abstract

Introduction

Shifts towards sedentary occupations in high-income countries (HIC) over the last decades and the health burden associated with physical inactivity have led to innovative strategies to promote physical activity (PA) among the working population. Financial rewards have been proposed to incentivise participation in programmes promoting PA or the achievement of PA-related goals. This review will assess (1) effects of financial incentives provided by employers to promote PA on programme adherence/compliance, PA and closely linked outcomes (eg, days of sick leave), (2) effects according to age, gender, implementation and study quality and (3) intervention costs and cost-effectiveness.

Methods and analysis

We will search for relevant studies in eight academic databases, two grey literature resources, two trial registers and on five organisational websites (oldest record/content to present). We will include (cluster) randomised controlled studies with a financial incentive to promote PA as intervention (≥1 months), conducted among employees in HIC and reporting data on at least one primary outcome in stages of screening (title/abstract, full text). To assess study quality and potential bias, we will use the revised Cochrane risk-of-bias tool (RoB 2). We will extract study data into prepiloted extraction sheets. Each task in screening, quality assessment and data extraction will be done by two authors independently. If a sufficient number of studies provide homogeneous data (ie, similar follow-up) for primary outcomes, meta-analyses will be carried out. We will report GRADE ratings to provide information on the certainty of the evidence.

Ethics and dissemination

For this review, no ethical approval will be required because only data of studies in which informed consent was obtained will be considered and analysed. The final review manuscript will be published in an Open Access journal. To ensure effective promotion of this review project, we will disseminate major findings through relevant communication channels.

PROSPERO registration number

CRD42020184345.

Keywords: preventive medicine, health policy, health economics


Strengths and limitations of this study.

  • This protocol outlines methods of an upcoming systematic review to investigate effects of financial incentives to increase physical activity (PA) among employees as a strategy of workplace health promotion.

  • Existing reviews on the use of financial incentives have not focused exclusively on the workplace setting—a setting where employees in high-income countries often remain sedentary for the majority of the workday.

  • Focusing on the emerging evidence on the use of financial incentives for increasing PA in the workplace will allow definitive conclusions to be drawn about their effectiveness in this setting.

  • Methods include a clear definition for financial incentives in this context, other relevant eligibility criteria (eg, (cluster) randomised controlled trial as included study designs) and the overall search approach for identifying studies.

  • Cross-country differences in private and institutional support to increase PA in the workplace setting as well as funding opportunities to finance incentive schemes may prevent formulation of universal recommendations for practice.

Introduction

Rationale

Physical inactivity is defined by WHO as an "insufficient physical activity level to meet present physical activity recommendations."1(p.vii) It is one of the most important behavioural risk factors for the development of chronic, non-communicable diseases and thereby remains a major societal problem, especially for high-income countries (HICs).2 Based on pooled estimates for HICs 36.8% of adults did not meet the ‘2010 WHO physical activity recommendations’ in 2016.3 According to WHO, adults can substantially improve their health status by regularly undertaking moderate-intensity aerobic physical activity (≥150–300 min per week) or vigorousintensity aerobic physical activity (≥75–150 min per week) or an adequate combination of both.1

Increasing physical activity levels can lead to improvements in physical and mental health and mortality reductions, in particular by decreasing the burden of cardiovascular disease, type 2 diabetes and depression.4–8 These diseases increasingly affect people of working age, likely also due to low levels of physical activity and high levels of sedentary behaviour at the workplace.9 Especially at increased risk are employees working in office environments or in the transportation sector. In 2017, the most frequent activity status of employees in the European Union while working was sitting (39%). In some countries (eg, Germany, Luxembourg, The Netherlands), one out of two employees predominantly carried out their work while sitting down.10 In the USA, private-sector jobs requiring at least moderate levels of physical activity dropped from 50% in 1960 to 20% in 2010. Levels of energy expenditure decreased accordingly, which has been linked to the observed increase in weight during the same period.11

The economic costs related to preventable chronic diseases are immense. They are caused by increasing healthcare needs, but also by reductions in productivity and increases in disease-related job exits.12 Hence, there is a need to increase physical activity among the working-age population. The workplace represents an important setting given the substantial amount of lifetime spent there. In 2018, workers in the Organisation for Economic Co-operation and Development member countries worked an average total of 1726 hours per year (approximately one-fifth of a year).13 Additionally, the commute to work is often carried out via car or public transport adding to the time spent being physically inactive due to employment-related activities, suggesting a potential to increase employment-related physical activity by promoting more active forms of commuting.14

Interventions that promote physical activity in the workplace setting can be of particular importance to at-risk groups with low levels of physical activity and high levels of sedentary behaviour during working hours. Evidence suggests that high levels of sedentary behaviour at work are rarely compensated for during leisure time.15–17

In many HICs, companies or employers (with or without the support of health insurance companies or funds) already provide access or subsidise physical activity programmes tailored to the specific needs of employees (eg, to encourage stair use, walking, active commuting, physical activity in the office), with mixed results on cost savings and cost-effectiveness.17–19 However, participation rates and programme adherence, especially among at-risk groups for sedentary behaviour, can remain low if participation is not actively supported or encouraged by the employer, even more so if activities are scheduled outside paid working hours.20 Financial incentives provide a potential way to encourage employees to actively participate in physical activity programmes or activities which ultimately could translate into increased levels of physical activity (ie, moderate and vigorous physical activity).

To decide if the use of financial incentives is a (cost-)effective strategy to increase physical activity in the working population, policy makers ideally require evidence from studies able to establish causal relationships, such as randomised controlled trials (RCTs), especially for quantification of a potential effect in a controlled best-case scenario.21

Existing reviews have assessed the effectiveness of financial incentives to promote physical activity.22–27 They have found moderate positive short-term effects of financial incentives on physical activity,22 25 27 exercise session adherence and exercise attendance,23–25 with mixed findings regarding long-term effectiveness (ie, after incentives were removed).22 24 26 However, so far none of these reviews have focused on the use of financial incentives in the workplace setting and overall included little evidence from workplace-based studies. A recent and comprehensive review by Luong et al,25 however, did show a significant increase in the evidence for the workplace setting over the last years. While this review again did not focus on the workplace setting alone but included evidence from many different settings and populations (eg, students, retirees, general population), it suggests that the available evidence base has increased substantially.25 The main contribution of this planned review is its focus on the workplace setting, which may differ substantially from other settings due to factors such as available income, time available for leisure activities, and age of the target population. The review will be able to provide a comprehensive overview of studies in this setting to draw conclusions about the effectiveness of financial incentives in workplace health promotion. To be able to more directly link changes in physical activity with monetary incentives, this review takes a narrower approach in the type of financial incentive it considers. Motivated by the real-life example from Germany, where public health insurers provide employers with the opportunity to use cash incentives to foster participation in workplace health promotion programmes,19 28 this review focuses on the specific case of positive financial incentives, that is, cash or cash-like payments for participating in a physical activity programme or the achievement of a predetermined static or adaptive physical activity goal (conditional incentive). This has two advantages: (1) Changes in physical activity can be directly attributed to a specific form of financial incentive and (2) it makes sure that the subjective and objective value of the incentive for participants is observable as it is directly measured in monetary units. This may not be the case for incentives based on vouchers for a specific item or lotteries, where in the case of vouchers for a specific item the subjective value may differ according to personal preferences, and in the case of lotteries, the payout is uncertain.29 Further, we do not consider negative incentives that make use of the concept of loss aversion where participants cannot gain but only lose financially.30 They may be effective strategies but are likely unattractive options in real-life workplace health promotion, especially taking equity considerations into account.27 29 Previous reviews including by Luong et al took into account several types of financial incentives, such as deposit contracts, and jointly evaluated the effectiveness of different types of incentives in fostering physical activity, preventing conclusions about the distinct effects of each incentive type.25

In general, previous reviews investigated different settings (eg, primary care setting),26 considered financial incentives where participants could not determine freely how to spend the financial reward (eg, free facility membership),22 24–27 or combined studies where financial incentives were linked to weight loss and not physical activity goals or physical activity programme participation.27 By taking this specific and more focused approach regarding setting, financial incentives and outcomes, this review aims to provide decision-makers in the space of workplace health promotion with information about the potential to use financial incentives to increase physical activity in the workplace based on the rapidly emerging evidence base from primary studies.31–35

How this intervention might work

As part of developing an "overall company culture of health",36 employers could support physical activity programmes both at work and during leisure time of employees. In addition to the individual health benefit for employees by participating in physical activity promotion activities, employers may benefit from higher productivity, fewer days of sick leave taken and later retirement of their workforce, which could translate into substantial savings for a company.37 A key question for the success of physical activity programmes is how to achieve sustained participation of the targeted groups. Otherwise, these programmes run the risk of remaining ineffective or even exacerbating existing inequalities in physical activity patterns when they primarily reach those that are already physically active.

One way to increase the attractiveness of physical activity is to offer financial incentives. Their use in the context of prevention is largely motivated by findings from behavioural economics, based on the assumption that people value the present more than the future.38 Hence, people may forgo physical activity today to carry out another more valued activity with little long-term health gain, even though they know that this may lead to worse health in the future. Expressed in economic terms, the (opportunity) costs of physical activity today (eg, physical exertion, loss of time due to physical activity, course fees) are larger than the positive long-term effects of physical activity on their health, preventing them from engaging in physical activity.39 The goal of using financial incentives is to reduce these immediate opportunity costs, making physical activity more appealing.40

Objectives

(1) To assess the effects of financial incentives provided by employers for promoting physical activity among employees on programme adherence/compliance, physical activity and days of sick leave (primary outcomes), as well as body mass indices (BMI)/weight status, health related quality of life and adverse events (secondary outcomes); (2) to conduct subgroup analyses for primary outcomes investigating effects according to age and gender as well as sensitivity analyses concerning intervention implementation and study quality and (3) to examine the costs and cost-effectiveness of the intervention.

Methods

Reporting standards and registration

This protocol is conceptualised in accordance with the ‘Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Protocols 2015 statement (PRISMA-P)’ (online supplemental file 1).41 42 The final review manuscript will adhere to the PRISMA statement.43

Supplementary data

bmjopen-2020-042888supp001.pdf (84.1KB, pdf)

Eligibility criteria

Population

We will include studies investigating effects in employees or apprentices/trainees (≥16 years) working in HICs (World Bank country classification of 2020).44 Studies on other study populations (eg, students) will be excluded, as these populations may differ notably from characteristics of the target group (eg, time available for leisure activities, income, age, etc). We will also not consider studies from low-income or middle-income countries where different labour markets, including a smaller formal service sector—an economic sector where jobs do not often require high levels of physical activity—and very different socioeconomic characteristics likely compromise comparability.45

Experimental intervention

We define eligible interventions as financial incentives provided by employers to promote physical activity programmes as part of workplace health promotion strategies from which employees, apprentices/trainees can gain a direct financial benefit. To be included the implementation of financial incentives has to share one of the following characteristics:

  • Financial rewards for participating in activities promoting physical activity (eg, participating in sports courses, visiting fitness centres/gyms/swimming pools, active commuting (walking or cycling to work)).

  • Financial rewards for achieving physical activity-related goals (eg, prespecified number of steps per day, prespecified number of visits to a fitness centre/gym/swimming pool).31–35

If necessary, the results of these two modes to implement the intervention will be reported separately in the review. We will exclude interventions where central aspects of a transparent incentive scheme are not fulfilled and financial rewards have no ‘money-like function’, which are:

  • Interventions, where participants stay uninformed about the magnitude of the financial reward.

  • Interventions, where successful programme participation does not lead to a guaranteed financial reward for participants (ie, lotteries).

  • Interventions, where participants cannot determine freely how they can spend the financial reward (eg, vouchers for one specific item/product or a store with a limited variety of products).

Control intervention

We will include any active control intervention promoting physical activity (eg, physical activity programmes without a financial reward system, with financial penalties or non-financial incentives to encourage participation) or passive control group (no intervention/usual support or waiting list) conducted and assessed in parallel. We will focus on studies with active controls, where the only difference between comparator and intervention is the provision of the financial incentive (ie, physical activity programmes without a financial incentive), for investigating causal mechanisms of the intervention.46

Outcomes

Primary outcomes
  • Adherence/compliance (eg, completion/retention, attendance, duration and intensity adherence).

  • Physical activity (eg, levels of moderate-to-vigorous physical activity (MVPA), steps per day).

  • Days of sick leave (taken by the employee).

Secondary outcomes
  • BMI/weight status (eg, BMI, waist-to-hip ratio).

  • Health-related quality of life (eg, Short Form-36 Health Survey (SF-36), EuroQol-5 Dimension (EQ-5D)).

  • Intervention costs/cost-effectiveness (eg, data on incremental cost-effectiveness ratios for primary outcomes of the review).

  • Adverse events (eg, musculoskeletal injuries, other types of ‘overuse’ injuries due to increased levels of MVPA).

Studies that do not present data on at least one primary outcome will be excluded. In general, we will prefer outcome data based on instruments with available information on validity and reliability. Potential biases at the stages of measuring outcomes, including potential instruments, will be taken into account in the risk of bias assessment.46

Study design

This review will include studies with experimental interventions and active or passive parallel control groups. We will focus on studies, which aim to minimise confounding and can establish reliable evidence regarding the relationship between financial incentives and the selected primary outcomes. Therefore, we will include:

  • RCTs,

  • Cluster RCTs (cRCT)

with a minimum follow-up of 1 month, so that also mid-term to long-term impacts of the interventions can be investigated. Other quantitative (ie, non-randomised studies, uncontrolled before-and-after studies) or qualitative studies will be excluded.46

Information sources

This review will integrate evidence from the fields of medical, economic and public health research, therefore, we selected relevant international and national electronic databases, grey literature resources, trial registers and websites across disciplines to identify relevant studies. By considering grey literature and study entries of trial registries, we aim to identify study reports and unpublished or ongoing research. Reference lists of included studies will be screened for potentially missed studies at a later stage to avoid evidence selection bias.47 We will search the following eight academic research databases to identify records of relevant studies without any date range restrictions:

  • Current Contents Medicine Database of German and German-Language Journals (CCMed) via LIVIVO (1917 to present).

  • Cochrane Central Register of Controlled Trials (CENTRAL) via Wiley (up to current issue).

  • Cumulative Index to Nursing and Allied Health Literature (CINAHL) via EBSCO (1937 to present).

  • EconLit via EBSCO (1969 to present).

  • Excerpta Medica database (Embase) via OvidSP (1947 to present).

  • MEDLINE via OvidSP (1946 to present).

  • PsycINFO via OvidSP (1887 to present).

  • Web of Science (SCI-EXPANDED, SSCI, A&HCI, CPCI-S, CPCI-SSH, BKCI-S, BKCI-SSH, ESCI, CCR-EXPANDED, IC) via Clarivate (1900 to present).

We will also conduct searches in two grey literature resources to identify studies:

  • Social Science Research Network (SSRN) eLibrary via SSRN (1994 to present).

  • Google Scholar via Google (first 50 hits).

For identifying planned, ongoing or unpublished studies, we will search one meta-register for trials and one register with a focus on RCTs of economic interventions/experiments:

  • WHO International Clinical Trials Registry Platform (ICTRP) via WHO (1988 to present).

  • American Economic Association’s registry for RCTs (AEA RCT) via AEA (2014 to present).

Our search approach for finding studies will be supplemented by free-text searches on websites of key organisations/institutions:

  • German Federal Institute for Occupational Safety and Health (BAuA).

  • CDC Prevention Research Centres (CDC PRC).

  • German Trade Union Confederation (DGB).

  • European Network for Workplace Health Promotion (ENWHP).

  • RAND (RAND Europe).

Search strategy

We followed recommendations outlined in the ‘Peer Review of Electronic Search Strategies’ (PRESS) guideline in the development process of a search strategy for MEDLINE (box 1).48 We discussed several iterations of the search strategy to ensure an appropriate translation of the review question and the correct use of Boolean/proximity operators, subject headings/index terms, text words and filters. Piloting was based on five potentially relevant studies that had been already identified by the review team and were then successfully retrieved by the search.31–35 The search strategy was finally approved by a librarian using the PRESS checklist (online supplemental file 2).48 The MEDLINE search includes relevant text words and subject headings regarding population/setting (eg, workplace, employee), intervention (eg, financial incentives, cash rewards) and outcomes of interest (eg, physical activity). Further, we modified the ‘Cochrane Highly Sensitive Search Strategies for identifying randomised trials in MEDLINE (2008 revision)’ (eg, by introducing additional search terms) to better account for ‘cluster’ RCTs (study design).21 We will use this filter, including search commands to exclude animal studies, to increase the overall sensitivity and precision of the search results. No other search filters or limits (eg, no limits on the timeframe for the search) will be used. The initial MEDLINE search strategy, including structure and syntax, will be translated to meet standard requirements of other electronic databases and will be further adapted as necessary (eg, to account for differences in controlled vocabulary across databases). Grey literature resources, websites and trial registers will be searched within the limitations of the search interface while using the most relevant search terms (eg, financial incentive). Articles or reports of studies in languages other than English, German, French or Spanish will not be considered.48

Box 1. Search strategy for Medline.

Search strategy for MEDLINE (via OVID SP)
  1. exp Workplace/

  2. adult.ab, kf, ti.

  3. adults.ab, kf, ti.

  4. apprentice.ab, kf, ti.

  5. apprentices.ab, kf, ti.

  6. employee*.ab, kf, ti.

  7. employer*.ab, kf, ti.

  8. employed.ab, kf, ti.

  9. job.ab, kf, ti.

  10. laborlabour.ab, kf, ti.

  11. labour.ab, kf, ti.

  12. staff.ab, kf, ti.

  13. trainee.ab, kf, ti.

  14. trainees.ab, kf, ti.

  15. work.ab, kf, ti.

  16. workers.ab, kf, ti.

  17. workplace*.ab, kf, ti.

  18. workforce.ab, kf, ti.

  19. or/1–18

  20. exp Reward/

  21. exp Financial Support/

  22. exp Employee Incentive Plans/

  23. exp Health Benefit Plans, Employee/

  24. ((cash or economic* or fee* or financ* or monetary or money or monies) adj2 (award* or bonus* or compensation* or incentiv* or gratification* or payment* or reward* or re?imbursement* or subsid* or support)).ab, kf, ti.

  25. gift.ab, kf, ti.

  26. gifts.ab, kf, ti.

  27. nudge.ab, kf, ti.

  28. nudges.ab, kf, ti.

  29. nudging.ab, kf, ti.

  30. lottery.ab, kf, ti.

  31. lotteries.ab, kf, ti.

  32. rebate*.ab, kf, ti.

  33. remuneration*.ab, kf, ti.

  34. voucher*.ab, kf, ti.

  35. or/20–34

  36. exp Exercise/

  37. exp Exercise Therapy/

  38. exp Fitness Centre/

  39. exp “Physical Education and Training”/

  40. exp Physical Conditioning, Human/

  41. exp Physical Fitness/

  42. exp Recreation/

  43. exp Sports/

  44. exp Weight Loss/

  45. ((active or bicycle or bicycling or bike or biking or cycle or cycling or walk or walking) adj2 (commute or commuting)).ab, kf, ti.

  46. (physical* adj2 activ*).ab, kf, ti.

  47. athlet*.ab, kf, ti.

  48. exercise*.ab, kf, ti.

  49. fitness.ab, kf, ti.

  50. gym*.ab, kf, ti.

  51. recreation*.ab, kf, ti.

  52. run.ab, kf, ti.

  53. running.ab, kf, ti.

  54. sport.ab, kf, ti.

  55. sports.ab, kf, ti.

  56. swim*.ab, kf, ti.

  57. walk.ab, kf, ti.

  58. walking.ab, kf, ti.

  59. wellness*.ab, kf, ti.

  60. or/36–59

  61. randomised controlled trial.pt.

  62. controlled clinical trial.pt.

  63. cluster.ab, kf, ti.

  64. randomised.ab, kf, ti.

  65. randomised.ab, kf, ti.

  66. randomly.ab, kf, ti.

  67. trial.ab, kf, ti.

  68. experiment.ab, kf, ti.

  69. study.ab, kf, ti.

  70. RCT.ab, kf, ti.

  71. cRCT.ab, kf, ti.

  72. groups.ab, kf, ti.

  73. or/61–72

  74. 19 and 35 and 60 and 73

  75. exp animals/ not humans.sh.

  76. 74 not 75

Supplementary data

bmjopen-2020-042888supp002.pdf (153.1KB, pdf)

Data extraction and analysis

Data management

We will save all database search export files locally and merge all retrieved references using EndNote.49 With this software, we will also deduplicate references for uploading unique references to a dedicated online screening software (Covidence).50 The review team will first test eligibility criteria against a limited set of references, to check for clarity and completeness of criteria for overall guidance in the screening process.47 Data extraction files and RoB tables will be shared among the review team and edited collectively via online tools allowing collaborative working. A backup of all relevant project data will be saved.

Study selection

Formal screening of all search results against our eligibility criteria will be first performed based on title/abstracts. Full texts of potentially relevant studies will be screened in a second step. Two authors will independently check title/abstracts or full texts for relevance using Covidence, a third author will be consulted if consensus cannot be reached.50 Reasons for exclusion in the full-text screening stage will be documented according to the PRISMA guideline. Internal hierarchy for documented reasons of exclusion will be as follows: ineligible intervention, ineligible study design/no comparator, ineligible population, ineligible outcomes, ineligible language. We will finally document search results across information sources and also the process of screening for and selecting studies using a PRISMA flow chart template.43

Data collection process and extraction

For data extraction, prepiloted extraction tables will be used. We will extract general information regarding the intervention and control group (eg, setting; delivery format, timing and amount of financial reward; number of participants), timing/length of follow-up, outcomes reported, outcome results and other relevant information based on data reported in articles/study reports. This will also include information on so-called PROGRESS factors at baseline to analyse equity aspects between the intervention and control groups.51 Other contextual factors and process evaluation criteria relevant to the implementation (eg, intervention intensity) will be extracted if available. All data in the extraction tables, such as study characteristics, will be checked by a second author, including the final data transferred to the review software. Dual data extraction by two independent reviewers will be done for outcome data to reduce both the risk of making mistakes and biased results due to partially selected data.52 Potential conflicts will be solved by consensus or the opinion of a third reviewer. We will extract data for primary and secondary outcomes, irrespective of whether they can later be analysed as dichotomous (eg, dichotomised data based on a specified threshold) or continuous (eg, indices such as BMI) outcome variables. The effect of providing financial incentives will be preferably reported as risk ratios for event data in the final analyses. If the same continuous outcomes were expressed in different measures not directly comparable (eg, scales) in primary studies, we will aim to report, at a minimum, the standardised mean difference. If necessary, further steps to harmonise data will be taken. We will also contact study authors via email or platforms such as ResearchGate if relevant data are missing.53

Outcomes and data items

We will extract and analyse outcome data that might be directly or indirectly influenced by financial incentives for employees to promote physical activity.36 On the one hand, ‘adherence/compliance’ is used to see whether uptake of physical activity programmes changes or employees comply with basic requirements for successful participation; the outcome ‘physical activity’ will be used to evaluate changes in physical activity levels (based on subjective (eg, exercise diaries) or objective data collection methods/instruments (eg, accelerometer data)) or achievement of physical activity goals between financial incentive intervention and control groups. On the other hand, the third primary outcome ‘days of sick leave’ will show potential general health effects for employees that might be explained by a change in individual physical activity levels and/or financial resources gained through the intervention.

Secondary outcomes will focus on areas that can change in the medium term due to changes in levels of physical activity (‘BMI/weight status’ and ‘health-related quality of life’). We will also look at potential ‘adverse events’, to investigate whether financial incentives can lead to ‘overuse’ injuries (eg, due to overestimation of individual baseline fitness levels) among participants of intervention groups (ie, musculoskeletal injuries).54 In addition, cost-effectiveness with regard to primary outcomes (eg, incremental cost-effectiveness ratios) and implementation costs will be extracted and analysed to see whether financial incentives are a useful approach from a cost perspective—information especially relevant to potential funders. We will extract and report intervention costs, if available, separately for the physical activity intervention part and the financial incentive part, to identify the budget requirements for implementing both parts of the intervention.55

For studies reporting multiple follow-ups, we will extract data for each time period individually. Cost data will be first extracted as reported in the original study, but later harmonised to be used in further analyses (eg, currencies, inflation), if necessary.46

Risk of bias

For assessing the quality of included studies with RCT and cRCT designs, we will use the latest version of the ‘Revised Cochrane risk-of-bias tool for randomised trials (RoB 2)’. The tool allows the assessment of the risk of bias arising from the randomisation process, deviations from the intended intervention, missing outcome data, measurement errors and the absence of relevant data in the report.56 The assessment will be performed independently by two authors and the process of solving potential conflicts will follow the same procedure as described for data extraction. We will use the ‘Grading of Recommendations Assessment, Development and Evaluation’ (GRADE) approach for primary outcomes, to provide additional information on the certainty of the evidence and the strength of potential recommendations.57

Data synthesis

Our review synthesis will focus on estimating the effectiveness of financial incentives provided by employers to promote physical activity programmes regarding primary outcomes in comparison to a control group, where the only difference between the control condition and intervention is the provision of the financial incentive. Data of intervention arms with the highest incentive provided or most similar incentive scheme (ie, timing of receiving the financial reward) across studies will be considered in the data synthesis. We do not expect to be able to conduct meta-analyses for all outcomes as the evidence base, especially for some secondary outcomes (ie, BMI), is likely too small.31–35 Outcome data of included studies (≤2 studies) will be pooled in a meta-analysis if they are sufficiently homogeneous (ie, similar scales/outcome measurements) and available (ie, data on intervention effect estimates, data to calculate weights), or the transformation of outcome data is possible (ie, transforming continuous outcome data into a dichotomous format for conducting the meta-analysis if relevant data are provided by the original study authors). This will include the analysis of dichotomous data by using the Maentel-Haenszel method and continuous data by using the inverse variance method. To take the potential heterogeneity of the studies into account, we will apply the random-effects method. Quantitative measurements of heterogeneity (eg, I², χ2) will be calculated and their information value discussed, especially if a meta-analysis is based only on a limited sample of studies. Forest plots will be used for visual representations of each meta-analysis to show the results and precision of individual studies, dispersion of outcomes, and summary effect estimates.

The most frequently reported scale/outcome measure (eg, SF-36 questionnaire score) for each outcome (eg, health-related quality of life) of all included studies will guide which data will finally enter the synthesis. In general, we will select estimates based on the longest follow-up (eg, ≥12 months) and most similar follow-up across studies (eg, at 6 months, at 12 months) and will perform analyses separately, if outcomes were reported for multiple time points. Similarly, we will prefer outcomes based on objectively collected data to outcomes based on subjectively collected data if validity is expected to be higher. In order to determine the heterogeneity of intervention effects and the robustness of our primary outcome results, subgroup analyses by age and gender, as well as sensitivity analyses by amount of financial incentive/reward and study quality (ie, based on the RoB assessment), will be performed if sufficient data are available.58

If calculating a meta-analysis is not possible due to heterogeneity concerns, or a reported control group estimate of a study cannot substantially contribute to the comparison between financial incentives to promote physical activity (intervention) versus the same physical activity intervention without the provision of a financial incentive (control), or a study provides multiple different outcome measurements which were not all considered in a meta-analysis (eg, meta-analysis for MVPA based on three studies, but only one study with data on VPA), we will conduct a narrative synthesis. Graphical and tabular methods will be used to present a summary of the effects, this also applies to studies where the only difference between intervention and control arms is the degree of financial incentive provided (ie, low vs high financial incentive). The narrative presentation of results will be supplemented by the use of harvest or effect-direction plots where appropriate, in line with current recommendations for the synthesis of complex interventions and the ‘Synthesis without meta-analysis’ reporting guideline.59–61

Meta-bias(es)

We will plot effect estimates of a primary outcome against a measure of precision to create funnel plots, if sufficient estimates from at least 10 studies are available. We will check for a potential asymmetry in the plot, that could stem from various biases, that is, reporting bias, including publication bias.62

Confidence in cumulative evidence

Two authors will independently assess the body of evidence included in this review using the GRADE approach for drawing conclusions about the certainty of evidence regarding primary outcome results. A third author will be consulted in case of disagreement and will decide on the final rating if no consensus could be reached. Because only randomised study designs (RCTs, cRCTs) will be included, each assessment will start with the best rating achievable (‘high’). The GRADE rating process foresees potential downgrades and upgrades. Domains which could affect final ratings (‘high’, ‘moderate’, ‘low’ or ‘very low’) are imprecision, inconsistency, publication bias, and other limitations. Ratings concerning the certainty of the identified evidence will be used in a ‘Summary of findings’ table for primary outcomes on a given comparison between intervention and control groups. Ratings will be reported alongside magnitudes of relative and absolute effects and the number of studies which provided data for each comparison. We will also report this information in the ‘Abstract’ and ‘Results’ section of our upcoming review to provide further information on the certainty for the body of evidence included.57

Ethics and dissemination

Ethics

This planned systematic review will use existing, published and unpublished data from completed RCTs, in which informed consent was obtained from the study participants. Our analyses will be based on aggregated data as reported in the original study reports or provided by study authors of primary studies (no individual patient data). Therefore, formal ethical approval is not required.

Dissemination and publication plan

Our review advisory board will be invited to discuss major findings of the review before publication to ensure relevancy for practitioners (eg, companies or health insurers). We aim to submit the final review manuscript including all planned analyses to an Open Access journal. At different stages of the review we will disseminate major findings through different communication channels. The scientific community will be mainly approached at conferences via poster or oral presentations. The final project report will be accompanied by press releases and social media postings of the involved institutions.

Data sharing statement

The full search strategy for all databases and all extracted data relevant to analyses and assessments will be part of the major publication (online supplemental files). Other data (eg, reference files, individual RoB assessment files) will be provided on request via email.

Patient and public involvement

We set up an advisory board for this research project comprising one representative of a provider in the German statutory health insurance system (Dr Julia Schröder, BKK, Berlin) and public health researchers with focus on incentive-based interventions (Professor Dr Lucia Reisch, Zeppelin University, Friedrichshafen) or physical activity (Professor Dr Claudia Pischke, Heinrich Heine University, Düsseldorf). The advisory board has already provided valuable input concerning the initial review question. Board members will be invited to take part in a research dissemination workshop at the end of this project. This workshop will provide an opportunity to reflect on major findings of the review and discuss the applicability and relevance to the German context.

Conclusion

Increasing physical activity levels among working populations, especially for employees in sedentary positions, poses a major public health challenge for HIC.2 4–9 One way to reduce opportunity costs of employees for engaging in physical activity is (1) to financially incentivise physical activity programme participation or (2) to financially reward the achievement of physical activity-related goals. The increasing number of RCTs with results on this topic provides an opportunity for their systematic assessment and synthesis to inform the use of this approach in the workplace setting by public health decision-makers, companies or health insurance funds which do support workplace-based interventions with a physical activity component.31–35 It thereby complements existing reviews that had taken a more general perspective.22–27 Differences in (1) study populations (eg, income level, baseline physical activity levels), (2) settings (eg, private/public sector), (3) intervention durations/follow-up and (4) physical activity programmes or activities linked to the incentive scheme across studies may lead to heterogeneity and may pose challenges for quantitative synthesis (ie, meta-analyses, subgroup, sensitivity analyses) and overall generalisability of the results. By using a comprehensive search strategy and focusing on study designs that can establish direct causal relationships, this review will provide an overview and critical assessment of the existing evidence on the effects of financial incentives on physical activity in the workplace setting. This will help policy-makers and companies to make decisions about ways to increase physical activity among employees based on the best available evidence.

Supplementary Material

Reviewer comments
Author's manuscript

Footnotes

Contributors: TS and TLH jointly developed the study question and drafted the background section of this protocol. TLH is the guarantor of this review and drafted a first version of the study selection criteria, synthesis of data and risk of bias assessment section. The search strategy was initially developed by TLH, later discussed with JF and finally approved by LC. All coauthors (TS, JF and LC) provided comments and critically read the final version of the manuscript before submission.

Funding: This research was funded by the German Federal Ministry of Education and Research (BMBF) under the project title AFKAB: grant number 01EL2031. The publication of this article was funded by the Open Access Fund of the Leibniz Association.

Competing interests: None declared.

Patient consent for publication: Not required.

Provenance and peer review: Not commissioned; externally peer reviewed.

Supplemental material: This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

References

  • 1.World Health Organization . WHO guidelines on physical activity and sedentary behaviour, 2020. Available: https://apps.who.int/iris/bitstream/handle/10665/336656/9789240015128-eng.pdf [Accessed 05 Jan 2021]. [PubMed]
  • 2.Lee I-M, Shiroma EJ, Lobelo F, et al. Effect of physical inactivity on major non-communicable diseases worldwide: an analysis of burden of disease and life expectancy. Lancet 2012;380:219–29. 10.1016/S0140-6736(12)61031-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Guthold R, Stevens GA, Riley LM, et al. Worldwide trends in insufficient physical activity from 2001 to 2016: a pooled analysis of 358 population-based surveys with 1·9 million participants. Lancet Glob Health 2018;6:e1077–86. 10.1016/S2214-109X(18)30357-7 [DOI] [PubMed] [Google Scholar]
  • 4.Arem H, Moore SC, Patel A, et al. Leisure time physical activity and mortality: a detailed pooled analysis of the dose-response relationship. JAMA Intern Med 2015;175:959–67. 10.1001/jamainternmed.2015.0533 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Lear SA, Hu W, Rangarajan S, et al. The effect of physical activity on mortality and cardiovascular disease in 130 000 people from 17 high-income, middle-income, and low-income countries: the PURE study. Lancet 2017;390:2643–54. 10.1016/S0140-6736(17)31634-3 [DOI] [PubMed] [Google Scholar]
  • 6.Mammen G, Faulkner G. Physical activity and the prevention of depression: a systematic review of prospective studies. Am J Prev Med 2013;45:649–57. 10.1016/j.amepre.2013.08.001 [DOI] [PubMed] [Google Scholar]
  • 7.Reiner M, Niermann C, Jekauc D, et al. Long-term health benefits of physical activity--a systematic review of longitudinal studies. BMC Public Health 2013;13:813. 10.1186/1471-2458-13-813 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Warburton DER, Nicol CW, Bredin SSD. Health benefits of physical activity: the evidence. CMAJ 2006;174:801–9. 10.1503/cmaj.051351 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Finger JD, Mensink G, Lange C. Arbeitsbezogene körperliche Aktivität bei Erwachsenen in Deutschland [Work-related physical activity among adults in Germany], 2017. Available: https://edoc.rki.de/bitstream/handle/176904/2647/24OvDoITLUVQ.pdf?sequence=1 [Accessed 13 Jul 2020]. [DOI] [PMC free article] [PubMed]
  • 10.Eurostat . Persons performing physical activity when working by type of activity, most frequent activity status, quantile and degree of urbanization, 2019. Available: https://data.europa.eu/euodp/en/data/dataset/rE6263Ct1IFXmwmOHizAA [Accessed 05 Jan 2021].
  • 11.Church TS, Thomas DM, Tudor-Locke C, et al. Trends over 5 decades in U.S. occupation-related physical activity and their associations with obesity. PLoS One 2011;6:e19657. 10.1371/journal.pone.0019657 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Ding D, Lawson KD, Kolbe-Alexander TL, et al. The economic burden of physical inactivity: a global analysis of major non-communicable diseases. Lancet 2016;388:1311–24. 10.1016/S0140-6736(16)30383-X [DOI] [PubMed] [Google Scholar]
  • 13.OECD . Hours worked (indicator), 2020. Available: 10.1787/47be1c78-en[Accessed 13 Jul 2020]. [DOI]
  • 14.Patterson R, Panter J, Vamos EP, et al. Associations between commute mode and cardiovascular disease, cancer, and all-cause mortality, and cancer incidence, using linked census data over 25 years in England and Wales: a cohort study. Lancet Planet Health 2020;4:e186–94. 10.1016/S2542-5196(20)30079-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Parry S, Straker L. The contribution of office work to sedentary behaviour associated risk. BMC Public Health 2013;13:296. 10.1186/1471-2458-13-296 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Clemes SA, O'Connell SE, Edwardson CL. Office workers' objectively measured sedentary behavior and physical activity during and outside working hours. J Occup Environ Med 2014;56:298–303. 10.1097/JOM.0000000000000101 [DOI] [PubMed] [Google Scholar]
  • 17.ECORYS . Physical activity at the workplace: literature review and best practice case studies: a report to the European Commission, 2017. Available: https://op.europa.eu/en/publication-detail/-/publication/9fc2b8a0-e537-11e7-9749-01aa75ed71a1 [Accessed 05 Jan 2021].
  • 18.Abu-Omar K, Rütten A, Burlacu I, et al. The cost-effectiveness of physical activity interventions: a systematic review of reviews. Prev Med Rep 2017;8:72–8. 10.1016/j.pmedr.2017.08.006 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Hollederer A, Wießner F. Prevalence and development of workplace health promotion in Germany: results of the iab establishment panel 2012. Int Arch Occup Environ Health 2015;88:861–73. 10.1007/s00420-014-1012-z [DOI] [PubMed] [Google Scholar]
  • 20.Ryde GC, Gilson ND, Burton NW, et al. Recruitment rates in workplace physical activity interventions: characteristics for success. Am J Health Promot 2013;27:e101–12. 10.4278/ajhp.120404-LIT-187 [DOI] [PubMed] [Google Scholar]
  • 21.Higgins JPT. Cochrane Handbook for systematic reviews of interventions (version 5.2), 2017. Available: https://training.cochrane.org/handbook/pdf-versions [Accessed 27 Mar 2019].
  • 22.Barte JCM, Wendel-Vos GCW. A systematic review of financial incentives for physical activity: the effects on physical activity and related outcomes. Behav Med 2017;43:79–90. 10.1080/08964289.2015.1074880 [DOI] [PubMed] [Google Scholar]
  • 23.Mitchell MS, Goodman JM, Alter DA, et al. Financial incentives for exercise adherence in adults: systematic review and meta-analysis. Am J Prev Med 2013;45:658–67. 10.1016/j.amepre.2013.06.017 [DOI] [PubMed] [Google Scholar]
  • 24.Strohacker K, Galarraga O, Williams DM. The impact of incentives on exercise behavior: a systematic review of randomized controlled trials. Ann Behav Med 2014;48:92–9. 10.1007/s12160-013-9577-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Luong M-LN, Hall M, Bennell KL, et al. The impact of financial incentives on physical activity: a systematic review and meta-analysis. Am J Health Promot 2021;35:236-249. 10.1177/0890117120940133 [DOI] [PubMed] [Google Scholar]
  • 26.Molema CCM, Wendel-Vos GCW, Puijk L, et al. A systematic review of financial incentives given in the healthcare setting; do they effectively improve physical activity levels? BMC Sports Sci Med Rehabil 2016;8:15. 10.1186/s13102-016-0041-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Gong Y, Trentadue TP, Shrestha S, et al. Financial incentives for objectively-measured physical activity or weight loss in adults with chronic health conditions: a meta-analysis. PLoS One 2018;13:e0203939. 10.1371/journal.pone.0203939 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Meierjürgen R. Präventionsgesetz: Ein wichtiger Schritt in die richtige Richtung [Prevention Act: An important step in the right direction]. Gesundheits- und Sozialpolitik 2016;69:25–31. [Google Scholar]
  • 29.Hoskins K, Ulrich CM, Shinnick J, et al. Acceptability of financial incentives for health-related behavior change: an updated systematic review. Prev Med 2019;126:105762. 10.1016/j.ypmed.2019.105762 [DOI] [PubMed] [Google Scholar]
  • 30.Kahneman D, Tversky A. Prospect theory: an analysis of decision under risk. Econometrica 1979;47:263–91. 10.2307/1914185 [DOI] [Google Scholar]
  • 31.Hunter RF, Brennan SF, Tang J, et al. Effectiveness and cost-effectiveness of a physical activity Loyalty scheme for behaviour change maintenance: a cluster randomised controlled trial. BMC Public Health 2016;16:618. 10.1186/s12889-016-3244-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Jones D, Molitor D, Reif J. What do workplace wellness programs do? Evidence from the Illinois workplace wellness study. Q J Econ 2019;134:1747–91. 10.1093/qje/qjz023 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Mitchell M, White L, Oh P, et al. Examining incentives to promote physical activity maintenance among hospital employees not achieving 10,000 daily steps: a web-based randomized controlled trial protocol. JMIR Res Protoc 2016;5:e231. 10.2196/resprot.6285 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Patel MS, Asch DA, Troxel AB, et al. Premium-based financial incentives did not promote workplace weight loss in a 2013–15 study. Health Aff 2016;35:71–9. 10.1377/hlthaff.2015.0945 [DOI] [PubMed] [Google Scholar]
  • 35.Royer H, Stehr M, Sydnor J. Incentives, commitments, and habit formation in exercise: evidence from a field experiment with workers at a Fortune-500 company. Am Econ J Appl Econ 2015;7:51–84. 10.1257/app.20130327 [DOI] [Google Scholar]
  • 36.CDC Workplace Health Research Network . Physical activity in the workplace - a guide for employers, 2016. Available: https://www.workhealthresearchnetwork.org/wp-content/uploads/2016/05/CDC-WHRN-Physical-Activity_Employer-Guide-FINAL.pdf [Accessed 13 Jul 2020].
  • 37.van Rijn RM, Robroek SJW, Brouwer S, et al. Influence of poor health on exit from paid employment: a systematic review. Occup Environ Med 2014;71:295–301. 10.1136/oemed-2013-101591 [DOI] [PubMed] [Google Scholar]
  • 38.O'Donoghue T, Rabin M. Doing it now or later. Am Econ Rev 1999;89:103–24. 10.1257/aer.89.1.103 [DOI] [Google Scholar]
  • 39.Loewenstein G, Brennan T, Volpp KG. Asymmetric paternalism to improve health behaviors. JAMA 2007;298:2415–7. 10.1001/jama.298.20.2415 [DOI] [PubMed] [Google Scholar]
  • 40.Marteau TM, Ashcroft RE, Oliver A. Using financial incentives to achieve healthy behaviour. BMJ 2009;338:b1415. 10.1136/bmj.b1415 [DOI] [PubMed] [Google Scholar]
  • 41.Shamseer L, Moher D, Clarke M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ 2015;350:g7647. 10.1136/bmj.g7647 [DOI] [PubMed] [Google Scholar]
  • 42.Heise TL, Frense J, Christianson L. Using financial incentives to increase physical activity among employees as a strategy of workplace health promotion, 2020. Available: https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42020184345 [Accessed 13 Jul 2020]. [DOI] [PMC free article] [PubMed]
  • 43.Moher D, Liberati A, Tetzlaff J, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009;6:e1000097. 10.1371/journal.pmed.1000097 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.World Bank . World bank list of economies, 2020. Available: http://databank.worldbank.org/data/download/site-content/CLASS.xls [Accessed 13 Jul 2020].
  • 45.World Bank . Services, value added (% of GDP) - All Countries and Economies, 2020. Available: https://data.worldbank.org/indicator/NV.SRV.TOTL.ZS [Accessed 13 Jul 2020].
  • 46.McKenzie JE, Brennan SE, Ryan RE. Defining the criteria for including studies and how they will be grouped for the synthesis. : Higgins JP TJ, Chandler J, Cumpston M, et al., . Cochrane Handbook for systematic reviews of interventions. 2 edn. Chichester (UK: John Wiley & Sons and The Cochrane Collaboration, 2019: 33–65. [Google Scholar]
  • 47.Lefebvre C, Glanville J, Briscoe S. Searching for and selecting studies. : Higgins JP TJ, Chandler J, Cumpston M, et al., . Cochrane Handbook for systematic reviews of interventions. 2 edn. Chichester (UK): John Wiley & Sons and The Cochrane Collaboration, 2019: 67–107. [Google Scholar]
  • 48.McGowan J, Sampson M, Salzwedel DM, et al. PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement. J Clin Epidemiol 2016;75:40–6. 10.1016/j.jclinepi.2016.01.021 [DOI] [PubMed] [Google Scholar]
  • 49.Endnote . Clarivate analytics. endnote X9 software, 2020. Available: https://endnote.com/ [Accessed 13 Jul 2020].
  • 50.Covidence . Covidence software [Internet], 2016. Available: https://www.covidence.org [Accessed 13 Jul 2020].
  • 51.O'Neill J, Tabish H, Welch V, et al. Applying an equity lens to interventions: using progress ensures consideration of socially stratifying factors to illuminate inequities in health. J Clin Epidemiol 2014;67:56–64. 10.1016/j.jclinepi.2013.08.005 [DOI] [PubMed] [Google Scholar]
  • 52.Higgins JPT, Lasserson T, Chandler J. Methodological expectations of Cochrane intervention reviews (version March 2020), 2020. Available: https://community.cochrane.org/mecir-manual [Accessed 05 Jan 2021].
  • 53.Li T, Higgins JPT, Deeks JJ. Collecting data. : Higgins JP TJ, Chandler J, Cumpston M, et al., . Cochrane handbook for systematic reviews of interventions. 2 edn. Chichester (UK): John Wiley & Sons and The Cochrane Collaboration, 2019: 109–41. [Google Scholar]
  • 54.Aicale R, Tarantino D, Maffulli N. Overuse injuries in sport: a comprehensive overview. J Orthop Surg Res 2018;13:309. 10.1186/s13018-018-1017-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55.Hunter RF, Murray JM, Gough A, et al. Effectiveness and cost-effectiveness of a Loyalty scheme for physical activity behaviour change maintenance: results from a cluster randomised controlled trial. Int J Behav Nutr Phys Act 2018;15:127. 10.1186/s12966-018-0758-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56.Sterne JAC, Savović J, Page MJ, et al. Rob 2: a revised tool for assessing risk of bias in randomised trials. BMJ 2019;366:l4898. 10.1136/bmj.l4898 [DOI] [PubMed] [Google Scholar]
  • 57.Ryan R, Hill S, Cochrane Consumers and Communication . How to grade the quality of the evidence, 2016. Available: https://cccrg.cochrane.org/author-resources [Accessed 13 Jul 2020].
  • 58.Deeks JJ, Higgins JPT, Altman DG. Analysing data and undertaking meta-analyses. : Higgins JP TJ, Chandler J, Cumpston M, et al., . Cochrane Handbook for systematic reviews of interventions. 2 edn. Chichester (UK): John Wiley & Sons and The Cochrane Collaboration, 2019: 241–84. [Google Scholar]
  • 59.Campbell M, McKenzie JE, Sowden A, et al. Synthesis without meta-analysis (swim) in systematic reviews: reporting guideline. BMJ 2020;368:l6890. 10.1136/bmj.l6890 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 60.Ogilvie D, Fayter D, Petticrew M, et al. The harvest plot: a method for synthesising evidence about the differential effects of interventions. BMC Med Res Methodol 2008;8:8. 10.1186/1471-2288-8-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 61.Thomson HJ, Thomas S. The effect direction plot: visual display of non-standardised effects across multiple outcome domains. Res Synth Methods 2013;4:95–101. 10.1002/jrsm.1060 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 62.Page MJ, Higgins JPT, Sterne JAC. Assessing risk of bias due to missing results in a synthesis. : Higgins JP TJ, Chandler J, Cumpston M, et al., . Cochrane handbook for systematic reviews of interventions. 2 edn. Chichester (UK): John Wiley & Sons and The Cochrane Collaboration, 2019: 349–74. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary data

bmjopen-2020-042888supp001.pdf (84.1KB, pdf)

Supplementary data

bmjopen-2020-042888supp002.pdf (153.1KB, pdf)

Reviewer comments
Author's manuscript

Articles from BMJ Open are provided here courtesy of BMJ Publishing Group

RESOURCES