Abstract
Excess sodium intake elevates blood pressure and risk for cardiovascular diseases. The use of low‐sodium salts is a potentially cost‐effective strategy to counter the rising global burden of cardiovascular diseases. This research aimed to understand the potential scale‐up of low‐sodium salt interventions by examining the availability of low‐sodium salts globally, synthesizing evidence about the effectiveness of low‐sodium salt interventions, and identifying the challenges and opportunities associated with implementing low‐sodium salt interventions. This study consists of three parts. The first part is a systematic online search of low‐sodium salts. The authors will use the advanced search functions of search engines and online shopping sites to execute the search. The second part is a systematic review of academic literature on the use of low‐sodium salts. A meta‐analysis will be performed to quantify the effectiveness of low‐sodium salt interventions. The third part is key informant interviews to understand the challenges of implementing low‐sodium salt interventions. Key informants will include policymakers, academic researchers, and salt industry representatives. The list of key informants will be generated through purposive sampling and snowball sampling based on the completed online search and the systematic review. The interview guides will be developed based on the RE‐AIM (Reach, Effective, Adoption, Implementation, and Maintenance) framework. The study received ethics approval from the University of New South Wales Human Research Ethics Advisory Panel (HC190921). Findings will be disseminated with academics and policymakers through a peer‐reviewed journal and conference presentations.
Keywords: environmental scan, low‐sodium salts, salt substitutes, study protocol
1. INTRODUCTION
Excess consumption of dietary sodium is a major cause of high blood pressure, leading to increased risk of cardiovascular disease (CVD) and mortality. Globally, 1.7 million annual deaths from cardiovascular causes are attributed to excess sodium intake. 1 Dietary sodium reduction has been identified as an effective public health strategy for reducing the risk of major CVDs. 2 , 3 Despite World Health Organisation (WHO) recommendations of a reduction in dietary sodium intake to <2 g/d (equivalent to 5 g/d of salt), 1 a study in 2014 suggested that the global mean intake of sodium is around 4 g/d (equivalent to 10 g/d of salt). Efficient implementation of effective sodium reduction strategies is urgently needed.
In contrast to sodium, potassium is a key mineral that helps to lower blood pressure by balancing the negative influence on the body's management of sodium. 4 A diet that includes a modest sodium restriction while increasing potassium intake serves as a strategy to prevent or control hypertension and decrease CVD morbidity and mortality. 5 Low‐sodium salts are table or cooking salts that contain a lower level of sodium chloride (or no sodium chloride) and replacing this amount with potassium chloride, magnesium sulfate, or other minerals. Low‐sodium salts are likely to be a cost‐effective intervention for the control of hypertension and cardiovascular disease burden. 6 , 7
In the past decade, there is a growing body of evidence supporting the use of low‐sodium salts as an effective intervention to reduce dietary sodium intake, manage blood pressure, and reduce CVD risk. 8 , 9 Previous systematic reviews about low‐sodium salts mainly focused on the quantitative impact on clinical outcomes from randomized controlled trials. 10 To add to this body of evidence, this study will gather a mixture of quantitative and qualitative data to provide a comprehensive understanding of the potential scale‐up of low‐sodium salt interventions.
2. OBJECTIVES
The overall objective of this study was to understand the potential scale‐up of low‐sodium salt interventions. Specifically, we aim to (a) better understand the availability and diversity of low‐sodium salts globally through a systematic online search, (b) summarize the academic researches related to low‐sodium salts and indicate important health benefits or potential harms of low‐sodium salts on a range of health outcomes through a systematic review, and (c) provide further insights into the potential scale‐up through key informant interviews about the challenges and opportunities associated with low‐sodium salt interventions.
3. METHODS
An environmental scan of low‐sodium salts will be performed from January 2020 to December 2020. The environmental scan has been described as an important and effective public health tool to review the current state of services and programs, evaluate needs, identify evidence gaps, inform policy development, and plan future initiatives. 11 , 12 This environmental scan is composed of three parts. The first part is a systematic online search of low‐sodium salts. The second part is a systematic review of academic literature on the use of low‐sodium salts. The third part is key informant interviews about the implementation of low‐sodium salt interventions.
3.1. Part I Low‐sodium salts in the global market
Low‐sodium salts in the market represent a novel way in reducing sodium intake with limited sacrificing taste. However, little is known about this emerging product in the global market. Systematic online search for low‐sodium salts in global market will help understand the availability of low‐sodium salts in global market and their formula, brands, and costs in different settings. This information will indicate the feasibility of using low‐sodium salts in different countries.
3.1.1. Search strategy
We will use the advanced search functions of the primary search engines (Google and Bing) to execute the search. The initial keywords are different terms describing low‐sodium salts, including “salt substitute,” “low sodium salt,” “potassium salt,” “mineral salt,” and “sodium reduced salt.” Detailed search strategies are shown in Table 1. To identify the availability of low‐sodium salts from as many countries as possible, initial keywords and country name will be combined for a Google advanced search. The first 25 results of each search will be examined for eligibility. Webpages in other languages will be translated into English in the Google Chrome browser. Efforts will be made to obtain any relevant products and documents that elaborate on the use of low‐sodium salts. This may involve searching the Web sites of low‐sodium salt patents, the World Health Organization for salt reduction, and the salt market research reports. Search terms will evolve during the search in an iterative process. We will also search from online shopping sites, including Amazon, eBay, Walmart, and RedMart to identify low‐sodium salts. Additionally, any low‐sodium salt identified from the systematic review in part II of the study will be considered for inclusion. At last, we will also supplement the Internet search with low‐sodium salts provided by the key informants during interviews in the third part of the study.
Table 1.
Key words and terms in Web site search strategy
| Initial key words | Terms of different aspects | |
|---|---|---|
| “salt substitute,” “low sodium salt,” “potassium salt,” “mineral salt,” “sodium reduced salt” | & | Formula: "formula," "composition," "ingredient," "constituent" |
| Price: "price," "cost" | ||
| Country: for example, "Finland," "UK," "US," "China" | ||
| Safety issue: "safety," “adverse events,” "warning," "danger," “threatening” |
3.1.2. Data extraction
Data will be extracted to a predesigned Excel sheet, including the brand, formula, price ($/100 g), label of nutrition facts, safety warning, Web site of the product, and search date. Price of the normal salt produced by the same salt company will also be extracted to compare with the low‐sodium salts. Normal salt is defined as salt that is not sodium reduced (containing about 99% sodium chloride), and iodized salt is included in this definition. Low‐sodium salt information will be summarized by country, which can indicate feasibility of scaling‐up low‐sodium salt interventions in the context of country.
3.2. Part II. Systematic review of low‐sodium salt studies
Low‐sodium salts bear promises of cardiovascular benefits primarily by reducing the contribution of daily sodium intake and thus decreasing the blood pressure, the main risk factors of cardiovascular diseases. However, evidence for other health benefits or potential harms due to use of low‐sodium salt is unknown. A broad range of health outcomes will be investigated to determine a possible association with low‐sodium salt use in a general population. Meta‐analyses will be conducted if comparable outcome data from two or more studies were available. Compelling evidence indicating important health benefits and potential harms of low‐sodium salt use on a range of health outcomes will provide evidence base for scaling‐up low‐sodium salt intervention in a general population. This systematic review protocol will be registered on the International Prospective Register of Systematic Reviews and adheres to the Preferred Reporting Items for Systematic Reviews and Meta‐Analyses Protocols (PRISMA‐P). Results of the systematic review will be reported in line with Preferred Reporting Items for Systematic Reviews and Meta‐Analysis (PRISMA) guideline. 13
3.2.1. Eligibility criteria
Participants
We will include studies with a general population of all individuals without reference to any specific characteristic, such as age, sex, or disease condition. We excluded studies in vitro and animal studies.
Intervention
The intervention includes any type of low‐sodium salt, either as a stand‐alone intervention or in combination with other salt reduction strategies. We exclude interventions that do not meet the definition of low‐sodium salt. We also exclude interventions with a duration of <7 days because the duration of intervention is too short to observe changes in outcomes.
Comparisons or controls
We will include studies that compare the low‐sodium salt intervention against the intake of any alternative sodium reduction intervention, no intervention, or placebo.
Outcomes
The outcomes of interest will be all health‐related outcomes and adverse events, including urinary sodium and potassium, blood pressure, pulse rate, weight/BMI, cholesterol level, cardiovascular events, renal diseases, hyperkalemia, and mortality. The adverse events and acceptability of low‐sodium salt will be the outcome for case studies.
Types of study
We will include all randomized controlled trials (ie, individual, crossover, or cluster). Non‐randomized controlled trials, prospective and retrospective cohort studies, case‐control studies, and cross‐sectional studies, before‐and‐after studies (ie, controlled or uncontrolled), interrupted time‐series studies, or repeated‐measures studies will also be included but analyzed separately. To explore the acceptability and the safety of low‐sodium salts, case studies will also be identified. Qualitative studies and ongoing trials will also be screened out to provide information for key informant interviews in part III. Ongoing trials will be found from clinical trial registries (clinicaltrials.gov) and International Clinical Trials Registry Platform (ICTRP). Review papers, letters, and editorials will be excluded. Inclusion and exclusion criteria of studies assessed using a PICOS (Participants, Interventions, Comparators, Outcomes and Study type) are summarized in Table 2.
Table 2.
PICOS approach to study eligibility
| Inclusion | Exclusion | |
|---|---|---|
| Population (P) | A general population of all individuals without reference to any specific characteristic, such as age, sex, or disease condition. | Animal studies |
| Vitro studies | ||
| Intervention (I) | Any kinds of low‐sodium salt interventions (including exposure to low‐sodium salt) |
|
| Comparators (C) | Placebo, regular salt, no intervention, and other interventions. | |
| Outcomes (O) | Health‐related outcomes | |
| Adverse effects | ||
| Acceptability of low‐sodium salt | ||
| Study design | Randomized controlled trials (RCTs), non‐randomized controlled trials, qualitative studies, case study, and unpublished and ongoing studies | Review papers, letters/comments, and editorials |
3.2.2. Search strategy
A systematic literature search in the following databases will be conducted in January 2020 from databases: MEDLINE, Embase, and Cochrane Library. A search strategy will be developed on MEDLINE and adapted for other databases. Search strategies are listed in the supplementary document 1. Reference lists of relevant papers will be searched for additional records. To identify ongoing or completed, but unpublished, trials, the WHO International Clinical Trials Registry Platform (ICTRP) search portal and ClinicalTrials.gov will be searched on March 2020. Searches will be not restricted by publication date or language.
3.2.3. Study selection
All titles and abstracts of records identified in the databases will be screened for eligibility by primary screener (XY). The full texts of papers identified as potentially eligible will be obtained for independent review by two reviewers (XY and KL). Reasons for exclusion will be recorded. Any differences between reviewers will be resolved through discussion, with reference to a third senior investigator (MT) where necessary.
3.2.4. Data extraction
Data from included studies will be extracted using a standardized, prepiloted data extraction form. Two reviewers will extract data independently, with discrepancies identified and resolved through discussion, including with a third author where necessary. Extracted information will include the following:
General information—title, authors, country/settings, year of publication, and registration number for clinical trials.
Study characteristics—study design, duration of intervention or follow‐up, method of randomization, allocation concealment, and blinding (patients, people administering treatment, and outcome assessors).
Interventions or exposure—intervention and comparators, formula of low‐sodium salts
Participants—inclusion and exclusion criteria of original study, total number of participants, and number in each group
Outcomes—primary outcomes and any other outcomes assessed.
Study corresponding author will be contacted for missing data or uncertain important information if necessary.
3.2.5. Risk‐of‐bias assessment
Two authors will independently assess the risk of bias for each included study. Any disagreements will be resolved by discussion or a third author. For the risk‐of‐bias assessment of randomized controlled trials, we will use the Cochrane risk‐of‐bias tool. 14 For non‐randomized controlled trials, we will use the ROBINS‐I tool (risk of bias in non‐randomized studies of interventions). 15 Other types of study will not be assessed the risk of bias. Funnel plots will be created when more than ten studies are available to assess the likelihood of dissemination bias. If substantial variation in risk of bias of included studies is found, results will be synthesized separately for studies at high risk and low risk of bias.
3.2.6. Quality of evidence
Both reviewers will independently apply the Grading of Recommendations Assessment, Development and Evaluation (GRADE) criteria to rate the overall quality of evidence across interventional studies, while the Newcastle‐Ottawa Scale (NOS) will be used to assess the quality of observational studies. 16 , 17 The evidence quality assessments made by the reviewers will be compared, and disagreements between assigned ratings will be discussed until a consensus is reached.
3.2.7. Data synthesis
We will calculate the risk ratios and their respective 95% confidence intervals for trials and cohort studies. Mean differences or standardized mean differences with 95% confidence intervals will be calculated for continuous outcomes. Meta‐analysis will be conducted if the studies/methods from two or more studies are sufficiently homogeneous. Heterogeneity will be quantified using the I 2 statistic. Random‐effects model will be adopted. If a meta‐analysis is possible, statistical analysis will be conducted using RevMan 5. If meta‐analysis is not possible due to substantial heterogeneity, a narrative synthesis of the findings from the included studies will be provided, structured around the type of intervention, target population characteristics, type of outcome, and intervention content.
3.2.8. Sensitivity analyses
If sufficient data are available, subgroup analyses will be conducted. These analyses will assess differences between study population, type of study, and quality and risk of bias.
3.3. Part III. Key informant interviews about low‐sodium salt intervention
Although published studies supported the health benefits of low‐sodium salts, efforts are required to enhance the implementation of low‐sodium salts and optimize the health benefits at a population level. In this part, we will investigate the barriers and facilitators of implementing low‐sodium salts from various perspectives through the interviews with the key informants, including the representatives from academics, industry, and government.
3.3.1. Key informant identification
The key informants will compose of academics, policymakers, and the salt or food industry representatives around the world. The sample size of the key informants will be determined by data saturation theory to inform the final sample size in this study. We will use a combination of purposive sampling and snowball sampling to recruit the interview participants. Purposive sampling will be informed by the results of the systematic online search in part I and the systematic review in part II. This purposive sampling method will involve contacting people with publicly available contact details. Snowball sampling will be also used to identify the potential participants. The interviewer may ask the recruited participants to pass on recruitment materials to other potential key informants to seek for their interest. The recruitment materials include a description of the study and study investigators’ contact information. If the potential participant does not response to the initial invitation, an email reminder will be sent 2 weeks after and no subsequent reminders will be sent after this.
3.3.2. Data collection
In‐depth interviews will be conducted by bilingual investigator (XY), who is fluent in both English and Chinese. Interviews will be conducted in English or Chinese as is appropriate. Key informant interviews will take place either over telephone or video conference or in the interviewees’ workplace. Semi‐structured interview guides are developed based on the RE‐AIM framework. 18 The framework has five domains addressing of the reach, effectiveness, adoption, implementation, and maintenance of the low‐sodium salt intervention, and interview guides will differ slightly, and be tailored to the different roles of academia, policymaker, and industry. (Table 3) Interviews will be audio‐recorded and transcribed verbatim. Participant identifiable information will not be collected and audio‐recorded during the interview. The interview is anticipated to take about 20‐30 minutes for each interviewee.
Table 3.
Semi‐structured questions based on RE‐AIM framework
| Academic investigators | Policymakers | Salt industry/food industry |
|---|---|---|
| Overview | ||
| Can you tell me about your project and describe the low sodium salt intervention in your study? | Were you or your work institute involved in salt reduction strategy in any way? Please tell us about your roles and responsibilities? | Can you tell me about your company (scale, sale area) and the low sodium salt products (formula, price, sales volume) of your study? |
| Is it a currently ongoing study? When and how long did it last? | Please tell us about your role and responsibility in your company? | |
| What is your responsibility in the study? | Why do you want to produce low sodium salts? | |
| Can you talk about the low sodium salt you used in your study? | Has your company involved in any research or public health activities? If yes, please tell me what the activities are? | |
| Why do you want to implement the low sodium salt intervention? | ||
| Reach | ||
| What is the target population for your salt‐substitute intervention? | Do you know if there is low sodium salt (low‐sodium salt) in your country? And what groups do you think are reached | What the target consumers of your low sodium salt products? |
| How widely used was the low sodium salt? | What groups do you think are not reached and why? | |
| Effectiveness | ||
| How has the use of low sodium salt impacted the research outcomes? | Do you think low sodium salts are an effective way to lower salt intake? Why or why not? | What do you think is the health impact of low sodium salt? |
| Do you think the low sodium salt intervention worked? Why or why not? | What settings or types of countries do you think low sodium salts will be effective in lowering salt intake? | Why do you think the low sodium salt intervention works (or does not work)? |
| What settings or types of countries do you think low sodium salts will be effective in lowering salt intake? | What settings or types of countries do you think low sodium salts will be effective in lowering salt intake? | |
| Adoption | ||
| Did the stakeholders involved in your study like to adopt low sodium salt intervention? Why or why not? | Would you like to use low sodium salts as an intervention? Why or why not? | What are the barriers for food industry to adopt the use of low sodium salts? And facilitators? |
| Can you suggest any strategies that would have increased the participant's use of the low sodium salts instead of normal salt? | What organizational issues impacted your capacity to carry out these responsibilities in reducing excessive salt intake in your country? | Compare with regular salt/ normal salt products, what are the barriers for consumers to adopt the use of low sodium salts? And facilitators? |
| Has the adoption of organization improved since the initial implementation phase? If so, can you provide some reasons why? | What do you think is the overall population's attitudes towards low sodium salt? | |
| Can you suggest any strategies that would have increased the participant's use of the low sodium salts instead of normal salt? | ||
| Implementation | ||
| What was your strategy for implementing low sodium salt intervention? | What was the strategy for implementing your low sodium salt intervention? (policy, health education campaign, industry engagement) What do you think is the role of low sodium salts in the overall salt reduction strategy? | Would you like to advocate for low sodium salts? What strategies you would like to use? |
| Can you describe any of the critical aspects that facilitated the implementation of the low sodium salt intervention? | Can you describe any of the key aspects that facilitated the implementation of the low sodium salt intervention? | Can you describe any of the key aspects that facilitated the use of low sodium salt? |
| Can you describe the barriers to implementation of the low sodium salt intervention? | Can you describe the barriers to implementation of the low sodium salt intervention? | What the barriers to increase the use of the low sodium salts instead of normal salt among consumer? And facilitators? |
| What the barriers to increase the use of the low sodium salts instead of normal salt among food industry? And facilitators? | ||
| Maintenance/Sustainability | ||
| Do you know whether the participants/organizations are still using or promoting low sodium salt after your study? | Can you suggest any strategies to ensuring the sustainability of the use of low sodium salts over normal salt? | From your company's perspective, do you want to invest in low sodium salt or use low sodium salt over normal salt in the food industry in future? Why and why not? |
| Can you suggest any strategies to ensuring the sustainability of the use of low sodium salts over normal salts? | ||
| Other opportunities | ||
| What do you think the feasible formulations for low sodium salts are? | What do you think are the feasible formulations for low sodium salts? | |
| From your perspective, are there safety issues in the use of low sodium salts? | From your perspective, are there safety issues in the use of low sodium salts? | |
| What opportunities do you have to promote using salt alternatives or substitutes? | Any opportunities to promote using salt alternatives or substitutes? | |
| What are the future research priorities on low sodium salts? | What is the future development plan or business strategy on low sodium salts? | |
| Is there anything we haven't mentioned that you would like to talk about related to the low sodium salt | Is there anything we haven't mention but you want to talk about low sodium salt |
3.3.3. Data analysis
Both inductive and deductive methods will be adopted to evaluate the interview transcripts. The headings of RE‐AIM will be used as an initial framework, and thematic analysis will be conducted to add new codes. The first round of the analysis will involve the first author (XY) reviewing the transcripts verbatim and inductively assigning codes to emergent concepts. Coding book will be generated by constant comparison until a hierarchy of conceptual codes, and no new concepts emerge. The second round of the analysis will use the same coding framework to apply codes to the transcripts by another coder independently. The research team will discuss the coding discrepancies and resolve by consensus to optimize intercoder reliability. The NVivo analytical software system will be used for data management and data coding.
CONFLICT OF INTEREST
JW is the director of the World Health Organization Collaborating Centre on Population Salt Reduction.
AUTHOR CONTRIBUTIONS
MT, HL, KT, and XY conceived the protocol review. MT, JW, SP, and CF provided guidance to the research. XY and KL designed the search strategy. XY wrote the review protocol. All authors provided a clinical perspective and commented on drafts of the protocol review and approved the final manuscript.
ETHICS AND DISSEMINATION
The study received its ethics approval from the University of New South Wales Human Research Ethics Advisory Panel (HC190921). The environmental scan will identify and describe the availability of salt substitute in the global market and the effectiveness, feasibility, and acceptability of low‐sodium salt interventions. The methodology used in this study combines systematic online search, systematic review, and key informant interviews. The key outputs from this study will be an understanding of the past and current efforts in low‐sodium salt interventions and will help to understand the enablers and barriers of implementing low‐sodium salt interventions. The study results will also inform future implementation science studies. We will disseminate the results of this review through publications in peer‐reviewed journals and at appropriate scientific meetings.
PATIENT AND PUBLIC INVOLVEMENT STATEMENT
Patient and public will not be invited to comment on the study design and the writing or editing of this manuscript.
Yin X, Liu H, Trieu K, et al. The effectiveness, feasibility, and acceptability of low‐sodium salts worldwide: An environmental scan protocol. J. Clin. Hypertens. 2020;22:2258–2265. 10.1111/jch.14054
Funding information
This research received no specific grant from any funding agency in the public, commercial, or not‐for‐profit sectors. XY and KL are supported by The University of New South Wales Scientia PhD Scholarship. JW is supported by a National Heart Foundation Future Leader II Fellowship (APP102039) and receives funding for work on salt reduction from The National Health and Medical Research Council, The World Health Organization, and The Victorian Health Promotion Foundation. KT was supported by an Early Career Fellowship (APP1161597) from The National Health and Medical Research Council of Australia (NHMRC) and a Postdoctoral Fellowship (Award ID: 102140) from The National Heart Foundation of Australia.
REFERENCES
- 1. World Health Organization . Global Status Report on Noncommunicable Diseases 2014. Geneva, Switzerland: World Health Organisation; 2014. [Google Scholar]
- 2. He FJ, Li J, MacGregor GAJB. Effect of longer term modest salt reduction on blood pressure: Cochrane systematic review and meta‐analysis of randomised trials. . BMJ. 2013;346:f1325. [DOI] [PubMed] [Google Scholar]
- 3. Mozaffarian D, Fahimi S, Singh GM, et al. Global sodium consumption and death from cardiovascular causes. N Engl J Med. 2014;371(7):624‐634. [DOI] [PubMed] [Google Scholar]
- 4. He FJ, MacGregor GA. Fortnightly review: beneficial effects of potassium. BMJ. 2001;323(7311):497‐501. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. He FJ, Li J, Macgregor GA. Effect of longer‐term modest salt reduction on blood pressure. Cochrane Database Syst Rev. 2013;4(7):CD004937. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Aaron KJ, Sanders PW. Role of dietary salt and potassium intake in cardiovascular health and disease: a review of the evidence. Mayo Clin Proc. 2013;88(9):987‐995. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Campbell NR, Lackland DT, Niebylski ML. 2014 dietary salt fact sheet of the World Hypertension League, International Society of Hypertension, Pan American Health Organization technical advisory group on cardiovascular disease prevention through dietary salt reduction, the World Health Organization collaborating centre on population salt reduction, and World Action on Salt & Health. J Clin Hypertens. 2015;17(1):7‐9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Roth GA, Johnson C, Abajobir A, et al. Global, regional, and national burden of cardiovascular diseases for 10 causes, 1990 to 2015. J Am Coll Cardiol. 2017;70(1):1‐25. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Peng YG, Li W, Wen XX, Li Y, Hu JH, Zhao LC. Effects of salt substitutes on blood pressure: a meta‐analysis of randomized controlled trials. Am J Clin Nutr. 2014;100(6):1448‐1454. [DOI] [PubMed] [Google Scholar]
- 10. Hernandez AV, Emonds EE, Chen BA, et al. Effect of low‐sodium salt substitutes on blood pressure, detected hypertension, stroke and mortality. Heart. 2019;105:953‐960. [DOI] [PubMed] [Google Scholar]
- 11. Graham P, Evitts T, Thomas‐MacLean RJCFP. Environmental scans: How useful are they for primary care research? Can Fam Physician. 2008;54(7):1022‐1023. [PMC free article] [PubMed] [Google Scholar]
- 12. Charlton P, Doucet S, Azar R, et al. The use of the environmental scan in health services delivery research: a scoping review protocol. BMJ Open. 2019;9(9):e029805. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Moher D, Liberati A, Tetzlaff J, Altman DGJAoim . Preferred reporting items for systematic reviews and meta‐analyses: the PRISMA statement. Ann Intern Med. 2009;151(4):264‐269. [DOI] [PubMed] [Google Scholar]
- 14. Higgins JP, Green S. Cochrane Handbook for Systematic Reviews of Interventions. Wiley Online Library; 2008. [Google Scholar]
- 15. Sterne JA, Hernan MA, Reeves BC, et al. ROBINS‐I: a tool for assessing risk of bias in non‐randomised studies of interventions. BMJ. 2016;355:i4919. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Guyatt G, Oxman AD, Akl EA, et al. GRADE guidelines: 1. Introduction‐GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011;64(4):383‐394. [DOI] [PubMed] [Google Scholar]
- 17. Peterson J, Welch V, Losos M, Tugwell P.The Newcastle‐Ottawa scale (NOS) for assessing the quality of nonrandomised studies in meta‐analyses. 2011.
- 18. Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE‐AIM framework. Am J Public Health. 1999;89(9):1322‐1327. [DOI] [PMC free article] [PubMed] [Google Scholar]