Abstract
Introduction
Most health information is verbal or written, yet words alone may not be the most effective way to communicate health information. Lower health literacy is prevalent in the US and is linked to limited understanding of one’s medical condition and treatment. Pictures increase comprehension, recall, adherence and attention in health settings. This is called pictorial superiority. No systematic review has examined the impact of pictorial health information among patients and consumers, including those with lower health literacy.
Methods and analysis
This systematic review and meta-analysis will assess the characteristics and effectiveness of pictorial health information on patient and consumer health behaviours and outcomes, as well as differentially among individuals of lower literacy/lower health literacy. We will conduct a systematic search across selected databases, as well as grey literature, from inception until June 2018. We will include randomised controlled trials in all languages with all types of participants that assess the effect of pictorial health information on patients’ and consumers’ health behaviours and outcomes. Two independent reviewers will conduct the primary screening of articles and data extraction for the selected articles with a third individual available to resolve conflicts. We will assess the quality of all included studies using the Cochrane risk of bias tool. We will combine all selected studies and do a test of heterogeneity. If there is sufficient homogeneity, we will pool studies into a meta-analysis. Independent of the heterogeneity of included studies, we will also conduct a narrative synthesis.
Ethics and dissemination
No ethics approval is required. The results will be published in a peer-reviewed journal and presented at relevant conferences.
PROSPERO registration number
CRD42018084743.
Keywords: health information, health communication, health literacy, picture, picture superiority, visual aid
Strengths and limitations of this study.
We included only randomised controlled trials in order to provide robust and comparable information on the impact of pictorial health information on patients’ and consumers’ health behaviours and other outcomes.
Our search strategy is very broad in order to capture as many relevant randomised controlled trials as possible.
We limited the search to randomised controlled trials, which is likely to exclude relevant quasi-experimental studies.
Introduction
From verbal consultations to lengthy after visit summaries, words pervade health communication.1 An estimated 75% of physicians routinely hand out written patient education materials.1 2 Because the average US adult reads at an eighth grade reading level or below, the National Institute of Health and American Medical Association recommend patient education materials be at a sixth grade reading level or below.3 4 Despite this, the reading level of most health literature is above eighth grade, indicating a discrepancy between what patients can understand and what is currently available to them.1 5–15 Research suggests words are not an effective way to communicate health information to patients and consumers across the literacy/health literacy spectrum.16–19Patients struggle to comprehend and retain written health information.20 In one study, average medication recall accuracy was only 53% 2 days after receiving a written discharge summary, regardless of health literacy.21
Language barriers and prior knowledge and experience are also frequent limitations to patients’ comprehension of written health information since this type of information is topic specific and necessitates prior knowledge.16 17 Language barriers and lack of experience can also make patients with lower literacy/health literacy less likely to comprehend words used in health communication.16 18 19 In one study, patients of lower health literacy reported poor communication because their doctors could not explain their condition in a way that they could understand.18 Accordingly, people with lower health literacy show limited understanding of their medical conditions or treatments and have poorer health outcomes.5 22 23 This is concerning since only 12% of the US adult population has proficient health literacy.24 In addition, a 2005 systematic review reported that the prevalence of limited health literacy in the US is 26%.25
Pictorial superiority is the tendency to understand and remember information more easily when it is presented as pictures rather than words.26 Pictures can facilitate word memory and conceptual processing27 and demand less cognitive effort to understand than words.6 28 Research indicates that all individuals can benefit from pictures as health information. Pictorial information is especially useful for individuals with lower textual literacy,20 29–31 suggesting that they compensate by being more visually literate.32 Pictures have been shown to improve comprehension when closely linked with text or spoken words compared with text alone.28 30 31 33 One study found that patients were 50% more likely to correctly answer comprehension questions when pictorial information was included in their discharge instructions.34 This relationship increased for patients with lower literacy/lower health literacy.34 Pictures can also increase attention, recall of information and adherence.30 In one study, patients provided with pictorial information in their discharge instructions could immediately recall 35% more information than their non-picture counterparts.35 In another study, the use of pictorial health information benefited individuals with lower health literacy in both adherence to and compliance of medication.20 The effects of pictorial superiority have been examined in many health information contexts and show promise for improving patients’ conceptual processing, attention, comprehension and recall of health information no matter their health literacy level.20 27 28 30 31 33–35
Despite these suggested benefits, previous reviews of pictorial superiority were not systematic and only included studies published in English.30 31 36 One review only compared text versus text with pictures,30 and another focused exclusively on pictograms used for patient education.37 Relevant reviews found in two preliminary searches contained no strict inclusion criteria, showed no limitation to randomised controlled trials and had no assessment of methodological quality.30 31 37 The reviews did not evaluate all possible health behaviour effects of pictorial health information, especially among those with lower literacy.31 32 36 38 No review currently exists that systematically evaluates how pictorial information affects all patient and consumer health behaviours and other outcomes in controlled contexts, as well as differentially among individuals with lower health literacy. In order to address these gaps, our systematic review aims to assess the effect of pictorial health information on patient and consumer health behaviours, as well as other outcomes. Our secondary, exploratory aim is to differentially evaluate the effect of pictorial health information on the health behaviours and outcomes of individuals who have lower literacy/lower health literacy. Our third aim is to examine the characteristics and delivery attributes of the pictorial health interventions that are intended for patients and consumers.
Methods and analysis
We will conduct a systematic review of the literature to assess the effectiveness and characteristics of pictorial health information on patient and consumer health behaviours (primary and tertiary aims) and focus on the impact of pictorial health information on individuals of lower literacy/lower health literacy (secondary/exploratory aim). This protocol was peer-reviewed by two reviewers (PB, FB). The following methods and analysis are aligned with Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols checklist criteria (see online supplementary appendix 1).39
bmjopen-2018-023300supp001.pdf (147.7KB, pdf)
The following research questions were used to guide the systematic review process:
What is the effectiveness of pictorial health information on patient and consumer health behaviours as well as other outcomes?
What is the effectiveness of pictorial health information on health behaviours and other outcomes in people of lower literacy/lower health literacy (exploratory analysis)?
What are the characteristics and delivery attributes of pictorial health interventions used in healthcare?
Patient and public involvement
Patients and the public were not involved in the design or data collection of this systematic review and meta-analysis.
Search strategy
We developed the search strategy with a research and education librarian from Dartmouth Biomedical Libraries and piloted in Ovid MEDLINE. We will perform electronic searches in MEDLINE, PsycINFO, Web of Science, Cumulative Index to Nursing and Allied Health Literature, Cochrane Database of Systematic Reviews, Cochrane Central Register of Controlled Trials, Cochrane Methodology Register, ERIC and the Database of Abstracts and Reviews of Effectiveness from inception until June 2018 (see table 1). We will write a list of keywords and subject headings in Ovid MEDLINE and run it in each database (see search strategy in online supplementary appendix 2).
Table 1.
Search strategy databases
Cumulative Index to Nursing and Allied Health Literature (CINAHL) | 1982–current |
Cochrane Database of Systematic Reviews (CDSR) | 1996–current |
Cochrane Central Register of Controlled Trials (CENTRAL) | 1996–current |
Cochrane Methodology Register (CMR) | 1980–current |
MEDLINE | 1946–current |
MEDLINE In-Process and Other Non-Indexed Citations | 1951–current |
PsycINFO | 1806–current |
Web of Science | 1900–current |
ERIC | 1964–current |
bmjopen-2018-023300supp002.pdf (2.6MB, pdf)
Additional search methods
Two independent reviewers (DS and CHS) will manually search the reference list of all included primary and relevant review articles to identify studies that have not been picked up by the electronic search. We will also perform a citation search using the ‘cited by’ option in Google Scholar (additional search methods are outlined in box 1).
We will search key journals, grey literature (ie, technical reports, works in progress) and conference proceedings (International Conference on Shared Decision Making, International Conference on Communication in Healthcare, Health Literacy Annual Research Conference, Institute for Healthcare Advancement Annual Health Literacy Conference and Wisconsin Health Literacy Summit).
We will use Google Scholar to search across the full text of cross-disciplinary articles. Two reviewers (DS and RWY) will manually search the first 100 hits in Google Scholar while documenting any discrepancies in the search results. We will use this search strategy: (Graphic OR Image OR Infographic OR Pictogram OR Pictorial OR Picture OR Visual OR Pictograph)(patient OR ‘health consumer’)(information OR instruction OR aid OR tool OR message OR education).
We will search ClinicalTrials.gov and the WHO International Clinical Trials Registry Platform for randomised controlled trials. We will adapt the search strategy appropriately to search within these databases.
Box 1. Additional search methods.
Websites
Google Scholar: http://scholar.google.com/
Clinical Trials: https://clinicaltrials.gov/
WHO International Clinical Trials Registry Platform: http://apps.who.int/trialsearch/
Conference proceedings
International Conference on Shared Decision Making
International Conference on Communication in Healthcare
Health Literacy Annual Research Conference
Institute for Healthcare Advancement Annual Health Literacy Conference
Wisconsin Health Literacy Summit
Two researchers (DS and SC) will independently assess the title and abstract of retrieved records and the full text articles meeting the inclusion criteria. We will resolve disagreements on inclusion by arbitration with a third person (M-AD).
Inclusion criteria
Types of studies
We will include all randomised controlled trials published in all languages that assess the effect of pictorial health information (all outcomes included) on adults and children who may or may not have an illness. We will also include prevention studies conducted in the general population (consumers). A certified translation company or a colleague fluent in the target language will translate studies in a language other than English.
Types of participants
We will include patients and consumers of all ages with or without an illness, as well as all conditions and clinical settings (e.g., lay care, primary, secondary care). We will not include medical students and health professionals as they are considered experts in health information.
For the secondary aim/exploratory analysis, we will examine outcomes in people of lower literacy and lower health literacy. We will accept multiple definitions of literacy and health literacy as authors have different definitions and markers of lower health literacy. For the purpose of this secondary analysis, we will include studies that have recruited at least 50% of people of lower literacy/lower health literacy or report results for that group separately.
Types of interventions
We will include all interventions that provide pictorial health information. This includes pictograms, pictographs and pictures in both paper and digital format. We will include studies with multiple interventions as long as one of the interventions includes pictorial information with or without supplementation of 100 words or less. We will also require a text, verbal or usual care control to clarify the effect of the pictorial health information.
Types of outcome measures
We will include all outcome measures that assess a health behaviour change in patients and consumers as well as other outcomes measured.
The primary outcome measure will be:
Health behaviour change in patients and consumers as a result of being exposed to pictorial health information. This may include adherence to prescribed medication, eating healthier foods, complying with discharge information and other health behaviours.
The secondary outcome measures will include all other outcome measurements such as:
Comprehension.
Information recall.
Accurate risk perception.
Exclusion criteria
We will not consider videos and three-dimensional models as a form of pictorial information as they add the feature of animation and movement. We will exclude graphs, icon arrays and bar charts since they are not truly pictorial. We will exclude studies that use text-only information or pictorial health information accompanied by a large amount of text (more than 100 words per picture) because the information needs to be primarily picture based. We will exclude studies with decision aids as they are a confounder to the effect of the pictorial health information. We will exclude pilot randomised controlled trials, and we will exclude studies that use graphic warning labels as there is a risk of too many confounding variables.
Preliminary searches
We conducted a preliminary search in Google Scholar to identify existing systematic reviews and assess the volume of potentially included articles using the following terms: pictorial health information, pictorial health decisions, pictograph patient adherence, pictograph patient comprehension and pictograph patient attention. We assessed literature reviews and the most cited randomised controlled trials from the Google Scholar search. The search retrieved several reviews assessing the impact of pictorial aids in medication instructions and the use of pictograms in healthcare, but no systematic review was identified that specifically investigated the impact of pictorial health information on patient behaviours and other outcomes.
We conducted a second preliminary search in PubMed using the following search strategy: (((pictorial health information[Title]) OR picture[Title]) OR visual[Title]) AND review[Title]. The search revealed 687 articles, none of which were a systematic review assessing the effects of pictorial health information on all health behaviours and other outcomes.
Assessment of methodological quality
We will consider and appraise the risk of bias of all included studies. We will rate the risk of bias of included randomised controlled trials using the Cochrane risk of bias tool where each section of the risk of bias tool will be categorised as either high, low or unclear.40 We plan to only assess each main outcome. Two independent assessors will use the risk of bias tool for all included studies. Each assessor will be trained on using the risk of bias tool before initiating the quality assessment. We will resolve discrepancies by discussion and consensus.
Data extraction
We will perform an independent double data extraction, using a predesigned form, adapted from the Cochrane Effective Practice and Organization of Care collection checklist.41 Four researchers will pilot the data extraction form independently using three studies specifically selected for this pilot exercise. We will resolve inconsistencies by discussion. We will review and consider all search results for inclusion using Rayyan, a web application designed for screening systematic review records.42 Since Rayyan is an online application, data are continuously backed up and managed by the website.
We will extract information about (1) the author(s); (2) publication year; (3) country; (4) type of study design; (5) aim(s) and research questions; (6) type of participants and sample size; (7) setting; (8) characteristics and duration of the delivery of the pictorial health intervention; (9) follow-up; (10) control condition; (11) number of participants included in analysis for both intervention and control groups and (12) outcome measures. We will adapt the Template for Intervention Description and Replication checklist to assess the description of each intervention.43
Data synthesis
We will assess heterogeneity using the χ2 test and I2 test.44 If there is sufficient homogeneity, we will pool studies in a meta-analysis with dichotomous outcomes presented as relative risks (RR) and continuous data as mean differences (MD). We will use a random effects model for our analysis. We will use funnel plots to evaluate potential publication bias. Significance will be assumed at p<0.05.
We will produce a narrative review independent of the heterogeneity of included studies, with an indication of whether the effect of the intervention was positive, negative or not statistically significant. Where possible, we will report dichotomous outcomes as RR and continuous data as MD, both with 95% CI.
Ethics and dissemination
The findings of this systematic review will be submitted for publication in a peer-reviewed journal and presented at relevant conferences. This protocol has been registered on the international PROSPERO, and the systematic review will be conducted according to the PRISMA statement. Should there be any protocol amendments, all amendments will be recorded and noted in the final systematic review manuscript.
Supplementary Material
Acknowledgments
Paul Barr and Frances Bunn peer-reviewed the protocol that is registered on PROSPERO. Heather Blunt and Pamela Bagley assisted in the development and running of the search strategy across the chosen databases.
Footnotes
Contributors: DS, M-A D: study conception. DS, SC, M-A D: search strategy with Dartmouth Biomedical Libraries. DS: manuscript drafting. DS, M-A D, RWY: critical revisions of manuscript. DS, M-A D, RWY, SC, PS, CHS, MvdM: final approval of the version to be published.
Funding: This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests: M-AD has developed the Option Grid patient decision aids, which are licensed to EBSCO Health. She receives consulting income from EBSCO Health and may receive royalties in the future. M-AD is a consultant for ACCESS Community Health Network.
Patient consent: Not required.
Provenance and peer review: Not commissioned; externally peer reviewed.
References
- 1. Rowlands G, Protheroe J, Winkley J, et al. A mismatch between population health literacy and the complexity of health information: an observational study. Br J Gen Pract 2015;65:e379–e386. 10.3399/bjgp15X685285 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Carrier E, Reschovsky J. Expectations outpace reality: physicians' use of care management tools for patients with chronic conditions. Issue Brief Cent Stud Health Syst Change 2009;129:1–4. [PubMed] [Google Scholar]
- 3. Doak CC, Doak LG, Friedell GH, et al. Improving comprehension for cancer patients with low literacy skills: strategies for clinicians. CA Cancer J Clin 1998;48:151–62. 10.3322/canjclin.48.3.151 [DOI] [PubMed] [Google Scholar]
- 4. Eltorai AE, Ghanian S, Adams CA, et al. Readability of patient education materials on the american association for surgery of trauma website. Arch Trauma Res 2014;3:e18161 10.5812/atr.18161 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Andrus MR, Roth MT. Health literacy: a review. Pharmacotherapy 2002;22:282–302. 10.1592/phco.22.5.282.33191 [DOI] [PubMed] [Google Scholar]
- 6. Michielutte R, Bahnson J, Dignan MB, et al. The use of illustrations and narrative text style to improve readability of a health education brochure. J Cancer Educ 1992;7:251–60. 10.1080/08858199209528176 [DOI] [PubMed] [Google Scholar]
- 7. Stossel LM, Segar N, Gliatto P, et al. Readability of patient education materials available at the point of care. J Gen Intern Med 2012;27:1165–70. 10.1007/s11606-012-2046-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Reicherter EA, Billek B, Chesbro S. An Analysis of the Readability of Educational Materials on the Consumer Webpage of a Health Professional Organization: Considerations for Practice. 2011. http://nsuworks.nova.edu/cgi/viewcontent.cgi?article=1360&context=ijahsp (accessed 9 Feb 2018).
- 9. Wilson M. Readability and patient education materials used for low-income populations. Clin Nurse Spec 2009;23:33–40. 10.1097/01.NUR.0000343079.50214.31 [DOI] [PubMed] [Google Scholar]
- 10. Estey A, Musseau A, Keehn L. Patient’s understanding of health information: a multihospital comparison. Patient Educ Couns 1994;24:73–8. 10.1016/0738-3991(94)90027-2 [DOI] [PubMed] [Google Scholar]
- 11. Shieh C, Hosei B. Printed health information materials: evaluation of readability and suitability. J Community Health Nurs 2008;25:73–90. 10.1080/07370010802017083 [DOI] [PubMed] [Google Scholar]
- 12. Paasche-Orlow MK, Taylor HA, Brancati FL. Readability standards for informed-consent forms as compared with actual readability. N Engl J Med 2003;348:721–6. 10.1056/NEJMsa021212 [DOI] [PubMed] [Google Scholar]
- 13. Cotugna N, Vickery CE, Carpenter-Haefele KM. Evaluation of literacy level of patient education pages in health-related journals. J Community Health 2005;30:213–9 http://www.ncbi.nlm.nih.gov/pubmed/15847246 10.1007/s10900-004-1959-x [DOI] [PubMed] [Google Scholar]
- 14. Comings JE, Garner BE, Smith CE. Annual Review of Adult Learning and Literacy. Volume 3. The Jossey-Bass Higher and Adult Education Series. 2002. https://files.eric.ed.gov/fulltext/ED478329.pdf (accessed 9 Feb 2018).
- 15. Baker DW, Parker RM, Williams M V, et al. The health care experience of patients with low literacy patients and methods. 2009. www.archfammed.com (accessed 9 Feb 2018). [DOI] [PubMed]
- 16. Lukoschek P, Fazzari M, Marantz P. Patient and physician factors predict patients' comprehension of health information. Patient Educ Couns 2003;50:201–10. 10.1016/S0738-3991(02)00128-3 [DOI] [PubMed] [Google Scholar]
- 17. Friedman AJ, Cosby R, Boyko S, et al. Effective teaching strategies and methods of delivery for patient education: A systematic review and practice guideline recommendations Journal of Cancer Education. Springer-Verlag, 2011;26:12–21. 10.1007/s13187-010-0183-x [DOI] [PubMed] [Google Scholar]
- 18. Schillinger D, Bindman A, Wang F, et al. Functional health literacy and the quality of physician-patient communication among diabetes patients. Patient Educ Couns 2004;52:315–23. 10.1016/S0738-3991(03)00107-1 [DOI] [PubMed] [Google Scholar]
- 19. Sentell T, Braun KL. Low health literacy, limited English proficiency, and health status in Asians, Latinos, and other racial/ethnic groups in California. J Health Commun 2012;17 Suppl 3:82–99. 10.1080/10810730.2012.712621 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Kessels RP. Patients' memory for medical information. J R Soc Med 2003;96:219–22. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Pavlik V, Brown AE, Nash S, et al. Association of patient recall, satisfaction, and adherence to content of an electronic health record (EHR)-generated after visit summary: a randomized clinical trial. J Am Board Fam Med 2014;27:209–18. 10.3122/jabfm.2014.02.130137 [DOI] [PubMed] [Google Scholar]
- 22. Wolf MS, Gazmararian JA, Baker DW. Health literacy and functional health status among older adults. Arch Intern Med 2005;165:1946–52. 10.1001/archinte.165.17.1946 [DOI] [PubMed] [Google Scholar]
- 23. Williams MV, Davis T, Parker RM, et al. The role of health literacy in patient-physician communication. Fam Med 2002;34:383–9. [PubMed] [Google Scholar]
- 24. Kutner M, Greenberg E, Jin Y, et al. The health literacy of America’s adults: results from the 2003 National Assessment of Adult Literacy. Education 2006;6:1–59. [Google Scholar]
- 25. Paasche-Orlow MK, Parker RM, Gazmararian JA, et al. The prevalence of limited health literacy. J Gen Intern Med 2005;20:175–84. 10.1111/j.1525-1497.2005.40245.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Hockley WE, Bancroft T. Extensions of the picture superiority effect in associative recognition. Can J Exp Psychol 2011;65:236–44. 10.1037/a0023796 [DOI] [PubMed] [Google Scholar]
- 27. Reed LA, Hoffman LG. Pictorial cues and enhancement of patient recall of instructions or information. J Am Optom Assoc 1986;5:312. [PubMed] [Google Scholar]
- 28. Delp C, Jones J. Communicating information to patients: the use of cartoon illustrations to improve comprehension of instructions. Acad Emerg Med 1996;3:264–70. 10.1111/j.1553-2712.1996.tb03431.x [DOI] [PubMed] [Google Scholar]
- 29. Sudore RL, Schillinger D. Interventions to improve care for patients with limited health literacy. J Clin Outcomes Manag 2009;16:20–9. [PMC free article] [PubMed] [Google Scholar]
- 30. Houts PS, Doak CC, Doak LG, et al. The role of pictures in improving health communication: a review of research on attention, comprehension, recall, and adherence. Patient Educ Couns 2006;61:173–90. 10.1016/j.pec.2005.05.004 [DOI] [PubMed] [Google Scholar]
- 31. Levie WH, Lentz R. Effects of text illustrations: a review of research. Educ Commun Technol 1982;30:195–232. [Google Scholar]
- 32. Alberto PA, Fredrick L, Hughes M, et al. Components of visual literacy. Sage 2007;22:234–43. [Google Scholar]
- 33. Brotherstone H, Miles A, Robb KA, et al. The impact of illustrations on public understanding of the aim of cancer screening. Patient Educ Couns 2006;63:328–35. 10.1016/j.pec.2006.03.016 [DOI] [PubMed] [Google Scholar]
- 34. Austin PE, Matlack R, Dunn KA, et al. Discharge instructions: do illustrations help our patients understand them? Ann Emerg Med 1995;25:317–20. 10.1016/S0196-0644(95)70286-5 [DOI] [PubMed] [Google Scholar]
- 35. Hill B, Perri-Moore S, Kuang J, et al. Automated pictographic illustration of discharge instructions with Glyph: impact on patient recall and satisfaction. J Am Med Inform Assoc 2016;23:1136–42. 10.1093/jamia/ocw019 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36. Katz MG, Kripalani S, Weiss BD. Use of pictorial aids in medication instructions: a review of the literature. Am J Health Syst Pharm 2006;63:2391–7. 10.2146/ajhp060162 [DOI] [PubMed] [Google Scholar]
- 37. Barros IM, Alcântara TS, Mesquita AR, et al. The use of pictograms in the health care: a literature review. Res Social Adm Pharm 2014;10:704–19. 10.1016/j.sapharm.2013.11.002 [DOI] [PubMed] [Google Scholar]
- 38. Campbell FA, Goldman BD, Boccia ML, et al. The effect of format modifications and reading comprehension on recall of informed consent information by low-income parents: a comparison of print, video, and computer-based presentations. Patient Educ Couns 2004;53:205–16. 10.1016/S0738-3991(03)00162-9 [DOI] [PubMed] [Google Scholar]
- 39. Shamseer L, Moher D, Clarke M, et al. PRISMA-P (Preferred Reporting Items for Systematic review and Meta-Analysis Protocols) 2015 checklist : recommended items to address in a systematic review protocol *. BMJ 2015;349:g7647. [DOI] [PubMed] [Google Scholar]
- 40. Higgins JP, Altman DG, Gøtzsche PC, et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ 2011;343:d5928 10.1136/bmj.d5928 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41. Cochrane Effective Practice and Organisation of Care Review Group DATA COLLECTION CHECKLIST Cochrane Effective Practice and Organisation of Care Review Group (EPOC) Data Collection Checklist. http://methods.cochrane.org/sites/methods.cochrane.org.bias/files/public/uploads/EPOC Data Collection Checklist.pdf (accessed 23 Feb 2018).
- 42. Ouzzani M, Hammady H, Fedorowicz Z, et al. Rayyan-a web and mobile app for systematic reviews. Syst Rev 2016;5:210 10.1186/s13643-016-0384-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43. Hoffmann TC, Glasziou PP, Boutron I, et al. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ 2014;348:g1687 10.1136/bmj.g1687 [DOI] [PubMed] [Google Scholar]
- 44. Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med 2002;21:1539–58. 10.1002/sim.1186 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
bmjopen-2018-023300supp001.pdf (147.7KB, pdf)
bmjopen-2018-023300supp002.pdf (2.6MB, pdf)