Skip to main content
BMJ Open logoLink to BMJ Open
. 2022 Mar 17;12(3):e058425. doi: 10.1136/bmjopen-2021-058425

Decision regret regarding treatments among women with early-stage breast cancer: a systematic review protocol

Jing Liu 1,, Sharyn Hunter 1, Jiemin Zhu 2, Regina Lai-Tong Lee 1, Sally Wai-Chi Chan 3
PMCID: PMC8932263  PMID: 35301213

Abstract

Introduction

Women with early-stage breast cancer (EBC) are commonly required to make treatment decisions. Decision regret regarding treatments is an adverse outcome that negatively affects women’s psychological well-being and quality of life. A systematic review will be conducted to synthesise evidence about decision regret among women regarding treatments for EBC. The study will focus on levels of decision regret, what is regretted, and the factors associated with decision regret.

Methods and analysis

A systematic review will be conducted following the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols 2015 checklist. Electronic databases, including CINAHL Complete, Embase, PubMed, Medline and Web of Science, will be searched for relevant articles published from 2000 to 2021. The reference lists of eligible studies will also be manually searched. All types of quantitative, qualitative and mixed-methods studies that report on decision regret regarding treatments among women with EBC will be included. The primary outcome of this review will be women’s levels of decision regret regarding breast cancer treatments. The secondary outcomes will include the content of their regrets, and the factors contributing to decision regret. The methodological quality of the studies will be assessed using the Joanna Briggs Institute appraisal tools. Meta-analysis and thematic synthesis approaches will be used to synthesise quantitative and qualitative data, respectively. A convergent parallel approach will be used to integrate quantitative and qualitative findings.

Ethics and dissemination

Ethical approval is not required for this systematic review. The findings of this work will be disseminated at international conferences and peer-reviewed journals. The findings of this systematic review will inform the development of decision interventions to improve the decision outcomes of breast cancer treatments.

PROSPERO registration number

CRD42021260041.

Keywords: breast neoplasm, regret, systematic review, qualitative, meta-analysis, factors


Strengths and limitations of this study.

  • This review will include all types of studies which differs from other reviews that only include randomised control trials.

  • Meta-analysis and narrative description will be used to analyse quantitative data, and a thematic synthesis approach will be used to synthesise qualitative data, producing robust evidence regarding decision regret about breast cancer treatments.

  • A third integrative analysis using a convergent parallel approach will be conducted to incorporate the quantitative and qualitative findings; thus, the review findings will be convergent and complementary.

  • This review only includes studies published in English; thus, eligible studies published in other languages may be missed.

Introduction

Breast cancer is the most prevalent malignancy among women, contributing to 11.7% of new cancer cases in 2020.1 In many countries, the majority of the women with breast cancer were found to have early-stage breast cancer (EBC) at the time of diagnosis.2 3

Women with EBC have a milder form of the disease, superior cure rates and more treatment options than those with advanced and metastatic breast cancer (stages 3 and 4). Clinical trials suggest that survival rates after mastectomy and breast conserving surgery (BCS) for women with EBC are equivalent4 5; therefore, it is important to empower these women to make treatment decisions for themselves to achieve ‘shared decision-making’ in breast cancer care.6 7 However, choosing among multiple treatment options can be difficult. For example, a mastectomy surgery benefits women by a lower risk of recurrence but causes relatively larger body image impairment, while a BCS helps women maintain breast image but exposes them to a higher risk of local recurrence.5 Therefore, when choosing between mastectomy and BCS, women must weigh the benefits and side effects of each option. Negative emotions, such as fear, can further complicate the decision-making about breast cancer treatments.8 9 Facing the difficult treatment choices, some women with EBC may make a decision that they will regret in the future.10 11 Thus, it is important to understand the decision-making behaviour of women with EBC.

In the context of healthcare, decision regret refers to ‘remorse or distress over a decision’.12 Decision regret is a significant indicator of treatment decision efficacy and may emerge when patients feel that they could have had a better outcome if they had chosen a different treatment.12 13 In a study from the United States of America (USA), Advani et al14 reported that 100 out of 421 (23.8%) older women with breast cancer (≥67 years) had experienced decision regret regarding some forms of local therapy (eg, lumpectomy with whole-breast radiotherapy, brachytherapy, or endocrine therapy or mastectomy). In this study, decision regret was associated with race, education level and the extent of nodal dissection performed, but not the type of therapy.14 In another survey among young women (<51 years), 42.5% of 449 women with breast cancer experienced decision regret 5 years after treatment. Of these women, 24.1% regretted having primary surgery and 21.5% regretted having chemotherapy or radiotherapy.15 Qualitative explorations also reported that women expressed regret about their treatment decisions,15–17 and their regrets were mostly associated with not engaging in the decision-making process16 and inadequate information.15 17

The physical and psychosocial consequences associated with decision regret have been discussed in previous literature.13 15 17–19 Regretting a treatment decision has been associated with a higher probability of undergoing a second round of treatment, which may result in delayed recovery and additional trauma. Regret regarding cancer treatments has also been associated with poor psychological well-being and quality of life.19 Experiencing regret about a treatment decision may also increase patients’ financial burdens, especially for patients from economically disadvantaged backgrounds. Additionally, when regret occurs, the unsatisfactory treatment outcomes may also harm the relationship between patients and their healthcare providers 20 21, such as distrust and low satisfaction with doctors. Thus, it is important for healthcare providers to support patients’ decision-making and minimise the occurrence of decision regret in clinical practice.

Reviews summarising evidence about patients’ decision-making about breast cancer treatments have been published.22 23 However, they did not specifically address the issue of regret about treatment decisions, and were not able to generalise to the whole population because these reviews only included older women22 and women who had a mastectomy.23 Previous reviews on decision regret related to breast reconstruction24 and risk-reducing treatment25 26 also could not provide a holistic understanding of decision regret regarding EBC treatments because other treatments, such as BCS, chemotherapy and radiotherapy, are also common choices for EBC. To date, there is a lack of literature synthesis regarding levels of decision regret, what patients regret, and factors associated with decision regret regarding breast cancer treatments among women with EBC. Without such an understanding, it is difficult for healthcare professionals to develop supportive interventions to help women make treatment decisions.

Review objectives

A systematic review will be conducted to assess studies dealing with decision regret regarding breast cancer treatments among women with EBC. The treatment approaches of interest will include unilateral mastectomy, BCS, chemotherapy, radiotherapy, endocrine therapy and targeted therapy. The Participants, Interventions, Comparators and Outcomes elements used for the systematic review are listed in table 1. The detailed objectives are:

Table 1.

The PICO elements used as selection criteria in this systematic review

Participants
  • Women with early-stage breast cancer (stage 0/carcinoma in situ, stage 1 and stage 2).

Intervention/Exposure
  • Experienced decision regret regarding their treatments.

Comparators
  • No restriction. The comparator depends on the study design.

  • For example, the comparator could be a group of the normal population for a cohort comparative study; if the included study is a randomised control trial to assess the effectiveness of a decision aid, the comparator could be the group of the population who had not received the intervention.

Outcomes
  • Levels of decision regret.

  • Events/processes/things that women regret.

  • Factors associated with decision regret.

Settings
  • No restriction.

  1. To assess levels of decision regret about treatments among women with EBC.

  2. To identify what women regret.

  3. To identify factors associated with decision regret.

Methods and analysis

This protocol has been registered with the International Prospective Register of Systematic Reviews. The Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols 2015 checklist is followed to report this protocol27 (see online supplemental file 1). This systematic review is anticipated to be performed during August 2021 and January 2022.

Supplementary data

bmjopen-2021-058425supp001.pdf (48.3KB, pdf)

Study selection

Information sources

The researchers will search electronic databases including CINAHL Complete, Embase, PubMed, Medline and Web of Science. A manual search of the reference lists of eligible studies will be also performed. This systematic review will include primary studies published from January 2000 to June 2021 in order to provide the most recent evidence.

Selection process

Two researchers will independently conduct the literature search (JL and SH). Another researcher will validate the search process to ensure accuracy (SW-CC). All studies will be exported using Endnote X9 software for duplicate removal and further screening. Thereafter, two researchers will independently review the titles, abstracts and full texts of these papers to determine their eligibility (JL and SH). Disagreement about study eligibility will be resolved through discussions among all researchers (JL, SH, JZ, RL-TL and SW-CC). The selection process will be presented in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow diagram (see figure 1).

Figure 1.

Figure 1

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow diagram and selection process.

Search terms

The keyword search terms are derived from the main concepts of the research topic. The key terms include “breast cancer”, “breast tumor”, “breast neoplasm”, “breast carcinoma”, “early-stage breast cancer”, “early breast cancer”, “regret”, “decision regret”, “treatment”, “intervention” and “therapy”. Search strategies for all databases are presented in online supplemental file 2.

Supplementary data

bmjopen-2021-058425supp002.pdf (17.2KB, pdf)

Inclusion and exclusion criteria

All relevant quantitative, qualitative and mixed-methods studies written in the English language will be included. There will be no restrictions regarding the design or setting of the study, as long as they are:

  1. Primary studies reporting decision regret regarding unilateral mastectomy, BCS, chemotherapy, radiotherapy, endocrine therapy or targeted therapy.

  2. Among women with EBC.

  3. Published in the English language.

Studies regarding risk-reducing treatment (eg, contralateral prophylactic mastectomy) and reconstructive surgery will be excluded because there have been three published reviews specifically addressing these surgeries.24–26 Secondary studies and grey literature will also be excluded.

Analysis

Outcome measures

The primary outcome of the systematic review will be women’s levels of decision regret regarding breast cancer treatments. Levels of decision regret were measured using different methods across the studies. For example, decision regret was measured using the validated 5-item, 5-point Likert Decision Regret Scale in Advani et al’s study. Researchers in this study considered scores 1, 2 and 3 as feeling regret, while 4 and 5 indicated no regret.14 Martinez et al 28 revised the items of the Decision Regret Scale. In this study, the ratings of each item were summed up as total scores ranging from 0 to 20, where higher scores indicated higher levels of regret.28 Regret was also measured by a single-item numerical rating scale in Yamauchi et al’s study, in which the researchers converted the scores into dichotomous variables as ‘having no regret’ or ‘having some regret’ about breast cancer treatments.29

The secondary outcomes will include what women regret when they recall their decision-making process and the factors contributing to their decision regret.

Data evaluation

Three researchers will conduct the data evaluation independently (JL, JZ and RL-TL), and another researcher (SW-CC) advice will be sought if there is any disagreement. The methodological quality of eligible studies will be evaluated using critical appraisal tools developed by the Joanna Briggs Institute. These tools were developed to assist in assessing the trustworthiness, relevance and results of quantitative and qualitative studies. For example, for qualitative studies, researchers are required to respond (yes, no, unclear or not applicable) to ten questions to determine whether a study has addressed the possibility of bias in its design, conduct or analysis.30 Following these questions, the researchers will decide if the study should be included or excluded, or if additional information should be sought.30 Disagreements regarding study inclusion will be discussed by the entire group of researchers.

Data extraction and synthesis

Two researchers will analyse the data independently (JL and SH). Included studies will be first categorised into quantitative, qualitative or mixed-methods studies according to the design. Information on the year of publication, author(s), setting, participant characteristics (eg, age, number of participants, cancer stage and treatments received), measures (eg, instruments, time of measurement and comparative groups), interventions (eg, blinding and randomised methods) and findings of interest (eg, levels of decision regret, what women regret and factors associated with decision regret) will be extracted. The extracted data will be compiled into an Excel spreadsheet by each researcher before being compared by both researchers for completeness and accuracy. Any discrepancies will be resolved through discussion within the research group (JL, SH, JZ, RL-TL and SW-CC).

Quantitative results will be pooled into Review Manager Software (RevMan) Version 5.4 to conduct a meta-analysis where appropriate.31 A forest plot will be created to present the pooled results. For example, if there are several interventional studies that have evaluated the effectiveness of decision aids in reducing patient-perceived regret, ORs (for dichotomous variables) or weighted mean differences (for continuous variables) and their 95% CIs will be calculated in order to precisely describe the impact of decision aids on decision regret. The I2 statistic will be used to assess heterogeneity, and a value lower than 50% will be considered to indicate low heterogeneity. In case of low heterogeneity, the fixed-effects model will be applied to assess the pooled results. Otherwise, the pooled results will be assessed using the random-effects model. Sensitivity analysis will be conducted if the pooled results have substantial heterogeneity, and the results will be carefully interpreted. A subgroup analysis on the interventions and types of received treatment (eg, chemotherapy or radiotherapy) will be performed where applicable. The publication bias will be indicated by the asymmetry of the funnel plot.32 The findings will be described in narrative form where meta-analysis is impossible. For example, if there is only one cross-sectional survey reporting regret after chemotherapy, results concerning the levels of regret, what women regret and related factors will be narratively described.

Qualitative evidence will be analysed using the thematic synthesis approach proposed by Thomas and Harden.33 Qualitative studies will be read and reread by two researchers (JL and SH), and findings associated with the three review questions will be identified and coded line-by-line. These initial codes will be compared and consolidated until a number of descriptive subthemes emerge. All researchers will discuss the subthemes until a consensus is reached regarding whether the subthemes comply with the meaning of the original study. Thereafter, similar subthemes will be further grouped based on their similarity in order to produce several analytical themes that are pertinent to the review questions. A coding sheet will be developed by the researchers to facilitate the data synthesis. The other three researchers (JZ, RL-TL and SW-CC) will comment on the synthesis process by reviewing the coding sheet until a final consensus is reached.

A third integrative analysis using a convergent parallel approach will be conducted to incorporate the quantitative and qualitative findings.34 Quantitative outcomes and qualitative themes will be combined to provide rich insights into the three review questions. It is anticipated the quantitative outcomes will quantify the qualitative findings, and the qualitative themes will help explain the quantitative outcomes; thus, the review findings will be convergent and complementary.35

Quality of evidence

The Grading of Recommendations, Assessment, Development and Evaluation (GRADE) guidelines will be followed to evaluate the certainty of reviewed evidence,36 and a GRADE evidence profile will be included when reporting the review findings.

Patient and public involvement

Patients and/or the public are not involved in the design, or conduct, or reporting or dissemination plans of this research.

Ethics and dissemination

Ethical approval is not required for this systematic review because no human participants will be involved. The findings of this study will be disseminated in international peer-reviewed journals and at nursing conferences. This review will also be disseminated as part of Jing Liu’s PhD thesis.

Supplementary Material

Reviewer comments
Author's manuscript

Acknowledgments

The authors would like to thank Editage (www.editage.com) for English language editing.

Footnotes

Contributors: All authors contributed to the design of this review. JL drafted the initial manuscript of this protocol. SH and SW-CC provided important comments to the manuscript. JL and SH will search and select the studies, and conduct the data analysis. JL, RL-TL and JZ will appraise the quality of included studies. SW-CC will validate the study selection, study evaluation and data synthesis process. All authors (JL, SH, SW-CC, RL-TL and JZ) will involve in resolving disagreement and drafting publications of the systematic review. All authors have approved the publication of this protocol.

Funding: This work was supported by China Scholarship Council, grant number 201808350089.

Disclaimer: The funder has no role in the study design, data collection, data analysis and manuscript preparation of this article.

Competing interests: None declared.

Provenance and peer review: Not commissioned; externally peer reviewed.

Supplemental material: This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Data availability statement

Not applicable for this protocol.

Ethics statements

Patient consent for publication

Not applicable.

References

  • 1.Sung H, Ferlay J, Siegel RL, et al. Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin 2021;71:209–49. 10.3322/caac.21660 [DOI] [PubMed] [Google Scholar]
  • 2.Walters S, Maringe C, Butler J, et al. Breast cancer survival and stage at diagnosis in Australia, Canada, Denmark, Norway, Sweden and the UK, 2000-2007: a population-based study. Br J Cancer 2013;108:1195–208. 10.1038/bjc.2013.6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Li J, Zhang B-N, Fan J-H, et al. A nation-wide multicenter 10-year (1999-2008) retrospective clinical epidemiological study of female breast cancer in China. BMC Cancer 2011;11:364. 10.1186/1471-2407-11-364 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Fisher B, Anderson S, Bryant J, et al. Twenty-Year follow-up of a randomized trial comparing total mastectomy, lumpectomy, and lumpectomy plus irradiation for the treatment of invasive breast cancer. N Engl J Med 2002;347:1233–41. 10.1056/NEJMoa022152 [DOI] [PubMed] [Google Scholar]
  • 5.Veronesi U, Cascinelli N, Mariani L, et al. Twenty-Year follow-up of a randomized study comparing breast-conserving surgery with radical mastectomy for early breast cancer. N Engl J Med 2002;347:1227–32. 10.1056/NEJMoa020989 [DOI] [PubMed] [Google Scholar]
  • 6.Elwyn G, Frosch D, Thomson R, et al. Shared decision making: a model for clinical practice. J Gen Intern Med 2012;27:1361–7. 10.1007/s11606-012-2077-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Lawhon VM, England RE, Wallace AS, et al. "It's important to me": A qualitative analysis on shared decision-making and patient preferences in older adults with early-stage breast cancer. Psychooncology 2021;30:167–75. 10.1002/pon.5545 [DOI] [PubMed] [Google Scholar]
  • 8.Ghaemi SZ, Keshavarz Z, Tahmasebi S, et al. Conflicts women with breast cancer face with: a qualitative study. J Family Med Prim Care 2019;8:27–36. 10.4103/jfmpc.jfmpc_272_18 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Vanni G, Materazzo M, Pellicciaro M, et al. Breast cancer and COVID-19: the effect of fear on patients' decision-making process. In Vivo 2020;34:1651–9. 10.21873/invivo.11957 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Leinert E, Kreienberg R, Wöckel A, et al. Survivors of primary breast cancer 5 years after surgery: follow-up care, long-term problems, and treatment regrets. Results of the prospective BRENDA II-study. Arch Gynecol Obstet 2020;301:761–7. 10.1007/s00404-020-05437-1 [DOI] [PubMed] [Google Scholar]
  • 11.Wang AW-T, Chang S-M, Chang C-S, et al. Regret about surgical decisions among early-stage breast cancer patients: effects of the congruence between patients' preferred and actual decision-making roles. Psychooncology 2018;27:508–14. 10.1002/pon.4522 [DOI] [PubMed] [Google Scholar]
  • 12.Brehaut JC, O'Connor AM, Wood TJ, et al. Validation of a decision regret scale. Med Decis Making 2003;23:281–92. 10.1177/0272989X03256005 [DOI] [PubMed] [Google Scholar]
  • 13.Connolly T, Reb J. Regret in cancer-related decisions. Health Psychol 2005;24:S29–34. 10.1037/0278-6133.24.4.S29 [DOI] [PubMed] [Google Scholar]
  • 14.Advani PG, Lei X, Swanick CW, et al. Local therapy decisional regret in older women with breast cancer: a population-based study. Int J Radiat Oncol Biol Phys 2019;104:383–91. 10.1016/j.ijrobp.2019.01.089 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Fernandes-Taylor S, Bloom JR. Post-Treatment regret among young breast cancer survivors. Psychooncology 2011;20:506–16. 10.1002/pon.1749 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Killoran M, Moyer A. Surgical treatment preferences in Chinese-American women with early-stage breast cancer. Psychooncology 2006;15:969–84. 10.1002/pon.1032 [DOI] [PubMed] [Google Scholar]
  • 17.Schulman-Green D, Cherlin E, Capasso R, et al. Patient and family caregiver considerations when selecting early breast cancer treatment: implications for clinical pathway development. Patient 2020;13:683–97. 10.1007/s40271-020-00426-7 [DOI] [PubMed] [Google Scholar]
  • 18.Karuturi MS, Lei X, Shen Y, et al. Long-Term decision regret surrounding systemic therapy in older breast cancer survivors: a population-based survey study. J Geriatr Oncol 2019;10:973–9. 10.1016/j.jgo.2019.03.013 [DOI] [PubMed] [Google Scholar]
  • 19.Tyner TE, Freysteinson WM. A concept analysis of decision regret in women with breast cancer. Nurs Forum 2022;57:112–20. 10.1111/nuf.12642 [DOI] [PubMed] [Google Scholar]
  • 20.Waljee JF, Hu ES, Newman LA, et al. Correlates of patient satisfaction and provider trust after breast-conserving surgery. Cancer 2008;112:1679–87. 10.1002/cncr.23351 [DOI] [PubMed] [Google Scholar]
  • 21.Taha SA, Matheson K, Paquet L, et al. Trust in physician in relation to blame, regret, and depressive symptoms among women with a breast cancer experience. J Psychosoc Oncol 2011;29:415–29. 10.1080/07347332.2011.582637 [DOI] [PubMed] [Google Scholar]
  • 22.Angarita FA, Elmi M, Zhang Y, et al. Patient-Reported factors influencing the treatment decision-making process of older women with non-metastatic breast cancer: a systematic review of qualitative evidence. Breast Cancer Res Treat 2018;171:545–64. 10.1007/s10549-018-4865-0 [DOI] [PubMed] [Google Scholar]
  • 23.Liu J, Hunter S, Zhu J, et al. Decision-Making experience related to mastectomy among women with breast cancer: an integrative review. Cancer Nurs 2021;44:E670–86. 10.1097/NCC.0000000000000981 [DOI] [PubMed] [Google Scholar]
  • 24.Flitcroft K, Brennan M, Spillane A. Decisional regret and choice of breast reconstruction following mastectomy for breast cancer: a systematic review. Psychooncology 2018;27:1110–20. 10.1002/pon.4585 [DOI] [PubMed] [Google Scholar]
  • 25.Braude L, Kirsten L, Gilchrist J, et al. A systematic review of women's satisfaction and regret following risk-reducing mastectomy. Patient Educ Couns 2017;100:2182–9. 10.1016/j.pec.2017.06.032 [DOI] [PubMed] [Google Scholar]
  • 26.Srethbhakdi A, Brennan ME, Hamid G, et al. Contralateral prophylactic mastectomy for unilateral breast cancer in women at average risk: systematic review of patient reported outcomes. Psychooncology 2020;29:960–73. 10.1002/pon.5379 [DOI] [PubMed] [Google Scholar]
  • 27.Moher D, Shamseer L, Clarke M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev 2015;4:1. 10.1186/2046-4053-4-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Martinez KA, Li Y, Resnicow K, et al. Decision regret following treatment for localized breast cancer: is regret stable over time? Med Decis Making 2015;35:446–57. 10.1177/0272989X14564432 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Yamauchi K, Nakao M, Nakashima M. Correlates of regret with treatment decision-making among Japanese women with breast cancer: results of an Internet-based cross-sectional survey. BMC Womens Health 2019;19:86. 10.1186/s12905-019-0783-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Lockwood C, Munn Z, Porritt K. Qualitative research synthesis: methodological guidance for systematic reviewers utilizing meta-aggregation. Int J Evid Based Healthc 2015;13:179–87. 10.1097/XEB.0000000000000062 [DOI] [PubMed] [Google Scholar]
  • 31.Higgins JPT, Thomas J, Chandler J, eds. Cochrane Handbook for Systematic Reviews of Interventions. Second Edition. Chichester, UK: John Wiley & Sons, 2019. [Google Scholar]
  • 32.Sterne JA, Egger M. Funnel plots for detecting bias in meta-analysis: guidelines on choice of axis. J Clin Epidemiol 2001;54:1046–55. 10.1016/s0895-4356(01)00377-8 [DOI] [PubMed] [Google Scholar]
  • 33.Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol 2008;8:45. 10.1186/1471-2288-8-45 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Creswell JW, Creswell JD. Research design: qualitative, quantitative and mixed methods approaches. Fourth Edition. Thousand Oaks, California: SAGE Publication, 2014. [Google Scholar]
  • 35.Östlund U, Kidd L, Wengström Y, et al. Combining qualitative and quantitative research within mixed method research designs: a methodological review. Int J Nurs Stud 2011;48:369–83. 10.1016/j.ijnurstu.2010.10.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Guyatt GH, Oxman AD, Kunz R, et al. Going from evidence to recommendations. BMJ 2008;336:1049–51. 10.1136/bmj.39493.646875.AE [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary data

bmjopen-2021-058425supp001.pdf (48.3KB, pdf)

Supplementary data

bmjopen-2021-058425supp002.pdf (17.2KB, pdf)

Reviewer comments
Author's manuscript

Data Availability Statement

Not applicable for this protocol.


Articles from BMJ Open are provided here courtesy of BMJ Publishing Group

RESOURCES