Skip to main content
BMJ Open logoLink to BMJ Open
. 2022 May 10;12(5):e061475. doi: 10.1136/bmjopen-2022-061475

How is recovery defined and measured in patients with low back pain? Protocol for a mixed study systematic review

Michael J Lukacs 1,, Katie L Kowalski 1, Nicole Peters 1, Meagan Stanley 2, Alison B Rushton 1
PMCID: PMC9092166  PMID: 35537787

Abstract

Introduction

This protocol describes the methods for a mixed study systematic review aiming to explore the definitions and measurements of recovery in patients with low back pain, and how perspectives of recovery differ between patients and providers. This review will be the first to review the concept of recovery in patients with low back pain across both quantitative and qualitative literature.

Methods and analysis

This protocol has been designed and reported in line with Preferred Reporting Items of Systematic Reviews and Meta-Analyses Protocols. The following databases will be electronically searched from database inception until 30 November 2021: Medline, EMBASE, CINAHL, Cochrane, PEDro. Grey literature will be searched for through targeted searching of ProQuest Dissertations and Theses and handsearching of the references of all included studies. Studies will be included if they include a patient population of >50% with low back pain (with or without leg pain), and mention the concept of recovery within the abstract, methods or results. The Mixed Methods Appraisal Tool will be used for quality assessment of both quantitative and qualitative included studies. Two independent reviewers will conduct the search, screen titles/abstracts and extract relevant data from full texts. Discrepancies between reviewers will be settled by a third reviewer with spinal pain expertise. For syntheses, thematic analysis will be used to analyse both qualitative and quantitative investigations to explore meanings, measurement and perspectives of recovery from a diverse evidence base. There is no clinical trial associated with this protocol.

Ethics and dissemination

There are no ethical issues associated with this systematic review, and ethics approval was not required. Once completed, the results of this review will be published in a peer-reviewed journal within the realm of spinal pain to help guide future research inquiries.

PROSPERO registration number

CRD42022295804.

Keywords: Low back pain, recovery, mixed study systematic review


Strengths and limitations of this study.

  • This is the first systematic review to establish definitions of recovery and explore how recovery has been measured across both quantitative and qualitative literature.

  • This review will also be the first to explore the concept of recovery across both surgical and conservative literature in patients with low back pain.

  • One limitation of this mixed study systematic review is that only articles available in English will be included for the synthesis.

Introduction

Despite being one of the most important questions within healthcare, the concept or definition of recovery from illness is not clear or consistent throughout the medical literature.1–5 ‘Recovery’ has been commonly used to mean a return to the preinjury/illness state, while used interchangeably with ‘improvement’ despite not likely representing the same construct.1 6 Furthermore, recovery has been described both as a process and a state or end-point.7 8 The process of recovery could be interpreted as a dynamic shift in health status, while the state could be interpreted as a position relative to a threshold of measurement, typically in the form of a cut-off score.9 10 In addition, it has been highlighted views regarding the meaning of recovery are likely very different between patients, researchers and clinicians.10 11 For example, depending on the cut-off used to define recovery, misclassification of patients can occur where the clinician rates the patient as recovered, but the patient does not support the same conclusion.7 12 13

To date, there remains no consistently agreed definition of recovery; an issue specifically highlighted for low back pain (LBP) and neck pain.6–8 This is especially problematic when considering that LBP remains the leading cause of disability globally.14 The concept of recovery may also be problematic to explore in the context of LBP in particular, as it is often challenging to identify the source of a patient’s LBP, and that patients with LBP experience persistent recurrence of their symptoms.15 As a result for those living with LBP, recovery may also include additional concepts such as readjustment and redefinition, especially when chronic pain is a component, as the resolution of persistent symptoms becomes less emphasised.7 16 17 Readjustment can be understood as methods employed to work around or avoid aggravation of a disorder, while redefinition may constitute new ideas of what life or, ‘being better,’ may be like.16 These concepts may be part of recovery, or could represent their own separate constructs. Thus, an understanding of the definitions of recovery across the literature is required to provide clarity in this area. Concurrently, non-invasive treatments for LBP management consistently show small to moderate treatment effects,18 and calls to re-revaluate outcome measurement selection in LBP management are growing.19 20 As a result, comparison of recovery outcomes across trials may prove futile if there is a lack of agreement on how recovery should be defined or measured.

The measurement of recovery has also been highlighted as difficult, owing to the number of indirect measures used in place of direct measures, in addition to disagreement over whether recovery is a continuous or dichotomous outcome.6 21 Scales that measure disability or symptom intensity have often been used as an indirect measure of recovery.6 7 However, these scales may be of relevance but not central importance,7 in addition to potentially not being informed by patients when they were first developed.22 Accordingly, if measurements of recovery within LBP are not in line with the perspectives of patients, then it becomes very difficult to conclude that the measurement of recovery is patient-centred. This is further complicated by clinical studies in which recovery is reported as an outcome of interest but data are not available, possibly reflecting reporting bias.23 Simultaneously, there has been a push for standardisation of both outcomes and outcome measures across clinical trials through initiatives like the Core Outcome Measures in Effectiveness Trials,24 as well as evaluation of their measurement properties.20

But before standardisation of a concept like recovery (and its measures) in LBP can be established, understanding regarding its definition and measurement is needed. Primary work in this area was completed in an initial systematic review by Kamper et al.6 This initial work concluded that almost every study assessed recovery in patients with LBP differently, and concluded that efforts to produce a consensus regarding its definition were sorely needed.6 However, Kamper et al (now 10 years old) included a narrow scope of studies and excluded populations such as spinal surgery. While conservative care is a primary treatment for LBP, invasive interventions such as surgery or injection may be needed, especially for persistent pain.25 Secondly, Kamper et al6 did not carry out a quality assessment of included studies, nor did they examine both quantitative and qualitative studies in their synthesis.6 Breadth of consideration of these perspectives would add considerable value to understanding the meaning of recovery for patients and providers, especially since the patient’s perspective is typically not incorporated in studies, and patient goals can be very different from information gained from practice outcome measures.11 21 26

Therefore, an updated broad review is required to synthesise the literature, especially with increased demand for outcome measures to include the perspective of patients.22 Understanding the concept of recovery from both patients and providers in LBP will aid in calibrating the landscape such that consensus can be reached regarding the meaning of recovery and its subsequent measurement. Thus, the aim of this review is to examine how recovery has been defined and measured for patients with LBP, within both the quantitative and qualitative literature.

Objectives

  1. To explore how the concept of recovery has been defined and measured for patients experiencing LBP.

  2. To examine how the perspectives of patients and providers regarding the recovery of LBP align or differ.

Methods and analysis

Design

This study is designed as a mixed study systematic review and this protocol is reported in accordance with Preferred Reporting Items of Systematic Reviews and Meta-Analyses (PRISMA) Protocols.27 28 This review is considered a mixed study systematic review as it will synthesise a meaning of recovery in the context of LBP across both quantitative and qualitative literature. Describing the definition and measurement of recovery using both quantitative and qualitative evidence will provide a more complete picture than either evidence base alone, and will also provide a diversity of views.29

Eligibility criteria

The eligibility criteria for studies were informed by the PICOS framework:

  • Population: Patients with musculoskeletal LBP with or without leg pain as the majority of participants (ie >50%).

  • Intervention: Either conservative (eg, physiotherapy) or non-conservative care (eg, surgical intervention).

  • Comparator: Not applicable.

  • Outcome: Definitions and/or measurements of recovery. Each study must report the concept of recovery in either the abstract, methods or results.

  • Study design: Quantitative and qualitative studies of any study design will be included. Studies will be excluded if they are written in a language other than English and a translation can not be found. Observational studies (eg, no intervention) are eligible inclusion as an extension of conservative care (ie, time). Additionally, studies that include LBP due to a non-musculoskeletal origin will be excluded (cancer, infection, metabolic disorders, etc). Lastly, studies for which quality assessment cannot be performed (ie, editorial/opinion pieces) will also be excluded (see the Quality Assessment section).

Information sources

The Medline, EMBASE, CINAHL, Cochrane and PEDro databases and the grey literature will be searched from inception until 30 November 2021. Handsearching of the references of included studies, as well as relevant qualitative journals, will take place to identify any articles that are inaccurately indexed provided they match the eligibility criteria for this systematic review. If studies cannot be accessed, the authors of included studies will be contacted. Lastly, we will consult with expert researchers in the field for identification of other possible potential studies.

Search strategy

The search strategy will be carried out by the lead author and will be checked by an independent librarian to ensure as broad and comprehensive of a search as possible (see online supplemental file 1). The search strategy for this review is informed by Kamper et al.6 This review, however, will emphasise a broader search with broader eligibility criteria to include both surgical patients as well as quantitative and qualitative literature. The search strategy has also been informed from an initial scoping of the literature and expert consensus. The search strategy has been piloted within MEDLINE (Ovid) (online supplemental file 1) and then adapted to the other included databases due to differences in specific search terms.

Supplementary data

bmjopen-2022-061475supp001.pdf (51.4KB, pdf)

Study records

Data management

All retrieved citations will be imported into Covidence (Covidence systematic review software, Veritas Health Innovation, Melbourne, Australia, www.covidence.org) at each stage of the review, and any duplicates will be removed.

Selection process

An onboarding session will take place to ensure agreement in article selection among reviewers. Two independent reviewers will evaluate the titles and abstracts of articles for inclusion in this study based on the above eligibility criteria. If articles do not match the eligibility criteria, they will be excluded. If there are articles that cannot be excluded, or there is doubt over their inclusion, they will be labelled as ‘maybe’. Next, full text copies of screened articles will be assessed against the eligibility criteria independently by two authors. At both points, if consensus is not reached, a third author with expertise in the domain of spinal pain will be consulted to resolve the disagreement. An unweighted kappa will be calculated to evaluate level of agreement between reviewers for both the titles and abstracts screening as well as the full-text assessment (IBM SPSS Statistics V.27).27 Inclusion of studies will follow the reporting process from PRISMA and be demonstrated visually.30

Data collection process

Data from included studies will be extracted by two reviewers using separate customised data extraction sheets for included quantitative and qualitative literature. Prior to data extraction, five articles will be piloted with reviewers to ensure consistency with extraction of relevant data.

Data items

Data extracted will include: study authors, publication year, country, type of study, research purpose, study participants, classification of LBP (eg, acute vs chronic), setting, description of how recovery was defined, measurement of recovery (eg, global rating of change), perspective (patient vs provider (ie, researcher or clinician)). Data items will be organised and presented in tabular form.

Outcomes and prioritisation

The main outcome for this study is recovery in terms of how it has been defined and measured. Operational definitions of recovery will be sought (eg, an improvement in a score or attainment of a specific cut-off point), as well as the various ways in which recovery has been measured (eg, through use of outcome-measures or self-report). Finally, this review will also explore if perspectives of recovery are different across patients and providers through comparison of themes based on study participants.

Quality assessment

Compared with the assessment of randomised control trials alone, there is much less guidance for quality assessment of mixed study systematic reviews.31 Quality assessment for this systematic review will use the Mixed Method Appraisal Tool Version 2018 (MMAT), which uses a single tool for assessment of mixed study research.32 The MMAT 2018 tool is an updated quality assessment tool first created in 2006 for the quality assessment of mixed study systematic reviews, including quantitative, qualitative and mixed-methods studies.32 The 2018 version has been content validated by expert opinion, and previous versions have demonstrated fair to perfect reliability.33–35 The MMAT tool was selected for use over other quality assessment tools due to its alignment with a constructivist viewpoint (which aligns with the positionality of the authors regarding the nature of recovery).36 Furthermore, the MMAT focuses specifically on reviews including quantitative, qualitative and mixed-methods studies.33 36 The MMAT tool will be piloted before use to ensure agreement in scoring. Two reviewers will independently score each included article, and if there is disagreement a third reviewer will be consulted. As an overall score of the MMAT is discouraged,32 the ratings of each criterion of the MMAT will be displayed to better inform the quality of included studies. Quality of included studies will be reported descriptively to help guide the reporting of the data synthesis of themes.

Data synthesis

There is a lack of guidance for reporting of mixed-methods systematic reviews, especially for those not focused on interventions or effectiveness.37 This review will follow a convergent qualitative synthesis,38 39 whereby both quantitative and qualitative literature will be synthesised together to answer the central research questions. A convergent qualitative synthesis is especially pertinent when the research questions in mind are concerned with questions of what, how or why.39 As the purpose of this review is to clarify definitions of recovery, and is not specific to interventions, thematic analysis40 41 will be used to synthesise the included qualitative and quantitative studies. The ‘qualitising’ of quantitative literature will enable synthesis and comparisons from included qualitative literature to allow for a more complete picture than either evidence base alone. Thematic analysis of the definitions, measurement, and perspectives of recovery will take place in three steps: line-by-line coding, free-coding and development of analytical themes.40 Common features of definitions of recovery will be grouped where appropriate (eg, types of interventions received, chronicity of LBP). Themes developed will also be interpreted in light of the quality of the included studies. From qualitative studies, thematic synthesis will be performed based on the characterisations of the themes of the definition of recovery to highlight areas of improvement and/or gaps.

Patients and public involvement

No patients were directly involved in the design of this review. The results of this mixed study systematic review will be disseminated to the Spinal Pain Patient Partner Advisory Group at Western University. The intent of this engagement is to ascertain if the measurements/definitions of recovery align with patient perspectives to help inform future research endeavours.

Discussion

This mixed study systematic review will enable definition and measurement of recovery. It will also help to inform the development of outcome measures specific to the measure of recovery if required. Findings will determine if alternative terms other than recovery should be used within the context of patients with LBP. The results of this review may help to develop shared meanings of recovery that may be applicable across other populations of patients. Ultimately, the results of this review will be integral to informing a core outcome set specifically for the concept of recovery in patients with LBP.

By exploring a diverse evidence base, a shared meaning of recovery can be explored or generated for patients with LBP. As such, this review will also be the first to inform a definition of recovery from the perspectives of both patients and providers, and in doing so will lay the groundwork for future research. This review will also highlight any potential differences or commonalities in the perceptions of recovery between patients and providers.

There are no ethical issues to report associated with this mixed study systematic review, and any potential amendments to the protocol will be reported in the final publication as well as on PROSPERO with pertinent details. This work is critical for setting the stage for, and informing, planned observational studies for determining measures that are meaningful and important to patients with LBP.

Ethics and dissemination

There are no ethical issues associated with this systematic review, and ethics approval was not required. Once completed, the knowledge generated from this mixed study systematic review will be published in a peer-reviewed journal within the realm of spinal pain to help guide future research inquiries. Finally, the results of this review will be shared with the Spinal Pain Patient Partner Advisor Group at Western University to help inform future research endeavours.

Supplementary Material

Reviewer comments
Author's manuscript

Footnotes

Twitter: @abrushton

Contributors: All authors (MJL, NP, MS and ABR) contributed to the design of the above systematic review and all authors have approved the final protocol. MJL, NP and MS will be responsible for screening articles and extraction of data, while all authors (MJL, NP, MS, and ABR) will contribute to data analysis. MJL is the guarantor of the review.

Funding: The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

Competing interests: None declared.

Patient and public involvement: Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.

Provenance and peer review: Not commissioned; externally peer reviewed.

Supplemental material: This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Ethics statements

Patient consent for publication

Not applicable.

References

  • 1.Adamowicz JL, Caikauskaite I, Friedberg F. Defining recovery in chronic fatigue syndrome: a critical review. Qual Life Res 2014;23:2407–16. 10.1007/s11136-014-0705-9 [DOI] [PubMed] [Google Scholar]
  • 2.Roberts G, Wolfson P. The rediscovery of recovery: open to all. Advances in Psychiatric Treatment 2004;10:37–48. 10.1192/apt.10.1.37 [DOI] [Google Scholar]
  • 3.Borrell-Vega J, Humeidan ML, Bergese SD. Defining quality of recovery – what is important to patients? Best Pract Res Clin Anaesthesiol 2018;32:259–68. 10.1016/j.bpa.2018.07.002 [DOI] [PubMed] [Google Scholar]
  • 4.Wiles R, Cott C, Gibson BE. Hope, expectations and recovery from illness: a narrative synthesis of qualitative research. J Adv Nurs 2008;64:564–73. 10.1111/j.1365-2648.2008.04815.x [DOI] [PubMed] [Google Scholar]
  • 5.Ellison ML, Belanger LK, Niles BL, et al. Explication and definition of mental health recovery: a systematic review. Adm Policy Ment Health 2018;45:91–102. 10.1007/s10488-016-0767-9 [DOI] [PubMed] [Google Scholar]
  • 6.Kamper SJ, Stanton TR, Williams CM, et al. How is recovery from low back pain measured? A systematic review of the literature. Eur Spine J 2011;20:9–18. 10.1007/s00586-010-1477-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Walton DM, Macdermid JC, Nielson W. Recovery from acute injury: clinical, methodological and philosophical considerations. Disabil Rehabil 2010;32:864–74. 10.3109/09638280903349511 [DOI] [PubMed] [Google Scholar]
  • 8.Walton D. A review of the definitions of 'recovery' used in prognostic studies on whiplash using an ICF framework. Disabil Rehabil 2009;31:943–57. 10.1080/09638280802404128 [DOI] [PubMed] [Google Scholar]
  • 9.Walton DM, Marsh J. The multidimensional symptom index: a new patient-reported outcome for pain phenotyping, prognosis and treatment decisions. Eur J Pain 2018;22:1351–61. 10.1002/ejp.1224 [DOI] [PubMed] [Google Scholar]
  • 10.Modarresi S, Walton DM, Reliability WDM. Reliability, discriminative accuracy, and an exploration of response shift as measured using the satisfaction and recovery index over 12 months from musculoskeletal trauma. Musculoskelet Sci Pract 2021;51:102300. 10.1016/j.msksp.2020.102300 [DOI] [PubMed] [Google Scholar]
  • 11.Hush JM, Refshauge K, Sullivan G, et al. Recovery: what does this mean to patients with low back pain? Arthritis Rheum 2009;61:124–31. 10.1002/art.24162 [DOI] [PubMed] [Google Scholar]
  • 12.Mehling WE, Gopisetty V, Acree M, et al. Acute low back pain and primary care: how to define recovery and chronification? Spine 2011;36:2316–23. 10.1097/BRS.0b013e31820c01a6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Cook CE, Learman KE, O'Halloran BJ, et al. Which prognostic factors for low back pain are generic predictors of outcome across a range of recovery domains? Phys Ther 2013;93:32–40. 10.2522/ptj.20120216 [DOI] [PubMed] [Google Scholar]
  • 14.James SL, Abate D, Abate KH, et al. Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990-2017: a systematic analysis for the global burden of disease study 2017. Lancet 2018;392:1789–858. 10.1016/S0140-6736(18)32279-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Buchbinder R, Underwood M, Hartvigsen J, et al. The Lancet series call to action to reduce low value care for low back pain: an update. Pain 2020;161 Suppl 1:S57–64. 10.1097/j.pain.0000000000001869 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Beaton DE, Tarasuk V, Katz JN, et al. "Are you better?" A qualitative study of the meaning of recovery. Arthritis Rheum 2001;45:270–9. [DOI] [PubMed] [Google Scholar]
  • 17.Myburgh C, Boyle E, Lauridsen HH, et al. What influences retrospective self-appraised recovery status among Danes with low-back problems? A comparative qualitative investigation. J Rehabil Med 2015;47:741–7. 10.2340/16501977-1987 [DOI] [PubMed] [Google Scholar]
  • 18.Chou R, Deyo R, Friedly J. Noninvasive treatments for low back pain. In: Effective Health Care Program - Comparative Effectiveness Review. 272, 2016. [Google Scholar]
  • 19.Froud R, Ellard D, Patel S, et al. Primary outcome measure use in back pain trials may need radical reassessment. BMC Musculoskelet Disord 2015;16:1–11. 10.1186/s12891-015-0534-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Chiarotto A, Ostelo RW, Boers M, et al. A systematic review highlights the need to investigate the content validity of patient-reported outcome measures for physical functioning in patients with low back pain. J Clin Epidemiol 2018;95:73–93. 10.1016/j.jclinepi.2017.11.005 [DOI] [PubMed] [Google Scholar]
  • 21.Hush JM, Kamper SJ, Stanton TR, et al. Standardized measurement of recovery from nonspecific back pain. Arch Phys Med Rehabil 2012;93:849–55. 10.1016/j.apmr.2011.11.035 [DOI] [PubMed] [Google Scholar]
  • 22.Cook CE, Wright A, Wittstein J, et al. Five recommendations to address the limitations of patient-reported outcome measures. J Orthop Sports Phys Ther 2021;51:562–5. 10.2519/jospt.2021.10836 [DOI] [PubMed] [Google Scholar]
  • 23.Machado GC, Ferreira PH, Yoo RIJ, et al. Surgical options for lumbar spinal stenosis. Cochrane Database Syst Rev 2016;2016:11. 10.1002/14651858.CD012421 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Prinsen CAC, Vohra S, Rose MR, et al. Core Outcome Measures in Effectiveness Trials (COMET) initiative: protocol for an international Delphi study to achieve consensus on how to select outcome measurement instruments for outcomes included in a 'core outcome set'. Trials 2014;15:247. 10.1186/1745-6215-15-247 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Jacobs WCH, Rubinstein SM, Willems PC, et al. The evidence on surgical interventions for low back disorders, an overview of systematic reviews. Eur Spine J 2013;22:1936–49. 10.1007/s00586-013-2823-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Gardner T, Refshauge K, McAuley J, et al. Patient led goal setting in chronic low back pain-What goals are important to the patient and are they aligned to what we measure? Patient Educ Couns 2015;98:1035–8. 10.1016/j.pec.2015.04.012 [DOI] [PubMed] [Google Scholar]
  • 27.Shamseer L, Moher D, Clarke M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ 2015;349:g7647. 10.1136/bmj.g7647 [DOI] [PubMed] [Google Scholar]
  • 28.Moher D, Shamseer L, Clarke M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev 2015;4:1–9. 10.1186/2046-4053-4-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Hong QN, Rees R, Sutcliffe K, et al. Variations of mixed methods reviews approaches: a case study. Res Synth Methods 2020;11:795–811. 10.1002/jrsm.1437 [DOI] [PubMed] [Google Scholar]
  • 30.Moher D, Liberati A, Tetzlaff J, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Int J Surg 2010;8:336–41. 10.1016/j.ijsu.2010.02.007 [DOI] [PubMed] [Google Scholar]
  • 31.Harrison R, Jones B, Gardener P. Quality assessment with diverse studies (QuADS): an appraisal tool for methodological and reporting quality in systematic reviews of mixed- or multi-method studies. BMC Health Serv Res 2021;21. 10.1186/s12913-021-06122-y [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Hong QN, Fàbregues S, Bartlett G, et al. The mixed methods appraisal tool (MMAT) version 2018 for information professionals and researchers. Education for Information 2018;34:285–91. 10.3233/EFI-180221 [DOI] [Google Scholar]
  • 33.Hong QN, Gonzalez-Reyes A, Pluye P. Improving the usefulness of a tool for appraising the quality of qualitative, quantitative and mixed methods studies, the mixed methods appraisal tool (MMAT). J Eval Clin Pract 2018;24:459–67. 10.1111/jep.12884 [DOI] [PubMed] [Google Scholar]
  • 34.Hong QN, Pluye P, Fàbregues S, et al. Improving the content validity of the mixed methods appraisal tool: a modified e-Delphi study. J Clin Epidemiol 2019;111:49–59. 10.1016/j.jclinepi.2019.03.008 [DOI] [PubMed] [Google Scholar]
  • 35.Souto RQ, Khanassov V, Hong QN, et al. Systematic mixed studies reviews: updating results on the reliability and efficiency of the mixed methods appraisal tool. Int J Nurs Stud 2015;52:500–1. 10.1016/j.ijnurstu.2014.08.010 [DOI] [PubMed] [Google Scholar]
  • 36.Pluye P, Gagnon M-P, Griffiths F, et al. A scoring system for appraising mixed methods research, and concomitantly appraising qualitative, quantitative and mixed methods primary studies in mixed studies reviews. Int J Nurs Stud 2009;46:529–46. 10.1016/j.ijnurstu.2009.01.009 [DOI] [PubMed] [Google Scholar]
  • 37.Stern C, Lizarondo L, Carrier J, et al. Methodological guidance for the conduct of mixed methods systematic reviews. JBI Evid Synth 2020;18:2108–18. 10.11124/JBISRIR-D-19-00169 [DOI] [PubMed] [Google Scholar]
  • 38.Pluye P, Hong QN. Combining the power of stories and the power of numbers: mixed methods research and mixed studies reviews. Annu Rev Public Health 2014;35:29–45. 10.1146/annurev-publhealth-032013-182440 [DOI] [PubMed] [Google Scholar]
  • 39.Hong QN, Pluye P, Bujold M, et al. Convergent and sequential synthesis designs: implications for conducting and reporting systematic reviews of qualitative and quantitative evidence. Syst Rev 2017;6:61. 10.1186/s13643-017-0454-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol 2008;8:45. 10.1186/1471-2288-8-45 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006;3:77–101. 10.1191/1478088706qp063oa [DOI] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary data

bmjopen-2022-061475supp001.pdf (51.4KB, pdf)

Reviewer comments
Author's manuscript

Articles from BMJ Open are provided here courtesy of BMJ Publishing Group

RESOURCES