Skip to main content
NCBI home page
British Paramedic Journal logoLink to British Paramedic Journal
. 2023 Sep 1;8(2):38–43. doi: 10.29045/14784726.2023.9.8.2.38

A pre-hospital mixed methods systematic review protocol

Tegwyn McManamny 1,✉, Marishona Ortega 2, Scott Munro 3, Paul Jennings 4, Gregory Adam Whitley 5
PMCID: PMC10477821  PMID: 37674915

Abstract

Introduction:

Mixed methods research, a methodology entailing the integration of qualitative and quantitative data within a single study, offers researchers the ability to investigate complex processes and systems in health and healthcare. The collective strength gained through the data combination can provide an enhanced understanding of research problems, providing an ideal solution to understanding complex clinical issues in a range of settings. In pre-hospital practice, where often uncontrollable variables and environmental considerations increase healthcare complexity, mixed methods has emerged as a valuable approach to research.

Aims:

Given the exponential growth of pre-hospital mixed methods research since the publication of our first systematic review in 2014, we aim to provide an update. Our review will explore how mixed methods is utilised in pre-hospital research and identify what standards of reporting are achieved.

Methods:

This systematic review update will search MEDLINE, CINAHL Complete, Embase and Scopus bibliographic databases from 1 January 2012 to 15 March 2023, using an updated pre-hospital search strategy. Study screening will be performed in duplicate. Articles reported in English, explicitly stating the use of ‘mixed methods’ in the pre-hospital ambulance setting, including helicopter emergency medical services and community first-responder services, will be included. Data related to underpinning philosophy or theoretical framework, rationale for utilising mixed methods, background of the corresponding author, mode of data integration, model of publication and adherence to reporting standards, utilising the good reporting of a mixed methods study (GRAMMS) guidelines, will be extracted and analysed. All extracted data from study articles will be summarised in a table, allowing analysis of included studies against specified criteria.

Keywords: methodology, mixed methods, paramedicine

Introduction

Pre-hospital care has evolved from a model of urgent transportation to an integrated component of the healthcare system. As systems have developed, so too has the quantity and quality of pre-hospital research, with individuals and organisations contributing to a growing evidence base for pre-hospital care and paramedic practice (Olaussen et al., 2022; Siriwardena & Whitley, 2022). An increasing number of clinical trials within the pre-hospital setting have prompted changes to existing guidelines and procedures (Benger et al., 2018; Snooks et al., 2017; Stub et al., 2015), and collaboration between pre-hospital practitioners, academics and organisations has been enhanced through large-scale multi-organisational research programmes (Turner & Lane, 2006). Research project design has similarly developed, with increased publication of research protocols, and the utilisation of a range of methodologies to answer research questions spanning a broad range of topics. The predominant use of quantitative research methods in pre-hospital studies has to some extent eased with the increased utilisation of qualitative and mixed methods, enhancing the depth and breadth of research investigating challenging topics such as paediatric pain management (Murphy et al., 2014), language barriers (Tate et al., 2016), adult sepsis (Wallgren et al., 2017) and pre-hospital palliative care management (Carter et al., 2019).

Since our initial systematic review was published (McManamny et al., 2015), the use of mixed methods in pre-hospital research has continued to grow. In addition, academic discourse on the use of mixed methods in pre-hospital research has also increased in quantity, growing the paradigm and contributing to an ever-increasing body of specialised knowledge (Whitley et al., 2020).

Given the rapid evolution of pre-hospital mixed methods research, it appears timely to update our understanding of the utilisation of mixed methods in pre-hospital research, by systematically reviewing and critiquing the available literature. In particular, we seek to assess the presence of underpinning philosophies or theoretical frameworks applied to the utilisation of mixed methods in pre-hospital research. Some researchers assert that the rationale for the utilisation of mixed methods in pre-hospital research indicates consideration of the way the chosen methodology influences the study – an essential component of best-practice research (Creswell et al., 2004). However, the challenges of conducting research in the pre-hospital environment, and the relative recency of the utilisation of mixed methods in pre-hospital research, may lead to mixed methodologies being employed for pragmatic reasons, in isolation to underpinning epistemology (McManamny, 2022).

Aim and objectives

The aim of this study is to update our previous systematic review (McManamny et al., 2015) and critique the utilisation of mixed methods in pre-hospital research. The study objectives are to describe the characteristics and adherence to reporting guidelines of mixed methods studies utilised in pre-hospital research.

In addition, this study will comment on the role of pragmatism in pre-hospital mixed methods research, and the influence that the pre-hospital setting has on research utilising mixed methodologies.

Methods/design

Study method

This review protocol was prepared using the preferred reporting items for systematic review and meta-analysis protocol (PRISMA-P) guidelines (Moher et al., 2015), along with the guidance provided by Garner et al. (2016). We completed the PRISMA-P checklist (Supplementary 1).

A systematic review of methodology will be conducted, building on the findings from our previous systematic review (McManamny et al., 2015). This review aimed to understand the utilisation and application of mixed methods within pre-hospital research, and to conceptualise the difficulties associated with mixed methods research in pre-hospital studies.

The review was not registered on the international prospective register of systematic reviews (PROSPERO), as it does not involve an outcome of clear relevance to the health of humans, and is instead a methodological review that assesses the quality of reporting.

Search strategy

The following databases will be searched from October 2012 (the last period searched in our previous systematic review) to 15 March 2023:

  • MEDLINE via Ovid,

  • CINAHL Complete via EBSCOhost,

  • Embase via Ovid and

  • Scopus.

Burgess et al.’s (2010) pre-hospital search filter was used in our previous systematic review. Since then, a more contemporaneous pre-hospital search filter has been published, which is optimised for pre-hospital researchers by maximising sensitivity (Olaussen et al., 2017). We have therefore chosen to revise our search strategy in line with Olaussen et al.’s (2017) search filter, while also taking the opportunity to add two new 2023 medical subject heading terms (‘Paramedics’ and ‘Paramedicine’) to maximise recency. A validated search filter for mixed methods studies was not available. El Sherif et al. (2016) produced a search filter for systematic mixed studies reviews; however, we felt this was not appropriate for our research question due to the high number of non-relevant records retrieved by this filter. We therefore chose to maintain the original ‘mixed methods’ search strategy used in our 2014 systematic review (McManamny et al., 2015) (see Table 1).

Table 1.

Search terms.

Line MEDLINE
Subject heading / text term		 Embase
Subject heading / text term		 CINAHL Complete
Subject heading / text term		 Scopus
Text term
1 Paramedics/ Paramedical Personnel/ – –
2 Paramedicine/ – – –
3 Ambulances/ Ambulance/ (MH “Ambulances”) –
4 Emergency Medical Technicians/ Rescue personnel/ (MH “Emergency Medical Technicians”) –
5 Air ambulances/ Air medical transport/ – –
6 Emergency Medical Services/ Emergency health service/ (MH “Emergency Medical Services”) TITLE-ABS (Emergency AND medical AND services)
7 Paramedic*.tw Paramedic*.tw TI Paramedic* OR AB Paramedic* TITLE-ABS (Paramedic*)
8 EMS.tw EMS.tw TI EMS OR AB EMS TITLE-ABS (EMS)
9 EMT.tw EMT.tw TI EMT OR AB EMT TITLE-ABS (EMT)
10 Prehospital.tw Prehospital.tw TI Prehospital OR AB Prehospital TITLE-ABS (Prehospital)
11 Pre-hospital.tw Pre-hospital.tw TI Pre-hospital OR AB Pre-hospital TITLE-ABS (Pre-hospital)
12 First responder*.tw First responder*.tw TI First responder* OR AB First responder* TITLE-ABS (First AND responder*)
13 Emergency medical technicians.tw Emergency medical technicians.tw TI Emergency medical technicians OR AB Emergency medical technicians TITLE-ABS (Emergency AND medical AND technicians)
14 Emergency services.tw Emergency services.tw TI Emergency services OR AB Emergency services TITLE-ABS (Emergency AND services)
15 Ambulance*.tw Ambulance*.tw TI Ambulance* OR AB Ambulance* TITLE-ABS (Ambulance*)
16 HEMS.tw HEMS.tw TI HEMS OR AB HEMS TITLE-ABS (HEMS)
17 Field triage.tw Field triage.tw TI Field triage OR AB Field triage TITLE-ABS (Field AND triage)
18 Out-of-hospital.tw Out-of-hospital.tw TI Out-of-hospital OR AB Out-of-hospital TITLE-ABS (Out-of-hospital)
19 1 OR 2 OR 3 OR 4 OR 5 OR 6 OR 7 OR 8 OR 9 OR 10 OR 11 OR 12 OR 13 OR 14 OR 15 OR 16 OR 17 OR 18 1 OR 3 OR 4 OR 5 OR 6 OR 7 OR 8 OR 9 OR 10 OR 11 OR 12 OR 13 OR 14 OR 15 OR 16 OR 17 OR 18 3 OR 4 OR 6 OR 7 OR 8 OR 9 OR 10 OR 11 OR 12 OR 13 OR 14 OR 15 OR 16 OR 17 OR 18 6 OR 7 OR 8 OR 9 OR 10 OR 11 OR 12 OR 13 OR 14 OR 15 OR 16 OR 17 OR 18
20 Mixed method*.tw Mixed method*.tw TI Mixed method* OR AB Mixed method* TITLE-ABS (Mixed AND method*)
21 (Quantitative OR survey*).tw (Quantitative OR survey*).tw TI (Quantitative OR survey*) OR AB (Quantitative OR survey*) TITLE-ABS (Quantitative OR survey*)
22 Qualitative.tw Qualitative.tw TI Qualitative OR AB Qualitative TITLE-ABS (Qualitative)
23 21 AND 22 21 AND 22 21 AND 22 21 AND 22
24 20 OR 23 20 OR 23 20 OR 23 20 OR 23
25 19 AND 24 19 AND 24 19 AND 24 19 AND 24
26 Limit 25 to (English language and yr=“2012 – Current”) Limit 25 to (English language and yr=“2012 – Current”) Limit 25 to (English language and yr=“2012 – Current”) Limit 25 to (English language and yr=“2012 – Current”)
Open in a new tab

The search strategy was developed with the assistance of a specialist senior academic subject librarian (MO). The search strategy was tested and revised over a period of several months to ensure an optimal level of sensitivity and specificity.

In addition, key journals will be hand searched to identify relevant studies, including paramedic journals not indexed in large databases. Google Scholar will be searched using key search terms and the top 200 results will be screened.

Reference lists of included studies will be screened and forward citation tracking will be performed to identify additional mixed methods studies conducted in the pre-hospital setting. Any protocol amendments will be tracked and dated, and documented in a ‘differences between protocol and review’ section of the completed systematic review.

Ethical approval for this review is not required because the work is carried out utilising published documents.

Inclusion criteria

  1. Reported in English.

  2. Published from October 2012 to 15 March 2023.

  3. Conducted in the pre-hospital ambulance setting, including helicopter emergency medical services and community first responders.

  4. Explicitly reporting the use of ‘mixed methods’.

  5. Integration of quantitative and qualitative data within the published study.

Exclusion criteria

  1. Community or primary care setting.

  2. Tertiary or academic setting.

  3. Studies within police or fire service organisations.

  4. Studies with a military, battlefield or combat focus.

  5. Multi-method studies with no explicit claim of using mixed methods.

  6. Protocols, pre-prints, conference abstracts, non-empirical research, reviews.

Study screening

Studies identified from the search strategy will be exported into Covidence Systematic Review Software (Veritas Health Innovation, Melbourne, Australia), where duplicate articles will be automatically removed. Titles and abstracts will be screened in duplicate.

Articles selected for full-text review will be retrieved and uploaded to Covidence. Full-text screening will be performed in duplicate according to the inclusion and exclusion criteria. Those selected for inclusion will undergo data extraction. A PRISMA flow diagram (Page et al., 2021) will be created to illustrate the study selection process, with reasons and references for full-text exclusions provided.

Data extraction

A bespoke data extraction template will be created and piloted. One author will conduct data extraction, verified by a second author. Data extraction will include items identified in the good reporting of a mixed methods study (GRAMMS) guidelines, as well as additional items required to answer the research question, including:

  • background of the corresponding author (clinical background, sex, country);

  • declaration of an underpinning philosophy or theoretical framework;

  • model of publication (single/separate); and

  • adherence to reporting guidelines (O’Cathain et al., 2008):
    • justification for utilisation of mixed methods approach;
    • design characteristics including purpose, priority and sequence of methods;
    • methodology relating to sampling, data collection and analysis;
    • data integration (where, how, who);
    • limitations; and
    • insights gained from method integration.

Disagreements between authors during data extraction will be resolved through discussion, or involvement of a third author.

Risk of bias assessment

The mixed methods appraisal tool will be used to assess the risk of bias for included studies (Hong et al., 2018). Risk of bias will be assessed in duplicate, with disagreements resolved through discussion or the involvement of a third reviewer. Results will be displayed in tabular format, with green, amber and red colour coding for ‘yes’, ‘can’t tell’ and ‘no’ responses, respectively. Studies will not be excluded from the review, irrespective of their risk of bias.

Data analysis

Included studies will undergo an assessment of their adherence to the GRAMMS guidelines (O’Cathain et al., 2008). A data analysis form using a ‘traffic light’ system has been created by the research team which allows studies to be coded according to the GRAMMS criteria, with the colour-coded options of ‘yes’ (green), ‘somewhat’ (yellow) and ‘no’ (red), with the additional option of ‘inadequate information’ (white). In addition, there is the opportunity for research team members to record open comments. Authors conducting the GRAMMS adherence assessment will undergo a benchmarking exercise where two papers will be assessed by all authors, followed by a discussion of the results.

Data related to the background of the corresponding author, declaration of an underpinning philosophy or theoretical framework and the model of publication will be reported narratively and in tables to highlight any disparity or areas of interest within the pre-hospital mixed methods literature.

Confidence in the cumulative evidence

Due to the focus of this review on methods rather than interventions or patient outcomes, assessment of the confidence in cumulative evidence using approaches such as grading of recommendations assessment, development and evaluation (GRADE) (Schünemann et al., 2022), and of confidence in the evidence from reviews of qualitative research (GRADE-CERQual) (Lewin et al., 2018), is not appropriate.

Discussion

The overarching aim of this study will be to undertake a contemporary systematic review around the utilisation of mixed methods in pre-hospital research. Determining how mixed methodologies are utilised in pre-hospital research will enable an enhanced understanding of the level of critical analysis that goes into the application of this research methodology. In addition, the evaluation of key components (including adherence to reporting standards) will allow assessment of the quality of mixed methods studies in pre-hospital research, and identify potential opportunities for improvement for pre-hospital researchers.

The findings of this research will have international relevance, as it will contribute to the understanding of the utilisation of mixed methods in pre-hospital research, a rapidly growing field. Findings from our work will be disseminated through manuscripts in peer-reviewed journals.

Author contributions

TM conceived the study. TM, GAW, MO, SM and PJ contributed to methodology development. MO tested and revised the search strategy. TM, GAW and PJ wrote the manuscript. TM, GAW, MO, SM and PJ contributed to review and editing, and project administration. TM prepared the manuscript for publication. TM acts as the guarantor for this article.

Conflict of interest

GAW is an associate editor of the BPJ.

Ethics

This research is exempt from ethics approval because the work is carried out utilising published documents. This protocol and the subsequent systematic review will be disseminated in peer-reviewed journals.

Funding

None.

Contributor Information

Tegwyn McManamny, Ambulance Victoria; Monash University ORCID iD: https://orcid.org/0000-0001-6512-0191.

Marishona Ortega, University of Lincoln ORCID iD: https://orcid.org/0000-0003-2647-264X.

Scott Munro, South East Coast Ambulance Service NHS Foundation Trust; University of Surrey ORCID iD: https://orcid.org/0000-0002-0228-4102.

Paul Jennings, Ambulance Victoria; Monash University ORCID iD: https://orcid.org/0000-0002-5605-7589.

Gregory Adam Whitley, East Midlands Ambulance Service NHS Trust; University of Lincoln ORCID iD: https://orcid.org/0000-0003-2586-6815.

References

  1. Benger J. R., Kirby K., Black S., Brett S. J., Clout M., Lazaroo M. J., Nolan J. P., Reeves B. C., Robinson M., Scott L. J., Smartt H., South A., Stokes E. A., Taylor J., Thomas M., Voss S., Wordsworth S., & Rogers C. A. (2018). Effect of a strategy of a supraglottic airway device vs tracheal intubation during out-of-hospital cardiac arrest on functional outcome: The AIRWAYS-2 randomized clinical trial. JAMA, 320(8), 779–791. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Burgess S., Smith E., Piper S., & Archer F. (2010). The development of an updated prehospital search filter for the Cochrane Library: Prehospital search filter version 2.0. Journal of Emergency Primary Health Care, 8, (4), 990444. [Google Scholar]
  3. Carter A. J. E., Arab M., Harrison M., Goldstein J., Stewart B., Lecours M., Sullivan J., Villard C., Crowell W., Houde K., Jensen J. L., Downer K., & Pereira J. (2019). Paramedics providing palliative care at home: A mixed-methods exploration of patient and family satisfaction and paramedic comfort and confidence. Canadian Journal of Emergency Medicine, 21(4), 513–522. [DOI] [PubMed] [Google Scholar]
  4. Creswell J. W., Fetters M. D., & Ivankova N. V. (2004). Designing a mixed methods study in primary care. Annals of Family Medicine, 2(1), 7–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. El Sherif R., Pluye P., Gore G., Granikov V., & Hong Q. N. (2016). Performance of a mixed filter to identify relevant studies for mixed studies reviews. Journal of the Medical Library Association, 104(1), 47–51. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Garner P., Hopewell S., Chandler J., MacLehose H., Akl E. A., Beyene J., Chang S., Churchill R., Dearness K., Guyatt G., Lefebvre C., Liles B., Marshall R., Martínez García L., Mavergames C., Nasser M., Qaseem A., Sampson M., Soares-Weiser K. . . . Schünemann H. J. (2016). When and how to update systematic reviews: Consensus and checklist. BMJ, 354, i3507. https://doi.org/10.1136/bmj.i3507. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Hong Q. N., Fàbregues S., Bartlett G., Boardman F., Cargo M., Dagenais P., Gagnon M.-P., Griffiths F., Nicolau B., O’Cathain A., Rousseau M.-C., Vedel I., & Pluye P. (2018). The mixed methods appraisal tool (MMAT) version 2018 for information professionals and researchers. Education for Information, 34, 285–291. [Google Scholar]
  8. Lewin S., Booth A., Glenton C., Munthe-Kaas H., Rashidian A., Wainwright M., Bohren M. A., Tunçalp O., Colvin C. J., Garside R., Carlsen B., Langlois E. V., & Noyes J. (2018). Applying GRADE-CERQual to qualitative evidence synthesis findings: Introduction to the series. Implementation Science, 13, (1), 2. https://doi.org/10.1186/s13012-017-0688-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. McManamny T. E. (2022). What is the health education role of paramedics for older individuals and communities within rural and regional Australia? A mixed-methods study [PhD thesis, Monash University, Melbourne, VIC, Australia; ]. [Google Scholar]
  10. McManamny T., Sheen J., Boyd L., & Jennings P. A. (2015). Mixed methods and its application in prehospital research: A systematic review. Journal of Mixed Methods Research, 9(3), 214–231. [Google Scholar]
  11. Moher D., Shamseer L., Clarke M., Ghersi D., Liberati A., Petticrew M., Shekelle P., Stewart L. A., & PRISMA-P Group. (2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Systematic Reviews, 4, (1), 1. https://doi.org/10.1186/2046-4053-4-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Murphy A., Barrett M., Cronin J., McCoy S., Larkin P., Brenner M., Wakai A., & O’Sullivan R. (2014). A qualitative study of the barriers to prehospital management of acute pain in children. Emergency Medicine Journal, 31(6), 493–498. [DOI] [PubMed] [Google Scholar]
  13. O’Cathain A., Murphy E., & Nicholl J. (2008). The quality of mixed methods studies in health services research. Journal of Health Services Research Policy, 13(2), 92–98. [DOI] [PubMed] [Google Scholar]
  14. Olaussen A., Bowles K., Lord B., & Williams B. (Eds.). (2022). Introducing, designing and conducting research for paramedics (1st ed.). Elsevier Health Sciences. [Google Scholar]
  15. Olaussen A., Semple W., Oteir A., Todd P., & Williams B. (2017). Paramedic literature search filters: Optimised for clinicians and academics. BMC Medical Informatics and Decision Making, 17, (1), 146. https://doi.org/10.1186/s12911-017-0544-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Page M. J., McKenzie J. E., Bossuyt P. M., Boutron I., Hoffmann T. C., Mulrow C. D., Shamseer L., Tetzlaff J. M., Akl E. A., Brennan S. E., Chou R., Glanville J., Grimshaw J. M., Hróbjartsson A., Lalu M. M., Li T., Loder E. W., Mayo-Wilson E., McDonald S. . . . Moher D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ, 372, n71. https://doi.org/10.1136/bmj.n71. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Schünemann H. J., Higgins J. P. T., Vist G. E., Glasziou P., Akl E. A., Skoetz N., & Guyatt G. H. (2022). Completing ‘summary of findings’ tables and grading the certainty of the evidence. In Higgins J. P. T., Thomas J., Chandler J., Cumpston M., Li T., Page M. J., & Welch V. A. (Eds.), Cochrane handbook for systematic reviews of interventions version 6.3 (updated February 2022). Cochrane. https://training.cochrane.org/handbook/current/chapter-14. [Google Scholar]
  18. Siriwardena A. N., & Whitley G. A. (2022). Prehospital research methods and practice. Class Professional Publishing. [Google Scholar]
  19. Snooks H. A., Anthony R., Chatters R., Dale J., Fothergill R., Gaze S., Halter M., Humphreys I., Koniotou M., Logan P., Lyons R., Mason S., Nicholl J., Peconi J., Phillips C., Phillips J., Porter A., Niroshan Siriwardena A., Smith G. . . . Russell I. T. (2017). Support and assessment for fall emergency referrals (SAFER) 2: A cluster randomised trial and systematic review of clinical effectiveness and cost-effectiveness of new protocols for emergency ambulance paramedics to assess older people following a fall with referral to community-based care when appropriate. Health Technology Assessment, 21(13), 1–218. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Stub D., Smith K., Bernard S., Nehme Z., Stephenson M., Bray J. E., Cameron P., Barger B., Ellims A. H., Taylor A. J., Meredith I. T., & Kaye D. M. (2015). Air versus oxygen in ST-segment-elevation myocardial infarction. Circulation, 131(24), 2143–2150. [DOI] [PubMed] [Google Scholar]
  21. Tate R. C., Hodkinson P. W., Meehan-Coussee K., & Cooperstein N. (2016). Strategies used by prehospital providers to overcome language barriers. Prehospital Emergency Care, 20(3), 404–414. [DOI] [PubMed] [Google Scholar]
  22. Turner J. V., & Lane J. (2006). Early barriers for university rural clinical placements. Education for Health: Change in Learning and Practice, 19(3), 375–379. [DOI] [PubMed] [Google Scholar]
  23. Wallgren U. M., Bohm K. E. M., & Kurland L. (2017). Presentations of adult septic patients in the prehospital setting as recorded by emergency medical services: A mixed methods analysis. Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine, 25, (1), 23. https://doi.org/10.1186/s13049-017-0367-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Whitley G. A., Munro S., Hemingway P., Law G. R., Siriwardena A. N., Cooke D., & Quinn T. (2020). Mixed methods in pre-hospital research: Understanding complex clinical problems. British Paramedic Journal, 5(3), 44–51. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from British Paramedic Journal are provided here courtesy of College of Paramedics

RESOURCES