Skip to main content
PMC home page
Family Practice logoLink to Family Practice
. 2019 Mar 19;36(5):666–671. doi: 10.1093/fampra/cmz010

Practical mixed methods strategies used to integrate qualitative and quantitative methods in community-based primary health care research

Navdeep Kaur 1,✉, Isabelle Vedel 1, Reem El Sherif 1, Pierre Pluye 1
PMCID: PMC6781936  PMID: 30888015

Abstract

Background

Mixed methods (MM) are common in community-based primary health care (CBPHC) research studies. Several strategies have been proposed to integrate qualitative and quantitative components in MM, but they are seldom well conceptualized and described. The purpose of the present review was to identify and describe practical MM strategies and combinations of strategies used to integrate qualitative and quantitative methods in CBPHC research.

Methods

A methodological review with qualitative synthesis (grouping) was performed. Records published in English in 2015 were retrieved from the Scopus bibliographic database. Eligibility criteria were: CBPHC empirical study, MM research with detailed description of qualitative and quantitative methods and their integration. Data were extracted from included studies and grouped using a conceptual framework comprised of three theoretical types of MM integration, the seven combinations of these types and nine practical strategies (three per type of integration) and multiple combinations of strategies.

Results

Among the 151 articles reporting CBPHC and MM studies retrieved, 54 (35.7%) met the inclusion criteria for this review. The included studies provided examples of the three theoretical types of MM integration, the seven combinations of these types as well as the nine practical strategies. Overall, 15 combinations of these strategies were observed. No emerging strategy was observed that was not predicted by the conceptual framework.

Conclusions

This review can provide guidance to CBPHC researchers for planning, conducting and reporting practical strategies and combinations of strategies used for integrating qualitative and quantitative methods in MM research.

Keywords: Community-based primary health care, integration strategies, methodological review, mixed methods, research designs, research methods

KEY MESSAGES.

  • Primary health care researchers benefit from staying abreast of mixed methods (MM).

  • MM can address the complex issues of primary care.

  • Several possibilities of MM integration strategies and combinations exist.

Introduction

Mixed methods (MM) consist of integrating qualitative and quantitative methods in program evaluation, primary research and literature reviews (1–5). Over the last 15 years, there has been a considerable increase in the number of MM publications in health sciences (Fig. 1). Likewise, MM are common in primary care and family medicine research (6–9). Indeed, MM help to understand the complexity of primary care (10) that deals with a variety of settings (from clinical to community) and patients with diverse health care needs (11). Furthermore, the integration of both qualitative and quantitative methods provides new insights to improve and address the complex and multi-faceted issues of primary care (12,13). Regarding research designs, MM have matured. However, with respect to practical MM integration strategies, knowledge remains fragmented.

Figure 1.

Figure 1.

Open in a new tab

This graph shows that the trend of use of MM in health sciences was generated using following search strategy: TITLE (‘mixed methods’ OR ‘mixed method’) AND DOCTYPE (ar) AND PUBYEAR > 2002 AND PUBYEAR < 2017 AND (LIMIT-TO (DOCTYPE, ‘ar’)) AND (LIMIT-TO (SUBJAREA, ‘MEDI’) OR LIMIT-TO (SUBJAREA, ‘SOCI’) OR LIMIT-TO (SUBJAREA, ‘PSYC’) OR LIMIT-TO (SUBJAREA, ‘NURS’) OR LIMIT-TO (SUBJAREA, ‘ARTS’) OR LIMIT-TO (SUBJAREA, ‘HEAL’) OR LIMIT-TO (SUBJAREA, ‘DECI’)) AND (LIMIT-TO (LANGUAGE, ‘English’)) AND (LIMIT-TO (SRCTYPE, ‘j’)). The 2003 benchmark was chosen because it was the year of the publication of the first edition of the SAGE handbook on MM (27)

Numerous strategies have been proposed to combine qualitative and quantitative methods in MM studies, but these have not been sufficiently theorized and described (5,6,14–18). Consequently, planning, conducting and clearly reporting the various MM strategies used to combine qualitative and quantitative methods is challenging. In view of this, our research question was: what are the practical strategies and combinations of strategies for integrating qualitative and quantitative methods in MM studies used by community-based primary health care (CBPHC) researchers?

The purpose of this review was to identify and describe the practical MM integration strategies and combinations of strategies that are used in CBPHC research. To achieve this goal, we used the conceptual framework proposed by Pluye et al. (5) This original framework is based on a review of the methodological literature on MM in health and social sciences. The following text of the conceptual framework is derived from Pluye et al. (5) with minor modifications:

At the theoretical level, the framework proposes three types of integration of qualitative and quantitative methods in MM research and seven combinations of these types (5). The three types of integration are as follows: the connection of qualitative and quantitative phases (hereafter T1), the comparison of qualitative and quantitative results (hereafter T2) and the assimilation of qualitative and quantitative data (hereafter T3). The seven combinations of types (a combination being one or more than one type) are three mono combinations (T1; T2; T3), three duo combinations (T1 and T2; T1 and T3; T2 and T3) and one trio combination (T1 and T2 and T3). At the practical level, the framework proposes nine strategies for integrating qualitative and quantitative methods in MM research and multiple possibilities of combinations of these strategies. The nine practical strategies are listed in Table 1 (three strategies per type of integration). Thereby, this framework offers multiple possibilities to combine strategies to plan, conduct and report the integration of qualitative and quantitative methods in MM research (5).

Table 1.

Nine practical strategies for integrating phases, results or data of QUAL and QUAN in MM

Three types of integration
Nine practical strategies		 Definitions
TYPE 1 (T1): Connection of phases Connection of the QUAL and QUAN phases
 1a. Phase QUAL to Phase QUAN Connect the results of a QUAL Phase 1 with the collection and analysis of a QUAN Phase 2 (1)
 1b. Phase QUAN to Phase QUAL Connect the results of a QUAN Phase 1 with the collection and analysis of a QUAL Phase 2 (1)
 1c. Special case of 1a and 1b: ‘Follow a thread’ Analyze the QUAL (or QUAN) data and identify the main themes (or variables) that require further study; choose a theme (or variable) and re-analyse through the QUAN (or QUAL) components (18)
TYPE 2 (T2): Comparison of results Compare the results of QUAL and QUAN
 2a. QUAL and QUAN results obtained separately Compare similarities and differences between QUAL and QUAN results obtained from separate data collection and analysis (1)
 2b. QUAL and QUAN results obtained in an interdependent manner Compare similarities and differences between QUAL and QUAN results from interdependent data collection and analysis (1)
 2c. Special case of 2a and 2b: Divergence of QUAL and QUAN results Compare by focussing on discrepancies (contradictions, discordances or dissonances) between QUAL and QUAN results (26)
TYPE 3 (T3): Assimilation of data Merging of QUAL and QUAN data
 3a. QUAL data into QUAN data Merging of QUAL and QUAN data by transforming QUAL data into QUAN data (1)
 3b. QUAN data into QUAL data Merging of QUAL and QUAN data by transforming QUAN data into QUAL data (1)
 3c. Merging of QUAL and QUAN data Merging of QUAL and QUAN data by merging them for each case in an additional database (1)
Open in a new tab

References: (1), (18) and (26)

(Reproduced from Pluye et al. (5) by permission of the International Journal of Multiple Research Approaches)

Methods

We conducted a methodological review with qualitative synthesis of MM studies in CBPHC. Methodological reviews identify key method-related aspects to inform future research (19). In this review, we have used the definition of the CBPHC provided by the Canadian Institutes of Health Research (CIHR). According to this definition, CBPHC satisfies the three following conditions: (i) health services (or health and social services) are given by professionals including (but not exclusively) dentists, dietitians, nurses, pharmacists, physicians, psychologists, public health practitioners and social workers; (ii) health services include (but not exclusively) disease prevention-diagnosis-treatment, public health, rehabilitation and end-of-life care and (iii) health services are provided in community settings including (but not exclusively) people’s homes, health care clinics, hospices, physicians’ offices, public health units and workplaces (20). By way of illustration, CBPHC includes common services such as (among other services): acute and chronic disease-related care, basic emergency services, healthy child development, nutrition counselling, palliative end-of-life care, primary maternity care, primary mental health care, psychosocial services, public health services, rehabilitation services and referrals to, and coordination with, specialized care.

Eligibility criteria

The following eligibility criteria and search strategy are centred on CBPHC, while other elements are derived from Pluye et al. (5) with minor modifications:

Inclusion criteria

For this review, we considered a study was MM if at a minimum one qualitative method and one quantitative method were used rigorously and integrated (1). Studies were included if they were CBPHC empirical studies published in English in 2015 that used MM and fulfilled at least the first three Good Reporting of A Mixed Methods Study (GRAMMS) recommendations (21), which are to describe: (i) the explanation for employing an MM approach for addressing the research question; (ii) the MM design in terms of the purpose, priority and sequence of methods and (iii) each method in terms of sampling, data collection and analysis. The following three last GRAMMS recommendations were applied but not used as inclusion criteria. They were used to describe: (iv) where MM integration has occurred, how it has occurred and who has participated in it; (v) any limitation of a method associated with the presence of the other methods and (vi) any insights gained from mixing or integrating methods. To be included, the qualitative methods, quantitative methods and their integration must be described minimally in one paragraph (as per the aforementioned first three GRAMMS recommendations).

Exclusion criteria

Studies were excluded if they reported: (i) a quantitative method with gathering and analysis of qualitative data without qualitative research methodology and method, (ii) a qualitative method with gathering and analysis of quantitative data without quantitative research methodology and method, (iii) a combination of quantitative methods, (iv) a combination of qualitative methods and (v) the use of qualitative and quantitative methods (like two distinct studies) without mixing the two methods. Non-CBPHC studies were excluded, such as hospital research (conducted outside any CBPHC clinic). Educational research (with students, residents or both), research on veteran services, in army settings and in prisons were also excluded.

Information source and search strategy

We searched the Scopus bibliographic data base with the following search strategy: ‘Title (mixed method*) AND SUBJAREA (MEDI OR SOCI)’. All records were imported to the collaborative online system, eSRAP©, for monitoring and filtering research publications (22).

Selection process

A three-step study selection process was followed. For each step, records and reviewers’ coding were imported in an Excel file to document the study selection process. First, all duplicates were removed. Second, two reviewers (NK and PP) screened records (titles and abstracts) using eligibility criteria and selected potentially relevant studies. Third, for each selected record, the two reviewers read the full-text papers and selected empirical CBPHC studies using MM with a detailed description of methods. These two reviewers selected records and full texts separately but not independently as they met to discuss and debate their decisions and reach consensus. When no consensus could be reached, the record or full text was deferred to a third party (IV). A fourth reviewer (RES) coded a random sample of 10% of the records to test the inter-rater reliability. The overall Cohen’s kappa calculated was 0.62 and considered ‘substantial agreement’ (Cohen’s kappa between 0.61 and 0.80) (23).

Data extraction and synthesis

For each included study, two reviewers (NK and PP) extracted the following data from full texts: (i) the type of MM design (sequential, convergent, multiphase, multilevel or multiphase-multilevel), (ii) the type or types of MM integration (theoretical level) and (iii) the applied MM integration strategy or strategies (practical level). The extracted data were synthesized using the grouping technique of qualitative synthesis (24). Based on the conceptual framework, the data were assigned codes for: (i) the MM design, (ii) the seven combinations of types of MM integration (T1; T2; T3; T1 and T2; T1 and T3; T2 and T3; T1 and T2 and T3) and (iii) for the practical MM strategies (1a, 1b, 1c, 2a, 2b, 2c, 3a, 3b, 3c). Then, the coded data were grouped into similar combinations of strategies and tabulated.

Results

Once duplicates had been removed from the 764 records retrieved, 760 unique records remained for screening. Among these, we identified 151 potentially relevant articles and selected 54 of these for inclusion in this review. The flow diagram is presented in Figure 2.

Figure 2.

Figure 2.

Open in a new tab

Flowchart of studies selection

Among the 54 included studies, 30 (55.5%) studies used a convergent design, 19 (35.1%) used a sequential design, three (5%) used a multiphase design and one (1%) used a multilevel design. One study (1%) used an ‘other’ type of MM design, that is, a variant of a sequential design where results of a Phase 1 convergent design (qualitative + quantitative) informed a Phase 2 quantitative. Only four studies (7.4%) conformed to all six GRAMMS recommendations for reporting MM studies. Sixteen (29.6%) met five of the recommendations, 19 (35.1%) met four of the recommendations and 15 (27.7%) fulfilled only the three first recommendations. The combinations of types of MM integration and practical MM integration strategies identified in the included studies are presented in Table 2.

Table 2.

Distribution of studies (n = 54) according to common types of MM design and seven combinations of MM strategies

Three types of integration
Nine practical strategies		 Combinations of strategies (according to the type of MM design)
(n = number of studies illustrating each combination)
Sequential Convergent Multiphase Multilevel Other*
T1. Connection of phase
 1a. Phase QUAL to Phase QUAN 1a (n = 4) 1a (n = 1)
 1b. Phase QUAN to Phase QUAL 1b (n = 4) 1b (n = 1)
 1c. Special case: following a thread 1c & 2b (n = 1)
T2. Comparison of results
 2a. QUAL and QUAN results obtained in a separate manner 2a (n = 9)
 2b. QUAL and QUAN results obtained in an interdependent manner 1a & 2b (n = 2)
1b & 2b (n = 6)		 2b (n = 15) 1a & 2b (n = 1)
 2c. Special case: divergence 2a & 2c (n = 1)
2b & 2c (n = 1)
T3. Assimilation of data
 3a. QUAL data in QUAN data 1a & 3a (n = 1) 2b & 3a (n = 1) 3a (n = 1)
 3b. QUAN data in QUAL data 3b (n = 1)
 3c. Merging of QUAL and QUAN data 1a & 2b & 3c (n = 1)
1b & 3c (n = 1)		 2b & 3c (n = 1)
2b & 3b & 3c (n = 1)
4. Emerging strategies
None found
Observed combinations of
TYPES of integration		 7 (n = 19) 8 (n = 30) 3 (n = 3) 1 (n = 1) 1 (n = 1)
Open in a new tab

(Reproduced from Pluye et al. (5) by permission of the International Journal of Multiple Research Approaches)

*Other design: variant of a sequential design where results of a Phase 1 convergent design (QUAL + QUAN) informed a Phase 2 QUAN.

The symbol ‘&’ (meaning ‘and’) allows to represent the combinations with several strategies.

At the theoretical level, all seven combinations of types of MM integration were observed. At the practical level, the following combinations of MM strategies were observed: six combinations including one strategy (1a; 1b; 2a; 2b; 3a; 3b), seven combinations of two strategies (1a and 2b; 1b and 2b; 1a and 3a; 1b and 3c; 2a and 2c; 2b and 2c; 2b and 3c) and two with three strategies (1a and 2b and 3c; 2b and 3b & 3c). No emerging strategy was observed that was not predicted by the conceptual framework.

Discussion

With this review, we identified and described practical MM strategies and combinations of strategies that are used for integrating qualitative and quantitative methods in CBPHC research. We found 54 CBPHC studies published in 2015 that illustrated the three theoretical types of MM integration, the seven combinations of these types as well as the nine practical strategies. Overall, 15 combinations of these strategies were identified and the conceptual framework was supported by the data.

We acknowledge that this methodological review has two limitations. First, this review was limited to records published in English in 2015. An explanation for this limitation is that the goal of the present review was not to conduct an exhaustive review of literature in this area but to identify and describe integration strategies and combinations of strategies using a manageable sample of studies. Second, we focussed only on the reporting quality of the included studies and did not assess their methodological quality. However, our sample of CBPHC studies that used MM was deemed sufficient to test the conceptual framework in CBPHC research and achieve the aim of the present review. The primary strength of the present review was the use of an innovative conceptual framework to systematically organize and describe the various strategies and combinations of integration strategies. This comprehensive description can provide guidance to CBPHC researchers to plan, conduct and clearly report their MM studies.

Integration is an inherent part of MM research and to make it explicit, transparent and reproducible, careful heed must be given to report how it is done (4). Yet, our results suggest that to date, the proportion of poor quality reporting of MM in CBPHC is substantial since 93% of included studies were not reported according to all six GRAMMS recommendations (the studies were published in 2015 and the recommendations in 2008). This is comparable to what has been found for MM by Pluye et al. (5) on patient-oriented research in health and social sciences (5). Thus, there is a need for advocacy of good planning and reporting practices among CBPHC researchers and editors through international organizations such as ‘The North American Primary Care Research Group’. Further, we suggest that for explicit, transparent and reproducible integration strategies, journals should include specific recommendations under ‘Guidelines to Authors’ to prompt explicit reporting of MM such as GRAMMS (25).

Conclusions

This review described practical strategies used for integrating qualitative and quantitative phases, results and data. Specifically, the combinations of integration strategies described in this manuscript provide ideas and possibilities to guide CBPHC researchers for planning and reporting their future MM studies. Additionally, this review has emphasized a need of advocacy of guidance for transparent reporting of MM studies among CBPHC researchers and editors, which can contribute to publish reproducible MM studies.

Acknowledgements

Authors gratefully acknowledge the support of the Method Development platform of the Quebec SPOR SUPPORT Unit, Department of Family Medicine, McGill University, Montréal, Canada. Authors are thankful to the International Journal of Multiple Research Approaches for providing the permission to reuse from our previous publication Tables 1 and 2. NK holds a Postdoctoral Fellowship from the Québec SPOR-SUPPORT Unit. RES holds a Doctoral Award from the Canadian Institutes of Health Research (CIHR). IV holds a CIHR New Investigator Fellowship and a Dawson Scholar (McGill University). PP holds a Senior Research Fellowship of the ‘Fond de recherche du Québec – Santé’.

Declaration

Funding: The review was sponsored by the Fond de recherche du Québec – Santé and the Quebec SPOR Support Unit.

Ethical approval: Ethical approval was not required since it was a literature review.

Conflict of interest: None.

References

  • 1. Johnson R, Onwuegbuzie AJ, Turner LA. Toward a definition of mixed methods research. J Mix Methods Res 2007; 1: 112–33. [Google Scholar]
  • 2. Pluye P, Hong QN. Combining the power of stories and the power of numbers: mixed methods research and mixed studies reviews. Annu Rev Public Health 2014; 35: 29–45. [DOI] [PubMed] [Google Scholar]
  • 3. Tashakkori A, Teddlie C (eds.) Handbook of Mixed Methods in Social and Behavioral Research. Thousand Oaks, CA: Sage, 2010. [Google Scholar]
  • 4. Creswell JW, Plano Clark VL.. Designing and Conducting Mixed Methods Research, 3rd edn Thousand Oaks: Sage, 2018. [Google Scholar]
  • 5. Pluye P, Bengoechea EG, Granikov V, Kaur N, Tang D. A world of possibilities in mixed methods: review of the combinations of strategies used to integrate the phases, results and quantitative and qualitative data (Invited paper). Int J Mult Res Approaches 2018; 10(1):41–56. [Google Scholar]
  • 6. Creswell JW, Fetters MD, Ivankova NV. Designing a mixed methods study in primary care. Ann Fam Med 2004; 2: 7–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7. Burt J. Following the mixed methods trail: some travel advice. Br J Gen Pract 2015; 65: 264–5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8. Guetterman TC, Fetters MD, Creswell JW. Integrating quantitative and qualitative results in health science mixed methods research through joint displays. Ann Fam Med 2015; 13: 554–61. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9. Stange KC, Gotler RS. Mixed methods and diverse perspectives. Ann Fam Med 2006; 4: 290–1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10. De Maeseneer JM, van Driel ML, Green LA, van Weel C. The need for research in primary care. Lancet 2003; 362: 1314–9. [DOI] [PubMed] [Google Scholar]
  • 11. van Weel C, Rosser WW. Improving health care globally: a critical review of the necessity of family medicine research and recommendations to build research capacity. Ann Fam Med 2004; 2 (suppl 2):S5–16. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12. Borkan JM. Mixed methods studies: a foundation for primary care research. Ann Fam Med 2004; 2: 4–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13. Vedel I, Kaur N, Hong QN et al. Why and how to use mixed methods in primary health care research. Fam Pract 2019; 365–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14. Greene JC. Is mixed methods social inquiry a distinctive methodology? J Mix Methods Res 2008; 2(1):7–22. [Google Scholar]
  • 15. Bryman A. Integrating quantitative and qualitative research: how is it done? QRJ 2006; 6(1): 97–113. [Google Scholar]
  • 16. Caracelli V, Greene JC. Data analysis strategies for mixed-method evaluation designs. Educ Eval Policy Anal 1993; 15: 195–207. [Google Scholar]
  • 17. Onwuegbuzie A, Combs JP. Emergent data analysis techniques in mixed methods research: a synthesis. In: Tashakkori A, Teddlie C (eds.). Handbook of Mixed Methods in Social and Behavioral Research. Thousand Oaks: Sage, 2010: 397–430. [Google Scholar]
  • 18. O’Cathain A, Murphy E, Nicholl J. Three techniques for integrating data in mixed methods studies. BMJ 2010; 341: 1147–50. [DOI] [PubMed] [Google Scholar]
  • 19. Randolph JJ. A guide to writing the dissertation literature review. Prac Assess, Res & Eval 2009; 14(13): e1. [Google Scholar]
  • 20. CIHR. Canadian Institutes of Health Sciences http://www.cihr-irsc.gc.ca/e/43626.html (accessed on 20 May 2018).
  • 21. O’Cathain A, Murphy E, Nicholl J. The quality of mixed methods studies in health services research. J Health Serv Res Policy 2008; 13: 92–8. [DOI] [PubMed] [Google Scholar]
  • 22. Granikov V, Tang DL, Bouthillier F, Pluye P. eSRAP: a system for collaborative monitoring of latest trends in patient oriented research. Paper presented at: Joint meeting of the Medical Library Association (MLA), the Canadian Health Libraries Association/Association des bibliothèques de la santé du Canada (CHLA/ABSC), and the International Clinical Librarian Conference (ICLC), Toronto, May 16, 2016; abstract published in the J Can Health Libr Assoc 2016; 37(2):96–100. [Google Scholar]
  • 23. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33: 159–74. [PubMed] [Google Scholar]
  • 24. Popay JR, Sowden H, Petticrew A et al. Guidance on the Conduct of Narrative Synthesis in Systematic Reviews. Lancaster, UK: Lancaster University, 2006. [Google Scholar]
  • 25. O’Cathain A, Murphy E, Nicholl J. Why, and how, mixed methods research is undertaken in health services research in England: a mixed methods study. BMC Health Serv Res 2007; 7(85): e85. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26. Pluye P, Grad RM, Levine A, Nicolau B. Understanding divergence of quantitative and qualitative data (or results) in mixed methods studies. Int J Mult Res Approaches 2009; 3(1): 58–72. [Google Scholar]
  • 27. Tashakkori A, Teddlie C (eds). Handbook of Mixed Methods in Social & Behavioral Research. Thousand Oaks, CA: Sage, 2003. [Google Scholar]

Articles from Family Practice are provided here courtesy of Oxford University Press

RESOURCES