Integrating Mixed Methods in Health Services and Delivery System Research

Turbulence, change, fragmentation, and multiple disruptive innovations characterize the current dynamic state of health services and delivery systems in the United States. Recent efforts to redesign and transform care delivery are searching for ways to overcome the challenges of fragmentation, inequality, and inappropriate care use while advancing the triple aims of better health and better care at lower cost for everyone. Health services research can contribute to these efforts by providing valid characterizations of the complex interactions among components of the current care delivery systems and by assessing impacts of efforts to redesign and improve care delivery. Mixed methods research can help investigators fully capture the complex interactions among system components, including interactions among multiple levels of analysis and over time. Through mixed methods, researchers can identify social, organizational, technical, and market contexts that shape the course and outcomes of improvement initiatives. Use of mixed methods may also make it easier for researchers to engage in dialogues with decision makers who formulate and implement programs of delivery system change, and to better communicate with other participants in the delivery system, including its users.

This special issue of Health Services Research, sponsored by the Agency for Healthcare Research and Quality, illustrates how mixed methods studies can make important contributions to health services research.1 Furthermore, it seeks to move the field of mixed methods research toward creating a methodological mosaic that better corresponds to the complex phenomena. In this introduction, we describe the papers and briefly indicate where they fit into the emerging mixed methods mosaic. Then we provide some suggestions about what may be missing and anticipate a future state of a more integrated approach to health services research and richer dialogue between researchers and those engaged in health care.

The growing role of mixed methods in health services research is evident from the publication of hundreds of mixed methods studies and recent overviews of best practices in mixed methods research (Palinkas, Aarons et al. 2011; Creswell et al. 2011; Curry et al. 2013; Ozawa and Pongpirul 2013; Zhang and Watanabe-Galloway 2013). This growing success reflects the capacity of mixed methods studies to capture the experiences, emotions, and motivations of people providing and receiving health care, as well as the objective conditions of care delivery. This wider use of mixed methods also reflects the ability of mixed methods to meet practical needs for assessing and understanding the complexity of health service delivery, which often results in an underlying emphasis on pragmatism. By applying a pragmatic mix of methods that work best in real-world situations, researchers thus find ways to overcome the assumed incompatibilities among research paradigms (Sale, Lohfeld, and Brazil 2002) and between qualitative and quantitative methods (Howe 2003; Onwuegbuzie and Leech 2005).

Nevertheless, the recent proliferation of mixed methods research has exposed four areas needing more attention. First, until recently, most mixed methods studies embodied a form of “separate and unequal” in the ways that methods are used, data collected, and results published. Thus, there has been limited discussion about how to actually integrate multiple methods. Second, there has been a lack of common terminology for describing mixed methods designs and the multiple ways of combining methods. Third, the literature provides limited guidance about how to build the collaborative, cross-disciplinary teams that are required to execute a complicated mixed methods project and effectively manage the accompanying methodological and epistemological challenges. Fourth, at times pragmatism is insufficient and some paradigm issues still require attention. The six manuscripts in this issue highlight many of the reasons that mixed methods research has become popular while also addressing these four challenges; thus, they should help move mixed methods research toward a more complete methodological mosaic.

The Papers

Fetters, Curry, and Creswell (2013) provide a common language and present an organizing framework for describing and examining mixed methods research designs and processes. Their work facilitates future methodological development, as well as comparisons of the strengths and weaknesses of different designs. This common language makes it easier to describe studies in manuscripts and grant applications and is a major improvement over the current confused state. Their classification framework describes many possible variations of mixed methods and allows for creation of hybrid designs and linkages. A further contribution is the treatment of ways that full methodological integration encompasses all phases of research—including design, data gathering, analysis, and reporting. Additionally, the authors anticipate possibilities for future developments of participatory designs and emergent improvisation.

Bowers et al. (2013) provide essential practical insights into how to organize and sustain research teams that are crucial for conducing mixed methods studies. They describe what it takes to build effective interprofessional teams, including a more active and creative role for funding agencies as motivators and facilitators. They make it clear that relationship development requires time, regular meetings, and facilitation.

Using the terminology introduced by Fetters, Curry, and Creswell, Scammon and colleagues illustrate the process of merging data using a convergent design within a case study framework where qualitative and quantitative data are gathered at the same time (Scammon et al. 2013). They present their findings using narrative weaving, in which they integrate divergent types of data in the results. They also provide ample illustrations of the messy complexity of accounting for all the “fits” (confirmation, expansion, and discrepancy). Like Bowers, they describe the challenging work involved in building an effective team and remind us of the practical importance of paying attention to all stakeholders and contexts. “Connecting the Dots” illustrates the multiple linkages of collection and analysis and gives many useful suggestions about ways to anticipate and navigate challenges in integrated mixed methods research. The article also illustrates another benefit of mixed methods research: the authors' methodology and findings helped overcome a strict separation between research and change management and facilitated more effective dialogue between the researchers and the leaders of the primary care redesign.

Zickmund et al. (2013) provide a richly documented example of integration through data transformation using a mixed methods convergent design. By carefully transforming semistructured interviews of cancer patients into numerical values, they identify the importance of self-view in cancer care and present results as a narrative in outcomes of care.

Hamilton et al. (2013) illustrate the use of mixed methods in quality improvement research using a convergent design within an intervention framework. The results are a great example of using a contiguous narrative to present their key findings for care of veterans with mental health problems. Their results show the importance of real-time feedback with improvement in both patient employment and care team actionable understanding.

Gilmer et al. (2013) illustrate the use of mixed methods in evaluation research, using a hybrid design that combines explanatory and exploratory sequential designs. This manuscript also provides an example of how to use Lawrence Palinkas' classification of mixed method functions (Palinkas, Horwitz et al. 2011). In evaluating the implementation of California's Full Services Partnerships, a state program caring for homeless persons, they describe the complexity and variability in finding homes, while also identifying strategies for enhancing fidelity across program sites.

Crafting a Mosaic of Mixed Methods Research

There are currently many opportunities to advance the knowledge on the integration of mixed methods. This is a period of health care transformation with the associated urgent needs for better understanding, rapid improvements, and innovations. All of these activities beg for mixed methods designs. Encouraging support for mixed methods work comes from the Patient Centered Outcomes Research Institute (PCORI), the NIH-supported Clinical and Translational Science Awards (CTSA) program, NIH Dissemination and Implementation, and PBRNs with their community focus.

Since its sponsorship of a special issue of Health Services Research on qualitative methods (Devers 1999), many funding announcements from the Agency for Healthcare Research and Quality (AHRQ) have encouraged use of a wide range of qualitative and quantitative research designs. Field-based implementation research studies and evaluations conducted under contract for AHRQ have often used mixed methods. A prominent recommendation from a 2011 meeting on Advancing Delivery System Research (http://www.ahrq.gov/professionals/systems/system/delivery-system-initiative/index.html#messages) was that researchers make more use of mixed methods designs and, particularly, that research designs and analysis techniques seek synergies across qualitative and quantitative data (Alexander and Hearld 2012). These methodological approaches can help illuminate ways that improvement initiatives and practices are shaped by the contexts in which they are implemented.

What are the research pieces still missing to help craft a more complete mosaic of mixed methods and health services? One set of pieces in the mosaic would enhance communication among researchers, decision makers, and practitioners of care. Knowledge must move through a complex maze to get translated into better policy or care. When researchers directly gather data from system leaders and practitioners, they gain opportunities to understand the concerns and perspectives of these actors. In applied research, this understanding can contribute to the crafting of findings and reports that better reflect user concerns. When researchers weave findings reflecting actors' perspectives and experiences into their reports, they make these findings more accessible to decision makers and practitioners than would be the case for findings based solely on statistical data. In this fashion, mixed methods studies can enhance the chances of engaging policy makers, system leaders, and practitioners in dialogue about the nature of the research and the implications of the findings.

Reporting stories to accompany numbers provides one particularly effective means of increasing the engagement of all the participants, including investigators, readers, “subjects,” decision makers, and practitioners. Stories communicate both emotions and facts and can sometimes affect people's beliefs and behavior in ways that numbers alone cannot. Mixed method designs are ideally suited for crafting research stories. Each method represents a different perspective on a specific phenomenon and pays attention to different phenomena within the scene being investigated. Integrating these divergent approaches is the work of crafting a mosaic from which an emotionally engaging and empirically valid research story is created. Learning how best to do this makes for richer science and should be a subject for methodological inquiry.

The need for greater user engagement is reinforced by some of the shared themes across the six articles in this issue. Spanning across a variety of settings and conditions, including nursing homes, primary care practices, cancer care, VA mental health, and state mental health, these articles highlight the importance of relationships, appreciating complexity, using narrative, paying attention to issues of identity, and acknowledging the value of contextually appropriate variability. Thus, democratizing knowledge-making through greater involvement of those being researched takes on more importance, as do calls for more participatory designs and more emergent, embedded designs. As an example, Donna Mertens proposes a “transformational paradigm,” seeking emergent understanding and a better life together (Mertens 2009). All of this requires paying attention to and enacting the essentials of successful inter-professional, collaborative team-building. Team relationship development needs time, regular meetings, facilitation, and appropriate leadership. This was abundantly clear in the articles in this issue by Bowers et al. for research team development and Scammon et al. about primary care transformation. Longer time frames will be necessary for projects so that they can take advantage of these relationships and have the time to more effectively engage the community/participants and sustain efforts so we can move from research to translated action and better services.

Paradigm questions still remain. Too often, the integrity and value of qualitative approaches succumb to serving the needs of the still dominant positivist paradigm. Too often, quantitative approaches are misapplied to phenomena requiring qualitative methods. The methods are based upon specific ontological and epistemological premises that result in different worlds being seen. Increasingly, health services and delivery system investigators seek to understand and/or evaluate a field with multiple phenomena. It is important to match each phenomenon to the most appropriate method and then fit each piece into the larger mosaic, that is, integrate through complementarity. As a guide to integrative work, we recommend six principles which assume that multiple ways of knowing are necessary for scientific inquiry (Stange, Miller, and McWhinney 2001). These are as follows: (1) know why you choose to do something; (2) preserve method and paradigm integrity; (3) pay attention to units of analysis; (4) remember the research questions; (5) ensure that strengths and weaknesses of each selected method complement each other; and (6) continually evaluate methodology throughout the study (Coward 1990; Patry 2013).

Summary Reflections

We live and perform our research at a time of great challenges in health care, and where our work could really matter. Together as a research community, we can use our collective imaginations and diverse methods to craft a more complete mosaic of intricate beauty from the many broken fragments of our current health services and delivery systems. Here are four challenges for optimizing the value and impact of our mixed methods work. Enlarge the mosaic and invite more people into the mixed methods process, while remembering to do the work of developing and sustaining relationships. Do it with feeling and tell creative, rigorous, and disciplined research stories that awaken emotions. Deepen and contextualize the practical and ethical questions and concerns related to health services. Trouble the waters and seek change, maybe even real transformation. This special issue begins to reveal a way.

Supporting Information

Additional supporting information may be found in the online version of this article:

Appendix SA1: Author Matrix.