Mixed Methods Studies: A Foundation for Primary Care Research

Two articles in this issue focus on mixed methods or multimethod techniques, offering a tantalizing glimpse of the potential of such investigations for primary care. One provides a taxonomy of criteria and models for constructing these types of studies, whereas the other represents an example of such work. Together they help to show the power that mixed methods and multimethod research possess: they suggest, discover, and test hypotheses; they give new insights on complex phenomenon; they allow the investigator to address practice and policy issues from the point of view of both numbers and narratives; they add rigor.

Mixed methods research refers to those studies or lines of inquiry that integrate one or more qualitative and quantitative techniques for data collection and/or analysis. Qualitative data collection methods, such as interviews, focus groups, or participant observation, are now almost routinely used as exploratory (hypothesis formation) phases of subsequent, more quantitative surveys in health care.¹ They have great potential for exploring new topics or familiarizing research teams with a new area. They may also assist in theory building, especially when methods aggregating qualitative studies are utilized.² Qualitative data collection techniques are also used broadly when patients’ or providers’ narratives or their lived experience are sought. Whereas quantitative methods may work best in isolating and identifying the correlates associated with variation at specific moments in time, qualitative techniques are particularly good at gaining insight into the processes and events that lead up to the observed variation³ and have the key advantage of providing unexpected insights. This last point should be emphasized, because the nature of quantitative research and its data collection tools allow the researcher to infer only about that which he or she is examining (you “see” only what you are “looking at”), whereas qualitative methods can expand the gaze to key elements that were never elucidated or even previously considered.

In the first article, Creswell, Fetters, and Ivankova⁴ have constructed an insightful conceptual framework for making sense of the mixed methods field, particularly around issues of quality. Mixed methods not only expand the research toolbox, they also provide the opportunity for synthesis of research traditions and give the investigator additional perspectives and insights that are beyond the scope of any single technique. The result is more than simple addition: the results often transcend the individual methods and disciplines. As Creswell et al note, “This form of research is more than simply collecting both quantitative and qualitative data; it indicates that data will be integrated, related, or mixed at some stage of the research process.” Not only do these types of investigations have the advantages of the deep descriptions and entrée to subjects’ lived realities explored by qualitative methods, they also have the potential to contribute the generalizability and statistical reliability that is the strength of quantitative research. In addition, as the authors note, mixed methods lend themselves to valuable opportunities for data triangulation and transformation and instrument design.

The major contribution of Creswell et al to the scientific literature is that they offer a coding template of important criteria for designing and evaluating mixed methods studies. This template builds on previous literature and includes 5 criteria: rationale for mixing, types of data collected and analyzed, the priority given to qualitative or quantitative research, the implementation sequence, and the phase of the study in which the integration occurred. It goes beyond previous efforts in this area^{5– 7} by suggesting 3 useful defining models that could form the basis for a taxonomy of such studies in primary care. Not only does it make explicit many of the implicit methodological issues, it throws down the gauntlet as to what criteria should be considered in measuring quality and demonstrates it on a sample of mixed methods studies.

That challenge is taken up by Schillaci and colleagues,⁸ who present an example of the both the synchronous and sequential use of multiple qualitative and quantitative methodologies in a study that explores the reasons for the dramatic fall in immunization rates among New Mexico’s children. This research demonstrates several of the strengths of mixed methods investigations, starting with the issue of flexibility when unexpected insights arise. While the investigators were performing an assessment of Medicaid managed care (MMC) using findings from the National Immunization Survey (NIS), they became aware that immunization rates for children were unpredictably dropping in a precipitous manner. The research team responded by reexamining their ethnographic data files for material on immunization and partially redirecting ongoing ethnographic activities during the remaining years of the study.

This type of reexamination of already collected qualitative data is akin to secondary analysis of quantitative material. Although it is considerably more methodologically sound than attempting to recreate the situation from informants’ distant memories, there is often a paucity of relevant raw data, because researchers might not have been attentive to this topic in the primary collection phase. Such exercises, however, can also serve as a starting point for redirection of the researchers’ gaze, an iterative process that picks up from the unexpected insight and refocuses the lens of the inquiry towards it. This powerful shift provides a dynamism to mixed methods research often lacking in less comprehensive designs.

The Schillaci et al study can be analyzed using the 5 criteria suggested by Creswell et al. The rationale for the mixing appears to be that although their quantitative population-based survey and temporal plots detected the statistical trend of decreased immunizations, the researchers needed qualitative, in-depth interviews and field observations to discern potential explanations for the drop—the why. The forms of data collection and analysis were a population-based survey, ethnography (in-depth interviews and field observations), and tracing of preventable adverse sentinel events. Analytical techniques were quite broad and included both statistical and qualitative interpretive processes. The priority given to them in this article appears to be equal, though separate, and the implementation sequence is first synchronous (multimethod assessment), then sequential (further qualitative inquiry after the survey phase). The analysis uses qualitative data to interpret the quantitative findings, and integration between qualitative and quantitative phases occurred at both the beginning and the end of the study—when the unexpected insight arose and when interpretations were applied to the temporal plots. Triangulation is present throughout this study, both between the qualitative and quantitative portions and through comparisons of documents, observations, and interviews, and likely fits most closely into the triangulation design model.

Given the numerous advantages of mixed methods research, as shown in the two articles in this issue of Annals, the casual observer might expect to find reams of such studies in the primary care literature. With the huge benefits of this type of comprehensive design, the average primary care researcher would have been expected to drop all their preconceived training and intellectual biases and jump enthusiastically, head first, into the mixed methods pond. How could anyone hold back from an approach that is not only uniquely applicable to the complexity of primary care research, but also offers powerful potential for instrument design, triangulation, data transformation, reflection, and maybe even transcendence? The promise of combined generalizability and contextual interpretive relevance, offered by comprehensive designs, may even be the holy grail of research and seems too tempting to resist. Yet, as Creswell and colleagues have noted, and as a search of the literature confirms, relatively few primary care investigators have taken this path and few studies can be found.

So why isn’t everyone doing mixed methods investigations, providing added value through more comprehensive designs? The authors point to some of the reasons (labor intensity, funding support, etc), yet more explanations can be found in the written and unwritten traditions of our research discipline. What is clear is that despite the advantages of such endeavors having been being raised in primary care during the past 2 decades, starting with Blake⁹ and Stange and Zyzanski,¹⁰ numerous practical and theoretical roadblocks remain to combining or integrating qualitative and quantitative research.

Perhaps the most important impediment is that the practical and theoretical path for doing these types of studies is still relatively unmarked, making them appear inordinately complex to accomplish. Much of this complexity has to do with academic turf—the various methods needed for mixed studies are owned by different fields. Even when multimethod assessments or inquiries are preformed, they might not be publicized as such and might avoid detection in the usual literature searches. For example, researchers who use sequential strategies starting with qualitative research might not publish their qualitative results because they are relegated to pilot data status. Even when each methodological portion is noteworthy, there may be few incentives for publishing the mixed methods findings in a single article.

The differences in audiences, reviewers, and journals for qualitative- and quantitative-derived insights may result in separate but equal publishing of the data from an individual study. Add to this mix the relatively small numbers of individuals or groups with expertise in these academically disparate areas and the words of caution from some authorities,⁶ and the current picture becomes less surprising.

The theoretical roadblocks to combining qualitative and quantitative research may also be daunting. Some say (often out of earshot of their colleagues in other fields) that the differences in underlying assumptions, worldview, and epistemology make true integration difficult. The roots of the argument go back to the debates about the nature of “truth” and “knowing.” Qualitative data collection relies on a variety of iterative techniques to generate data, rather than more linear quantitative questionnaires or measurements. Qualitative research may also be defined by its analytical tools or styles,¹¹ which, although diverse, do not generally rely on statistics or quantification to arrive at conclusions. In contrast to reductionist quantitative research, its basis is antipositivistic, which means that insights are interpreted rather than discovered and that instead of searching for truths, the investigator seeks valid and rigorous meanings or interpretations. Qualitative research is generally inductive, beginning with observations of reality, formulating hypotheses and building theories. In opposition, quantitative investigations are generally deductive. They start with theories, construct hypotheses grounded in those theories, and then gather data to prove or disprove them. In this way, observations are often fit to prior ideas.

Despite all the roadblocks, there is the alluring possibility that family medicine researchers, given the transdisciplinary nature of our field, might be able to develop further the huge potential that can come from bringing together qualitative and quantitative methods in creative and integrated manners. Family medicine researchers, perhaps following the incredible example provided by their clinical colleagues, can overcome the practical and theoretical concerns. Family medicine has successfully maintained a generalist model in the face of increasing specialization and efforts to break the body, the patient, and the family into infinite, reducible units. Our research efforts can lead the charge of integration and comprehensiveness, as well.

I cannot be more pleased that the new Annals has decided to include two mixed methods studies for publication in one of its premier issues. I hope this is a signal of interest in this underserved area. As Creswell et al suggest, “The discussion initiated here holds promise for designing rigorous proposals for funding and clarifies the complex designs inherent in this form of inquiry.” Stange and Zyzanski’s words from 1989 still ring true and may finally be heard:

If the only tool researchers have is a hammer, they tend to see every problem as a nail. An appreciation of both quantitative and qualitative approaches can enhance a researcher’s ability to answer complex questions in a manner which is efficient, internally valid, and generalizable.¹⁰

Perhaps these two articles can be seen together as a call to action. So pick up your hammer, your loom, your screwdriver, your statistical software, and your tape recorder and weave collaborative, mixed method, theory-based primary care research into your future!