Publishing Multimethod Research

THE POTENTIAL AND THE PROBLEM

An extraordinary surge in the methodological diversity of clinical research studies has occurred during the past 2 decades, with a recent dramatic rise in the scope and sophistication of mixed methods designs.^1,2 Mixed methods (also known as multimethod) research involves integrating quantitative and qualitative approaches to generating new knowledge³ and can involve either concurrent or sequential use of these 2 classes of methods to follow a line of inquiry.^4,5 Combining methods activates their complementary strengths and helps to overcome their discrete weaknesses.⁶ Increasingly, integrated mixed methods designs allow researchers to follow emerging questions, rather than limiting their research to questions that are amenable to a particular method.⁷ Multimethod research brings together numbers and narratives, description, hypothesis testing, hypothesis generation, and understanding of meaning and context to provide fuller discernment and greater transportability of the phenomenon under study.

In the past, proponents of quantitative and qualitative methods have been divided into separate camps with different skills and world views.⁸ The benefits of integrating these methods, however, are increasingly recognized and acted on in the medical and health care arena,^1,2,6 as they have been for a longer time in the ethnographic tradition from anthropology and the case study tradition from educational research.^7,9 Among many examples, a 1999 National Institutes of Health task force issued guidelines for rigorous qualitative and multimethod research.¹⁰ Two years ago, the (US) National Cancer Institute hosted a conference on mixed methods research that identified multimethod research as an important approach to solving some of the most intractable problems in cancer control research. An article by the Medical Research Council on the design and evaluation of complex interventions to improve health¹¹ identified the use of mixed methods as essential.

Even so, the dramatic advances in the scope and sophistication of conducting mixed methods research have not been met with parallel progress in ways of disseminating the results of mixed methods studies. From our point of view, a major dilemma is that the results of multimethod studies often are segregated in different publications that reach limited and often nonclinical audiences. For example, Wilson and colleagues¹² reported in JAMA on the ineffectiveness of nicotine gum in smoking cessation, while in Social Science and Medicine, Willms¹³ reported qualitative findings from the same study that the meaning patients attributed to their cigarettes was more influential in stopping smoking than either counseling or nicotine gum. Both articles have important messages, but the JAMA article does not reference the other, and they are published in journals with very different readers. Thus, different fields only come to know part of the research—reminiscent of the story of the 4 blind men each feeling a different part of the elephant and thus unable to develop a coherent idea of the whole.

SOLUTIONS

There are a number of options that authors and journals can use for integrating the publication of multi-method research, several of which are represented in this issue of Annals. We propose 5 approaches for this integration, including an additional opportunity to use publication to develop cross-disciplinary knowledge communities.

1. Publish quantitative and qualitative papers in separate journals, but with clear references and links to the other article(s). This option reflects the current trend toward publishing mixed methods research in separate journals that often represent different disciplines. We suggest an important addition, however. The article(s) based primarily on quantitative methods should prominently cite the article(s) using qualitative methods and vice versa. In addition, they should use their discussion sections to draw overarching inferences that emerge from interpreting the quantitative and qualitative findings in light of each other. This publication option is most appropriate when the different methods yield findings that are most relevant to disparate audiences served by different journals. When the quantitative and qualitative analyses are published at different times, this approach also requires the initial article to foreshadow the forthcoming one, and the subsequent article(s) to refer in some detail to the article(s) that came before.

As more journals move to free or open-access publication, cross-talk between journals becomes much easier for the reader. When the linked paper is published in an online journal, authors should be sure to cite the URL as well as the print citation so that readers of the companion article can find it easily. In the emerging era in which many journals are read online, it becomes easy to hot link references, making moving from one journal to another almost seamless. Obviously, charges for online access put a large wrinkle in accessibility for most readers.

The article by Rabago and colleagues in this issue¹⁴ portrays qualitative findings from a previously published clinical trial and is an example of this type of sequential, separate publication of quantitative and qualitative findings from the same study.

2. Publish concurrent or sequential quantitative and qualitative papers in the same journal. Sometimes the findings are most clearly and succinctly conveyed by portraying the quantitative and qualitative methods and results in separate manuscripts. When this is the case, submitting the manuscripts for review at the same time allows the journal to consider them together and potentially to publish them in the same issue. Sometimes, the editors can choose to commission an editorial to provide an additional perspective on the conjoint findings. In any case, the articles should refer to and reference each other, and the discussions should interpret the findings in light of the other article. The companion articles by Solberg¹⁵ and Hroscikoski¹⁶ and their colleagues in this issue are an example of this genre.

3. Publish an integrated single article that describes both methods and findings and draws overarching lessons, with or without appendices that provide study details. This option is the most straightforward way to make both qualitative and quantitative findings available to readers, but it requires considerable parsimony in portraying methods and findings. This approach allows the discussion of the paper to draw out the overarching lessons and enhances portrayal of cross-cutting results that involve both quantitative and qualitative data.

Publishing a single multimethod article may at times require flexibility by journals in expanding word length requirements, as well as work by authors to be succinct. In general, readers appreciate parsimony, particularly if they have the option to find more details easily if they are interested. An increasingly used strategy is to portray the main methods and findings in a tightly worded manuscript while providing additional detail in appendices. Often journals will publish appendices online to make them readily available to readers and save expensive print journal pages. Appendices are particularly good places to portray detailed quotations or other text data from qualitative research or supplementary tables from quantitative research. In addition, appendices can be used to show details of methods or materials that may be relevant only to readers with specialized interests.

The studies by Bennett,¹⁷ Sussman,¹⁸ Wittink,¹⁹ and their colleagues are examples of the portrayal of mixed methods findings in a single article that uses appendices to convey important additional detail.

4. Copublish separate qualitative and quantitative papers accompanied by a third paper that draws overarching lessons from analyses across the 2 methods. There are times when quantitative and qualitative methods and results are each sufficiently complex and informative that each deserves its own paper, but some of the most important lessons emerge from comparisons across the 2 sets of findings. In this situation, it makes sense to write the quantitative and qualitative findings in separate manuscripts, and also to conduct additional cross-cutting analyses that are conveyed in a separate paper. Publishing these 3 papers in the same issue of a journal provides readers with access to a line of inquiry that often tests a priori hypotheses but then moves beyond to portray exciting emergent findings. This option should be used only when the quantitative, qualitative, and cross-cutting analyses each are of sufficient impact to justify their own paper.

5. Develop an online discussion of readers and invited commentators to foster cross-disciplinary communities of knowledge. Inviting diverse individuals and groups to contribute to an online discussion can help foster a community of knowledge that is not limited by discipline, readership, or methodological leaning. The Annals TRACK online discussion is a good example in which authors and editors identify different constituencies potentially affected by the published research and invite representatives of these constituencies to submit comments.²⁰ Synthesizing these discussions (as the Annals does in its On TRACK feature)²⁰ provides an opportunity for others to learn from themes and new information identified in the discussions and to advance thinking about the application, interpretation, implications, and further research options that emerge from diverse readers interacting with published research.

We encourage the expansion of multimethod research and the creative and effective portrayal of the insights that result from the integration of highly complementary quantitative and qualitative approaches to generating new knowledge. We also encourage readers to share experiences and additional ideas by commenting on this article at http://www.AnnFamMed.org.