Making Meaning Matter: an Editor’s Perspective on Publishing Qualitative Research

Abstract

The term qualitative research refers to a family of primarily non-numeric methods for describing, analyzing, and interpreting the lived experiences of people in their day to day lives. Originally developed to study social problems such as poverty, juvenile delinquency, and race relations, qualitative research methods have been used in the health sciences since the 1960s to better understand the socialization of medical professionals and the culture of medical education and practice. More recently, qualitative research has been employed in health services research to address and improve the quality and safety of care. While quantitative researchers generally ask “what” or “how many” questions, qualitative researchers generally ask, “why” or “how”? Publishing qualitative research comes with a number of challenges, among them, manuscript length, unfamiliarity of reviewers with qualitative traditions, and sample sizes that, by design, are difficult to generalize from. In addition, while there is general agreement about the quality of evidence and types of research designs used in quantitative studies, the same is not yet the case for qualitative and mixed methods research although a variety of useful guidelines have recently appeared. From the perspective of journal editors, we raise and offer guidance on three important questions: (1) Is the study under review suitable for this journal? (2) What is the rationale for using qualitative methods to carry out the research? (3) What are editors/reviewers looking for in a qualitative submission? In unpacking the third question, we describe common strategies editors use and challenges that we have encountered in the abstract, background, methods, results, discussion, and conclusions sections of qualitative submissions we and our colleagues have reviewed.

“Not everything that counts can be counted and not everything that can be counted counts”

Attributed to Albert Einstein

Background

Qualitative research is a term social scientists use to describe an array of primarily non-numeric methods for describing, analyzing, and interpreting the lived experiences of people in their day to day lives. First used systematically by sociologists and anthropologists in the early twentieth century to understand social relationships and problems such as poverty, juvenile delinquency, and race relations, they have evolved in sophistication and use. In the early 1960s, qualitative research was employed in medical and nursing education to study the socialization process of becoming a professional.¹ Since then, qualitative research in medicine and nursing has expanded steadily from fewer than 100 articles in 1960 to more than 110,000 in 2021.

In addition to health professions education, qualitative approaches have increasingly been used in health services research to study such topics as quality and safety in the delivery of care.² In 2001, the Institute of Medicine in its highly cited monograph, Crossing the Quality Chasm, recommended employing multiple designs for health research including quantitative alone, qualitative alone, and mixed methods which include a sequence of quantitative followed by qualitative (explanatory sequential design), qualitative followed by quantitative (hypothesis generating), and simultaneous use of qualitative and quantitative (synergistic) methods.^3–5 With the rapid rise of published qualitative studies have come various guidelines for conducting and reporting research results.^6–9 As useful as these guidelines are, there are aspects of the review process that we believe will be of interest to readers and are worth sharing.

Some (Hopefully) Helpful Tips on Preparing for and Surviving the Manuscript Review Process

Translating any type of research findings into a publishable manuscript is no easy task. Based on our experience as qualitative researchers and editors over several decades, we offer some practical advice about how to successfully prepare manuscripts and anticipate questions that are likely to be raised during the peer review process. Researchers trying to publish qualitative findings may face some unique challenges, given some reviewers’ lack of familiarity with the breadth of qualitative methods, standards by which to evaluate,¹⁰ and conventional manuscript styles and length.

Three Questions Editors Typically Ask Themselves when Reviewing a Qualitative Manuscript

Question 1: Is the Study Suitable for This Journal?

Some medical journals simply do not publish qualitative research or do so only infrequently. For example, several years ago, a controversy over publishing qualitative research occurred in the British Medical Journal (BMJ). In an open letter to the editor, 77 prominent researchers claimed that the journal appeared to have developed a policy of rejecting qualitative research on the grounds that such studies were “low priority,” “unlikely to be highly cited,” “lacking practical value,” and/or “not of interest to our readers.”¹¹ The journal defended its actions on the grounds that it was simply identifying an area of emphasis and stuck to its policy of limiting publication of qualitative studies, at least in the parent journal¹² . By contrast, the Journal of General Internal Medicine welcomes qualitative research, has published 191 articles using qualitative methods in the last 3 years, and has recognized the need for additional space to report qualitative narrative results by accepting manuscripts up to 4000 words as compared to non-qualitative research which is limited to 3000 words.

Before deciding on a journal, it may be wise to look on its website to get a sense of the types of research questions and methodologies that are heavily represented in its pages. This is not to say that you should always avoid submitting to journals that have a limited history of publishing qualitative research. If the paper breaks new ground on an important topic, it may be considered by top-tier journals. For instance, William Branch and colleagues published a qualitative study of critical incidents related to the professional development of medical students that appeared in the New England Journal of Medicine, surely one of the most quantitatively oriented medical journals in the world.¹³ The worst that can happen in submitting to a top-tier journal is that the paper will receive a quick rejection (usually within 24–48 h) and in some cases an editor will make suggestions of other journals to try.

Question 2: What Is the Rationale for Using Qualitative Methods to Carry Out the Research?

In simplified terms, qualitative research focuses on how people make meaning and at a higher level of abstraction how their lived experience can be categorized into themes and patterns. It is most useful when little is known about a topic or question and where the aim is to discover new knowledge or deepen our understanding of current knowledge. In that regard, qualitative research is not designed to be predictive or generalizable. T.S. Elliot put it well when he wrote, “We shall not cease from exploration, and the end of all our exploring will be to arrive where we started and know the place for the first time.”¹⁴ In other words, qualitative research is designed to illuminate and increase our appreciation of the particulars, of a particular situation or phenomenon, not control, predict, or draw generalizations from it. Quantitative research asks the question, “What?” while qualitative research asks the question “Why?”.

The importance of recognizing and embracing the built in limitations of qualitative research cannot be overstated. A common problem in qualitative research reporting is trying to provide quantitative answers from qualitative approaches. In reviewing hundreds of qualitative research papers for the Journal of General Internal Medicine and elsewhere, we often encounter statements of the following sort, “Based on our results (of a sample of 12 first year medical students from a single institution) we believe that it is beyond time for radical reform in all training programs from undergraduate through continuing medical education.” While this may be a true statement, the conclusion that that the results apply to all of medical education is unwarranted. A better and more accurate statement might be, “Based on the results from our inquiry into the lived experiences of a small group of first year medical students, there are several hypotheses that we intend to pursue in larger randomized samples from multiple institutions.”

In addition to the question of choosing among quantitative, qualitative, and mixed methods for a study, it is important to recognize that qualitative research represents a family of approaches with different philosophical roots and traditions. (See Table 1 for a description of the five major qualitative research traditions and the types of questions underlying them)^15,16 .

Table 1.

Five Questions to Select a Qualitative Design*

Questions to ask to discover preferred approach	Associated tradition
1. If I could discover the meaning of one person’s lived experience, I would ask an individual) about____	Biography
2. If I could discover the shared lived experiences of one quality or phenomenon in others, I would want to know about____	Phenomenology
3. If I could experience a different culture by living/ observing it, I would choose to experience____	Ethnography
4. If I could discover what actually occurred and was experienced in a single lived event, that event would be ___	Case study
5. If I could discover a theory for a single phenomenon of living as shared by others, I would choose to discover the theory of ____	Grounded theory

^*From, McCaslin M, Scott KW. The Five-Question Method For Framing A Qualitative Research Study. Qual Rep 2003;8 https://doi.org/10.46743/2160-3715/2003.1880

As such, not all qualitative approaches are equally rigorous and it is possible that editors and reviewers may have their own implicit or explicit biases based on their own training and research. In ways similar to quantitative research, where meta-analyses and randomized double-blind control trials are considered the highest form of evidence,¹⁷ qualitative researchers recognize that there are inherent limitations in some approaches. Research based on focus group data, for example, is subject to “demand characteristics” of the situation¹⁸ and peer pressure.¹⁹ By comparison, audio- and video-based transcript analysis of doctor/patient recordings incorporates a greater degree of reproducibility and data trustworthiness although it too is subject to its own limitations²⁰ . Although the hierarchy for evidence in qualitative studies is not as advanced as it is for quantitative research, there are steps being taken in that direction (in a BMJ journal, no less)²¹ . Table 2 presents a comparison of the advantages and disadvantages of the most frequently used qualitative approaches.

Table 2.

Comparison of Qualitative Assessment Methods

	Advantages	Disadvantages
Audio visual records	• Reproducible, low cost	• Storage and coding can be time and labor intensive
	• Allows direct observation and coding of actual events	• Analysis is often painstaking and results may be delayed
	• Useful for teaching, coaching, and feedback	• Easy to become overwhelmed by too much data
Individuals and group interviews	• Relatively inexpensive	• Real-time observation may mask important cues and clues
	• Allows for exploration of attitudes and beliefs	• Subject to social desirability and observer bias
	• Allows for rapid results	• Little guidance for conducting systematic analysis
Self-report	• Relatively easy to obtain	• Can raise issues of accuracy
	• Access to inner experience	• Difficult to independently confirm observations
	• Encourages self-awareness and self-disclosure as part of the analytic process	• Subject to conflicting points of view by different observers
Checklists	• Good for discerning patterns of behavior	• Results may be bound to artificial checklist categories
	• Easy to establish norms or performance guidelines	• Potential for oversimplifying issues
	• Results are easily quantified	• Results may be contextually insensitive
	• Good for group comparisons	• Difficult to use for giving specific feedback

Question 3: What Are Editors/Reviewers Looking for in a Qualitative Submission?

Abstract

The abstract is likely the first thing an editor will look at in the review process and it is probably the most important gateway into the study. Abstracts, like movie trailers, inform the reader of the highlights of the study and must do so in an accurate and compelling way. The abstract is often the last thing authors think about having reached a state of complete exhaustion wrangling with the body of the paper. For better or worse, the quality of the abstract will likely determine whether the paper gets sent out for peer review or is rejected out of hand. Readers often decide whether or not to read the full paper based on the abstract. Given these constraints, the abstract should receive special attention using “fresh eyes,” which means imagining that the reader is a total stranger encountering your work for the first time. We often use the “granny” test to assess abstracts. After reading the abstract could you describe to your grandmother or grandfather what the study is about, the methods that were used, the findings, and the implications for clinical care. While this might not disqualify a paper from being reviewed, simplicity, clarity, and conciseness have been found to be favorable qualities to have in an abstract (and the body of the paper as well)²² .

Some journals require structured abstracts that can pose challenges for reporting qualitative work. For example, since most qualitative research is based on description and hypothesis generation, a structured abstract based on experimental or quasi-experimental methods may be difficult to navigate with qualitative data. Likewise, unstructured abstracts run the risk of losing the reader in a wealth of detail that may make it hard to discern the logic and sequence of steps used in conducting the study.

Background

Editors expect a concise description of the state of knowledge about the topic under study especially focusing on gaps in the literature that will be addressed by your research. Beyond the abstract, it is important for the paper to add sufficient detail about the literature to convince the reader/reviewer that you know and understand the current state of the science. It is not necessary to be exhaustive in the background section especially for research on well-known topics. Rather, for completeness, the background section should tell the story of why your research is positioned to address a known gap in the literature or if little to no literature exists, why  your research is groundbreaking. A helpful approach to thinking about the background and literature is to do a narrative review²³ that creates a timeline and arc that describes the evolution of research on the topic of interest and how this has led to the need for the current study. The review can provide the resources you will need to describe the issue, what is known about the issue to date, and current gaps in knowledge (that your study will fill).

Methods

Many editors, ourselves included, move directly from the abstract to reviewing the methods section of the paper. One important question editors ask is whether the methods are appropriate to the topic under study. The next question is whether the methods are described in enough detail so that the study could independently be replicated by another researcher. For qualitative research, it is very important to make the reader understand exactly what you did in your study. If the answer to either of these questions is no, it will often result in questions about the soundness of the science that informs the study. For instance, if the authors in the methods section assert, “We used grounded theory to conduct our analyses,” without explaining what grounded theory is, why it was appropriate for the question being asked, how it was applied, and what safeguards were in place to ensure the trustworthiness of the analysis, it would likely be judged by an editor to be insufficiently detailed and might be treated as a fatal flaw. The analogy in quantitative research would be reviewing the methods section in which the investigators assert, “We used standard statistics to do our analyses,” clearly failing to include details of the types of statistics and a rationale for why they were chosen.

Results

One of the most challenging aspects of reporting qualitative results is striking a balance between being too detailed and not detailed enough. It is common, especially for researchers who are new to qualitative research, to categorize many individual statements as potential themes. Moving up the ladder of abstraction and reducing statements to provisional and final themes is both challenging and sometimes results in a loss of meaning. A study of the hidden curriculum of medical education might start out with 64 different types of influences on learning reported by medical students. Reducing them to three primary categories may be efficient for reporting purposes but may also lose the subtlety and texture of experiences that make up the phenomenon and, in the process, lose the reader’s or editor’s interest. Fortunately, there are excellent qualitative data reduction and presentation techniques available²⁴ .

A common error that we often encounter in reviewing qualitative research studies is a kind of circular reasoning regarding themes. For example, the following assertion might appear in the results section of an interview study, “We found that the frequency of negative experiences (such as symptoms of cyanide poisoning) from eating raw lima beans emerged as a theme in our analysis.” While this is certainly a possibility if the topic of eating raw lima beans spontaneously arose during the interviews, it would be incorrect to call it “emergent” if the appendix containing the interview guide included a question such as “What have your negative experiences been with eating raw lima beans?” While not necessarily a fatal flaw, it does reflect a potential misunderstanding of what is understood in the qualitative research community by the term “emergent theme.”

Discussion

For editors and eventual readers, the discussion section builds upon the background, methods, and results, to further the narrative arc of the story that is being told. It is the “so what” of the manuscript and its role is to provide an interpretation of the results and put them in context. Too often, in both quantitative and qualitative work, authors simply recapitulate the main findings of the study without adding any interpretation of the findings and their implications. We encourage authors to take the next step in their thinking and come up with a synthesis of where the field is and how the study advances and creates new knowledge. New does not always mean the first of its kind, although if it is a first, and it is well done, chances of publication usually go up. Confirmation of what others have found can also be a source of new knowledge and this is especially true in qualitative research.

Conclusion

If you think about a research paper in literary terms, the conclusion brings together the characters who are introduced at the beginning of the story (abstract), their relationships, (background), how they fare over time (methods), how they resolve or fail to resolve challenges (discussion), and how they leave the story (conclusion). In a well-plotted novel, the elements all work together seamlessly. The same holds for qualitative research. Editors and readers are interested in how all the elements of the study cohere to tell a compelling story that advances knowledge and understanding. As mentioned earlier, qualitative research is, by design, non-generalizable and it is important not to overstate the conclusions in terms of their relationship to policy and practice.

One other point to consider is whether the conclusion offers credible suggestions about follow-up research in the topic area. Doing so conveys a sense that the manuscript being submitted is not just a “one off” but is part of a line of research in which the next steps are logical and appropriate.

Closing Comment

We end where we began by recognizing that qualitative approaches have a critical role to play in better understanding and improving health, illness, and health care services. As editors, we welcome your comments, suggestions, and criticisms in hopes of continuing a stimulating and informative dialog about qualitative research and making meaning memorable.

Declarations

Conflict of Interest

The authors have no conflicts of interest to declare.