Qualitative Research in Medicine and Health Care

Qualitative research is becoming more prominent in medicine and health care. Recently, editorials advocating a larger role for qualitative research as a way to address both “clinical” and “biopsychosocial” phenomena have appeared in, for example, the Journal of General Internal Medicine,¹ the Annals of Internal Medicine,² and the British Medical Journal.³ Increasing numbers of reports from qualitative studies have begun to appear in these and other prominent journals.

Physicians and clinical and health services researchers, however, may be unfamiliar with qualitative research and unsure how it relates to their interests. The evidence-based medicine movement has taught that clinical practice and health policy should be based on critical review of the best available evidence.⁴ To appreciate the evidence supplied by qualitative research, one must be able to address the general question: What are the goals of qualitative research? Principles of evidence-based medicine and behavioral sciences that further suggest that to review any study critically, one must be able to answer several more specific questions: Is the design of the study appropriate to its goals? How valid are its results? How well do they apply to one's practice or circumstances?

Unfortunately, the discussion of qualitative research in the medical literature provides no easy answers to these questions. Rather, it reveals considerable controversy, if not confusion about how qualitative research can address either “clinical” or “biopsychosocial” questions. Thus, in this article we try to raise and discuss these questions based on our reading of some of the relevant literature. We hope that the resulting dialogue will contribute to the clarification of their answers, and help physicians and clinical and health services researchers to understand some of the limitations and potential contributions of qualitative research better.

WHAT ARE THE GOALS?

Addressing Clinical Questions

There is controversy about whether qualitative research can address and answer not only clinical but also biopsychosocial research questions. In their Journal of General Internal Medicine editorial, Berkwits and Aronowitz argued for the broad clinical applicability of qualitative research: “qualitative primary care research aims to identify the essential component parts of clinical phenomena,” and is “especially suited to areas that have both social and clinical dimensions.”¹ They suggested, for example, that qualitative research can identify “essential and fundamental” cardiovascular risk factors, leading to “a better understanding of which aspects of individual and group behavior and physiology ‘hang together’ and which are amenable to population interventions, individual behavior change, or physiologic manipulation.” However, they did not explain how qualitative research could be used to answer specific clinical questions about cardiovascular risk, or more generally, how it could answer other traditional clinical questions about etiology, diagnosis, prognosis, or therapy. They also did not explain how qualitative research could answer clinical questions that refer to essentially quantitative data, e.g., blood pressure measurement or diagnostic test results.

In contrast, other advocates of qualitative research disagree and suggest its scope is much narrower and does not include traditionally “clinical” concerns. For example, Hull proposed that qualitative research deals with social, as opposed to clinical phenomena.⁵ Helman noted qualitative researchers collect only data about “what people say,”“what people actually do,”“what people actually believe,” and “the context in which people answer questions,” but, by inference, not data about, for example, cardiac physiology.⁶

To bolster their arguments about the clinical usefulness of qualitative research, Berkwits and Aronowitz went on to criticize strongly the way in which quantitative research answers clinical questions. However, the justification of these criticisms is not clear.

Berkwits and Aronowitz asserted that epidemiologic studies have often overlooked “essential or fundamental” risk factors because they only recognized variables that were independently predictive in multivariate regression analysis. For example, they stated “obesity may fail as an independent risk factor for coronary heart disease (CHD) in regression analysis, but it may nevertheless be an essential factor.” This statement may be true, but it should not be used to indict quantitative research. In fact, it is well known that failure to achieve significance as an independent predictor of risk in a multivariate model does not necessarily mean the variable has no clinical importance. Reasons a variable may fail to achieve such significance include insufficient sample size and correlation with some other variable in the model.⁷ Furthermore, epidemiologic researchers have not ruled out obesity as a risk factor for CHD, but have been concerned with possible confounding by both the causes of obesity (e.g., nutritional patterns and energy expenditures) and its consequences (e.g., other risk factors like diabetes).⁸

Berkwits and Aronowitz also implied that quantitative studies often cannot answer “appropriate and worthwhile” questions because they focus on what is measurable, and worse, may “reduce many dimensions of clinical experience into a single and misleading numerical dimension.” For example, they suggested that quantitative studies cannot address how tobacco advertising or price subsidies affect peoples' use of tobacco, because such factors cannot be entered into “cardiovascular risk factor equations,” which “require that putative risk factors operate at the level of the individual.” This argument is hard to follow. In fact, not all quantitative research involves the use of multivariate models. Moreover, some multivariate models, like the generalized estimating equation, can analyze correlated data about individuals who are clustered into groups.⁹ Furthermore, quantitative researchers have not ignored questions about why people smoke. Several studies have addressed, for example, the effect of advertising on smoking behavior.^10–14 In any case, it is not clear how these criticisms of quantitative research can advance an argument that qualitative research is an appropriate method to answer clinical questions.

In conclusion, advocates of qualitative research need to explain convincingly how it could be applicable to traditionally clinical questions. At present, it remains unclear whether qualitative research can address such questions.

Addressing Biopsychosocial Questions

There is also controversy about whether qualitative research can answer biopsychosocial research questions. A number of authors believe that qualitative research can answer some kinds of research questions, often referred to as “biopsychosocial,” which they believe cannot be addressed at all by quantitative research. McWhinney argued that “we use qualitative methods because they are the only ones that can answer certain questions.”¹⁵ More specifically, Jones stated that “to answer questions about the reasons for variations in adherence . . . requires a qualitative approach.”³ Pope and Mays wrote that “qualitative research can explore complex phenomena not amenable to quantitative research” and it “can reach aspects of complex behaviours, attitudes, and interactions which quantitative methods cannot. As a result, it has been extremely useful for examining clinical decision making.”¹⁶ Gilchrist and Engel wrote “qualitative research answers questions for clinicians that quantitative research cannot. These are questions about individuals' motivations, perceptions, expectations, and meaning.”¹⁷

These authors do not clearly justify why they consider the scope of quantitative research to be so limited. It is easy to demonstrate, however, that quantitative researchers have tackled many of the areas beyond the limits they postulate. Armstrong also has noted that “proponents of qualitative research sometimes take on a missionary zeal in promoting its virtues over more traditional quantitative research” and “they sometimes minimize the extent to which the quantitative tradition has managed to address some key problems.”¹⁸ Jones' assertion is contradicted by numerous quantitative studies on why people's compliance with treatment regimens may vary.^{19, 20} Gilchrist and Engels' statement disregards a prodigious amount of scholarly work. The quantitative social and behavioral sciences have a long tradition of addressing questions about motivations, perceptions, expectations, meaning, cognition, and social interactions. Experimental studies of motivation, social interaction, the cognitive underpinnings of interpersonal phenomena, and more recently, even affect and emotion, have been in the mainstream of experimental social psychology for decades. By the late 1960s and early 1970s, methodologic efforts in experimental social psychology included many that focused on accurately representing in natural settings even such complex phenomena as helping and altruism.^{21, 22} Webbs'Unobtrusive Measures(1966) offered insightful quantitative methods for assessing and observing behavior systematically even in nonexperimental studies.²³ These kinds of efforts show that although it may take innovation, creativity, time and effort to measure complex phenomena quantitatively, this does not mean it cannot be done.

There is also a substantial body of relevant quantitative work in the context of medicine and health care. To cite only a few of many possible examples, in 1986 Eisenberg summarized multiple quantitative studies of physicians' decisions which addressed their motivations,²⁴ and more specifically in 1991 Bradley similarly summarized many quantitative studies of what motivates physicians' drug-prescribing decisions.²⁵ Numerous quantitative studies of specific influences on physicians' decision making have appeared since. These have addressed diverse aspects of decision making for real patients in complex environments: for example, how clinical differences in patient populations can lead to practice variation ^{26, 27}; how physicians decide about resuscitation for patients with AIDS compared with those with other diseases with different meanings but similar prognoses ²⁸; and how physicians' risk preferences ²⁹ and anticipated regret ³⁰ affect their decisions, to name only a few. Meanwhile, several studies of patients' utilities have addressed the meanings they attach to health states. A few recent examples include work by Tsevat and colleagues,³¹ Hult and colleagues,³² and Llewellyn-Thomas and colleagues.³³ Further, researchers in the field of health psychology have conducted quantitative, empirical studies investigating real-life health-related phenomena ranging from coping with traumatic events such as spinal-cord injury or rape to measures self esteem and subjective well-being.^{34, 35}

We urge those who suggest that qualitative research is the only method that can answer certain kinds of research questions to reassess their conclusions about the limitations of quantitative research. For now, it remains unclear whether qualitative research can answer questions quantitative research cannot answer. In any case, as we stated above, it is not clear how unsupported criticisms of quantitative research advance arguments about the usefulness of qualitative research.

In this context it is also disturbing that some qualitative studies have attempted to answer biopsychosocial questions with data that seems insufficient to do so. For example, Rich and Stone,³⁶ Kravitz and colleagues,³⁷ and Allery and colleagues ³⁸ made statements about causation based on observations of case series without control groups. These investigators seemed to assume that some feature they found in persons who displayed the phenomenon of interest was not found in persons who did not display this phenomenon, but made this assumption without actually acquiring any data about those who did not display it. For example, Kravitz and colleagues performed qualitative interviews of 90 patients who felt their expectations were not met by office visits. The patients reported having multiple concerns, e.g., about the intensity, impact on function, duration, or seriousness of their symptoms, and about their vulnerabilities to illness. The study did not include patients who had no clear expectations for their visits or whose expectations were met. It is quite possible, however, that such patients would also have had similar concerns. Without a comparison group that did not have unmet expectations, one cannot conclude that the multiple concerns lead to or caused unmet expectations. The authors did conclude, however, that the concerns listed above were the causes of their patients' unmet expectations: “our study . . . elucidates the multifactorial ontogenesis of these expectations.” (Also, see letters critiquing the studies by Rich and Stone,³⁹ and Allery and colleagues.⁴⁰)

Generating Research Questions and Hypotheses

There is little controversy about the usefulness of qualitative research to describe new phenomena and generate research questions. This ability of qualitative research to raise interesting hypotheses about some aspects of medicine was demonstrated by recent studies. For example, Als suggested doctors may use desktop computers as “magic boxes,” sources of authoritative clinical information that goes beyond the data the computer is capable of displaying.⁴¹ Generation of hypotheses is an important and necessary part of the research process; however, hypotheses also must be tested.

HOW VALID ARE THE RESULTS?

Readers of reports of qualitative research studies in medicine or health care must be able to evaluate the evidence the studies supply critically. This is true for studies that simply describe a phenomenon, as well as for those that try to test specific hypotheses about its causes and outcomes, especially for phenomena that involve unobservable mental processes like beliefs, feelings, interpretations, and motives. To evaluate a study critically, readers must be able to answer several questions: Is the design of the study appropriate to its goals? How valid are its results? How well do they apply to one's practice or circumstances? Unfortunately, the clinician or health services researcher looking for guidance about the critical review of qualitative studies whether about clinical or biopsychosocial issues may again find considerable controversy and confusion.

Existing Methodologic Standards

A number of articles have suggested methodologic standards for qualitative research or guidelines about critical review of qualitative studies. Inui and Frankel proposed terse standards for “trustworthiness,” which is analogous to validity.⁴² Reid proposed brief guidelines to help physicians “assess the quality of qualitative work,”⁴³ as did Shmerling and colleagues,⁴⁴ and Ryan and Denz-Penhey.⁴⁵ Mays and Pope proposed somewhat more detailed standards for validity.⁴⁶ Elder and Miller also proposed somewhat more detailed standards in an article meant to help “physicians learn how to read and assess qualitative studies.”⁴⁷ Other efforts in this area were made by Britten and colleagues,⁴⁸ Goering and Streiner,⁴⁹ Kuzel and colleagues,⁵⁰ and Miller and Crabtree.⁵¹ Articles in the British Medical Journal series addressed some methodologic issues relevant to particular modalities of qualitative research: observational studies,⁵² interviews,⁵³ focus groups,⁵⁴ consensus methods for expert opinion,⁵⁵ and case study evaluation.⁵⁶ Frasier and colleagues also provided methodologic standards for using focus groups to evaluate the medical curriculum.⁵⁷

These articles successfully introduced the terminology of qualitative research, described different kinds of qualitative studies, and listed methodologic options available to qualitative researchers. Many of them stressed the importance of clearly describing the research effort and justifying the methodologic options chosen. Unfortunately, none of these articles provided enough detail to create a usable operational framework for the critical review of qualitative research studies.

To review a qualitative research study critically, one must be able to evaluate the appropriateness of the chosen methodologic option to a particular study's goals or situation and the problems that might result from using that option; and one must be able to assess whether the option was executed satisfactorily and the problems that might result from the way it was executed. For example, there are numerous sampling methods used in qualitative research, e.g., using key informants, purposeful sampling, and homogeneous sampling.⁴⁷ Suppose a study used key informants. How could one determine the appropriateness of this sampling method to its goals and situation? How could one determine what problems might result from using this method? How could one determine whether the right key informants were picked and what problems might result from the choice of key informants in the study? None of the articles we read could help us answer such specific questions. For example, the relevant statement by Britten and colleagues is reasonable but not very helpful: “There is no right or wrong way to sample in qualitative research. The important thing is that the sampling decisions should be clearly thought out and appropriate for the research question, not dictated by convenience.”⁴⁸ Without guidance about the appropriateness of specific methods to a particular study's goals and situation, however, the reader is left hanging. How the reader should evaluate the methods chosen from the numerous available options for data collection and analysis is similarly obscure. Elder's statement, “researchers need to choose the method best suited for their study and adequately describe [it] . . . as well as why it was chosen,”⁴⁷ is approrpiate but underlines the need for some consensus about the suitability of particular methods to particular kinds of applications.

Confusion over vague methodologic standards may be why Armstrong chided qualitative researchers for not practicing what they preach: “Mays and Pope . . . recommend that independent corroboration for the identification of themes in a transcript is the mark of rigorous research. Yet, ironically such ‘rigour’ has not been used by qualitative methodologists themselves—never mind the poor neophyte reading the BMJ for guidance. . . .”¹⁸

Advocates of qualitative research must develop operationally usable guidelines to help readers critically review qualitative studies. This will not be an easy task. Further it must not ignore the century-long development of scientific psychology and behavioral science, which began in Wilhelm Wurdt's laboratory with dedication to images as the data and introspection as the method of inquiry (a qualitative data collection effort) but evolved into quantitative, objective, verifiable methods, because of problems of reliability and validity of even trained experts' images and introspections.⁵⁹ Until they develop such guidelines, it will remain very difficult to review such studies critically, and to determine the contribution of the evidence they may provide.

Conceptual Controversies

The development of rigorous methodologic standards for qualitative research may have been hindered by another controversy within the field of qualitative research itself, one about its conceptual underpinnings. This controversy calls into question whether rigorous standards for qualitative research should exist at all.

Armstrong noted that “many qualitative researchers would argue against the appropriateness of assessing reliability, claiming that a ‘multiple reality’ will be expected to produce multiple interpretations.”¹⁸ In Clinical Epidemiology, Fletcher and colleagues defined validity in the following terms: “a clinical observation is valid if it corresponds to the true state of the phenomenon being measured.”⁵⁸ If reality is multiple, in the sense that people can hold multiple levels of meaning about a real thing simultaneously, then one must choose an appropriate level to address for a particular purpose, and the validity of the observation of that phenomenon depends on that choice.

Yet, other advocates of qualitative research have asserted that the phenomena of interest to them have no true states apart from their observers. For example, Peacock (quoted by Helman), stated that no “fact” has objective reality, but is “a construction reflecting both the perspective of perceivers and the world that they perceive.”⁶ Griffiths, summarizing a British conference on qualitative research, wrote that “there is a lot of feeling behind questions we ask, they are not innocent, they are knowing and are not so much questions as statements of a point of view. All facts, all data are values.”⁶⁰ She later stated, “a fact is a precept viewed through a frame of reference.”

It is tempting to assume that these statements are reasonable assertions about the different ways humans perceive, and think and communicate about the “real world,” dressed up, perhaps, in rhetorical excess. As noted above, many have stressed the application of qualitative research to how people perceive, interpret, and communicate about the “real world.” Obviously, humans work within cognitive and perceptual limits. Not everything one believes to be true is true. It is hard for anyone to communicate flawlessly what he or she believes to be true in all instances. Furthermore, although different people may attach different meanings to the same fact, they may also share perceptions and meanings about that fact. For example, a glass ashtray may evoke different meanings to smokers and nonsmokers, and yet it may also evoke some common meanings in these different groups as well.

Further, there is reason to suspect that some advocates of qualitative research challenge the notion that the observer can ever be separated from the observed, for example, that one's reaction to an ashtray can be separated from the reality of a piece of glass. This is based on an argument that the conceptual foundations of qualitative research are separate from and even in conflict with the foundations of what most physicians may consider science. Henwood and Pidgeon argued that the ongoing debate within the social sciences between quantitative and qualitative research “has been anchored within two apparently opposed epistemological positions. The two poles are known variously as ‘experimental,’‘hypothetico-deductive’ or ‘positivist’ and the ‘naturalistic’ or ‘interpretative’ approaches respectively.”⁶¹ Inui echoed this notion, describing two “permanently important world views.” One, contrasting with the traditional “positivist” view that posits an objective reality, emphasizes a “subjective reality that is characterized by complexity, apparently infinite variation, and incommensurability without perturbation. The laws of cause and effect operate, but always within a unique set of circumstances determined by multiple factors.”² Gilchrist and Engel also argued that the two world views are in opposition. They noted the “naturalistic” world view holds that “what is being studied is inseparable from the scientist, who devises mental constructs of his/her experiences with it as a means of characterizing his/her understanding of its properties and behavior.”¹⁷

The “naturalistic” world view thus rejects the notion that reality is ever external to its human observer, or as Henwood and Pidgeon wrote, it “challenges the dualistic distinction between knower and known.”⁶¹ Those who hold this world view have scoffed at “positivist” social scientists' attempts to be objective: “what the parochial view in the social, behavioral, and service sciences has touted as ‘science’ in historical and practical myth. . . .”⁶² They reject the notion that scientists studying social phenomena should provide general rules that “can explain phenomena in retrospect . . . [and] permit us to predict what will happen in the future.”⁶³ Finally, they believe that the traditional notions of methodologic rigor, “the classical canons of reliability, validity, and objectivity,”⁶¹ are irrelevant to their kind of qualitative research. So Henwood and Pidgeon stated, “there is no easy resolution to the problem of judging the adequacy of a particular piece of research. A powerful case can be made for methodological anarchy . . . !”⁶¹

We are puzzled and troubled by these arguments flowing from the “naturalistic” world view. How can one evaluate arguments made by people who deny the existence of truth? Advocates of qualitative research will need to clarify the nature of the reality they seek to study, and reconcile it with the fundamental goal of science, to discover the truth, before they can satisfactorily develop methodologic standards for their work. Until they do so, we reiterate the need for caution about making use of the evidence that may be supplied by qualitative research.

Measurement Bias

Understanding the conflict between what some call the “positivist” and the “naturalistic” world views may help explain the curious controversy among qualitative research advocates about measurement bias. In medical and health services research, one usually thinks of methodologic bias as something to avoid. For example, in Clinical Epidemiology, Fletcher and colleagues wrote “most of this book is about how to recognize, avoid, or minimize bias.”⁵⁸ Many qualitative researchers would agree. Relatively unstructured data collection techniques and researchers' frequent immersion in the day-to-day lives of their subjects make some qualitative studies especially susceptible to measurement bias. Britten and Fisher suggested that measurement bias should be a major concern in qualitative research.⁶⁴ Mays and Pope wrote about strategies to reduce it.⁴⁶

In contrast, other qualitative researchers conceded that measurement bias may greatly affect qualitative research, advocated full disclosure of biases, but seemed uninterested in minimizing them. For example, Charon and colleagues stated, “the observer is not considered to be detached and unbiased. Rather, the researcher's own values assume importance in the search for meaning of the data and must be made explicit in the analysis . . . Medical researchers who use these methods make their own philosophical and political positions clear when they report findings.”⁶⁵ Elder and Miller also were not explicitly concerned with minimizing bias: “the investigator, rather than trying to eliminate as many biases as possible using a rigid structure, actually becomes part of the research, and describes rather than eliminates known biases.”⁴⁷ Full disclosure of bias seems a worthy goal. However, it begs the question: When is there so much (disclosed) bias present as to render the study invalid for any purpose other than self-expression by the researchers?

Furthermore, Berkwits and Aronowitz make the remarkable assertion “‘bias’ is not something to be eliminated, but is a productive element, a foundation for formulating questions and understanding answers in the process of research.”¹ Henwood and Pidgeon also viewed the traditional goal of reducing bias as counterproductive, leading one “to eliminate idiosyncrasy and creativity, [whereas] criteria for judging the quality of generative [qualitative] research must recognize aspects of these personal characteristics in the search for theory that is relevant and good.”⁶¹ If bias is productive in qualitative research, somewhat facetiously we wonder when is there enough bias present to make a study worthwhile? Gehlbach, however, wrote: “authors who set forth the methods they use to guard against measurement error are more likely to gain our confidence.”⁶⁶ Authors who promote bias are likely to gain little confidence.

SUMMARY

Qualitative research is becoming more prominent in medicine. It is still not clear how it can address either clinical or biopsychosocial research questions. Methodologic standards and guidelines for qualitative research in medicine and health care remain too sketchy to help one evaluate a qualitative study critically. Alternatives for addressing complex real-life questions quantitatively exist. Until better guidelines for qualitative research become available, we urge caution about using evidence from qualitative studies. Developments of such standards and guidelines are perhaps being hindered by continuing controversies among advocates of qualitative research about whether truth exists independent of its observer, and whether bias should be eliminated, disclosed, or actively encouraged. These controversies undermine the credibility of qualitative research for clinical and health services research audiences.