Studies employing qualitative methods are an important part of scholarship in a variety of fields and are becoming increasingly popular and accepted. This trend means that systematic standards for conducting and evaluating such research are also more important than ever. Despite a renewed focus on quality, few comprehensive guides for researchers exist. The fact that qualitative research is, to one extent or another, diverse and idiosyncratic only complicates matters further.
Margaret Roller and Paul Lavrakas tackle this daunting task in a hands-on, innovative, integrated methods text, Applied Qualitative Research Design: A Total Quality Framework Approach. They do so through the mechanism of their Total Quality Framework (TQF), a comprehensive and interrelated way of thinking that addresses the major threats that can undermine the accuracy and value of qualitative studies.
They want to put the “quality” back into qualitative research through a philosophical underpinning linked to corresponding sets of tools for guiding research practice. The book “provides researchers with a flexible way to focus on quality issues, examine the sources of variability and possible bias in their qualitative methods, and incorporate features into their designs that try to mitigate these effects and maximize quality outcomes” (9). This is really a “spiritual user’s manual” for qualitative research.
Drawing on their decades of practical and professional research experience, along with an extensive review of the existing literature, Roller and Lavrakas’s aim is to create a more systematic way of creating and evaluating quality in these types of studies. They appear to adapt some of the perspectives of traditional (not necessarily quantitative, but within that tradition) research designs. These include precepts like representativeness of the target population, construct validity, using and justifying appropriate analytic techniques, and applicability of a study to larger contexts or generalizability. Yet, at the same time, they recognize the important distinctions between qualitative and quantitative research and seek to ensure the value of qualitative approaches. In their admirable view, qualitative research does not mean less rigor, less validity, and more limited generalizability than quantitative research.
The framework provides principles and practices for researchers to apply throughout the research process so that they are more likely to gather high-quality data, come to a more complete and valid interpretation of their data, and generate and present useful outcomes. Another laudable goal is to provide consumers of the research, be they readers, clients, or reviewers, a toolkit for judging how well a study was done and the value of its findings.
The TQF has four interrelated components: credibility, analyzability, transparency, and usefulness. Each of these corresponds to a different stage of a research project. Credibility addresses the completeness and accuracy of the data gathered by the qualitative study.
Analyzability refers to the completeness and accuracy of the analyses and interpretations of the data. Transparency is concerned with the completeness and full disclosure of all aspects of the research in the documentation of the study. Usefulness has to do with the “ultimate goal of all qualitative research, which is the ability to ‘do something’ of value with the findings and recommendations of the study” (22).
One of the more notable emphases of TQF is the use of “thick description.” This is a practice of documentation of all aspects of the study—including the topic, rationale for the research, and especially details about how it was done and how its conclusions were drawn—thus laying an audit trail to help consumers of the study independently judge its merits on their own.
After an in-depth discussion of the framework, the book contains chapters on each of the main qualitative methods: in-depth interviews, focus groups, ethnography, content analysis, and case studies. The authors, in keeping with the integrated approach of their model, in parallel fashion apply each of the four elements of the framework to each technique and its particular requirements. This feature alone is a major strength of this work.
But each method chapter contains some additional valuable elements, such as skills researchers must bring to that particular technique, strengths and weaknesses of that approach, and ethical considerations. The authors also provide a case study at the end of each chapter that is an example of a research project utilizing the TQF standards for that specific method. If that weren’t enough, the final chapter focuses on how to use the TQF to assess the quality of qualitative studies done by others—or at least, grant or project proposals.
Readers of this journal will likely be most interested in those chapters dealing with interviewing, focus groups, and content analysis. The authors’ knowledge and experience are evident here, addressing issues such as interviewee recordkeeping and interpretation, the training of coders and discussion moderators, why and why not to use these approaches, and so forth. But all of the chapters give readers useful and straightforward advice on how to go about utilizing that particular technique.
The one exception was the chapter on case study approaches. It was less clear and harder to follow, perhaps because the authors argue that case study and narrative research essentially use multiple methods of the other kinds. Such a tactic made the chapter more complicated than the others. Granted, this is in part due to lack of clarity within the scholarly community on what constitutes a “case,” which the authors did address. But it did seem to me that one could do a case analysis with only one method, such as content analysis.
Similarly, for a methodology book there was little discussion of appropriate case selection or identification, which presumably is key. In comparative political science, for example, the selection of cases in, say, qualitative historical studies, especially for relevant variables of comparison if not for theory testing, is a crucial criterion. In any event, there are plenty of other sources on case-based research, some listed in the book’s references, that address this issue.
The book itself is very well organized and well written. True, by its nature it won’t keep you up all night like the latest thriller, but for a text it is written in a clear, accessible, and reasonably engaging style. Section headings and parallel organization make it easy to jump back and forth, and chapter summaries along with an extensive glossary likewise aid the reader.
The authors hope their framework will become the gold standard for this kind of research. One potential problem with this aim, of course, is getting their manifesto accepted by the scholarly, and especially editorial, community. For example, it might be difficult to include detailed narratives of method and procedure called for under “thick description,” along with the mandatory findings and implications, under the word-count limits of most professional journals (and grant or book proposals, too).
Nevertheless, the ways of thinking about, designing, implementing, and reporting on research within TQF should help all qualitative researchers produce better work and better-written products. Some of the advisory material, like on representativeness or ethics, is rather intuitive or common sense—indeed, I have applied some of their suggestions without realizing it—and if nothing else provides fodder for arguments to rebut potential blind reviewers. Still, it is good to be reminded of these elements, as well as have a ready-made template for following best practices of others.
Overall, Roller and Lavrakas have done social science scholars of all stripes a big favor. This is a fine—if not essential—reference for those interested in, embarking upon, or just considering qualitative research for their next study, or for those teaching methods courses.
