Appraising Qualitative Research in Health Education: Guidelines for Public Health Educators

Abstract

Research studies, including qualitative studies, form the basis for evidence-based practice among health professionals. However, many practicing health educators do not feel fully confident in their ability to critically appraise qualitative research studies. This publication presents an overview of qualitative research approaches, defines key terminology used in qualitative research, and provides guidelines for appraising the strengths and weaknesses of published qualitative research. On reading, health educators will be better equipped to evaluate the quality of the evidence through critical appraisals of qualitative research publications.

Published research studies, including qualitative studies, provide the evidence for the selection of evidence-based practices in health education and promotion. For health educators who may not be comfortable with their skills in determining the quality of research studies, critically reading research studies can be time consuming and challenging. Health educators can however increase their confidence in appraising research studies by using the guidance described in this article.

Jack et al. (2010) offer general guidance and recommendations for increasing one’s skills and confidence in reading scientific publications that included

1. becoming more familiar with the key components of a research publication and

2. using key questions and guidelines presented in the article to critically appraise the strengths and weaknesses of published studies.

Research publications should provide adequate information in order to assess the strengths and weaknesses of any research study. The reader needs to have a basic knowledge of qualitative research in order to appropriately appraise a qualitative study and determine the value of the evidence. The purpose of this article is to provide an overview of qualitative research. In addition, this article will provide health educators with general guidelines for appraising the quality of published qualitative research studies.

QUALITATIVE RESEARCH

There is no universal definition of qualitative research, as it is an umbrella term that covers several approaches. However, Burns and Grove (2007) describe qualitative research as focusing on the human experience through systematic and interactive approaches. Qualitative research methods are usually used when little is known about the topic and allows the researcher to explore meanings and interpretations of constructs rarely observed in quantitative research. Studies are conducted in natural settings and provide a context to observed phenomena. The information sought focuses on how something is experienced or processed and not specifically about facts and figures. The main approaches are phenomenology, ethnography, and grounded theory.

Qualitative research studies begin with the identification of a problem. The research question may be implied in the problem statement or stated separately. Additional research questions may emerge from the data as the study progresses. Generally, qualitative research studies do not begin with a hypothesis, although some studies may result in the formation of hypotheses that are later tested using quantitative methods (Greenhalgh & Taylor, 1997).

THE KEY COMPONENTS OF A QUALITATIVE RESEARCH PUBLICATION

Jack et al. (2010) suggest that any quantitative research publication typically include the following key components: publication title, abstract, introduction, method, data analysis, results and discussion/conclusion. These components are consistent with those found in qualitative research publications. However, characteristics of these components may vary in qualitative studies. A brief description of these components as they pertain to qualitative research follows.

Publication Title

The title generally consists of a heading that provides insight into the reported research study by including reference to the research problem or concept studied, the population, and the research design.

Abstract

An abstract provides the reader with a brief description of the overall research study, the sample, how the study was conducted, data collection and analysis, relevant results, and important implications and or recommendations.

Introduction

This section provides the rationale for conducting the study by elaborating on the concept along with introductory information as to what is included in the remaining sections of the article. The introduction frequently includes the reason for using a qualitative approach along with respective philosophical or theoretical underpinnings and the review of the published literature that provides a rationale or purpose for conducting the particular study.

Method

This component provides information on qualitative research and the selected approach. Reasons for the selection of the particular research approach along with the problem statement, purpose or the research questions, recruitment strategies, and sampling plan are included. A description of the sample includes inclusion and exclusion criteria used to identify participants eligible for the study. Data collection and analysis procedure(s) should be included in this section.

Results

This section provides the results of the data analysis and includes information needed to evaluate the strength of the study’s evidence. The results section of a qualitative research publication should include the identified themes or patterns, along with participant quotes that depict the essence of the data. The study’s limitations may also be included and researchers should discuss how each limitation may influence the applicability of study results.

Discussion/Conclusion

This section summarizes important findings and results as well as discusses how the various themes and/or patterns relate to the concept studied or answers the research question. The discussion section should also provide an explanation as to whether the study results are consistent with existing literature, which aids in the interpretation of the meaning of study results. Finally, this section should explain how the results can be transferred to other groups of people along with recommendations for future research and the advancement of health education.

CRITICALLY APPRAISING THE STRENGTHS AND WEAKNESSES OF PUBLISHED QUALIFIED RESEARCH

A thorough understanding of the research study is especially needed in order to determine the strengths and weaknesses of the methodology, to evaluate the quality or strength of the study’s evidence, and to identify the appropriateness for use in the reader’s practice. This understanding is achieved by critically appraising the research publication. A review of the literature has revealed six questions that may guide the evaluation of qualitative research articles (Curtin & Fossey, 2007; Fowkes & Fulton, 1991; Greenhalgh, 1997; Greenhalgh & Taylor, 1997; Henderson & Rheault, 2004; LoBionda-Wood, Haber, & Krainovich-Miller, 2006; Malterud, 2001).

1. Did the qualitative research describe an important health education practice–related problem addressed in a clearly formulated research question?

The research question helps the researcher to decide whether to conduct a quantitative or a qualitative study. However, the research question in qualitative studies is not always provided as a bona fide question but may be implied or immersed in the purpose or the aim of the study. In addition, qualitative research questions may change as data emerge.

1. Was the qualitative approach appropriate?

The use of the qualitative research is likely to be justified if the purpose of the research study is to seek descriptive, in-depth insights into a phenomenon about which little is known from the participant’s perspective and it is appropriate for the research question. Although multiple approaches exist, the particular qualitative approach that was used may not be explicitly stated. The more commonly used approaches are described.

Phenomenology

Carpenter (2007) defines phenomenology as “a science whose purpose is to describe a particular phenomenon or the appearance of things, as lived experiences” (p. 43). Those topics that are central to people’s lived experiences are best suited for phenomenological research.

Ethnography

Ethnography involves describing and learning from a culture (Spradley, 1979). Generally, fieldwork, characterized by some type of participant observation, provides the basis for data collection. Because ethnography focuses on culture, it is an appropriate approach to use when studying cultural influences on health.

Grounded Theory

Grounded theory research involves the generation of theory from data (Glaser, 1998; Glaser & Strauss, 1967). Realization that the theory actually emerges from the data provides the key to understanding grounded theory research. Awareness that groups of people have common or shared problems and have similar ways, or processes, of solving the problems helps researchers contribute to that understanding.

1. How were the participants selected?

Appraisal of the source of the sample, sampling method, sample size, and inclusion and exclusion criteria helps to ensure that the study’s sample is representative of the population from which it is drawn. An overview of the four common types of samples is provided.

Purposeful/Purposive

Participants in a purposive sample are selected according to the needs of the study (Morse, 1991). Initially, the researcher may desire to interview individuals with broad knowledge about the concept or those who meet a general criterion. As the study progresses, sample needs may change somewhat and selection criteria may be altered. Purposive sampling may evolve into theoretical sampling, which involves selecting participants or data sources that will contribute to the emerging theory.

Nominated, Network, Snowball

Nominated, network, or snowball sampling involves participants suggesting other people as possible study participants. This approach has been found to be helpful in aiding people establish trust with the researcher and the research study.

Volunteer/Convenience

Volunteer samples are composed of individuals who are not known to the researcher or other participants but have volunteered to participate in the research study (Morse, 1991).

Total Population

Total population sample refers to a sample in which all participants live or work in the same confined area, such as all health educators employed in a large school district (Morse, 1991).

The adequacy of a qualitative sample is evaluated by the quality and amount of the data—not the number of participants. Sample size in qualitative research studies is much smaller than in quantitative studies because of the nature and processes of the method. Sample size is determined by the recognition of data saturation, which occurs when there are no new data emerging and redundancy occurs.

1. What were the researchers’ roles in conducting the study and has this been taken into account?

Qualitative researchers acknowledge that there is a possibility that their values and beliefs may influence their research studies (Porter, 1993; Jootun, McGhee, & Marland, 2009). Therefore, it is the researcher’s responsibility to be self-aware of one’s own reactions, reflections, and even personal growth along with the researcher and participant relationship. The possible effect of the researcher’s influence can be reduced through bracketing and reflexivity. Bracketing refers to the researcher’s acknowledgement of the possible influence and an intentional setting aside of conscious thoughts and decisions influenced by the particular mindset. Reflexivity involves the researcher’s self-awareness and the strategies the researcher used to manage potentially biasing factors while maintaining sensitivity to the data (Porter, 1993;Speziale & Carpenter, 2007; Jootun et al., 2009).

1. What methods did the researcher use for collecting data—and are they described in appropriate detail?

Traditional sources of qualitative data include in-depth interviews, focus groups, and participant observation. The selection of the data source depends on the research question, choice of qualitative approach, sensitivity of subject matter, available resources as well as the investigator’s skills and experience (Streubert-Speziale, 2007). An overview of data sources follows.

Interviews

Interviews are commonly considered to be the mainstay of qualitative research. Most interviews consist of the researcher using an interview guide to ask semi-structured open-ended questions that are intended to help the participant openly share personal experiences. Interviews vary in length and are traditionally audio-taped in mutually agreeable locations where privacy can be ensured.

Focus Groups

Focus groups are sessions conducted by a group leader, who uses question or interview guides for the purpose of discussing a particular topic. They are considered to be effective for addressing sensitive topics, are relatively inexpensive, and provide cumulative information from multiple participants (Streubert-Speziale, 2007).

Participant Observation

Various forms of participant observation, which refers to varying degrees of researcher involvement or observation, have been used to study participants’ activities.

1. What methods did the researcher use to analyze the data and what measures were used to ensure that scientific rigor was maintained?

Researchers need to describe the methods used to manage and analyze the qualitative data. Assessment of methods and approaches used to analyze the qualitative data contributes to evaluation of the rigor or scientific strength of the study. Qualitative research studies can generate large volumes of data. Therefore, prior to any data analysis, the investigator needs to have an organized plan for data management.

Qualitative data can be analyzed manually or via computer software. Whether the data is managed manually or electronically, the researcher interprets the data through processes aimed at identifying recurring themes and/or patterns that are then clustered into increasingly abstract levels or groupings. Thus, each level of clustered data is more abstract than the previous level. Throughout these processes, the researcher compares each new piece of data to previous data and to existing literature as means of confirming preliminary interpretations. Eventually, the theory or final interpretation of the data emerges from the data.

Streubert-Speziale (2007) specified that “the goal of rigor in qualitative research is to accurately represent study participants’ experiences” (p. 49). Any research study that lacks rigor or scientific strength should be considered less than desired, and little credence should be given to the strength of its evidence. Because of the differences between the two methods, the rigor of a qualitative study should not be evaluated by the same criteria used to determine the strength of rigor in a quantitative study. Reliability and validity, commonly associated with quantitative research, have, for the most part, been replaced with “trustworthiness” when evaluating qualitative studies. According to Lincoln and Guba (1985), trustworthiness refers to the “truth value” of the study’s findings or how accurately the investigator interpreted the participant’s experiences. Generally, rigor in qualitative research is established through the study’s confirmability (or auditability), credibility, and fittingness (or transferability; Cutcliffe & McKenna, 1999; Lincoln & Guba, 1985; Sandelowski, 1986; Streubert-Speziale, 2007).

Confirmability, or auditability, refers to the documentation, or paper-trail, of the researcher’s thinking, decisions, and methods related to the study (Polit, Beck, & Hungler, 2006; Streubert-Speziale, 2007). Field notes, memos, transcripts, and the researcher’s reflexivity journal or diary allow the reader to follow the researcher’s decision making.

Credibility refers to the confidence in the truth value or believability of the study’s findings (Polit, Beck, & Hungler, 2006; Sandelowski, 1986; Streubert-Speziale, 2007). Credibility is demonstrated through strategies such as data and method triangulation (use of multiple sources of data and/or methods), repeated contact with participants, peer debriefing (sharing questions about the research process and/or findings with a peer who provides an additional perspective on analysis and interpretation), and member checking (returning findings to participants to determine if the findings reflect their experiences). The researcher’s reflexivity also contributes to the study’s credibility as it helps to make the reader more aware of possible influences on the study.

Fittingness or transferability of research findings refers to the study findings’ fitting outside that particular study. Fittingness also refers to the possibility that the findings would have meaning to another group or could be applied in another context (Byrne, 2001; Streubert-Speziale, 2007). An accurate and rich description of research findings demonstrates fitting-ness or transferability by providing adequate information for evaluating the analysis of data.

CONCLUSION

Because the selection of the research method depends on the research questions being asked, qualitative research provides an excellent approach to collecting and analyzing information to important questions in health education research. The guidelines, questions, and explanations provided in this article are not intended to be all inclusive. However, the information can provide the reader with a deeper understanding and appreciation for published qualitative research. No study is perfect, nor does any study answer all questions. Accordingly, it is recommended that qualitative studies are read critically and that the value of evidence be critically assessed. As readers become more comfortable in reading and appraising qualitative research, it is anticipated that readers will become more confident in their understanding of the various terminologies, methods, and approaches used in conducting and reporting qualitative research.