Study Design and Use of Inquiry Frameworks in Qualitative Research Published in Health Education and Behavior

Abstract

Qualitative methods help us understand context, explore new phenomena, identify new research questions and uncover new models of change. To better understand how researchers in health education and health behavior use qualitative methods, we reviewed qualitative papers published in Health Education and Behavior from 2000 to 2015. We identified 48 papers that met our inclusion criteria and extracted information on the qualitative inquiry framework, use of theory, data collection methods, sampling strategy, general analysis approach and reporting of results. Use of common qualitative inquiry frameworks was rare, with just one grounded theory study, five ethnographies and one case study. No studies were framed using phenomenological or narrative inquiry approaches. Theory was used most commonly to select sensitizing constructs for analysis (41.7%) and to inform development of data collection instruments (27.1%). Interviews were the most common data collection method (66.7%), with focus groups next most common (39.6%). Sampling was typically purposive (87.5%), although often not labeled as such. Almost all (95.8%) of the papers used quotes to illustrate themes and more than half (58.3%) used descriptors of magnitude (e.g., most, some) to report findings. The use of qualitative methods by health education and behavior researchers could be enriched with more intentional application of a broader range of inquiry frameworks. More deliberate application of a range of inquiry frameworks has the potential to broaden the types of research questions asked, application and generation of theory, study design, analytic strategies and reporting of results.

Introduction

For decades, researchers who apply social and behavioral sciences to public health have acknowledged the potential of qualitative methods to inform their work (Backett, 1992; Broughton, 1991; Foster, 1987; Gerhardt, 1990; Glik, Parker, Muligande, & Hategikamana, 1986; Goodman, Steckler, Hoover, & Schwartz, 1993; Steckler, McLeroy, Goodman, Bird, & McCormick, 1992; Yach, 1992). Qualitative methods are key to understanding context; exploring new phenomena from multiple perspectives; generating, refining and extending theory; and guiding quantitative studies by illuminating new research questions or informing more nuanced or culturally appropriate measures (Creswell & Poth, 2018; Hennink, Hutter, & Bailey, 2011; Miles, Huberman, & Saldana, 2014; Patton, 2015; Saldana, 2016; Saldana & Omasta, 2018). Qualitative methods can provide essential process evaluation data on facilitators and barriers to implementation, can elucidate models of change and logic model pathways, and provide evidence of policy and program impact through case studies and causal qualitative analysis (Hennink et al., 2011; Patton, 2015).

In 1987, Health Education Quarterly (the former name of Health Education and Behavior) published its first paper arguing for more qualitative research in health education (Basch, 1987). A few years later in 1992, Health Education Quarterly published a theme issue that highlighted methods for combining qualitative and quantitative research (Steckler et al., 1992). Although not labeled “mixed methods” at the time, Steckler and colleagues underscored the value of qualitative methods, both combined with quantitative methods and as a stand-alone paradigm. Recently, HE&B published a theme issue highlighting how qualitative methods can advance health equity (Griffith, Shelton, Kegler, 2017). The Society of Public Health Education’s (SOPHE) practice journal, Health Promotion Practice, has published tools and support for qualitative research (Lewis, 2015; Watkins, 2012). Recently updated health education competencies include both design of qualitative research and qualitative data analysis as necessary skills for master’s level practitioners (National Commission for Health Education Credentialing, 2015).

As qualitative researchers, teachers of qualitative methods in MPH curricula, and members of HE&B’s editorial team, we have noticed great variability in how qualitative research is reported, from how qualitative inquiry frameworks are applied to explanations of the analytic process. This led us to question whether our field was using qualitative methods to their full capacity, whether we were building on existing methodological knowledge, and whether we could be more intentional in our application of qualitative methods. These observations and our curiosity were the catalyst for the review of qualitative articles reported on here. The current paper presents findings from a review of all qualitative papers published in HE&B between 2000 and 2015. As the flagship journal for SOPHE and a leader in shaping intellectual discourse in the field, we felt reviewing papers in HE&B would provide an appropriate gauge of how we in health education and health behavior are conducting and using qualitative research. We sought to learn what inquiry frameworks were commonly used and how they informed study design, use of theory, application of analytic strategies, and reporting of results.

Methods

Articles were eligible for inclusion if they exclusively used qualitative research methods (i.e. not mixed methods), and were published in HE&B between 2000 and 2015. PubMed was used to identify eligible articles using the search string “((qualitative) AND “Health education & behavior: the official publication of the Society for Public Health Education”[Journal]) AND (“2000”[Date - Publication]: “2015”[Date - Publication])”. Two investigators independently reviewed each title and abstract and labeled each article as “include”, “exclude”, or “review in full text”. Ultimately, 48 articles were included. Of the 56 articles excluded, 50 were excluded through the title and abstract review, and an additional 6 were excluded after full text review. The two investigators agreed on all but one determination, which was resolved through discussion, and ultimately included. The primary reason for exclusion was the use of mixed methods (n=32). Although not a formal systematic review as we only reviewed articles from one journal, Figure 1 presents the PRISMA flow chart.

Figure 1.

PRISMA flow diagram.

Note. PRISMA = Preferred Reporting Items for Systematic Reviews and Meta-Analyse. From Moher, Liberati, Tetzlaff, Altman, and The PRISMA Group.

(2009). For more information, visit www.prisma-statement.org

The abstraction form was designed to assess the key methodological and analytic approaches used in each article. Topics and coding categories were adapted from the domains presented in O’Brien et al. (O’Brien, Harris, Beckman, Reed, & Cook, 2014) and Tong et al.’s (Tong, Sainsbury, & Craig, 2007) qualitative reporting standards, in combination with our pooled expertise in conducting and reviewing qualitative research. We also reviewed sampling and analytic strategies as described by Patton for possible categories (Patton, 2015). A draft form was shared with all co-authors, whose feedback was incorporated into a final version comprised of 10 main sections: qualitative inquiry framework, sampling strategy, data collection methods, data management software, coding approach, analytic approach, reporting of results, use of theory, indicators of trustworthiness, and reflexivity. Tables 1–3 show the categories and codes used in abstraction for the data reported here.

Articles were divided among the six co-authors, and each article was double-coded. Two authors each coded half of the articles, the other four coded one-quarter. One member of each coding pair reviewed the completed abstraction forms and marked areas of disagreement. This coder corrected discrepancies easily resolved by re-reviewing the full text, and a third coder (not in the original coding pair) reviewed all other discrepancies. The final reconciled forms were entered into an Access database, and queries were generated to summarize the results for each topic. The results were shared with all co-authors for review, and then discussed as a group.

New topics of interest emerged from the first round of reviews, particularly related to the analysis. Two of the authors conducted a second round of review focused on common analytic pathways or trajectories including: the use of software in analysis, the use of matrices, how and when authors discussed achieving consensus on codes, the variation in use of the terms code versus theme, how themes were identified, and the process of grouping codes or themes. These findings are reported in a companion paper (Raskind et al., 2018).

Results

Qualitative Inquiry Framework

The articles were reviewed to determine if the research was based on a qualitative inquiry framework (Table 1). Thirty-seven papers (77.1%) did not mention a qualitative inquiry framework. Five papers mentioned ethnography (10.4%), and one paper mentioned grounded theory (2.1%). The former was exemplified in a paper by Thompson and colleagues (Thompson, Gifford, & Thorpe, 2000) who spent almost two years deeply engaged in an Aboriginal community in Australia to contextualize behavioral risk factors for diabetes within society and culture. The grounded theory paper described iterative interviews and focus groups with physically active African American women (Harley et al., 2009). The authors developed the physical activity evolution model to explain the changes, both psychological and behavioral, involved in the process of becoming and staying physically active. Additional frameworks were referenced in only one paper such as community-based participatory research and mutual knowledge creation, and life history interview methods. Some of these terms are general approaches to research and others are often used to describe analysis methods; we considered them as inquiry frameworks when the authors used them to describe an overall approach to their research.

Table 1.

Qualitative Inquiry Frameworks and Use of Theory (n=48)

Design Element	N	Percent
Qualitative Inquiry Framework
Ethnography	5	10.4
Grounded theory	1	2.1
Phenomenology	0	0.0
Narrative inquiry	0	0.0
Other	5	10.4
None mentioned	37	77.1
Use of Theory
Sensitizing constructs for analysis	20	41.7
Informed data collection instruments	13	27.1
Mentioned in introduction and discussion only	8	16.7
Informed sampling strategy	2	4.2
Developed as a result of analysis	2	4.2
Other	2	4.2
No mention of theory	17	35.4

Use of Theory

Theory can be generated and refined through qualitative methods or it can be used to guide various phases of a study design. Of the papers we reviewed (Table 1), the most common use of theory was to provide sensitizing constructs for analysis (41.7%). Cortes and colleagues, for example, described a study of the activation and empowerment process in the context of increasing Latino patient participation in mental health treatment decisions (Cortes, Mulvaney-Day, Fortuna, Reinfeld, & Alegria, 2009). Results were organized by components of a published empowerment model. Theory was also used fairly often to inform development of data collection instruments (27.1%). Barnidge et al. (2011) used the sustainable livelihoods framework to identify structural factors that influence health in rural communities (Barnidge, Baker, Motton, Fitzgerald, & Rose, 2011). The framework guided photo-elicitation interview questions, a start list of codes for analysis, and organization of the results. In two of the papers, theory informed the sampling approach and in one paper, theory was developed as a result of the analysis. Interestingly, a significant number of the papers did not mention theory or frameworks at all (35.4%). In some cases, however, theory was mentioned in the introduction and discussion sections, but the theory was not evident in the research process itself (16.7%).

Data Collection Methods

The most common data collection methods were one-on-one interviews (Table 2), described in approximately two-thirds (66.7%) of the articles, and focus groups, described in approximately one-third of papers (39.6%). Several (10.4%) of the papers described observation in combination with interviews or other methods. The majority (78.1%) of interviews were conducted in-person; several papers (9.4%) reported conducting interviews via telephone and several did not specify the platform used (15.6%). Nearly two-thirds (62.5%) of the interviews were conducted using semi-structured interview guides.

Table 2.

Data Collection Methods and Sampling (n=48)

Study Element	N	Percent
Data Collection Methods
Interviews	32	66.7
Focus groups	19	39.6
Observation	5	10.4
Photovoice	1	2.1
Secondary sources/texts (e.g., newspapers, blogs)	3	6.3
Other	5	10.4
Sampling Strategy
Purposive	42	87.5
Snowball	6	12.5
Census	7	14.6
Convenience	7	14.6
Random	2	4.2
Other	3	6.3

Sampling Approach

A large majority of studies (87.5%) used purposeful sampling techniques (Table 2). Approximately one third (33.3%) of these studies explicitly labeled their sampling approach as either purposeful or purposive. The remaining studies, while not explicitly labeled by the authors as purposeful, described sampling approaches wherein the selection of participants or cases was clearly guided by the study purpose. The degree of detail provided regarding purposeful sampling techniques varied: many authors provided a rationale for their chosen approach (e.g. recruiting from sites frequented by the population of interest, or recruiting for demographic diversity within the population of interest), some authors cited specific types of purposeful sampling strategies (such as maximum variation, intensity, and snowball sampling), and some solely described participant eligibility criteria and listed locations where recruitment occurred. Other commonly described sampling strategies were convenience sampling (14.6%) and census approaches (14.6%). Papers were only coded as using convenience sampling if the term was explicitly stated by the author. Census approaches were most often used in the context of program or intervention evaluations (e.g. recruitment of all intervention participants).

Sample Size and Saturation

Study sample sizes varied widely. For studies using interview techniques, the median sample size was approximately 35, and ranged from 5 to 252 participants. The median number of focus groups conducted was approximately 10, and individual focus groups ranged in size from 2 to 13 participants. Less than one-third (29.2%) of the studies discussed data saturation, and there was wide variation in how it was addressed. A few studies provided detail on how they determined saturation, when it was reached, and how it informed the sampling strategy, while others only stated that it had been attained and provided a brief definition.

Data Analysis

Thematic analysis (22.9%) and content analysis (20.8%) were the most common analytic approaches named by authors (Table 3). Grounded theory (16.7%) and the related approaches of constant comparison (10.4%) and open or axial coding (6.3%) were next most common, along with use of matrices (6.3%). Single and multiple case studies were also included, with just one of each method reported. Additional papers used the terms cases and matrices, but did not explicitly describe case study methods or use of matrices as their analytic approach. A variety of other approaches were named by authors, with none of them commonly used across the studies (e.g., thematic content analysis, ethnographic content analysis, compare and contrast method, framework analysis technique, coder categorizing technique, standard qualitative research methods).

Table 3.

Data Analysis Approach and Reporting of Results (n=48)

Study Element	N	Percent
Analytic Approach according to Authors
Thematic analysis	11	22.9
Content analysis	10	20.8
Grounded theory	8	16.7
Constant comparison	5	10.4
Open and/or axial coding	3	6.3
Data matrices	3	6.3
Case study	1	2.1
Multiple case study	1	2.1
Other approach	19	39.6
Approaches to Reporting Results
Quotes to illustrate themes	46	95.8
Descriptive categories (i.e., most, many, some, a few)	28	58.3
Themes with no subthemes	24	50.0
Percentages/counts of respondents per theme	10	20.8
Themes and explicitly named subthemes	7	14.6
Percentages or counts of mentions or text units	2	4.2
Visual display	3	6.3
Resulting conceptual model	1	2.1
Other	9	18.8

Reporting of Results

Table 3 presents how results were typically reported. Including quotes to support themes was most common by far (95.8%). Over half (58.3%) of the articles used semi-quantitative descriptors such as most, many, some, and a few when presenting themes. Another set of papers reported percentages or counts of respondents per theme (20.8%) or percentages or counts of mentions or text units (4.2%). Visual displays of the findings were uncommon (6.3%), as were conceptual models resulting from the analyses (2.1%).

Discussion

In our review of qualitative research published in HE&B from 2000 to 2015, we found that qualitative inquiry frameworks were underutilized. The majority of studies included in our review were not explicitly guided by a qualitative inquiry framework. Our review showed that less than 25% of the articles explicitly stated that a qualitative inquiry framework had guided the research, and only 12.5% used a well-established approach such as ethnography or grounded theory to shape their studies. Ethnography, with its emphasis on culture, extended time in the field, ongoing observation, and building of rapport with community members, captures the emic perspective of groups and communities (Hennink et al., 2011; Padgett, 2012). Several papers included in our review stated they used an ethnographic approach. These were generally characterized by data collection over time and/or multiple forms of data collection. Our review identified one paper in particular that exemplified this approach with significant time spent deeply engaged with a community (Thompson et al., 2000). Grounded theory was mentioned more often as an analytic strategy rather than as an overarching inquiry framework. When used to describe the analysis, it often seemed synonymous with inductive coding. We did note one study (Harley et al., 2009) that developed new theory using a grounded theory approach which typically involves an iterative process of data collection and analysis (Hennink et al., 2011; Patton, 2015).

We did not find any papers that framed the research as phenomenology or narrative inquiry. Phenomenology, with its emphasis on the essence of a lived experience, requires the researcher to suspend their preconceived beliefs to explore how people make meaning of a particular event or phenomenon (Creswell & Poth, 2018; Patton, 2015). The phenomenon is the target of study, such as experience of stigma or adoption of new guidelines within a federally qualified center. Many of the studies in our review were about a phenomenon, but curiously, this inquiry framework was rarely named. Narrative inquiry, based on stories from a small number of individuals who represent a typical or critical case, values lived experience as a critical foundation for knowledge and an important source of understanding (Creswell & Poth, 2018; Padgett, 2012; Patton, 2015). Stories are often told using chronologies with temporal or episodic ordering. Life histories or life narratives fit within this approach. Again, no papers were explicitly framed as a narrative inquiry, although some studies used associated methods (e.g., life history interviews). Case studies were also rare in our review.

Explicit use of inquiry frameworks and deliberate use of associated analytic methods have considerable potential for theory-building. Grounded theory, which can help to illuminate how a process, action or interaction unfolds (Charmaz, 2000; Corbin & Strauss, 1990; Creswell & Poth, 2018; Strauss & Corbin, 1994), may yield especially valuable insights for more effective change strategies at higher levels of the social ecologic model, such as changing organizational policies, altering neighborhood or community conditions that influence health, and influencing public policies at various levels of government. Case studies also have significant potential for theory development at the organizational, community and policy levels. As with the limited use of inquiry frameworks, we were surprised by the number of studies not explicitly grounded in existing theory. Qualitative methods have considerable potential to refine and extend our existing theories to new contexts or sub-cultures (Hennink et al., 2011), which becomes increasingly important as we attempt to reduce disparities and acknowledge inequities in foundational conditions that contribute to health (Shelton, Griffith, & Kegler, 2017).

Most of the studies we reviewed employed widely used qualitative sampling and data collection approaches. As expected, the majority used purposeful sampling techniques. While not always explicitly labeled as such, most authors provided an explanation for their chosen approach, illustrating how case selection followed from, and was aligned with, the stated study purpose. Of note, few studies named the type of purposeful sampling technique used, such as maximum variation, critical case, theory-based, deviant case, typical case, stratified purposeful, or random purposeful (Creswell & Poth, 2018; Miles et al., 2014; Patton, 2015). A more deliberate and thoughtful approach to sampling may deepen our use of qualitative methods beyond description of a population or phenomenon.

Additionally, only a minority of authors discussed data saturation, a surprising result given the centrality of the principle to many qualitative sampling strategies and analytic processes (Guest, Bunce, & Johnson, 2016; Sandelowski, 1995). Further, many of the articles that did discuss data saturation provided little information on how, when, and why it was assessed, and its implications for the research process. As saturation may be defined and applied differently depending upon the research goals (O’Reilly & Parker, 2013), the limited detail provided made it difficult to assess whether the approach taken was well aligned with the study design and purpose.

Nearly all studies used interviews or focus groups as their primary mode of data collection; very few used observation techniques. Rooted in cultural anthropology, observation can provide in-depth understanding of the context within which behaviors occur, and elucidate how environments are experienced and perceived by the population of interest. These strengths are naturally aligned with social ecological approaches to understanding health behavior, making observation a potentially untapped resource for health researchers. Other data collection methods and sources, such as documents and artifacts were generally rare, as were open-ended conversational interviews.

The most common approach to reporting was to present themes with illustrative quotes. While reporting was limited to what is acceptable in peer-reviewed journals such as HE&B, we were struck by the lack of visual displays of data given the relatively common use of qualitative software with these features. This may be related to our tendency to describe phenomenon with themes supported by quotes, and lack of training on the value of visual displays, or perhaps skepticism about the validity (or credibility) of showing connections between concepts using qualitative approaches. Authors often reported results using semi-quantitative descriptors such as “most” or “many.” While useful as an indicator of salience, other approaches are to highlight the wide range of perspectives or nuances uncovered through the research. Again, a more systematic use of inquiry frameworks would broaden options for how to report results, with increased use of stories, case studies, conceptual models and data displays. Other approaches for organizing and reporting should also be considered, such as chronology, critical incidents, and processes (Patton, 2015).

Limitations

This review has a number of limitations. Many of the papers we reviewed were written before consensus had emerged on how to report qualitative research. Consequently, authors were inconsistent in what they reported in the article and they likely received little additional guidance during the review process. This is not meant to be a critique of the authors of the papers we reviewed, but rather to reflect on what these papers mean for how we should use and report qualitative methods going forward. Additionally, HE&B decreased allowable word count during the time period covered by this review which may have also affected the level of detail provided in the papers. Our analysis is dependent on what was reported in the papers and it may be that some of our conclusions do not reflect how the study was actually conducted. Lastly, although we developed the abstract form collectively and it appeared to work well when pilot tested, we achieved low levels of consensus on some of the domains and chose not to report them here (i.e., Patton’s (2015) types of purposive sampling methods such as comparison-focused versus group-focused and his analytic strategies such as indigenous typology and analyst-generated concepts).

Implications

Gaining insight into how various determinants of health are experienced and understood by the people whose health we seek to promote is fundamental to health education. Interdisciplinary programs such as those in health education and behavior often cover qualitative methods with a semester-long course, which is occasionally supplemented by independent readings or apprenticeship. Several of the major inquiry frameworks are embedded in disciplinary traditions such as sociology, education, anthropology, history and philosophy. Our less-established and interdisciplinary field may benefit from concerted efforts to embed, or at least strategically apply, these frameworks to our training programs and our research. For example, when developing course syllabi, faculty could deliberately assign readings that exemplify a range of inquiry frameworks, even outside of qualitative courses. Or, we could actively seek colleagues from the traditional social sciences to join our research teams or guest lecture in our classes.

Consideration of the paradigms that underlie our research may also shed light on our methodologic choices. Much of the training in health education and behavior is consistent with a positivist paradigm that serves as the foundation for experimental and quantitative research in the social sciences, with an emphasis on objectivity, internal validity, hypothesis-testing and generalizability (Patton, 2015; Hennink et al., 2011). Qualitative research, in contrast, is generally consistent with a constructivist paradigm which purports there are multiple realities that are social constructed (Hennink et al., 2011, Creswell & Poth, 2018; Saldaňa & Omasta, 2018). Some qualitative scholars (Creswell & Poth, 2018; Patton, 2015) describe a pragmatic paradigm that is flexible and not committed to a particular philosophy or paradigm allowing for research methods to vary depending on the research question. The rise of mixed methods as a valued approach in health education and behavior is consistent with a pragmatic paradigm. Thus, our tradition of a positivist and/or pragmatic paradigm may contribute to a hesitancy to fully embrace inquiry frameworks deeply grounded in a constructivist paradigm. To help guard against this, journal editors and reviewers could reflect on their approach to reviewing qualitative research (e.g., not automatically rejecting qualitative papers with small sample sizes) and guard against inappropriate positivist thinking and language (e.g., quantifying results, discussing lack of generalizability as a limitation), both in submitted manuscripts and in reviews.

Practical considerations may also explain our modest use of inquiry frameworks. For example, qualitative inquiry frameworks such as ethnography typically include extensive time in the field. This timeframe is not generally feasible in public health research where qualitative methods are often used to support or contextualize quantitative data or inform future hypothesis-driven studies and where study timelines are dictated by grant funding. Given these constraints, it may be that we need to acknowledge the usefulness of a pragmatic paradigm that can accommodate the interdisciplinary nature and practical considerations of our field, and pursue opportunities to innovate and publish new methods such as combining ethnography with community-engaged research in which community members serve as co-investigators and bring their understanding of the community culture to every phase of the research process or make more use of “rapid” ethnography as long as the essence of the approach is not lost (Hernandez et al., 2017; Needle et al., 2003).

Going forward, we recommend the explicit stating, use and integration of qualitative inquiry frameworks in health education and behavior research. The use of an established framework justifies research decisions and provides transparency. A more deliberate application of a range of inquiry frameworks has the potential to broaden the types of research questions asked, application and generation of theory, study design, analytic strategies and reporting of results. Additionally, as we embrace a social ecologic approach in our intervention and descriptive research, qualitative methods for theory-building may prove useful in understanding how to create and sustain change within organizations, communities and complex systems.