The Use of Closing Questions in Qualitative Research: Results of a Web-Based Survey

Abstract

Background

Scarce and differing reasons for including closing questions in qualitative research exist, but how data generated from these questions are used remains uncertain.

Objectives

The purpose of the study was to understand if and how researchers use closing questions in qualitative research; specifically, the research questions were: (a) “Why do qualitative researchers include or exclude closing questions during interviews?”; and (b) “How do qualitative researchers use data from closing questions?”

Methods

A qualitative descriptive design using a single, asynchronous, web-based, investigator-designed survey containing 14 items was used to collect data. Convenience and snowball sampling were used to recruit participants. Data were analyzed using descriptive statistics and qualitative content analysis. Codes were developed from the qualitative data. Subcategories were derived from similar codes, and these subcategories were further scrutinized and were used to create broad categories.

Results

The number of respondents per question ranged from 76 to 99; most identified nursing and sociology as their academic disciplines, lived in the United States, and were involved in qualitative research for 1 to 10 years. Data, the interview, the interviewee, and the interviewer were broad categories to emerge as reasons for including closing questions. Only one respondent reported a reason for excluding closing questions. The uses of closing question data were described in four broad categories: analysis; data; the interview guide; and inquiry.

Discussion

Researchers frequently included closing questions in qualitative studies. The reasons for including these questions and how data are used varies and support for closing questions is limited in previously published literature. One unique reason, adding “new breath” to the interview, emerged. Study findings can aid qualitative researchers in deciding whether to include closing questions.

Qualitative researchers use various approaches to conclude interviews. The absence of standard procedures or rules contributes to methodological variability (Brinkmann & Kvale, 2014). Some researchers terminate an interview by thanking participants and acknowledging their contributions (Baumbusch, 2010; Whiting, 2008); others add closing questions. Limited and divergent accounts exist regarding researchers’ choice of executing closing questions, question format, and rationale for including these.

A closing question may allow research participants time to reflect, share additional information, and decompress; however, how this information informs the research is unclear. King and Horrocks (2010) suggest using closing questions to give more control to participants and ask if they want to share anything else or inquire about the research. Rationale for this approach is to minimize tension or anxiety from discussing personal and emotional experiences, and/or concerns about the research process. Similarly, Brinkmann and Kvale (2014) add a final debriefing stage to allow participants to process emotions and share further information. During debriefing, the interviewer may summarize the main points and ask participants to comment and provide feedback. The authors provide no guidance on how the additional information and feedback are used. In contrast, other authors use closing questions to signal the conclusion of the interview (Castillo-Montoya, 2016; Wengraf, 2001). Although not considered primary research questions, reflective closing questions may add valuable information or raise additional issues for the researcher to consider (Castillo-Montoya, 2016).

Few authors describe how they use or analyze data generated from closing questions. Krueger and Casey (2015) include three types of closing questions with focus groups, which they believe are critical for data analysis and directing future interviews. Participants are asked an all-things-considered question (e.g., “Of all the needs we discussed, which one is most important to you?”) allowing participants time to reflect, comment on critical areas of concern, and clarify their positions. Responses inform interpretation of conflicting comments and assign weight of importance. A summary question (e.g., “Is this an adequate summary?”) and then an insurance question (e.g., “Have we missed anything?”) are used for considering topic importance, and can inform modifications of interview guides. Lincoln and Guba (1985) describe terminating an interview by completing a member check where the interviewer summarizes what they believe they just learned, and the interviewee is given time to react and comment on the validity of the constructs made. All information learned, including additional thoughts sparked by the summary, is available for possible triangulation and further member checking as the study proceeds (Lincoln & Guba, 1985).

Objective

Due to limited and variable literature regarding closing questions, the purpose of our study was to understand if and how researchers use closing questions during qualitative interviews. Specifically, our questions were: (a) “Why do qualitative researchers include or exclude closing questions during interviews?); and (b) “How do qualitative researchers use data from closing questions?”

Methods

Study Design

A qualitative descriptive design was utilized for this study. No a priori conceptual framework was used in this study; however, the study was informed by the axioms of the naturalistic paradigm as described by Lincoln and Guba (1985). The [institution 1] and [institution 2; blinded for review] institutional review boards deemed this study exempt.

Participants

A convenience sample of researchers known to the authors as having engaged in qualitative research were recruited initially; snowball sampling was used to recruit additional participants. First, the corresponding author emailed 15 colleagues to describe the study and included a link to complete the web-based survey. Additionally, coauthors sent emails to 66 others, and these 81 (15 + 66) initial contacts were asked to forward the survey to others engaged in qualitative research. Given the recruitment strategy, it is unknown how many emails were sent inviting people to complete the survey; therefore, we are unable to calculate a response rate.

Measurements

A single, asynchronous, web-based, investigator-designed survey was used to collect data. Before launching the survey, coauthors pilot tested it to improve the questions’ structure and sequence. The final survey included 14 questions: Nine multiple-choice and five free text responses. Respondents were asked about themselves, their experiences with conducting qualitative research and, specifically, if and how they used closing questions during qualitative interviews. The survey was administered via SurveyMonkey® and was available from December 11, 2017 until January 8, 2018.

Data Analysis

Descriptive statistics were used to analyze data from the multiple-choice questions; however, the focus of this article is on the responses to two of the free text questions: (a) “Please tell us why you include or do not include closing questions in your interviews.”; and (b) “Please tell us how you use the data generated from closing questions.” Individual responses to these questions were analyzed using qualitative content analysis (Graneheim & Lundman, 2004; Sandelowski, 1995). Individual codes that were similar in content and meaning were subcategorized. Subcategories were further scrutinized and placed into broader categories (Morse, 2008). Table 1 provides an example of how similar codes were collapsed into subcategories, which were then collapsed into a category. Separate codebooks were created for each question given that the purpose of each differed (i.e., reasons for excluding/including closing questions and how data from them are used). Data were analyzed by the corresponding author and final categories were reviewed and approved by all authors. Techniques to establish trustworthiness included the development of a coding system, peer debriefing, and maintaining an audit trail of decisions related to analysis (Abboud et al., 2017; Lincoln & Guba, 1985; Morse, 2015).

Table 1.

Example of Category Development

	Category	Subcategories	Codes
	Q12. Please tell us why you include or do not include closing questions in your interviews
Data		Adding to existing data	Add richness to data
			Detailed descriptions
			Enhance the conversation
			Ensure research question explored fully
			Tie up loose ends
		Collect data	Capture missed relevant information
			Generate data
			Prompt interesting comments
			Retrieve data from an inaccessible area
			Uncover hidden findings
		Modify questions	Ideas for questions
			Modify interview protocol/guide
		Uncover new areas/topics to explore	Expose areas to explore
			Finding something unique

Results

Respondents per question ranged from 76 to 99. Academic discipline/profession was reported by 99 respondents. Most identified with nursing (49%) and sociology (17%); 17% were not explicit (e.g., “social science” and “researcher”). Current country of residence varied; the United States was the most frequent answer (83%). Other reported locations were Canada (7%), the United Kingdom (2%), and Brazil, India, Ireland, Nepal, Nigeria, Peru, Switzerland, and Turkey (1% each). Years involved in qualitative research was reported by 96 respondents; 15 years was the most frequent (35%) followed by 6–10 years (27%), more than 15 years (25%), less than one year (7%), and 11–15 years (5%).

The frequency of including closing questions was reported by 95 respondents; 81% reported doing so “always”, 15% “sometimes”, and 4% “never”. The inclusion of closing questions by interview type varied among the 91 respondents to this question; 91% for semistructured interviews; 34% for structured interviews; and 29% for unstructured interviews (this question allowed respondents to choose all answers that apply).

Reasons for Excluding or Including Closing Questions

Free text answers from 85 participants regarding reasons for including closing questions during interviews were analyzed and four broad categories emerged: data; the interview; the interviewee; and the interviewer. Table 2 includes representative quotes from each of these categories. Only one respondent offered a reason for excluding closing questions: “I feel these kinds of closing questions are often unhelpful, as most will just respond ‘no’ or go on a tangent that is not necessarily related to your research questions.”

Table 2.

Reasons for Including Closing Questions

Category	Representative quote
Data	“Provide participants a final opportunity to share anything related to the research that they have not done so already”
The interview	“I always include closing questions to wrap up the interview”
The interviewee	“A closing question like ‘is there anything else you’d like to tell me that you think might be useful?’ honors that we are not the expert on the topic, that they are”
The interviewer	“To make sure I have obtained what I need for the study”

Data

Respondents described using closing questions as a means of collecting data, including adding to existing data, uncovering new areas or topics to explore, and for modifying interview guides.

The Interview

Most responses in this category indicated that closing questions are included as a means for signaling the close of the interview. Additionally, respondents offered that these questions are a way of indicating the existence of the relationship between the interviewee and the interviewer. Notably, one respondent indicated, “i [sic] always include closing questions because they can give an [sic] new breath to the interview”, which contrasts most reasons for including them.

The Interviewee

Many respondents reported asking closing questions to acknowledge the interviewee as a person. These acknowledgments took many forms, including conferring respect and honoring participants, conveying that participants were heard, and that their experiences were valued. Closing questions were also noted to give interviewees “a feeling of agency” and voice as they expand on interview topics, particularly those most important to them.

Responses in this category moved beyond simply garnering interviewees’ replies to questions, but rather, provided opportunities for researchers to hear additional information free of interviewers’ assumptions and removed from their own research agenda. Such questions were also described as benefiting interviewees by providing an outlet for reconsidering or rethinking topics discussed during interviews, summarizing and finalizing thoughts, and gaining closure to the experience of being interviewed or of speaking about their experiences.

The Interviewer

This category concerns use of closing questions for the purpose of serving the interviewer. Closing questions provide them opportunities to obtain feedback for instrument development, clarify interviewees’ responses, and terminate the relationship. In addition, closing questions are used to ensure data needed for the research endeavor have been obtained, and to attain information to support an argument or stance related to the area being studied.

The Uses of Closing Question Data

In addition to understanding why closing questions are used, we were also interested in learning how respondents use the data. Free-text answers from 81 respondents were gathered and from these four broad categories emerged from these data: Analysis, data, the interview guide, and inquiry. Table 3 includes representative quotes from each of these categories.

Table 3.

Uses of Closing Question Data

Category	Representative quote
Analysis	“I include these responses as part of the overall coding/thematic analysis”
Data	“They may just become part of the corpus of data I’ve collected”
The interview guide	“Although sometimes, I add in questions to my interview protocol based on the way previous participants have answered the closing questions”
Inquiry	“They may open up new lines of inquiry”

Analysis

Some respondents described using closing question data similarly to how they use all other study data, while others reported separate analysis of these questions. Respondents also described using data from closing questions in all phases of the analytic process, including generating codes and themes and for higher levels of analysis, such as data interpretation. Data also served as a source of reflection for researchers relative to what they are learning from the data and how they come to understand them.

Data

Data are used as new information or to supplement and clarify other existing data, or, more generally, to add to the overall body of data. Some respondents noted they rarely or ever use these data, offering that closing questions generate little or no data, and that use is dependent on some other condition (e.g., if relevant to the purpose of the interview).

The Interview Guide

Some respondents reported using data to inform and modify interview guides, such as via addition of questions and prompts to be included in subsequent versions.

Inquiry

Finally, data are used to discover new areas for future inquiry. Examples include generating new research questions or components of future studies (e.g., informing the development of a survey).

Discussion

To our knowledge this is the first study to describe why researchers include closing questions and how data from them are used. We found most respondents use closing questions; however, the rationale for inclusion and how the data are used varies. As with Castillo-Montoya’s (2016) suggestion, some respondents had a functional intention for the closing question: Signaling the end of the interview and validating the relationship between the interviewer and interviewee. Our study unearthed a previously unreported reason for including a closing question: Adding “an [sic] new breath” to the interview. This runs counter to ending an interview (Wengraf, 2001) which is inherent in the name “closing” question.

Respondents shared they intended for the closing question to augment or expand what had been shared by the interviewee, striving for completeness in the data in case something pertinent was not captured previously in the interview. Echoing King and Horrocks (2010), some respondents felt closing questions provide a space where the interviewee may have a stronger sense of agency and autonomy. The open-endedness of the questions does not impose the theoretical or philosophical lens that may have informed the interview guide, giving interviewees more control. Interestingly no respondents used closing questions as an opportunity for the interviewee to emotionally debrief from the interview as suggested by Brinkmann and Kvale (2014).

Respondents included closing questions to generate data to inform future topics of inquiry and modify interview guides, and analyzed data produced by closing questions jointly and separately from the rest of the data. Closing question data augments existing data when analyzed jointly. Reflecting on Krueger and Casey’s (2015) and Lincoln and Guba’s (1985) use of the closing question, data analyzed separately from the rest of the data were treated as new information, supplemental, or used for triangulation. This strategy further strengthens the trustworthiness of the data and rigor of the study (Lincoln & Guba, 1985). Limitations include convenience and snowball sampling and a design that did not allow us to ask follow-up or probing questions to further explore answers to free text response questions. Additionally, our survey was created by and for those whose primary language is English.

Conclusion

We found that researchers include closing questions in qualitative research, which is consistent with the limited existing literature and reinforces that valuable data can be collected via closing questions. We recommend building on our results to further the discussion regarding alternative views, usefulness of established criteria researchers use to guide closing question development and use, and as a process of sparking continued conversations about this commonly used component of data collection.