Table 6.

Recommendations to ensure the quality of cognitive interviews

Component	Recommendations	Rationale for recommendations
Scope of survey tool tested	The cognitive interview guide includes a reasonable number of survey questions to complete in 1.5 hours (likely around 30 questions) Additional guides should be developed if the total interview length exceeds 1.5 hours	The cognitive interview requires discussion time and probing about each survey question A 1- or 1.5-hour quantitative survey tool will require over 10 hours of cognitive interview time if each question is examined in the cognitive interview; therefore, a priority sub-set of draft survey questions must be selected or multiple guides developed and the sample expanded
Developing the cognitive interview guide	The cognitive interview guide includes survey questions, scripted probes and guidance on areas to explore through emergent probes	Researchers require guidance on which areas of the survey question to explore; they must also be encouraged to develop emergent probes throughout the course of the interview
Recruiting and training researchers	Researchers should be highly educated social scientists with prior qualitative research experience; quantitative survey research experience is desirable Researchers must be fluent in the local language, and, when relevant, the language of the broader team Ample time should be allocated for training, in order to cover: orientation to the purpose of the larger study, detailed instruction on cognitive interviewing, in-depth topic area teaching, question-by-question examination of the guides and role play	Cognitive interviewing is complex and quite different from mainstream qualitative research in terms of purpose, interview skills and debriefs. The field researchers drive the quality of the interviews and are fundamental in creating the final revised survey questions Linguistic issues are the most common problems with surveys that have been translated from English into regional languages. The researchers must be fluent in both languages in order to ensure nuance is captured across the translation, while adapting the language to local norms Extensive training is vital to orienting the researchers, including ensuring they understand the topics being assessed, the intent of each survey question, and how to carry out effective cognitive probing
Participant sample characteristics	Cognitive interview participants are from the same geographic area as the target respondent population Cognitive interview participants have similar socio-demographic characteristics to the survey target population Within the socio-demographic profile of the sample population, individuals with lowest levels of education, literacy and mobility and most marginalized should be prioritized	Cognitive interviewing enables local adaption and thus requires local participants who mimic the characteristics of the intended survey respondents Cognitive failures in the drafts survey questions are most efficiently and comprehensively identified by interviewing participants who are most likely to struggle with the material
Conducting interviews	Interviews should be carried out in pairs of trained qualitative researchers: one to conduct the interview and one to take notes throughout the interview	Notetaking is as important as leading the cognitive interview because debriefs and revisions of the survey questions depend on the notes taken during data collection. Notetakers must be as well trained and experienced as the interviewers
Debrief and analysis	Balance data collection with debriefing: Conduct a few (approximately 6 or 7) cognitive interviews and then allocate a day or more for debriefing and revision—do not gather a lot of data without time for reflection Multiple rounds of data collection should be conducted to test subsequent versions of the draft survey questions	Conducting a large number of cognitive interviews before pausing to debrief and revise the survey question is inefficient and impractical
Supporting quantitative survey enumerator training	Researchers who conducted the cognitive interviews should attend the survey enumerator training	Researchers who carried out the cognitive interviewing can explain the rationale for the final wording of the questions to the survey enumerations, provide locally grounded orientation to field realities (including local vocabulary) and help the enumerators anticipate the types of challenging responses they are likely to receive in the field