Box 2. Pretesting interview discussion template
Introduction to the choice experiment
This is the introduction to the next part in the survey. Do you mind reading this aloud?
Can you read the description out loud? Is anything unclear?
Do you have any issues or questions?
Review of attributes/levels (one attribute at a time)
Can you explain the attribute and its levels in your own words based on what you know so far?
Are the description of the attribute and its levels clear? Would you make any change to how they are described?
What do you think about the levels of the attribute presented? Are they too similar or too different?
Do you think the levels for this attribute are presented in the right order?
Example task
Can you explain to me in your own words what you think is being asked in this task?
Is the choice we are asking you to make clear? Can you imagine the scenario we are describing here?
Do you think you could answer this question?
Was this example task useful? Do you feel prepared to do the next task on your own?
Choice tasks
Can you think aloud for me as you review this choice task?
Do you like either of these profiles better than the other?
If so, can you explain why?
Was there any tradeoff you had to make in choosing that profile?
Was there something you like better in the profile you didn’t choose?
Was there anything in the profile you chose that was not ideal?
Were any of the attributes more important in your decision making than others?
Is there any information outside of what we’ve presented here that factored into your choice?
Should there have been an opportunity to choose neither profile? As in, to opt-out of making a choice?
How many of these tasks could you do before getting tired? Is there anything we could change to make it easier to do more of these tasks?
[If presenting multiple choice task formats] Which of these formats do you like more?
Debriefing
How consistent were your answers with your preferences?
How easy were the choice tasks to understand?
How easy were the choice tasks to answer?
How long do you think it would have taken you to complete X tasks?
Format and structure
What kind of device do you usually use to take surveys (e.g., phone, computer)?
If by phone, what orientation do you typically use when taking surveys on your phone? Would you flip it to portrait to accommodate a wide choice task? Or prefer it to be up/down?