Table 2.

Overview of qualitative approaches to assessing engagement with considerations and example questions.

Qualitative approach	Description	Example items	Considerations (pros/cons)
Semistructured interviews	Provide an opportunity for sharing of lived experiences and feelings to uncover concealed perceptions related to digital health intervention or the technology; includes informal conversational interviews (spontaneous-suited to ethnographic research), semistructured interviews (interview guide used to steer otherwise spontaneous conversation), or standardized open-ended interviews (worded questions used for all participants).	Microlevel: Tell us what you think about the content; How did completing that module make you feel?; Please explain your pattern of use?; Why did you log on when you did?; Macrolevel: Did you notice any change to your thinking as a result of using the …(“app”)?; What impact did using the ... (“website”) have on how you are going about changing your behavior?	Pros: inform modifications to increase acceptability, interactivity and tailor to end-user needs; identify a range of issues associated with use (both short and long term); augment interpretation of quantitative evaluation; generally small sample sizes. Cons: subject to bias (eg, recall and social desirability), especially if leading questions are asked; time consuming to collect and transcribe; time consuming to analyze and often requires more than 1 person to decide on and confirm themes.
Think aloud	Aim to capture the experience of using the technology in real time. The user is provided with a specific task to complete and is observed while they perform the task. The user is prompted to think aloud throughout the process.	Microlevel: Tell me what you are thinking; What are you looking at?; What’s on your mind?; How are you feeling?; Why did you click on this?; Why did you frown/smile/sigh?; Macrolevel: Are you learning anything new?	Pros: can be used at various stages of development and implementation to understand how intervention features impact on engagement; occurs in real time, so less subject to recall bias. Cons: subject to observer bias; can be cognitively difficult for participants and requires practice; may require additional resources such as video or sound recording equipment to obtain a comprehensive picture. Acquired data can be time consuming and complex to analyze; may be most useful for exploring microlevel engagement.
Focus groups	Used to identify the social and contextual factors in specific population subgroups that influence engagement with digital health intervention and needs for technological characteristics and operations that promote user alignment and functional utility.	Microlevel: What did you think of the intervention?; Which components caught your attention the most?; What about them caught your attention?; Were there any components that caused frustration?; Did any aspects make you feel guilty? Macrolevel: How often did you think of the intervention during the week?; Was the intervention in the back of your mind?; How did the intervention help or hinder you reach your goals?	Pros: allow for spontaneous discussion of topics and subsequent voicing of ideas and perceptions that may go unnoticed in semistructured or structured interviews; Can obtain rich data from multiple people at the same time. Cons: subject to group or social desirability bias; some participants may not express themselves as fully in a group situation; requires practice to manage group discussion; can take a long time to transcribe due to interruptions/butting in; time consuming to analyze and often requires more than 1 person to decide on and confirm themes.