Table 1.
Technique | Brief description | Example for SITBa research | ||||
Formative evaluation | ||||||
|
Elicitation-focused techniques | |||||
|
|
Card sorting | Sort a set of cards with constructs relevant to the intervention into groups that make sense to the end user | Individuals with lived SITB experience sort emotion regulation features in terms of the most to least important for SITB management | ||
|
|
Concept mapping | Create an illustration mapping the relationship between the research question and related concepts | Individuals with lived SITB experience create a pen-and-paper map of emotions, thoughts, and contexts and how these factors relate to certain coping strategies | ||
|
|
Diary or EMAb studies | Prompt end users to share relevant details about their perception or experience at scheduled or random intervals over time | Individuals with lived SITB experience are prompted 3 times daily to understand contexts or environments associated with heightened suicidal or self-injurious thoughts | ||
|
|
Focus groups | Stakeholders meet for a moderated discussion related to the research question | Clinicians, parents, and young people with SITB are gathered to discuss day-to-day needs | ||
|
|
Interviews | End users meet with the researcher for a structured or semistructured interview | Interviewing individuals with current SITB to understand how they conceptualize and engage in SITB self-management and identify needs [28] | ||
|
|
Personal inventories | End users describe and share artifacts that are personally valuable or relevant in their life | Individuals with lived SITB experience are asked to bring to an interview session the personal items that provide them hope or comfort when times are difficult | ||
|
|
Task analysis | Identify steps that the end user takes to complete a task | Individuals with lived SITB experience are asked to access and practice coping tools in an app | ||
|
Design-focused techniques | |||||
|
|
Co-design workshops | Designers, end users, and stakeholders meet to collaborate on the design of an intervention | Individuals with lived experience of SITB and clinicians meet with the purpose of ideating and designing prototypes that will help end users when they experience urges to self-injure [29] | ||
|
|
Passive storyboards | End users are presented with scenarios—or narratives describing a set of contexts and actions—and probed for how they would engage or navigate the scenario | In the presence of qualified practitioners, individuals with lived SITB experience are asked to reflect on a situation when they experienced an urge to self-injure and asked how they managed the urge and what resources they would usually use | ||
|
|
Interactive storyboards | End users are presented with a narrative of the intervention in a series of panels and participate in the narrative while being probed on whether the product or solution meets requirements | Individuals with lived SITB experience are provided a visual narrative of a proposed intervention that aims to intervene in moments when suicidal distress occurs and are asked to consider its acceptability through each interaction | ||
|
|
Blue sky prototyping | Considering designs that were previously ignored because of feasibility (eg, cost, risk, or complexity) to encourage radical innovation and creativity | Through a focus group, clinicians are provided with an implementation plan for a suicide prevention intervention that they believed had liability concerns. Rather than disregarding it, they brainstormed implementation strategies to mitigate their concerns | ||
|
|
Parallel prototyping | Multiple design concepts are embodied and compared concurrently. Parallel prototyping can help provide critical feedback for concept selection | Individuals with lived SITB experience are provided with similar variants of prototypes that differ by look and feel and are asked to describe their perceived acceptability in moments of suicidal crisis | ||
|
|
Wizard of Oz-ing | A prototype to simulate the full functionality of the intervention with a human operating behind the scenes so that all interactions seem to be computer-driven | Individuals with lived SITB experience are provided with an app prototype and are asked to interact with the app while a researcher navigates display behind the scenes | ||
Summative evaluation | ||||||
|
Usability testing techniques | |||||
|
|
Cognitive walkthrough | End users are presented with an intervention design and asked to show how they would use it based on design features and prompts within the intervention | Individuals with lived SITB experience are presented with screenshots from a web-based program designed to teach distress tolerance and are asked to describe how they would navigate the screen based on prompts | ||
|
|
Think-aloud exercises | End users engage with the intervention and are asked to speak aloud when they complete a task | Individuals with lived SITB experience interact with an avatar-and-text interface and are asked to speak aloud when they make decisions about how and when to engage [30] | ||
|
|
Heuristic evaluation | Designers or researchers evaluate prototypes to identify potential usability problems based on heuristics | Clinicians are invited to evaluate early prototypes to determine whether the intervention would be useful and usable for patients with SITB [31,32] | ||
|
|
Laboratory-based usability testing | End users identify features or interactions that are not usable and are in need of refinement while using the product in a laboratory setting | Individuals with lived SITB experience are brought into the laboratory to engage with the DMHIc to identify issues | ||
|
|
Field testing | End users engage with a prototype in natural context (eg, daily life and inpatient unit) | Individuals with SITB use the DMHI over the course of the planned length of an intervention (eg, 8 weeks) and respond to usability measures across this period |
aSITB: self-injurious thoughts and behaviors.
bEMA: ecological momentary assessment.
cDMHI: digital mental health intervention.