Table 2.

An overview of the methods and outcomes and reflection on these methods of the value specification.

Research goal	Method	Target group	Main results	Conclusions	Lessons learned
Generating multiple ideas on the use of VRa in forensic mental health care, based on the outcomes of the contextual inquiry	Idea generation—scenarios: In 3 sessions, all project team members individually brainstormed about ideas for VR applications. The 6 most promising ideas were worked out in a template (see Multimedia Appendix 1: 1.5) by multiple project team members. On the basis of these templates, scripts were written and 6 short videos were filmed	Not applicable	A short video was created for each of the 6 ideas. All videos had the same underlying structure: the goal of VR, its use during treatment, an example, and the desired outcomes. The videos (with English subtitles) can be watched on YouTube [38]. An example of a scenario can be found in Multimedia Appendix 1 (1.6)	The videos made clear that there are a lot of promising possibilities for VR in forensic mental health, so it appeared to be necessary to make decisions about what to prioritize and why	The structured approach in which multiple templates were used worked well in this project: it forced all different members of the project team to work and think in a similar way. Each member of the project team had a clear role with individual responsibilities. This was experienced as helpful in motivating the team members and ensuring that all of their perspectives were present in the 6 ideas. Creating scripts and videos was very time-consuming, so motivated members who are willing to invest time and effort and enough budget were necessary preconditions for making videos.
Identifying (1) the preferences of stakeholders of the 6 ideas and (2) the stakeholders’ values regarding VR in forensic mental health care	Web-based questionnaire: After asking sociodemographic questions, the 6 videos were presented to the participants in random order. After each video, the PIIb [39,40] a question about the participant’s grade for the idea, and 3 open questions on positive points, points of improvement, and suggestions for the idea were provided	Patients (n=19); therapists (n=89); other stakeholders (n=38), such as parole officers or researchers from different Dutch forensic institutions	There were no significant differences between the grades and PII scores for ideas. A broad range of positive and negative aspects and remarks were identified via inductive coding. These can be found in Multimedia Appendix 1 (1.7)	The results of the questionnaire were mostly in line with the interviews but provided more detailed and specific information, for example, how VR should be personalized and which skills should be trained	The answers of the patients fitted the research questions of the questionnaire better than the answers that were given by patients in the interviews. This indicated that the concrete, scenario-based videos were a better way to include the patient perspective than the broad, abstract interviews. Although the goal was to make this method less time-consuming, filling in the questionnaire still took about 30 minutes, which might explain why a large share of the participants (55.4%) did not fully complete it. A shorter questionnaire might have led to more response but also would mean that less information would have been retrieved. The quantitative measures indicated no major differences between opinions about ideas. Although it was not clear if this was an issue regarding validity or if there actually were no differences, it was still useful to ask for a grade for each idea. The PII was not of added value in this questionnaire. Although this method proved to be useful to further specify previously found results, it would not have been suitable as an initial method to gather in-depth information, partly because no probing questions could be asked, and answers were relatively short.
Formulating values that capture what the added value of the technology should be for people and context, according to the stakeholders	Value formulation: On the basis of all previous results, 2 researchers created attributes that summarized the needs or wishes of stakeholders [17]. On the basis of categories of related attributes, accompanying values that stated what VR should achieve, improve, or add according to the stakeholders [6,17] were formulated. The values were discussed by the project team and minor adjustments were made accordingly	Not applicable	A total of 43 attributes and 13 values were formulated. An example of how a value was created can be found in Multimedia Appendix 1 (1.8). The following values were formulated: fit with patient; improvement of skills; insights into behavior, thoughts, and feelings; bridge between treatment room and practice; generalization of skills to daily life; safety; treatment motivation; unique addition to current treatment; ease of use within treatment; cooperation between patient and therapist; wide applicability; affordability; and constant adaptation of the application	Formulating values proved to be a very good way to get to the point and summarize the essence of the results so far. It forced the project team to critically think about the overall added value and goals of the VR app and prevented them from getting lost in details or a tunnel vision	Values might be difficult to understand for outsiders as they are abstract, concise summaries of the needs and wishes. Consequently, clear definitions of the values were provided to prevent misunderstandings. Besides their importance for development, the project team determined that values could also be useful to determine what to evaluate: to what extent was the added value actually achieved in practice? This way of thinking about values allowed the project team to think ahead in terms of implementation and evaluation and facilitated a broader view on the VR application. In hindsight, the process of formulating values was more complex than expected. The project team had to account for the results of all used research methods, combine them in an abstract way, and make decisions about conflicting values, such as the importance of visual realism. A clear guideline for formulating values would have been useful.
Generating a concept for a VR application based on the values and previously gathered results	Idea generation—prototyping: The project team discussed the values, attributes, and outcomes of all research activities and their implications for a VR application. Via multiple brainstorming sessions in which multiple low-fidelity prototypes were created, a first version of an idea was developed	Not applicable	The main goal of the VR application was to support therapists and patients in identifying triggers that can elicit undesired behavior and search for helpers that can support the patient in dealing with these triggers. Patient and therapist can together build personalized scenarios via a dashboard with several categories that contain elements that can be added to a scenario (see Multimedia Appendix 1: 1.9 for the prototype)	The developed concept was a combination of elements of all 6 videos that were created by the project team. Also, important concepts that already arose from the interviews were present in the idea, for example, personalization, skills training, and new insights	To ensure the consistency of the development process, the idea generation process started with discussing the implications of all earlier conducted studies, even though it was more appealing for the project team to start creating the idea right away. Visualization of ideas via low fidelity (lo-fi) prototypes appeared to work well during the idea generation process to make abstract concepts more concrete. For example, the team drew multiple dashboards and visualized the structure of the dashboard with post-its. This was experienced as helpful by all members of the project team.
Investigating (1) how far the stakeholders’ opinions of the concept match the previously formulated values and (2) if changes to the concept are required for it to optimally fit the stakeholders’ preferences	Interviews—idea finalization: In the first part, open-ended questions, based on an adapted version of the TAMc [41], were asked to check the attitudes toward the concept of the VR application. The second part focused on the participant’s overall opinion of the VR application. The developed low-fidelity prototype and a scenario on its use in treatment were used	Patients (n=10) and therapists (n=8) from all different locations of Transfore, the forensic hospital	The first part was coded deductively using the constructs of the TAM (see Multimedia Appendix 1: 1.10), the second part was coded deductively with the 13 formulated values (see Multimedia Appendix 1: 1.11). Overall, the idea was evaluated positively, but there were some concerns about the ease of use of the application. All values were, to some extent, present in the participants’ answers. Most positive remarks were about the added value for treatment, for example, fit with patient and new insights. Points of attention were related to the implementation in treatment	Overall, the idea fits the values of the participant, mostly with regard to the unique added value to treatment. No major changes to the basic idea were necessary. In later stages, attention should be paid to the usability of the application, training, and protocols to successfully embed VR in treatment	This second set of interviews was considerably shorter than the first one: they only took about 15 to 20 minutes. It proved to be easier to include patients, which might be because of the relatively little time that was required to participate. Using the values to code these interviews was useful to determine the positive and negative aspects of the idea in relation to the added value that it should have had. In this way, it became very clear what the points of improvements were, which might not have been the case with an inductive, bottom-up coding process. It also allowed the project team to check whether the idea was still in line with the values. The TAM was used in the interview scheme and coding process. Although it helped to structurally ask about and analyze the participants’ attitudes and intentions, it provided hardly any information about the treatment context and characteristics of (other) persons [42,43]. The second part, in which the added value in general was discussed, appeared to be necessary to paint a full picture of the participants’ opinion. Merely using the TAM would not have sufficed in this interview.

^aVR: virtual reality.

^bPII: personal involvement inventory.

^cTAM: technology acceptance model.