Table 4.

Examples of feedback on aesthetic, content and learnability elements of Breathe

Elements	Negative feedback	Positive feedback	Changes applied
Layout	“It took me a couple of questions to realize that I wasn’t answering the question below. So that may be something you just want to either get rid of the white spacing or insert the spacing after the answer.” (Y01) “I think there’s almost too much information on this one page… Maybe if that page was broken down into three or four other pages, like that section right there being one page.” (C06)	Yes I’m very impressed with the layout of this… it’s concise and the information is good… I like how there are pictures and if you click the picture there’s lot of info. I found that really cool… it was quick and easy.” (Y03) “Yes, it was very good…kind of leaving some room for just self-directed user discovery which is always kind of fun, especially with sites designed intuitively like the Breathe site is. The layout is very simple and minimalistic, and I can definitely kind of see persistence, kind of enjoying the experience of learning how to use the site.” (Y02)	Converted larger chunks of text into bulleted lists. Limited amount of scrolling users would do. Broke long sections into multiple pages. Created more open text boxes and ‘other’ category for users to express ideas Allowed users to go back over previously completed pages more easily
Audio/Visual	“I also don’t know if you want to stick any images in here because from just my immediate viewing of the page… it just appears as a giant block of text, you may want something to break it up a little.” (Y01) “The voice I found a little bit on the quiet side. It was kind of that background quiet soft hiss… I wondered if the background voice could be louder, or the person speaking, if that could be louder and more crisp?” (C07)	“I kind of like the fact that you recycled the videos of raw images… because it gives a sense of continuity in that each situation can be approached from different angles.” (Y01) “I think [the video] was really interesting because it introduced you to a real person going through something that eventually other youth are going through.” (Y03)	Rerecorded and/or slowed audio recordings for videos. Added visual assets (i.e., photos, graphics, icons and diagrams) to break up page layout.
User-friendliness	“So I can actually print off this page? Okay, because I’m not sure what to do, now that I have finished this log, I’m not sure what I’m supposed to do.” (C08) “There’s just a lot of different things that could be under, what did you do, and I don’t think you can cover all of them, but maybe try to balance them a little bit more, putting in another box so they could type it in.” (Y04)	“One thing I liked was things to try out. I like that it gives you a choice between the two because it’s not always fun to visit one or two things, I like having the choice between the two.” (Y02) “No, nothing looks confusing. Everything looks pretty clear, and maybe that’s why I read the one message and then jumped right in sort of thing. Because it looked pretty easy to use.” (C06)	Modified the tutorial video to provide overview of steps in each module. Changed words on some of the buttons (‘submit’ changed to ‘save’) Drop down menus were added for self-evaluation tasks and open-ended text boxes for adding additional information.
Completeness	“Just a question about this. This might be too complicated, but we have 9-1-1, but is there an ability to link to local hospitals or numbers?” (C07) “I know certainly from personal experiences…it was hard for me to get started, simply in the fact that I didn’t see results immediately. I think you’ve certainly done a good job, but maybe if you say that they actually have to insert some sort of progress report before they receive the next module.” (Y01)	“The thing that impresses me is how comprehensive it is. It’s touching all the bases, so I think that’s a huge advantage. And it’s kind of laid out in a systematic way, so they can do this kind of progressive work through it. And I would think with each module they’re going to feel some benefit, and that’s going to reinforce their motivation to continue.” (C05) “I think it’s nice and brief and to the point, and I do like how it’s showing what you answered the week before. I think it helps put things into perspective… if their moods fluctuate day by day and they are not thinking about success that they had the past week.” (C09)	Created lists of mental health organizations and providers to be filtered by geographic location of user. Provided options for completing extra optional practice activities. Created a Progress Summary graphic to appear at the check-in section of each module.
Relevance	“it seems kind of geared toward youth who are at the start of looking at their anxiety or learning about it, not so much maybe for someone who’s been in counselling for a while or whose already learned a lot about that stuff.” (Y04) “you have to compete with everything else that’s on the Internet and on TV and that’s tough, in order to hold their interest… you almost have to disguise the information that you’re giving them, or the help that you are trying to offer.” (C06)	“It was much more engaging simply because of the use of your images, of the text boxes, of the actual questions. The actual situations you’ve put up are very relatable.” (Y01) “The video is excellent. That’s the comment. That’s what I literally do with teenagers who come in. Yes, and showing it from the side is key because they don’t always understand that. It’s an excellent technique and the explanation is really good too.” (C07)	Rerecorded audio with different voices (e.g., boys and girls speaking). Made some activities optional or fewer “required” activities to be completed Layered some content into image maps, pop down text boxes etc. so youth could decide if they wanted/needed more information
Credibility	“I’m not sure if this would be possible to attach the University of Alberta’s name to the Website or whatever, department or faculty you guys and your research team is based out of…to really communicate to youth that this isn’t just some little thing that just some random person put together somewhere. It is something that people have put a lot of thought and planning and work into, that it’s been evaluated and tested will really help.” (Y02) “I’m thinking about the introductory video… I think I’d be a tad more explicit that there’s lots of documentation that these interventions, these principles are highly effective, so we can say with confidence that you should indeed make progress… and it’s really a function of how much effort you put into this.” (C05)	“I think it’s good that you emphasize the confidentiality of it all. I feel like with the confidentiality they can really express what they’re thinking.” (Y03) “yeah they’re getting real < <emphasis> > answers. I just know there’s a lot of ask the experts sort of things where it’s blah, blah, blah, blah, a little blurb, follow-up with your family doctor, blah, blah, blah…”(C09)	Added “Our Team” page to the site, including photos, bios and logos from all participating institutions) Created a tutorial video about the main features of CBT and research evidence of its effectiveness.