Table 5.

Focus group result summary.

AICFa function	Representable pros	Representable cons	Therapist quotes for pros	Therapist quotes for cons
Emoji check-in	Emoji check-in provided facilitators sufficient feedback on participant emotions to address the absence of visual cues. Emoji check-in produced less invasive but critical information than a distress warning. Emoji check-in helped bridge the gap between the postsession report and clinical outcomes such as engagement and distress.	Emoji check-in results were not shown in the session in real time, limiting facilitators’ situational awareness. Facilitators did not have control over the deployment of emoji check-in when needed.	…The emojis help address the lack of non-verbal feedback. [Therapist 3] …The emoji check-in helped provide more granular information regarding distress compared to distress warnings. [Therapist 3] …Sometimes the post-session reports don’t line up with the red bar or other analyses, however, emojis help address this gap. [Therapist 1]	…Facilitators can’t see the emojis during the session, so participants might feel ignored if their concerns aren’t being taken seriously. [Therapist 3] …It would be beneficial if we could deploy the emoji check-in when we believe it is appropriate. [Therapist 3]
Engagement score	Engagement score was helpful in identifying inactive participants.	The system could not show the typing of participants as an engagement indicator. If a participant left early, they would be flagged as low engagement. Text might be insufficient to show engagement as participants were thinking or feeling beyond what they could express in text. Other indicators such as heart rate might be complementary to the text-based analysis.	…The engagement score is really helpful to see who is actually inactive. [Therapist 1]	…I would love to see the participant typing. [Therapist 2] …Is there a way the system knows if the person has left early and is not just disengaged? [Therapist 1] …Engagement isn’t always shown through text. What someone is thinking or feeling beyond what text messages they are sending. [Therapist 1]
Distress warning	Facilitator felt they could provide more support to participants with distress warnings during the session. Distress warnings provided a reminder for the facilitator to follow up with the distressed patient.	Distress warnings could not help therapists to distinguish between healthy and unhealthy positivity, which refers to participants who refused to acknowledge their negative emotions. The distress warning score needed fine-tuning as most participants were usually above average. The distress graph needs to be simplified.	…I had a patient with a distress warning, so I directed the group to provide more support. I was really happy for the group support. [Therapist 1] …If I see the distress warning, it reminds me to follow up with them after the session. [Therapist 1]	…When a participant was showing toxic positivity, their messages were still read as “positive.” [Therapist 1] …The system needs improvement on setting an average, since most participants were above the red bar. [Therapist 1] …Make the distress graph easier to read. [Therapist 2]
Group cohesion score	The group cohesion score was helpful and is relative to other participants.	There were some discrepancies between the cohesion score and facilitator's judgement or experience concerning group cohesion.	…It is helpful that the group cohesion scores are relative to other participants. [Therapist 1]	…A recent session I facilitated had a red cohesion score, however, this feedback does not fit with my experience with the group. [Therapist 1]
Resources recommender	The recommender system could standardize the distribution of reference materials to participants to maintain group cohesion and fluidity. The recommender system was helpful to track participants’ progress on the reference materials and their ratings on the usefulness.	Facilitators preferred to read and add additional materials into the automated email content before sending to patients.	…I want everyone to read the same material, it can help improve group cohesion and fluidity. [Therapist 3] …It’s really handy to see if participants have opened and clicked on the material and I can see whether or not it’s useful. [Therapist 2]	…Sometimes there are resources I want to add, but I don’t want to send them another email on top of the automated email. [Therapist 2]
Dashboard	The conversation summary on the dashboard effectively summarized patient emotions and concerns.	The distress graph was visually overwhelming as it showed the status of all participants. It was recommended that only abnormal distress levels be shown during the session. Facilitators suggested the need for a more succinct summary with the most critical information.	…The conversation summary was useful to look at patients’ feelings and concerns during the session. [Therapist 1]	…It’s [distress graph] visually busy. Unless there is someone whose fluctuating out of the usual boundary leaves it out maybe, when in range it’s not too important to know. [Therapist 2] …The summary should only present the most important information and put the other details somewhere else. [Therapist 2]

^aAICF: artificial intelligence–based co-facilitator.