Table 2.

Examples of benefits and limitations of using Blackboard Collaborate.

Presence factors		Specific examples
Benefits	Anonymity	Moderators or test users might be more willing to share honestly or critically if less visible/identifiable.
	Test users have a sense of control	Moderators or test users could mute the session if they wanted to limit noise.
		Moderators could employ the “private chat” feature.
		Test users have control of when and where their session was held (some test users completed testing at home or at work).
	Authentic use in context	Test users could mute the session for reasons such as: check on children, take a phone call, speak to a coworker, eat.
		By having an unstandardized testing approach, the teams were given insights into the nature of technology use in people’s everyday lives and routines. This was valuable information about how the eHealth interventions being tested might also be used.
Limitations	Lack of visual cues, “personality,” or human element in the virtual space	The lack of visual cues led to moderators feeling they were checking in with the test user more than necessary. If there was silence, or no movement on the screen, moderators couldn’t be sure if the test user was done or just attending to another task.
	Quickly establishing rapport and relationships	At times, moderators experienced anxiety about getting technical problems solved quickly to reduce test user stress and to ensure not too much testing time was taken up by technical problems. In Project 1, the moderator was on the same campus as some of the test users and was requested to come in person to the test user’s office to set up the audio prior to the test session.
	Concept of time	The technical setup took longer than anticipated, so at times moderators felt rushed for time to complete usability tasks.
		There was no “clock” tool to help provide test users or moderators with cues about how much time a task had taken.
	Surveillance	Moderators’ virtual presence was constant and all-encompassing. Test users’ every click was monitored and every task was recorded. Moderators felt that the testing context might have led to feelings of being surveilled, obligations to have opinions, or pressure on test users to perform as expected.