Usability and Navigability of an HIV/AIDS Internet Intervention for Adolescents in a Resource Limited Setting

Abstract

Use of Internet is growing in Sub Saharan Africa. Evidence of computer and Internet effectiveness for reduction in risk behaviors associated with HIV shown in U.S. settings has yet to be replicated in Africa. We describe the development, usability and navigability testing of an Internet-based HIV prevention program for secondary school students in Uganda, called CyberSenga. For this work, we used four data collection activities, including observation of (a) computer skills and (b) navigation, (c) focus group discussions, and (d) field assessments to document comprehension and usability of program content. We document limited skills among students, but youth with basic computers skills were able to navigate the program after instruction. Youth were most interested in activities with more interaction. Field-testing illustrated the importance of using a stand-alone electrical source during program delivery. This work suggests delivery of Internet-based health promotion content in Africa requires attention to user preparedness and literacy, bandwidth, Internet connection, and electricity.

Introduction

The use of computers for health promotion has been growing in the U.S. for more than a decade.¹ Although trend data aren’t available, recent reports suggest it’s popularity in developing country settings as well ². Ybarra et al. (2008) demonstrated that 38% of youth surveyed in Mbarara, Uganda, a rural community in East Africa, had accessed the Internet to seek health information.² Groundbreaking work in the United States shows the Internet can be used effectively to improve healthy sexual behaviors;³ we have yet to replicate these programs on a large scale, and to do so in a financially resource poor context.

Literature supports the need for careful technological assessment and beta-testing of technological programs to ensure the intended audience can indeed access and use them; that the programs function as intended; and that audiences appreciate and engage with interactive program elements.^{1, 4} In this paper, we report findings from our development work for the CyberSenga program, an Internet-based comprehensive sexuality education program for youth in secondary school in rural Uganda, East Africa. The Senga in western Uganda is the name given to the paternal aunt within families, generally considered responsible for advising the girls as they come of age on issues related to running a household, including sexuality and sexual health. Given economic and political pressures in Uganda that have lead to migration to large cities, extended families in rural areas are diminishing, and youth no longer have universal access to a Senga. CyberSenga was conceptualized as one way to integrate a culturally salient symbol into an innovative program that takes advantage of growing Internet access.

The content in the CyberSenga Internet based program is tailored to the user’s gender (male or female) and sexual experience (i.e. those who have had sex receive different content than those who have not). Six modules, lasting approximately one hour each, were created based upon the Information-Motivation-Behavioral Skills Model of HIV Preventive Behavior ^{5, 6}: 1) Information about HIV/AIDs; 2) Problem solving and communication skills for negotiating safer behavior; 3) Motivations to be abstinent versus sexually active; 4) Skills for correct and consistent condom use; and 5) Components of a healthy relationship. Module 6, originally planned as a comprehensive review, was ultimately delivered as a “booster” session two months after completing module 5. The “booster” had no new information, but rather summarized and reviewed the information from the previous 5 modules. This paper offers a description of the methods utilized to develop the program: (a) observation of computer skills, (b) alpha test for program navigation, (c) wireframe focus group discussions, and (d) field assessments of the logistics and program design.

Methods

The study takes place in Mbarara, the sixth largest municipality in Uganda. Mbarara is about 250 kilometers southwest of Kampala, with a population of 69,000 (based upon the 2002 census) and is the 6^th largest urban center in Uganda ⁷, although population density is low. Mbarara municipality serves mainly a rural population.

Participants were students from five partner secondary schools. One is an all-girls Catholic school; two are all-boys government schools; one is a mixed-sex Muslim school, and one is a mixed-sex government school. In most cases, data were collected with all or some members of the CyberSenga Youth Advisory Committee (YAC). The YAC consisted of peer-nominated popular opinion leaders identified in a quantitative survey conducted at each of the five partner schools. One student was recruited in each of the four grades at each school, resulting in 20 members. The YAC was convened multiple times with the overarching goal of offering iterative and detailed information on acceptability, usability, and saliency of the CyberSenga program content. Research staff were trained as a group to observe each development activity, and staff were debriefed as a group after each day in the field to document their experience. Specific recruitment detail for each development activity follows.

Computer skills

To test computer literacy, we invited 20 secondary school youth randomly selected from our five partner schools. Because we anticipated lower skills to be represented among lower grade youth, the recruitment focused solely on students from Secondary 1 and 2 classes. Research staff asked participants to turn on the computer, and use he computer mouse to open a web browser and navigate to three sites offering information about HIV/AIDS. They also were asked to create a free e-mail account, and then navigate to a threaded discussion group and post a comment there. Finally, research staff asked youth to self-administer a three-item survey.

One observer viewed each participant as they completed these tasks, and documented their skill level on a scale (based on a scale of 1–3, where 1=task was extremely difficult and 3=task was very easy) on the observation log. The observer also noted any specific problems or comments of the participant in completing these tasks.

Alpha test for program navigation

On a different occasion, 13 YAC members came to the CyberSenga office and reviewed an alpha version of the CyberSenga module (i.e. one with anticipated elements of the program, but without all the functionality present in the beta-version) about Problem Solving and Communication Skills related to sexual behavior. Using a talk-aloud procedure, participants were asked to navigate from one page to another, and to verbally express their reactions and experience in a stream of consciousness fashion, while research staff seated near them documented challenges they faced on each page and reactions to each page. Research staff documented problems encountered (e.g. program will not advance; computer freeze; power outage), and perceived reactions to content (e.g. boredom, confusion, possible literacy issues). Research staff also documented time to complete each module and a participant assessment of how engaging and interesting the module was (using a scale of 1–5, where 1=not at all and 5=completely), and whether the participants would recommend the module to a friend.

Wireframe Focus Group Discussions

Fourteen YAC members came to the CyberSenga office to participate in one of two focus groups (one for each biological sex), where we showed preliminary sketches of the CyberSenga content for four program modules. These preliminary sketches are called wireframes; these are the first mock ups of interactive content ideas (see Figure 1 for examples). These wireframes were meant to show students the content and general flow for each section. Focus group discussions centered on how easy scenarios were to understand, and salient and familiar scenarios were, and what improvements might be made to the design and the content.

Figure 1.

selected wireframe examples and screen shot from final program

Field assessments of program logistics and usability

For the fourth and final activity, two field assessments were conducted: the first to test the logistics of the RCT; and the second, to test a beta version of the CyberSenga program. Fifty-three randomly identified students took part in the first (logistics) field test; YAC members participated in the second (beta) test. We randomly recruited an additional 15 students to replace YAC members lost to attrition due to graduation to achieve our target sample size of 20. All students completed the modules on netbooks at their schools. Research staff observed students, identified any technical challenges (e.g., internet connectivity, modules not properly loading), and assisted them with navigating the modules as needed (e.g., helping with logging in). Students completed process questions querying: 1) level of privacy; 2) perceived helpfulness of staff; and 3) adequacy of time to complete the lesson. Students were also asked to indicate: 4) one thing most liked and 5) disliked about the module; and 6) any confusion they experienced. Research staff also noted perceptions of any student navigation or comprehension difficulty.

Results

Data on participant demographics for the YAC members in each activity are shown in Table 1.

Table 1.

Number and demographics of participants in each CyberSenga formative activity

	Computer Skills Testing, N=10		Navigation Observation, N=13		Focus Groups, N=14		Field assessment, N=20
Demographic	Number	% of total	Number	% of total	Number	% of total	Number	% of total
Male	7	70%	8	61.5%	5	35.7%	15	75%
Female	3	30%	5	38.5%	9	64.3%	5	25%
Senior 1	5	50%	3	23%	none		5	25%
Senior 2	5	50%	5	38%	5	35.7%	7	35%
Senior 3	None		2	15%	6	42.8%	8	40%
Senior 4	None		3	23%	3	21.4%	None
Day	3	30%	1	8%	1	7.1%	6	30%
Boarding Student	7	70%	12	92%	13	92.8%	14	70%
Muslim	1	10%	3	23%	4	28.5%	2	10%
Catholic	5	50%	4	30%	4	28.5%	8	40%
Protestant	4	40%	6	46%	6	42%	10	50%

Computer skills

Results from the assessment of basic computer skills are shown in Table 2. The majority of participants could perform each task with ease and no confusion. There was variability however, with some (n=3) participants unable to turn on the computer, launch a web browser, or complete an online survey without direct assistance.

Table 2.

Basic Computer Skills among CyberSenga Audience (N=20)

Skill	Mean (mode) skill*	% completely capable
Turn on power source	2.1 (3)	50
Launch a web browser	2.6 (3)	40
Access Internet	2.5 (3)	70
Find three web sites w/HIV/AIDS information	2.3 (3)	50
Sign up for a free e-mail account	2.7 (3)	80
Post a message to a message board	2.9 (3)	90
Self-administer online survey questions	2.3 (2)	40

^*using a scale of 1–3, where 1=no skill and 3= completely skilled

Alpha test for program navigation

As shown in Table 3, the challenges documented by observers included initial difficulties with navigation where participants seemed to be confused about how to advance to the subsequent page; this diminished once participants got through the first several pages. Observers documented cases where material was confusing; participants noted that they were unsure about what they were supposed to do on a page, either because they did not understand instructions that were offered or those offered did not coincide readily with the activity on the page. They documented instances where content was challenging due to linguistics or literacy. For example, two of the participants were confused about stages in problem solving (the introductory part) and they were not able to comprehend the meaning of the some words (e.g., “determine” [the problem]; “evaluating” [the possible solutions]). The amount of text itself also seemed to be a barrier, with students becoming visibly tired. On the other hand, opportunities for engagement with the content, such as when youth could accrue points for correct answers to questions seemed to be especially well liked by youth.

Table 3.

Navigation skills and challenges among Youth Advisory Council users of CyberSenga Beta-modules

Skill and Challenge	Description	Percent occurring (N) (of 52 pages)
Navigation	Most participants who experienced problems with navigation exhibited these in the first several pages	36 (19)
Page loading or freezing	Participants experienced slow internet connections that could time out	9 (5)
Power outage	Participants experienced a total loss of power	0
Material too simplistic	Participants expressed boredom or criticism that material was too basic or overly simple	1 (2)
Material confusing	Participants expressed confusion over content; lack of understanding of what they were supposed to do on the page	30 (16)
Material passed by too quickly	Participants indicated the page loaded too quickly and moved forward before they were ready	13 (7)
Material passed by too slowly	Participants indicated the page loaded too slowly and they were ready to move on before the program did	5 (3)
Material difficult to comprehend	Participants had difficulty understanding a word, phrase or concept	30 (16)

Wireframes Focus Group Discussions

For all discussions, participants were shown wireframes or drafts of the elements intended for each module in the program and asked to respond to them. Examples of what participants reviewed are shown in Figure 1.

Problem solving segment

Participants were shown the problem-solving segment from Module 2. To help youth internalize the steps related to problem solving, a problem someone their age might face is presented. Participants were asked to react to the realism of the problem and potential solutions and whether a different problem might seem more salient to the participants. The problem presented to both groups was the same: the youth characters in Cybersenga, Eunice and Moses, had decided to have sex; they want to use condoms, but they do not know where to get them.

The boys agreed that this was a common problem faced: “These problems are very common and the characteristics of adolescents is shyness; they always find it hard to get these condoms”. Girls, on the other hand, thought that the issue was less about where to get condoms, and more about whether or not to have sex in the first place: “No I don’t think it is the real problem because most times adolescents do not mind where to get them [condoms]…they mind about whether they should have sex or not.” The girls’ group agreed however, that it would be difficult to get condoms: “I think they may find it difficult to go and get [condoms] …they may find it hard to know where to get a condom.”

Generally, youth felt the module was helpful in building their skills to solve any problem, not only those related to HIV prevention. One participant, responding to the opportunity to develop skills in identifying and solving problems, said, “I have learnt personally that you have to first identify a problem then take your time to evaluate the solutions unlike most of us who just pick the solutions the way they want them.”

Communication skills segment

To teach youth about different types of communication, also from Module 2, users choose conversation starters and responses that are assertive rather than aggressive or passive. Users learn that passive communication is often met with passive communication; assertive with assertive; and aggressive with passive or aggressive communication. Participants reacted to this section by asking that we emphasize strategies for how to avoid passive language. They indicated that adolescents are often passive, and expected to be so: “I think it relates because most of the adolescents do not speak to the point….and it makes them to have quarrels here and there.” Both groups thought felt girls particularly needed strategies for assertive communication to avoid feeling overpowered in the relationship if they want to use a condom.

Choosing between abstinence and sex

Participants were shown a segment aimed at exploring the motivations for being abstinent versus sexually active. The ‘good’ and ‘bad’ things associated with each choice were depicted on puzzle pieces. The user is told to put the good things together, and the bad things together; thus creating two separate puzzles. Once complete, the Senga observes that the puzzle for good things about abstinence is bigger than the puzzle about bad things; and vice versa with respect to having sex. The intention is to acknowledge that there are benefits and drawbacks to each choice, but that there are more benefits and fewer drawbacks for abstinence.

Participants responded positively to the exercise. One girl explained: “Me, I think that although there are good things about playing sex I think it shows that there is a reason to abstain because the bad things are many.” There was an appreciation for the creative approach to demonstrating a complex concept: “It is creative because it is a good way because teenagers they do not like reading many words but since in this one is…a game I think it may help them.”

Condom acquisition and use segment

In this segment, participants were asked to read testimonials of ‘people like them’ aimed at norming the use of condoms. The boys’ group reiterated the thought that it is difficult for young people to get condoms: “It is a very small percentage who can do it”, and indicated it is more common to get condoms from friends than from a shop or a clinic. The girls’ group instead thought that even if you use a condom the first time, it is difficult to sustain condom use in the relationship: “Yeah I think it is realistic because when they tell the adolescents that use a condom, they will be able to use a condom but the second time they will be like, the first time I did not know him but now I know him without even testing, so they go on to have sex without one and they end up having STDs”.

Healthy relationships section

In this segment, participants were shown a Baobob tree that would grow or wither based upon whether healthy or unhealthy relationship behaviors were chosen. Multiple scenarios were shown to the right of the tree (e.g., “Giving your girlfriend a mobile phone so that she will play sex with you”). When placed on the tree, it would grow or whither depending on the type of behavior. Participants felt the picture was a good way to illustrate the effect of positive and negative behaviors in a relationship: “if you pick a solution that is good and you see the tree growing and if you pick the bad solutions and you see the tree shrinking I think this can make you to choose the right thing.” Participants were able to offer additional suggestions for examples of unhealthy actions in relationship (e.g. boys said they could deceive a girl by telling the girl they are too young to impregnate them). See the Figure for a draft page from this module.

Field assessments of program logistics and usability

Transportation and scheduling

Our original plan to create a cybercafé and have students come to the café (located at the CyberSenga offices) was thwarted by challenges in transportation and school schedules. We altered plans to instead create a ‘mobile café’, bringing Netbooks to the schools and having the students complete the intervention there. In some cases, classrooms were big enough to allow sufficient space between students for privacy; but this was not true in all cases. So, we added privacy screens for all Netbooks. In some, but not all cases, class rooms had outlets and electricity. This was addressed by purchasing a car battery and inverter, and Netbooks operated on these batteries at each school site.

In field beta tests of CyberSenga the average length of time to complete each module ranged from 20 minutes (Module 4) to 92 minutes (The registration and initial informational and introductory module) with the average of 54 minutes per modules for each of the six modules. Lesson completion rates during the field assessment slightly decreased each additional module. 100% (20/20) of students finished the baseline survey and Module 0–1; between 80–100% completed each additional module, and 40% (8/20) completed the entire program through Module 6. Factors that influenced completion of all of the modules included missed attendance related to academic commitments, illness, or boarding students being sent home to collect school fees. Module 6 had the lowest completion rate; this was because schools diverged from the original schedule for field testing, shortening the timeline. This is turn meant that programmers had inadequate time to debugging the module, and it was therefore not finalized for testing in two schools.

General computer use skills

There was wide variability in computer skills among participants using the Netbooks in the beta test. Some participants seemed to be quite literate on the computer, and were able to download and play games (until we put a filter on the browsers), open up the video camera and take pictures of themselves, etc. On the other hand, many participants (especially younger students) seemed to be unfamiliar with computer use. Because it requires motor skills and spatial skills, both the desktop mouse and Netbook trackpad seemed to be particularly difficult. Typing was difficult for users with low computer skills.

We fielded the intervention at two schools with higher computer skills first. Given the lack of skills observed for some students, we chose to apply an additional eligibility criterion of computer use in the past year before fielding in the two schools with generally lower computer skills. To ensure a basic level of computer skills, research staff started doing an ‘orientation’ with each student. Skills included moving cursor up/down and left/right using the trackpad, scrolling the page up/down, and how to drag and drop objects (click and hold left button, and drag finger along the trackpad). Despite this orientation to the computer, participants had problems with navigating the first few modules. Specifically, games requiring increased coordination and motors skills were initially challenging for students to navigate. For example, the drag and drop scenario in Module 1 — where students needed to select a Myth and drag the box over to be on top of the Fact that disproves the myth — was particularly difficult. Students were better able to navigate games that required less coordination, such those with point and click elements. As time went on and repetition of skills increased, students were able to learn necessary and after completion of Module 3, most students were able to finish modules with ease.

User feedback

Participant responses to the daily process questions indicated that program likeability was high. Students reported they liked the exercises with animation and activity, such as a soccer game for boys and a dress-up game for girls. Exercises that included pictures without an action (e.g., pictures of a road block that was subsequently ‘removed’ after clicking on a multiple choice option) were not attention-keeping. Exercises that required clicking and had some action (e.g., clicking on a particular frog to categorize whether the behavior has high risk, some risk, or low risk; the frog’s tongue would then come out to lick the selected response) were deemed somewhat entertaining, but not as much as the soccer and dress-up exercises. Students suggested more interactivity and multiple-choice problems would improve the program. Participants frequently described the research space as ‘very private’. All netbooks were equipped with privacy screens. This along with measures taken to move desks so that students had their backs to walls, seemed to work well.

Survey

Several important observations were made during the baseline survey and follow-up survey (which were completed via a Vovici survey, an online survey program www.vovici.com). Because of the participants’ challenge with a mouse, scrolling was very difficult. The survey had multiple questions per page and the need to scroll to see the questions or review material in tables and grids placed a burden on these students. Particular types of questions seemed to be difficult to comprehend and confusing; especially those from the CESD-R, which require a student to indicate frequency in the past two weeks a statement about themselves applied to them (e.g., my appetite was bad). In some cases, time limits prevented module completion. The survey did not have a log in/log out system, so we opted to have incomplete data for these students.

Technological difficulties

A technology-related challenge encountered during the field assessment was the program’s load time, lasting in some cases up to 30 minutes for the module to load, limiting the time students had to complete the module. To address this issue, the research staff loaded the day’s intended lesson on each of the individual laptops at the Cybersenga office prior to traveling to the schools, enabling files to be stored in the Internet browser cache. As a result, at the school, the modules loaded without delay.

Although the pre-Module before Module 1 was helpful in showing students how to use the program, it seemed that retention of the skills demonstrated in the pre-Module were low. Many participants forgot skills for the subsequent Modules; or were unable to generalize skills from a specific example to all related content (especially the drag-and-drop exercises).

Structural difficulties

School schedules seemed to constantly fluctuate. A few days prior to the beginning the field assessments, research staff visited each school to meet with the school administrators, primarily to confirm the academic schedule to ensure we could meet with the YAC students on the intended days and time. Nonetheless, three of the four partner schools had deviations from the schedule originally provided. Two schools changed their exam schedule to begin one day earlier, and one school had changed their mandatory weekly assembly to occur at the same time we were planning to be at the school. To adapt to school schedule changes we encountered during the field assessment, we made arrangements to either 1) field over the weekend; or 2) field at the school on the same day, but at a different time.

Program modifications

We identified and resolved a bug that compromised load times and then compressed the files for faster loading for the subsequent randomized controlled trial. Content was resized to fit on the screen to avoid scrolling; and all drag-and-drop exercises were changed to point-and-click. Instructions for exercises were modified to be more detailed, and clear. A pulsating arrow was created for the bottom of the page to prompt users to click on it to move to the next page. Instead of one pre-Module with navigation instructions, a mini pre-Module was added to each of the first three Modules that demonstrated the skills specific to that Module. The need for more regular coordination with school administrators was recognized and research staff recommitted to meeting with a school contact at least weekly when fielding the randomized controlled trial. Overall, the need for flexibility during the fielding process was highlighted and integrated into subsequent research staff trainings.

Discussion

Use of technologies, such as the Internet and mobile phones, for health promotion will only increase in resource poor settings as access increases in the coming decade. Programs ranging from the very simple with few interactive elements, to those that are more complex will benefit from formative usability and navigability assessments. The development activities that we went through to refine CyberSenga were time consuming and iterative, but ultimately led to an intervention that was not only culturally salient, but also took into account the structural challenges faced by the setting.

The program is presented in English, as this is the shared common language in Uganda as well as the language of instruction for our participants. English is a second language for Ugandans however, suggesting that comprehension would be facilitated with simpler words and sentences. Findings from the alpha test led to reduced text, and readability was reduced to the fourth grade level. Text was presented in small paragraphs; and efforts were made to keep the number of words on each page minimal. Figure 1 illustrates a panel demonstrating this from the final program.

The general observation of user fatigue during the alpha test led to a complete re-thinking of the program design. Originally, many of the concepts were based upon stories that involved conversations between two people. Because the bandwidth could not carry audio files, this meant that the students had to read all of the conversation content. Subsequent to the alpha test, the design team met and identified ways to ‘game-out’ program concepts in more entertaining and varied presentation formats.

Focus groups with the program wireframes were useful both in getting feedback about the intended program components, but also in getting quotable scenarios that could be subsequently integrated into the content. It was useful to hear how participants reconciled their real life experiences (e.g., it’s difficult to get condoms; the question is not whether to use condoms but whether to have sex) with those that they constantly hear about abstinence. This nuance and dichotomy was integrated into the scenarios, leading to richer and hopefully more meaningful content.

The field test led to many important refinements to the program. Indeed, the value of having both a logistics test and a program test cannot be overstated. In environments with variable electricity, low technology exposure, and other structural challenges, it was extremely helpful to test exactly how we were going to conduct the research in the field before we tested whether the program itself was able to affect behavior change. Despite the findings from the navigation test suggesting that students may have general computer skills, most of the students in the field tests need an orientation as well as constant help for the first modules. Indeed, the navigation test seemed to have no direct applicability to the CyberSenga user experience. We were reassured however, that with longer exposure to computers, youth could likely quickly learn the basics needed for accessing and utilizing CyberSenga. An introductory “pre-Module” session that provides basic instructions on how to advance through the program; and complete surveys, quizzes and exercises throughout the program is necessary, but insufficient. Research staff must be trained to provide technical support as part of one of their core activities.

Specific program elements were difficult to navigate and potentially required elimination or reconfiguration. Certain exercises that required dragging and dropping were particularly challenging for youth. To address this issue, we replaced this functionality with a ‘click’, thus requiring less computer skill. Instructions for exercises were not always completely understood. The need for explicit, step-by-step instructions (e.g., ‘click the next arrow to begin’; ‘If you want to create a space in between words, use the space bar – the long bar at the bottom of the keyboard) is appreciated. To eliminate the need for scrolling, the program was resized so that it would fit on a 10.1” netbook screen.

Finally, additional technology issues may be uncovered in field-testing that are not previously discovered. In our case, the issue discovered was a bug in the programming that resulted in unacceptable load times.

Limitations

Without doing similar work in other developing countries, it is impossible to know how generalizable our findings are. We posit that some will translate (e.g., computer skills) while others may be specific to our setting (e.g., challenges with schools schedules). Even if the specific problems and associated solutions do not translate, the methods describe here offer a template for beta-testing other technology-based health promotion endeavors in Africa and in other resource poor settings.

Conclusions

Certainly, the promise of technology needs to be weighed against its potential challenges. Whether the setting is in a developed or developing country, health and social service providers need to be sure that the delivery mechanism matches their target population and infrastructure. Sometimes compromises can be made. For example, if computers are available but Internet is not, perhaps a computer-based program that relies on a flash drive or CD ROM or DVD is a better fit. Sometimes technology is simply not the right choice. For example, even if mobile phones are widely used, if your target population is illiterate, perhaps a text messaging-based program is not advisable. Traditional approaches such as group-level education, while often deemed less innovative should be considered if they are better able to reach your target audience.

CyberSenga was designed in anticipation of increased Internet access in Uganda. Several advances have been made towards this goal in the country, including the availability of wireless Internet through cell phone providers; and more affordable netbook computers. Nonetheless, access is far from ubiquitous. We hope that the extensive development process ensures that when the Internet becomes more widespread, the CyberSenga program will provide a uniquely salient option for HIV prevention. The CyberSenga program is currently being implemented in Uganda with 366 secondary school students, and results from the randomized controlled trial of CyberSenga efficacy for HIV prevention will be available after 2011.

Table 4.

Module completion rate among Youth Advisory Council users of CyberSenga Beta-modules (n=20)

Module	Completion rate	Average length of time to complete
Registration/ Survey	20/20 (100%)	58 min (range: 39– 82 min)
M0 & M1	20/20 (100%)	71 min (range: 60–92 min)
M2	19/20 (95%)	56 min (range: 30–75 min)
M3	18/20 (90% )	39 min (range: 25–58 min)
M4	17/20 (85%)	28 min (range: 20–35 min)
M5	16/20 (80%)	40 min (range: 22–72 min)
M6	8/20 (40%)	36 min (range: 30–43 min)