SiHLEWeb.com: Development and Usability Testing of an Evidence-Based HIV/STI Prevention Website for Female African-American Adolescents

Abstract

African-American adolescent girls are at disproportionate risk for HIV infection. Although numerous evidence-based risk-reduction interventions exist, dissemination and implementation resources remain limited, and prevention services remain notably inaccessible to the very populations at highest risk for HIV infection. Internet delivery of HIV risk-reduction programming has promise as a mechanism for extending the reach of existing prevention efforts and overcoming barriers associated with traditional service delivery. This article: (1) details the development process for the creation of SiHLEWeb, a web-adapted version of an evidence-based, culturally-informed HIV prevention program traditionally delivered to female African-American adolescents via an in-person group format; and (2) presents findings from quantitative and qualitative usability testing conducted among 18 African-American girls (13–18). Results suggest that users found the website improved knowledge and learning, was helpful, efficient to use, and generally attractive. Users reported some concerns about website navigation. Implications for internet delivery of health prevention programming are discussed.

INTRODUCTION

The HIV/AIDS epidemic is currently impacting the lives of over 1.1 million U.S. adults and adolescents (1). African-Americans are at disproportionate risk for HIV infection. Recent data indicates African-Americans have an HIV diagnosis rate 799% that of Caucasian Americans (2). National data also indicate that African-Americans aged 13 and older have the highest prevalence of undiagnosed HIV infection in the U.S. African-American females represent nearly two-thirds (64%) of all HIV-positive females in the United States, a rate 18 times that of their Caucasian counterparts (3). Although these statistics are disheartening, the fact that the majority of these new infections were transmitted through sexual contact means that this is a largely preventable epidemic. Further, these data highlight the critical role that effective HIV/Sexually Transmitted Infections (STI) prevention curricula – specifically those addressing sexual risk behavior and promoting healthier decision making in this population – can play in stemming this public health crisis.

As such, dissemination and implementation of evidence-based, behaviorally-focused prevention strategies have been identified as public health priorities (4). The growing epidemic of new HIV/STI infections among African-American adolescents, in particular, underscores the need for culturally-informed prevention services targeting ethnic minority youth. As evidence suggests, these interventions are more effective than standard services (5) (6) (7). As of 2011, 74 HIV/STI risk reduction and prevention interventions had been developed and identified as evidence-based by the Centers for Disease Control in the United States; nearly a third of these interventions have culturally-tailored elements intended to increase engagement among minority/African-American populations (8). However, many of these interventions require significant resources, including financial, personnel, and time to implement, and may not be readily available or accessible to populations at highest risk (9, 10). Impediments to the effective dissemination through traditional means include, but are not limited to, lack of service providers (particularly in more rural areas), high staff turnover, lack of transportation to services, potential stigmatization, and other client characteristics such as substance use (11)

Considering their relative low cost to produce and disseminate, adapting evidence-based programming for computer-based and web-based delivery is one method of addressing these barriers and extending the potential reach of these health-focused interventions. Specifically, computer technology-based, behaviorally informed HIV/STI interventions that target increasing condom use have shown a significant impact when evaluated across a variety of at-risk populations and when specifically evaluated with e-Health HIV Intervention Development 6 populations at high-risk for HIV infection (12–15). Computer-based technology has been used to deliver adapted content for multiple HIV prevention interventions that target a range of populations including, Tailored Information Program for Safer Sex (13), Project LIGHT (16) and Willow (17).

Most pertinent to the current study, Sistas Informing Healing Living Empowering (SiHLE) program, which delivers culturally informed HIV prevention materials to African-American female adolescents, was adapted into a 2-hour, computer-delivered, individual intervention called Multimedia SiHLE (18). The preliminary efficacy of Multimedia SiHLE was supported by results of a randomized controlled trial (RCT) (19). Sexually active and non-sexually active, African-American females (n=178), aged 14 to 18 years were recruited from the San Francisco Bay area and randomized to receive either Multimedia SiHLE or general health education that consisted of a computer-delivered, video-based, 65 minute session that provided information on diet and nutrition. Three-month follow-up data indicated significant increases in STI knowledge for both groups, although significantly greater increases were observed among the intervention group. Further, intervention participants, and not control participants, demonstrated significant increases in condom self-efficacy and condom use at follow-up. Whereas results provide preliminary support for the efficacy of the intervention, Multimedia SiHLE was developed for quick completion (average of two-hours) and, importantly, for computer-based and not web-based dissemination. Participants had to complete the program on-site rather than from the convenience of their own home or community-based organization, thus limiting the potential reach of the intervention, as well as the related confidentiality issues that are sometimes relevant for young people.

Given the need for a more easily accessible and portable interactive HIV prevention curriculum, our group recently developed a web-based version of the SiHLE intervention as one critical component of a multi-faceted capacity building initiative for minority HIV prevention in the Southeast United States. To address traditional barriers of accessibility and stigma, our team chose an e-Health (versus standard computer-based) platform to foster the development of a publicly available, evidence-based HIV prevention intervention that could be widely disseminated to and usable by both community-based organizations and unmediated end users – African-American adolescent girls. E-health is a promising approach to health education and prevention that has long been growing momentum (20–23). The current paper has implications for other HIV-related intervention efforts in that the steps followed in this mixed method study can be applied in the development and evaluation of other web-based interventions. The current article has two aims: (1) To outline key developmental decisions and processes e-Health HIV Intervention Development 8 involved in the creation of SiHLEWeb – a web-based adaptation of the SiHLE program; and (2) To present qualitative and quantitative results of usability testing of SiHLEWeb conducted with a sample of African-American female adolescents.

METHODS: WEBSITE DEVELOPMENT AND DESIGN

Procedures: SiHLEWeb Development

Intervention Adaptation

SiHLE is an evidence-based HIV prevention program designed specifically for sexually active African-American females between the ages of 14 and 18 years (24, 25). SiHLE, in its original format, consists of four, 4-hour sessions delivered in small groups. Content of the intervention focuses on enhancement of ethnic and gender pride, HIV prevention strategies, and assertive communication skills training. SiHLE is traditionally delivered by a female African-American adult facilitator (who is responsible for the health education portion of the curriculum) and one to two African-American peer facilitators (who are responsible for modeling positive HIV prevention behaviors and promoting group norms supportive of HIV prevention). In an RCT (n=522 African-American adolescents), SiHLE demonstrated efficacy in comparison to an attention-matched control group with respect to: (1) increases in consistent condom use; (2) percentage of condom-protected vaginal sex acts; (3) frequency of applying condoms on a sex partner; and (4) condom use during last sexual encounter (25). Outcomes were sustained through 6 and 12-month follow-up assessments. Furthermore, SiHLE intervention participants reported significantly fewer new vaginal sex partners, less likelihood of Chlamydia diagnosis, and fewer episodes of unprotected vaginal sex during the 6- and 12-month follow-up assessments.

Consumer Feedback

In order to increase the relevance, appeal, and usability of the SiHLE website, the development team relied heavily on input from local members of the target audience. At the outset of the adaptation process, three focus groups (N = 25) were conducted to gather information regarding: (a) websites that African-American teens frequently visited, (b) reasons that teens visited/spent time on those websites, and (c) opinions on various existing websites targeting teen HIV/STI prevention, dating violence, and pregnancy prevention. Focus group feedback guided our decisions to use bright colors, include options for personalization, minimize text in favor of images that conveyed meaning, and avoid the use of heavy animation and random sound effects. Further, nearly unanimous feedback from focus group participants encouraged the use of humor throughout the website content to incentivize and engage users. In addition to informal focus group testing among local-area youth, a Youth Advisory Board (YAB) was formed to guide design and development decisions on an ongoing basis. Members for the YAB (n=7) were recruited from community partner agencies, local high schools, and area church youth groups. All YAB members were African-American girls (aged 14–18) and represented a variety of socioeconomic backgrounds. The YAB members met regularly to provide informal consumer feedback, review site design mock-ups, guide development of website content, and inform delivery modalities (e.g., videos, activities, quizzes).

Website Design

The website design and user interface was developed with parallel input from consumer feedback, programming experts, and project staff, as well as consideration of best practices in e-learning (26, 27), including the United States Department of Health and Human Services’ most recent e-health usability guidelines (http://guidelines.usability.gov). The resulting product, SiHLEWeb, was a highly interactive, streamlined, graphically oriented product. Consistent with the curriculum for the in-person groups, the SiHLE content remained divided into four primary sessions – or modules – entitled “We are SiHLE,” “It’s My Body,” “SiHLE Skills,” and “Power and Relationships.” Content within each module was distilled to core informational and activity elements that could be presented within a one-hour time frame through a process of review by project staff and SiHLE developers, Drs. DiClemente and Wingood. To further divide information into manageable units and provide users with logical opportunities to exit and re-enter the site, each of the four modules was divided into four subsections.

The webpage layout consisted of four main spaces: (1) Top, which contained title and navigation information; (2) Bottom, which contained progress indicators; (3) Right, which contained video content; and (4) Center, which contained a workspace for text or activities (see Figure). In response to feedback from the YAB and as an incorporation of a popular element of social media, the website design incorporated photography of African-American women and youth engaging in daily activities; this provided the backdrop for website content.

Figure 1.

SiHLEWeb Screen Shot

Video was used to deliver the majority of site content for several reasons: (1) to keep with the overall aim of the website to emulate a SiHLE group experience; (2) to overcome reading comprehension issues; and (3) to increase engagement among users. Three main types of videos were included. First, we utilized Adult Facilitator and Near Peer videos, which delivered much of the didactic content. Second, we included SiHLE group participation videos, which mimicked content that may occur during a typical SiHLE group discussion among group members in response to activities or information presented by the Facilitator and Near Peer. Third, we incorporated SiHLE group member “confessionals”, which emulated popular reality TV and provided insight into the group members’ reactions and personal experiences relevant to the material discussed. Finally, interactive activities were presented via quizzes and demonstrations, which allowed the user to practice or demonstrate mastery of certain skill or knowledge of certain information. After completion of activities, users received video-based feedback from the “adult facilitator.”

Website Production

Input from our target demographic extended beyond developmental focus groups and advisory boards to include casting local African-American teen girls to play roles of “SiHLE group members” for the video footage used in SiHLEWeb. To recruit actresses, advertisement regarding the project was distributed to schools and partnering agencies, resulting in approximately 50 African-American girls auditioning for one of five roles as a SiHLE participant. For casting auditions, each of the youth was given the opportunity to introduce themselves, select a character (from among those created by the project staff), and participate in a brief sampling of activities from the SiHLE program. Each audition was taped and reviewed and rated by the project staff on characteristics of voice (including volume and dialect), screen presence, use of nonverbal communication, and improvisational interactions with other actresses. Professional video production experts (housed at the home medical university setting) were responsible for filming and editing video footage for the site.

In addition to project staff, the SiHLE developers reviewed all video footage. Team, consultant, and YAB feedback guided decisions regarding wardrobe, makeup, introducing diversity (with respect to character biography), and set construction. In addition, the project team reviewed videos to ensure that the content and spirit of each activity was consistent with that of the SiHLE program. Team and consultant feedback also were heavily relied upon in the process of translation of group activities for web-based delivery.

METHODS: USABILITY TESTING

Participants

Participants were African-American females (N=18) between the ages of 13 and 18 years (M= 16.29, SD = 1.48), who reported being either sexually active – defined as having had consensual oral, vaginal, and/or anal sex in their lifetime – or sexually-contemplating – defined as having a strong interest and/or intent in having sex in the next 12 months. Girls with significant cognitive disabilities, developmental delays, or pervasive developmental disorders that would prevent learning from a web-based prevention program were excluded from participation. Participants were recruited from a range of community settings (e.g., Department of Juvenile Justice, area high schools, youth centers) through the use of advertisement, referrals from community partner agencies, and snowball recruitment methods (28). Participants were compensated $35 for their time.

Procedures

The Institutional Review Board (IRB) of a large southeastern medical university approved all procedures for the website usability testing. Following completion of informed consent procedures with both the teen and legal guardian, participants were scheduled to complete testing at a time most convenient for them. Participants were directed to a private room and provided a laptop on which to access the SiHLEWeb intervention website. Participants were assigned a unique user-ID that was used to log in to the website and also denoted which sections of the website users would complete. Given that SiHLEWeb was designed for completion in multiple visits to the site and would take approximately 4 hours to complete in its entirety, participants were randomly assigned two (of the total of 16) sections to review to allow time for provision of feedback while keeping participation within a 60 to 75 minute timeframe.

In line with standard cognitive interviewing procedures (29, 30), participants were video recorded as they completed and shared a ‘think aloud’ approach, which involves a running dialogue about their opinions, impressions, and thoughts on each of the pages of the website. When participants visibly changed affect or were silent, they were prompted by the facilitator to share their thoughts and opinions using a general guideline for each page of the Web intervention to guide the interviewers’ open-ended thematic interview questions. However, the majority of user responses were participant elicited and were based on the actions and responses of participants as they navigated through the intervention. This ‘think aloud’ approach, accompanied by prompts, is a well-established, effective approach in the assessment of web-based intervention usability testing (31, 32). Following completion of their designated sections of the intervention, participants completed the Website Analysis and Measurement Inventory (WAMMI) (33), a self-report website usability standardized measure, which is described below.

Measures

The Website Analysis and Measurement Inventory (WAMMI) (33, 34). The WAMMI is a standardized 20-item assessment that captures users’ personal opinions on a given website’s ease of use. The WAMMI asks users to rate various aspects of their experience with the site (e.g., content, navigation, design) on a 5-point Likert scale from “strongly agree” to “strongly disagree.” Items are then scored to produce 5-subscales measuring Website Attractiveness, (level of visual interest of the site both in terms of function and information provision); Website Control, (navigation ease); Website Learning (user’s ability to easily understand the content and learn what they expect to learn); Website Helpfulness (usefulness and expected content and structure); and Website Efficiency (users’ ability to quickly find and do what interests them on the site). The WAMMI was developed using latent variable analysis, has high reliability, and reports standardized scores (e.g., 50 = average; 100 = perfect) for the 5 aforementioned themes based on a reference database of websites (35).

Statistical Analyses

A mixed-method approach has been strongly recommended for strengthening qualitative or quantitative methods (36). The overlapping but unique and differing depth of the data obtained through WAMMI questionnaire responses and cognitive interviews was well suited for a mixed-method approach and allowed for triangulation. The mixed-method approach of triangulation allows for both the depth associated with qualitative analyses from the cognitive interviews, but also gives the project a wider breadth than might be expected in quantitative analyses of the WAMMI questionnaire on its own (for review) (36–39). As such, the thematic qualitative analyses of the utterances observational data and quantitative analyses of the questionnaire responses were conducted concurrently and interpretations were interpreted in light of both results (40).

Thematic Coding

Following completion of the interviews, digital videos were transcribed verbatim for both the interviewer and participant by a trained master’s level transcriptionist, checked by a doctoral level co-investigator, and imported into NVivo-9 qualitative analyses software (41). Following transcription, a doctoral level co-investigator conducted qualitative analyses of the transcripts in NVivo-9 informed by Constructivist Grounded Theory (42). Primary coding using line-by-line coding identified 69 thematic codes to users’ impressions about the website. During secondary coding, the constant comparative method was used to merge and create new codes to ensure maximal convergence and minimal divergence within each code. Further, only codes that were held for 2–3 cases (or 10–15% of cases) were retained (36) thereby reducing the codes by approximately half (29 thematic codes). As a final step in coding, the emerging hierarchical structure of constructs were reviewed and refined based on a co-occurrence and non-occurrence of features (36). The data derived hierarchical structure produced 7 parent (superordinate) themes of Content (including Overall Content, Activity Content, Video Content, Character Content); Layout and Design, and Navigation and Direction, and 9 Child (subordinate) codes.

RESULTS

For triangulation across quantitative and qualitative data, specific application of parent themes was examined for overlap with WAMMI subscales. Matrices analyses in Nvivo were used to explore the overlap or triangulation of participants’ overall scores for the WAMMI subscales of Learning, Helpfulness, Attractiveness, and Controllability. Users values for these subscales were entered into NVivo-9 qualitative analyses software for comparison with qualitative themes from cognitive interviews. Resulting overlap is presented below and in Table 1 by WAMMI theme.

Table 1.

Percentage and frequency of website themes organized by WAMMI subscale theme

Parent Theme	Overlapping WAMMI Subscale	% of users by WAMMI Subscale	Child Theme	% of sample (number of cases)	References to theme in sample
Design and Appeal	Attractiveness	75% Strongly Agree 15% Agree
			Pictures in background are appealing “I like these pictures in the back, they are cool”	11% (2)	5
			Website is cool, funny, and attractive “It’s very nice how yall put some stuff that we relate to into the website that’s not boring, it’s very attractive”	50% (9)	21
Content	Learning	80% Strongly Agree 20% Agree
Activity Content			Poems activity liked “It’s [poem] better that way the girls were saying it”	17% (3)	6
Activity Content			Activity teaches and is engaging “You learn things and it’s interesting and I like the game you, like, you understood the concept”	33% (6)	16
Questions Content			Question and Answers activity helpful “These questions are good because they give you the options to see if you learned everything”	50% (9)	19
Character Content			Characters facilitated learning “She [character] did good. An explanation why to use, well, and not why. She didn’t say only why, but she had a good point on using condoms”	28 (5)	6
Text Content			Content terminology confusing “Compassion, this like mean with somebody else?”	44% (8)	17
Content Generally			Content informational “I learned like when you have all the SIHLE girls telling them SIHLE girls what is truth and what is myth”	44% (8)	20
Content Generally			Website is educational generally “I like this website, it’s very educational and fun”	11% (2)	2
Video Content			Video was informative and had content teens should learn ”I think this was easy to understand because of the video”	56% (10)	53
Content	Helpfulness	50% Strongly Agree 50% Agree
Content Generally			Content agreed with, expected “This stuff is true with some teens today that that’s why they don’t use condoms and things like that”	17% (3)	6
Activity Content			Content disagreed with, unexpected “That [activity] was questionable”	11% (2)	3
Content Generally			Content redundant, doesn’t follow other content naturally “And it basically took what you just watched and said it all back again”	17% (3)	6
Activity Content			Activity is easy to do “Easy to do because everyone else [characters] had a different answer for everything”	11% (2)	5
Questions Content			Question and answer activity is confusing, hard to do “Like the way the questions are at the bottom I expect there should be some sort of answer”	11% (3)	4
Character Content			Character activity helpful and liked “I like it because girls actually sit down and they actually talk about stuff like that, they are actually like that that they have those problems and helps”	28% (5)	5
Character Content			Character was not realistic, not relatable, unexpected “I didn’t get her. That first girl was was kinda, kinda, ghetto”	17% (3)	7
Character Content			Character was real, relatable, expected “Its like feedback from your peers”	39% (7)	10
Content Generally			Content describes relatable and expected mistakes	11% (2)	2
Video Content			Video content unexpected or objectionable “I ‘ain’t want to say that’s aggressive either, but the way she talking sound aggressive.”	33% (6)	14
Video Content			Videos redundant and too much talking for user needs “The only thing was I didn’t like her talking so long in that beginning”	22% (4)	10
Navigation	Controllability	60% Strongly Agree 35% Agree
			Navigation confusing and difficult “Why can’t it just be like a process where you just read one page and click to the next page?”	11% (2)	2
			Navigation easy and liked “They show you far you’ve come and how far you have to go and that you are going in the right direction”	33% (6)	15
			Technical problems getting through site “Why am I supposed to click? Ummm”	67% (12)	75
			Multiple ways of presenting content was helpful and aided progression through site ”I like how you break it down and um show people how the website goes. Cause um, most websites don’t break it down for you and you have to figure it out for yourself basically” 33%	(6)	21

The final version of the parent theme of Content included codes indicating users’ overall satisfaction with the main learning mechanisms of the website (activities/games, text, videos, character presentations), pleasure with learning the content, and the degree to which the website was as expected and realistic for this population. This overlapped strongly with the constructs included in the WAMMI subscales of Learning and Helpfulness. Most of the users stated they enjoyed the videos, specifically noting they learned a lot from the videos, and found the content relevant and expected for their demographic, but many also noted that some of the videos were long and included some unexpected tone and content (i.e., statements made by one of the SiHLEWeb characters who was intentionally provocative). Similarly, most enjoyed the active learning activities of the site; users reporting considerable learning from the question and answer activities and the poems, generally found them helpful and as expected, with no concerns noted. Participants were more mixed on the characters portrayed, although several users stated that the characters were useful to facilitate and enhance concepts learned in the website, many others noted that one or two characters were unexpected, not relatable and/or outright disliked, and the remaining users volunteered that they generally related to, expected, and liked all the characters. The multiple modalities of information being portrayed were emphasized by users as facilitating learning and retention of information across sessions. WAMMI results are consistent with users’ cognitive interviewing report that they were generally learning a lot from the website and found the website content to be interesting and important to learn and presented in a way that taught the material well.

The final version of the parent theme of Navigation included comments regarding users’ difficulties with order of activities and obtaining the information sought after or anticipated information. Many users reported problems with being confused about how to navigate between pages of sections and difficulty locating the position of feedback after activities. Navigation difficulties occurred most often when an activity or video page then navigated to a text page. However, negative comments regarding navigation were not universal; several other users noted the website was easy to navigate and find feedback. This Navigation parent theme overlapped greatly with the WAMMI subscale of Controllability. Overall WAMMI results for this dimension were lower than ratings for other components of the website, consistent with many users having reported frustration with navigation and the numerous references to these problems in the interviews.

The final version of the parent theme of Layout Design included codes indicating users’ overall satisfaction with the graphical design and appeal of the site. This theme largely overlapped with constructs assessed by the WAMMI subscale of Attractiveness, with scores indicating that the majority of users found the site attractive. Although users did not have many consistent statements about the attractiveness of the site, several noted it was fun, cool, and funny. A few participants noted that they specifically liked the pictures in the background of girls engaging in everyday activities.

The users in this sample did not direct any comments during cognitive interviewing to indicate their specific concerns about the efficiency of the website. As such, no parent themes overlapped with the WAMMI subscale of Efficiency. WAMMI results generally indicated that users felt the website will was moderately efficient, which may be why users did not feel the need to volunteer comments pertaining to this domain.

DISCUSSION

Access to health care, prevention and treatment services, has long been a topic in the public arena. Specific to the goals of the current paper, although evidence-based HIV prevention interventions exist for African-American teen girls, cost, transportation, local availability, and confidentiality concerns inherently limit access to these existing services. E-Health, and particularly web-based, health-promoting interventions, offer a promising alternative to traditional in-person services, due to their typically low-cost, high accessibility, and lower resource requirements. The process of developing a web-based health intervention, including translation of existing evidence-based in-person interventions for web format, as well as the evaluation of the usability of the newly created website, is of critical importance. If the web-product is not developed with an eye on dissemination and is not found to be engaging or helpful to the target consumer population, the effectiveness of such a service is inconsequential. Thus, beyond the presentation of the mixed-method results of the current study, a related goal of this paper was to describe the process by which SiHLEWeb was developed and its usability was evaluated to offer readers a template for related e-Health adaptation and evaluation efforts.

The objective of the development of SiHLEWeb was to provide HIV prevention information to adolescents through a preferred medium and develop a tool that could be broadly disseminated if found to be efficacious. Several components of our SiHLEWeb development process contributed to overall positive feedback garnered from usability testing participants. First and foremost, it was critical that consumers were included throughout the full development process, from graphic design of the site to format and content of activities to length and utility of videos. For example, on average, individuals will only spend 10–20 seconds on a website before deciding to click something else or to remain on the page (43). Hence, attractiveness of a webpage is as important as its content. Similarly, establishing different forums for which consumer input could be collected (e.g., focus groups, Youth Advisory Board, casting community members as actors for the site) allowed for assessment of reliability in feedback of the website across youth, as well as further engagement of the youth for the intervention. HIV prevention is not often described as a “fun, cool, and funny” topic among adolescents; however, as detailed in the Results section, these were the exact words used to describe this HIV prevention site during the usability testing. In other words, consumer inclusion and feedback helped promote the likelihood that SiHLEWeb would be found to be helpful and interesting to the participating youth. Second, in keeping with the theme of the original in person SiHLE intervention, creating learning opportunities through highly interactive activities was important and valued by the youth. Third, involvement of the developers of the original health intervention helped ensure that the fidelity to the evidence-based SiHLE content was maintained while still allowing for the necessary adaptations for the web-based format.

The qualitative and quantitative results of the usability testing conducted on the fully developed site complemented one another and supported the utility of the website for the target population. Consistent with the suggestions given by consumers during the development phase, usability participants liked the interactive videos and activities of the site, including questions and answers and feedback to their answers, as well as the use of videos to teach content and to meet the SiHLE participants. The WAMMI responses and the interview responses highlighted that users appreciated the graphic design, color schemes, and use of photography in the background of the site (i.e., overall attractiveness).

Whereas the majority of feedback was positive, some components of SiHLEWeb garnered negative feedback during usability testing. As mentioned above, navigation problems were the most frequently noted critique of the site. Fortunately, we were able to specifically target and fix identified errors in navigation upon completion of the usability testing. Indeed, navigation ease has been highlighted as a critical component of a webpage (44); even if the content of a website is engaging, problems with navigation can lead to users quickly exiting a webpage. Usability testing also resulted in mixed feedback regarding the SiHLE group participant characters; not all users found all characters to be likable. However, the development team anticipated this finding, as it was implicit in the goals of replicating a traditional SiHLE group that our characters would portray a range of attitudes, personalities, backgrounds, preferences and interaction styles. In addition to conveying that no two African-American teen girls were completely alike, the decision to script a range of character types allowed for a parallel range of teaching opportunities. In fact, one of the characters was intentionally challenging in her interaction style, which provided opportunities to discuss aggression versus assertiveness and healthy decisions versus unhealthy decisions in the context of the SiHLE group. Finally, the results of the current study also highlight the value of utilizing a mixed method approach in the evaluation of usability of a website. For example, while the quantitative results indicated overall satisfaction with the main learning mechanisms of SiHLEWeb, as well as finding pleasure with learning the content on the site, qualitative interviewing and observation was necessary to identify specific aspects of the website to which the users attributed fun in learning the content (e.g., the videos and activities following the poems). Also, while the quantitative results indicated that relative to the other aspects of the website (e.g., Attractiveness), controllability received the lowest ratings, the lower ratings only suggested problems of navigation. Specific problems in navigation around the website could not be gleaned from such responses on the WAMMI. However, qualitative interviewing allowed for more specific probing and observation of challenges while navigating through the site and enabled our team to pinpoint and fix these problems in navigation prior to the next stage of evaluation (feasibility testing), which is described below.

Despite these promising results, the limitations of the current study should be noted. First, the development of the website and the usability testing were both completed with African-American youth who lived in the same geographic location of the country (southeastern United States). Different areas of the country have different dialects and potentially different ways of communicating. Although this is an adequate first step in the evaluation process, it will be important to pursue evaluation of the website among African-American teen girls from other regions of the country to determine the relevance and reach of the site-important considerations for determining larger scale effectiveness and dissemination. Second, the usability testing was completed with a relatively small number of participants (n=18). The sample size is standard for this type of qualitative work and permitted the attainment of theoretical saturation of feedback; nonetheless, it is possible that different feedback or clarification of mixed feedback would have been captured among a larger sample of participants. Third, support of usability of the website is necessary but not sufficient for the evaluation of SiHLEWeb as an effective tool for HIV prevention. It is the first step in a long line of research questions, which will ultimately make that determination.

Our research team has already taken the next step in this line of research by conducting an open pilot trial with 40 African-American teen girls following the completion of usability testing to evaluate the feasibility of SiHLEWeb. Specifically, we sought to determine if a community sample of girls would indeed log onto SiHLEWeb and complete its content in their natural environment (e.g., home). Results from the feasibility trial were promising, with 88% logging onto the website at least once and the majority (63.4%) completing the site in its entirety (45). Encouraged by the results of both usability and feasibility testing, next steps in the evaluation of SiHLEWeb include larger scale investigation of the effectiveness of the website in preventing HIV risk behaviors through an RCT. Our investigative team is in the process of designing a large, well-controlled hybrid efficacy-effectiveness trial of SiHLEWeb for this purpose. Cost effectiveness will also be an important consideration in evaluating this approach to HIV prevention in comparison to the traditional, in-person delivery of such services. If the website is indeed found to be effective, a range of dissemination and implementation questions will also need to be examined, including identifying best methods for promoting awareness and incentivizing completion of the website among the target population, such as social media methods. Given the possibility of low cost, high effectiveness, and wide reach of web-based, health-promoting prevention interventions, these are key questions not just for SiHLEWeb, but also for the field of the e-Health on a broader scale.