Development of a Sexual Health Screening Tool for Adolescent Emergency Department Patients

Abstract

Objective

Develop a content-valid audio computer-assisted self-interview (ACASI) sexual health survey (SHS) that is understandable and acceptable to adolescents and can be feasibly implemented in a pediatric emergency department (ED) for sexually transmitted infection (STI) risk assessment.

Methods

Multi-step iterative qualitative study utilizing a Delphi panel of key informants for survey development and content validity, cognitive interviews with end-users to evaluate understanding, and pilot testing with end-users to evaluate acceptability and feasibility.

Results

We developed a 20-item questionnaire through an iterative modified Delphi process with experts in adolescent and pediatric emergency medicine. All items were assessed as understandable by >90% of adolescents during the cognitive interviews. All respondents found the SHS easy to use. 76.5% preferred answering questions related to sexual health through the SHS compared to face-to-face interviews. Mean length of survey completion was 17.5 minutes (SD +/−6.7) and 88.6% of participants found survey length to be “just right.” With respect to feasibility testing, there was no statistically significant difference in median ED length of stay between those who piloted the SHS and those who did not (230.0 vs 219.0 minutes; p=0.7).

Conclusions

We developed a content-valid ACASI for the identification of adolescents at risk for STIs that was understandable, acceptable, and easy to use by adolescent patients and feasible for implementation in the pediatric ED. Future planned steps include the evaluation of the SHS in providing clinical decision support for targeted STI screening in the ED.

Introduction

Adolescents are disproportionately affected by sexually transmitted infections (STIs), and account for 9 million of the 19 million new cases of STIs each year.¹ Failure to diagnose and treat STIs in a timely fashion can result in serious reproductive morbidity, including pelvic inflammatory disease, ectopic pregnancy, infertility, and facilitation of the transmission of Human Immunodeficiency Virus (HIV). Most importantly, failure to diagnose STIs leads to increased transmission of disease, further perpetuating this cycle and contributing to the STI epidemic.

High rates of STIs have been documented among an adolescent emergency department (ED) population.^2–4 Recognizing the opportunity for the ED to serve as a strategic site to reach high risk populations, the Centers for Disease Control and Prevention recommends that HIV and STI screening expand into the ED setting,⁵ however, this has not yet become routine in most EDs.^6–8 Furthermore, even when patients present with symptoms suggestive of an STI, sexual histories and STI testing are not routinely conducted by ED physicians.^6–11

There are several barriers to the assessment and treatment of STIs in the ED. Although adolescents are interested in learning about sexual health in the ED,^12–14 and a majority of teens presenting to the ED even for non-reproductive health issues are accepting of STI testing,¹³ adolescent patients are not always comfortable disclosing sensitive health information to physicians in face–to-face interviews.^15–20 Moreover, patients may not recognize that their symptoms might be STI-related, or that they may be at risk for STIs, and therefore, may not disclose their sexual behavior unless prompted. Furthermore, the chaotic nature of the ED setting often makes it difficult to conduct sexual health interviews in a confidential manner.

Audio computer-assisted self-interviews (ACASI) are an effective method for obtaining sensitive health information accurately and efficiently.^19,21 Implementation of computerized screening may help overcome the barriers to obtaining sensitive information accurately when compared to face-to-face interviews or self-administered questionnaires.^19,21–24 Adolescents specifically report more comfort with and the ability to be more truthful with computerized surveys collecting sensitive information.¹⁹ Moreover, ACASI allows for standardization of questions and can simplify complex surveys by employing skip logic and internally cross-checking related responses for consistency to maximize the specificity of the interview. Finally, audio capabilities can assist those with low reading literacy.

Despite many encouraging studies on ACASI, and its preliminary success in the pediatric ED for mental health²⁵ and violence screening,²⁶ ACASI has yet to be evaluated for sexual health assessment in the adolescent ED population. Therefore, our goal was to develop a content-valid ACASI-based sexual health survey (SHS) that was understandable and acceptable to adolescents and could be feasibly implemented into a pediatric ED for STI risk assessment.

Methods

This was an IRB-approved multi-step qualitative study for survey development and testing.

Survey Development: Modified Delphi Process

We began the process of ACASI SHS development by gathering expert opinion regarding the scope and content of items to ensure content validity. We convened a modified Delphi panel and held 30-minute individual phone interviews with 13 key informants who are national leaders in adolescent medicine and pediatric emergency medicine. The goal of these interviews was to identify the presence of previously developed sexual health screening tools that may be adapted for the ED setting, to discuss the benefits/limitations of these existing tools, and to identify questions/items that would be most important in such a tool for use in a pediatric ED. Based on the key informant interviews, we based the initial questionnaire development on questions from two previously validated surveys, the Youth Risk Behavior Survey (YRBS)²⁷ and the National Survey of Family Growth (NSFG),²⁸ and incorporated items identified as essential for inclusion by the experts. Through an iterative electronic Delphi process, we identified key questions crucial for an effective sexual health screen in the ED setting and discovered areas of missed content or inappropriate wording. We compiled a master list of potential items to send electronically to our expert panel, asking them to: 1) rate each item as “useful” or “not useful”; 2) if necessary, restructure the wording of any useful item; and 3) suggest content not represented in any of the current questions. Questions identified as “not useful” by at least 75% of experts were eliminated. The remaining questions, along with any new suggested items, were distributed for a second round of the Delphi process, in which experts were again asked to rate each item as “useful” or “not useful” and items identified as “not useful” by at least 75% of experts were eliminated. For the third round, the experts were asked to rate each remaining item on level of importance using a 5-point Likert scale (“1” indicating “unimportant” to “5” indicating “very important”). A priori, we decided that questions with mean scores >=4 were retained and questions with mean scores <=3 were removed. Questions with scores <4 but >3 with significant variability were also removed.

Survey Development: Cognitive Interviews

Once the initial SHS questions had been selected by our expert panel, we performed cognitive interviews with our end-users - adolescent ED patients - to further refine the questions for clarity and content validity. This study population consisted of a purposive sample of adolescent males and females ages 14–19 who had presented to the ED with and without STI-related symptoms. This sampling strategy was designed to achieve maximal input from each age strata (14–15 years and 16–19 years), gender, and presence or absence of STI-related chief complaint. Chief complaints of lower abdominal pain, dysuria, ano-genital lesions, and/or vaginal pain/discomfort, discharge, or bleeding in females and ano-genital lesions, dysuria, and/or urethral discharge in males were categorized as STI-related. We excluded patients from participating in the cognitive interviews if they had a history of developmental delay, presented to the ED with altered mental status or psychiatric complaints, were non-English speaking, or were critically ill.

Two research staff performed cognitive interviews to pretest the survey for clarity and understanding using the “Think Aloud” technique,²⁹ a technique in which patients are asked to talk through their thoughts, reasoning, and feelings as they complete a set of tasks, allowing the investigator to gauge meaning and understanding of the task. The principal investigator developed a script for the cognitive interviews, which was followed by the research staff for standardization. For this phase of the SHS development we used a paper questionnaire and allowed for participants to review each question and its corresponding answer choices during the interview. For each item, we asked participants if they thought other people their age would understand what the question was asking, what the answer choices meant, if the question and/or answers should be asked using different words, and to restate the question in their own words. We modified existing questions based on this feedback, and repeated the process with every modification of the survey until we received no further suggestions for refinement. We subsequently measured the Flesch-Kincaid grade level of the developed questionnaire.³⁰

SHS Pilot Testing

Once the SHS questions had been modified based on adolescent participant feedback, we uploaded our survey onto an ACASI-based survey platform, Illume Next (DatStat Inc^©, Seattle, WA) that allowed for use of branching logic. We then pilot tested the SHS with an additional group of 40 purposively sampled adolescent ED patients for ease of delivery, functionality, and satisfaction. After completing the SHS, participants then completed a second computerized questionnaire, which asked participants to provide responses about acceptability, specifically : a) ease of navigating the computer platform; b) survey length; c) comfort sharing responses via the SHS; and d) satisfaction completing the SHS. Each of these dimensions was assessed with a mean total Likert scale with a possible range of 1 to 5. If specific survey dimensions were not endorsed favorably by greater than 75% of the participants, we planned review and revision of the SHS. In order to meet the goal of developing a simple and feasible adolescent SHS tool, we also: a) measured the duration of survey completion for each participant; b) tracked any computer malfunctions; c) tracked patient length of stay (LOS). We used the Mann-Whitney test to compare the LOS of patients who pilot tested the SHS to the LOS of patients who met eligibility criteria for study participation but were not approached for enrollment during the same study period. We considered a p-value <0.05 as a statistically significant difference. In addition, to gauge the potential value of the SHS, we also asked participants whether information about their sexual history was obtained by their treating physician, and inquired about comfort in sharing this information during the medical visit.

During both the cognitive and pilot testing, medical care took precedence over survey administration. The survey could be interrupted for patient care and resumed after any necessary examinations were completed. Research assistants discussed timing of survey administration with each patient’s bedside nurse to minimize interruptions.

Results

Modified Delphi Process

Expert interviews yielded recommendations to adapt questions for the SHS from the YRBS and NSFG. None of the 13 Delphi panel experts had knowledge of adolescent sexual risk assessment surveys that were currently being administered in a clinical setting. The following themes regarding the development of a sexual health screening tool emerged from the interviews: confidentiality, infrastructure, and length. With respect to confidentiality, a few expressed concerns about incorporation of the SHS into the health record without identification of an additional system to safeguard sensitive sexual health information. In addition, a few expressed that adolescents should be offered a private area to complete the SHS or a privacy filter should be used on the tablet screen. Furthermore, others suggested that to increase comfort with participating in the SHS, the staff should explicitly review why the SHS is being administered and who will have access to the information. Another theme that emerged was infrastructure. Many of the experts felt that prior to such a screening tool being incorporated into clinical care, “buy in” from the clinical staff must be obtained and that clinicians must be comfortable addressing issues of sexual health. Further, staff to administer the SHS and follow up for patients with positive STI results should be identified. Additionally, seamless integration into current ED workflow would ensure the best success for incorporation of the SHS into clinical care. The last theme to emerge was related to survey length, and all experts agreed that the SHS should be short enough that adolescents would not lose interest in survey completion and so it would not prolong ED LOS.

We identified 25 items (including 4 demographic questions) from the NSFG and/or YRBS for inclusion in the first Delphi round. Twelve of the 13 experts participating in the key informant interviews provided electronic feedback on the incorporated items. There were no questions in which over 75% of the experts scored the item as not useful on the first Delphi round and therefore all 25 items were retained for the second Delphi round. One expert suggested a question regarding substance use prior to initiation of sexual activity, and this was also added to the second Delphi round, for a total of 26 items. Three questions were revised for clarity based on edits suggested by panel members.

The 26 items in round two of the modified Delphi panel were evaluated by 10 experts, and none were rated as not useful by >75% of the experts, so all were retained for Round 3 of the Delphi process.

Nine of the original 13 experts participated in round 3 of the Delphi panel. Of the 26 items, 8 were rated as important/very important by all of the experts. These questions related to presence of STI-related symptoms, history of prior STI/HIV testing, history of prior STI, confidential contact information for STI test results notification, method of contraception used to prevent pregnancy at last sexual intercourse, concern for risk of pregnancy, and interest in speaking with a health care provider about emergency contraception. No questions had a mean score <=3. Four questions had a mean score <4 with high variability (standard deviations >=1.2) and were removed from the SHS. These questions related to: lifetime number of sexual partners, ever having sexual intercourse without use of a condom, and gender of those to whom patient is sexually attracted. There was significant variability regarding an item asking if a patient had ever been forced to have sexual intercourse against one’s will (mean score: 3.4; SD 1.8). Many of respondent comments about this item related to concerns regarding mandatory reporting laws. Therefore, this question was removed from the final questionnaire. Twenty items (including 4 demographic items) were retained from the modified Delphi process (Appendix 1).

Cognitive Interviews

A purposive sample of 40 adolescents being cared for in the ED participated in cognitive interviews designed to evaluate understanding of the SHS. Almost all (93%) participants responded that that people their age would understand all 20 items. When asked to restate the questions in their own words, the participants were able to appropriately restate the purpose of all the questions. Conversely, although 90% of the participants reported people their age would understand an item regarding emergency contraception, only 7 participants specifically mentioned pregnancy prevention when asked to define emergency contraception. For this reason, we planned to provide an explanation of emergency contraception in the second iteration of the SHS.

All cognitive interview participants thought people their age would understand the answer choices on the sexual health questions, with the exception of items asking about methods of pregnancy prevention used during sexual intercourse. Three participants suggested that not all people their age would understand what some of the methods of pregnancy prevention were, specifically citing injectable birth control, birth control ring, implantable birth control, intrauterine device, withdrawal, and emergency contraception as potentially confusing for peers to understand. To address these concerns, we planned to revise the SHS with information detailing these different forms of birth control.

Participants answered how comfortable they were answering the questionnaire items, using a 5-point Likert scale ranging from “Very Uncomfortable” to “Very Comfortable.” Eighty-four percent responded that they were comfortable answering sexual health questions on the survey (27% “very comfortable”, 57% “comfortable”). The remainder of the respondents stated they were “neutral” and no participants responded that they were “Uncomfortable” or “Very Uncomfortable” answering these questions. The Flesch- Kincaid grade level score for the SHS was 6.3. When asked if there were any items they thought should be added to the questionnaire, suggestions included items about sexual abuse (3 persons), suicide (1 person), peer pressure in regard to sexual activity (2 persons), partner violence (2 persons), abortion (1 person), information on different types of STIs (1 person), how to talk to a partner about STIs (1 person), and where to go for STI testing (1 person). In response to these suggestions, we developed a publicly available website to provide information about STIs, location of STI testing sites by zip code, and information on how to protect oneself from STIs, including communication with one’s partner.

Pilot Testing

Because feedback from the cognitive interviews resulted in minimal modifications, we simultaneously performed a second round of cognitive interviews while pilot testing the ACASI-SHS that we uploaded onto the host survey platform. We then recruited another purposive sample of 40 adolescent ED patients based on the sampling schema previously described. All participants answered that people their age would understand all items on the questionnaire that ask about reproductive/sexual health with no questions or suggestions about any of the survey items.

The majority of patients (80%) reported having “lots of experience” with computers. All participants responded that the computer-assisted SHS was “easy” (65.7%) or “very easy” (34.3%) to use. The majority of the patients “liked/very much liked” the look of the computer-assisted survey. With respect to survey length, the majority of the participants (88.6%) responded that the length was “just right” with only 2 participants responding that the survey length was “too long.”

The majority of participants (76.5%) responded that they preferred to answer sexual health-related questions on the computer, with only 2 respondents stating they preferred a face-to-face interview with a physician/nurse (Figure 1). The majority of respondents (94.1%) believed that computers were a good way to ask sexual health related questions and 82.3% stated that they would recommend their friends to complete the SHS if they came to the ED for care.

Figure 1.

Preference for Answering Questions Related to Sexual Health

The mean and median length of the SHS survey completion was 17.5 minutes (+/−6.7 minutes) and 15.0 minutes, respectively. No study participants used the audio feature of the survey. There were no computer malfunctions and minimal interruptions. Furthermore, there was no difference in median LOS between SHS participants (230.0 minutes) and adolescents who met inclusion criteria but were not approached for participation in SHS pilot testing (219.0 minutes) (p=0.7).

Over half (52.9%) of the respondents reported that they had not been asked about sexual experience by a healthcare provider in the ED during their visit. Of the patients who had been asked about sexual activity by a health care provider, 40.0% reported that it was easier to answer sexual health-related questions on the computer than face-to-face with the provider, and another 53.3% reported that it was just as easy to answer sexual health questions on a computer as compared to a face-to-face interaction with a provider. Overall, the majority (94.1%) of the participants “liked” or “did not mind” answering sexual health questions via a computerized screener.

Discussion

In this article, we have described a rigorous and iterative research process using qualitative and survey methodology to develop an ACASI SHS that can efficiently and effectively be delivered to adolescents in the ED. Overall, the survey was well received by adolescent patients and did not prolong patient LOS. The development of the ACASI-based SHS is innovative. To our knowledge, there are no ED-based STI screening programs that have been incorporated into routine practice. The SHS has the potential to efficiently, accurately, and confidentially elicit sexual health information in a standardized manner that could be useful for determining whether a patient would benefit from STI testing and for the provision of risk-tailored interventions.

We used a modified electronic Delphi process to incorporate expert opinion for the development of the SHS. Using a rigorous idea-generating and prioritization process, we developed a 20-item content-valid questionnaire. The concept of safeguarding sensitive health information in an adolescent patient’s health record arose during our one-on-one telephone interviews with our expert informants. This concern has been raised by many national organizations, including the American College of Obstetricians and Gynecologists and the Society for Adolescent Health and Medicine, and most recommend notifying adolescents that parents may have access to their medical records. This would require clinicians to work with their hospitals to develop policies and procedures that safeguard sensitive health information.^31–33 We have begun these discussions within our institutions. Our experts also recommended that the screening tool should be short enough to maintain the interest of adolescents completing the survey and not prolong ED LOS. During pilot testing, adolescents found the SHS length to be acceptable and those who participated did not have an increased length of ED stay compared to a sample of adolescents who did not participate.

Cognitive interviews with adolescent ED patients found that the SHS was understandable by our adolescent patient population. Furthermore, pilot test participants in our pediatric ED found the SHS favorable. Patients preferred completion of the ACASI SHS over face-to-face interviews with clinicians for sexual health-related questions. This finding is consistent with other studies that have found that computer-assisted surveys may overcome barriers to obtaining accurate sensitive health information.^19,21,22,24 Our study methodology followed robust processes for item development and refinement: we combined expert and adolescent opinion for content, performed cognitive testing to assess understanding, and pilot tested to ensure questions were considered appropriate and understandable by our end-users. If future investigators wish to modify survey items to tailor the survey to other settings, we recommend that they follow a similar process of cognitive interviews and pilot-testing to ensure the survey is valid for their intended population.

There are some potential limitations to this study. We only recruited adolescents in the ED at two pediatric ED sites, both large urban academic pediatric hospitals serving patient populations of primarily low socioeconomic status. Therefore, our results may not be generalizable to all adolescent ED patients or patients at other EDs. However, our purposive sampling did provide the perspectives of varying age groups, gender, and chief complaints. Additionally, in concordance with the demographics of the population served by our ED, more than 75% of the patients who participated in pilot testing were of non-Hispanic black race and ethnicity. Our study was not designed to assess racial/ethnic differences in attitudes towards the SHS. Because there is no reference standard for the measurement of sexual risk behaviors in an adolescent ED population, we were unable to perform criterion-related validity testing of the SHS. We employed rigorous methodology to perform translation (face and content) validity, and produce a survey that was understandable and acceptable to adolescent ED patients. We measured construct validity during our cognitive interviews by ensuring that our end-users were able to explain the meaning of each survey item and that it was consistent with our intention for each question. We did not test for internal consistency because our survey questions were factual rather than attitudes and beliefs. An example is: “During the past 3 months, with how many people have you have sexual intercourse? Additionally, as with all survey data, responses may be subject to social desirability bias and/or confidentiality concerns. However, our participants reported greater comfort sharing sexual health information with the ACASI SHS as compared to face-to-face interviews with ED providers. Finally, patients were approached early in their ED stay, and therefore, this may have accounted for there being no difference in ED length of stay between the study participants and non-participants. However, we deliberately approached patients early in their ED stay to simulate ideal conditions for when the SHS would be incorporated into clinical practice, because the goal of the SHS is to provide real-time clinical decision support during an ED visit.

In conclusion, we successfully developed an ED-based, understandable, acceptable, and content-valid ACASI-based sexual health screening tool that can be used to identify adolescents at risk for STIs. Further research will evaluate the predictive validity of the SHS in identifying youth who have STIs, as well as the utility of the SHS for clinical decision support and provision of risk-tailored behavioral interventions. Studies evaluating the effectiveness of the SHS once it has been incorporated into routine clinical care are also warranted.

Table 1.

Demographic Description of Participants Involved in Pilot Testing SHS

Demographic		n (%)
% Female		20 (50.0%)
Mean age (years)		15.8 (SD +/−1.6)
Racial/Ethnic Group	Non-Hispanic White	3 (7.5%)
	Non-Hispanic Black	32 (80.0%)
	Hispanic	5 (12.5%)
Highest Completed Grade	7th	3 (7.5%)
	8th	4 (10.0%)
	9th	9 (22.5%)
	10th	10 (25.0%)
	11th	7 (17.5%)
	12th	7 (17.5%)
Prior Sexual Experience		19 (47.5%)
Potential STI-Related Chief Complaint		20 (50.0%)