Development and Clinical Use of Rapid Assessment for Adolescent Preventive Services (RAAPS) Questionnaire in School-based Health Centers

Abstract

Introduction

The purpose of this study is to describe the development and clinical use of Rapid Assessment for Adolescent Preventive Services (RAAPS), a time-efficient screening tool to assess for multiple adolescent risk behaviors.

Method

A retrospective chart audit was conducted to obtain descriptive data of middle school (N = 106) and alternative high school (N = 39) adolescents who completed the 17- to 18-item RAAPS questionnaire. Surveys assessed providers’ evaluations of the RAAPS.

Results

Descriptive statistics and qualitative analysis indicated that providers using the RAAPS were able to identify risk behaviors/factors, provide counseling for these behaviors, and refer 26% of 9- to 15-year-olds and 43% of 16- to 20-year-olds for further assessment or ongoing risk counseling. In one brief clinic visit, the providers were able to address and document most risk behaviors/factors reported by the adolescents.

Discussion

Although psychometric analysis is needed, the RAAPS is a time efficient and comprehensive risk assessment tool. Early risk identification can assist providers in tailoring specific preventative education counseling and intervention programs that are geared to meet the specific needs of the adolescent population.

The majority of mortality and morbidity of youth today is attributed to social and behavioral factors (Eaton et al., 2006). In fact, more than 70% of all illnesses, disabilities, and deaths among youths and young adults are related to six types of health risk behaviors (Eaton et al.). These behaviors include intentional and unintentional injuries and violence, alcohol and other drug use, sexual behaviors leading to unintended pregnancies, sexually transmitted diseases and HIV/AIDS, tobacco use, unhealthy dietary behaviors, and physical inactivity (Eaton et al.). Many of these risky behaviors result in adverse health consequences and are related to behavioral decisions made by youths (Klein et al., 2001). Furthermore, risk behaviors are interrelated and likely to co-occur (Jessor, 1991; Weden & Zabin, 2005).

With more than 70% of adolescents visiting a health care provider each year (Fleming, Elster, Klein, & Anderson, 2001), many health care providers believe that cost-effective, comprehensive clinical preventive services for adolescents can reduce risky behaviors, lower overall medical costs, and decrease the burden of suffering (Brindis, Park, Ozer, & Irwin, 2002). Through health counseling and screening, nurse practitioners can play a vital role in reducing risk and improving adolescent well-being (Grunbaum et al., 2002). In an effort to screen and prevent risk behaviors, national health organizations have developed comprehensive preventive service guidelines such as the Guidelines for Adolescent Preventive Services (GAPS), Bright Futures, Put Prevention Into Practice, and tools from American Academy of Family Physicians and American Academy of Pediatrics to be used by primary care providers (Centers for Disease Control and Prevention [CDC], 2003; Child and Adolescent Health Program, 1998; Green, Palfrey, Clark, & Anastasi, 2001; U.S. Department of Health and Human Services, 2000).

Despite the availability of various screening instruments and the recommended national guidelines for provision of clinical preventive services, providers still do not adequately address the recommended preventive services for adolescents (Halpern-Felsher et.al., 2000). Barriers to adolescent preventive services include issues pertaining to access to care, provider issues, and adolescents’ perception of providers (Brindis et al., 2002). Provider barriers include lack of time, confidence, training, inadequate reimbursement for prevention, and reluctance to address psychosocial issues that contribute to adolescent mortality and morbidity (Brindis et al.; Fleming et al., 2001). Although providers are willing to deliver preventive services (Grayson, Bush, & Ryan, 2000), lack of time has been listed as the most prominent barrier in providing appropriate preventive care to patients (Stange, Flocke, & Goodwin, 1998). Thus, a time-efficient, comprehensive screening tool is needed that nurse practitioners, physicians, and social workers can use to assess for multiple risk behaviors of adolescents.

A time-efficient, comprehensive screening tool is needed that nurse practitioners, physicians, and social workers can use to assess for multiple risk behaviors of adolescents.

RAPID ASSESSMENT FOR ADOLESCENT PREVENTIVE SERVICES

Although the GAPS has been considered the gold standard comprehensive screening tool for adolescents, the GAPS assessment tool is lengthy (four pages, 60-71 questions), time consuming, and often condensed for practice. Written permission was received from the American Medical Association (AMA) to modify questions from the GAPS and to use them as part of the Rapid Assessment for Adolescent Preventive Services (RAAPS). Consequently, the authors developed an abbreviated version of an adolescent risk assessment modeled on the GAPS tool, the RAAPS, which was then used at three school-based health centers (SBHC). This article describes the development and clinical use of the RAAPS, a time-efficient preliminary screening tool to assess for multiple adolescent risk behaviors at SBHC.

Development of RAAPS

The RAAPS was developed based on the AMA GAPS assessment form, CDC Youth Risk Behavior Survey (YRBS), and a panel of expert adolescent providers. In addition, special emphasis was placed on the top six risk behaviors/factors stated previously that contribute most to the mortality and morbidity of adolescents as identified by the CDC. Using a format similar to the AMA GAPS, two forms of the RAAPS were developed: the RAAPS Middle School Questionnaire and the RAAPS High School Questionnaire. The RAAPS was developed to assess the most serious risk behaviors in each of the identified areas. On each RAAPS form, the health risk questions are divided into nine sections: eating/weight, physical activity, safety/violence, tobacco, alcohol, drugs, development, emotions, and friends/family. Questions relating to other adolescent risk behaviors/factors were omitted from the RAAPS to keep it concise and user friendly. However, pertinent questions were combined to obtain the most information with the fewest number of questions (e.g., “Do you have any serious issues or worries at home or at school?”). A positive response to this item is followed up with further investigation and counseling by the providers during the clinical visit. This strategy allows the providers to determine the type and level of the risk and potential need for referral.

Questions from the AMA GAPS assessment form and the YRBS were reviewed for relevance and succinctness based on our identified highest priority risk behaviors/factors. For instance, in the middle-older AMA GAPS form, 11 questions are used to screen for development. Questions such as, ”Are you using a method to prevent pregnancy? (Which?)” and “Do you and your partner(s) always use condoms when you have sex?” were combined to ask, “If you do have sex, do you always use a method to prevent pregnancy (birth control pills/condoms)?” in the high school RAAPS. Many of the omitted areas from the AMA GAPS and YRBS referred to friend use. For example, questions such as, “Do any of your close friends ever use marijuana or other drugs, or sniff inhalants?” and “Have any of your close friends ever had sexual intercourse?” were excluded.

Overall, the RAAPS questions are designed to include developmentally appropriate characteristics (e.g., concrete vs. abstract) and age-associated risk behaviors. Both forms had Flesch-Kincaid Grade Level readability scores at the 5th-grade reading level. In the middle school questionnaire, a question regarding sexual activity is used in the form of popular jargon, that is, “hooked up,” with “(had any type of sex including oral)” following the jargon for further explanation. This word was added upon review by the providers who were familiar with the adolescents’ language at the schools. On the other hand, in the high school RAAPS questionnaire, the question regarding sex is explicitly stated and is followed by an additional question of contraceptive use. A question regarding wearing protective gear or a helmet when rollerblading, biking, or skateboarding was omitted because the majority of high school adolescents are no longer engaging in these activities; instead, they are driving or riding in vehicles to get around. Additionally, with the opportunity to drive and increased developmental likelihood to participate in risk behaviors, the high school RAAPS has additional questions regarding drinking or drug use and driving, inappropriate use of nonprescription drugs, and contraceptive use. On both forms, one question, the desire to know more about abstinence, HIV/AIDS, and other sexually transmitted diseases, was included as an information-seeking behavior rather than a risk behavior.

The RAAPS demonstrated good face validity because the questions were modeled after the GAPS and included the top adolescent risk behaviors. In addition, the adolescents were able to complete the RAAPS without assistance, and any questions were addressed by the providers during the visit. The content validity of the items on the RAAPS were reviewed by adolescent health care providers composed of health professionals (family and pediatric nurse practitioners, physicians, and social workers) and pediatric risk behavior researchers who were familiar with the intended audience and had content expertise in adolescent health through research and professional experience. After 3 months of use in the SBHCs, the RAAPS was revised based on a survey of the RAAPS findings and input from adolescent providers who used the tool. The revisions included the addition of two questions related to the issue of threatening behavior and bullying for middle school RAAPS and drinking or using drugs and driving for the high school RAAPS (and minor wording and format changes to facilitate documentation). Revisions since have included an addition of a separate sexual abuse question and rewording of the physical and emotional abuse question to assist in obtaining the most accurate information from the students completing the RAAPS. An abbreviated sample of the Middle School RAAPS can be seen in Figure 1. The final versions of the RAAPS can be found and downloaded from the following Web site: http://www.a2schools.org/rahs/raaps_survey. The results reported in this article are from data collected using the original RAAPS prior to revisions.

FIGURE 1.

Abbreviated Middle School RAAPS Questionnaire. To download the complete versions of the Middle School and High School RAAPS, please visit the following Web site: http://www.a2schools.org/rahs/raaps_survey.

METHODS

This descriptive study investigated the clinical use of the RAAPS for adolescent risk assessment in SBHCs located in the suburbs of southeastern Michigan. A retrospective chart audit was conducted to obtain descriptive data of adolescents who completed the RAAPS questionnaire. The RAAPS Provider Evaluation surveys consisted of five open-ended questions about their opinions toward using the RAAPS with the adolescents in SBHCs (i.e., “What are the strengths of RAAPS? Weaknesses? Revision suggestions? Other comments? Approximate length of time for students to complete the RAAPS?”). Another set of questions inquired about providers’ attitudes regarding the use of the RAAPS to screen for risk behaviors and develop rapport with the students, along with ease of use. The University of Michigan Health Sciences Institutional Review Board approved the study.

Sample

Located in the suburbs of south-eastern Michigan, the SBHCs participating in this study include two middle schools and one alternative high school. The alternative high school enrollment encompasses adolescents who have failed traditional high schools for a variety of reasons, including those who have been suspended or expelled. Four nurse practitioners, three social workers, and two physicians provide ongoing care at the SBHCs. Generally, the experience of the main providers ranged from 10 to 25 years in pediatric and adolescent health care settings. Because the RAAPS was designed as a quick, initial screening tool regardless of the type of clinical visit, all adolescents except for those present for an initial health maintenance examination (N = 50) completed the RAAPS questionnaire. The adolescents who received an initial health maintenance examination completed the AMA GAPS assessment as required by SBHC protocol for these annual examinations. The other adolescents who were identified as high risk from the RAAPS questionnaire completed the AMA GAPS assessment in their next visit.

Chart audits were conducted on the medical record charts of all adolescents who completed the RAAPS during September 2004 to December 2004 (N = 213). Sixty-eight questionnaires had incomplete data; thus, data were collected and analyzed only for those who had complete information (N = 145). This group included charts of 106 younger adolescents aged 9 to 15 years (64% female students and 36% male students) and 39 high school adolescents aged 16 to 20 years (62% female students and 38% male students). The distribution of female and male students is consistent with the SBHC populations. Provider evaluations of RAAPS were received from three social workers and four nurse practitioners, but not from the two physicians.

Procedure

Clinical administration

Adolescents seen at the SBHC, except for those who were present for an initial health maintenance examination, completed the RAAPS prior to meeting with one of the health providers. The provider then reviewed the RAAPS and interviewed the teens to obtain additional information based on the reported risk behaviors identified in the RAAPS. This method of adolescent reporting and provider interviewing allowed for the providers to clarify with the adolescents their risk behaviors, develop risk reduction plans, and then determine the need for further management or referral. For each question, the SBHC developed a protocol for the providers to follow if a risk behavior was identified (e.g., counseling using motivational interviewing, provision of health education materials, and referrals). To facilitate documentation, the RAAPS includes a section/column for the providers to document the specific intervention that was used for each of the identified risk behaviors. This practice allows for documentation to occur directly on the questionnaire correlating to each identified risk behavior, reducing time spent on documentation in clinical notes. If a clinical note was generated, a “see RAAPS” is all that is necessary to link the clinical note to the RAAPS that was discussed during the visit.

Data collection

A research assistant collected the RAAPS data from SBHC visits and removed all identifiers prior to data entry. Data from the RAAPS forms then were entered in a confidential manner with anonymity of the adolescents preserved by the primary author. After the fourth month of RAAPS implementation, the adolescent providers evaluated the RAAPS through a survey. Content analysis was used to evaluate the qualitative comments from the provider surveys.

Data Analysis

Descriptive statistical analyses were used to summarize risk behaviors reported by the adolescents and interventions documented by providers. Using the Statistical Package for the Social Sciences (SPSS), Version 13.0 (SPSS, Chicago), the frequencies of the risk behaviors/factors were analyzed by total sample, sex, and school status. Chi-square test of proportions was used to assess gender differences among risk behavior responses. Qualitative content analysis was used to determine the adolescent providers’ responses.

RESULTS

Sample Characteristics

Table 1 shows the sample characteristics of adolescents who participated in the RAAPS questionnaire. More than half of the sample was African American and between the ages of 12 and 13. The mean age of students was 12 years for middle school adolescents and 17 years for alternative high school adolescents. The average number of risk behaviors identified by younger female adolescents (M = 2.3, SD = 1.4), and male adolescents (M = 2.4, SD = 1.6) were similar. Higher mean total number of risk behaviors was reported by the older adolescents than by the younger adolescents, but the mean number of reported risk behaviors/factors was similar for both older female adolescents (M = 6.3, SD = 2.9) and male adolescents (M = 6.3, SD = 1.9).

TABLE 1.

Characteristics of adolescents

Characteristic	N = 145	%
Sex
Female	92	63.4
Male	53	36.6
Race/ethnicity*
Black	73	54.1
White	46	34.1
Other (Asian,   Hispanic, Biracial)	16	11.8
Age (y)
9-11	22	15.2
12-13	69	47.6
14-15	14	9.6
16-17	21	14.5
18-20	19	13.1

^*Race/ethnicity data from 10 students were missing. The total N = 135 was used.

Risk Behavior/Factor Prevalence

The three most prevalent risk behaviors/factors reported by middle school adolescents included not usually wearing protective gear or helmet when rollerblading, biking, or skateboarding (77%); often feeling sad or down in the past month (30%); and physical inactivity (not exercising or playing hard for at least 20 minutes, three or more times a week) (17%). Although one question on the RAAPS, the desire to know more about abstinence, HIV/AIDS, or other sexually transmitted diseases, was not a risk behavior/factor, it was selected in the RAAPS by one quarter of the middle school adolescents (Table 2). Among the alternative high school adolescents, the three most prevalent risk behaviors/factors were ever having sex (97%), ever gotten drunk or high on beer, wine, or other alcohol (74%), and smoking or chewing tobacco (67%).

TABLE 2.

Top five risk behaviors/factors of 9-to 15-year-old adolescents

Risk behaviors/factors	Male % (N = 38)	Female % (N = 68)
Do not usually wearing protective gear/helmet  when rollerblading, biking, skateboarding	82	75
Often feel very sad or down for past month	26	32
Want to know more about saying no to sex,  HIV/AIDS, or other sexually transmitted diseases	21	29
Do not exercise or play hard (e.g., running, dancing,  basketball, swimming, fast bicycling, etc.) for at least  20 minutes, three or more times a week	25	8
Do not always wear a lap/seat belt when  in a car, truck, or van	16	9

Tables 2, 3, and 4 represent the most prevalent reported items on the RAAPS questionnaire separated by sex and school status. The only significant gender differences were found in physical inactivity for both younger (χ² = 8.65, P < .003) and older adolescents (χ² = 10.41, P < .001), with girls reporting more inactivity than boys.

TABLE 3.

Top five risk behaviors/factors of 16-to 20-year-old male adolescents

Risk behaviors/factors	Male % (N = 15)
Ever had any type sexual intercourse (sex)	93
Ever gotten drunk or high on beer, wine coolers, or other alcohol	87
Smoke cigarettes or chew/use smokeless tobacco	80
Smoke marijuana, or use other street drugs, steroids, or inhalants	53
Often feel very sad or down for past month	67

TABLE 4.

Top five risk behaviors/factors of 16-to 20-year-old female adolescents

Risk behaviors/factors	Female % (N = 24)
Ever had any type sexual intercourse (sex)	100
Ever gotten drunk or high on beer, wine coolers, or other alcohol	67
Smoke cigarettes or chew/use smokeless tobacco	58
Do not exercise or play hard (e.g., running, dancing,  basketball, swimming, fast bicycling, etc.) for at least  20 minutes, three or more times a week	58
Often feel very sad or down for past month	46

Providers’ Intervention Documentation

Providers documented intervention of the identified risk behaviors/factors for 89.5% (N = 221) of 9- to 15-year-olds and 95.5% (N = 234) of 16- to 20-year-olds. The most common intervention technique used by providers for identified risk behaviors was clarification and review of the risk behaviors/factors through direct questioning, motivational interviewing, and/or provision of health educational materials. Only 34 identified risk behaviors did not have provider intervention documented on the RAAPS. From these, the provider did not document any intervention on the entire RAAPS form or the provider missed documenting an intervention for one or more of the identified risk behaviors. More referrals were made for 16- to 20-year-olds (43.6%) than for 9- to 15-year-olds (26.4%). The majority of the adolescents were referred to a social worker if they reported highly sensitive risk behaviors/factors (e.g., carrying a weapon, abuse, or suicide) or if after the provider RAAPS assessment, the report of sadness for the past month was significant enough to be referred.

SBHC Providers’ Evaluation of the RAAPS

The providers indicated the RAAPS was a “quick, easy way to identify kids at risk.” According to the providers, the strengths of the RAAPS include its concise format, ease of use, ability to assess for major risks in a short time, and provision of the opportunity to establish a rapport with the adolescents. A nurse practitioner stated that by using the RAAPS, “You get to know the teens right away.” In contrast, the weaknesses to the RAAPS were mainly related to its grammatical structure. Providers commented that the grammatical structure of some of the questions was difficult to understand and interpreted “too literally” by middle school adolescents. Another provider stated that the RAAPS could be improved by using a “strength approach.” The RAAPS has since been revised to reduce these reported weaknesses. Despite the suggested grammatical changes of the instrument, the providers thought that the RAAPS stimulated meaningful discussions during the clinical visit and allowed the providers to clarify any questions and misinterpretations. On average, the providers stated the RAAPS took about 5 to 10 minutes for adolescents to complete.

DISCUSSION

Nurse practitioners can use the RAAPS as a comprehensive, time-efficient adolescent risk assessment questionnaire in clinical practice. By being able to quickly identify risk behaviors, nurse practitioners and adolescents would have more time and opportunities to discuss behaviors that are specific to each adolescent. The SBHC providers perceived that the RAAPS encouraged communication and disclosed risk behaviors. Using the RAAPS, the SBHC providers were able to address and document the most common adolescent health risk behaviors in a single visit. This finding suggests that a large number of relevant health concerns can be addressed in a single visit. Furthermore, this supports previous research done by Epner, Levenberg, and Schoeny (1998), who demonstrated that providers attempted to discuss all reported health risk behaviors during a single clinical visit.

Because providers do not always record everything that occurs during a clinical visit, it is questionable whether chart audits accurately capture the patient encounter (Gadomski, Bennett, Young, & Wissow, 2003). Depending on the expectations of the working environment, providers may document more than what actually occurred or document less, especially related to counseling (DiMatteo, Robinson, Heritage, Tabbbarah, & Fox, 2003). Regardless, the high percentage of provider-documented intervention using the RAAPS is consistent with other GAPS assessment research, which demonstrated increased rates of health counseling and preventive services documentation after implementation of the AMA GAPS (Gadomski et al., 2003; Klein et al., 2001). For the risk behaviors/factors that were not documented, the providers may have addressed the concerns in the progress notes section of the medical record. However, for this research, we did not have access to the complete medical records.

Past research has shown that the consistent use of structured encounter forms improves the completeness of providers’ documentation on preventive services (Grant, Skinner, Fleming, & Bean, 2002). If used annually, the providers have documentation to track improvements or changes in the lives of the adolescents. In addition, the tool assists providers with documentation and guides the direction of the clinical visit. Because the RAAPS is a screening tool, it is highly recommended that protocols for risk intervention should be established prior to its use.

In the 2005 National YRBS, 85.9% of adolescents rarely or never wore their helmets while riding their bicycles (Eaton et al., 2006). This item was also the highest risk behavior identified in the YRBS. In our middle school RAAPS, 77% of adolescents reported not usually wearing protective gear or a helmet when riding, biking, or skateboarding. Despite a much smaller sample size in the RAAPS, it appears that the RAAPS is sensitive to adequately detect a similar percentage of potentially unsafe riding behaviors of adolescents.

Reports of feeling sad in the past month was the second most common risk behavior/factor. It was identified by 30% of the younger adolescents and by more than 50% of the older adolescents on the RAAPS. The reported prevalence of adolescent depression in the community is 4% to 8%, but reports of elevated depressive symptoms occur in about 15% of the adolescent population (Richardson and Katzenellenbogan, 2005). Inquiring about feelings of persistent sadness and hopelessness are two methods to briefly screen adolescents with potential depression and dysthymia (Richardson and Katzenellenbogan). Identification of youth with depressive symptoms can lead to early intervention that can preclude the development of other co-morbid disorders associated with depression (American Academy of Child and Adolescent Psychiatry, 1998).

The desire to acquire more information regarding abstinence, HIV/AIDS, and other sexually transmitted diseases was reported by one quarter of the younger adolescents in the RAAPS survey. Despite what the public may have feared, comprehensive sex and HIV education programs do not increase teen sexual intercourse (Kirby, 2002). In fact, some of these programs have shown to decrease or delay sexual intercourse and increase condom or contraceptive use among sexually active teens (Bennett & Assefi, 2005). Currently, the middle schools use an abstinence-based program while the high school uses a comprehensive sexual education program (J. Salerno, personal communication, June 23, 2005). This finding is a possible indication of the need for the schools affiliated with the SBHC to evaluate or modify the current reproductive curriculum. By extracting successful components of comprehensive sex education programs, abstinence-based programs may be more beneficial in reducing teen sexual intercourse.

The RAAPS reported top-three risk behaviors/factors that were similar to the findings of the 1998 CDC Alternative High School Youth Risk Behavior Survey (ALT-YRBS) (Grumbaum, Lowry, & Kann, 2001). Adolescents attending alternative high schools are at greater risk of participating in health risk behaviors that jeopardize their current and future health compared with adolescents attending traditional high schools (Grumbaum et al., 2001). The high rate of sexual activity also is combined with other health risk behaviors such as alcohol use and smoking. This finding is consistent with the research on risk behavior interrelationships (Weden & Zabin, 2005) and findings that engaging in multiple risk behaviors increases with age (Brener & Collins, 1998). In addition, more adolescents in alternative schools come from disadvantaged home backgrounds, engage in multiple health risk behaviors, and experience greater emotional problems than do adolescents from regular high schools (Denny, Clark, & Watson, 2004). Therefore, routine screening and health guidance in these types of settings is of vital importance to reducing morbidity and mortality of adolescents.

Although the medical records provided a feasible method of risk behavior data collection, adolescent behavior outcomes were not available at the time of data collection. For instance, provider documentation of clinic visits does not capture the full breadth of the patient-provider encounter and does not completely address the outcomes of counseling provided. Face and content validity were provided, but additional measures of construct validity and reliability are needed. The RAAPS was developed for and used at SBHCs for risk assessment and intervention; hence, the current version of the RAAPS is not generalizable to other settings.

Future research should focus on further refining the RAAPS and examining its clinical impact. Focus groups with providers and adolescents who have used the RAAPS in this study would allow for an assessment of clarity and readability of items and other design considerations to facilitate its use. In addition, a comparison study using the original AMA GAPS and the RAAPS may further aid in the development of the RAAPS. After the revisions, the RAAPS should be used with a larger, more diverse sample to test the validity, reliability, and clinical feasibility of the screening tool. Because the RAAPS was designed for SBHCs, additional testing with other community health centers and primary care offices are warranted. Finally, the RAAPS could be used to examine adolescentprovider interaction and its influence on adolescent health behavior outcomes over time.

This study is an initial effort to conduct an adolescent risk assessment through a time-efficient, comprehensive screening questionnaire. Results of initial application of this tool in three SBHCs showed that providers using the RAAPS were able to detect the top risk behaviors/factors associated with poor outcomes in adolescents and refer them for further management. Furthermore, providers were able to discuss and document risk behaviors/factors reported by the adolescents in a single clinical visit. The RAAPS has clinical implications for tailoring of adolescent preventative education and intervention programs that are geared to meet the specific interests of the adolescent population.