Sexual Abuse Prevention Concept Knowledge: Low Income Children are Learning but Still Lagging

Abstract

School based child sexual abuse (CSA) prevention programs have proliferated since beginning over 30 years ago. Research on program effectiveness has emphasized summative outcomes while under-reporting implementation and process fidelity, limiting reliable comparisons across programs and populations. External validity is further limited by racially and economically homogenous samples, or a failure to report such demographics. The current work presents data from a CSA prevention program delivered to 2^nd and 3^rd grade public school children. A formative concept and item level analysis on the Children’s Knowledge of Abuse Questionnaire at baseline and four-week follow-up, as well as fidelity and implementation data, are reported. Results show that children learn and retain certain concepts but vary widely in knowledge across items. This sample of low-income, minority children also show lower baseline and “learned” abuse prevention knowledge compared to published norms, which is not attributed to implementation variability. These data give critical insight into workshop effectiveness that is masked in summative reporting, yet essential to adapting and strengthening school-based CSA programs. Knowing the long-term consequences of adverse childhood experiences and disparate health outcomes linked to race and socioeconomic status, it is imperative to adequately assess CSA prevention program impact across diverse populations.

School based child sexual abuse (CSA) prevention programs are a popular prevention strategy, and have gained significant traction since their introduction in the late 1970s (Collin-Vezina, Daigneault, & Hebert, 2013; Finkelhor, 2009; Zeuthen & Hagelskjaer, 2013). The need for effective prevention and education efforts is driven by increased understanding of the rampant prevalence and harmful consequences of adverse childhood experiences, and CSA in particular. It is estimated that 1 in 5 children will be sexually assaulted before the age of 18, with varying incidence and prevalence rates depending on the definition of abuse used, the age of the cohort sampled, and whether it is justice system- or self-reported (Finkelhor, 1990, 2005; Centers for Disease Control and Prevention, 2016; Turner, Finkelhor, & Ormrod, 2006). One constant, however, is that CSA is widely underreported (Collin-Vezina et al., 2013; Hébert, Tourigny, Cyr, McDuff, & Joly, 2009).

CSA is associated with numerous negative psychological, behavioral, and emotional effects, including post-traumatic stress disorder, sexual revictimization in adulthood, and substance abuse (Paolucci, Genuis, & Violato, 2001; Chiu et al., 2013; Heffernan et al., 2000; Bensley, Van Eenwyk, & Simmons, 2000). These issues may be mediated via neural and neuroendocrine changes stemming from traumatic abuse, such as hypothalamic-pituitary-adrenal dysfunction (Trickett, Noll, Susman, Shenk, & Putnam, 2010) and reduced hippocampal volume (Stein, Koverola, Hanna, Torchia, & McClarty, 1997). Costs associated with the negative outcomes of CSA, including mental health care and medical services, attenuated earnings, and victims’ low quality of life, are conservatively estimated at $24 billion nationally (Miller, Cohen, & Wiersema, 1996). Thus, effective prevention and awareness efforts, rather than reactive treatment, are instrumental to help reduce the individual and systemic toll of CSA.

School based CSA prevention programs are favorable because of their low administration costs, relative ease of implementation, and ability to reach large numbers of children without stigmatizing any one particular group (Finkelhor, 2009). Although the number of school based programs currently implemented in schools is unknown (Wurtele, 2009), this primary prevention method is preferred over others, such as treating individuals that have perpetrated abuse (tertiary prevention; Letourneau, Eaton, Bass, Berlin, & Moore, 2014). Offender management (e.g. longer incarceration, offender registries) is also widely accepted because of its punitive nature, but is based on an inaccurate stereotype that abusers are strangers and ‘monsters’ (Collin-Vezina et al., 2013), rather than the reality that offenders are almost always known to the victim.

Many positive effects of school based programs have been reported, including increases in CSA prevention-related knowledge, self-protective skills, and increases in self-esteem, with little evidence of anxiety, or sexualized or concerning behaviors linked to program content (Zeuthen & Hagelskjaer, 2013). There is, however, growing concern within the literature regarding replicability and consistency of CSA prevention program implementation. A recent meta-analysis of 22 CSA efficacy studies highlights pervasive inconsistency in reporting implementation details, an almost complete absence of fidelity measurement, and a bias of reporting outcomes but not process (Topping & Barron, 2009). This has negatively impacted the ability to compare programs and outcomes across studies and replicate published work.

Understanding the impact of CSA prevention programs has also been hindered by a focus on summative outcomes. As highlighted by Tutty (1994) more than two decades ago: “reporting average gains in knowledge, while important, provides little information as to whether children are less familiar with certain concepts even before participating in a prevention program, or whether children learn and remember all concepts” (p. 182). With the magnitude of program impact generally on the order of only one to two questions, albeit with moderate to large effect sizes (Zwi et al., 2007), reporting detailed analyses of concept and item specific learning is of great importance.

Comprehensive data reporting is necessary to evaluate other moderating factors, such as age, ethnicity, and socioeconomic status (SES). Few studies have assessed CSA program effectiveness in racially and economically diverse samples, but reduced effects have been reported (Daigneault, Hebert, McDuff, & Frappier, 2012). Additionally, while the association between race or SES and increased risk for sexual abuse is inconclusive, living in poverty is a predictive factor for children to experience both physical and sexual abuse (Collin-Vezina et al., 2013). Therefore, while CSA affects children of all ages, races, and SES, it is possible that demographic factors moderate prevention learning in disadvantaged children, resulting in reduced program effectiveness. In following, the current study provides item level and implementation fidelity data from a randomized controlled study of Safe Touches, a sexual abuse prevention program delivered to low-income, racially diverse public school children.

Program Description

Safe Touches is a classroom-based, culturally sensitive workshop that teaches children about sexual abuse prevention and is designed for grades K – 3. The workshop targets the age range when CSA often begins, with one-third of reported cases occurring before the age of 8, and an additional 19% occurring before age 11 (U.S. Department of Health and Human Services, 2015). More potent detrimental effects of CSA are found to result from abuse occurring during highly sensitive developmental periods, between 3 – 7 years old (Grabell & Knight, 2009), providing further support for prevention programs targeted at young age groups.

Curriculum.

The Safe Touches curriculum was developed by a New York City-based nonprofit organization and has been implemented in local schools and organizations since 2007. The curriculum was iteratively tested and refined by integrating established CSA prevention concepts, age appropriate language, and a culturally inclusive model. As the curriculum is focused on CSA prevention, stranger danger and other forms of child abuse (i.e. physical) are not emphasized. During the 50-minute interactive workshop, culturally appropriate puppets are used to role-play scenarios and help children learn and practice safety concepts. Key concepts covered in the workshop include: defining parts of the body that are considered private, the difference between safe and not safe touches, that not safe touches can be given by someone the child knows, and that the child is not to blame for receiving a not safe touch. Facilitators guide children in making lists of who they can tell if they experience a not safe touch, and practicing the assertive language skills to express discomfort. Workshop language is purposefully sensitive to the fact that some children viewing the workshop may have already been, or might currently be, victims of abuse, as this is a statistical certainty.

Facilitator Training.

Workshop facilitators must complete an eight-hour training, which includes education on CSA, proper handling of disclosures, conducting minimal facts interviews, and procedures for reporting abuse. The training also covers practical workshop skills like puppet handling, audience management techniques, and responding to children’s frequently asked questions. Facilitators must practice the workshop in pairs for at least two weeks, and are assessed using a fidelity checklist before being cleared to facilitate an in-class workshop. This ensures that all workshop components are adequately executed. Planning, recruitment, and implementation for the current work occurred from May 2012 – June 2014. Institutional Review Board (IRB) approval was obtained from the NYC Department of Education and an independent IRB.

Method

A delayed intervention, cluster randomized controlled trial design was used to examine program effectiveness in multiracial, low-income sample of children. On site study activities occurred at three time points: T1 (Baseline), T2 (Post Test), and T3 (Follow Up; see Figure 1). Details of study design and outcomes at T2 have been published elsewhere (Pulido et al., 2015).

Figure 1.

Timeline for Study Activities

Timeline for study activities, using a delayed intervention, cluster randomized design (T2) with short term follow up (T3). The Safe Touches workshop occurred at T2, and Children’s Knowledge of Abuse Questionnaire (CKAQ) administration occurred at all three time points. CKAQ data are reported for Baseline (T1) and Follow Up (T3).

Planning and Implementation

The Data and Safety Monitoring Board assembled for this work established a definition of an adverse event and developed a protocol and form for documentation (see Pulido et al., 2015). An adverse event did not include child disclosure of abuse. Facilitators were trained on the protocol for responding to a child making concerning statements suggestive of abuse, behaviors indicating discomfort with workshop content, or a disclosure during or after the workshop. Upon such an event, the facilitator responds calmly to the child’s statement or behavior and asks to speak with the child afterward. A private location is sought by the facilitator after the workshop, with one school staff member present, to clarify the statement or behavior. If necessary, a minimal facts interview is conducted, and a call is made to the State Central Register or local police if warranted. This procedure was made clear to teachers, guidance counselors, and parents (via consent form) prior to conducting the workshops.

School Outreach

Outreach to schools occurred on a rolling basis from May 2012 - November 2013. Schools were sourced through publicly available websites (e.g. www.insideschools.org) resulting in identification of 101 schools meeting eligibility criteria of: > 75% of students receiving free lunch, < 25% White students, < 1-hour travel time to the school from the non-profit location, and at least two 2^nd or 3^rd grade classrooms available for randomization.

Enrollment

Research staff worked closely with school staff to schedule study activities. Multiple rounds of bi-lingual English/Spanish parent consent forms written at or below an 8^th grade reading level were distributed in classrooms, starting at least one month prior to T1. Across all schools, 890 children were eligible to participate (437 2^nd grade, 453 3^rd grade), 528 (59%) returned signed parental consents, 55% of children completed assessments at T1 and T2, and 411 (46%) completed assessments at T3. School level ethnicity means were: 70% Latino, 23% African American, 3% White, 3% Other (Pulido et al., 2015). Assessment data from the 411 children completing the four week follow up (T3) are included in this paper. The mean age of the T3 sample ranged from 7–10 years (M = 7.9, SD = .71), with the majority (n = 192) of children 8 years old, and a handful (n = 5) 10 years old at the time of testing. No age or grade differences were found between the enrolled and final samples.

Outcome and Fidelity Measures

Children’s Knowledge of Abuse Questionnaire (CKAQ).

The CKAQ was used as the main outcome measure, and is a standardized tool that assesses knowledge of CSA prevention concepts and skills that could prevent abuse (Tutty, 1995, 2003). The CKAQ has 33 items scored “true,” “false,” or “don’t know”, and is comprised of two subscales measuring knowledge of inappropriate touch (ITS) and knowledge of appropriate touch (ATS). The 24-item ITS assesses children’s ability to recognize not safe touches, situations, and people, and acquisition of self-protective skills. It has an established internal consistency alpha of .87, and one month test-retest reliability of .88 (Tutty, 1997). The 9-item ATS assesses knowledge and actions regarding appropriate touch, and was created to gauge potential adverse reactions (e.g. anxiety) in the form of over-generalization of safety concepts to appropriate touches (Tutty, 1995; 1997). In the current study sample, the ITS had adequate internal consistency with a Cronbach’s alpha of .67, while ATS consistency was modest at .54. One month test-retest reliabilities for the ITS and ATS in this sample were r = .65 and r = .55.

CKAQ testing protocol.

Children that had returned parental consent forms were removed from their classrooms at each time point and administered the CKAQ in groups of three to four. Research staff read assent forms aloud to the children immediately prior to testing, to circumvent variability in children’s reading abilities, and for clarity and consistency. Children who did not assent were returned to their classrooms and withdrawn from the study. The CKAQ was then distributed, instructions were read aloud to the children, and study staff explained that answers would not be shared with teachers or parents, or become part of the their grade. Researchers read each item aloud in a neutral tone, ensured that all children had completed an item before moving onto the next one, and stopped after every 3–5 items to make sure all students were on the same question. Intermittently, the groups were reminded that there were no right or wrong answers and that it was okay if a child did not know the answer. CKAQ administration took 15–20 minutes on average for a group of 3–4 children.

Fidelity Measures and Adverse Event Tracking

Workshop Implementation Checklist (WIC).

The WIC measured fidelity of Safe Touches workshop delivery and documented the extent to which all components of the workshop were administered consistently and according to protocol. The WIC was completed by a workshop facilitator or independent observer at T2, following each workshop. Items on the WIC included: distribution of handouts, adherence to the workshop script, and use of puppets/props. Qualitative items asked: reasons for workshop protocol deviation, children’s reactions during the workshop, and the content and nature of any concerning statements or behaviors.

Data Collection Checklist (DCC).

Child assent and CKAQ administration fidelity was tracked using the DCC and was completed after each CKAQ administration session. DCC items included: the number of research staff present, number of children in testing groups, assenting, and adherence to the CKAQ instruction script. Qualitative items asked about: deviation from the data collection protocol, and children’s reactions to the testing or specific CKAQ items.

Teacher Follow-Up Checklist (TFC).

The TFC was created to track child disclosures, concerning statements, and adverse events occurring subsequent to the day of the Safe Touches workshop. The TFC was administered in person at T3 to teachers of participating classrooms. Teachers were asked to comment on observed changes in children’s academic performance, peer interactions, pattern of attendance, and overall behavior. If any changes were noted, teachers were asked to explain the behavior in detail, and whether they felt that the behavior change was related to the workshop or CKAQ testing.

Analysis

Data are included from children that completed the CKAQ at both T1 and T3 (N = 411). Item-level data were analyzed by conducting McNemar’s tests on each question to determine change in the proportion of correct responses (marginal frequencies). This is a distribution free, nonparametric method, which is recommended when working with matched pairs of subjects to examine dichotomous trait data and assess change in individual test items. It is useful for formative analyses, in contrast to t-tests and ANOVA’s, which are better suited for summative, normally distributed data (Adedokun & Burgess, 2012). McNemar’s analysis is particularly applicable to pre-post study designs, when a participant serves as their own control and a nominal response scale is used (Siegal, 1956). Analyses were conducted on the overall sample (N = 411), as well as separately for 2^nd and 3^rd grade subgroups.

Concept Categorization.

The 24 item ITS subscale contains four ‘stranger danger’ questions, and two questions related to assertiveness with peers, which are not concepts directly covered in the Safe Touches workshop, and were created to ‘establish a comfortable response’ to the questionnaire (Tutty, 1992). Thus, the remaining18 workshop-related ITS questions were divided into sub-categories to gain a better understanding of which concepts children directly learned and retained, and those with which children struggled. Questions were grouped into four categories based on conceptual content (see Table 1) which mirrors the design logic of the CKAQ, and reflects topics covered by most CSA prevention programs (Tutty, 1995; 2000). The ATS subscale has nine questions, three of which are ‘stranger danger’ questions. The ATS was not categorically subdivided for this analysis.

Table 1.

Children’s Knowledge of Abuse Questionnaire Items and Concept Categories

Inappropriate Touch Scale (ITS) Items	T1	T3	Δ	T1	T3	Δ	T1	T3	Δ
Category 1: Personal Agency re: grown ups	N = 411			Gr. 2 (n=197)			Gr. 3 (n=214)
5. Sometimes it’s OK to say “no” to a grown-up.	50	66	16***	48	63	15***	51	69	17***
15. If a grown-up tells you to do something you always have to do it.	20	42	21***	16	31	16***	24	51	27***
22. You have to let grown-ups touch you whether you like it or not.	49	65	16***	37	53	16***	60	76	15***
31. If your baby-sitter tells you to take off all your clothes but it’s not time to get undressed for bed, you have to do it.	65	77	12***	57	68	11**	72	85	12***
Category 2: Personal Agency re: trusting feelings
1. You always have to keep secrets.	37	57	20***	29	49	20***	44	64	20***
12. If someone touches you in a way you don’t like, it’s your own fault.	70	75	6*	59	66	7	80	84	4
16. Some touches start out feeling good then turn confusing.	42	63	21***	37	59	22***	48	67	20***
17. You can trust your feelings about whether a touch is good or bad.	53	63	10**	50	57	7	56	68	12**
25. Boys don’t have to worry about someone touching their private parts.	59	63	4	51	56	5	67	70	3
Category 3: What to do if a not safe touch happens
6. It’s OK to say “no” and move away if someone touches you in a way you don’t like.	74	79	5*	63	77	14***	84	81	−2
10. If someone touches you in a way you don’t like, you should tell someone you trust.	83	87	4(.07)	80	85	5	86	90	4
13. If you don’t like how someone is touching you, it’s OK to say “no”.	76	83	7**	72	77	5	79	88	8*
23. If someone touches you in a way that does not feel good you should keep on telling until someone believes you.	76	83	8**	69	75	6	82	91	9**
32. If someone walks in while you are having a bath, and you feel uncomfortable, you should just keep quiet.	49	44	−4	30	29	−2	49	44	−4
Category 4: Unsafe touch from a familiar person
8. Even hugs and tickles can turn into bad touches if they go on too long.	55	56	1	49	53	4	61	59	−1
20. Even someone you like could touch you in a way that feels bad.	37	42	5(.09)	37	40	4	38	44	7
24. Sometimes someone in your family might touch you in a way you don’t like.	56	60	4	53	61	8	59	59	0
30. Someone you know, even a relative, might want to touch your private parts in a way that feels confusing.	25	46	21***	28	45	17***	22	46	24***
Appropriate Touch Scale (ATS) Items
2. It’s OK for someone you like to hug you.	76	76	−1	77	78	1	76	74	−2
4. Most kids like to get a kiss from their parents before they go to bed at night, so, for them, that would be a safe touch.	89	93	4*	90	93	3	88	92	4
9. If you fell off your bike and hurt your private parts, it would be OK for a doctor or nurse to look under your clothes.	58	76	18***	63	82	19***	54	71	17***
18. It's OK to have a hug from a grown-up you like.	83	80	−2	83	81	−2	83	79	−3
21. A pat on the back from a teacher you like after you've done a good job at school is a safe touch.	82	92	10***	83	90	8*	81	93	13***
28. If you won a contest for drawing the best picture in your school and a neighbour you liked gave you a quick hug to congratulate you, that would be a safe touch.	72	82	11***	77	82	6	67	83	16***

Items and concept categories for Safe Touches workshop-related CKAQ items. McNemar’s tests were conducted to compare change in proportion of correct responses from T1 to T3. Results are presented for each item for the total sample (N = 411), and by grade.

^*p < .05,

^**p < .01,

^***p < .001

Fidelity Measures.

Quantitative and qualitative data from the three fidelity forms were examined for the total sample, from all three time points. A total of 38 WIC forms, 144 DCC forms, and 40 TFC forms were collected and reviewed, resulting in approximately six single-spaced pages of comments. Qualitative review was completed (Miles and Huberman, 1994) using a comprehensive code sheet constructed using open, axial, and selective coding.

Results

Children’s Knowledge of Abuse Questionnaire

Mean scores increased significantly from T1 to T3 on the full 33 item CKAQ, t (1, 410) = 1.97, p < .001, as well as on the individual ITS and ATS subscales, indicating an overall increase in knowledge that was retained at least one-month post-workshop. There was no effect of sex on scores at any time point. Regarding ITS item level data, the proportion of children endorsing correct responses significantly increased on 12 of 18 ITS questions from T1 to T3 (See Table 1). At the concept level, children significantly increased the proportion of correct responses on eight of nine items across “personal agency” Categories 1 and 2, providing considerable support that concepts of assertiveness with grownups and trusting feelings were enhanced as a result of participating in the workshop. Children voiced confusion at the negative wording of one question in Category 2, which is the only item that did not significantly increase.

In the action-oriented “what to do” Category 3, children demonstrated significant gains on 3 of 5 questions, with one additional approaching significance (Q10; p = .07), suggesting moderate enhancement of concept knowledge in this domain. However, baseline knowledge was higher in this category overall, compared to categories 1 and 2. Finally, children appeared to show less mastery of Category 4, “unsafe touch from familiar person”, where the proportion of correct responses increased significantly on only one of four questions. This is an area of concern, as sexual abuse is most often perpetuated by someone known to the child. Because this sample of children showed low baseline knowledge on all of these items, a ceiling effect would not account for this finding. Finally, there was no significant change on three of the four stranger danger questions (see supplemental materials). On the ATS subscale, a significant proportional increase in correct responses was found on 4 of 6 questions. Two questions that did not show statistically significant improvement had a high proportion of correct responses at T1 (Q2: 77% and Q18: 83%).

When the data were parsed by grade, third graders demonstrated a higher mean baseline score on the full 33 item CKAQ, t (1,197) = 1.97, p < .01, as well as on the 24 item ITS subscale (See Table 2). On the 18 ITS questions, there were three items that showed a significant increase at T3 in the combined sample analysis that remained significant for 3^rd, but not 2^nd graders when separated by grade (Q17, Q13, Q23). The opposite was true for Q6. Finally, 2^nd graders demonstrated ‘poor’ understanding (< 50% correct) on half of the 18 ITS items at baseline, while 3^rd graders showed ‘poor’ understanding on only six items. Second graders demonstrated ‘good’ understanding (> 75% correct) on only one item at baseline, while 3^rd graders showed ‘good’ understanding on five items.

Table 2.

CKAQ Inappropriate Touch Scale Data Compilation

			% Correct at Baseline
			ITS	ITS Category from Table 1
	Age	Grade		Cat. 1	Cat.2	Cat. 3	Cat. 4
Current study	7 – 9	2nd	48	40	45	63	42
	8 – 10	3rd	60	52	59	76	45
Tutty 2000	5 – 7	n.a.	63	52	71	65	65
Tutty 1997	n.a.	2nd	73	n.a.	n.a.	n.a.	n.a.
		3rd	76
Tutty 1995	6 – 7	1st	53	n.a.	n.a.	n.a.	n.a.
	8 – 9	2nd	77
Daignault et al., 2012	5 – 11	1st – 4th	50	n.a.	n.a.	n.a.	n.a.

Summary of CKAQ Inappropriate Touch Scale (ITS) data. The first row shows data from the current study. The following rows were compiled from data available in prior published work. Studies were included if they provided item level data from the CKAQ, or ITS means, and the study sample grade or age. ITS category means for Tutty 2000 were created based on Table 1 category items. Only ITS means were available in the other studies. Data from Daignault et al. used a French translation of the CKAQ.

Workshop Fidelity

A total of 38 workshops were conducted across six schools (4–12 per school). Quantitative data from the WIC forms show that all workshops were delivered with high fidelity to the program model, with little variability across schools. Specifically, for items including: using all props according to script, facilitators following the script, and activity books given to teachers to distribute, 100% fidelity was obtained. In only two cases, teachers did not receive the pre-workshop informational handout. Qualitative data from the WIC forms yielded three themes:

Concerning statements.

Concerning statements were made by six children during the workshop or during the question and answer period. Statements included: “What if you are in a car and a person tries to touch you?”, “If someone says they won’t touch you and they do it later, is that a safe touch?”, “Would you need a video or photo to prove that [a not-safe touch] happened?”. Additionally, three children disclosed feeling uncomfortable with specific individuals (i.e., doctor, sister’s boyfriend). In all instances, according to the developed disclosure protocol, these children were calmly asked to stay after the workshop to discuss their questions further, and a minimal facts interview was conducted by the facilitator in the presence of a school staff member. No calls to the SCR or police were deemed to be necessary. School personnel agreed to follow up with parents as they felt was appropriate. Thus, for the full sample of 492 children, there was less than 2% concerning statement rate, and 0% disclosure rate.

Child engagement.

Facilitators observed high levels of engagement, interest in the interactive segments and the question and answer format, and to the puppets. Children appeared to grasp the concepts being taught, and verbalize understanding the difference between safe and not safe touches. Facilitators noted that some concepts took longer to grasp than others. For example, “there was some difficulty understanding that a doctor touching private parts would be a safe touch, but after facilitators reviewed this, children were able to verbalize understanding”.

Teacher involvement.

Teacher presence at the workshops was required by the research team and the DOE. Facilitator observations indicated that teacher involvement greatly impacted student attention during the workshop, increased student participation and understanding of content, and decreased disruptive behavior. When teachers were less supportive, facilitators noted that it was harder to maintain students’ attention. In these cases, facilitators implemented student engagement strategies outlined in the Safe Touches Facilitator Training Manual, such as having children say “1–2-3 Action!”, before the start of each skit.

Data Collection Fidelity

A total of 144 CKAQ administration sessions and DCCs were completed across 59 testing days (including T1, T2, and T3). Again, high fidelity to protocol was documented. In 97% of the sessions, there were 4 or less children per testing group, and on only one occasion a child chose not to assent. Qualitative review of the DCC revealed the following themes:

Test administration.

Notable test administration events included children having difficulty circling the answer on the correct line of the CKAQ, saying answers out loud, and two children appeared to circle “don’t know” or “true” for the majority of items. Research staff responded by regularly checking that children were completing the CKAQ correctly, reminding children to keep their answers private, and reviewing and reinforcing the meaning of the response options. Several children seemed to struggle with reverse-worded items on the CKAQ: “Children [knew] the answer but got stuck on what true and false means for reverse questions, i.e. boys don’t have to worry about someone touching their private parts. The child said ‘that’s not true, boys do have to worry’, and circled [the wrong answer] even though they knew the right answer.”

Concerning Statements.

Thirteen comments were made during CKAQ testing that required follow up conversations, although none of the instances required an SCR report. Four comments that required a minimal facts interview related to children indicating that someone touched their private parts, one saying “my dad does that”, another saying his mom does that, one related to ‘tickles’, and one to a cognitively impaired cousin touching the child’s private parts. In the first three instances, the situations were not indicative of abuse. In the last instance, the child had told his mother already, and she no longer allowed them to play alone together. The guidance counselor was present for the interviews, or was notified if follow up was required.

Cheating.

Despite instructions given, cheating and copying answers was prevalent in early testing groups. “[The children] would either wait to answer until the others did and then copy the answer, or see that someone else had answered one way and then change their answer.” After noticing this problem, the research protocol was changed to include using “mini cubicles” between each child with binders or large books. Following the first testing cycle [T1], cardboard partitions were purchased for subsequent testing sessions.

Testing Space.

Issues regarding testing space included uncomfortable conditions and distractions, such as cramped seating and noisy testing areas due to proximity to a gym. Temperature control was an issue, and distractions occurred when testing rooms were used for other purposes by school employees (guidance office, employee lunch room).

Scheduling.

CKAQ testing scheduling challenges included conflict with other school activities, needing to change locations in the middle of testing due to room availability changing, and children being removed from class during the workshop.

Teacher Follow-Up Interview

Brief follow-up interviews were conducted with teachers in participating classrooms at T3 using the TFC, four weeks after workshop delivery (N = 38). Overall, no negative study impacts were reported. One teacher reported a change in peer interactions, and another reported a change in attendance patterns, neither of which were said to relate to the workshop or CKAQ questions. No additional concerning statements or disclosures were reported.

Discussion and Limitations

The current work provides item level and implementation fidelity data from a study assessing the effectiveness of Safe Touches, a school-based CSA prevention program, delivered to a low-income, racially diverse sample of children whom are historically underrepresented in CSA prevention research (Baker, Gleason, Naai, Mitchell, & Trecker, 2012; Finkelhor, 2009; Kenny, 2010). The results provide insight into specific knowledge this population of children had prior to receiving the workshop, and what they learned, retained – and struggled with – four weeks after participating.

Three findings from this work are essential to characterizing the relative effectiveness of Safe Touches, as well as informing other existing and emerging CSA prevention programs. First, children in this study demonstrated and retained CSA prevention knowledge four weeks after participating in the workshop. This increase in mean CKAQ scores was driven by large increases in the proportion of correct responses on certain items from T1 to T3. Concept-level findings revealed that the workshop is effective in increasing knowledge regarding “personal agency”, specifically when it is appropriate to demonstrate assertiveness toward adults, and in trusting one’s feelings. These two important safety concepts had mean category increases of 16.3% and 12.2%, respectively. These categories also contained items with the largest proportional increases, including: “If a grown-up tells you to do something you always have to do it”: 21%, and “Some touches start out feeling good then turn confusing”: 21%, which are notably sophisticated distinctions for young children to make. In contrast, there appeared to be a lack of conceptual learning related to receiving not safe touches from familiar people on all except one item (Q30), which showed large gains. It is unclear why children improved on this one question by such a large margin. This item contained the term “private parts”, and the baseline correct response mean was extremely low compared to the other category items. It is possible that children did not understand the term “private parts” prior to seeing the workshop, and the increase reflects learning the term, not the concept. We continue to emphasize this important message in our ongoing workshops, and reiterate the need within the CSA prevention field for more detailed analyses to accurately and substantively report program impact.

Second, in the present study, developmental differences in learning were evident. Children in 3^rd grade had a significantly higher proportion of correct responses and ‘good’ understanding of more items at baseline, and showed significant improvement on more program-related ITS questions than 2^nd grade children. Similar developmental differences in learning have been found in the few CSA program studies that have reported results by age or developmental stage (Saslawsky, 1986; Tutty, 2000, 1997). Furthermore, a prior item analysis found that older children increased correct responding on certain items while younger children decreased or stayed the same, resulting in non-significant effects when combined (Tutty, 1994). Conversely, significant increases reported summatively may be carried by one group of children, masking a lack of acquisition by another group of children, and leading to imprecise interpretation of program impact. In the current study this pattern was evident on 3 items, where 3^rd but not 2^nd graders showed significant improvement, and significance was retained when combined. These findings may reflect more advanced cognitive and moral development in older children (Tutty, 1994) resulting in better learning acquisition or test performance. While these differences in learning are not necessarily surprising, it cannot be concluded that they are due solely to age, and not mediated by other factors like SES, ethnicity, or implementation fidelity, which are rarely reported in CSA prevention program studies. In the current work, it is unlikely that differences found between grades are attributable to program content or delivery, as implementation and testing fidelity was closely monitored and varied little.

Third, in comparison to published CKAQ outcome data, children in the current study show considerably poorer baseline knowledge relative to same-aged peers. Table 2 shows a compilation of CKAQ data when results were available by age or grade. It is clear that mean ITS scores in studies of middle class, mainly Caucasian samples (Tutty, 1995, 2000, 1997) are substantially higher than in the current sample of children, whom are predominantly racial minority and low SES. Compared to the 1997 dataset of 2^nd and 3^rd graders, children in the current study scored more than 1 standard deviation and 20% lower on average. Remarkably, the current sample of 2^nd and 3^rd graders also scored below children from Tutty’s 2000 dataset that, by age, would be in kindergarten and 1^st grade (ages 5 – 7), which is troubling.

Cultural factors may impact how children learn prevention concepts (Baker et al., 2012; Topping & Barron, 2009). These concepts may oppose norms in certain ethnic groups more strongly than others and contradict messaging that the child is exposed to at home, such as placing great value on respect for elders (Baker et al., 2012), spirituality, and masculinity/machismo (Welland & Ribner, 2010). These norms could preclude or discourage children from accepting certain workshop concepts or using certain safety skills. In addition, although evidence linking risk for sexual abuse and race/ethnicity is far from conclusive (Douglas & Finkelhor, 2005), some studies have reported an association between certain racial subgroups and risk for CSA (Hussey, Chang, & Kotch, 2006; Sedlak et al., 2010). This collectively points toward a need to better understand how children of all races perceive and interpret CSA prevention messaging.

In contrast, there is a strong and reliable association found between poverty and poor academic performance (Sirin, 2005), as well as between poverty and sexual abuse risk (Hussey et al., 2006; Sedlak et al., 2010). There is also evidence, although sparse, that low-SES children show fewer knowledge gains after participating in CSA prevention programs compared to higher SES counterparts (Briggs & Hawkins, 1996; Daigneault et al., 2012), but again, it is not certain how race may have modulated these findings. If SES is impacting CSA knowledge acquisition in the current sample, or shows the same trend as other studies, it would result in reduced prevention program efficacy for these at-risk children. Taken together, it is clear that more detailed exploration and reporting of CSA prevention program outcomes is required to tease apart the impact of demographic factors on program efficacy, children’s acquisition of CSA knowledge, prevention behaviors, and sexual abuse disclosure.

Finally, a complementary goal of the current work was to track and report fidelity to workshop delivery and data collection. This assists in interpreting the present results and addresses some factors that could potentially confound study outcomes. Overall, the results show rigorous adherence to program execution and the established data protocols, supporting the expectation that all children were receiving the same content, and that variability in outcomes due to implementation deviation was negligible. The fidelity tracking tools also allowed for documentation of challenges specific to CSA program implementation, and issues related to administering a sexual abuse prevention assessment to young children in busy public school settings. During CKAQ testing, there were issues with the physical testing environment, as well as those due to children copying each other’s answers, which should be preemptively addressed in future work to enhance assessment integrity. Qualitative data revealed that children expressed confusion on some items, such as “If someone walks in while you are having a bath…you should just keep quiet”, and “Boys don’t have to worry about someone touching their private parts”. The latter is a key concept repeated throughout the workshop, suggesting that this confusion may underlie lower correct baseline or follow up responding. Again, prior work has shown that young children have higher baseline response rates and show improvement at post-test on this question, which may again underscore moderating demographic factors in the current sample.

Establishing CSA program fidelity and consistency across sites and studies is critical to interpreting program outcomes (Bickman et al., 2009), and is strongly suggested within the prevention program literature (Jaycox et al., 2006). To minimize program disruptions and enhance fidelity, qualitative results confirm that teacher engagement, having numerous scheduling checks, and pre-implementation discussions with school points of contact to review protocols in the event of a disclosure, are essential. As anticipated, we found that recruiting and engaging schools was labor intensive, there was a low response rate, and school staff turnover reduced enrollment interest, which is consistent with other school-based studies (Jaycox et al., 2006). There were refusals due to concern that CKAQ preparation and administration time – not the in-class workshop – would be too great. This suggests that even resource limited and overburdened schools could be amenable to a CSA prevention program like Safe Touches that is administered in one session for less than an hour. Acceptance is enhanced by fostering good relationships with target schools, being flexible, and offering a minimally intrusive program (Lytle et al., 1994). We did not find that the highly sensitive nature of workshop content was a factor for declining, although we did not systematically solicit reasons for non-participation.

A limitation of the current study is that SES and ethnicity data were captured at the school level, so direct comparisons between knowledge gain and these variables cannot be made. However, a strength of the study design is the large sample size and strategic school selection process. Overall, 96% of students in the included school districts participated in income-based free lunch programs, and 97% identified as racial/ethnic minority. With respect to the between study CKAQ comparisons, it is possible that higher scores in other samples were not due to race or SES, but rather to program dose or execution. While Tutty’s work did have longer workshop duration (90–120 minutes over two days) than the current study, the Daigneault study also found relatively low ITS scores with workshops lasting at least 40 minutes longer than Safe Touches.

Conclusion

The current study shows that young children increase knowledge of certain CSA prevention concepts after a single 50-minute workshop, which is sustained at short term follow up. Findings suggest that a more granular review of outcomes is critical to consider when creating, evaluating, and reporting the effectiveness of CSA prevention programs. This will help clarify factors that promote or inhibit children’s understanding of sexual abuse prevention concepts, and impact a child’s ability or motivation for disclosing abuse, reacting to potentially abusive situations, and taking steps to keep themselves safe from preventable harm (Topping & Barron, 2009). This, together with increased attention to program fidelity, will strengthen the knowledge base for continued replication and expansion of CSA programs for all children.

Biography

Mary L. Pulido, PhD, is the Executive Director of The New York Society for the Prevention of Cruelty to Children. She has published in the areas of child abuse and neglect, child sexual abuse prevention, preventing secondary traumatic stress, and supervised visitation. Dr. Pulido is President of the American Professional Society on the Abuse of Children – New York, and adjunct Assistant Professor at the Silberman School of Social Work at Hunter College.

Jacqueline L. Holloway, PhD, is the Director of Research and Evaluation at The New York Society for the Prevention of Cruelty to Children, where she manages research initiatives across the agency’s supervised visitation, trauma recovery, and child sexual abuse prevention programs. Her research background is in the psychophysiological bases of learning and behavior, and the effects of stress and trauma on pathology and resilience.