Evaluation of an Assessment Instrument for a Sexual Health Curriculum for Nurses and Midwifery Students in Tanzania: The Sexual Health Education Professionals Scale (SHEPS).

Abstract

We assessed the structure and content of a new scale, the SHEPS, to assess change in sexual health confidence, knowledge and attitudes in nursing students following an intervention (a 2-day standardized workshop on sexual health). Students were 78 Tanzanian nursing students attending a University of Health Sciences, who were assessed immediately before and after the workshop on matched pre- and post-workshop questionnaires. Data confirmed significant changed pre- and post-test on knowledge and confidence on topics taught in the workshop, with the effect also extending to some topics not or minimally covered, suggesting that there was a general increase in confidence and a perception of increased knowledge following the workshop. There was power to detect differences even with a moderate sized matched sample. Correlations between knowledge and confidence on the same content items were between 0.52 and 0.63, suggesting that respondents could clearly distinguish between knowledge and confidence. There were no significant differences pre- and post-test on several controversial cultural and religious attitudes including on abortion and non-vaginal penetrative sex. Alpha coefficients were 0.93 for pre-test and 0.90 for post-test. This field test demonstrates the preliminary appropriateness of the SHEPS as a tool for evaluating sexual health interventions in health care workers.

Introduction

Sexual health training is regarded as a core competency for nurse practitioners and clinicians, (Santa Maria et al., 2016) but it is also an essential component of the education and training for health workers, and nurses. Despite often being the first point of patient contact, many health workers and nurses have not had any training in sexual health. Stand-alone 2-day workshops (such as the University of Minnesota-PAHO sexual health programme for health professionals: Coleman & Mazín, 2016) can provide essential training on sexual health interviewing, diagnostic decision-making, and management skills, both for knowledge and comfort. Adequate evaluation is crucial to the development of effective sexual health teaching and training.

A variety of sexual health training programs for nurses have been evaluated internationally. Santa Maria et al. (2016) implemented a parent-based adolescent health intervention for nursing students in Texas and found significantly increased outcome expectancies, reduced barriers to sexual health communication, increased self-efficacy, and experience in applying sexual health counseling. In Pennsylvania, Wang et al. (2015) noted that lack of training and provider discomfort were the major barriers to oncology nurses raising sexual health concerns with their patients. In a pilot study, they implemented a brief training aimed at patients with breast cancer, and reported comfort level with providing, and self-reported frequency in addressing, sexual health questions.

In Hong Kong, Yip et al. (2015) found that while most adolescent health nurses were very aware of the importance of sexual healthcare, they rarely felt knowledgeable or comfortable discussing sexual health issues with their patients. Nearly half (40%) identified inadequate or lack of training as the most significant barrier to providing adolescent sexual health screening and counselling. In Taiwan, Tsai et al. (2014) found that nurses who had training and experience in assessing patient’s sexual health care were more likely to agree that sexual health care was an appropriate nursing role and those with less training and experience had greater learning needs in assessing patient sexual functioning, communication about patient’s intimate relationships, and approaches to sexual health care. Oskay et al. (2014) in Turkey found that the most frequent reasons for ignoring sexual counselling included nurses believing that they had insufficient skills and that patients would become embarrassed. Similarly, Bell and Bray (2014) in the U.K. noted that nurses often struggle to care with patients with sexually transmissible infections and identified attitudes stemming from lack of education which negatively influence patient care. Specifically, they identified changes in the nursing curriculum including small group debates and service user involvement to allow students to express and challenge their beliefs in a safe and supportive environment. In Taiwan, Sung and Lin (2013) also identified self-efficacy as a crucial determinant in addressing nursing student’s needs in providing sexual healthcare. Even in Sweden, regarded as a leader in the field of sexual health education for health professionals, while more than 90% of nurses understood how disease might affect patient’s sexuality, only two thirds felt comfortable discussing sexual issues (Saunamäki, Andersson & Engstrӧm, 2010). Despite the importance of training in sexual health, few good standardized tools for evaluating changes in knowledge, attituses and confidence exist.

Measurement of sexual knowledge and attitudes in health professionals has been sparse. Miller and Lief (1979) developed the Sexual Knowledge and Attitudes Test (SKAT)© in 1967 which was designed as a teaching and research instrument. Response range was multiple choice or true/false for knowledge and 5-point Likert scales for attitudes, and yielded 4 attitude and 1 knowledge scores. Reliability using the KR-21 was 0.87 (Lief, 1998). There is also an adolescent version, the SKAT-A^©, which was used to evaluate the impact of sexual health interventions. At the two Summits on Medical School Education in Sexual Health held at the University of Minnesota, a group was formed to develop a modern instrument, which assesses specific content areas of sexual health in clinical practice, and to measure both communication and knowledge and equivalent attitudes as a tool for evaluating sexual health educational interventions for health professionals in detail for specific content areas.

The aim of the present study was to administer a form of the newly developed Sexual Health Education Professionals Scale (SHEPS) to assess its format, acceptability and psychometric characteristics as a pilot test of its suitability as a measurement for sexual health education interventions. The full SHEPS can be found in the Appendix.

Method

Two 2-day workshops of the University of Minnesota/PAHO Sexual Health workshop (Coleman & Mazín, 2016) were given to senior nursing students at the Muhimbili University of Health Sciences (MUHAS) School of Nursing in Dar es Salaam, Tanzania. Workshop places were advertised (at no cost) on non-teaching days at the end of the term, with priority given to more senior nursing students. The workshops were rapidly subscribed, with a waiting list. Participants were assessed before and after exposure to the intervention, which was the workshop. There was no effort made to specifically target the same content areas in the workshop as would be assessed in the questionnaire. Workshops were limited to a maximum of 40 students each, with preference given to nursing graduates and senior undergraduate students. Demographic characteristics of the attendees appear in Table 1. The workshop was taught by MR, SR, MT, JW and LG, with four MUHAS nursing faculty as trainee teachers. Each workshop was 16 hours over 2 days and taught in an interactive format in a large lecture theater, using PowerPoint presentations and sexual history taking role-plays in dyads. During the workshops didactic lectures, interactive question and answer sessions, panels, dyads for sexual history taking, dyads and small groups for topic discussion, and an exercise where students formed a “linear scale” where those with very positive attitudes went to one end of the class and those with very negative attitudes to the other, with opinions strung out all the way between, to respond to a number of sexually-related attitudes and beliefs (e.g. “Oral sex is an acceptable sexual practice”) were used.

Table 1:

Sample Demographics

Gender	Female	42
	Male	41
Age	Mean	26.3
	Median	23
	Range	19–53
Year of nursing studies		1	5
		2	28
		3	18
		4	20
		Diploma or Masters	10
Nursing specialty intended		Public & Community health		6
		Critical care/Emergency/Trauma		10
		Diploma		2
		Doctorate		2
		General nursing		13
		Named hospital		2
		Maternal & Child health/Pediatrics		4
		Midwifery		21
		Nursing education		1
		Nursing research		1
		Prefer not to answer		10
Sexuality	Heterosexual	26
Asexual	21
	Bisexual	4
	Gay/Lesbian	1
	Unsure	5
	Prefer not to answer/no answer		26

(Not all responses sum to the total n given some missing values)

Content of the workshop appears in the Appendix. The workshops were based on the curriculum developed jointly by the University of Minnesota and the Pan American Health Association (regional office for the Americas of the World Health Organization (WHO)): the full workshop can be accessed and downloaded at z.umn.edu/manual. The language of instruction in high school and at university in Tanzania is English, so participants were fluent in both English and Swahili.

We used the SHEPS, which was developed by a group led by Drs. Carey Bayer and Alan Shindel at the first and second Sexual Health Education for Medical Schools summits held at the University of Minnesota in 2012 and 2014. The full SHEPS has over 100 items, arranged in 3 scales: communications skills; knowledge; and attitudes. All items were scored on a 7-point Likert scale, with the scale for the communications and knowledge items being Very confident, Moderately confident, Slightly confident, Neither confident nor unconfident, Slightly unconfident, Moderately unconfident, Very unconfident, Don’t know, and Prefer not to answer (the latter two scoring zero points). For attitude items, the poles were defined by the anchors “Strongly agree” and “Strongly disagree”. For the communications and knowledge scales, the same content items are repeated, assessing first, confidence in being able to communicate on the content item, and second, confidence in having adequate knowledge on the item. Items appear in Table 2 (communication) and Table 3 (knowledge). Because of the large item burden and limited time available for pre-test and post-test, and because this was a pilot test of the format and characteristics of the instrument, we used half of the items for each scale, in alternating 4-item blocks. The scale was given as a pretest before the first presentation, and at the end of the 2-day workshop just before feedback to faculty. Modal time taken to fill the half-burden SHEPS out was approximately 15 minutes.

Table 2:

Pre- and Post-Test Changes in Confidence in Sexual Topic Communication

Variable	Mean Difference	SD	t, df	p
A. How confident are you in your ability to communicate/assess/discuss sexuality and sexuality-related topics with…
the parents of a fetus or newborn with a disorder of sex development (e.g. ambiguous genitalia)	1.05	2.03	4.56, 76	.00
a pre-pubescent child (i.e. masturbation, genital exploration of self and other children, questions about sex, “birds and the bees”	0.95	1.46	5.66, 75	.00
a pubescent person (i.e. body changes with puberty, becoming sexually active, decision making)	0.35	1.20	2.56, 76	.01
a person with sexual problems/dysfunctions or concerns?	1.56	1.82	7.50, 76	.00
a person with sexual problem(s) related to a medical, pharmacological, or surgical treatment	1.44	1.81	7.00, 75	.00
a person whose gender and/or sex is different from your own?	1.20	1.95	5.32, 74	.00
a person who identifies as asexual	1.86	2.34	6.91, 75	.00
a person who engages in non-normative sexual practices (e.g. sadomasochism, paraphilias, or fetishes)	1.90	2.29	7.10, 75	.00
a person who masturbates	1.56	2.16	6.33, 76	.00
a person who engages in sex with a committed partner (i.e. dyadic relationship)	1.16	1.74	5.83, 76	.00
a person who is coercive or abusive to their sexual partner(s)	1.43	2.22	5.62, 75	.00
a person who is coerced or abused by their sexual partner(s)	1.13	1.86	5.31, 75	.00
a person with questions about safer sex and sexually transmitted infections	0.39	1.38	2.48, 76	.02
a person infected with the human immunodeficiency virus (HIV)	0.91	2.03	3.92, 76	.00
a person with conservative sociocultural beliefs about sexuality	1.33	1.76	6.68, 77	.00
a person with liberal sociocultural beliefs about sexuality	1.46	1.87	6.90, 77	.00
a person with religious/spiritual convictions about sexuality	1.05	1.89	4.88, 77	.00
a person who informs you of a topic that requires mandatory reporting (e.g. STI, threat of harm to others, etc)	0.85	1.60	4.66, 77	.00

Table 3:

Pre- and Post-test Changes in Sexual Knowledge

Variable	Mean Difference	SD	t, df	p
Do you feel confident that you have the knowledge to care for patients when discussing sexuality and sexuality-related topics in…
the parents of a fetus or newborn with a disorder of sex development (e.g. ambiguous genitalia)	1.21	1.71	6.19, 76	.00
a pre-pubescent child i.e. masturbation, genital exploration of self and other children, questions about sex, “birds and the bees“	’ 0.92	2.03	3.98, 76	.00
a pubescent person (i.e. body changes with puberty, becoming sexually active, decision making)	0.52	1.81	2.52, 76	.01
a person with physical disability (e.g. cerebral palsy, spinal cord injury, amputations)	1.23	1.84	5.88, 76	.00
a persons with sexual problems/dysfunctions or concerns?	1.23	1.84	5.81, 74	.00
a person with sexual problem(s) related to a medical, pharmacological, or surgical treatment	1.12	1.75	5.57, 75	.00
a person whose gender and/or sex is different from your own?	1.00	1.76	4.96, 75	.00
a person who identifies as asexual	1.65	1.64	8.73, 74	.00
a person who engages in non-normative sexual practices (e.g. sadomasochism, paraphilias, or fetishes)	1.53	2.13	6.23, 74	.00
a person who masturbates	1.03	2.07	4.26, 73	.00
a person who engages in sex with a person other than a partner in a dyadic relationship WITH the other partners knowledge and consent (e.g. “open relationship”)	0.83	1.87	3.88, 75	.00
a person who is coercive or abusive to their sexual partner(s)	1.13	1.93	5.08, 74	.00
a person who is coerced or abused by their sexual partner(s)	1.13	1.99	4.94, 74	.00
a person with questions about safer sex and sexually transmitted infections	1.03	1.45	6.17, 75	.00
a person with conservative sociocultural beliefs about sexuality	1.31	1.97	6.17, 75	.00
a person with liberal sociocultural beliefs about sexuality	1.19	2.19	4.70, 74	.00
a person with religious/spiritual convictions about sexuality (in this context it refers to persons whose convictions stem from an organized religious group such as Christianity, Islam, Judaism)	1.26	1.78	6.06, 74	.00
a person who informs you of a topic that requires mandatory reporting (e.g. STI, threat of harm to others, etc)	0.95	1.77	4.61, 73	.00

To ensure that all data would be anonymous and unidentifiable, questionnaires were matched pre- and post-test by asking students to write a word or code that they knew they could remember on the pre-test and the post-test questionnaires, e.g. some combination of initials or a significant date or word. On collating questionnaires, 5 could not be matched on this characteristic so they were matched on gender and age, with handwriting style as a check. Because over the two workshops some attendees arrived late and did not fill out the post-test, total n for matched questionnaires was 77. Not all analyses sum to 77 as some items had missing responses. As this was an anonymous evaluation of an educational intervention, the study was exempted from IRB approval at the University of Minnesota and at MUHAS. Participation in both the workshops and filling out the questionnaires was voluntary.

Analysis:

Data were entered in SPSS format and analyzed using version 22. Demographic characteristics were calculated using means ± SDs for interval or ratio data, and percentages and ns for categorical data. For each item, pre- and post-test differences were calculated using paired t-tests, for communication items (Table 2), knowledge items (Table 3), and attitudinal items (Table 4). Finally, because the content areas of items for both communication and knowledge were the same, we calculated Pearson correlations between each matched communication and knowledge content item, first for pre-test data and then for post-test data (Table 5). Because a random half of the SHEPS items were presented because of time limitations, we do not report on factor analysis of the structure. The item:subject ratio was well below that to allow for an exploratory factor analysis and as a consequence, we cannot determine latent dimensions and thus subscale structure without a larger sample.

Table 4:

Pre- and Post-Test Differences in Attitudes Regarding Sexual Health

Variable	Mean Difference	SD	t, df	p
Please state your level of agreement or disagreement with the following statements. Please answer truthfully; there are no “wrong” answers!
Educating teenagers on sex makes them more likely to do it	−1.49	2.45	−5.34, 76	.00
Masturbation is a healthy part of human development	2.35	2.53	8.20, 77	.00
Oral sex is an abnormal sexual practice	−0.45	3.09	−1.28, 77	.20
Anal sex is an acceptable sexual practice	0.51	2.60	1.71, 76	.09
I won’t be able to provide care for patients with sexual problems	−0.56	2.13	−2.34, 77	.02
People who get sexual pleasure from inflicting and/or experiencing (sadomasochism) pain with consenting partners are sick	−0.83	2.89	−2.55, 77	.01
Abortion should be available to women for whatever reason they choose	0.11	2.51	0.37, 75	.72
Sex is not an issue that physicians should deal with in their Practices	−0.19	1.71	−0.99, 77	.33
Abortion is only allowable in special cases (e.g. rape, incest, threat to health of mother)	−0.01	2.56	−0.04, 76	.97
Abortion is murder	−1.87	2.50	−4.52, 76	.00
People who are transgender deserve to receive care to help them conform to their chosen gender	0.55	2.23	2.14, 76	.04
People should be allowed to marry someone of the same sex	1.03	2.34	3.87, 77	.00
All pornography should be banned	−0.92	2.72	−3.00, 77	.00
One can never be too old for sex	0.50	2.60	1.70, 77	.00

Table 5:

Paired Communication and Knowledge Questions at Pre-test and Post-test

	Paired Pre-Test Correlation	Paired Post-test Correlation	P Pre	Post
Paired Pre-Test Correlation the parents of a fetus or newborn with a disorder of sex development (e.g. ambiguous genitalia)	.48	.63	.00	.00
a pre-pubescent child (i.e. masturbation, genital exploration of self and other children, questions about sex, “birds and the bees”	.41	.73	.00	.00
a pubescent person (i.e. body changes with puberty, becoming sexually active, decision making)	.27	.51	.02	.00
a person with physical disability (e.g. cerebral palsy, spinal cord injury, amputations)	.69	.73	.00	.00
a person with sexual problems/dysfunctions or concerns?	.65	.87	.00	.00
a person with sexual problem(s) related to a medical, pharmacological, or surgical treatment	.57	.56	.00	.00
a person whose gender and/or sex is different from your own?	.56	.91	.00	.00
a person who identifies as asexual	.53	.84	.00	.00
a person who engages in non-normative sexual practices (e.g. sadomasochism, paraphilias, or fetishes)	.36	.85	.01	.00
a person who masturbates	.31	.72	.01	.00
a person who engages in sex with a committed partner (i.e. dyadic relationship)	.47	.87	.00	.00
a person who is coercive or abusive to their sexual partner(s)	.65	.37	.00	.00
a person who is coerced or abused by their sexual partner(s)	.52	.76	.00	.00
a person with questions about safer sex and sexually transmitted infections	.45	.38	.02	.00
a person infected with the human immunodeficiency virus (HIV)	.59	.76	.00	.00
a person with conservative sociocultural beliefs about sexuality	.56	.38	.00	.00
a person with liberal sociocultural beliefs about sexuality	.56	.35	.00	.00
a person with religious/spiritual convictions about sexuality	.67	.82	.00	.00
a person who informs you of a topic that requires mandatory reporting (e.g. STI, threat of harm to others, etc)	.60	.76	.00	.00

Results

Data are presented in Tables 1 to 5. The sample was evenly split between male and females, with median age in the early 20s, and almost all years of nursing and nursing specialties (or intentions to specialize) represented. There was a wide spread of years of nursing education, ranging from first year to graduate study. Almost all pre- to post-test communication and knowledge comparisons were significant at p<.01, despite the relatively small n. For attitudes, for a relatively small number of attitudinal items, notably on abortion and oral and anal sex, and transgender issues, there were no or borderline significant changes. Correlations between content area items in communication and knowledge were a mean of 0.52 (range, 0.27–0.69) for pre-test and 0.63 (range, 0.35–0.91) for post-test. Cronbach’s alpha reliability coefficients were 0.93 for pre-test and 0.90 for post-test. Because the scale tested represented approximately half the items, we could not make comparisons between gender or years of education on a total scale score. No question had more than 5 missing responses, and most only one or two.

Discussion

These data are the first on the SHEPS format and structure, and obtained from pre- and post-test data from two formal workshops with Tanzanian senior and graduate nursing students at MUHAS in Tanzania. Data indicate that the SHEPS was able to measure significant improvements in self-assessed ability to communicate, knowledge, and positive changes in most attitudes toward sexual issues. We conclude that, at least for the half of the content issues which were randomly sampled in the SHEPS, it has the sensitivity to measure changes following a standardized sexual health workshop, even with a relatively small sample. A further strength is that the workshop took place outside North America, which suggests that as an instrument designed in North America with the design intent of cross-cultural effectiveness and applied in East Africa, it has a degree of cross-cultural robustness.

The format (using specific content areas to asses change in confidence in communication skills and knowledge, and attitudes associated with areas of controversy in sexual health) appears, from these data, to be appropriate. We speculated that the correlations between paired pre-test communications and knowledge items, and paired post-test communications items, would be high and that students may not be able to differentiate communications and knowledge. However, the correlations were moderate at a mean of 0.52 for pre-test items and 0.63 for post-test items, which suggests that students were well able to distinguish between communications skills and knowledge.

The lack of change in some attitudinal items was interesting. Items with strong, often religiously-based items like abortion, and culturally sensitive issues like non-vaginal penetrative sex, showed no or borderline significant change. We conclude that this is again a confirmation that the SHEPS is picking up change in attitudes that do change while distinguishing those that do not. Tanzania is a country with approximately equal numbers of Christian and Moslem citizens, at least in Dar es Salaam, and both Islam and Christianity have some strong positions on these issues. The government of Tanzania strongly discourages asking research participants their religion, primarily to avoid fostering religious intolerance, but our observations and those of our colleagues would confirm that the workshop was probably evenly divided between Moslem and Christian-identified students. We particularly noted (during an exercise where students formed a “linear scale” where those with very positive attitudes went to one end of the class and those with very negative attitudes to the other, with opinions strung out between), that there were also a number of attitudes where there was polarization between males and females.

There were improvements on content items which were not covered or were minimally covered in the workshops, such as “the parents of a fetus or newborn with a disorder of sex development (e.g. ambiguous genitalia)” and “a person with physical disability (e.g. cerebral palsy, spinal cord injury, amputations).” These data suggest that any improvements in communication and knowledge have a more general effect on communication and knowledge confidence – a rising tide lifting all boats. General improvements in communications skills such as taking a sexual history may have an impact on confidence in sexual knowledge. The increase in reported knowledge may have been recorded as a function of greater optimism in confidence rather than of knowledge.

This study has a number of weaknesses. Only half of the content items of the SHEPS were used because of workshop time constraints, and we therefore cannot carry out psychometrics such as factor analyses to determine if there were any latent dimensions within the content item scales. Such psychometrics must await use of the full scale in much larger samples. While the internal reliability was high, test-retest over time would need a longer timeframe for assessment. Even so, it is possible that people might lose their confidence over time, so the effectiveness of the intervention might not be durable. Nor can we be sure that the content items had the same meaning in Tanzania as they had in North America. The relatively small sample size is a potential issue, although it appeared to give more than adequate power to detect significant differences, and there were few missing responses. There might be a concern that the intervention and the SHEPS were closely aligned from the outset. However, the SHEPS and the Workshop were developed completely independently and by different personnel, so there was no deliberate intention to provide knowledge and influence attitudes in certain content areas, knowing that these would be assessed in the SHEPS. A potential limitation is that the scale was not compared to any other knowledge and attitude scale, so nothing is known about its comparative sensitivity. Finally, for clinicians, the optimum assessment remains competence-based assessment rather than self-report.

Nevertheless, these data from the SHEPS in the first “live” test of some of its structural and sensitivity components, suggests that it is an excellent and sexual health professional content-specific measure of the effectiveness of an intervention in varying cultural settings. Additional psychometrics and assessment are needed on the full scale to clarify its dimensional structure with regard to possible content subscales.