The Development and Validation of Sexual Health Indicators of Canadians Aged 16–24 Years

Abstract

Objectives

We developed and validated a set of self-administered, multi-dimensional indicators of sexual health among Canadians aged 16–24 years.

Methods

This study used a mixed-method qualitative and quantitative approach to develop and validate indicators of sexual health. We used the four-stage Dillman method to identify, focus-test, pilot-test, and validate key metrics to measure sexual health. We collected quantitative data to validate the measures through a computer-assisted self-interviewing program among a purposive sample of 1,158 people aged 16–24 years recruited from four Canadian provinces.

Results

The survey contained 75 items measuring five dimensions of sexual health: (1) physical, mental, emotional, and social well-being in relation to sexuality; (2) approach to sexuality; (3) sexual relationships; (4) sexual experiences; and (5) discrimination, coercion, and violence. Principal components analysis for composite measures found seven components with eigenvalues ≥1. The factor structure was stable across gender, age, size of area of residence, and language in which the survey was completed. Cronbach's alpha coefficients ranged from 0.79 to 0.90. Indicators of condom use at last vaginal sex, protection self-efficacy, sexually transmitted infection/HIV testing self-efficacy, and sexual orientation also showed good construct validity.

Conclusions

The indicators constituted a conceptually grounded survey that is easy for young adults to complete and contains valid, reliable, and psychometrically robust measures. The survey instrument provides a tool for future research to collect population-level data to measure and monitor trends in the sexual health of young people in Canada.

The World Health Organization (WHO) definition of sexual health emphasizes both positive and negative aspects of sexual health. Sexual health includes physical, emotional, mental, and social well-being in relation to sexuality and requires a positive, responsible approach to sexuality and sexual relationships, as well as safe sexual experiences free from coercion, discrimination, and violence.¹ Initiatives to support sexual health are important to improve people's ability to make healthy sexual decisions, to avoid sexually transmitted infections (STIs) or unintended pregnancy, and to access preventive sexual health services. Public health professionals rely on population-level sexual health data to inform these programs and monitor their effectiveness.

Despite their importance for public health, national data on sexual health in Canada are currently limited to the reported number of positive tests of STIs (e.g., chlamydia, gonorrhea, infectious syphilis, and human immunodeficiency virus [HIV]) and rates of pregnancy. Limited data on sexual behavior result from the sporadic inclusion of indicators in national health surveys.² Unlike Australia,³ the United States,^4,5 Ireland,⁶ France,^7,8 and Great Britain,⁹ Canada lacks comprehensive national data from which to develop a profile of the sexual health of Canadians.

The WHO has articulated concepts to be considered in measuring both the positive and negative aspects of sexual health.¹⁰ This study represents the first attempt at operationalizing these concepts and validating a comprehensive set of positive and negative indicators of sexual health in Canada. This article describes the development and validation of a comprehensive set of indicators of sexual health that may be shared with policy makers and public health professionals for use in national, regional, or local surveys to monitor the sexual health of young people in Canada. This set of indicators constitutes a self-administered survey on sexual health that is grounded in previous work by the WHO.¹⁰ We present data on the validity and reliability of these measures demonstrating that this survey provides a robust instrument to measure and monitor sexual health among young people in Canada.

METHODS

We used the Dillman method to develop and validate survey indicators involving (1) focus groups with experts, (2) focus testing of the survey with the target population, (3) pilot testing of the instrument with the target population, and (4) revision of the instrument.¹¹

Survey development: a qualitative approach

We identified constructs representing various dimensions of the WHO definition of sexual health from the literature and presented them to experts in sexual health from across Canada in semistructured focus groups in June 2007. Seventy-nine participants provided feedback on the components, commented on their relevance to sexual health, identified existing indicators to measure the components, and identified missing components to be included in the survey. Focus group data were collected until no new ideas emerged. Focus group discussions were recorded and transcribed verbatim. We used a general inductive approach to analyze the data. Four researchers coded transcripts into categories, which were then conceptualized into broad themes following further discussion and inter-coder verification.

Guided by these data, the researchers developed an 88-item survey measuring five dimensions of sexual health: (1) physical, mental, emotional, and social well-being in relation to sexuality; (2) approach to sexuality; (3) sexual relationships; (4) sexual experiences; and (5) discrimination, coercion, and violence. The survey also measured individual-level determinants of sexual health including mental health (e.g., substance use, depression, and suicide ideation) and chronic conditions that can impact sexual health. Approximately 42 items were adapted from previously validated measures and 46 items were newly developed by the research team.¹²

To assess face validity of the items, the survey was sent for review to four Canadian sexual health experts. Face validity of the survey was also assessed through focus groups conducted with young people in March 2010 in Quebec and British Columbia. Participants were purposively sampled to maximize the diversity of participants along the lines of age, gender, race/ethnicity, sexual orientation, work/school status, and size of area of residence. Demographic information on the 32 focus group participants is provided in Table 1.

Table 1.

Overview of demographic characteristics of participants aged 16–24 years in the pretesting of a survey to measure sexual health and its determinants: British Columbia and Quebec, March 2010 (n=32)

^aPercentages do not total 100% due to rounding.

^bPercentages do not total 100% because of one missing value on this survey item.

Focus groups were conducted until no new ideas emerged. Six focus groups, three in each province, were stratified by gender. Focus groups were conducted at universities, community organizations, and drop-in centers by trained facilitators aged 20–30 years to make participants feel at ease in discussing issues of sexuality.

After giving their consent, participants completed the survey using a computer-assisted self-interviewing (CASI) system created specifically for this project. Participants kept notes as they answered the questions. Immediately following survey completion, participants engaged in a discussion group to comment on the scope, length, and acceptability of the items; confusing wording; and their experience using the CASI system. Discussion groups lasted 60–90 minutes. Participants were compensated with $15; however, compensation was not overtly advertised in participant recruitment. The discussion guide, thematic analysis techniques, and results of these focus groups have been published elsewhere.¹²

Items that were unclear, misunderstood, or rarely answered by participants were removed from the survey or reworded, resulting in a pilot survey comprising 75 items. Table 2 maps the items in the pilot survey onto the elements of the WHO working definition of sexual health.

Table 2.

Number of items in the Canadian Sexual Health Indicators Survey as they relate to the elements of the WHO working definition of sexual health^a

^aThe WHO definition of sexual health emphasizes both positive and negative aspects of sexual health. Sexual health includes physical, emotional, mental, and social well-being in relation to sexuality and requires a positive, responsible approach to sexuality and sexual relationships, as well as safe sexual experiences free from coercion, discrimination, and violence. From: WHO. Defining sexual health: report of a technical consultation on sexual health, 28–31 Jan 2002. Geneva: WHO; 2002.

WHO = World Health Organization

Item validation: a quantitative approach

Sampling.

Pilot-test participants aged 16–24 years were sampled from youth service organizations, universities, colleges, community organizations, drop-in centers, health clinics, and shopping malls in British Columbia, Alberta, Quebec, and Nova Scotia. The sampling frame used to purposively sample participants in this phase of the study was the same as for the focus group phase. Participants were recruited indirectly through the use of posters, mass e-mail distribution, advertisements on organizations' websites, and word of mouth. Research assistants visited the recruitment locations during peak hours of the day to recruit participants in person and to answer potential participants' questions.

Data collection.

Surveys were administered, in both English and French, at drop-in centers, health clinics, universities, and colleges from April to June 2010. Participants signed a consent form explaining the study and ensuring them of the confidentiality of their involvement. Participants were given $15 upon termination of their survey; however, this compensation was not overtly advertised during recruitment posters or other recruitment media. Surveys were self-administered by participants using the CASI system.

Data analysis.

We used SPSS® version 16.0 for all statistical analyses.¹³ We used descriptive statistics to summarize sample characteristics. We used means and standard deviations to summarize continuous data. Categorical data were summarized using frequencies and percentages. These analyses assessed the accuracy of skip patterns, proportions of nonresponse, and potential for nonresponse bias.

Content validity refers to how well the items measure what they intend to measure.¹⁴ We examined the content validity of seven composite measures using principal components analysis (PCA) with varimax rotation. Eigenvalues ≥1 were retained.¹⁵ Items were retained in a dimension with factor loadings of >0.4. We assessed discriminant validity by the extent to which items correlated more highly with the dimensions that they were hypothesized to represent than they did with the other dimensions.¹⁶ The internal consistency of each scale was measured using Cronbach's alpha coefficient.¹⁷ The relative contribution of each item to the scale was accounted for by weighing each item by its loading in creating the scale. To assess the stability of the factor structure, we stratified analyses by age (16–18 years and 19–24 years), gender (male and female), size of area of residence (urban and rural), and language in which the survey was completed (English and French).

We assessed construct validity using bivariate correlation, Chi-square tests, and independent sample t-tests, as they were appropriate to measurement level of the item.

RESULTS

Sample characteristics

A total of 1,185 participants completed the pilot survey. Completion time ranged from 10 to 90 minutes, with a mean completion time of 27 minutes. The majority (77%) of the sample completed the survey in English. Table 3 summarizes the demographic characteristics of the pilot-test sample.

Table 3.

Demographic characteristics of participants in the pilot testing of the Canadian Sexual Health Indicators Survey administered to young people aged 16–24 years in British Columbia, Alberta, Quebec, and Nova Scotia: March–June 2010 (n=1,185)

^aNot all percentages total 100% due to nonresponse and rounding.

^bTwo-spirit is a term used by Aboriginal lesbian, intersexed, gay, bisexual, and transgender people to identify themselves.

^cFirst Nations refers to status and non-status Indian peoples in Canada.

^dMétis refers to one of three recognized Aboriginal peoples in Canada, along with the Indians (or First Nations) and Inuit.

Item analysis

The proportion of missing data for most items was low, with a mean proportion missing of 6.7%. The proportion of missing values on each item was regressed against its item number as an indicator of its placement in the survey to assess whether there was a linear pattern among missing values. The item number was found to be moderately correlated with the proportion of missing values, suggesting that the proportion of missing data increased with progression through the survey (r=0.057, p≤0.001).

Content validity

Factor structure.

PCA resulted in seven components with eigenvalues ≥1. Components were labeled according to the nature of their constituent items as protection self-efficacy (eight items), STI/HIV testing self-efficacy (five items), sexual communication self-efficacy (four items), sexual limit-setting self-efficacy (three items), sexual assertiveness (five items), partner violence victimization (four items), and sexual coercion victimization (four items). Table 4 summarizes the loadings of constituent items on each component. The seven-component structure accounted for 67% of the total variance. Item discriminant validity was confirmed, as the correlation of each item with its associated component was higher than its correlation with the other components.

Table 4.

Assessment of content validity through rotated principal components analyses of seven scale items using data from the pilot test of the Canadian Sexual Health Indicators Survey administered to 16- to 24-year-olds in British Columbia, Alberta, Quebec, and Nova Scotia: March–June 2010 (n=1,185)

^aVariables were coded as 1 = strongly disagree, 2 = disagree, 3 = neither agree nor disagree, 4 = agree, and 5 = strongly agree.

^bVariables were coded as 0 = never, 1 = once, 2 = two or three times, 3 = four or five times, and 4 = more than five times.

^cVariables were coded as 0 = never, 1 = rarely, 2 = sometimes, 3 = almost always, and 4 = always.

STI = sexually transmitted infection

HIV = human immunodeficiency virus

Internal consistency reliability.

The internal consistency of these seven composite measures was good, demonstrating high reliability. Cronbach's alpha coefficients ranged from 0.79 to 0.90.

Stability of the factor structure.

The stability of the seven-component structure was tested across age (16–18 years and 19–24 years), gender (male and female), size of area of residence (urban and rural), and language in which the survey was completed (English and French). There was no effect of age, gender, size of area of residence, or survey language on the factor structure.

Construct validity

Construct validity is the degree to which the measures in the survey are associated with other measures in ways that are predicted by theory or in ways that are consistent with previous research. We assessed the construct validity of condom use at last vaginal sex, protection self-efficacy, STI/HIV testing self-efficacy, sexual attraction, and sexual orientation in this study.

Condom use at last vaginal sex.

Participants were asked whether (coded 1) they had ever experienced any difficulty in using protection (e.g., condoms, dental dams, or other barrier methods) because of any chronic medical condition or disability. A lower proportion of both females (31.8%) and males (38.1%) who reported difficulty using protection also reported condom use at last vaginal intercourse as compared with participants who did not report any difficulty using protection (59.3% of females, 62.7% of males) (p≤0.05). This relationship was stable across age, size of area of residence, and survey language (data not shown).

Protection self-efficacy.

We used independent sample t-tests to assess differences in mean protection self-efficacy scale scores between participants who reported condom use (coded as 1) and non-use (coded as 0) in the previous 12 months. The mean scores on the protection self-efficacy scale were 24.00 for participants who reported using and 22.13 for participants who reported not using condoms in the previous 12 months (p≤0.001). This relationship was stable across gender, age, size of area of residence, and survey language (data not shown).

STI/HIV testing self-efficacy.

We used independent sample t-tests to assess differences in mean STI and HIV testing self-efficacy scale scores between participants who reported getting tested (coded as 1) and those who reported not getting tested (coded as 0) for STIs or HIV in the previous 12 months. The mean scores were 15.27 for participants who reported getting tested and 13.93 for those who reported not getting tested for STIs in the previous 12 months (p≤0.001). The mean scores were 15.32 for participants who reported getting tested and 14.04 for participants who reported not getting tested for HIV in the previous 12 months (p≤0.001). This relationship was stable across gender, age, size of area of residence, and survey language (data not shown).

Sexual orientation.

Participants were asked to describe their sexual orientation using the following response categories: (1) heterosexual (attracted to the opposite sex), (2) “two-spirit” (i.e., a term used by Aboriginal lesbian, intersexed, gay, bisexual, and transgender people to identify themselves), (3) bisexual (attracted to both men and women), (4) gay or lesbian (attracted to the same sex), (5) queer, (6) asexual, (7) other, and (8) don't know yet. Participants were also asked to whom they are attracted using the following response categories: (1) only people of the opposite sex, (2) mostly people of the opposite sex, (3) equally attracted to people of the opposite and same sex, (4) mostly people of the same sex, (5) only people of the same sex, and (6) not attracted to either sex. Chi-square tests demonstrated an association between participants' self-identified sexual orientation and the sex of those to whom they are attracted (p≤0.001). The majority of participants who identified as heterosexual (84%) also reported being attracted to only people of the opposite sex. Among those who self-identified as gay or lesbian, 50% reported being attracted to only people of the same sex and 44% reported being attracted to mostly people of the same sex. The responses among participants who identified as bisexual and two-spirit were distributed across the three middle categories, with the largest proportion (43%) reporting being attracted equally to people of the opposite and same sex. This relationship was stable across gender, age, size of area of residence, and survey language (data not shown).

DISCUSSION

The purpose of this study was to develop and validate a comprehensive set of indicators of sexual health for young people aged 16–24 years, conceptually grounded in previous work by the WHO in defining and measuring sexual health.¹⁰ The indicators developed measure both positive and negative aspects of sexual health. The findings of both the qualitative and quantitative phases of this study suggest that the indicators are valid and reliable measures for young people aged 16–24 years.

The development of the survey based on the Dillman method strengthened the face validity of the measures.¹¹ Analysis of quantitative data from a pilot sample of young people confirmed that all measures were acceptable and valid, and that composite measures met standards regarding psychometric properties of scales with good convergent and discriminant validity, internal consistency reliability, and construct validity.

The measures in the survey were acceptable, as evidenced by the low average rate of missing data (<7%). The high response rate suggests that participants were willing to answer sensitive questions and were able to select from one of the response choices presented. Significantly higher proportions of missing data at the end of the survey suggest that either the survey was too long or participants were less willing to answer the most sensitive questions, which were placed at the end of the survey. The mean completion time of 27 minutes in this study was less than the completion time of other similar studies that have produced valid data, which suggests that the survey length was acceptable.^18,19 Therefore, further qualitative research with young people on the best placement of the most sensitive questions is warranted.

PCA revealed seven components of sexual health representing self-efficacy in protection use, testing for STIs and HIV, sexual communication, and sexual limit-setting; sexual assertiveness; partner violence victimization; and sexual coercion victimization with high internal consistency reliability (Cronbach's alpha >0.7). These findings support the multidimensional nature of sexual health articulated in the WHO definition and suggest that the items measured each of these domains reliably. Further studies using confirmatory factor analysis could be considered to verify the seven components found in this study.

Several relationships explored in the data support the construct validity of the measures. Participants who reported difficulty in using protection, such as condoms, due to a medical condition or disability reported using condoms at last vaginal intercourse in significantly lower proportions than participants who did not report difficulty in using protection. This result is consistent with studies that found that young people with chronic illnesses or disabilities were less likely to report condom use.^20,21

Participants who reported using condoms in the previous 12 months had significantly higher protection self-efficacy scores than participants who reported not using condoms during this period. Further, participants who reported getting tested for STIs or HIV in the previous 12 months had significantly higher STI/HIV testing self-efficacy scale scores than participants who reported not getting tested during this period. Previous research has shown that high sexual self-efficacy is correlated with a lower likelihood of engaging in risky sexual behavior, including unprotected sex, and a higher likelihood of engaging in healthy sexual behavior.^22,23 Young people who believe they can take action become more inclined to do so and remain more committed to this decision.^24,25

Finally, participants' self-identified sexual orientation was significantly correlated with the sex of those to whom they are attracted in the expected manner. The majority of participants who identified their sexual orientation as heterosexual also indicated that they were attracted only to people of the opposite sex. Likewise, the majority of participants who identified their sexual orientation as gay or lesbian also indicated that they were only or mostly attracted to people of the same sex. Existing theory and research on sexual orientation suggest that sexual orientation is a complex identity comprising many dimensions, including attraction, self-identification, behavior, emotional preference, and social preference.²⁶ The congruence between the two dimensions in this study supports the need for multidimensional measures of sexual orientation.

Limitations

This study was subject to the following limitations. First, the sample was a purposive sample across four provinces in Canada. While the heterogeneity of the sample was of interest for determining the validity and reliability of the measures among a diverse population, the inferential statistical findings are not meant to be generalized to the entire population. Second, this study was unable to test the stability of responses over time, as repeated surveys were not conducted. The consistency of responses over time may be established with the use of the survey in future research, most notably longitudinal studies with cohorts of young people. Finally, this study was only able to demonstrate the validity and reliability of these indicators for one age group in the population. Further study is needed to determine the quality of these measures for measuring the sexual health of populations across the life span.

CONCLUSIONS

The items contained in this survey are valid and reliable measures of the sexual health of young people in Canada. These measures address an existing gap in measuring both positive and negative elements of sexual health and further expand the scope of measures currently used. This study provides robust pre- and post-test survey measures for future research to evaluate the impact of programs on the sexual health of participants. More importantly, this study provides a tool for health authorities and academics to monitor the sexual health of young people in Canada and to develop programs, policies, and strategies to improve the health of the population. To this end, these measures will be used by the Public Health Agency of Canada in a study to collect national-level data in Canada on the sexual health of young people aged 18–20 years.

Table 5.

Assessment of construct validity through independent sample t-tests for differences between mean scores on protection self-efficacy and STI/HIV testing self-efficacy among 16- to 24-year-olds in the pilot test of the Canadian Sexual Health Indicators Survey in British Columbia, Alberta, Quebec, and Nova Scotia: March–June 2010 (n=1,185)

STI = sexually transmitted infection

HIV = human immunodeficiency virus