TEST–RETEST RELIABILITY FOR SELF-REPORTS OF SEXUAL BEHAVIOR AMONG THAI AND KOREAN RESPONDENTS

Abstract

The purpose of this study was to examine the test-retest reliability for reports of sexual behavior from the National Institutes of Mental Health Multisite HIV Prevention Trial survey for two Asian samples. Thai (N = 37) and Korean (N = 46) respondents aged 19–37 years (M= 29, SD = 4.61) completed face-to-face interviews to assess their sexual risk in their ethnic language 3–7 days apart. Test-retest coefficients ranged from .65 to 1.00 demonstrating acceptable reliability of the survey among Thai and Korean adults. The discussion focuses on the development of HIV risk assessments that take into consideration ethnic diversity found within the Asian and Pacific Islander community.

Asian and Pacific Islanders (A/PIs) residing in the United States are an understudied minority in HIV research (Chin, 1999). A/PIs make up 3.3% of the U.S. population and .7% of AIDS cases (Centers for Disease Control and Prevention [CDC], 1998). Low rates of HIV/AIDS among A/PIs might be used to infer that they are not at risk for contracting HIV. Previous research, however, has documented that A/PIs are at risk for contracting HIV through sexual behavior and incidence of sexually transmitted diseases (STDs) (Baldwin, Whiteley, & Baldwin, 1992; Cochran, Mays, & Leung, 1991; Gellert et al., 1995). An analogous situation existed for African Americans and Latinos in the mid-1980s. Both groups had a relatively low incidence of HIV. The incidence of STDs in the same time period (CDC, 1988), however, indicated that individuals within both African American and Latino communities were engaging in behavioral practices that put them at risk for HIV. By the late 1990s, it became evident that new AIDS cases among both communities were dramatically increasing (CDC, 1999). In 1999, for example, African Americans and Latinos made up 34% and 45% of all new AIDS cases in Los Angeles County, California, respectively (HIV Epidemiology Program, Los Angeles County Department of Health Services, 2000). The presence of risk behavior coupled with an increase in HIV prevalence in the A/PI community could lead to a significant increase in the number of HIV cases among A/PIs in the United States similar to increases found in African American and Latino communities.

There is a dearth of research on the development of culturally appropriate methods and measures to assess HIV risk behavior taking into consideration ethnic diversity among A/PI populations in the United States. Reliable and valid HIV risk assessments need to be developed prior to mounting studies to examine HIV risk among the various ethnic groups represented in the A/PI community.

In particular, the Thai American population may be at risk for HIV infection owing to the combined factors of the HIV epidemic in Thailand, risky sexual behavior, and heavy travel between the United States and Thailand among both American and Thai residents (World Health Organization, 1998). For example, public health officials in northern Thailand have recognized the commercial sex industry as a significant source of HIV transmission (Mastro & Limpakarnjanarat, 1995). In these regions, 65% of female commercial sex workers were estimated to be HIV-positive (Celentano et al., 1996). As sex-oriented tourism industry remains alarmingly high (Morris, Pramualratana, Podhisita, & Wawer, 1995), the potential for international transmission is high as well, putting Thai Americans at particular risk.

Less is known about the risk for HIV infection among Korean Americans. However, cultural factors among that population as well as other A/PI groups render them vulnerable for HIV infection. In particular, the enduring stigma of HIV among A/PI has been documented (Chin & Kroesen, 1999; Gock, 1994). In addition, cultural issues such as the reticence of A/PIs to discuss sex and sexuality openly also increase HIV risk (Chin, 1999). Thus HIV risk needs to be assessed among A/PI groups, including Thai and Korean Americans.

In prior research, the HIV sexual risk practices of Korean (e.g., Kim, Celentano, & Rosa, 1998) and Thai (e.g., Cash, Anansuchatkul, & Busayawong, 1999) samples outside of the United States has been examined. There is a lack of research on the reliability of these reports of sexual behavior and even less research on the self-reported HIV risk practices among people of Thai and Korean descent residing in the United States. The purpose of this study is to establish the test-retest reliability of the NIMH (National Institutes of Mental Health) HIV prevention trial baseline survey for assessing the HIV risk behaviors among individuals of Thai and Korean descent in their ethnic language. This survey was selected because it has been used and validated in a large and nationally representative sample of African Americans and Latinos in the United States (NIMH Multisite HIV Prevention Trial, 1997). The reliability of this survey, however, has not been established for different Asian ethnicities represented in the United States.

METHOD

PARTICIPANTS

Thai (N = 37) and Korean (N = 46) respondents were recruited from three community-based organizations in Los Angeles County, California. The eligibility requirements were that participants had to be (a) bilingual in English and their ethnic language or monolingual in their ethnic language, (b) sexually active in the last 90 days (vaginal or anal sex), and (c) single (never married, divorced, or widowed). The age of the participants ranged from 19 to 37 (M= 29, SD = 4.61) years and 51% were male. Only18% percent of the participants reported the United States as their country of birth. Approval for this study was obtained from the Institutional Review Board (IRB) at the University of California, Los Angeles.

PROCEDURE

An interviewer administered a slightly modified version of the NIMH HIV prevention trial baseline survey at two points via face-to-face interview. The interviews were conducted in the ethnic language of the participant at their home or recruitment site approximately 3 – 7 days apart. Each participant was paid $10 per interview for a total of $20. Responses to questions on sexual behavior can change as a result of sexual episodes that may take place between assessment periods. To limit the effect of sexual experiences between assessments on test-retest coefficients, we chose to conduct follow-ups within a short period of time.

All interviewers received extensive training on how to conduct interviews on sensitive topics prior to conducting their first interview. The training included mock interviews with feedback given to the interviewer to identify areas for additional training. Specific areas in the training included nondirectional probing, emotional tone used during questioning, and nonjudgmental responses to answers.

INSTRUMENT

The participants completed a slightly modified version of the NIMH Multisite HIV Prevention Trial survey¹ (NIMH Multisite HIV Prevention Trial, 1997). Modifications of the survey included the exclusion of some items concerning socioeconomic status, performance in high school, and STD symptoms. In addition, the original survey included partner specific questions for the last five sexual partners over the last 90 days. In the modified survey, the respondents were only asked to report sexual behavior with their last three sexual partners over the last 90 days. The final version of the modified survey included items on sociodemographics, acculturation, health and mental health status, STD history and symptoms, alcohol and drug use, sexual behavior in the last 90 days, exposure to unwanted/uninvited sex, and HIV testing experience. For the purposes of this paper, only items concerning sociodemographic, acculturation, sexual behavior and HIV testing were included for analyses.

Translation and Cultural Adaptation of Measures

A professional language service that specializes in Asian languages and dialects was hired to translate the English version of the survey into Thai and Korean. Native speakers fluent in English as a second language were used to translate each survey into Thai and Korean, respectively. A second pair of translators were then selected to provide back-translations of the survey. There were no major discrepancies between the English and translated versions of the Thai and Korean surveys.

Socio-Demographic and Background Information

Gender and marital status (never married, divorced, separated, or widowed) were self-reported. The participants completed a five-item version of Marin, Sabogal, Marin, Oter-Sabogal, and Perez-Stable’s (1987) acculturation scale measuring language use with the following response categories: (a) ethnic language only, (b) ethnic language more than English, (c) both equally, (d) English more than ethnic language, and (e) English only. Examples of scale items included, “What languages do you read and speak now?” and “What language do you usually speak at home?” For this sample, the internal consistency of the scale was excellent (α = .91).

Sexual Behavior

The respondents were asked to recall whether a sexual behavior or event took place (categorical items) and the number of times that behavior or event took place (continuous items). Examples of these items included “Have you ever had a sexually transmitted disease?” and “In the last 90 days, how often did you use condoms when you had vaginal or anal sex?” The respondents were also asked to recall behaviors with their last three sex partners over the last 90 days. Examples of questions with a specific sexual partner included, “Of those times that you had vaginal sex with [your partner], were you or [your partner] ever high on drugs or alcohol?” and “How many times did you have vaginal sex with [your partner]?” Finally, the respondent was asked to respond to questions on HIV testing. Examples of these items included, “Have you ever been tested for the virus that causes AIDS?” and “How many times have you been tested [for HIV]?”

DATA ANALYSES

Previous research has established that variations in HIV sex risk is based on the particular type of sex act or concurrent risk behavior (Rotheram-Borus, Marelich, & Srinivasan, 1999). In the current study, each item represents a specific behavioral aspect of sexual behavior. Thus examination of these behaviors on an individual basis is optimal in order to appropriately assess their reliability (see Shrout & Fleiss, 1979). The frequency of self-reports to responses for categorical items (yes, no, and don’t know) are reported. Means and standard deviations for continuous items are presented. Chi-square analyses and independent group t - tests were used to examine differences between the Thai and Korean samples on age, gender, and acculturation. These analyses were conducted to provide a general description of the populations sampled.

Kappa coefficients were used to assess the test-retest correspondence between categorical items. Kappa values can be interpreted according to the criteria found in Fleiss (1981): a value of .75 or greater represents excellent agreement, values between .40 and .75 represent fair to good agreement, and values under .40 represent poor agreement. Kappa values cannot be estimated when the responses of participants are not distributed among all valid response categories for both assessment periods. For example, if participants’ responses are distributed among categories “yes” and “no” at Time 1 and categories “yes,” “no,” and “don’t know” at Time 2, then Kappa cannot be calculated. In such cases, Kendall’s Tau is used to show the correspondence between the two assessments of a categorical item. There is no standard for equating Kappa and Tau coefficients. However, the Tau coefficient is similar to a Pearson bivariate correlation coefficient with values closer to 1 indicating better agreement. Intraclass correlations (ICCs) were used to assess the test-retest correspondence between continuous items. ICC values greater or equal to .70 indicate acceptable test-retest reliability (Shrout & Fleiss, 1979).

As mentioned, the respondents were asked to self-report sexual behaviors with their last three sexual partners in the last 90 days. A majority (88%) of respondents reported only having one sex partner in the last 90 days. Based on these results, all test-retest coefficients for Partner-specific questions over the last 90 days are calculated for only one partner.

RESULTS

FREQUENCY OF BEHAVIOR

The percentage of respondents endorsing “yes” to each index of sexual risk at Time 1 are presented in Table 1. The responses for the follow-up assessment (not presented in the table) were nearly identical. The mean and standard deviations for all behaviors assessed on a continuous scale at Time 1 are presented Table 2. Again, the mean responses were very similar across each test administration. There were no significant differences between Time 1 and Time 2 self-reports of behavior.

Table 1.

Percent of Respondents Responding “Yes” to Categorical Items at Time 1 and Kappa Coefficients.

Itemsa	N	Time 1 (%)	Thai κ(N)	Korean κ(N)
1. Did you or your partner use anything to prevent  pregnancy the last time you had sex?	83	49.4	.95 (37)	.88 (46)
2. Are you or your partner pregnant or trying to get  pregnant?	83	4.8	.94(37) d	.87(46)
3. Have you ever had a sexually transmitted disease?	83	13.3	.88 (37)	1.00 (46)
4. Have you ever had sex using a condom?b	43	81.4	.72 (19)	.80 (24)
5. Have you ever had sex when a partner used a con-  dom?c	57	10.8	1.00 (18)	1.00 (22)
6. During the past 90 days, have you given someone  money or drugs to have sex with you?	83	3.6	1.00 (37)	1.00 (46)
7. During the past 90 days, did you have receptive oral  sex?	83	39.0	1.00 (37)	1.00 (45)
Partner-Specific Questions Over Last 90 Days
8. Of those times [you had vaginal or anal sex], did  you ever use a condom?b	33	51.5	1.00 (18)	1.00 (14)
9. Of those times [you had vaginal or anal sex], did he  ever use a condom?c	38	44.7	1.00 (18)	.92 (20)
10. Of those times [you had vaginal or anal sex], were  you [or your partner] ever high on drugs or alcohol?	71	21.1	1.00 (36)	1.00 (34)
Questions Related to HIV Testing
11. Have you ever personally known anyone who has  tested positive for the virus that causes AIDS or  who has been diagnosed as having AIDS?	83	33.7	1.00 (37)	1.00 (46)
12. Have you ever been tested for the virus that causes  AIDS?	83	66.3	1.00 (37)	1.00 (46)

^aOrder of presentation of items in table differs from order of items found in survey.

^bItems answered by males only.

^cItems answered by females only.

^dKendall’s Tau coefficients are shown for italicized values.

Table 2.

Mean (M) and Standard Deviations (SDs) for Responses to Continuous Items at Time 1 and Intraclass Correlations (ICC)

Itemsa	N	Time 1 M (SD)	Thai ICC (N)	Korean ICC (N)
1. First, how old were you the first time you  had vaginal or anal  sex?	83	19.67 (4.5)	1.00 (36)	1.00 (44)
2. How many of these partners were male?	82	.72 (.87)	1.00 (37)	1.00 (43)
3. How many of these partners were female?	82	.50 (.67)	1.00 (37)	1.00 (43)
4. On how many days did you have vaginal  or anal sex in the last 90 days?	82	17.60 (14.8)	1.00 (36)	.97 (45)
5. On how many days did you use a condom  when you had vaginal or anal sex?	77	11.03 (15.4)	1.00 (37)	.99 (37)
6. In the last 90 days, how often did you use  condoms when you had vaginal or anal  sex?	79	2.94 (1.9)	1.00 (37)	.99 (42)
7. How often does your partner use a con-  dom when you have receptive oral sex?	32	4.03 (1.3)	1.00 (5)	.99 (26)
8. How often does your partner(s) ejaculate  (cum) in your mouth when you have receptive  oral sex?	31	4.19 (.98)	1.00 (5)	1.00 (25)
Partner Specific Questions Over Last 90 Days
9. How many times did you have vaginal sex  with [your partner]?	64	17.31 (12.6)	1.00 (33)	.97 (29)
10. How many times did [(you/he)] use a  condom?	38	18.71 (13.2)	1.00 (13)	.97 (20)
Questions Related to HIV Testing
11. How many people have you personally  known who have tested positive or diag-  nosed as having AIDS?	28	2.82 (3.6)	1.00 (9)	1.00 (15)
12. How many times have you been tested  [for HIV]?	55	2.91 (3.8)	.65 (28)	.86 (27)

^aOrder of presentation of items in table differs from order of items found in survey.

DIFFERENCES BETWEEN THAI AND KOREAN SAMPLES

There were no significant differences found between the Thai and Korean samples for age or gender. The mean scores on the acculturation scale ranged from 1 to 4.60 (Thai M= 1.38, KoreanM= 2.98). Higher scores for the Korean sample indicate higher levels of acculturation, t = 8.6 (82), p <.001.

TEST-RETEST CORRESPONDENCE

Kappa coefficients for categorical items to assess sexual behavior and HIV testing among the Thai and Korean respondents are presented in the last two data columns of Table 1. The coefficients ranged from .80 to 1.00 indicating excellent agreement between the two test periods for both the Thai and Korean sample. As mentioned, Kappa coefficients cannot be calculated when the responses of participants are not distributed among all valid response categories for both assessment periods. Under this criteria, Kendall’s Tau coefficients are presented for Item 2 and Item 4 (see italicized values in Table 1). The Tau coefficients were high and significant. The lowest test-retest coefficients were for Item 4, “Have you ever had sex using a condom?” Respondents showed the lowest correspondence on this item between the two assessment periods.

Intraclass correlations to assess the test-retest reliability for sexual behaviors assessed on a continuous scale are presented in the second two data columns of Table 2. The ICCs ranged from .65 to 1.00 for the Thai and Korean samples. With the exception of item 12, all ICCs were greater than .97 indicating acceptable test-retest reliability. For item 12, the participants were asked to recall how many times they had been tested for HIV. The ICC for item 12 was below the acceptable range for the Thai sample.

DISCUSSION

In the current study, the test-retest reliability of the baseline survey of the NIMH Multisite HIV Prevention Trial for the assessment of sexual behavior among individuals of Thai and Korean descent was examined. Test-retest coefficients on self-reports of sexual behaviors ranged from .65 to 1.00 demonstrating acceptable reliability of the survey among Thai and Korean adults. Our findings are comparable to the similar research with African American and Latino samples in which test-retest coefficients of sexual behavior over a 3-month period ranging from .53 to .89 were observed (NIMH Multisite HIV Prevention Trial, 1997).

Very few studies to document HIV risk among A/PI populations have taken into consideration the diversity found within the A/PI community. Possibly due to the small number of A/PI participants in many studies, A/PIs are often considered as one ethnic group for analyses. The combination of A/PI ethnic groups may conceal important group differences that should be taken into consideration when developing appropriate HIV prevention interventions. In the current study, we compared test-retest coefficients self-reports of sexual behavior for both Thai and Korean respondents. In general, we found that these items have appropriate psychometric properties for both Thai and Korean respondents.

Because our findings demonstrate the applicability of these instruments for these distinct cultural groups instead of for A/PIs as a whole, we can now begin to embark on studies of each particular A/PI group. Thus the development of appropriate research instruments is the first and necessary step towards a greater understanding of how sexual risk operates in these particular groups and therefore towards the prevention of HIV among these groups.

There are several factors that could affect the magnitudes of the test-retest coefficients found in this study. It should be noted that the magnitudes of the test-retest coefficients in this study are likely positively affected by the relatively short period between assessments. However, we chose a short period between assessments in order to limit the possibility of additional sexual episodes influencing the test-retest coefficients.

STUDY LIMITATIONS

A few of the behaviors examined in this study were highly skewed. Some of the skewed items tend to indicate lower risk, for example a few Thai respondents reported that ejaculation took place with oral sex. Other skewed items may indicate greater HIV risk. For example, only (approximately) half of the males in the sample reported that they used condoms when having vaginal sex. Even though skewed items have behavioral meanings that are interpretable, they may still limit the generalizability of our results. There is a substantial amount of diversity within A/PI populations found in the United States. Issues of diversity can include acculturation, generation status, immigrant status, language use, ethnicity, and sexual preference. The findings presented in this study are limited to a nonrandom sample of Thai and Korean participants. Caution should be used in generalizing these findings to other A/PI populations.