Validity of coital diaries in a feasibility study for the Microbicides Development Programme trial among women at high risk of HIV/AIDS in Mwanza, Tanzania

Abstract

Objectives

To compare coital diaries and face‐to‐face interviews (FFIs) in measuring sexual behaviour among women at high risk of HIV. To assess the effect of differing levels of support from researchers on reporting in coital diaries and FFIs.

Methods

Three groups of 50 women were randomly selected from a cohort of food and recreational facility workers participating in a microbicide trial feasibility study and received differing levels of researcher support. Minimum support involved delivering and collecting coital diaries weekly; medium support included a weekly FFI and discussion of concerns; intensive support also included an unscheduled mid‐week visit when diaries were checked and concerns addressed. All respondents participated in an exit FFI, including questions on sexual behaviour over the four‐week study period and study acceptability.

Results

Sexual behaviours were generally reported more frequently in coital diaries than weekly or exit interviews. Vaginal and anal sex, male and female condom use, vaginal cleaning and lubrication, sex during menstruation and sex with irregular and regular partners were reported more frequently in coital diaries than exit interviews. In coital diaries, level of support was associated with reporting of vaginal sex and cleaning. In exit interviews, support level was associated with reporting of vaginal sex, vaginal cleaning and sex with regular, irregular and commercial partners. Women with minimum support reported least satisfaction with the research process. Women with intensive support were most likely to report that they informed someone about their study participation and that they completed diaries daily.

Conclusion

Compared with FFIs, coital diaries resulted in higher reporting of socially stigmatised activities, and sexual behaviour reporting varied less by level of support. More researcher support enhanced study acceptability.

Coital diaries are data collection tools in which sexual practices of research subjects are recorded on a daily or other periodic basis or after each sex act. They have been used in sexual health studies to examine the efficacy or acceptability of products¹,² and track behaviour.³,⁴,⁵ Suitable sexual behaviour data collection methods are needed for developing countries, where most HIV infections occur. Low literacy and a desire to conceal socially proscribed activities may hamper the measurement of sexually transmitted infection (STI) risk behaviour.⁶ Valid sexual behaviour data collection tools are necessary to test the efficacy and acceptability of HIV prevention technologies, but are difficult to devise.⁷,⁸

Our study formed part of a feasibility study leading to a phase III trial of vaginal microbicides for HIV prevention, currently being conducted by the Microbicides Development Programme in six sites in sub‐Saharan Africa.⁹ Accurate data on the frequency of sexual practices are needed to show the behavioural conditions under which trial results were achieved. Among the aims of the feasibility phase was to develop data collection tools for use in the trial. This paper assesses the validity of coital diaries and the acceptability of coital diary study implementation by:

• Comparing the results of coital diaries with face‐to‐face interviews (FFIs);

• Assessing the sensitivity of coital diary results to different levels of support to participants by researchers;

• Assessing whether the acceptability of using coital diaries varied by the level of support provided.

Methods

Research subjects were selected from among 1573 women enrolled into the feasibility study in Mwanza, Tanzania, who worked in food or recreational facilities (cooked food or local beer vending, bars, restaurants, guesthouses and hotels).⁹ Women in these occupations in Tanzania are at high risk of HIV/STI.¹⁰,¹¹,¹²

The feasibility study included structured FFIs at enrolment (n  =  1573), ethnographic fieldwork in 68 food or recreational facilities, focus group discussions on sexual behaviour (six with food or recreational facility workers, two with male customers) and nine open‐ended interviews on vaginal hygiene practices. For the coital diary study, these informed the variables chosen, the design of pictures and instruments and the interpretation of results.

After pre‐testing of five diary designs, a design with pictures and tick boxes was selected. The coital diary covered 7 days and could be used to record up to four sex acts per day. It was 16 cm by 13 cm to fit into a pocket or handbag, and contained seven data entry sheets (representing each day of the week), and information sheets with explanations for each picture. The diary text was translated into Kiswahili and back before production. Fig 1 shows a collection entry page.

Figure 1 Coital diary collection entry page. Notes: Pictures show, from left to right: 1. Sex act (anal, vaginal). 2. Male condom. 3. Female condom. 4. Contraception. 5. Decrease and increase in lubrication. 6. Vaginal cleaning. 7. Menstruation. 8. Sex partner type (sex for money or gifts, irregular, regular).

Coital diary evaluation study

We assessed the impact on diary responses of three levels of support by researchers. Diary results were compared with data from FFIs covering two different recall periods (previous week or previous four weeks), either of which FFI might be employed at regular intervals during microbicide trials. The acceptability of the study methods was assessed by interview.

In discussions among Microbicides Development Programme coordinators, it was debated whether FFIs on sexual behaviour should aim to record frequencies over the past four weeks or one week. The preference initially was for four weeks, because a longer time period was thought likely to offer a more representative picture of sexual behaviour, and repeatedly interviewing each week would be more costly. In this paper we therefore focus primarily on comparisons between coital diaries and FFIs covering four weeks.

We selected a subsample of 150 women from participants who were due to attend for a three, six, nine or 12‐month follow‐up visit to mobile study clinics during a six‐week interview period (March to April 2004; N  =  538). The sample size of 150 participants was chosen according to logistical and human resource constraints, given a number of competing demands on the research team in meeting feasibility study objectives. Recruitment data suggested that there were few differences in women's characteristics or STI prevalence between the 10 wards of Mwanza City from which women were recruited to the feasibility study. For each level of researcher support, five wards were randomly chosen and a random sample of 50 women was selected from eligible participants in these wards. To reduce travel between interviews, we ensured a minimum of 10 participants in each ward for the three support levels combined.

Participants were asked to complete diaries for 28 days. Levels of researcher support were defined as follows:

Minimum – Participants received a scheduled weekly visit to their home or workplace to deliver and collect coital diaries. A FFI (exit interview) was conducted a few days after the final visit, including questions on frequency of sexual behaviour over the entire four‐week study period and on diary study acceptability.

Medium was the same as minimum support, plus during the weekly visit an FFI was administered relating to the previous 7 days and, if participants raised concerns about diary completion, these were discussed.

Intensive – An unscheduled weekly visit was added, at home or the workplace. During this, fieldworkers checked that participants were completing the diary daily and asked about and helped them with any concerns related to the diary.

Questions asked in weekly and exit interviews were closed‐ended, with the exception of some questions concerning interviewee experiences of completing diaries. Responses were recorded manually on questionnaires.

All participants signed or thumb‐printed an informed consent form. Ethics clearance was obtained from the National Medical Research Coordinating Committee, Tanzania and the London School of Hygiene and Tropical Medicine, United Kingdom.

Statistical analysis

Coital diary and FFI data were summarised as the frequency of each behaviour (a count) and the number of women reporting one or more occurrence of each behaviour (a percentage). Paired statistical tests were used to compare responses on coital diaries, four‐week FFIs and weekly FFIs within women, whereas comparisons of researcher support strategies were between women and so were unpaired. Reported frequencies of behaviours were not normally distributed; they were positively skewed, with many women reporting no occurrences of the behaviours in question. Non‐parametric methods were therefore used for statistical analysis.

For comparisons between coital diaries and the four‐week FFI, the Wilcoxon signed rank test was used for behaviour frequencies and the McNemar test was used for the percentage of women reporting one or more occurrence. These tests were repeated to compare coital diaries with weekly FFIs.

For comparing data by level of researcher support, the Kruskal–Wallis analysis of variance was used to compare behaviour frequencies. Chi‐squared tests were used for the percentage of women reporting one or more occurrence and to analyse the acceptability of the study by support level.

Kappa was used to measure agreement between pairs of instruments in reporting at all, and Spearman's rho to assess agreement in frequencies reported.

In the first week, interviewers mistakenly used a version of the weekly questionnaire that did not contain questions on vaginal cleaning or sex by partner type. Full four‐week data are therefore only available on the weekly interview for other variables.

Results

Characteristics of evaluation study sample

Participants in the coital diary study were selected to be representative of the feasibility study population (n  =  1573). Differences could be introduced at two selection stages: the choice of sample during our six‐week recruitment window (N  =  538) and the selection of participants from this sample (N  =  150). Feasibility study enrolment questionnaire data showed that diary study participants were similar to the feasibility study population regarding most sociodemographic characteristics, prevalence of HIV, herpes simplex virus type 2 and syphilis (data not shown). Of the coital diary study sample, 23% were aged under 25 years and 35% were over 35 years, compared with 30% and 25% of the feasibility study population (p = 0.004). Sixty‐six per cent were cooked food or local beer vendors, 19% were guesthouse or hotel workers and 15% were bar, restaurant or grocery workers, compared with 61%, 18% and 21% of the feasibility study population (p = 0.02). Twelve per cent of the coital diary study participants had secondary education or above, 20% were illiterate, 21% were HIV positive and 75% were herpes simplex virus 2 positive.

Of 150 women recruited, all accepted the invitation to participate. During the four weeks of data collection, however, six women dropped out. A total of 144 participants completed an exit interview after four weeks, 141 completed all four coital diaries, and 93 completed all four weekly interviews (weekly interviews were only administered to the 100 women who received medium or intensive support; fig. 2).

Figure 2 Flow chart of participation in the coital diary study.

Comparisons between coital diaries and FFIs

Reported frequencies of behaviours tended to be higher on coital diaries than exit interviews; this was so for vaginal sex, anal sex, male and female condom use, vaginal cleaning and lubrication, sex with an irregular partner and with a regular partner. Substantial differences were found for vaginal sex, male condom use, vaginal cleaning and sex with an irregular or regular partner (p<0.005). Comparing coital diaries and exit interviews, on diaries greater numbers of respondents reported any occurrence of male condom use, female condom use, vaginal cleaning, sex during menstruation and sex with an irregular partner. Very pronounced differences were found for vaginal cleaning, sex during menstruation and sex with an irregular partner (p<0.001; table 1).

Table 1 Four‐week totals of reported frequencies of behaviours, and percentages of respondents reporting behaviours at least once, on coital diaries and exit interviews.

	Mean		Standard deviation		Median		p Value (Wilcoxon signed rank test)	Percentage of respondents reporting behaviour at least once		p Value (MacNemar test)
	Exit FFI	Coital diary	Exit FFI	Coital diary	Exit FFI	Coital diary		Exit FFI	Coital diary
Vaginal sex	7.91	13.01	8.13	16.65	5	8	<0.001	77.3	75.9	0.687
Anal sex	0.03	0.05	0.24	0.38	0	0	0.180	1.4	2.1	1.000
Male condom use	1.96	3.65	5.10	10.41	0	0	0.002	25.5	37.6	0.001
Female condom use	0.01	0.25	0.17	1.58	0	0	0.051	0.7	4.3	0.125
Contraceptive use	NA	2.38	NA	8.55	NA	0	NA	NA	19.3	NA
Decreasing lubrication	0.01	0.07	0.17	0.77	0	0	0.655	0.7	0.7	1.000
Increasing lubrication	0.00	0.04	0	0.25	0	0	0.102	0.0	2.1	NA
Vaginal cleaning	2.10	5.88	4.3	12.7	0	0	<0.001	36.4	52.9	<0.001
Sex during menstruation	NA	0.70	NA	3.14	NA	0	NA	0.7	10.7	<0.001
Sex in exchange for money or gifts	0.87	0.84	3.12	3.49	0	0	0.657	14.2	12.1	0.664
Sex with irregular partner	0.20	1.57	0.97	3.87	0	0	<0.001	6.4	28.4	<0.001
Sex with regular partner	6.87	10.6	7.34	14.67	5	6	0.001	75.2	71.6	0.267

FFI, Face‐to‐face interview; NA, not available. Questions on frequency of contraceptive use and sex during menstruation were not asked in the exit interview. In exit interviews, however, respondents were asked if they had sex during menstruation in the past 4 weeks. All calculations are based on 141 cases with complete data on both instruments, with the exceptions of those for decreasing lubrication (138 cases), increasing lubrication and vaginal cleaning (140 cases).

The online appendix presents comparisons of coital diaries with weekly interviews. Whenever differences were considerable (p<0.05), the frequencies and percentages of individuals reporting were higher on coital diaries than weekly interviews. The differences were, however, generally smaller between coital diaries and one‐week recall FFIs than between coital diaries and four‐week recall FFIs. The online appendix can be viewed on the Sexually Transmitted Infections website (http://sti.bmj.com/supplemental).

Section 2 of the online appendix provides an analysis of agreement between data collection instruments.

Level of support

Over the four‐week study period, level of support appears to have had little impact on the reported frequency of behaviour in coital diaries, although lowest frequencies were reported in the medium support group for vaginal sex, contraceptive use, vaginal cleaning and sex with a regular partner. For vaginal cleaning, there was a markedly lower average frequency with medium support (p<0.001). In exit interviews, however, somewhat larger differences in frequencies reported by support level were found for vaginal sex, sex in exchange for money or gifts, sex with a regular partner, sex with an irregular partner and vaginal cleaning. For all but the last two of these, frequencies increased as the level of support increased. Strong relationships between reported frequencies and level of support were found for vaginal sex, vaginal cleaning and sex with a regular partner (p<0.001). In exit interviews, in each case when differences were found to be significant at the p<0.05 level, the number of acts reported was highest with intensive support (table 2). For eight of the 10 variables measured on both instruments, differences in frequencies between coital diaries and exit interviews were smallest either with medium or with intensive support. For vaginal sex, the positive difference between coital diary and exit interview frequencies fell as the level of support increased (F‐test, p = 0.023).

Table 2 Reported total frequencies of behaviours over four weeks, and percentage of respondents reporting behaviour at least once over four weeks, by level of support for coital diaries and exit interviews.

Level of support	Mean			p Value (Kruskall–Wallis test)	Percentage of respondents reporting behaviour			p Value (Pearson χ2 test)
	Minimum	Medium	Intensive		Minimum	Medium	Intensive
Coital diary variables
Vaginal sex	14.2	10.9	13.9	0.177	65.3	76.1	87.0	0.048
Male condom use	4.2	3.3	3.3	0.420	30.6	37.0	45.7	0.317
Contraceptive use	3.5	1.5	2.1	0.207	14.3	17.8	29.8	0.146
Vaginal cleaning	8.2	2.6	6.7	<0.001	55.1	30.4	73.3	<0.001
Sex during menstruation	0.8	1.0	0.3	0.806	12.2	11.1	8.7	0.851
Sex in exchange for money or gifts	1.2	1.0	0.2	0.985	12.2	10.9	13.0	0.949
Sex with irregular partner	1.5	1.8	1.4	0.527	28.6	21.7	34.8	0.382
Sex with regular partner	11.3	8.1	12.3	0.117	61.2	71.7	82.6	0.069
Exit interview variables
Vaginal sex	4.9	7.1	11.9	<0.001	65.3	78.7	89.6	0.016
Male condom use	1.1	2.7	2.0	0.221	18.4	31.9	25.0	0.309
Vaginal cleaning	2.7	0.1	3.4	<0.001	51.0	6.4	50.0	<0.001
Sex during menstruation	NA	NA	NA	NA	0.0	2.1	0.0	0.354
Sex in exchange for money or gifts	0.1	1.1	1.4	0.015	6.1	10.6	25.0	0.020
Sex with irregular partner	0.1	0	0.5	0.010	4.1	0.0	14.6	0.010
Sex with regular partner	4.7	6.1	9.9	<0.001	63.3	76.6	87.5	0.021

NA, not available. Variables indicating anal sex, female condom use, increasing and decreasing lubrication were excluded from comparisons of reporting by level of support because only six or fewer respondents reported these behaviours at all on coital diaries or FFIs. A question on frequency of sex during menstruation was not asked in the exit interview. In exit interviews, however, respondents were asked if they had sex during menstruation in the past 4 weeks. Coital diary sample size N  =  141. Minimum support, 49; medium support, 46; intensive support, 46. Exceptions are: Decreasing lubrication N  =  138. Minimum support, 49; medium support, 43; intensive support, 46. Increasing lubrication N  =  140. Minimum support, 49; medium support, 45; intensive support, 46. Vaginal cleaning N  =  140. Minimum support, 49; medium support, 46; intensive support, 45. Exit interview sample size N  =  144. Minimum support, 49; medium support, 47; intensive support, 48.

On coital diaries, there were increases in the number of respondents reporting one or more event as the level of support increased for vaginal sex, contraceptive use and sex with a regular partner. For vaginal cleaning, the number of individuals reporting was substantially lower with minimum support (p<0.001). On exit interviews, there were increases in the number of respondents reporting one or more event as the level of support increased for vaginal sex, sex in exchange for money or gifts, and sex with a regular partner (p<0.05 in each case). For vaginal cleaning and sex with an irregular partner, the number reporting was lowest with minimum support (p<0.001 and p = 0.010, respectively). On exit interviews, the number reporting was generally highest with intensive support, except for the vaginal cleaning variable when reporting was roughly equal between minimum and intensive support (table 2).

Evaluation of coital diary and survey process

Less than 5% of participants with intensive support said they found any diary pictures difficult to understand, unacceptable or embarrassing. In contrast, several pictures were reported embarrassing by more than 5% of participants in either the minimum or medium support group. Women with minimum support reported least satisfaction with the research process and were most likely to report that someone had discovered their diary. Women with intensive support were most likely to say they informed someone about their participation in the study and that they completed their diaries daily (table 3).

Table 3 Reported acceptability of the coital diary study by level of support*.

Level of support	Minimum	Medium	Intensive	All	p Value (Pearson χ2 test)
	Percentage who reported the picture was embarrassing
Picture
Sex act	10.2	12.8	0	7.6	0.046
Male condom	0	8.5	0	2.8	0.014
Female condom	2.1	10.6	4.2	5.6	0.168
Contraception	0	4.3	0	1.4	0.123
Lubrication	2.0	29.8	4.2	11.8	<0.001
Vaginal cleaning	38.8	17.0	0	18.8	<0.001
Menstruation	6.1	12.8	0	6.3	0.037
Sexual partner	2.0	10.6	2.1	4.9	0.081
	Percentage responding “Yes”
Did you feel you were able to understand the diary?	73.5	100	97.9	90.3	<0.001
Were you given enough information to be able to complete the diary every day?	95.9	100	97.9	97.9	0.375
Were you happy about the number of visits you received during the diary completion period?	85.7	100	100	95.1	0.001
Did you inform anyone about the diary study?	14.3	17.0	60.4	30.6	<0.001
Did anyone discover your diary either at home or in your place of work?	38.8	10.6	10.4	20.1	<0.001
Did you complete your diary on a daily basis?	57.1	76.6	83.3	72.2	0.011

*Based on exit interview data, n  =  144.

Discussion

This study has shown a general pattern of more frequent reporting of sexual behaviour in coital diaries than in FFIs. The level of support from researchers apparently affected the frequencies of acts reported and the numbers of individuals reporting on a greater number of sexual behaviour variables in four‐week FFIs than in coital diaries. Level of support was associated with the acceptability of the diary design and completion processes.

Strengths and weaknesses of the study in assessing the validity of coital diary data

Assessing the validity of quantitative sexual behaviour data collection tools among high‐risk populations generally entails comparisons between self‐reporting methods.¹³ Validation through partner reports is largely impractical. Frequency data cannot be validated using biomarkers.⁸ This study used several types of comparison between self‐report methods to assess validity, including measuring reporting patterns, using qualitative methods to explain patterns, assessing stability under different research conditions and examining strength of agreement.

For socially sensitive behaviours, higher reporting is argued to indicate higher validity and lower social desirability bias.¹⁴,¹⁵,¹⁶ We found a general pattern of higher frequencies and reporting in diaries than exit interviews, including that of sexual behaviours found in qualitative research during our microbicides feasibility study to have low social acceptability (male condom use, sex with an irregular partner, sex during menstruation). Qualitative methods may be used to help explain patterns in quantitative data, with reference for example to prevailing social norms.⁸,¹⁷ Condom counselling was provided in the feasibility study, but qualitative research revealed that the use of male condoms symbolised sexual infidelity and commercial sex and was unacceptable when a partner wished for a long‐term relationship, or when women wished to please their partner sexually. This suggested that social desirability bias, if it existed, would tend to deflate condom use reporting.

Diaries encourage frequent data recording, reducing recall bias. The importance of recall is suggested by the pattern of reporting we found, with higher reporting on coital diaries but larger differences with four‐week interviews than with weekly interviews.

Several studies in developing countries have explored the potential of coital diaries¹⁸,¹⁹,²⁰ or other methods to reduce social desirability bias.⁶,⁸,²¹ These have generally shown lower reporting of socially sensitive behaviours in FFIs, but have not analysed the impact of the level of support on reporting. The stability of findings under differing research conditions (such as researcher support) is a further criterion for validity assessment.²² Although the picture is not clear‐cut, coital diary results were generally more consistent than exit interviews in the face of differences in support. Frequencies reported in exit interviews were generally highest with intensive support, and differences between diaries and exit interviews were smallest with medium or intensive support. If we accept both higher reporting of socially proscribed behaviours and stability in results as methods of assessing validity, coital diaries emerge as more valid, because reporting on coital diaries was higher and exit interview results only tended to converge with coital diary results when researchers provided more support.

The problem of reactivity occurs when awareness that behaviour is being measured influences the behaviour being studied.²³ As in other coital diary studies, different tools were administered to the same respondents, so the use of one may have influenced reports on others; this is a variant of the reactivity problem. Future studies may be strengthened by administering different methods to different random samples.

“Construct validity” refers to whether variables measure what they are supposed to measure.¹⁶ A method to assess this is agreement between items measuring the same construct.¹⁸,²⁴ The online appendix shows and discusses agreement between coital diaries and FFIs.

Given the sample size (less than 50 per group), our study may have been only moderately powered to detect some differences of potential importance. Substantial differences were found despite this, however, and these were consistent with expected patterns according to the literature and our qualitative research. Previous studies comparing coital diaries with other methods of recording sexual behaviour have been conducted with samples of less than 150,¹⁸,²⁴,²⁵,²⁶,²⁷,²⁸,²⁹,³⁰ in contrast with several studies comparing other self‐report methods, which have used larger samples,⁶,⁸,²¹ Further attention should be paid to the validation of coital diaries, using larger sample sizes.

Utility of coital diaries in evaluating sexual health technologies

The need for quantitative sexual behaviour data collection methods appropriate for developing countries is likely to increase with the growth in clinical trials of STI prevention technologies. An advantage of daily diaries is that they permit the collection of per‐sex‐act data, showing the co‐occurrence of behaviours.¹⁶,²⁵ For example, in interpreting microbicide trial results showing changes in HIV incidence, we may wish to know, for those sex acts in which a microbicide was used, what number were accompanied by behaviours that may influence the impact of the microbicide on HIV transmission. Using coital diaries, one can calculate the proportion of reported sex acts with microbicides in which, for example, either condoms or vaginal cleaning, or both, were reportedly used. Recalling individual events and proportions in FFIs covering longer periods may be challenging.

The generation of per‐sex‐act data is costly, however, because data on each individual sex act must be entered separately. Computer data entry for diaries may be more time‐consuming than for FFIs.

We found that reportedly the coital diary study was more acceptable to participants with higher support. In our study, however, medium support cost 23% more than minimum; intensive support added a further 15%. As there was little impact on reporting in coital diaries, the advantages of higher support appear to be in acceptability. The burden to participants of daily completion over extended periods should also be considered.¹⁸

Our results support the validity of coital diaries compared with four‐week FFIs, because on coital diaries there was higher reporting of socially proscribed behaviour, somewhat greater stability of results in the face of differing levels of researcher support, and patterns of reporting were consistent with expectations from qualitative research. The burden and costs of coital diary completion must, however, be considered. Regular data collection with random samples of participants in HIV prevention trials may be worthwhile to collect per‐sex‐act data and estimates with higher validity than may be possible using FFIs.

The online appendix can be viewed on the Sexually Transmitted Infections website (http://sti.bmj.com/supplemental).

Abbreviations

FFI - Face‐to‐face interview

STI - sexually transmitted infection