Challenges in Measuring the Sequencing of Life Events Among Adolescents in Malawi: A Cautionary Note

Abstract

Using data from two rounds of the Malawi Schooling and Adolescent Survey, this research note examines consistency of retrospective reporting on the timing and sequencing of sexual initiation, school leaving, and marriage. The analysis, which compares reporting of events both within and between rounds, indicates substantial inconsistency in reporting of event sequences and highlights difficulties in measuring transitions to adulthood in sub-Saharan Africa with survey data.

Introduction

Experiences during adolescence shape future life circumstances; thus, considerable attention has been devoted to describing transitions to adulthood in the developing world (Lloyd 2005). In particular, demographers focused on sub-Saharan Africa have attempted to explain trends and differences in the timing of sexual initiation, school leaving, pregnancy, and marriage (Biddlecom et al. 2008; Lloyd and Mensch 2008; Marteleto et al. 2008). Given recall problems and misreporting of age in the region, where birthdays are often not known and the month in which a particular transition takes place has little salience, ordering of life events may be subject to considerable error. Without accurate data on sequencing of events, however, it is difficult to assess whether school attendance plays a role in reducing high-risk behavior, increasing age at marriage, and delaying childbearing. Nor is it possible to estimate the degree to which premarital sexual activity or early marriage raises the likelihood of school dropout. Although numerous studies have examined the validity of sexual behavior data in the region (Gregson et al. 2009; Langhaug et al. 2011; Lindstrom et al. 2010; Luke et al. 2011; Mensch et al. 2003; Mensch et al. 2008; Minnis et al. 2009; Nnko et al. 2004), this research note goes beyond prior methodological investigations in assessing consistency of retrospective reporting on the timing and sequencing of sexual initiation, school leaving, and marriage using data from Rounds 3 and 4 of the Malawi Schooling and Adolescent Study (MSAS).

Data

The MSAS is a longitudinal study of 2,649 adolescents aged 14–16 at the start of 2007 who were resident in two contiguous rural districts in the southern region of the country. The initial sample consisted of 1,764 students randomly selected from enrollment rosters at 59 randomly selected primary schools in Machinga and Balaka districts. The probability of a particular school being included was proportional to its enrollment in 2006. At each school, approximately 30 students stratified by gender and age and enrolled in standards 4–8 (the last four years of primary school) were interviewed. An additional sample of 885 adolescents not enrolled in school was drawn from the communities surrounding the selected primary schools. The ratio of 14- to 16-year-olds attending standards 4–8 to those out of school (67:33) was based on the proportion observed in the 2004 Malawi Demographic and Health Survey (70:30). The study successfully reinterviewed 91 %, 90 %, and 88 % of the original sample in 2008 (Round 2, R2), 2009 (Round 3, R3), and 2010 (Round 4, R4), respectively.

In R3, the question on age at first marriage was asked via face-to-face interview (FTFI). The question on age at sexual initiation was asked via audio computer-assisted self-interview (ACASI), with the intent of increasing privacy and minimizing social desirability bias.¹ In contrast to questions on marriage and sexual initiation, age upon leaving school was not directly asked. For the in-school sample, we used the current status question on school attendance for each round, determined the rounds between which the adolescent dropped out, and took the midpoint of the current ages reported in the rounds preceding and following dropout. For the out-of-school sample, we compared the response to a question on age at start of the most recent school year attended with age at last standard attended calculated from a school history grid, and used whichever was higher. We then added six months to account for those who might have had a birthday between the start of the school year and the point of dropout. Note that estimated ages were used only for analyses generated by FTFI/ACASI; see Fig. 1.

Fig. 1.

Event sequences reported with FTFI/ACASI and life event cards in Round 3 (2009). Because of the estimation procedure used to generate the age at leaving school in the FTFI, the figure indicates the upper bound of events occurring at the same age. ^aOrder of events at the same age cannot be established with FTFI/ACASI. L = leaving school, M = first marriage, and S = sexual initiation

Because many adolescents reported discrepant ages within and between rounds for sexual initiation and marriage (Soler-Hampejsek et al. 2013), after the structured survey was completed, we began in R3 to ask participants to order a set of life event (LE) cards that illustrated major transitions: sexual initiation, school leaving, and first marriage.² The respondent, after placing the cards in the sequence in which events occurred, was asked his or her age at each transition. Interviewers were trained to reconcile inconsistencies between LE cards and FTFI in the reporting of a transition except for sexual initiation, which was asked via ACASI.

Results

Comparison of FTFI/ACASI Reporting With Reporting From LE Cards

Comparison of R3 current status reports of sexual initiation collected via ACASI with those from LE cards used in the same interview reveals no significant differences for both boys and girls, despite ACASI being considered more confidential (56.7 % ACASI vs. 54.1 % LE for boys, p = .209; and 66.3 % ACASI vs. 62.6 % LE for girls, p = .057). We next compare data from FTFI/ACASI with those derived from LE cards for reporting of sequences of sexual initiation, marriage, and school leaving in R3 (see Fig. 1). The most important difference between the two methods of data collection is that order can be established with LE cards when events occurred at the same age. This advantage is not critical for establishing sequencing for boys, since only 7 % reported events occurring at the same age; however, it is for girls, since up to 40% reported at least two events—and often three—as having occurred at the same age. Timing of events by month was collected to resolve ties for FTFI/ACASI, but because months do not appear to be a pertinent reference point for respondents,³ the data were considered unreliable for generating sequences.

Among respondents who provided an age for sexual initiation both in ACASI and LE cards in R3, only 36.6 % of adolescents reported the same age with the two methods, with little variability by sex (boys = 36.3 %, and girls = 36.9 %). On average, the age reported in ACASI for both males and females is lower than the age reported using the LE cards. Although inconsistency in reporting age at sexual initiation within R3—specifically, 63.4 %—is not as elevated as inconsistency reported between R1 and R2 within the ACASI mode—approximately 75 %—it is still very high and indicates just how problematic age reporting is in settings such as Malawi.

Consistency of Reporting With LE Cards

Figure 2 compares LE reports in R3 and R4 and indicates that at least at the aggregate level, the sequences specified across the two rounds are not incompatible; with the passage of another year, both boys and girls experience more transitions.

Fig. 2.

Event sequences reported with life event cards in Round 3 (2009) and Round 4 (2010). L = leaving school, M = first marriage, and S = sexual initiation

To examine individual-level consistency across rounds, we restrict the comparison to girls who experienced all three transitions by R3 because girls are much more likely to have experienced multiple events. In the aggregate, the sequence of events specified in R3 and R4 is quite similar (Fig. 3). However, of the 472 girls who reported three events in R3 and were reinterviewed in R4, only 38.6 % reported the same sequence of events in R4 as they had in the prior round. Figure 4 displays the sequences designated by girls in R4 according to the sequence reported in R3. Of the 323 respondents who reported premarital sex in R3, 28.5 % reversed the order and reported that they were virgins at marriage in R4. Of the 107 respondents who indicated they had sex prior to leaving school in R3, only 21.5 % reported in R4 that they became sexually active while still attending school.

Fig. 3.

Event sequences reported with life event cards in Round 3 (2009) and Round 4 (2010) for girls who had reported all three transitions by Round 3 (N = 472). ). L = leaving school, M = first marriage, and S = sexual initiation

Fig 4.

Event sequences reported with life event cards in Round 4 (2010) by sequence reported in Round 3 (2009), for girls who had experienced all three transitions by Round 3 (N = 472). L = leaving school, M = first marriage, and S = sexual initiation. “Other” comprises respondents who in R4 reported no event, two events, or the three-event sequence “SML”

In the adolescent sample more generally, of the 1,470 adolescents who reported sexual activity or school leaving or both in R3 and were interviewed again in R4, 339 (23.1 %) reversed the sequence reported or implied in the previous round (Fig. 5); an additional 9.8 % denied one or both events. Consistency of reporting was slightly higher for sexual activity and marriage, with 218 (17.3 %) of the 1,259 adolescents who reported one or both of these events in R3 reversing the sequence, and an additional 10.7 % denying one or both events.

Fig. 5.

Consistency of reporting of event pairs (sexual initiation and school leaving; sexual initiation and first marriage) with life event cards in Rounds 3 and 4 (boys/girls). ^aTwenty-six adolescents, 20 boys and 6 girls, who indicated that they had left school in Round 3 but reported that they were still in school in Round 4, returned to school between Rounds 3 and 4. Of these 26, 4 (3 boys and 1 girl) denied in Round 4 that they had initiated sex, having reported it in Round 3. Therefore, 22 of these adolescents are considered to be consistent in our analysis

We examined whether events for respondents who reported inconsistent sequences might have been reversed because they occurred at the same age. If the interval between events is very short, the sequence may not be particularly salient in the mind of the respondent. However, only 43.1 % (94 of 218) of those reporting sexual initiation and school leaving at the same age in R3 reported that these events occurred at the same age in R4. Likewise, only 54.3 % (157 of 289) of those reporting sexual initiation and marriage at the same age in R3 reported that these events occurred at the same age in R4, suggesting that temporal proximity of events does not fully explain the observed sequence reversals between R3 and R4.

Summary and Conclusion

As is the case for others who have examined age at sexual initiation and age at first marriage with longitudinal data in African populations (Gregson et al. 2009), we find a high level of inconsistency in reporting by MSAS respondents, both within rounds across interview modes and between rounds with the same interview mode. Our analyses indicate the following:

• The aggregate difference in the prevalence of premarital sex reported with LE cards and with ACASI is not significant.

• The level of inconsistency in reporting age at first sex is high even within rounds; in R3, almost two-thirds of adolescents who indicated they had initiated sexual activity reported an age at sexual debut different in ACASI from that reported in LE cards, with a lower average age reported in ACASI.

• Inconsistency in reporting event sequences is high; among respondents who reported at least one event in R3, between 17 % and 23 % reversed the reported or implied sequence between rounds. Of the girls who reported three events in R3, fewer than 40 % reported the same sequence in R4.

• Temporal proximity of events does not fully explain observed sequence reversals across rounds. Only 43 % of males and females reporting the same age for sexual initiation and school leaving in R3, and 54 % of those reporting the same age for sexual initiation and marriage, reported those events as occurring at the same age in R4.

• Sequence switching across rounds for premarital sex and school leaving, and premarital sex and marriage, operates in both directions; further analysis of the determinants of inconsistent reporting is warranted.

The inconsistent reporting that we observed in our data is substantial. The measurement challenges that we faced are likely not unique to Malawi and reveal the difficulties encountered in documenting transitions to adulthood in sub-Saharan Africa using retrospective data. We recommend that researchers conducting longitudinal surveys of adolescents take advantage of the prospective nature of the data and ask about current status at each round, which should be reported more reliably than events further in the past. For multiple transitions occurring between rounds, questions can be included about the sequence. When faced with conflicting data in longitudinal surveys, we suggest that sensitivity analyses be performed. Finally, we caution researchers conducting adolescent surveys in Africa to maintain a healthy skepticism about their data and the ability to draw definitive conclusions regarding the timing and sequencing of transitions to adulthood.