Response of Sensitive Behaviors to Frequent Measurement

Abstract

We study the influence of frequent survey measurement on behavior. Widespread access to the Internet has made important breakthroughs in frequent measurement possible—potentially revolutionizing social science measurement of processes that change quickly over time. One key concern about using such frequent measurement is that it may influence the behavior being studied. We investigate this possibility using both a population-based experiment with random assignment to participation in a weekly journal for twelve months (versus no journal) and a large scale population-based journal-keeping study with weekly measurement for 30 months. Results reveal few of the measured behaviors are correlated with assignment to frequent measurement. Theoretical reasoning regarding the likely behavioral response to frequent measurement correctly predicts domains most vulnerable to this possibility. Overall, however, we found little evidence of behavioral response to frequent measurement.

1.1 Introduction

Like placing a cool thermometer into a warm beaker of liquid to measure the temperature of the liquid, every measurement we take has the potential of distorting the thing we aim to measure. This issue is just as relevant in social research as in other areas of science (Zwane et al., 2011). Every measurement we take from human beings has the potential to affect the human behavior we hope to measure (Fitzsimons and Moore, 2008; Warren and Halpern-Manners, 2012; Zwane et al., 2011). Even as scientific attention to these issues grows (Crossley, de Bresser, Delaney, and Winter, 2014; Schneider, Tahk and Krosnick, 2007; Williams, Block, and Fitzsimons, 2006; Wilson and Howell, 2006), the demands for more repetitive measurement— especially multiple measures of the same person—are growing at an even faster pace (Dunton and Atienza, 2009; Ginexi et al., 2013; Schlam et al., 2012). We focus on repeated measures over time—an area of substantial investment of effort, with recent breakthroughs significantly enhancing our ability to conduct measurements of the same people frequently. Here we investigate the potential for frequent measurement—frequently repeated survey data collection—to affect the very behaviors we aim to measure.

The substantive focus of our investigation comes from family sociology/demography, in which a great deal of effort has been invested to create detailed measures of human behavior over time. Two of the largest national longitudinal surveys in the United States—the National Survey of Families and Households (1980s/90s) and the National Longitudinal Survey of Adolescent Health (1990s/2000s)—were devoted to these topics (Acock, no date; Carver, Joyner, and Udry, 2003; Sweet, Bumpass, and Call, 1988; Udry, 1997, 1998;). Other regional panel studies and specialized measurement techniques have also been devoted to measuring change over time in family processes (Phelps, Furstenberg, and Colby, 2002; Thornton, Axinn, and Xie, 2007). Key reasons for these investments is that young people's relationships, sexual behavior, and contraceptive use all change rapidly over early adulthood, are causally intertwined, and can have substantial long-term consequences (Bearman, Moody, and Stovel, 2004; Bumpass, 1990; Bumpass, Sweet, and Cherlin, 1991; Thornton, Axinn, and Xie, 2007). Thus careful investigation of cause and effect in early-life family events requires detailed measurement of events over time to adjudicate timing and sequencing of specific events. Our investigation of measurement effects on behavior focuses on the latest advance in measurement methods in this substantive field—the use of electronic journal keeping—to gather weekly measures of relationship status, sexual behavior, contraception, and pregnancy.

Widespread access to the Internet has made important breakthroughs in journal-keeping measurement possible (Bolger, Davis, and Rafaeli, 2003). On the web, respondents can easily provide frequent updates about their behavior with a high level of privacy and confidentiality. Geographic mobility need not hamper access, and alternative modes such as telephone can be used when Internet access is interrupted. As a result, this technique has the potential to greatly advance researchers' ability to measure behavior frequently. One key concern about using such frequent measurement is that it also has the potential to influence the behavior being measured. In the paragraphs below we investigate this possibility by drawing on two complementary sources. The first is a population-based experiment with both pre and post measurement of outcomes and random assignment of half the participants to completion of a weekly journal for twelve months. The second is a large scale population-based journal-keeping study that featured weekly measurement for 30 months. Both included the same journal-keeping measures focusing on relationships, sexual behavior, contraception, and pregnancy among young women. Together they provide a unique opportunity for understanding the potential for frequent measurement to influence behavior in these substantive domains.

2.1 Journal Keeping

New technologies for acquiring measures from human subjects have the potential to revolutionize social research, in general, and the ability of researchers to measure the relative timing of personal events, in particular. Recent advances in computer-assisted interviewing technologies are at the core of this revolution. Computer-assisted interviewing has now become routine in social and behavioral data collection, opening many new possibilities for measurement of difficult topics, self-interviewing, electronic linking of data records, and enhanced quality control (Couper et al., 1998). In addition, relatively recent changes in the US population, such as widespread access to computers and the Internet as well as cellular and other telephone technologies, have opened substantial new avenues for social and behavioral measurement (Couper, 2005). These changes make large-scale electronic journal data collection a real and attractive option for social and behavioral measurement.

A small number of studies has both demonstrated the feasibility of and foreshadowed the scientific potential of these methods. These studies have been limited in their sample selection (e.g., Barrett and Barrett, 2001; Helzer et al., 2006; Kaminer et al., 2006; Kranzler et al., 2004; Lee, Choi, and Beal, 2006; Toll et al., 2006; Vivoda and Eby, 2006), variety of method use (e.g., Armeli et al., 2008; Baer, Saroiu, and Koutsky, 2002; Herbenick et al., 2011; Kiene et al., 2009; Moloney et al., 2009; Park, Armeli, and Tennen, 2004), topical focus (e.g., Armeli et al., 2008; Baer, Saroiu, and Koutsky, 2002; Herbenick et al., 2011; Kiene et al., 2009; Moloney et al., 2009; Park, Armeli, and Tennen, 2004), and time span of data collection (e.g., Aldridge-Gerry et al., 2011; Merz and Roesch, 2011). Although these studies have made important contributions to the usage of new technologies in data collection, few, if any, studies have attempted frequent measurement using self-administered methods on a probability-based sample.

Broader populations have been reached using Internet and telephone data collections (e.g., Couper, 2000, 2008; Galesic, Tourangeau, and Couper, 2006; Kreuter, Presser, and Tourangeau, 2008; Tourangeau, Steiger, and Wilson, 2002). However, these data collections rarely use repeated measurement within a short time frame. Furthermore, telephone and Internet have not been combined into a single tool to enhance both measurement quality and improve representation of the population for greater inferential value.

The study we report here takes the next step in this technological revolution in social and behavioral measurement—scaling up the technology by constructing a tool that can be used across a wide range of topics and in population-based studies. The key novelty is in moving the use of these methods beyond the limits of small-scale lab-based studies among volunteers to probability-based samples (Barber, Kusunoki, and Gatny, 2012). Furthermore, this new tool extends the time frame of the measurement beyond the limits of most studies conducted to date. The tool mixes two modes of data collection (Internet and phone), with the goal of maximizing the benefits and minimizing the drawbacks of each. Finally, the journal keeping system we describe involves measurement that is repeated weekly, covering a variety of topics and tailored to fit each respondent's individual circumstances. This last feature creates the greatest risk for measurement error: the potential for frequent repeated measurement to influence participants, biasing results.

2.1.1 The Potential for Frequent Measurement to Influence Behavior

The theoretical basis for expecting frequent measurement—whether interview-based or self-administered—to influence human behavior is grounded in social and psychological theories of human behavior (Zwane et al., 2011). The fundamental idea is that the conditions and social interactions surrounding each individual person shape her or his understandings of the world, create beliefs and dispositions, and drive behaviors (Mead, 1967 [1934]). In day-to-day life, exposure to words, concepts, and ideas can shape beliefs and attitudes, and those attitudes become a guide to behavioral choices (Ajzen, 1988; Fishbein and Ajzen, 1975). Even while most theories of behavior recognize long term continuity of behavior, driven by biological, social, and psychological forces (Elder, 1974, 1983), frameworks for studying change in behavior focus on the exposure of individuals to new stimuli in their environments, including messages in all forms (Ajzen, 1988; Barber, 2004; Mead, 1967 [1934]).

Social psychology identifies “mere exposure” as a potentially powerful mechanism that may influence behaviors following measurement. We know that new experience with a behavior creates more positive attitudes toward that behavior, increases familiarity with the behavior, and increases the likelihood of subsequently repeating the behavior (Ajzen, 1988; Axinn and Thornton, 1993; Thornton, 1985; Thornton, Axinn, and Xie, 2007). By measuring either intentions to engage in a behavior, or the past performance of a behavior, we risk influencing future engagement in that behavior via merely exposing respondents to the idea of engaging in it. When measuring intentions, this mere exposure increases attitude accessibility so that the respondent may either increase or decrease their engagement in the behavior, depending on their attitudes toward it (e.g., Fitzsimons, Nunes, and Williams, 2007; Morwitz and Fitzsimons, 2004). So, the mere exposure to the idea of contraceptive use may increase positive attitudes toward contraceptive use. By repeating exposure to the idea of contraceptive use through frequent measures, individuals are likely to form increasingly positive attitudes toward contraception (Barber, 2004; Mead, 1967 [1934]; Zajonc, 1968, 2001). These more positive attitudes toward contraception, in turn, are likely to increase the behavioral use of contraceptive methods (Ajzen, 1988; Brauner-Otto, 2013; Fishbein and Ajzen, 1975). Thus repeated questioning about sex and contraception is expected to change contraceptive use behaviors through a potentially powerful psychological mechanism.

Mere measurement of past behavior may be expected to operate in a similar manner. Repeated questioning about contraceptive use, for example, increases exposure to the idea of using contraception and can affect future behavior. In fact, as a person is repeatedly asked about their past behavior, their attitudes around that behavior can become more salient.¹ If their past behavior does not fit with their attitudes, this can influence their future behavior in a way that more closely matches their attitudes (Spangenberg et al., 2003). In the case of a behavior that is common, it is likely that repeat measurement will increase future engagement in the behavior (Torche et al., 2012). On the other hand, asking about risky behaviors, which are not common, may increase engagement in the risky behavior if a person has positive underlying attitudes toward it (Fitzsimons and Moore, 2008; Moore and Fitzsimons, 2008). In the case of contraceptive use in the United States—a behavior which is common and a behavior in which non-engagement may be more risky than engagement—we expect that measurement of past experience is likely to produce greater contraceptive use in the future.

Finally, mere measurement may be more likely to shape behavioral choices within a particular domain of behavior than it is to lead to initiation of an entirely new behavior. For example, repeated measurement may be more likely to shape choices among exercises than it is to lead a person to begin exercising for the first time. In the domain we study here, frequent measurement may be more likely to shape choices among contraceptive methods than it is to lead to a person beginning contracepting for the first time. Contraceptive method switching is relatively common in early adulthood in the United States, especially as sexual relationships end and new ones begin (Grady, Billy, and Klepinger, 2002). So, we expect effects of frequent measurement on contraceptive use may be greatest on contraceptive switching behavior.

2.1.2 Repeat Measurement of Individual versus Joint Behaviors

Repeat measurement is likely to have a stronger influence on behavior in domains in which individuals can act alone than in domains which require joint behaviors with other individuals. Sexual behavior is the outcome of a complex process involving the behavior of at least one other person—the individual's sexual partner. In fact, there is evidence that repeatedly asking questions about sex has no effect on the actual behaviors reported by respondents (Halpern, Udry, and Suchindran, 1994; Jaccard et al., 2004). Halpern et al. (1994) find that “comparisons… indicate that frequent, even weekly, assessment of sexual activity over a 2-year period does not systematically change behavior to any greater degree than does a single pretest completed 2 years prior to a second assessment” (p. 51). As it is a joint behavior, sex is less likely to be affected by repeated questioning to a specific individual than a behavioral domain in which the outcome can be determined by that single individual's actions.

Contraceptive use is an example of a behavior that can be done alone, depending on the method used. For example an oral contraceptive pill can be taken without the sexual partner having any knowledge, whereas when condoms are used both partners must at least be aware. In an individual behavioral domain like oral contraceptive use, the behavioral consequences of frequent measurement are likely to be greater than in a joint behavioral domain like sex. Condom use is likely to fall in between these two, with more potential to be affected by measurement than sex but less than pill use. At least from the female perspective, condom use requires active participation from a partner, whereas use of female-only methods, such as oral contraceptive pills, does not.

These contrasts between individual and joint behaviors are important for the study of measurement effects on sexual and contraceptive behaviors. Each fall within a similar domain of intimate personal behaviors, but they vary in the degree to which other factors are likely to shape behavior. All three behaviors—pill use, condom use, and sex—are known to be shaped by many factors other than measurement (Martinez, Copen, and Abma, 2011; Mosher and Jones, 2010; Mosher et al., 2004). But among them, sex is highly dependent on the complex interplay of at least two people, condom use also can involve both people, and pill use can be completely determined by a single person. This contrast offers an important window into the conditions that may make repeated measurement more or less likely to affect behavior. Across all three of these behaviors we expect more frequent measures (more measures within a fixed period of time) and more total measures to increase behavioral response, with the strongest influence on pill use and the least influence on sexual behavior. Following the logic regarding contraceptive switching behavior outlined above, if frequent measurement does shape this kind of “switching” behavior, then we would expect to find switches toward contraceptive methods that can be used by individuals acting alone. That is, more switching toward contraceptives such as the oral contraceptive pill than toward contraceptives such as condoms.

3.1 Data

The Relationship Dynamics and Social Life (RDSL) study uses a mixed mode approach to survey research (Barber, Kusunoki, and Gatny, 2012). The study is based on a population-based sample of 1,003 young women between the ages of 18 and 19 (at baseline) from one county in Michigan. Although the age and geographic restrictions limit the generalizability of the sample, the county chosen for this study closely mimics the demographic and income distributions of the State of Michigan, placing the population near the median for the United States. The sampling frame used was the Michigan Department of State driver's license and Personal Identification Card (PID) data. This frame has high coverage of this age group, with 96% agreement between the frame count and Census-based population projections. The frame is updated every six months, and replicate samples were drawn quarterly, with recruitment taking place between March 2008 and March 2009. Eligible women were initially contacted via mail, with a letter that informed them of the upcoming baseline interview and included a $5 incentive to participate.

Sixty-minute face-to-face baseline interviews were conducted with each woman at the start of the study to gather information on her family background; education and career plans; attitudes, values, beliefs, and knowledge about sexual practices; romantic relationships; and sexual experiences. A total of 1,418 women were sampled from the database; of these women 218 were found to be ineligible. The baseline interview yielded a response rate of 83.6% (RR1; AAPOR, 2011), for a sample of 1,003. After the baseline interview, the women were each invited to participate in the weekly journal portion of the study. Over 99% of respondents who completed the baseline survey enrolled in the weekly surveys (n=992) (Barber et al., 2012). These weekly surveys lasted for the next 2.5 years, during which the women were asked to complete the surveys either online or by phone (92% chose online and 8% chose phone). Among the sample of 992 young women, 34% were Black, 14% had parents with an income of less than $15,000, and 63% had a 12^th grade education or less (Barber et al., 2013).

Significant effort was taken to keep these young women enrolled in the weekly journal-keeping study. Monetary incentives of $1 per weekly journal and a bonus of $5 for having completed five weekly journals on time were given, and small gifts—such as pens and lip balm—were also given to encourage retention (Gatny et al., 2009). Respondents who failed to complete the journal on time were first contacted by phone, then by email and letter. After 60 days of not completing a journal, increased incentives were offered for the next journal entry.² At the completion of the journal-keeping study, 84% of respondents who were interviewed at baseline had participated in journal-keeping for at least 6 months, 79% for at least 12 months, and 75% for at least 18 months (Barber et al., 2013).

To investigate the potential of this journal keeping to affect behavior, we added an experiment to the RDSL. For this experiment, 263 additional respondents were randomly selected from the same sampling frame, 200 of whom agreed to be interviewed with the same baseline interview (response rate of 76%). Of those 200 women interviewed for the experiment, 100 were randomly assigned to participate in the weekly journal for 12 months. After 12 months all 200 women interviewed in the baseline were contacted for a closeout interview. Over 90% participated in the closeout interview, yielding 94 women in the control group (no journal between baseline and closeout) and 92 women in the treatment group (weekly journals for 12 months between the baseline and close out). We use this experimental design to estimate the effect of journal keeping on key measures of romantic, sexual, and contraceptive use behavior.

4.1 Analysis Strategy

Our analysis of these two different data sources proceeds in three steps. First, we use the experimental data to perform t-tests of mean differences, allowing us to use the power of the random assignment to document whether participation in the weekly journal is associated with sexual behavior, contraceptive use, and pregnancy. Second, we use the large scale weekly journal-keeping data to estimate multivariate models of the hazard of contraceptive use methods that appeared most affected by journal keeping in the experiment. This strategy allows us to test for the effects of repeated measurement in more cases, with controls for a broad range of baseline characteristics, and measures of the dynamic nature of contraceptive use in these age groups. Third, based on results from the first two steps and using the same multivariate approach as in step two, we isolate contraceptive switching behavior, from condom use to oral contraceptive pill use, and investigate the potential for repeated measurement to accelerate this switch. All empirical estimates are calculated using the SAS statistical software.

5.1 Results

5.1.1 Behavioral Consequences of Random Assignment to Weekly Journal Keeping

The first step in our investigation is to use this embedded experiment to evaluate the gross effects of weekly journal keeping on measures of behavior. We begin by comparing all the behavioral measures from the treatment group (kept a journal) to all the behavioral measures in the control group (did not keep a journal). In total, there were 36 behavioral measures in the closeout survey administered to both groups that provided a large enough number of responses from each group to perform a means comparison test.^3,4 Of these only three measures (less than 10%) indicated significant gross differences (at p < .05) for those who received the weekly journals compared to those who did not receive the weekly journals.⁵ These three items were: (1) ever use of condom (lower for those with journal), (2) use of condom every time respondent had sex in past 12 months (lower for those with journal), and (3) ever use of withdrawal in past 12 months (higher for those with journal). Next we investigate these observed differences in more detail.

More detailed examination of the effects of journal keeping on sexual activity, contraceptive use among those who have ever been sexually active, and pregnancy experiences requires consideration of specific sub-samples who are at risk of each behavior. Key results from this analysis are displayed in Table 1. First, note that journal keeping appears to have no statistically significant relationship to whether or not a young woman is sexually active (Row 1, Table 1). Second, among the sexually active, those who completed the weekly journal have somewhat elevated levels of sex with multiple partners and oral contraceptive pill use, though these differences are not statistically significant (Rows 2 and 3, Table 1). Third, among the sexually active, those who completed the weekly journal have significantly lower levels of condom use (Rows 4 and 5, Table 1). The use of withdrawal also remains significantly different for those who kept a journal, compared to those who did not (Row 6 of Table 1). Finally, note that among those with at least one pregnancy, numbers of pregnancies is significantly higher in the group which received the weekly journal (among all sexually active women this difference is in the same direction but not statistically significant).

Table 1. Mean Differences in Treatment and Control Groups: Closeout Interview.

	Did not keep a Journal (Control)	Kept a Journal (treatment)	Differencea
Full Sample	n=94	n=92
Ever had sex	0.819	0.826	0.007
Sexually Active in Last 12 months (with specific partner)	n=56	n=60
Had sex with more than one personb	0.179	0.317	0.138
Pill Useb	0.4119	0.500	0.089
Condom useb	0.875	0.700	-0.175*
Condom use: Every time b	0.321	0.150	-0.171*
Withdrawal b	0.643	0.817	0.174*
Sexually Active in Last 12 months (with specific partner) and Has Had at least One Pregnancy	n=26	n=28
Number of pregnancies	1.308	1.857	0.550*

^aDifference in means, two-tailed tests

^*p < .05

^bNote that these measures refer to the past 12 months.

Our findings for condom use are somewhat surprising. The hypotheses of behavioral response to frequent measurement predict that exposure to repeated questions about condom use should increase the likelihood of using condoms (or have no effect), not decrease the likelihood of using condoms. Numerous rival mechanisms may be at work, such as measurement error, or behavioral predictors not included in the model. For example, it is possible that repeated measurement reduces the potential exaggerating error from a social desirability effect—by being asked frequently about condom use perhaps young women become more accustomed to reporting accurately and less likely to exaggerate their use of condoms. But the simplicity of this random assignment experiment eliminates many such potential explanations. Because we did not find significant differences for other sensitive behaviors measured, even such a social desirability effect must be constrained to a specific item and circumstance. We do not find evidence of a general change in social desirability or other mechanisms shaping the accuracy of recall for journal participants versus nonparticipants.

Because the empirical relationship between frequent measurement and contraceptive use offers special insight into the mechanisms generating behavioral response to measurement, as we reasoned above, we prioritize further investigation of the effects on condom use. One crucial potential explanation comes from the study of contraceptive use behavior, rather than the study of behavioral response to measurement. From decades of research on contraceptive use, which has focused attention on the initiation of contraceptive use, we know that young people often initiate use of one method and subsequently switch to the use of a different method (Frost, Singh, and Finer, 2007; Grady, Billy, and Klepinger, 2002; Moreau, Cleland, and Trussell, 2007). This is particularly rational when sexual activity first begins, especially outside of marriage. When sex is rare, coitally specific, temporary methods have important advantages. Condoms are a clear example. Condoms are easy to obtain, require little advanced preparation to use, and have no long-term side effects (Hatcher et al., 1995; Trussell and Raymond, 2007). When sex is rare, as is usually the case when young women first begin sexual relationships, condoms are often the contraceptive method of choice (Abma, Martinez, and Copen, 2010; Laumann et al., 1994). However, as sexual relationships intensify and sex becomes more frequent, young people often switch to, or sometimes add, coitally independent methods such as oral contraceptive pills (Grady, Billy, and Klepinger, 2002; Moreau, Cleland, and Trussell, 2007; Trussell and Vaughan, 1999). Such methods require more effort and advanced planning to obtain, and may be characterized by side effects even when not sexually active, but when sex is frequent they have the advantage of being independent of sexual activity so there is no additional effort each time a person has sex (Hatcher et al., 2007; Ryder and Westoff, 1971; Westoff and Ryder, 1977). Within the specific context of a sexually active relationship it is possible that the “mere exposure” resulting from frequent journal questions about use of specific methods independently promotes contraceptive switching, from condoms to other methods such as oral contraceptive pills.

5.1.2 Behavioral Consequences of Long Term Weekly Journal Keeping

To investigate the potential of journal-keeping effects on contraceptive switching behavior, we turn from the 12-month weekly journal experiment to analyses of the 30-month weekly journal observational RDSL study. Though the observational design introduces the potential for rival explanations because journal keeping is not randomly assigned, it has the advantage of providing observations from many more respondents (n=953 rather than n=186).⁶ This advantage is crucial for studying contraceptive switching because the investigation requires both a large number who are sexually active and using one method, and that some of them switch from that first method to a second method. The lack of random assignment of journal keeping creates the possibility of unobserved correlations between enrollment in the journal keeping and predictors of contraceptive use. However, the RDSL study has the advantage that respondents are chosen systematically from a population of young women and almost all are enrolled in the journal (i.e., not a self-selected sub-sample). Furthermore, the study provides measures of known predictors of contraceptive use at baseline.

The Hazard of Contraceptive Use in the Journal

The weekly journal measures from RDSL provide the means to operationalize the hazards of first pill use and the hazards of first condom use during the journal among young adult women. Because the measures are collected through a journal with discrete-time daily measurement, we have chosen to operationalize these hazards in discrete time using person-days as the unit of analysis.⁷ The discrete time approach yields results similar to a continuous approach because the incidence of pill or condom use in any one day is quite low, but the discrete time approach allows us to avoid making any parametric assumptions regarding the distribution of the underlying baseline hazard (Yamaguchi, 1991). We use data from the first 365 days that respondents were enrolled in the weekly journals, to match the time period for the experimental data. To estimate the hazards of pill and condom use, we use the sample of respondents who completed at least one journal during the first 365 days of the weekly data collection. This produces a sample of 953 respondents who generate a total of 165,523 person-days of exposure to pill use who have no missing data and are included in our event history analysis of pill use. Likewise the sample generates 153,915 person-days which have no missing data and are included in our event history analysis of condom use. Because our focus is on these effects of journal keeping on subsequent contraceptive use, the hazard is conceptualized as first use of pills or condoms after the first journal is completed, even if respondents used these methods before journal keeping began. Of this sample of women, 475 use pills at some time during the year of journal keeping and 571 use condoms.

Our main objective is to evaluate the relationship between the number of journals a young woman has completed and her subsequent likelihood of using a specific contraceptive method as the number of journals grows over time. This objective focuses our attention on a measure of the total number of journals completed that is operationalized as a time-varying covariate, so that each new person-day of risk of contracepting has an updated tally of previous journals completed, a tally that grows as weeks pass. We construct two extreme models to estimate this relationship—one with virtually no other controls for factors predicting contraceptive use and the other with controls for all likely predictors of contraceptive use measured in RDSL—to establish the limits of the likely true estimate of the effects of journal keeping on contraception in this observational design. Our minimum controls model includes measures of the number of weekly journals completed, respondents’ age at baseline, and time (the time-varying number of days into the study). Our maximum controls model adds measures of a broad range of demographic and family factors that are likely to influence contraceptive use: race, school enrollment, receipt of public assistance, religious importance, romantic cohabitation, mother's age at first birth, family structure, mother's education, parental income, age at first sex, number of sexual partners, ever had sex without birth control, and number of prior pregnancies (Kusunoki, 2010). These domains were selected based on the extensive existing literature on factors affecting the hazards of pill and condom use in the United States (Mosher and Jones, 2010; Martinez, Copen, and Abma, 2011; Mosher et al., 2004; Kusunoki and Upchurch, 2011) and the specific measures were derived directly from substantive research focused on contraceptive use using these same data (Kusunoki, 2010).

The results of this exercise are presented in Table 2 below. First and foremost, the effects of journal keeping on contraceptive use do not reach statistical significance (p>.05) in any of the models. Though this result is not consistent with behavioral theories of response to frequent measurement, from the perspective of substantive research it is something of a relief to discover that the frequent measurement does not appear to significantly alter behavior. Second, time since the baseline has a statistically significant hyperbolic relationship to the hazards of both pill and condom use in these age groups. The shape is flat in the beginning of the interval and rises rapidly as respondents move through the year—entirely consistent with what is known about rapidly rising rates of contraceptive use among US teen women at the end of their teenage years (Glei, 1999).⁸ Third, the estimates of the effects of other known individual and family predictors of contraceptive use influence the hazards of pill and condom use as expected (Kusunoki and Upchurch, 2011).

Table 2. Odds Ratio Estimates of the relationship between Journals Completed and the hazard of Oral Contraceptive Pill Use and the hazard of Condom Use (discrete time), 365 days.

	Pill Use, Minimum Controls a	Pill Use, Maximum Controls a	Condom Use, Minimum Controls a	Condom Use, Maximum Controls a
Journals completed (time-varying covariate)	0.996	0.989	0.992	1.008
Time, days converted to months	0.184***	0.193***	0.368***	0.359***
Time, days converted to months, squared	1.130***	1.128***	1.075***	1.074***
Sociodemographic Characteristics
Age (ref: 18 years)
19 years	0.859	0.843+	0.926	0.871
20 years	0.865	0.841	0.850	0.807
African American		0.726*		1.098
School enrollment and type (ref: 4 year college)
Not enrolled and did not Graduate		0.483**		0.725+
Not enrolled and did graduate		0.829		0.906
High school		0.876		0.882
2 year college/vocational/technical/other		1.424**		0.906
Receiving public assistance		0.888		1.036
Religious importance		0.864**		0.955
Living with romantic partner		1.253+		1.023
Biological mother <20 years old at first birth		0.900		1.023
Family structure (ref: Two parents)
Single biological parent only		0.797*		1.122
Other		1.106		0.980
Mother's education <high school graduate		0.950		0.900
Parental income (ref: $15,000 or more)
$14,999 or less		0.628**		0.914
Don't know/refused		0.767*		1.018
Sexual, Contraceptive, and Pregnancy Experiences
Age at first sex (ref: 17 years or greater/never had sex)
14 years or less		1.246		0.943
15-16 years		1.498**		1.079
Lifetime number of sexual partners		1.907***		2.446***
Ever had sex without birth control		0.773*		1.114
Prior pregnancies (ref: None)
One		0.721*		0.926
Two or more		0.856		0.881
Person-days	165523	165523	153915	153915
Total persons in sample	953	953	953	953
Persons adopting method	475	475	571	571

^aTwo –tailed tests

⁺p < 0.10,

^*p < .05,

^**p < .01,

^***p < .001

Journal Keeping and Contraceptive Switching

To examine the influence of journal keeping on contraceptive method switching, we build directly on the model of hazard of first pill use presented above (Table 2). We add a time-varying measure of condom use to the model of pill use. This allows us to see how experience using condoms shifts the hazard of first pill use, and more importantly, it allows us to interact the time-varying number of journals completed with time-varying condom use. This interaction provides an estimate of the effects of journal keeping conditional on previous use of condoms; if respondents who are sexually active and using condoms switch to using contraceptive pills after repeated journal keeping, this interaction should be statistically significant and should drive the rate of first pill use upward.⁹

In Models 3 and 4 of Table 3, the first row demonstrates that the interaction of condom use and journal keeping does in fact have a statistically significant positive effect on pill use, increasing the hazard of first pill use. For those young women who are using condoms for contraception, the more journals they complete, the more likely they are to switch to using oral contraceptive pills. This strong, significant effect is robust against other known predictors of pill use in the model; our estimate changes very little when we add controls for these factors (compare coefficients in Row 1 of Model 3 to those in Row 1 of Model 4). Though observational in nature, this evidence is consistent with the conclusion that repeated journal measurement can increase oral contraceptive pill use among sexually active individuals who are using condoms.¹⁰

Table 3. Odds Ratio Estimates of the relationship between Journals Completed and the hazard of Oral Contraceptive Pill Use, controlling for time-varying Condom Use, 365 days.

	Pill Use	Pill Use	Pill Use	Pill Use
	Model 1 a	Model 2 a	Model 3 a	Model 4 a
Journals completed (time-varying covariate)*Condom use (time-varying covariate)			1.028*	1.029*
Journals completed (time-varying covariate)	0.998	0.989	0.984	0.973+
Condom use (time-varying covariate)	1.617***	1.575***	1.41**	1.372**
Time, days converted to months	0.177***	0.186***	0.181***	0.190***
Time, days converted to months, squared	1.133***	1.131***	1.130***	1.129***
Sociodemographic Characteristics
Age (ref: 18 years)
19 years	0.848+	0.834+	0.848+	0.834+
20 years	0.885	0.863	0.889	0.867
African American		0.708**		0.706**
School enrollment and type (ref: 4 year college)
Not enrolled and did not graduate		0.483**		0.489**
Not enrolled and did graduate		0.838		0.840
High school		0.896		0.899
2 year college/vocational/technical/other		1.459**		1.465**
Receiving public assistance		0.930		0.934
Religious importance		0.861**		0.862**
Living with romantic partner		1.270+		1.273+
Biological mother <20 years old at first birth		0.891		0.882
Family structure (ref: Two parents)
Single biological parent only		0.798*		0.798*
Other		1.116		1.117
Mother's education <high school graduate		0.964		0.957
Parental income (ref: $15,000 or more)
$14,999 or less		0.614**		0.611**
Don't know/refused		0.763*		0.760*
Sexual, Contraceptive, and Pregnancy Experiences
Age at first sex (ref: 17 years or greater/never had sex)
14 years or less		0.614		1.277
15-16 years		1.484**		1.479**
Lifetime number of sexual partners		1.736***		1.729***
Ever had sex without birth control		0.745*		0.738**
Prior pregnancies (ref: None)
One		0.695*		0.694*
Two or more		0.861		0.865
Person-days	165523	165523	165523	165523
Total persons in sample	953	953	953	953
Persons adopting method	475	475	475	475

^aTwo –tailed tests

⁺p < 0.10,

^*p < .05,

^**p < .01,

^***p < .001

To see the substantive results of this interaction more clearly, we translate the estimates into a graphical presentation in Figure 1. Here we calculate predicted values for the hazards of pill use at varying levels of condom use and journal keeping exposure. Condom use is either 1 (for using) or 0 (for not using), represented as two different lines on the graph in Figure 1. The number of journals completed forms the x-axis of the graph and the predicted hazard of pill use forms the y-axis of the graph. The slight downward slope on the line for not using condoms means that for those women the hazard of pill use falls slightly as they complete more journals. This slope is in strong contrast to the steep upward slope of the line for those women who have used condoms. Among condom users the hazard of pill use grows rapidly as they complete more and more journals. The weekly questions about their contraceptive use appear to serve both to cognitively engage them in consideration of contraceptive method choice when they are not having sex and to remind them of the set of choices they face. Thus based on the results described by Figure 1, it appears that many sexually active women who are currently using condoms to contracept become pill users as the number of times they are asked grows.

Figure 1. Predicted Values of Interaction between Journals Completed and Condom Use, with Pill Use as Outcome, Holding Time (converted to months) at Mean, 365 days ^a.

^a Figure based on a model in which no other controls were included, aside from journals completed, condom use, and time (converted to months).

However, these results are not as simple as they appear. First, contraceptive “switching” may not be the correct concept at all, as oral contraceptive pills and condoms are frequently used together. Oral contraceptive pills are coitally independent, have side effects that some women want (controlled menstruation), and can be extremely effective at preventing pregnancy. Condoms, on the other hand, can be used to prevent transmission of sexually transmitted diseases, and are often used even when couples are also using pills. In the ages 17 – 22, condom use is also a strong predictor of pill use in the US national population (NSFG Cycle 6), but much of this is young women choosing to use both condoms and pills. In the RDSL study a full 49% of the women reporting any condom use in the last 7 days also report using oral contraceptive pills.

Second, and even more important, because age has a large influence on the hazard of pill use among young women, and age is rising as the number of journals completed is growing, we cannot demonstrate that the effect of journal keeping is independent of the effect of age. US national data for women aged 17 – 22 also demonstrates that age has a strong positive influence on using oral contraceptive pills (NSFG Cycle 6). In our regional sample, we re-estimated the models testing for journaling effects on the hazard of pill use, now adding a term for the interaction between time since the baseline interview (age in these models) and condom use. The result is similar to the results for the interaction between journals completed and previous condom use (not shown in tables). These two estimated effects are not just similar, they are related, and they are not independent. Both remain positive—both the numbers of journals completed and the passing of time increase the hazard of pill use among those who had used condoms. But neither of these effects is statistically significant when they are both included in the same model (not shown in tables). Completing more journals may affect the pill use behavior of young women who had used condoms, but so does the simple passage of time as they go through life completing these journals, such that we cannot observe any independent effect of journal keeping on contraceptive use behavior.

6.1 Conclusion

Overall, the results of this investigation provide some encouraging news for social scientists interested in using electronic journal keeping methods to collect frequent measures of human behavior. Using an experiment with random assignment of young women to journal keeping, we find that frequent measurement through weekly journal keeping rarely appears to affect the behaviors we study. Our analysis is limited to the substantive domains of relationships, sexual behavior, contraceptive use and pregnancy—domains in which there is theoretical reason to expect consequences of frequent measures. This overall finding is an important step toward more widespread use of these new technologies to gather data. The scientific demand for more measurement from the same people is unabated; tools of this type provide a practical means for collecting such measures.

However, we do find some evidence of behavioral change associated with frequent journal keeping. Particularly interesting, the effects of journal keeping on condom use are statistically significant, but with participation in weekly journals producing less condom use: the opposite of our prediction. We use observational panel study data with many more cases to show this odd result may be produced by condom users switching to oral contraceptive use at high rates as they complete more and more journals. In these panel data, frequent measurement does appear to affect pill use exactly as predicted, but mainly among the sexually active who have used condoms. Frequent journal keeping measurement is associated with high rates of initiating oral contraceptive pill use among 18 and 19 year old women who have used condoms. It may be that the repeated measurement serves as a reminder of the different contraceptive choices available to these young women. Over time, young women may recognize that pills offer greater independent control over their pregnancy risk. Their choice to switch to this method may be a conscious, rather than nonconscious, one because of the recognition of the benefits of oral contraceptive pills (Fitzsimons and Moore, 2008).

Complicating matters, among this age group the mere passage of time has a similar influence on contraceptive choice, with more young women who have used condoms initiating pill use as time passes. Because time must pass in order to complete more journals, these two influences on pill use are not independent, and the effect of journal keeping is not statistically significant once the differential effects of time among condom users versus non-users are added to our models. Even if frequent measurement does influence behavior, the magnitude of this effect is quite mild relative to other known factors. Moreover, the effect of frequent measurement is neither of greater magnitude or independent of the effect of the passage time itself.

Substantively, these rich longitudinal measures of relationships, sex, and contraceptive use raise many important new opportunities to investigate the micro-dynamics of the timing and sequencing of events in young women's lives (Barber et al., 2012). These investigations are already underway and yielding important new information regarding these processes (Barber et al., 2013; Kusunoki, 2010; Moreau et al., 2013). The results presented here point toward contraceptive switching and simultaneous use of multiple contraceptive methods as particularly interesting topics. Investigations of these topics using journal keeping measures that feature frequent interviews must confront the possibility of measurement effects, but our results are consistent with the prediction that such measurement effects are likely to be minimal.

Within the substantive domain of relationships, sex, and contraception, the vast majority of behavioral measures are largely unaffected by the frequent measurement. But among the sexually active already using contraception, frequent questions about contraceptive use appear to stimulate a shift toward more use of coitally independent oral contraceptive pills. Crucial to our understanding of behavioral response to frequent measurement, this is also a behavioral shift (across contraceptive methods) among those already engaging in the behavior in general (contraceptive use). Frequent measurement appears to have no effect on sexual activity and little influence on initiating contraception. Instead, among those already using contraception, it may lead them to a change of methods. An analogy might be changing from cigarettes to cigars among smokers, or perhaps brands of cigarettes among cigarette smokers. It is likely that switching specific behavioral choices within a behavioral domain responds more to frequent measurement than initiation of new behaviors. However, in the study we report even this effect is not independent of the effects of the mere passage of time itself, indicating minimal risk of distortion of these behaviors from frequent measurement.

Of course, other behavioral domains could respond differently to frequent measurement. Evidence from the study of consumers demonstrates that measurements of intention do appear to shape purchase behavior, at least with respect to the purchase of automobiles (Fitzsimons and Morwitz, 1996). The purchase of automobiles is a particularly interesting case because these may frequently reflect joint decisions of multiple adults. It is possible that repeated measurement in some behavioral domains that require two or more people to act together does influence behavior. More research across multiple different behavioral domains is needed to differentiate joint-decision circumstances from individual decision circumstances, including possible differences in the extent to which influences persist over time (Moore et al., 2012).

Our results from the study of contraceptive use constitute an interesting finding that may help scientists target their investigations of behavioral response to frequent measurement. Frequent measurement is not likely to lead people to engage in an entirely new behavior, but may be more likely to lead people to change an existing behavior from one form to another form. It may be that this type of “switching” behavior represents a lower threshold for behavioral changes that is more susceptible to “mere exposure” types of effect similar to those documented for attitudinal change (Barber, 2004; Mead, 1967 [1934]; Zajonc, 1968, 2001). We can expect the greatest change toward behaviors that can be accomplished by the individual being measured acting completely alone. The study we report here advances what is known about the most likely behavioral responses to frequent measurement, helping to target future research on these questions.

What do these results mean for the future of electronic journal keeping measurement, or similar forms of frequent measurement from people? In the substantive domains we investigate, we find that frequent measurement does not affect behaviors related to getting into or out of relationships or the behaviors within relationships, including sex. A sample representing a different age range or men might be expected to produce different results. However, among this sample of young women, for whom these behaviors are especially relevant there is little overall influence on contraceptive use, with frequent measurement potentially creating mild contraceptive method switching. Though this is an interesting and important result, the magnitude of this effect is neither large nor statistically independent of the passage of time. Any behavioral consequence of frequent measurement in this domain at this mild level is unlikely to influence the overall outcomes of contraceptive use, such as sexually transmitted diseases and pregnancies. In domains not represented in this study but with less theoretical reason to expect effects of frequent measurement, it appears unlikely that weekly measurement will alter the behaviors being measured. Overall we find little evidence for behavioral response to frequent measurement, and the response we find gives us much better information regarding the situations in which we should expect to find a response.

Weekly electronic journal keeping can be effectively used to measure the dynamics of relationships, sex, and pregnancy. Even in the domain of contraceptive use, we expect little overall effect and caution data users to control for the cumulative number of measures and the passage of time when analyzing these sources of evidence. As frequent electronic journal keeping is expanded to a wider range of substantive domains, the vast majority of topics measured are unlikely to be affected by frequent measurement. Data users should devote care to analyses using such data to investigate choices among different types of a specific behavior among those already engaging in the behavior, but otherwise these results should give social scientists confidence that they can use electronic journal keeping to collect more frequent measures effectively in the domain of sensitive behaviors and possibly across a broad array of behaviors.

Highlights.

• Frequent measurement via weekly journal keeping rarely affects behavior.

• Frequent measurement is most likely to affect behavior choices of a single person.

• More young women who have used condoms initiate pill use as time passes.

• Electronic journaling is viable for frequent measurement.

Appendix

Survey Items Included in Means Tests:

General Attitudes/Beliefs

If a woman asks her partner to use a condom, he will think that she doesn't trust him.(Response options: Strongly Agree, Agree, Neither Agree nor Disagree [if R insists], Disagree, Strongly Disagree)

Young people should not have sex before marriage. (Response options: Strongly Agree, Agree, Neither Agree nor Disagree [if R insists], Disagree, Strongly Disagree)

Being a mother and raising children is the most fulfilling experience a woman can have. (Response options: Strongly Agree, Agree, Neither Agree nor Disagree [if R insists], Disagree, Strongly Disagree)

Relationships between men and women can improve after they have a baby together. (Response option Strongly Agree, Agree, Neither Agree nor Disagree [if R insists], Disagree, Strongly Disagree)

If a woman waits for the perfect time to have a baby, she will probably have trouble getting pregnant. (Response options: Strongly Agree, Agree, Neither Agree nor Disagree [if R insists], Disagree, Strongly, Disagree)

It is alright for a woman to have a child without being married. (Response options: Strongly Agree, Agree, Neither Agree nor Disagree [if R insists], Disagree, Strongly Disagree)

It is alright for a couple to live together without planning to get married. (Response options: Strongly Agree, Agree, Neither Agree nor Disagree [if R insists], Disagree, Strongly Disagree)

Children cause worry and emotional strain for their parents. (Response options: Strongly Agree, Agree, Neither Agree nor Disagree [if R insists], Disagree, Strongly Disagree)

What do you think is the ideal number of children for the average American family?

Individual Preferences for Family Related Behavior

Getting pregnant at this time in your life is one of the worst things that could happen to you. (Response options: Strongly Agree, Agree, Neither Agree nor Disagree [if R insists], Disagree, Strongly Disagree)

Suppose that your life turned out so that you never married, how much would that bother you? Please give me a number from 0 to 5, where 0 means not at all and 5 means extremely.

Suppose your life turned out that you never had children, how much would that bother you?

Coombs Scale A: The number of children people expect and want are not always the same. If you could have just the number you want, what number of children would you want to have when your family is completed? (Responses ranged from 0 to 6)

Coombs Scale B: Now, I would like to know how you feel about some other possible family sizes. If you could not have [Coombs Scale A], would you want to have [Coombs Scale A - 1] or [Coombs Scale A + 1] children? (Coded so that 1= Coombs Scale A – 1, and 0= Coombs Scale A + 1)

Coombs Scale C: And if you did not have [Coombs Scale B], would you rather have [Coombs Scale B -X] or [Coombs Scale B + Y] children? (Coded so that 1= Coombs Scale B – X, and 0= Coombs Scale B + Y, where: IF Coombs Scale B = 1, THEN X = 1; IF Coombs Scale B = 5, THEN X = 2; IF Coombs Scale B = 1, THEN Y = 2; IF Coombs Scale B = 5, THEN Y = 1)

Coombs Scale items A through C were recoded into a single item, as follows:

Sometimes what people want and what they expect are different because they are not able to do what they want. How many children do you expect to have?

Many people do not get as much education as they would like. How far do you think you will actually go in school?

If necessary: Do you think you will graduate from high school, graduate from a two year community college, earn a specialized certificate from a vocational or trade school, attend a 4-year college, graduate from a 4-year college, get more than 4 years of college, or do something else?

Life History

Have you ever had sexual intercourse? Sexual intercourse is when a man inserts his penis into a woman's vagina. (Response options: Yes, No)

How old were you the first time you had sexual intercourse?

With how many total partners have you had sexual intercourse?

During the last 12 months, that is, since [M/Y], how many men, if any, have you had sexual intercourse with? Please count every male sexual partner, even those you had sex with only once.

(If R is unable or unwilling to give an exact number, interviewer says: “If you would prefer, you can give me a range.” ENTER DK or RF as appropriate, and enter range in the next two fields.)

Do you think there might be a chance that you are pregnant right now? (Response options: Yes, No)

Please think of all the times you have been pregnant, whether you are currently pregnant or the pregnancy ended in live birth, miscarriage, stillbirth, abortion, or ectopic pregnancy.

1. How many times have you been pregnant in your life?

2. [Number of pregnancies minus current pregnancy.]

In which way did the [1^st] pregnancy end? (Response options are coded into a series of dummy measures: Miscarriage, Stillbirth, Abortion, Ectopic or Tubal Pregnancy, Live Birth by Cesarean Section, or Live Birth by Vaginal Delivery. Three percent or more of the total sample of 186 women experienced the following [and, thus, these items were included in our gross comparisons analysis]: Miscarriage, Abortion, Live Birth by Cesarean Section, or Live Birth by Vaginal Delivery.)

Right before you became pregnant with your [1^st] pregnancy, did you yourself want to have [a/another] baby at any time in the future? (Response options: Yes, No)

So would you say you became pregnant too soon, at about the right time, or later than you wanted? (Response options: Too soon, Right time, Later, Didn't Care)

In which way did the [2^nd] pregnancy end? (Response options are coded into a series of dummy measures:

Miscarriage, Stillbirth, Abortion, Ectopic or Tubal Pregnancy, Live Birth by Cesarean Section, or Live Birth by Vaginal Delivery. Three percent or more of the total sample of 186 women experienced the following [and, thus, these items were included in our gross comparisons analysis]: Miscarriage or Live Birth by Vaginal Delivery.)

Right before you became pregnant with your [2^nd] pregnancy, did you yourself want to have [a/another] baby at any time in the future? (Response options: Yes, No)

Have you ever used anything that can help people avoid becoming pregnant? (Response options: Yes, No)

Are you currently using anything that can help people avoid becoming pregnant, even if you are not using it to keep from getting pregnant yourself? (Response options: Yes, No)

In the past 12 months, (since [M/Y]), did you ever take birth control pills for any reason? (Response options: Yes, No)

In the past 12 months, (since [M/Y]), did you ever use a Depo-Provera shot or any other type of contraceptive shot? (Response options: Yes, No)

Are you currently married? (Response options: Yes, No)

Have you ever been married? (Response options: Yes, No)

Current Relationship

Are you currently engaged to be married [to someone else]? (Response options: Yes, No)

Are you currently in a special romantic relationship with anyone? (Response options: Yes, No)

Are you currently in any type of relationship that involves physical or emotional contact, such as kissing, dating, spending time together, sex, or other activities with a partner? (Response options: Yes, No)

Have you talked with [Name] about birth control? (Response options: Yes, No)

In the past 12 months, since [M/Y], did you ever have sexual intercourse with [Name]? By sexual intercourse, we mean when a man puts his penis into a woman's vagina. (Response options: Yes, No)

In the past 12 months, (since [M/Y]), did you ever have sexual intercourse with anyone other than [Name]? (Response options: Yes, No)

In the past 12 months, (since [M/Y]), did you or your [partner/partners] use some method of birth control every time you had intercourse (even if you are not trying to prevent pregnancy)? This could be a method you mentioned earlier, or a method you haven't mentioned such as condoms, pills, or another method. (Response options: Yes, No)

The time or times that you did not use a method of birth control, did you make the decision, did your partner make the decision, or both? (Response options: You, Partner, Both)

In the past 12 months, since [M/Y], did you ever use a condom? (Response options: Yes, No)

Did you use a condom every time you had intercourse? (Response options: Yes, No)

In the past 12 months, (since [M/Y]), did you ever use spermicide? (Response options: Yes, No)

In the past 12 months, since [M/Y], did you ever use the morning after pill? (Response options: Yes, No)

In the past 12 months, (since [M/Y]), did your partner ever withdraw before ejaculating? (Response options: Yes, No)

In the past 12 months, (since [M/Y]), did you ever do anything else to avoid becoming pregnant that you haven't mentioned today? (Response options: Yes, No)

Knowledge/Information about Pregnancy and Birth Control

Most women's periods are regular, that is, they ovulate or are fertile fourteen days after their periods begin. (Response options: True, False)

When putting on a condom, it is important to have it fit tightly, leaving no space at the tip. (Response options: True, False)

The most likely time for a woman to get pregnant is right before her period starts. (Response options: True, False)

Even if the man pulls out before he ejaculates, even if ejaculation occurs outside of the woman's body, it is still possible for the woman to become pregnant. (Response options: True, False)