The Stability of Self-Reported Marijuana Use Across Eight Years of the National Longitudinal Survey of Youth

Abstract

Introduction

This study examined teen marijuana report stability over eight years. The stability of self-reports refers to the consistency of self-reported use across several years.

Method

This study used fives waves of data across eight years from the National Longitudinal Survey of Youth. Analyses were conducted to examine the internal or within wave consistency as well as external or across waves consistency for self-reported marijuana use. Further tests were conducted to identify if there were any differences for age, ethnicity and sex for report consistency.

Results

Report stability was higher for lifetime use reports than the age of onset reports. Wave-by-wave differences revealed stability remained at acceptable levels in nearly all comparisons at agreement being about 75%. Overall, report agreement was higher for females, older adolescents, and Non-Hispanic/Non-Black youth in bivariate analyses. However, only older chronological age remained consistently significantly associated with better report stability in multiple logistic regression models. Implications regarding misclassification of users for prevention programs and measurement issues are discussed.

INTRODUCTION

The recent epidemiological data suggests the illicit use of marijuana among adolescents and young adults is fairly common, with 42.6% of high-school seniors reporting lifetime use and 32.4 % reporting past year use (Johnston et al., 2008). Based on these reports, marijuana would be the third most commonly used psychoactive substance behind only alcohol and tobacco respectively Johnston et al., 2008). Although such epidemiological data provide insight into drug use trends, they are based on self-reports which are subject to potential reporting errors—both intentional and not.

A recent editorial in Addiction noted there is “an undeniable problem” among survey respondents in longitudinal studies “recanting” accounts of previous substance use (Fendrich, 2005, p.143). Research examining sources of error related to such self-reports is important to the field's ability to estimate epidemiology, model etiology, and evaluate prevention initiatives (Shillington et al., in-press; Matt et al., 2003; Culverhouse, 2003). Current evidence examining the stability of recall suggests that numerous factors including age of the respondent (Johnson and Mott, 2001), race/ethnicity (Shillington and Clapp, 2000), SES and occupation (Johnston and O'Malley, 1997), and substance type (Frendrich and Vaughn, 1994; Golub et al., 2000) are all associated with report stability of substance use.

Demographically, younger respondents are less stable in their reports across survey waves than older respondents (Johnson and Mott, 2001); however, the longer the duration between age of first use of a substance and the most recent report of the same substance is often a strong predictor of inaccurate reports (Johnson, Gerstein and Tasinski, 1997; Golub, Johnson and Labouvie, 2000).

In addition to age related variables, racial and ethnic variables also predict substance use report instability. African Americans have been found to have lower levels of report stability than their counterparts (Johnson, Mott, 2001; Johnson and O'Malley, 1997; Fendrich and Rosenbaum, 2003). Hispanic respondents have also been found to have lower substance use report stability than non-Hispanics and to be the ethnic group most likely to recant use (Frendrich and Vaughn, 1994).

Similar to the above studies, in a study examining data from the National Longitudinal Survey on Youth (NLSY), Fendrich and Kim (2001) found that ethnicity was the very best predictor of recanting substance use. Other factors, however, also contributed to recanting previously reported substance use including socio-economic status and marital status (married people were more likely to recant). Additionally, Johnson and O'Malley (1997) found occupation to be related to recanting past substance use, with police, fire fighters and military personnel being more likely to recant than respondents working in other occupations.

Several studies have also found that report stability is also a function of the substance being reported on. Reports of marijuana use have been found to be more stable than reports of cocaine use (Johnson and Mott, 2001), and almost all cocaine users have been found to recant their previously reported use of the drug at some point (Fendrich and Rosenbaum, 2003). However, Fendrich and Vaughn (1994) reported that under-reporting of past marijuana use was very common.

This study builds on the current literature by examining report stability for marijuana across eight years.

METHODS

Sample

The NLSY has been conducted in the U.S. since 1979 and uses a multistage stratified random sampling technique. It had an original sample of 5,828 females and 5,578 males, who aged 14-21 when they were first interviewed with an over-sampling of Blacks and Hispanics. These same participants have been interviewed annually with a retention rate at twelve-year follow-up of 90.5% (Baker et al., 1993). In 1986 the study protocol was changed to also include the children of the original female respondents. These children, however, were born to young mothers and therefore are not a nationally representative sample of children.

The surveys for the NLSY children changed during the first few waves of data collection. Initially, a self-report instrument entitled the Child Self Administered Supplement (CSAS) was utilized with the children when they reached 10 years of age and older. The children have been queried every two years. This instrument queried areas such as child-parent relationships, attitudes, religiosity, deviance, and substance use. Beginning in 1994 a new instrument was added to the protocol for those who were aged 15 years and older. The new survey was entitled the Young Adult Survey (YAS), and the children aged 10-14 still completed the CSAS. The YAS asked about cigarette use, alcohol and other substance use, sexual activity, delinquent activities and relationships. The sample for this study consists of youth aged 10 and older beginning with those interviewed from 1990 to 1998.

Measures

The first wave of data collection (Time 1) consisted of responses to marijuana use questions in the 1990, 1992, 1994, and 1996 surveys. Those who were 10-14 years in age responded to the CSAS while those who were 15 and older completed the YAS. Across the surveys, many children aged out of the CSAS and started responding to the YAS. Although the responses to the marijuana use questions were sometimes from the two different surveys the questions that queried marijuana use are nearly the same within each wave. For most waves of data a respondent would answer a question asking if they ever used marijuana (no or yes). However there were 1-2 waves of data collection (depending on whether it was the CSAS or YAS) during which the youth was asked to respond to a question regarding the number of times they'd used marijuana. If a youth reported “0” for the number of times of marijuana use, they were coded as “no” for lifetime use and if he/she reported 1+ the code was a “1” for “yes” lifetime use.

For this study we used two variables to examine external consistency--agreement and discrepancy--for lifetime use of marijuana for each wave-to-wave comparison. We also examined incident cases, that is those who reported that they never used marijuana at the first wave of data collection or Time 1 but then did report use at Time 2 or the survey that followed two years later. Because the two sources of data are from the adolescents’ self-reports it is not possible to validate if new users from Time 1 to Time 2 are true incident cases. Thus incident cases are presented separately and not considered a type of report agreement.

We excluded adolescents who reported that they “never used” marijuana at both Time1 and Time 2 for two reasons. First, prior research has shown that youth who report no use for each substance category are significantly younger than ever users (Shillington and Clapp, 2000). Second, the inclusion of never users in an analysis of report stability artificially decreases discrepancy rates (Bailey, et al., 1992).

External consistency

Users were coded as either “consistent” or “discrepant” for marijuana use for each wave of data to the next among those who reported marijuana use at a prior wave of data collection. Adolescents who reported marijuana use at both Time 1 and Time 2 were coded as consistent reporters. For example if a respondent reported lifetime use of marijuana at Time 1 and then reported such use again two years later they were categorized as being in agreement or consistent in their reports for marijuana use.

If an adolescent marijuana use at Time 1 but then two years later reported ‘never used’ marijuana they were coded as discrepant reporters. For example, if an adolescent reported lifetime marijuana use during the 1992 survey but in 1994 responded that they never smoked marijuana, then they met criteria for being discrepant in their reports. A variable was created for consistent use for five interviews across 4 timeframes 1990-92, 1992-94, 1994-96 and 1996-98.

The reported age of onset for marijuana was another aspect of report stability examined across the multiple waves of data collection. A variable was created to calculate the difference between the age at onset reported at Time 2 and the age at onset reported at Time 1. This calculation could result in a “0” which would indicate that the age of onset was the same at both times. However, a positive value such as “+2” for example would indicate that a youth reported their age of onset two years older at Time 2 compared to Time 1. A negative value would be for those reporting a younger age of onset from Time 2 compared to Time 1.

Internal consistency

To examine internal consistency of reports (consistency based on the logical skip pattern within a specific survey), responses to two variables at each wave of data collection were compared. If the respondent reported lifetime use of marijuana they were then asked to report how recently they used marijuana. If they reported use within the past 30 days they were then asked to report the frequency of their past 30 day use. Internally consistent reports were defined as responses of 30-day use and then reports of using marijuana 1+ times during the past 30 days. Inconsistent reports were defined as those respondents who reported past 30-day use but then in response to the frequency of past 30 day use, they reported that they had “never used marijuana in the last 30 days”. This internal inconsistency is an illogical set of responses. Inconsistent reports could also be defined as a situation where the youth reported no 30-day use but did answer the 30-day frequency question. These questions were available in the 1994, 1996 and 1998 interviews only.

Demographic Variables

The dataset consisted of all youth who were age 10 or older by the last wave of data collection in 1998 included for this study. Ethnicity was coded as 1= Hispanic, 2 = African American and 3 = Non-Hispanic/Non-African American. Age ranges changed based upon the year of interview. The minimum age was always 10 years but the upper age range increased with each interview, as did the mean age. In 1990 the age range was 10 - 20 years with the upper age range increasing by two years at each later wave of data collection. By 1998 the mean age was 16.26 years with a range of 10 - 28 years. When age related analyses were conducted, only the chronological age at the time of the survey was used for the analyses. For example, when examining if there were any differences in report stability of marijuana use by chronological age, a separate analysis for each year's stability variable and the same year's age were conducted.

Statistical methods

Data analyses were conducted using SPSS version 15.0.1.1. Chi-square analyses were conducted to compare the report agreement and discrepancy in reports by gender and ethnicity. Chronological age differences for those with report agreement and discrepancies was examined using analysis of variance and Scheffé post-hoc means tests to reduce type I error. Exact agreement for reported age of onset for marijuana use was tested using the Intraclass correlation coefficient (ICC), which is an agreement, based analysis for reliability. The two-way mixed model analysis of variance where the respondents were considered the random factor and the two time points for comparisons were the fixed factor (McGraw & Wong, 1996; Shrout & Fleiss, 1979; Bartko, 1966) was used for ICC calculations. This test-retest agreement is considered excellent when the ICCs are 0.75 or higher; considered good from 0.60-0.74; fair for values of 0.40 – 0.59; and poor for values below 0.40 (Johnson & Mott, 2001; Fendrich, Weissman, Warner & Mufson, 1990). Logistic regression analyses were conducted to test the association between the demographic variables and recanting marijuana use.

When testing for internal consistency with logically linked questions, the analyses included a phi and Cramer's V coefficient. These analyses were conducted for each wave-to-wave comparison as well as the within year interviews for the internal consistency analyses.

RESULTS

External Consistency

The results for external consistency can be found in Table 1. The number of youth who reported marijuana use at both Time 1 and Time 2 increased from 59 by 1992 to 644 in 1998. The percentage of those who reported their use consistently from Time 1 to Time 2 increased across time. As can be seen, the bulk of marijuana users for each time period were incident cases ranging from 43.2% - 77.8% of those reporting use at Time 2. Agreement rates were low for the 1990-92 comparison, but it increased to about 75-80% in the other three comparisons (with incident cases removed). Chi-square analyses were conducted to examine for differences between ethnic and sex groups. No ethnic differences were found for report agreement and only one analysis resulted in significant differences between males and females. In this one analysis (not presented in a table) it was found that report discrepancies were higher among males than females 23.9 % vs. 14.0%, Chi-square value 3.47, p< 0.05.

Table 1.

Report agreement, report discrepancy and incident cases for marijuana users over eight years.

Years of Interviews	Total N Reported Use Time 1 or Time 2	Agreement Yes-Yes N (%)	Use Discrepancy Yes- No N (%)	Incident Cases N (%)
1990-92	59	6 (31.6)	13 (68.4)	40 (67.8)
1992-94	225	37 (74.7)	13 (25.3)	175 (77.8)
1994-96	537	178 (80.9)	42 (19.1)	317 (59.1)
1996-98	644	269 (73.5)	97 (26.5)	278 (43.2)

Table 2 presents the results from ANOVA tests examining the mean age differences between consistent and inconsistent reporters. It was found that consistent reporters were significantly older than inconsistent reporters on three of the four wave-to-wave comparisons. The greatest difference was found between consistent and inconsistent reporters’ ages in the 1990-1992 comparison in which the consistent reporters were 3.18 years older than the inconsistent reporters. The age difference decreased with the lowest difference being 0.57 years between the two groups in 1996-1998.

Table 2.

Mean chronological age differences between those who are consistent or inconsistent reporters for lifetime marijuana use from Time 1 to Time 2.

2 Year Comparison	Report Marijuana Use Both Waves Mean Age (SD)	Denied Use at Time 2 Mean Age (SD)	ANOVA F Value
1990 - 1992	17.33 (1.86)	14.15 (1.40)	17.20***
1992 – 1994	18.46 (1.48)	16.08 (2.56)	16.58***
1994 – 1996	19.22 (2.03)	18.62 (1.99)	NS
1996 - 1998	18.72 (1.67)	18.15 (2.07)	7.27**

NS = not significant

^**p < .01

^***p < .001

Age of Onset

The analyses for consistent reporting of age of onset are presented in Table 3. The N size for the cells for the 1990-92 comparison were too low and thus not included in the table. It was found that about 35% - 40% of youth reported their age onset exactly the same from the first wave of data collection to the second. This increased when a one-year allowance was made with the agreement across waves increased to 70% - 74%. The remaining one-third of youth was either forward telescopers or backward telescopers. The patterns are different across the waves of data collection. From 1992-94 it was found the more youth with a greater than 1 year difference in age of onset from Time 1 to Time 2 were primarily backward telescopers. This indicates that they reported a younger age at Time 2 than Time 1. Yet in the latter waves of data collection, it was found that the majority of youth were forward telescopers in that they reported their age of onset as two or more years older at Time 2 compared to Time 1.

Table 3.

Percentage who reported their age of onset accurately and the variability of inaccuracy across waves of data collection for marijuana use.

Reported Age of Onset Exactly the Same Wave to Wave	Years of Interviews	± 1 YEAR	Backward Telescopers ≤ 2 YEARS	Forward Telescopers ≥ 2 YEARS
40.0%	1992-94	70.0%	20.0%	10.0%
35.1%	1994-96	74.2%	5.8%	20.0%
34.8%	1996-98	72.3%	8.0%	19.8%

Multi-wave comparisons using intraclass correlation coefficients (ICC) are presented in Table 4. As can be seen, the ICC values decrease somewhat as the number of years between the baseline report and latter reports increases. For example, marijuana users who reported use in 1992 and 1994 had an ICC of .721 which decreased slightly but remained stable at about .66 in 1992-96 and 1992-98. All values for the ICCs for multi-wave comparisons for age of onset remained in the excellent to good range and drop on slightly as the time range increases.

Table 4.

Reliability of Age of Onset for Marijuana Use Across Multiple Waves of Reports.

N-Size	Year to Year Comparison	Intraclass Correlation Coefficient (ICC)	Significance
1992
N = 40	1992 – 1994	.721	.000
N = 37	1992 – 1996	.664	.000
N = 13	1992 – 1998	.672	.01
1994
N = 174	1994 – 1996	.814	.000
N = 89	1994 – 1998	.711	.000
1996
N = 264	1996 – 1998	.762	.000

Analysis of variance statistics were conducted to test if there were differences among varying types of age of onset reports and demographic variables. As presented in Table 5, differences were found for two of the analyses for ethnicity. It was found that Hispanic adolescents were most likely to report their age of onset exactly the same as they reported it two years earlier. They were followed by the Non-Hispanic, Non-African American youth with African American teens having the lowest percentages with reported age of onset matching across waves. The Non-Hispanic, Non-African American youth were more likely to be backward telescopers while the African American teens were more likely to be forward telescopers. When examining chronological age (not presented in a table), it was found that only in the 1992-94 comparison reached statistical difference. In that one analysis, those who reported their age exactly the same two years later had a mean age of 18.0, backward telescopers were older at 20.0 and forward telescopers had a mean age of 18.13 (F value 8.13, p < .001). None of the analyses for 1992-94 reached statistical significance and was not included in the table.

Table 5.

Differences between three ethnic groups for marijuana use consistently, backward and forward telescopers.

Years of Interviews	Reported Age of Onset Exactly the Same Wave to Wave N (%)	Backward Telescopers N (%)	Forward Telescopers N (%)	ANOVA F Test
1994-96
Hispanic	24 (53.3)	7 (15.6)	14 (31.1)	13.96**
African American	12 (20.3	13 (22.0)	34 (57.6)
Non-Hispanic, Non-African American	25 (35.7)	18 (25.7)	27 (38.6)
1996-98
Hispanic	34 (43.0)	18 (22.8)	27 (34.2)	15.83**
African American	14 (19.7)	13 (18.3)	44 (62.0)
Non-Hispanic, Non-African American	44 (38.6)	28 (24.6)	113 (36.8)

1992-1994 not significant

^**p < .01

Internal Consistency

The results for internal consistency are presented in Table 6 and it can be seen that for the three years examined, 13-20% of youth made an error in their self-reports regarding past thirty-day use and frequency of such use. Of those with an error, the errors were mostly among those who reported they used during the prior thirty days but then in a later question reported “no use” frequency of thirty-day use. The Phi and Cramer's V statistic was similar for all three years at about .63-.67.

Table 6.

Internal consistency between the recency of marijuana use question and the frequency of marijuana use question.

Substance	Recency of last use
Frequency of 30 day	No Use past 30 days n (%)	Used past 30 days n (%)	Total Inconsistencies	Phi and Cramer's V
1994
No use past 30 days	124 (44.4)	5 (1.8)	55 (19.7)	.646***
Used 1+ times past 30 days	50 (17.9)	100 (35.8)
1996
No use past 30 days	227 (41.5)	3 (0.5)	119 (21.7)	.634***
Used 1+ times past 30 days	116 (21.2)	201 (36.7)
1998
No use past 30 days	302 (47.1)	4 (10.6)	121 (13.6)	.669***
Used 1+ times past 30 days	117 (18.3)	218 (34.0)

* = p < .05

** = p < .01

^***p < .001

When testing for ethnicity and sex differences it was found that females were more consistent in their reports than males (Table 7). In both 1994 and 1998 85% of females were consistent across the recent use questions compared to about 76% of males. Only one comparison was significant for the ethnic groups in the 1998 survey, with about 77% of Hispanic and African American teens being consistent compared to 86% of the Non-Hispanic, Non-African American youth.

Table 7.

Differences of internal consistency for marijuana use by ethnicity and sex

	Internally Consistent	Not Internally Consistent	Chi-Square Value
1998
Hispanic	130 (77.8)	37 (22.2)	8.39*
African American	152 (76.8)	46 (23.2)
Non-Hispanic, Non-African American	238 (86.2)	38 (13.8)
1994
Male	113 (75.8)	36 (24.2)	4.00*
Female	111 (85.4)	19 (14.6)
1998
Male	246 (76.6)	75 (23.4)	8.46**
Female	274 (85.6)	46 (14.4)

^*p < .05

^**p < .01

Multiple Logistic Regression

To examine the association of the demographic variables with self-report inconsistencies for marijuana use (external consistency), logistic regression models were conducted for each wave-to-wave comparison with age, sex and ethnicity. For ethnicity, the referent group was Non-Hispanic, Non-African American, and for sex the referent group was female. Results are not reported in a table because chronological age is the only variable to remain statistically significant in three of the five models. The models in which chronological age remained significant are 1990, 1994 and 1998. The findings indicate that younger age was significantly associated with report inconsistency in each model. The odds ratio for chronological age ranged from 1.20 in 1996 to 2.50 in 1992. The odds ratios show that with each year of increased chronological age the odds of being a consistent self-reporter of marijuana use increased significantly. The significant sex and ethnicity findings from the bivariate analyses did not remain in the multivariate models.

DISCUSSION

In this study the self-report stability of a national group of children, adolescent and young adults was examined across eight years. It was found that chronological age was the one consistent variable associated with self-report consistency across varying two-year time intervals. Although sex and ethnicity had statistically significant associations in some of the wave-to-wave comparisons, such associations did not continue when controlling for chronological age in the multivariate logistic regression. Such findings demonstrate that the youngest self-reporters of marijuana use are not as stable in their self-reports across time as their older aged peers.

Other research on self-report stability has identified differences between varying ethnic groups. Fendrich and Vaughn (1994) noted that minority respondents were more likely to underreport their substance use compared to whites and this was replicated in a later studies as well (Fendrich and Kim, 2001; Fendrich and Rosenbaum, 2003). Although our findings in the bivariate analyses identified a similar pattern, such differences did not remain after controlling for chronological age. The internal consistency of three surveys was examined for the ability to answer questions logically within a single survey. As with the report stability analyses, females were found to be more consistent than males. Again, sex differences do not remain significant after adjusting for chronological age. Similar findings are reported in Shillington et al., (in press) in that chronological age at the time of report was the most consistently significant variable in multivariate analyses for alcohol report stability.

Age of onset stability was high if a one-year allowance was made. With stricter criteria, that demanded exact agreement in age of onset from one wave to the next, the agreement was low at 34-40%. However with the one-year allowance, the agreement increased so that nearly three-fourths of youth had their reports match. Forward telescopers were more prevalent among marijuana users than backward telescopers. So inconsistent reporters of age of onset for marijuana use are more likely to report their age of onset as older at a later wave of data collection.

Strengths and Limitations

The study design is a strength. It utilized a national, panel study with two-year follow-ups spanning eight years. With it we were able to examine report stability for use, age of onset, and logical consistency for the same youth. Further, this is the first study to examine such measures of stability for marijuana use across nearly a decade of self-reported use. However, although marijuana use is illicit, the findings here would likely vary greatly from that of other illicit drug self-reports.

One limitation of this study is that the phrasing of the ever use questions changed slightly during 1-2 survey years. These changes could have impacted how a youth interpreted the questions. However, upon examination of the data there are no significant shifts in report consistency related to such changes. Also, such changes were in place for all respondents so this would not explain the differences identified here. The data for this study were self-reports of use. There was no validation of the self-reported use by way of a saliva test for example and therefore should be interpreted as such.

Another limitation of the current study is that the adolescents queried are the older children to young mothers. So, although the sample is a national sample and the mothers are nationally generalizable, the findings from the children are not generalizable to all youth in this age range.

Further, lifetime use questions were phrased as “ever use” questions. Some of the report instability may be due to the issue of the youth's interpretation of such a threshold. Children who reported prior use and denied it later may have reported “yes” to marijuana use but had only a puff or two. Later denial may be a result from a decision that such use is not true use. Future research may examine this issue of question phrasing. So, with the lack of data trends coinciding with wording changes and the need of research that examines this important issue, the limitation is noted but thought to be acceptable. Because there is so little research that examines these specific issues, it is important to advance our understanding due to so many AOD studies collecting data with cross-sectional designs.

Future Research and Conclusions

To move the field forward, future research needs to conduct qualitative research and longitudinal research with a theoretical framework to guide queries into report discrepancies as they occur. Thus, this topic needs to be examined within the context of a broader theoretical perspective regarding factors affecting the reporting of potentially sensitive behavior. Additionally, such research would benefit from the inclusion of biomeasures to validate the self-reported marijuana use. Clinical implications would include the stability of adolescents reporting their use of marijuana with diagnostic instruments. Such inconsistencies may lead to some adolescents being under diagnosed and not referred to treatment. Since we don't know the reasons for the inconsistencies it is difficult to state whether the older adolescents are better reporters or if they have motivations to disclose their marijuana use that differ from younger adolescents.

Our findings indicate a strong impact of chronological age on report accuracy. More research is needed to understand the reasons for this. It is not possible to delve into this issue with the current longitudinal data. Future research should be designed to specifically identify report instability and query the youth in-depth regarding such inconsistency. Because the findings from this research indicate that younger age is important to understanding self-report stability a strong focus on cognitive development would be essential.