Abstract
Objective:
Assess agreement between self-report and urine toxicology measures assessing use of two illicit simulants (methamphetamine and cocaine) during early pregnancy.
Methods:
This cross-sectional study of 203,053 pregnancies from 169,709 individuals receiving prenatal care at Kaiser Permanente Northern California (KPNC) between 1/1/2011 and to 12/31/2019 assessed agreement (kappa, sensitivity, and specificity) between self-reported frequency and urine toxicology measures of methamphetamine and cocaine early in pregnancy.
Results:
Prenatal use of the illicit stimulants was rare according to toxicology (n=244 [0.12%]) and self-report (n=294 [0.14%]) measures. Agreement between these measures was low (kappa <0.20). Of the 498 positive pregnancies, 40 (8.03%) screened positive on both measures, 204 (40.96%) screened positive on toxicology tests only, and 254 (51.00%) screened positive by self-report only. Relative to toxicology tests, sensitivity of any self-reported use was poor with 16.39% (95% confidence interval [CI]:11.75%−21.04%) of pregnancies with a positive toxicology test self-reporting any use in pregnancy. Relative to self-report, sensitivity of toxicology tests was also poor with 13.61% (95% CI:9.69%−17.52%) of pregnancies who self-reported any use having positive urine toxicology tests. The sensitivity improved slightly at higher frequencies of self-reported use: daily=17.50%, 95% CI:5.72%−29.29%; weekly=25.00%, 95% CI:11.58%−38.42%; and monthly or less=11.06%, 95% CI: 6.89%−15.23%. Specificity was high (>99%), reflecting the high negative rate of use.
Conclusions:
Findings suggest utilizing self-report and toxicology measures in combination likely provides the most accurate information on methamphetamine and cocaine use in early pregnancy. Findings also highlight the need to provide supportive, non-stigmatizing environments in which pregnant individuals feel comfortable disclosing substance use without fear of punishment.
Keywords: Agreement, validity, self-report, urine toxicology, illicit stimulant, pregnancy
Introduction
Healthcare workers often rely on self-report measures to assess use of illicit stimulants, such as methamphetamine and cocaine, in pregnancy, as they are cost effective to administer (1). However, these measures are subject to self-report biases. For example, pregnant individuals may misreport stimulant use due to a lack of knowledge about the actual identity or purity of the substance they used. Relying on self-reports of illicit stimulant use in pregnancy may also lead to an underestimation of prenatal use due to underreporting from fear of social stigma, legal consequences, or discrimination (2). Urine toxicology testing is also routinely used in clinical practice to identify stimulant use, and is often viewed as a valid and objective method to test biological samples for stimulants. However, the ability of urine toxicology screenings to detect stimulant use depends on multiple factors, including the analytical approach to testing, the test performance characteristics (including cutoff levels), substance use patterns, urine quality and pH, and many patient-specific physiological characteristics e.g., body mass and renal and liver functioning (3). Moreover, methamphetamine can typically only be detected in urine for 2 days after the last use, and cocaine metabolites can typically only be detected in urine for 2 to 4 days after the last use (3). Given the limitations of both self-report and urine toxicology measures, there is no gold standard for measuring illicit stimulant use in pregnancy. Understanding the validity of measures of illicit prenatal stimulant use is vital for healthcare professionals to adequately provide services to pregnant individuals with illicit stimulant use, and for researchers to accurately estimate both the prevalence and associated maternal and child health risks of illicit stimulant use in pregnancy.
Self-reports of illicit stimulant use may be good at detecting negative tests of drugs in biological specimens but poor at detecting positive tests of drugs in biological specimens. For example, a meta-analysis of seven studies conducted with non-pregnant individuals reported high specificity (i.e., how often tests are negative when this substance is absent; 89%) but low sensitivity (i.e., how often the tests are positive when the substance is present; 59%) of self-reported cocaine use relative to laboratory assays (1). Research assessing agreement between self-reports and laboratory assays of illicit stimulant use among pregnant individuals is sparce. However, a small study of 83 pregnant individuals found results consistent with the meta-analysis, with high specificity (≥ 90%) and low sensitivity (47%) of self-reported prenatal cocaine use relative to urine toxicology tests (4).
Given the lack of research on the validity of measures of illicit stimulant use in pregnancy, the primary aim of this study was to assess the agreement of self-reported and urine toxicology measures of methamphetamine and cocaine use during early pregnancy in a large sample of pregnant individuals universally screened for substance use at entrance to prenatal care.
Methods
Study population
We utilized data from a large integrated health care delivery system, Kaiser Permanente Northern California (KPNC). KPNC provides care to over 4 million racially and socioeconomically diverse members who are representative of the insured population in Northern California (5). All first-trimester pregnancies that were screened for prenatal substance use by self-report questionnaires during standard prenatal care (January 2011-December 2019) and by urine toxicology test within +/− one week of the self-report questionnaire were considered for inclusion. The Institutional Review Board at KPNC approved this data-only study and waived informed consent.
Measures
Methamphetamine and cocaine use
Data on methamphetamine and cocaine use were obtained from the electronic health record (EHR). Pregnant individuals are universally screened for substance use, including use of methamphetamine and cocaine at entrance to prenatal care, which typically occurs at ~8 weeks gestation. The screening includes both a self-administered questionnaire, which asks how often the patient has used each substance “since pregnancy” with the answer options: “daily”, “weekly”, “monthly or less”, and “never”, as well as a urine toxicology test to which patients provide consent.
Amphetamine urine screening tests were performed on a Beckman Coulter AU680 chemistry analyzer using the DRI Amphetamines Assay with a cutoff of 500 ng/mL. Confirmatory testing for the presence of amphetamine species was performed by liquid chromatography-tandem mass spectrometry for all positive immunoassay results. For this study, amphetamine species detected by the confirmation assay included only methamphetamine. The positive cutoff for the confirmation assay was 250 ng/mL.
Cocaine urine screening tests were performed on a Beckman Coulter AU680 chemistry analyzer using the Emit II Plus Cocaine Metabolite Assay which detects the cocaine metabolite, benzoylecgonine, with a positive cutoff of 150 ng/mL. Confirmatory testing for the presence of benzoylecgonine was performed by liquid chromatography-tandem mass spectrometry for all positive immunoassay results. The positive cutoff for the confirmation assay was 100 ng/mL.
Sociodemographic characteristics
We extracted information on sociodemographic characteristics measured during the first trimester of pregnancy from the EHR. This information included maternal age at first day of last menstrual period (less than 25, 25–34 years, 35 or more years), parity (0, 1 or more, missing), and self-reported race/ethnicity (Asian/Islander, Black, Hispanic, Other/Unknown/Multiracial, non-Hispanic White).
Statistical analyses
Main analyses
First, we calculated kappa values to assess the agreement between urine toxicology tests and (a) any, (b) daily, (c) weekly, and (d) monthly or less self-reported use. Second, we assessed the sensitivity and specificity of self-reported use relative to urine toxicology tests. We calculated sensitivity of self-reported use as the proportion of individuals with a positive urine toxicology test who self-reported any methamphetamine or cocaine use, and specificity of self-reported use as the proportion of individuals with a negative urine toxicology test who self-reported no methamphetamine and cocaine use. Third, we calculated sensitivity and specificity of urine toxicology tests relative to (a) any, (b) daily, (c) weekly, and (d) monthly or less self-reported use. We calculated sensitivity of urine toxicology tests as the proportion of individuals with self-reported use who had a positive urine toxicology test and specificity of urine toxicology tests as the proportion without self-reported use who had a negative urine toxicology test. We calculated sensitivity and specificity using both self-report and urine toxicology measures as the reference because there is no established gold standard for measuring illicit stimulant use.
Sensitivity analyses
Because main analyses included individuals who were given a urine toxicology tests within +/− one week of the self-reported questionnaires, we conducted sensitivity analyses with the subset of individuals who completed the urine toxicology test and self-report questionnaire on the same day to evaluate if agreement would be better when measures were collected at the same time.
Results
Study population
Among the 331,995 eligible pregnancies, we excluded 2,381 (0.72%) with a missing response on the self-report questionnaire about prenatal methamphetamine or cocaine use and 28,450 (8.57%) without a urine toxicology test for methamphetamine or cocaine use. We also excluded 98,111 (32.58%) who did not complete the urine toxicology test within +/− one week of the self-report questionnaire. The final study sample included 203,053 pregnancies from 169,709 individuals. Supplement 1 presents a flowchart of the cohort creation.
In the final sample of 203,053 pregnancies, 0.25% (n=498) screened positive by either measure; 0.12% (n=244) tested positive by urine toxicology tests; and 0.14% (n=294) self-reported use (0.02% [n=40] daily; 0.02% [n=40] weekly; 0.11% [217] monthly or less use). In the final sample of 203,053 pregnancies, most pregnancies occurred to individuals between the ages of 25 and 34 years (62.86%), with 1 or more previous births (55.26%), and of non-White race/ethnicity (26.14% Asian/Islander, 5.92% Black, 25.47% Hispanic, 4.61% Other/Unknown/Multiracial, 37.86% White).
Agreement
Agreement between urine toxicology and self-report measures was low regardless of whether assessing any, daily, weekly, or monthly or less self-reported use (kappa values all less than <0.20; Table 1). Of the 498 pregnancies that screened positive according to either measure, 40 (8.03%) screened positive on both, 204 (40.96%) screened positive on the toxicology test only; 254 (51.00%) screened positive based on self-report only.
Table 1.
Agreement between self-report and urine toxicology measures among individuals who completed urine toxicology tests +/− one week of the self-report questionnaires (n=203,053)
| Urine toxicology test results among those with self-reported use | ||||
|---|---|---|---|---|
| Sample: Pregnant individuals who completed urine toxicology tests +/− one week of the self-report questionnaires (n=203,053) | Sample: Pregnant individuals who completed urine toxicology tests and self-report questionnaires on the same day (n=132,788) | |||
| Positive urine toxicology n (%) |
Negative urine toxicology n (%) |
Positive urine toxicology n (%) |
Negative urine toxicology n (%) |
|
| Any self-reported use | 40 (13.61%) | 254 (86.39%) | 34 (17.53) | 160 (82.47) |
| Daily self-reported use | 7 (17.50%) | 33 (82.50%) | 6 (21.43) | 22 (78.57) |
| Weekly self-reported use | 10 (25.00%) | 30 (75.00%) | 8 (26.67) | 22 (73.33) |
| Monthly or less self-reported use | 24 (11.06%) | 193 (88.94%) | 21 (15.22) | 117 (84.78) |
| Estimates of agreement between urine toxicology tests and self-reported frequency of use | ||||
| Sample: Pregnant individuals who completed urine toxicology tests +/− one week of the self-report questionnaires (n=203,053) | Sample: Pregnant individuals who completed urine toxicology tests and self-report questionnaires on the same day (n=132,788) | |||
| Kappa (95% CI) | Kappa (95% CI) | |||
| Any self-reported use and urine toxicology test | 0.15 (0.11, 0.19) | 0.18 (0.13, 0.24) | ||
| Daily self-reported use and urine toxicology test | 0.05 (0.14, 0.08) | 0.06 (0.01, 0.10) | ||
| Weekly self-reported use and urine toxicology test | 0.07 (0.03, 0.11) | 0.08 (0.03, 0.13) | ||
| Monthly or less self-reported use and urine toxicology test | 0.10 (0.06, 0.14) | 0.13 (0.08, 0.18) | ||
| Sensitivity (95% CI) | Specificity (95% CI) | Sensitivity (95% CI) | Specificity (95% CI) | |
| Urine toxicology test as reference | ||||
| Any self-reported use versus urine toxicology test | 16.39% (11.75%, 21.04%) | 99.87% (99.86%, 99.89%) | 19.21% (13.41%, 25.01%) | 99.86% (99.86%, 99.90%) |
| Self-reported use as reference | ||||
| Urine toxicology test versus any self-reported use | 13.61% (9.69%, 17.52%) | 99.90% (99.89%, 99.91%) | 17.53% (12.18%, 22.88%) | 99.89% (99.87%, 99.91%) |
| Urine toxicology test versus daily self-reported use | 17.50% (5.72%, 29.29%) | 99.88% (99.87%, 99.90%) | 21.43% (6.23%, 36.63%) | 99.87% (99.85%, 99.89%) |
| Urine toxicology test versus weekly self-reported use | 25.00% (11.58%, 38.42%) | 99.88% (99.87%, 99.90%) | 26.67% (10.84%, 42.49%) | 99.87% (99.85%, 99.89%) |
| Urine toxicology test versus monthly/less self-reported use | 11.06% (6.89%, 15.23%) | 99.89% (99.88%, 99.91%) | 15.22% (9.22%, 21.21%) | 99.88% (99.86%, 99.90%) |
Note: CI=confidence interval.
Sensitivity and specificity
With urine toxicology tests as the reference, the sensitivity of any self-reported methamphetamine or cocaine use was poor, with only 16.39% (95% confidence interval [CI]: 11.75%−21.04%) of those with a positive urine toxicology test self-reporting any use; however, the specificity of any self-reported use was excellent with 99.87% (95% CI: 99.86%−99.89%) of those with a negative urine toxicology test self-reporting no use (Table 1).
With self-reported frequency of methamphetamine and cocaine use (covering the period from the start of pregnancy to the time the questionnaire was completed) as the reference, the sensitivity of urine toxicology tests was poor with only 13.61% (95% CI: 9.69%−17.52%) of those who self-reported using any methamphetamine or cocaine having a positive urine toxicology test. The sensitivity improved only slightly at higher frequencies of self-reported use. Specifically, 17.50% (95% CI: 5.72%−29.29%) of those who self-reported using daily had a positive urine toxicology test, 25.00% (95% CI: 11.58%−38.42%) of those who self-reported using weekly had a positive urine toxicology test, and 11.06% (95% CI: 6.89%−15.23%) of those who self-reported using monthly or less had a positive urine toxicology test. When using self-reported frequency of use as the reference, the specificity of urine toxicology tests was excellent, though this mostly reflects the high negative rate of overall use. Regardless of whether self-reported use was daily, weekly, or monthly or less, more than 99% of those who self-reported methamphetamine or cocaine use since pregnancy had a negative urine toxicology test (Table 1).
Sensitivity analyses
The majority of pregnancies (65.40% of the analytic sample; n=132,788) were screened via both measures on the same day. Among the those who completed the self-report and urine toxicology screenings on different days (n=70,265), 42.12% (n=29,594) had screenings that were 1 to 2 days apart, 23.52% (n=16,527) had screenings were 3 to 4 days apart, and 34.36% (n=24,144) had screenings that were 5 to 7 days apart.
Among the subsample of pregnancies with their urine toxicology test collected on the same day as their self-report questionnaire, the general pattern of results was the same as the results from analyses conducted in the whole sample. However, we found slightly higher sensitivity for: (a) any self-reported use relative to urine toxicology tests (19.21%, 95% CI: 13.41%−25.01%) and (b) urine toxicology tests relative to self-reports (any self-reported use=17.53%, 95% CI: 12.18%−22.88%; daily self-reported use=21.43%, 95% CI: 6.23%−36.63%; weekly self-reported use=26.67%, 95% CI: 10.84%−42.49%; monthly/less self-reported use=15.22%, 95% CI: 9.22%−21.21%; Table 1).
Discussion
We found poor agreement between a self-reported measure of methamphetamine and cocaine use (completed in the first trimester, covering the period from pregnancy start to questionnaire completion) and urine toxicology results of methamphetamine and cocaine use (measured within one week of the questionnaire) in a large, representative sample of pregnant individuals screened for substance use early in pregnancy as part of routine prenatal care. Self-report questionnaires were excellent at identifying negative urine toxicology tests, and urine toxicology tests were excellent at identifying pregnancies without self-reported methamphetamine or cocaine use. However, self-report questionnaires were poor at identifying positive urine toxicology tests, indicating that many pregnant individuals with positive illicit stimulant urine toxicology tests may choose not to disclose their use. In general, urine toxicology tests were also poor at identifying self-reported methamphetamine and cocaine use. This finding may be due to the short half-life of methamphetamine and cocaine. It is important to note that urine toxicology tests can only typically detect methamphetamine and cocaine use that occurred up to 2–4 days prior (6), and our results showed that urine toxicology tests were better able to identify pregnant individuals with more frequent (daily or weekly versus monthly or less) self-reported use.
These findings have important clinical implications. First, they suggest that neither urine toxicology tests nor self-reports are good at identifying methamphetamine and cocaine use in pregnant individuals and utilizing both measures in combination likely provides the most accurate information to health care professionals and researchers. For example, clinicians could follow-up with individuals who have inconsistent results with additional conversations. Second, given that most individuals with a positive urine test for methamphetamine or cocaine use chose not to disclose any use in pregnancy, clinicians, healthcare systems, and policymakers should take care in providing supportive, non-stigmatizing environments in which pregnant individuals feel comfortable disclosing substance use. Further, punitive policies that criminalize individuals for prenatal substance use may further prevent individuals from disclosing their use and seeking treatment, when needed (2). In California, substance use during pregnancy is not considered child abuse or grounds for civil commitment (7), allowing the KPNC Early Start program to screen for substance use via urine toxicology testing along with non-judgmental, non-punitive substance use counseling integrated into prenatal care with improved outcomes for mothers and their babies (8). Understandably, the sensitivity of self-reported prenatal illicit stimulant use may be even lower in settings without such programs.
These findings should be considered in light of several limitations. First, self-report questionnaires queried methamphetamine and cocaine use since start of pregnancy, rather than current use, and urine toxicology tests can only detect methamphetamine and cocaine that has been used within 2 to 4 days of testing. Second, our sample included pregnant individuals who were screened for prenatal substance use during the first trimester of pregnancy in KPNC as part of standard prenatal care. Therefore, findings may not generalize to pregnant individuals within KPNC who were not screened for substance use, to those who enter prenatal care late, to those outside California, or to those without access to health care. Future studies need to examine how later entry into prenatal care may impact findings.
Conclusions
Both self-report and urine toxicology measures had low sensitivity. Therefore, using both measures in combination likely provides the most accurate information on methamphetamine and cocaine use during pregnancy (e.g., following-up with individuals that have inconsistent results with additional conversations). Importantly, many individuals who test positive for methamphetamine and cocaine use via urine toxicology tests chose not to disclose use, indicating that clinicians, healthcare systems, and policymakers should take care in providing supportive, non-stigmatizing environments in which pregnant individuals feel comfortable disclosing substance use and seeking help when needed, without fear of punishment.
Supplementary Material
Funding:
This study was supported by the National Institute on Drug Abuse (K01 DA043604 and R01 DA048033 and R01 DA047405) and the National Institute on Alcohol Abuse and Alcoholism (K08 AA027551). The funding organizations had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication.
Footnotes
Conflicts of interest: All authors declare no conflict of interest.
Adherence to preprint policy: This manuscript is an original contribution, has not previously been published, and is not under review elsewhere. This manuscript was not posted on a preprint server.
References
- 1.Large MM, Smith G, Sara G, Paton MB, Kedzior KK, Nielssen OB. Meta-analysis of self-reported substance use compared with laboratory substance assay in general adult mental health settings. International Journal of Methods in Psychiatric Research. 2012;21(2):134–48. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Stone R. Pregnant women and substance use: fear, stigma, and barriers to care. Health & Justice. 2015;3(1):2. [Google Scholar]
- 3.Moeller KE, Kissack JC, Atayee RS, Lee KC. Clinical Interpretation of Urine Drug Tests: What Clinicians Need to Know About Urine Drug Screens. Mayo Clin Proc. 2017;92(5):774–96. [DOI] [PubMed] [Google Scholar]
- 4.Garg M, Garrison L, Leeman L, Hamidovic A, Borrego M, Rayburn WF, et al. Validity of Self-Reported Drug Use Information Among Pregnant Women. Maternal and Child Health Journal. 2016;20(1):41–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Gordon NP. Similarity of Adult Kaiser Permanente Members to the Adult Population in Kaiser Permanente’s Northern California Service Area: Comparisons based on the 2017/2018 cycle of the California Health Interview Survey. Kaiser Permanente Northern California Division of Research, Oakland, CA; 2020. [Google Scholar]
- 6.Moeller K, Lee KC, Kissack JC. Urine Drug Screening: Practical Guide for Clinicians. Mayo Clinic Proceedings. 2008;83(1):66–76. [DOI] [PubMed] [Google Scholar]
- 7.Guttmacher Institite. Substance use during pregnancy 2022. [Available from: https://www.guttmacher.org/state-policy/explore/substance-use-during-pregnancy.
- 8.Goler NC, Armstrong MA, Taillac CJ, Osejo VM. Substance abuse treatment linked with prenatal visits improves perinatal outcomes: a new standard. J Perinatol. 2008;28(9):597–603. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.