Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 Jan 2.
Published in final edited form as: Subst Use Misuse. 2015 Dec 18;51(1):23–33. doi: 10.3109/10826084.2015.1073326

Measuring current drug use in female sex workers and their non-commercial male partners in Mexico: Concordance between data collected from surveys versus semi-structured interviews

Lawrence A Palinkas 1,✉, Angela Robertson Bazzi 2, Jennifer L Syvertsen 3, Monica D Ulibarri 4, Daniel Hernandez 5, Gudelia Rangel 6, Gustavo Martinez 7, Steffanie A Strathdee 8
PMCID: PMC4802973  NIHMSID: NIHMS769623  PMID: 26683591

Abstract

Background

Self-reports are commonly used to assess prevalence and frequency of drug use, but it is unclear whether qualitative methods like semi-structured interviews are as useful at obtaining such information as quantitative surveys.

Objectives

This study compared drug use occurrence and frequency using data collected from quantitative surveys and qualitative interviews. We also examined whether combining data from both sources could result in significant increases in percentages of current users and whether the concordance between the two sets of data was associated with the type of drug use, age, gender and socioeconomic status.

Methods

Self- reports of recent marijuana, heroin, crack, cocaine, crystal/methamphetamine, inhalant, and tranquilizer use were collected using both methods from a cohort of Mexican female sex workers and their non-commercial male partners (n = 82).

Results

Participants were significantly less likely to report marijuana, cocaine and tranquilizer use and frequency of use during the qualitative interviews than during the quantitative surveys. Agreement on frequency of drug use was excellent for crystal/methamphetamine, heroin and inhalant use, and weak for cocaine, tranquilizers and marijuana use. Older participants exhibited significantly higher concordance than younger participants in reports of marijuana and methamphetamine use. Higher monthly income was significantly associated with higher concordance in crack use but lower concordance with marijuana use.

Conclusions

Although use of such data can result in an underreporting of drug use, qualitative data can be quantified in certain circumstances to triangulate and confirm the results from quantitative analyses and provide a more comprehensive view of drug use.

Keywords: drug use, measurement, surveys, qualitative interviews, concordance, female sex workers

Introduction

One of the greatest challenges faced in the collection and reporting of sensitive information is determining validity. Individuals may be reluctant to accurately report conditions or behaviors that are stigmatized (e.g, mental health issues, sexual risk behaviors) (Del Boca & Noll, 2000; Mirzazadeh et al., 2014) or illegal (e.g., drug use, delinquency, criminal activity) (Catania, Gibson, Chitwood, & Coates, 1990; Solbergsdottir, Bjornsson, Gundmundsson Tyrfingsson & Kristinsson, 2004). Self-report may also be affected when participants possess marginalized social status (e.g., women, younger adults, sex workers, drug users, lower SES) (Del Boca & Noll, 2000; Fenrich & Johnson, 2005; Magura, Goldsmith, Casriel, Goldstein, & Lipton, 1987; Schwarz, 1999; Syvertsen, Robertson, Rolon, et al., 2013) or reduced capacity for accurate recall (e.g., due to impaired cognition, intoxication) (Del Boca & Noll, 2000) or by the type of drug used (Colon Robles & Sahai, 2001; Fendrich, Johnson, Sudman, Wislar, & Spiehler, 1999; Ledgerwood, Goldberger, Risk, Lewis, & Price, 2008). Reluctance or inability to accurately report such information could result in underestimates of prevalence, misdiagnoses or lack of diagnoses, inappropriate treatments, or missed opportunities for prevention or treatment (King & Bruner, 2003).

In addition to the characteristics of behaviors themselves or the individuals reporting on behaviors, the validity of information obtained about sensitive behaviors may be related to the methods through which the information is obtained. With respect to drug use, biological measures are considered to have the highest validity because they are less likely to be influenced by characteristics of the behavior (i.e, stigma, legality) and/or characteristics of the individual exhibiting the behavior (although it is possible to falsify biological specimens). However, collection of such data is not always feasible or even desirable (Day, Best, Cantillano, Gaston, Nambamali & Keaney, 2008; Winhusen, Somoza, Singal, Kim, Horn, & Rotrosen, 2003). Consequently, many studies rely on self-report and ethnographic methods (Singer et al., 2000). Such methods are widely used because they have the advantage of flexibility, adaptability, relatively low cost, and efficiency (Wilcox, Bogenschutz, Nakazawa & Woody, 2014). However, the extent to which they are valid and reliable is problematic (Buchan, Dennis, Tims, & Diamond, 2002; Magura & Kang, 1996; Williams & Nowatzki, 2005) and may vary depending on how the data are collected and the circumstances or contexts in which data collection occurs (Denis et al., 2012; Houck, Forcehimes, Gutierrez, & Bogenschutz, 2013; Wilcox et al., 2014).

Self-reported information can take two forms, quantitative data collected through structured surveys and standardized instruments, and qualitative data collected through extended interviews. Previous studies comparing the reliability of data collected from high-risk populations via face-to-face and self-administered surveys have reported mixed results (Islam et al., 2012), with some studies reporting no difference between the two methods (Bongers & Van Oers, 1998; Williams et al., 2000), and other studies finding self-administered quantitative surveys to generate more reliable data (Islam et al., 2012; Macalino, Clentano, Latkin, Strathdee, & Vlahow, 2002), a pattern attributed to social desirability bias (White, Day & Maher, 2007), presumed anonymity of surveys (Del Boca & Noll, 2000), and an ability to self-pace the data collection effort and to reflect on behavior and circumstances (Del Boca & Noll, 2000). In contrast, characteristics of quantitative data collection believed to potentially reduce validity and reliability include mode of administration (e.g., in-person interview, telephone interview, paper-and-pencil questionnaire, computer-assisted instrument), complexity and duration of the task, clarity of instructions, and instrument design (e.g., item wording, question sequencing, response format) (Del Boca & Noll, 2000).

Although qualitative data are generally not relied upon exclusively to assess prevalence and frequency of drug use, such data are increasingly being used in combination with quantitative data in mixed method designs (Gibson, Exner, Stone, Linquist, Cowen, & Roth, 2011; Lopez, Bourgois, Wenger, Lorvick, Martinez, & Kral, 2013; Mayhew, et al., 2009; Pollini et al., 2010; Rhodes, Wagner, Strathdee, Shannon, Davidson, & Bourgois, 2012; Wagner, Davidson, Pollini, Strathdee, & Palinkas, 2012). Each set of methods possesses a distinct set of strengths and limitations (Cresswell & Plano Clark, 2011; Palinkas, 2014; Patton, 2002; Teddlie & Tashakkori, 2003). They are used simultaneously or sequentially to achieve complementarity of perspectives (e.g., qualitative methods can explore issues or describe process and contexts while quantitative methods test hypotheses and describe outcomes), to expand or explain the findings of one set using another set of data, to develop new conceptual frameworks or questions, to sample study participants, or to triangulate results through convergence (Cresswell & Plano Clark, 2011; Lopez et al., 2013; Palinkas, 2014). Using qualitative data to triangulate or confirm the findings of quantitative analyses is especially useful when the sample size of the quantitative data provides insufficient statistical power (Palinkas, Horwitz, Chamberlain, Hurlburt, & Landsverk, 2011). It can also be used to provide more comprehensive estimates of the number of cases with a particular characteristic or set of characteristics such as members of social networks or current drug users (Rice et al., 2014). However, the practice of “quantitizing” qualitative data for the purpose of triangulation or testing hypotheses remains the subject of considerable debate, and there is even debate regarding whether quantitizing qualitative data fits the criteria of mixed methods designs (Onwuebuzie & Teddlie, 2003; Sandelowski, 2003). While some methodologists argue that quantitizing does little justice to the potential of qualitative data to develop in-depth knowledge of particular phenomena, other methodologists assert that under certain circumstances it can be performed with rigor to validly and reliably describe the breadth of a phenomenon (Miles & Huberman, 1995; Padgett, 2008).

In this study, we compared self-reports of drug use collected from quantitative surveys with self-reports collected from face-to-face qualitative interviews among female sex workers and their non-commercial male partners participating in a study of HIV risk in Ciudad Juarez and Tijuana, Mexico. Our aims were to determine whether: 1) there were any significant differences in estimates of the numbers of current drug users and frequency based on data collected using these two methods; 2) combining the two sets of data would produce estimates of the number of current drug users that was either significantly higher than produced by the data from the quantitative survey alone or remain unchanged; and 3) the concordance between the two sets of data was associated with the type of drug use or certain demographic characteristics of the user believed to be associated with underreporting of drug use, i.e., younger age, female gender, or lower socioeconomic status. While it is generally assumed that number of drug users and frequency are quantitative measures that are better captured through surveys specifically designed for that purpose, it is unclear whether a similar representation of these characteristic of drug abuse could be obtained by quantitizing qualitative data, especially since qualitative methods create opportunities for eliciting more detail and in-depth information (Patton, 2002), including sensitive topics like drug use (Lopez et al., 2013; Singer et al., 2000).

Methods

Participants

This study utilized data obtained from baseline interviews of 428 participants (214 couples) enrolled in Proyecto Parejas (Couples Project), a prospective social epidemiological study of HIV, sexually transmitted infections (STIs), and associated risk behaviors among female sex workers (FSWs) who use drugs and their non-commercial male partners in Tijuana (n = 212) and Ciudad Juarez (n = 216), Mexico (authors, 2012). Eligibility criteria for women were: being at least 18 years old; having exchanged sex for money, drugs, shelter, or goods in the past 30 days; ever using heroin, cocaine, crack, or methamphetamine; having a non-commercial male sexual partner for at least 6 months; having sex with that partner in the past 30 days, and agreeing to receive antibiotic treatment for STIs should they test positive. Women were excluded if they planned to break up with their non-commercial partner, move in the next 24 months (participants were to be followed up every 6 months for twenty-four months), or if they reported severe intimate partner violence (IPV) in their current relationship or fear of IPV as a result of their potential participation in the study. Eligibility criteria for the male partners were: being 18 years of age or older; being in a verified non-commercial relationship for six months or more with an eligible FSW; and having had sex with this partner within the past 30 days.

Procedures

We recruited women first using targeted sampling by pairs of outreach workers in areas where sex workers and drug users were known to frequent (bars, motels, streets, alleys). Snowball sampling also occurred in which enrolled FSWs could refer other women they knew to the study. Complete details on the development and methods of the screening and couples verification process, our IPV safety protocol, number of potential participants screened, and reasons for ineligibility are published elsewhere (authors, 2012).

Couples who passed the screening process provided written informed consent. Institutional review boards at the respective institutions approved all study protocols. As described in the sections below, all enrolled participants underwent confidential baseline quantitative surveys (i.e., no information from individual interviews were shared with individuals’ partners). A subset of couples also participated in qualitative interviews both together and separately. Gender-matched interviewers conducted all data collection. Participants received $20 USD for each individual interview, and couples received $20 USD for joint interviews.

Bilingual team members translated the drafts of the survey and semi-structured interview guide into Spanish. All translations were reviewed by field team members to assure accuracy and appropriateness. Attention was given to incorporating local drug terminology and other street slang (e.g., “ referring to crack as piedra and curar for “to cure” instead of “to inject” The Mexican field staff provided feedback to assess the content of the questions, add additional questions, and identify problematic words or phrases in the translations. All instruments were field tested in Tijuana by two coauthors (JS, AB), who suggested further edits to the questions and overall screening process based on that experience. Field-testing was an essential component to the protocol development.

The research team held training for field workers from both research sites in Tijuana. Interactive training activities such as role playing allowed fieldworkers to gain experience with the quantitative survey protocol and qualitative interviewing techniques and provide each other with feedback on their performance. While many of the field staff at both sites already had extensive experience working with FSWs and conducting quantitative data collection, training specifically aimed at working with FSWs in relationships provided guidance to address couple-based issues of gender and relationship power. To the extent possible, interviewers were matched with study participants by gender and ethnicity. Participants were also offered the option to complete the survey and semi-structured interview in English or Spanish.

Quantitative Data Collection

Beginning in 2010, we conducted 1.5-hour individual quantitative surveys to measure individual- and relationship-level characteristics and behaviors related to HIV/STI risk at baseline and every six months for two years. Surveys were administered to each partner separately in private rooms by trained interviewers using laptop computers. All measures were administered using computer-assisted personal interviewing (CAPI; NOVA software, MD, USA). Measures included demographic information such as age, gender, highest level of schooling completed, and monthly income. Participants were also asked if they had ever smoked marijuana; smoked or injected heroin; smoked or injected crack; smoked, snorted/inhaled, or injected cocaine; smoked, snorted/inhaled, or injected crystal/methamphetamine; used inhalants, or swallowed tranquilizers. Information concerning each type of drug and route of administration was obtained by asking three questions: 1) “have you ever used” [yes or no], and if yes, 2) “how long ago was the last time you used [drug],” and if six months or less, 3) “since [six months ago], how often have you used [drug]?” Response options to the third question included 1) one time per month or less, 2) 2–3 days per month, 3) 1 time per week, 4) 2–3 days per week, 5) 4–6 days per week, 6) one time per day, and 7) more than once a day. Participants were also asked about use of other drugs like barbituates, hallucinogens and PCP, but references to current use of these drugs during the qualitative interviews were too few to allow for meaningful analysis here.

Qualitative Data Collection

We conducted individual and joint qualitative interviews lasting 1–2 hours to explore the social and relationship dynamics surrounding HIV/STI risk with a sub-sample of 41 couples (n = 82 individuals) who were purposively selected for maximum variation in age, relationship duration, drug use and male employment status. We used the same semi-structured interview guide for couple and individual interviews. For this study, we identified responses to the following questions: 1) Do you use drugs; 2) Does your partner use drugs; and if yes, 3) What kinds of drugs do you and/or your partner use; and 4) How often? The first three questions were included as part of the interview guide, but participants who indicated they were using drugs were usually but not always asked the fourth question if the information was not voluntarily provided. As we were interested in how individuals reported their drug use quantitatively or qualitatively, we relied on information collected during the individual interviews only.

We digitally recorded and transcribed all qualitative interviews following a structured protocol (McLellan, MacQueen & Neidig, 2003). Interviews conducted in Spanish were translated into English. With topics of interest in this study (i.e., prevalence and frequency) determined a priori, we employed a primarily deductive coding strategy with the baseline data (Patton, 2002). Our bilingual team of qualitative data analysts carefully reviewed all individual and couple transcripts for drug use-related content. Analysts used MAXQDA software to manage, merge, and analyze the transcript data, interview summaries, and memos in an integrated system. For each participant, mention of current drug use and frequency of drugs used were extracted from these data and coded using the same scale and frequency criteria used in survey questionnaires; in some instances, participants described their drug use in terms such as “rarely”, which assigned a code of 1 (once a month or less) and “occasionally” which was assigned a code of 3 (one time per week).

Statistical analysis

Data collected from the quantitative survey were used to calculate percentages of participants who reported any use of a specific drug in the past six months and their frequency of use ranging from never to more than once a day. Estimates of the number of current drug users for each of seven drug classes (marijuana, heroin, crack, cocaine, crystal/methamphetamine, inhalants, and tranquilizers) based on self-reports elicited from the quantitative survey and from the qualitative interviews were compared using the McNemar’s tests, and ordinal measures of frequency of use were compared using the Wilcoxon test, Spearman’s rank order test, and Cohen’s kappa statistics. According to Landis and Koch (1977), empirical concordance levels for kappa range from 0 < .20 = weak, 0.21 ≤ 0.40 = sufficient, 0.41 ≤ 0.60 = good, 0.61 ≤ 0.80 = excellent, and 0.81 ≤ 1.00 = almost perfect. We also calculated percent agreement (i.e., use or non-use identified by both forms of data collection) as a categorical measure of concordance obtained by the two data collection methods, and then used chi-square tests to compare concordance by age (less than 35 years old versus 35 years or older), gender, monthly income (less than 3500 pesos versus 3500 or more pesos), and level of education (none or some primary, completed primary school, some secondary, completed secondary school or higher).

Results

Among 428 FSWs and their intimate male partners who completed baseline surveys, median age was 36 years (interquartile range [IQR]: 31–42). Thirty-six percent reported having a monthly income under $200 USD (2500 pesos). Recent drug use in the entire cohort was common, with participants reporting in the quantitative surveys using heroin (62%), methamphetamines (31%), marijuana (37%), tranquilizers (24%), cocaine (19%), crack (14%), and inhalants (8%) in the past six months. Among 41 couples completing qualitative interviews (n=82 individuals), quantitative sample characteristics mirrored those of the overall cohort and there were no significant differences in the number of users of any of the seven drugs (Table 1).

Table 1.

Prevalence of drug use by quantitative survey or qualitative interview

Drug Total Sample
(n = 428)		 Survey
(n = 82)		 Interview
(n=82)		 Combined
(n=82)
n % n % n % n %
Marijuana 158 36.9 28 34.1 9 11.0*** 29 35.4
Heroin 267 62.4 53 64.6 55 67.1 57 69.5
Crack 59 13.8 15 18.3 12 14.6 19 23.2
Cocaine 83 19.4 15 18.3 6 7.3* 17 20.1
Methamphetamine 134 31.3 25 30.5 23 28.0 28 34.1
Inhalants 34 7.9 8 9.8 5 6.1 8 9.8
Tranquilizers 101 23.6 25 30.5 15 18.3* 30 36.6
Open in a new tab
*

p < 0.05,

**

p < 0.01,

***

p < 0.001 for differences between quantitative survey and qualitative interview percentages.

There were numerous instances of individuals who reported drug use in quantitative surveys who did not report the same use during their qualitative interviews, including reports of marijuana (n = 20), heroin (n = 2), crack (also known as piedra) (n = 7), cocaine (n = 9), crystal/methamphetamine (n = 5), inhalants (used exclusively by participants in Ciudad Juarez; n = 3), and tranquilizers (primarily ritrovil; n = 15). Differences consistent with underreporting in qualitative interviews were statistically significant for marijuana (p < 0.001), cocaine (p = 0.02) and tranquilizers (p = 0.04). At the same time, 5 qualitative study participants who reported tranquilizer use in the past 6 months during the qualitative interviews, 4 who reported heroin use, 4 who reported crack use, 3 who reported crystal/ methamphetamine use, 2 who reported cocaine use, and 1 who reported marijuana use did not mention such use on the quantitative survey. When the two data sets were combined, there was a slight but statistically non-significant increase in the numbers of tranquilizer, heroin, and crystal/methamphetamine users.

A comparison of frequency of drug use by data collection method is described in Table 2. The percentages of participants who reported using drugs in the past six months in the qualitative interviews but failed to indicate how often they used drugs ranged from 17% for crystal/methamphetamine to 80% for tranquilizers. The reported frequency of drug use was significantly less during qualitative interviews than quantitative surveys for marijuana (z = −4.17, p < 0.001), cocaine (z = −2.04, p = 0.04), and tranquilizers (z = −3.37, p = 0.001). Mirroring these results from the Spearman rank order test, kappa values ranged from weak for tranquilizers (k = .11) and marijuana (k = .15) to excellent for heroin (k = .62) and crystal/methamphetamine (k = .67).

Table 2.

Frequency of drug use by quantitative survey or qualitative interview

Drug Wilcoxon Spearman Kappa Missing frequency
data from
Qualitative
Interviews
Z score p rho K SE (1) n % of
drug
users (2)
Marijuana 4.17 <0.001 .45*** .15*** .06 4 44.4
Heroin 0.58 0.57 .87*** .62*** .07 12 21.8
Crack 1.81 0.07 .40*** .36*** .67 7 58.3
Cocaine 2.04 0.04 .28* .09 .07 3 50.0
Methamphetamine 0.98 0.33 .80*** .67*** .08 4 17.4
Inhalants 1.16 0.10 .72*** .65*** .18 2 40.0
Tranquilizers 3.37 0.001 .36** .11* .05 12 80.0
Open in a new tab
*

p < 0.05;

**

p <0.01;

***

p < 0.001

(1) SE = standard error

(2) Identified from qualitative interviews

The associations between concordance of quantitative survey and qualitative interview reports of drug use and demographic characteristics are provided in Table 3. Highest concordance (percent agreement) was observed for inhalant, heroin and crystal/methamphetamine use, and lowest concordance was observed for marijuana and tranquilizer use. Adults older than 35 years of age were significantly more likely than younger adults to be consistent in reporting use of marijuana (χ2 = 5.19, p = 0.02), crystal/ methamphetamine (χ2 = 4.99, p = 0.026) and inhalants (χ2 = 4.27, p = 0.04) in the past 6 months. Concordance was unrelated to level of education but was significantly associated with higher monthly income for reported crack use (χ2 = 5.55, p = 0.02) and lower monthly income for reported marijuana (χ2 = 4.61, p = 0.03) and crystal/methamphetamine (χ2 = 4.67, p = 0.03) use.

Table 3.

Demographic characteristics of 82 qualitative interview participants and relationship of concordance (percent agreement) in responses to questions regarding drug use

Variable Marijuana Heroin Crack Cocaine Meth Inhalants Tranquilizers
% % % % % % %
Age
  18–35 63.4 97.6 87.8 85.4 82.9 92.7 68.3
  36+   85.4* 87.8 85.4 85.4   97.6* 100.0 82.9
Sex
  Male 70.7 95.1 82.9 87.8 95.1 95.1 80.5
  Female 78.0 90.2 90.2 82.9 85.4 97.6 70.7
Education
  None/Some 71.4 95.2 85.7 90.5 95.2 90.5 66.7
Primary
  Primary School 77.8 92.6 92.6 85.2 77.8   100.0 74.1
  Some Secondary 82.4 94.1 82.4 82.4 100.0   100.0 94.1
  Secondary/Higher 64.7 88.2 82.4 82.4 94.1 94.1 70.6
Monthly income
  < 3500 pesos 85.0 95.0 77.5 85.0 97.5 97.5 75.0
  3500+ pesos   63.3* 90.5   95.2* 85.7 83.3 95.2 76.2
Total Concordance 74.4 92.7 86.6 85.4 90.2 96.3 75.6
Open in a new tab
*

p < 0.05;

**

p < 0.01

Discussion

In this socially marginalized population of Mexican female sex workers and their non-commercial male partners, we found significant underreporting of marijuana, cocaine and tranquilizer use during semi-structured interviews. Several studies have noted underreporting of injection drug use during semi-structured qualitative interviews (Islam et al., 2012; Macalino et al., 2002), while other studies have found no difference between the two methods (Ray, Hart, & Chin, 2011; Williams et al., 2000). In this study, survey questions were read aloud to study participants and recorded directly on the computer by a study investigator. Hence, the differences in prevalence may not be attributed to the participant’s reactions to the interviewer or to the opportunity to spend more time contemplating responses, as has been previously suggested (Del Boca & Noll, 2000). However, differences in how the questions were framed may account for differences in responses. In the survey, participants were asked three identically worded questions regarding drug use in the past six months for each class of drugs and given a choice of two possible answers (yes/no) to the first question, time since last use to the second question, and seven possible answers (ranging from less than once a month to more than once a day) to the third question. In the qualitative interview, participants were asked four open-ended questions during the course of the interview (e.g., Do you use drugs?; Does your partner use drugs?; and if yes, What kinds of drugs do you and/or your partner use? and How often?). However, drug-related questions were often raised at various points throughout the qualitative interviews and not necessarily in the same order. Moreover, frequency of use was not always asked by the interviewer or was volunteered by the participant without prompting by the interviewer. Also, while the quantitative survey asked about drug use in the past six months, participants were asked about what they were currently using, raising the possibility that a participant may have been using six months ago or less but was not using at the present time. Hence, in contrast to the survey questionnaire, the relative lack of structure in drug use assessment during qualitative interviews and the ways the questions were framed may have accounted for the underreporting during the interviews.

We also found that underreporting occurred with some drugs but not others. Highest concordance between reporting methods was observed for inhalant, heroin, and crystal/ methamphetamine use, whereas it was lowest for tranquilizer and marijuana use. Compared to heroin, inhalant and methamphetamine, marijuana and tranquilizer use may have been perceived as either less addictive or having fewer consequences, which may have contributed to underreporting because they were viewed during the interview as less important or less relevant by either the interviewer or interviewee. Alternatively, the emphasis of the study in general and the qualitative interviews in particular on risk behaviors engaged by couples may have led to an over-reporting of forms of drug use that are usually performed together by both partners within a couple (i.e., heroin injection) (authors, 2013) and underreporting of forms of drug use that are usually performed individually (i.e., marijuana and tranquilizer use). This possibility is supported by a post hoc analysis of prevalence and frequency of use alone and with partner in the entire cohort. Participants reported using heroin and crystal/methamphetamine with their partners at least two or three times a month in 65.1 and 71.9 percent of cases, respectively, while cocaine, marijuana, and tranquilizers were used with partners at least two or three times a month in only 23.2, 29,1, and 33.7 percent of cases, respectively.

In this study, adults aged 36 years and older exhibited higher levels of concordance than younger adults for all classes of drugs except for heroin and crack and significantly higher levels of concordance for marijuana and crystal/methamphetamine use. Studies of other drug-using populations have reported age-related differences in concordance between semi-structured qualitative interviews and quantitative survey data and attributed such differences to social pressures of peers, characteristics of the examiner, and perceived threat to confidentiality (Magura, et al., 1987; Schwarz, 1999). Other studies of this population (authors, 2013) have demonstrated a greater reluctance of female sex workers to disclose risk behavior, with and without the presence of their non-commercial partner, out of concern that it might harm the relationship. However, in this study, we found no gender differences in concordance, which is consistent the findings of studies of other drug using populations, (Digiusto, Seres, Bibby, & Batey, 1996; Magura et al., 1987). We also found no differences in concordance by educational attainment. We did observe associations between monthly income and concordance in reports of marijuana and crack use, but they were in different directions with a positive association with crack use and a negative association with marijuana use. Although education and income are often used as measures of socioeconomic status, they may carry less value in a population widely viewed in Mexico as well as the United States as living on the margins of society.

Our results point to the limitations of relying solely on data obtained from semi-structured interviews to assess individual and group patterns of drug use. Consistent with the finding of previous studies (Islam et al., 2012), such methods lead to an underestimate of the prevalence of certain forms of drug use. Given that qualitative methods like semi-structured interviews are intended to provide a depth of understanding to complement the breadth of understanding afforded by quantitative methods (Cresswell & Plano Clark, 2011; Palinkas, 2014; Patton, 2002), it is not surprising that such methods are poorly suited for collecting data that are meaningful when only when described in a quantitative fashion. However, the findings from this study also point to the value of utilizing data from multiple sources to obtain more accurate and comprehensive assessments of patterns of drug use than is afforded by single methods alone. While reliance on data from qualitative interviews alone would have led to a failure to identify 61 instances of drug use in this sample, reliance on the quantitative survey alone would have led to a failure to identify 19 instances on drug use. Such instances of drug use are at risk of being undercounted if reliance on assessment of numbers of current drug users is placed solely on the standardized survey questionnaire. Moreover, the high concordance in reports of heroin, crystal/methamphetamine and inhalant use suggests that qualitative data can reliably be used to confirm the validity of self-reports of certain types of drug use obtained from quantitative surveys. In this instance, the focus of the parent study on couples, their risk behaviors and their relationships, may have provided an appropriate context for the use of qualitative data to assess numbers of current users and frequency of classes of drugs that are often used as a couple.

There were several limitations to this study that should be acknowledged when evaluating our findings. Although we found no differences in the demographic characteristics and percentage of current users as determined by the survey data between the subsample of study participants who completed the qualitative interviews (n = 82) and participants who only completed the quantitative survey (n = 342), the purposeful nature of sampling the qualitative subsample limits our ability to generalize our findings to the entire group of study participants or to other female sex workers and their non-commercial partners. We utilized snowball sampling methods (in addition to targeted sampling conducted by outreach workers at selected venues) to reach this “hidden” or understudied population of FSWs and their partners, who had never been sampled before in this setting. Although other methods of recruitment such as RDS may have benefits over snowball sampling in terms of assessing dependencies between recruiters and recruiters, RDS has not been successful in recruiting female drug users in this setting or sex workers in this or other settings. Although snowball sampling generates dependencies between individuals, particularly in regard to the variables such as drug use, our binational study team, including local outreach workers, deemed this form of sampling to be more appropriate than RDS. Our objective was to compare different methods of data collection, rather than to provide estimates of drug use prevalence within the larger community of FSWs or to generalize our findings to the rest of the population. Nevertheless, the linkages between study participants, especially those couples who are in a noncommercial relationship, suggest that the assumptions of independence underlying the use of inferential statistics may have been violated. Second, although we found statistically significant differences in concordance across the different drug classes and between older and younger adults and men and women, these findings must be interpreted in the context of the relatively small sample and multiple comparisons. Finally, in the absence of biological data, it is difficult to ascertain the validity and reliability of either form of self-report data. Our position is not that a yes answer to a question related to current use in one assessment is a valid yes, even if there is no report in the other assessment. Rather, it is that use of both assessments provides a more comprehensive picture of drug use and potentially helps to address reasons for underreporting of drug use in either assessment. We recognize that in counting a yes response to either an interview question or survey item increases the likelihood of generating a false positive. However, in adopting a targeted approach to prevention, which would certainly be the case with this particular population, sensitivity of a screening tool would take higher priority than specificity. There is always a trade-off between optimizing sensitivity and specificity, but if the goal is to capture as many people as possible who are using drugs (e.g., to refer them to drug treatment, or to screen for HIV/STIs), then erring on the side of conservatism by maximizing sensitivity and sacrificing specificity seems to be reasonable.

Conclusion

In a high risk population of female sex workers and their non-commercial male partners, our assessment of the numbers of current drug users and frequency of use through semi-structured interviews led to underreporting of the number of marijuana and tranquilizers users and the frequency of marijuana, cocaine, and tranquilizer use when compared to standardized, quantitative survey methods. However, although this study focused specifically on the issue of concordance between prevalence and frequency of drug use data obtained from structured surveys and semi-structured interviews, our findings support the conclusions raised by other drug researchers (Lopez et al., 2013; Wagner et al., 2012) that the integration of mixed methods results may provide a sum that is greater that the individual qualitative and quantitative parts. While not specifically designed to replace epidemiologic methods in assessing prevalence and frequency, qualitative methods can be an important tool in eliciting a more comprehensive understanding of patterns of drug use in high risk marginalized populations and in confirming the validity of data on these patterns obtained through quantitative methods.

Footnotes

Declaration of Interest

The authors report no conflicts of interest

Contributor Information

Dr Lawrence A Palinkas, Email: palinkas@usc.edu, University of Southern California, School of Social Work, Montgomery Ross Fisher Building, Los Angeles, 90089-0411 United States.

Dr Angela Robertson Bazzi, Email: abazzi@bu.edu, Boston University, Community Health Sciences, Boston, United States.

Dr Jennifer L Syvertsen, Email: syvertsen.1@osu.edu, The Ohio State University, Anthropology, Columbus, United States.

Dr Monica D Ulibarri, Email: mulibarri@ucsd.edu, University of California, San Diego, Psychiatry, La Jolla, United States.

Mr Daniel Hernandez, Email: d7hernan@gmail.com, University of California, Davis, School of Medicine, Sacramento, United States.

Dr Gudelia Rangel, Email: grangel2009@gmail.com, El Colegio de la Frontera Norte, Tijuana, Mexico.

Dr Gustavo Martinez, Email: gmartinez@femap.org.mx, Federacion Mexicana de Asociaciones Privadas, Cuidad Juarez, Mexico.

Dr Steffanie A Strathdee, Email: sstrathdee@ucsd.edu, University of California, San Diego, Medicine, La Jolla, 92093-0507 United States.

References

  1. Bongers IMB, Van Oers JAM. Mode effects on self-reported alcohol use and problem drinking: Mail questionnaires and personal interviewing compared. Journal of Studies on Alcohol. 1998;59:280–285. doi: 10.15288/jsa.1998.59.280. [DOI] [PubMed] [Google Scholar]
  2. Catania J, Gibson D, Chitwood D, Coates T. Methodological problems in AIDS behavioral research: Influence on measurement error and participation bias in studies of sexual behavior. Psychological Bulletin. 1990;108:339–362. doi: 10.1037/0033-2909.108.3.339. [DOI] [PubMed] [Google Scholar]
  3. Colón HM, Robles RR, Sahai H. The validity of drug use responses in a household survey in Puerto Rico: Comparison of survey responses of cocaine and heroin use with hair tests. International Journal of Epidemiology. 2001;30:1042–1049. doi: 10.1093/ije/30.5.1042. [DOI] [PubMed] [Google Scholar]
  4. Cresswell JW, Plano Clark VL. Designing and conducting mixed method research. 2nd. Thousand Oaks, CA: Sage; 2011. 2007. [Google Scholar]
  5. Day E, Best D, Cantillano V, Gaston RL, Nambamali A, Keaney F. Measuring the use and career histories of drug users in treatment: Reliability of the Lifetime Drug Use History (LDUH) and its data yield relative to clinical case notes. Drug and Alcohol Review. 2008;27:171–177. doi: 10.1080/09595230701829504. [DOI] [PubMed] [Google Scholar]
  6. Del Boca FK, Noll JA. Truth or consequences: The validity of self-report data in health services research. Addiction. 2000;95(Suppl 2):S347–S360. doi: 10.1080/09652140020004278. [DOI] [PubMed] [Google Scholar]
  7. Denis C, Fatseas M, Beltran V, Bonnet C, Picard S, Combourieu I, Auriacombe M. Validity of the self-reported drug use section of the Addiction Severity Index and associated factors used under naturalistic conditions. Substance Use and Misuse. 2012;47:356–363. doi: 10.3109/10826084.2011.640732. [DOI] [PubMed] [Google Scholar]
  8. Digiusto E, Seres V, Bibby A, Batey R. Concordance between urinalysis results and self-reported drug use by applicants for methadone maintenance in Australia. Addictive Behaviors. 1996;21:319–329. doi: 10.1016/0306-4603(95)00064-x. [DOI] [PubMed] [Google Scholar]
  9. Fendrich M, Johnson TP. Race/ethnicity differences in the validity of self-reported drug use: Results from a household survey. Journal of Urban Health. 2005;82(2 Suppl 3):iii67–iii81. doi: 10.1093/jurban/jti065. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Fendrich M, Johnson TP, Sudman S, Wislar JS, Spiehler V. Validity of drug use reporting in a high-risk community sample: a comparison of cocaine and heroin survey reports with hair tests. American Journal of Epidemiology. 1999;149:955–962. doi: 10.1093/oxfordjournals.aje.a009740. [DOI] [PubMed] [Google Scholar]
  11. Gibson EK, Exner H, Stone R, Linquist J, Cowen L, Roth EA. A mixed methods approach to delineating and understanding injection practices among clientele of a Victoria, British Columbia needle exchange program. Drug and Alcohol Review. 2011;30(4):360–365. doi: 10.1111/j.1465-3362.2010.00219.x. [DOI] [PubMed] [Google Scholar]
  12. Houck JM, Forcehimes AA, Gutierrez ET, Bogenschutz MP. Test-retest reliability of self-report measures in a dually diagnosed sample. Substance Use and Misuse. 2013;48:99–105. doi: 10.3109/10826084.2012.731674. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Islam MM, Topp L, Conigrave KM, van Beek I, Maher L, White A, Day CA. The reliability of sensitive information provided by injecting drug users in a clinical setting: Clinician-administered versus audio computer-assisted self-interviewing (ACASI) AIDS Care. 2012;24:1496–1503. doi: 10.1080/09540121.2012.663886. [DOI] [PubMed] [Google Scholar]
  14. Kab V, Evans J, Sansothy N, Stein E, Claude-Couture M, Maher L Young Women’s Study Collaborative. Testing for amphetamine-type stimulant (ATS) use to ascertain validity of self-reported ATS use among young female sex workers in Cambodia. Addiction Science and Clinical Practice. 2012;7:11. doi: 10.1186/1940-0640-7-11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. King M, Bruner G. Social desirability bias: A neglected aspect of validity testing. Psychology and Marketing. 2003;17:79–103. [Google Scholar]
  16. Landis JR, Koch GC. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–174. [PubMed] [Google Scholar]
  17. Ledgerwood DM, Goldberger BA, Risk NK, Lewis CE, Price RK. Comparison between self-report and hair analysis of illicit drug use in a community sample of middle-aged men. Addictive Behaviors. 2008;33:1131–1139. doi: 10.1016/j.addbeh.2008.04.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Lopez AM, Bourgois P, Wenger LD, Lorvick J, Martinez AN, Kral AH. Interdisciplinary mixed methods research with structurally vulnerable populations: Case studies of injection drug users in San Francisco. International Journal of Drug Policy. 2013;24:101–109. doi: 10.1016/j.drugpo.2012.12.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Macalino GE, Celentano DD, Latkin C, Strathdee Sa, Vlahov D. Risk behaviors by audio computer-assisted self-interviews among HIV-seropositive and HIV-seronegative injection drug users. AIDS Education and Prevention. 2002;14(5):367–378. doi: 10.1521/aeap.14.6.367.24075. [DOI] [PubMed] [Google Scholar]
  20. Magura S, Goldsmith D, Casriel C, Goldstein PJ, Lipton DS. The validity of methadone clients’ self-reported drug use. International Journal of the Addictions. 1987;22:727–749. doi: 10.3109/10826088709027454. [DOI] [PubMed] [Google Scholar]
  21. Magura S, Kang SY. Validity of self-reported drug use in high risk populations: A meta-review. Substance Use and Misuse. 1996;31:1131–1153. doi: 10.3109/10826089609063969. [DOI] [PubMed] [Google Scholar]
  22. Mayhew S, Collumbien M, Qureshi A, Platt L, Rafiq N, Faisel A, Hawkes S. Protecting the unprotected: Mixed-method research on drug use, sex work, and rights in Pakistan’s fight against HIV/AIDS. Sexually Transmitted Infections. 2009;85(Suppl. 2):ii31–ii36. doi: 10.1136/sti.2008.033670. [DOI] [PubMed] [Google Scholar]
  23. McLellan E, MacQueen KM, Neidig JL. Beyond the qualitative interview: Data preparation and transcription. Field Methods. 2003;15(1):63–84. [Google Scholar]
  24. Miles M, Huberman M. Qualitative data analysis: An expanded source-book. 2nd. Thousand Oaks, CA: Sage; 1994. [Google Scholar]
  25. Mirzazadeh A, Nedjat S, Navadeh S, Haghdoost A, Mansournia M, Mohammad K. HIV and related risk behaviors among female sex workers in Iran: Bias-adjusted estimates from the 2010 National Bio-Behavioral Survey. AIDS and Behavior. 2014;18:S19–S24. doi: 10.1007/s10461-013-0548-3. [DOI] [PubMed] [Google Scholar]
  26. Onwuegbuzie AJ, Teddlie C. A framework for analyzing data in mixed methods research. In: Tashakkori A, Teddlie C, editors. Handbook of mixed methods in the social and behavioral sciences. Thousand Oaks, CA: Sage; 2003. pp. 351–385. [Google Scholar]
  27. Padgett DK. Qualitative methods in social work research. 2nd. Los Angeles: Sage; 2008. [Google Scholar]
  28. Palinkas LA. Qualitative and mixed methods in mental health services and implementation research. Journal of Clinical Child and Adolescent Psychology. 2014;43:851–861. doi: 10.1080/15374416.2014.910791. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Palinkas LA, Horwitz SM, Chamberlain P, Hurlburt M, Landsverk J. Mixed method designs in mental health services research. Psychiatric Services. 2011;62(3):255–263. doi: 10.1176/ps.62.3.pss6203_0255. [DOI] [PubMed] [Google Scholar]
  30. Patton MQ. Qualitative research and evaluation methods. 3rd. Thousand Oaks, CA: Sage; 2002. [Google Scholar]
  31. Pollini RA, Lozada R, Gallardo M, Rosen P, Vera A, Macias A, Strathdee SA. Barriers to pharmacy-based syringe purchase among injection drug users in Tijuana, Mexico: A mixed methods study. AIDS and Behavior. 2010;14(3):679–687. doi: 10.1007/s10461-010-9674-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Ray LA, Hart EJ, Chin PF. Self-rating of the effects of alcohol (SRE): Predictive utility and reliability across interview and self-report administrations. Addictive Behaviors. 2011;36:241–243. doi: 10.1016/j.addbeh.2010.10.009. [DOI] [PubMed] [Google Scholar]
  33. Rhodes T, Wagner K, Strathdee SA, Shannon K, Davidson P, Bourgois P. Structural violence and structural vulnerability within the risk environment: Theoretical and methodological perspectives for a social epidemiology of HIV risk among injection drug users and sex workers. In: O’Campo P, Dunn JR, editors. Rethinking social epidemiology: towards a science of change. New York: Springer Science+Business Media; 2012. pp. 205–230. [Google Scholar]
  34. Rice ER, Holloway IW, Barman-Adhikari A, Fuentes D, Brown CH, Palinkas LA. A mixed methods approach to network data collection. Field Methods. 2014;26:252–268. doi: 10.1177/1525822X13518168. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Sandelowski M. Tables of tableaux? The challenges of writing and reading mixed methods studies. In: Tashakkori A, Teddlie C, editors. Handbook of mixed methods in the social and behavioral sciences. Thousand Oaks, CA: Sage; 2001. pp. 321–350. [Google Scholar]
  36. Schwarz N. Self reports: How the questions shape the answers. American Psychologist. 1999;54(2):93–105. [Google Scholar]
  37. Singer M, Stopka T, Siano C, Springer K, Barton G, Khoshnood K, Heimer R. The social geography of AIDS and hepatitis risk: Qualitative approaches for assessing local differences in sterile-syringe access among injection drug users. American Journal of Public Health; 2000;90(7):1049–1056. doi: 10.2105/ajph.90.7.1049. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Solbergsdottir E, Bjornsson G, Gundmundsson LS, Tyrfingsson T, Kristinsson J. Validity of self-reports and drug use among young people seeking treatment for substance abuse or treatment. Journal of Addictive Diseases. 2004;23(1):29–38. doi: 10.1300/J069v23n01_03. [DOI] [PubMed] [Google Scholar]
  39. Teddlie C, Tashakkori A. Major issues and controversies in the use of mixed methods in the social and behavioral sciences. In: Tashakkori A, Teddlie C, editors. Handbook of mixed methods in the social and behavioral sciences. Thousand Oaks, CA: Sage; 2003. pp. 3–50. [Google Scholar]
  40. Wagner K, Davidson PJ, Pollini RA, Strathdee SA, Palinkas LA. Reconciling incongruous qualitative and quantitative findings in mixed methods research: Exemplars from research with drug using populations. International Journal of Drug Policy. 2012;23:54–61. doi: 10.1016/j.drugpo.2011.05.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Wilcox CE, Bogenschutz MP, Nakazawa M, Woody G. Concordance between self-report and urine drug screening data in adolescent opiod dependent clinical trial participants. Addictive Behaviors. 2013;38:2568–2574. doi: 10.1016/j.addbeh.2013.05.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Williams ML, Freeman RC, Bowen AM, Zhao Z, Elwood WN, Gordon C, Signes C. A comparison of the reliability of self-reported drug use and sexual behaviors using computer-assisted versus face-to-face interviewing. AIDS Education and Prevention. 2000;12:199–213. [PubMed] [Google Scholar]
  43. Williams RJ, Nowatzki N. Validity of adolescent self-report of substance use. Substance Use and Misuse. 2005;40:299–311. doi: 10.1081/ja-200049327. [DOI] [PubMed] [Google Scholar]
  44. Windhusen TM, Somoza EC, Singal B, Kim S, Horn PS, Rotrosen J. Measuring outcome in cocaine clinical trials: A comparison of sweat patches with urine toxicology and participant self-report. Addiction. 2003;98:317–324. doi: 10.1046/j.1360-0443.2003.00311.x. [DOI] [PubMed] [Google Scholar]

RESOURCES