Validation of a screening tool for labor and sex trafficking among emergency department patients

Abstract

Objective

Patients with labor and sex trafficking experiences seek healthcare while and after being trafficked. Their trafficking experiences are often unrecognized by clinicians who lack a validated tool to systematically screen for trafficking. We aimed to derive and validate a brief, comprehensive trafficking screening tool for use in healthcare settings.

Methods

Patients were randomly selected to participate in this prospective study based on time of arrival. Data collectors administered 5 dichotomous index questions and a reference standard trafficking assessment tool that requires 30 to 60 minutes to administer. Data collection was from June 2016 to January 2021. Data from patients in 5 New York City (NYC) emergency departments (EDs) were used for tool psychometric derivation, and data from patients in a Fort Worth ED were used for external validation. Clinically stable ED adults (aged ≥18 years) were eligible to participate. Candidate questions were selected from the Trafficking Victim Identification Tool (TVIT). The study outcome measurement was a determination of a participant having a lifetime experience of labor and/or sex trafficking based on the interpretation of the reference standard interview, the TVIT.

Results

Overall, 4127 ED patients were enrolled. In the derivation group, the reference standard identified 36 (1.1%) as positive for a labor and/or sex trafficking experience. In the validation group, 12 (1.4%) were positive by the reference standard. Rapid Appraisal for Trafficking (RAFT) is a new 4‐item trafficking screening tool: in the derivation group, RAFT was 89% sensitive (95% confidence interval [CI], 79%–99%) and 74% specific (95% CI, 73%–76%) and in the external validation group, RAFT was 100% sensitive (95% CI, 100%–100%) and 61% specific (95% CI, 56%–65%).

Conclusions

The rapid, 4‐item RAFT screening tool demonstrated good sensitivity compared with the existing, resource‐intensive reference standard tool. RAFT may enhance the detection of human trafficking in EDs. Additional multicenter studies and research on RAFT's implementation are needed.

1. INTRODUCTION

1.1. Background

Human trafficking is the recruitment, harboring, transportation, provision, and/or obtaining of a person, by the use of force, fraud, and/or coercion, for the purpose of labor and/or sexual exploitation. ¹ Cases have been reported in all 50 states and Washington, DC. ² People who have been trafficked seek healthcare both during and after their trafficking experience ³ , ⁴ , ⁵ and can experience trafficking‐related health and social consequences for years after a trafficking experience. ⁶ , ⁷ , ⁸ Clinicians often fail to recognize trafficking experiences among their patients because they lack trafficking‐specific training and comprehensive (labor and sex trafficking), validated screening tools. ³ , ⁴ , ⁵ , ⁹

1.2. Importance

Multiple healthcare institutions and organizations have developed trafficking screening tools, ⁹ , ¹⁰ yet only the Child Sex Trafficking Screening Tool is validated for healthcare settings and it is explicitly for sex trafficking screening of adolescents presenting with specific chief complaints. ¹¹ The social, and, in some states, legislative pressure to identify trafficking has led institutions to use unvalidated trafficking screening tools. ¹² , ¹³ Unvalidated tools lack sensitivity and specificity with the potential to negatively impact individual patient care and ultimately public health data collection. ¹⁴

1.3. Goal of this investigation

This study's objective was to derive and externally validate a comprehensive screening tool, Rapid Appraisal for Trafficking (RAFT), to facilitate adult patient disclosures about labor and sex trafficking experiences.

2. METHODS

This prospective study's methods, including site selection, are described in extensive detail in a prior publication. ¹⁵ The Mount Sinai Health System and John Peter Smith Hospital institutional review boards deemed the study exempt from review. No identifying information was collected; participants verbally consented, and the findings are reported here following the Standards for Reporting of Diagnostic Accuracy reporting guideline (eTable1 in the Supporting Information).

2.1. Study population

Clinically stable ¹⁵ adult (aged ≥18 years) ED patients seeking care at any of the participating hospitals, speaking any language, were eligible for participation. In the 5 New York City (NYC) EDs (Mount Sinai Hospital, Mount Sinai West, Mount Sinai Morningside, Mount Sinai Beth Israel, Elmhurst Hospital; annual censuses 65,000–107,000 adult visits), patients were randomly recruited based on time of arrival between June 2016 to January 2021. Data collection was paused during the March–August 2020 peak of NYC's COVID‐19 infections. The Fort Worth site (John Peter Smith Hospital) data collection (same eligibility criteria; annual census ≈120,000) took place between May 2018 and March 2020.

The prevalence of human trafficking in a general population is unknown. Study sample size was planned based on preliminary prevalence findings of 1.2% to 1.4% after a year of data collection. Planning for a precision of < 0.10 and an instrument that would be at least 80% sensitive to trafficking recognition, enrollment of at least 3667 participants was anticipated.

2.2. Procedures and measures

The Trafficking Victim Identification Tool (TVIT) was the first validated, comprehensive trafficking assessment tool for use in social service settings ¹⁶ and was used as the reference standard. The index test was composed of 5 candidate RAFT items (eFigure1 in the Supporting Information). These were dichotomous questions from the TVIT with the highest odds of predicting a trafficking experience on the TVIT. Prefacing, normalizing language was added to build rapport between the data collector and participant; prefaces were adopted from the Human Trafficking Interview and Assessment Measure, a trafficking assessment tool also derived from the TVIT ¹⁷ and validated for use only among homeless young adults. Index questions were posed by the data collector, and then the participant completed the reference 30‐ to 60‐minute TVIT interview with the same data collector; interviews were completed in the ED. ¹⁸ All participants were offered the opportunity to speak with an ED social worker. ¹⁵ Data were entered directly into Research Electronic Data Capture (REDCap), ¹⁹ , ²⁰ not the electronic health record. Data collectors were members or employees of Mount Sinai Emergency Medicine or John Peter Smith Emergency Medicine research programs or divisions who were trained to interpret TVIT responses. ¹⁵ If participants did not complete the index test, their data would not be used for analysis. If the TVIT interview (reference standard) ended early (eg, for healthcare interventions) but a trafficking experience was already recognized, the data would be used for analysis. If the TVIT interview was not completed and trafficking had not yet been identified, the data would not be used for analysis.

Data from patients in 5 EDs in NYC were used for tool derivation; data from the ED in Fort Worth were used for external validation of RAFT. That is to say, the larger data set (5 NYC EDs) was used for derivation, and the smaller data set (1 Fort Worth ED) was used to validate for generalizability. Data from TVIT interviews were used to determine a lifetime experience of trafficking. ¹⁵ Data about participant language, gender, and age was captured in the interview (self‐report); race, ethnicity, and presenting complaint were self‐reported upon registration, and data collectors documented from view of the electronic health record.

2.3. Psychometric analysis

To identify reliable and valid labor and sexual exploitation screening measures, DS (author 3) used innovations in Testlet Response Theory (TRT) ²¹ to evaluate the following: (1) the performance of items in measuring each of the 5 individual domains (ie, testlets: Force, Fraud, Coercion; Isolation; Labor Exploitation; Harm; Sexual Exploitation) and (2) the performance of items in assessing labor or sexual exploitation experiences. Methods based in TRT allow hierarchical evaluation of multiple subsets of items. Each subset may elaborate a specific content area, but all subset content areas are thought to be related to a larger primary construct. Models provided quantification of each item's ability to distinguish levels of exploitation (discrimination parameter estimate) and the levels of exploitation associated with each item endorsement (threshold parameter estimates). Before fitting item response models, full information maximum likelihood confirmatory factor analysis (CFA) evaluated fit of models to organize the scales’ items. Model fit indexes including Akaike information criterion (AIC), Bayesian information criterion (BIC), and −2 log likelihood, comparative fit indexes, and root mean square error guided decisions.

The Bottom Line.

This study's objective was to derive and externally validate a 4‐question screening tool, Rapid Appraisal for Trafficking (RAFT), to facilitate adult patient disclosures about labor and sex trafficking experiences. When validated with 4127 patients in 6 major emergency departments (EDs), RAFT demonstrated good sensitivity compared with the existing, resource‐intensive reference standard tool. RAFT may enhance the detection of human trafficking in EDs. Additional multicenter studies and research on RAFT's implementation are needed.

The following 4 criteria were established when selecting items for an efficient index of trafficking: (1) minimizing redundancy of the content, (2) ensuring maximum coverage of 5 TVIT domains, (3) selecting items providing strong relationships with overall levels of exploitation, and (4) selecting items that performed similarly (least differential item functioning [DIF]) across examined samples. During DIF analysis, DS examined items across NYC and Fort Worth samples by fitting successive models 1 at a time while using the remaining items to anchor both samples on the same metric. ¹⁹ To quantify the difference in parameter estimates in each sample, DS repeatedly (10,000 times) drew a parameter estimate from each sample posterior distribution and subtracted them.

2.4. Statistical analysis

The psychometrically identified items were predicted on the binary outcome of trafficked/not trafficked in a logistic regression model, and a receiver operator curve (ROC) was computed from this model evaluation. Sensitivity, specificity, and positive predictive value performance from participant response to the items were evaluated for effect of increasing score and analyzed by demographics (gender, race/ethnicity, language of interview, age) as planned. ¹⁵ These computations were conducted by GTL (author 4) using Statistical Analysis System 9.4 (SAS 9.4) software. ²²

3. RESULTS

3.1. Characteristics of study participants

Of the 6290 eligible participants, 3292 ED patients from the 5 NYC EDs (derivation data set) wanted to participate and completed the TVIT interview (Figure 1). The majority of NYC participants identified as women (61%), and the sample was racially and ethnically diverse (Table 1). In the NYC sample, there was a 1.1% prevalence of a lifetime experience of labor and/or sex trafficking, yielding 36 cases: 20 were labor trafficking, and 16 were sex trafficking. Of the participants, 3 with a trafficking experience were in their trafficking situation at the time of the interview; another had just recently left their situation. Of all NYC participants, 186 (5.7%) wanted to speak with an ED social worker; of the 36 participants with a trafficking experience, 12 (33%) opted to meet with the social worker. Participants with a trafficking experience presented to the ED with a variety of complaints (Figure 2). Of the 1677 eligible participants at the Fort Worth site (external validation data set), 835 randomly selected patients wanted to participate and completed the interview (Figure 1). About 54% of the participants identified as men, and a larger proportion identified as White (Table 1). In this sample, 1.4% had a lifetime experience of labor or sex trafficking, with 8 cases being labor trafficking and 4 cases being sex trafficking. In both the NYC and Fort Worth data sets, all participants with a trafficking experience completed the reference standard interview.

FIGURE 1.

The Standards for Reporting of Diagnostic Accuracy flow diagrams to report flow of participants through the study. RAFT, Rapid Appraisal for Trafficking; TVIT, Trafficking Victim Identification Tool

TABLE 1.

Participant demographics

	New York City EDs		Fort Worth ED
	Not trafficked (n = 3256)	Trafficked (n = 36)	Not trafficked (n = 823)	Trafficked (n = 12)
Age
Median, years	45.0	45.0	53.0	49.0
25th percentile	30.0	28.5	42.0	40.0
75th percentile	60.0	55.0	61.0	51.5
Gender, n (%)
Female	1979 (61)	17 (47)	380 (46)	6 (50)
Male	1273 (39)	19 (53)	443 (54)	6 (50)
Other	4 (0.12)	0 (0)	0 (0)	0 (0)
Race, n (%)
African American/Black	1114 (34)	11 (31)	313 (38)	4 (33)
American Indian/Alaska Native	8 (0.25)	0 (0)	1 (0.12)	0 (0)
Asian	32 (0.98)	0 (0)	0 (0)	0 (0)
Native Hawaiian or other Pacific Islander	2 (0.06)	1 (3)	1 (0.12)	1 (8)
White	575 (18)	8 (22)	368 (45)	6 (50)
>1 race	17 (0.52)	0 (0)	2 (0.24)	0 (0)
Other	657 (20)	3 (8)	1 (0.12)	0 (0)
Unknown	163 (5)	2 (6)	2 (0.24)	0 (0)
Ethnicity, n (%)
Hispanic/Latino	753 (23)	10 (28)	137 (17)	1 (8)
Country of birth, n (%)
United States	2478 (76.2)	29 (81)	766 (93)	11 (92)
Other country	774 (23.8)	7 (19)	57 (7)	1 (8)
Years of schooling, n (%)
1–6 years	122 (4)	2 (6)	22 (3)	0 (0)
7–12 years	1189 (37)	21 (58)	439 (53)	8 (67)
>12 years	1932 (59)	12 (33)	360 (44)	4 (33)
Other	12 (0.37)	1 (3)	2 (0.24)	0 (0)
Type of trafficking, n (%)
Labor trafficked		20 (56)		8 (67)
Sex trafficked		16 (44)		4 (33)

Note: Participants self‐identified upon emergency department registration and could select >1 racial and/or ethnic category. Both electronic health record systems use “Hispanic/Latino” rather than the most contemporary and inclusive term, “Latinx.”

FIGURE 2.

Trafficked participants’ chief complaints by city. ED, emergency department; GI, gastrointestinal; SOB, shortness of breath

3.2. Item derivation

In the NYC sample, CFA models AIC, BIC, and log likelihood supported the improved fit of the proposed hierarchical model over a unidimensional model or a model with 5 correlated factors (eTable 2 in the Supporting Information). In the Fort Worth sample, we also observed an improved fit of the hierarchical model over other models. With support for a hierarchical model, we proceeded to describe each item using a 2‐parameter logistic testlet model ²³ and performed tests of differences in parameter estimates for the NYC and Fort Worth samples. ²⁴ In the Supporting Information, eTable3 shows the resulting marginal item discrimination and threshold parameters with 95% credible intervals for the NYC and Fort Worth samples.

Of the 5 candidate items, 4 items were identified (Figure 3) from each of the Labor Exploitation, Harm, Sexual Exploitation, and Force TVIT domains (eTable4 in the Supporting Information). Selected items had the strongest relationships with levels of exploitation within each domain and tended to reflect more severe examples of exploitation. The difference in parameters across settings is presented in eTable2 in the Supporting Information, which also indicates when 95% or more of the comparisons were <0. The “unsafe work” item had slope and threshold parameters that differed in >95% of the comparisons, although absolute differences in parameters suggest that these effects were unlikely to impact overall scores. The relative severity of exploitation reflected by the “forced work” item differed across settings, although again magnitudes of the difference were small.

FIGURE 3.

RAFT Items

3.3. Analyses of items as predictors of trafficking and ROC analysis

Of the 4 items (“forced work,” “threats at work,” and “payment for sex”), 3 were associated with trafficking (eTable5 in the Supporting Information). All 4 items contributed to a robust C‐statistic (area under the curve) equal to 0.90 using data in both data sets (eFigure2 in the Supporting Information). Using data in both data sets, affirmation to any 1 of the 4 items yielded a sensitivity of 92% and a specificity of 72% (Table 2). Sensitivity and specificity for planned subgroups (gender, race/ethnicity, language of interview, age) are shown in eTable6 in the Supporting Information; the range in sensitivity (75%–96%) is good to strong throughout the spectrum of stratified demographics.

TABLE 2.

Sensitivity and specificity by items with positive responses

	No. of positive responses	Sensitivity		Specificity		Positive predictive value		Negative predictive value		Likelihood ratio positive	Likelihood ratio positive	Likelihood ratio negative	LR negative
Site		Estimate	95% CI	Estimate	95% CI	Estimate	95% CI	Estimate	95% CI		95% CI		95% CI
All sites	1	0.92	0.84–0.99	0.72	0.70–0.73	0.04	0.03–0.05	0.99	0.99–1.00	3.22	2.90–3.54	0.12	0.01–0.23
	2	0.77	0.65–0.89	0.91	0.91–0.92	0.10	0.07–0.13	0.99	0.99–0.99	9.01	7.35–10.67	0.25	0.12–0.38
	3	0.38	0.24–0.51	0.98	0.98–0.99	0.20	0.12–0.29	0.99	0.99–0.99	21.54	12.24–30.85	0.64	0.50–0.78
	4	0.23	0.11–0.35	0.99	0.99–0.99	0.69	0.46–0.91	0.99	0.99–0.99	186.95	−3.38 to 377.29	0.77	0.65–0.89
New York City sites	1	0.89	0.79–0.99	0.74	0.73–0.76	0.04	0.02–0.05	0.99	0.99–1.00	3.43	2.99–3.88	0.15	0.01–0.29
	2	0.72	0.58–0.87	0.92	0.91–0.93	0.09	0.06–0.13	0.99	0.99–0.99	9.44	7.22–11.66	0.30	0.14–0.46
	3	0.33	0.18–0.49	0.98	0.98–0.99	0.21	0.11–0.32	0.99	0.98–0.99	24.67	11.17–38.17	0.68	0.52–0.83
	4	0.19	0.07–0.32	0.99	0.99–1.00	0.78	0.51–1.00	0.99	0.98–0.99	316.56	−169.91 to 803.02	0.81	0.68–0.94
Fort Worth site	1	1.00	1.00–1.00	0.61	0.58–0.65	0.04	0.02–0.06	1.00	1.00–1.00	2.59	2.37–2.81	0.00	−0.00 to 0.00
	2	0.92	0.76–1.00	0.88	0.86–0.90	0.10	0.04–0.15	0.99	0.99–1.00	7.54	5.65–9.44	0.09	−0.08 to 0.27
	3	0.50	0.22–0.78	0.97	0.96–0.98	0.18	0.05–0.31	0.99	0.98–0.99	15.24	4.93–25.55	0.52	0.22–0.81
	4	0.33	0.07–0.60	0.99	0.99–1.00	0.57	0.20–0.94	0.99	0.98–0.99	91.44	−35.13 to 218.02	0.67	0.40–0.94

Abbreviation: CI, confidence interval.

4. LIMITATIONS

Although this is the largest study of its kind, given the low prevalence of trafficking experiences, a larger multicenter investigation may offer more information about the proposed tool. In addition, TVIT interrater reliability was not tested; the “raters” were the interviewers and the reference standard interviews were time intensive (30‐60 minutes). Reference standard (TVIT) interrater reliability testing would have been too burdensome on patient participants and too resource intensive.

Another study limitation is that patients that were ineligible for participation may have been at higher risk for having a trafficking experience. For example, patients who were not able to consent to participation in research (including patients that presented with intoxication, or substance use disorder or mental illness complications), could not speak with the interviewer alone, or who presented and were dispositioned in the middle of the night may have been at higher risk of exploitation. Data were not collected during peak COVID‐19 infection rates, as institutional research activities were halted. These exclusion criteria may have decreased the prevalence of trafficking identified and yielded a precision of < 0.11 instead of the goal of < 0.10. In addition, 48% and 50% of the NYC and Fort Worth eligible patients, respectively, declined to participate (Figure 1); it is difficult to assess the direction, if any, of bias introduced by patients who declined participation. RAFT should be validated in other EDs and other kinds of areas, including rural settings, reservations, and free‐standing EDs.

Most participants with a trafficking experience in this study were not in their situation at the time of the interview, but previous literature demonstrates that a connection to health‐based or community‐based organization (CBO) services may still be desirable or beneficial, and the ED is the most accessible route of connection for many survivors. ³ , ⁴ , ⁵ , ⁶ , ⁷ , ⁸ , ²⁵ Finally, RAFT's specificity is 72%. This means that some people will screen falsely positive, but the patient implications of a false screen on RAFT are fairly benign and may still be beneficial to such patients: patients who screen positive can be offered the opportunity to connect with local antitrafficking organizations for further assessment. If a CBO determines that the person does not have a trafficking experience, they can be connected to other relevant resources.

5. DISCUSSION

To our knowledge, this is the first study to externally validate a trafficking screening tool for use in a healthcare setting. ⁹ It is useful for clinicians to be able to rapidly screen for both labor and sex trafficking as screening for only 1 trafficking type may lead to under‐detection of the other type of trafficking, and using separate tools can be time‐consuming in under‐resourced EDs. Because many with a trafficking experience present for healthcare and experience complications and sequelae but are not recognized by care teams, ³ , ⁴ , ⁵ , ⁶ , ⁷ , ⁸ , ²⁵ trafficking recognition in healthcare settings is of increasing federal, state, and institutional priority, with some states even mandating trafficking identification among patients. ¹² , ¹³ RAFT may allow patients with trafficking experiences the opportunity to tell their care teams about their experiences and finally be connected to social work in the ED and/or CBOs, receive trafficking‐specific medical care when relevant, ²⁶ and make reports to the National Human Trafficking Hotline ²⁷ and/or law enforcement. Patients may accept all, none, or some of these connections or services. ²⁸ Another advantage of RAFT is that it expands communities’ capacity to accurately assess the number of people with a trafficking experience. Understanding which populations are experiencing trafficking, and in what proportions, can help communities equitably allocate resources based on evidence.

6. USING RAFT

Although RAFT demonstrates good sensitivity, EDs should first develop trafficking response protocols with trafficking survivors and local trafficking response organizations. ²⁹ , ³⁰ In light of a low‐trafficking prevalence in a general population sample, a positive RAFT screen does not necessarily indicate a trafficking experience. However, this tool may be used to recognize which patients may need expert assessment for antitrafficking or other resources and/or interventions. In‐depth assessment need not occur in the ED and is likely best performed by local antitrafficking experts, for example, community‐based antitrafficking organizations. Likewise, a negative RAFT screen does not rule out a trafficking experience. A patient may not feel comfortable telling their truth to the interviewer, may not have understood the questions, or the trafficking experience may not have been captured by RAFT. Still, trafficking survivor–perspective literature demonstrates that patients appreciate when healthcare teams support their path to well‐being by directly engaging on this topic (rather than self‐administered surveys). ³¹ , ³²

Psychometric analysis demonstrates that the 4 RAFT questions perform well together, but the ideal administration is not yet known. As a 4‐item screener, it takes about 2 minutes to administer, and optimal strategies for where, when, and by whom RAFT should be asked will vary in different EDs. In some sites, triage may be the most private and best setting for these questions; at other sites, the primary nursing assessment may be more appropriate. We anticipate that most of the training on RAFT would be around a site's protocol for what happens when a patient screens positive.

CONFLICT OF INTEREST

This study was supported by an Emergency Medicine Foundation grant and a Robert Wood Johnson Foundation grant (75200; principal investigator, Makini Chisolm‐Straker). David Strong, George T. Loo, and Cindy Clesca received salary support or consulting fees for data management or analysis. No other author received consulting fees or honoraria, fees for participation in review activities such as data monitoring boards or statistical analysis, payment for writing or reviewing the manuscript, and/or provision of writing assistance, medicines, equipment, or administrative support in connection with this study.

AUTHOR CONTRIBUTIONS

Makini Chisolm‐Straker is the primary investigator for the study, she and Elizabeth Singer, Lynne D. Richardson, George T. Loo, and David Strong conceived of this analysis and designed it together. Cindy Clesca, Naomi Alanis, James d'Etienne managed the data collection. David Strong and George T. Loo managed the analysis. Makini Chisolm‐Straker, David Strong, and George T. Loo are the primary authors of the article. All of the authors participated in the revision of the manuscript. Makini Chisolm‐Straker takes responsibility for the paper as a whole.

Supporting information

Supporting Information

Biography

Makini Chisolm‐Straker, MD, MPH, is an Associate Professor of Emergency Medicine at the Icahn School of Medicine at Mount Sinai in New York, NY, USA.

Chisolm‐Straker M, Singer E, Strong D, et al. Validation of a screening tool for labor and sex trafficking among emergency department patients. JACEP Open. 2021;2:e12558. 10.1002/emp2.12558