The Dynamic Assessment and Referral System for Substance Abuse (DARSSA): Development, Functionality, and End-user Satisfaction

Abstract

The Dynamic Assessment and Referral System for Substance Abuse (DARSSA) conducts a computerized substance abuse assessment; prints personalized summary reports that include tailored substance abuse treatment referral lists; and, for individuals who provide authorization, automatically faxes their contact information to a “best match” substance abuse treatment provider (dynamic referral). After piloting the program and resolving problems that were noted, we enrolled a sample of 85 medical patients. The DARSSA identified 48 (56%) participants who were risky substance users, many of whom had not been identified during their routine medical assessment. Mean satisfaction scores for all domains ranged between “Good” to “Excellent” across patients, nurses, doctors, and substance abuse treatment providers The median completion time was 13 minutes. Of the 48 risky substance using participants, 20 (42%) chose to receive a dynamic referral. The DARSSA provides a user-friendly, desirable service for patients and providers. It has the potential to improve identification of substance abuse in medical settings and to provide referrals that would not routinely be provided. Future studies are planned to establish its efficacy at promoting treatment initiation and abstinence.

1. Introduction

Use and abuse of psychoactive substances, including tobacco, alcohol, and other drugs, kills more Americans than any other class of health behavior (Mokdad et al., 2004). The well-recognized health consequences of risky substance use have prompted recommendations for healthcare providers to be proactive in screening, brief intervention, and referral to treatment (SBIRT). However, implementation of SBIRT practices within many healthcare settings has fallen short of recommendations (Anda et al., 1987; Frank et al., 1991; Goldstein et al., 1997; Marbella et al., 2003; United States Department of Health and Human Services, 2005). For example, although tobacco SBIRT models like the National Cancer Institute’s Five A’s -- Ask, Advise, Assess, Assist, Arrange -- are known to be effective, healthcare providers often fail to implement them (Anda et al., 1987; Goldstein et al., 1997; Marbella et al., 2003). Neglect of counseling and referral may be even more common in settings not traditionally viewed as suitable for preventive health efforts, such as emergency departments (EDs) and trauma units (Bernstein et al., 2006; Boudreaux et al., 2005; Wei and Camargo, 2000). This is true despite promotion of SBIRT models for alcohol and drug use in these settings by the Substance Abuse and Mental Health Services Administration (SAMHSA; Substance Abuse and Mental Health Services Administration, 2006), the Society of Academic Emergency Medicine (SAEM; Bernstein and Becker, 2002; D’Onofrio et al., 1998), and the American College of Surgeons (ACS; American College of Surgeons, 2007).

To maximize the changes of SBIRT procedures being successfully implemented in clinical practice, they should be brief, convenient, require little investment of resources, require little or no specialized training, and be perceived as efficacious by providers (Cabana et al., 1999; Goldberg et al., 1993; Ockene et al., 1994; Prochazka et al., 1995; Roche and Freeman, 2004). Traditional strategies often do not meet these requirements, leading to inconsistent adoption. Technological advances such as computerized assessments, personalized feedback reports, and electronic health records hold much potential for facilitating the implementation of SBIRT practices in healthcare settings (Etter and Perneger, 2001; Fiore et al., 1995; Holtz et al., 2001; Krishna et al., 1997; Prochaska et al., 2001; Revere and Dunbar, 2001; Rhodes et al., 2001; Velicer et al., 1999; Zeiler et al., 2002). We have created a computer program designed to facilitate SBIRT called the Dynamic Assessment and Referral System for Substance Abuse (DARSSA). This paper describes the DARSSA’s development, its functionality, and the results of our initial field testing and evaluation.

2. Materials and Methods

2.1. DARSSA Overview

The DARSSA is comprised of three integrated modules: (1) a computerized assessment of substance use and other psychosocial variables, (2) a report generator, and (3) a referral generator. The DARSSA was funded by a Small Business Technology Transfer (STTR) grant from the National Institute on Drug Abuse. STTRs fund partnerships between small business concerns (i.e., Polaris Health Directions, Inc) and research institutions (i.e., Robert Wood Johnson Medical School) and proceed in two phases. Phase 1 is designed to create and pilot test a commercial product that has the potential to improve health, and Phase 2 is designed to evaluate the product’s ability to actually impact important health outcomes. This paper summarizes the DARSSA’s Phase 1 results. The studies were approved by the Institutional Review Boards for Cooper University Hospital and Polaris Health Directions, Inc., and were in accordance with the standards of the Helsinki Declaration of 1975. All participants signed a consent form.

To derive our core design principles, we conducted focus groups and in-depth interviews with 65 healthcare professionals from a variety of specialties representing a range of provider types, as well as 13 tobacco, alcohol, and drug using medical patients.¹ This early end-user input was critical to maximize the goodness of fit between the DARSSA and a broad array of medical settings. The recommendations from all end-users informed the development of the assessment, report, and referral modules.

2.1.1. Assessment module

The assessment module provides for the self-administered assessment of tobacco, alcohol, illicit drug, and licit drug use and abuse. The foundation of the assessment consists of an abbreviated version of the Addiction Severity Index (ASI; McLellan et al., 1980), the most widely used addiction assessment instrument in both research and clinical settings (McLellan et al., 2006). The DARSSA’s Alcohol and Drug scales: (1) provide a reliable evaluation of the presence and severity of addiction, (2) are brief, (3) are psychometrically strong (Butler et al., 2001; Grissom et al., 2004; McLellan et al., 1980), (4) have Polaris data for over 15,000 substance using patients, and (5) provide for linkage to the Polaris assessment system for substance abuse treatment settings (Polaris Chemical Dependency). Because tobacco use is only superficially assessed in the original ASI, the investigators constructed a Tobacco scale that mirrored the ASI’s Alcohol and Drug subscales.

The response-adaptive programming logic ensures that an individual is presented only with the questions appropriate to his or her situation. For each substance use category (tobacco, alcohol, and drugs), the DARSSA assesses lifetime use, use in the past 30 days, the amount of use, last use, dollars spent, substance-related problems and consequences, readiness to change, readiness to enter treatment, and withdrawal symptoms. For patients reporting any readiness to change, the DARSSA assesses interest in a dynamic referral to a substance abuse treatment provider, a process that is further described in the Section 2.1.3. In addition to substance use and abuse, the DARSSA assesses (1) demographic and socio-economic variables, including insurance status and carrier; (2) general health status; (3) psychiatric history, including hospitalizations, diagnoses, and psychotropic use; (4) current mental health symptoms, including screeners for depression, labile mood, anxiety, post-traumatic stress, suicidal ideation, and suicidal intent; and (5) recent interpersonal victimization.

2.1.2. Report generator

The report generator produces three reports using the data collected through the assessment: (1) the Healthcare Provider Report, (2) Patient Feedback Report, and (3) Referral Provider Report. The one-page Healthcare Provider Report summarizes the substance use parameters deemed most important for clinical decision-making by healthcare providers during our early end-user interviews, including lifetime use, current use, amount, last use, readiness to change, and referral information.² It also summarizes the other psychosocial history variables assessed, like psychiatric history and depression. This report was designed to be included in the patient’s medical chart and was specifically formatted to comply with third-party provider documentation requirements to support billing efforts.

The patient receives a Patient Feedback Report consisting of one to six pages divided into three sections: (1) the Patient Assessment Summary, (2) Quick Fact Sheets, and (3) Motivation Review Worksheet. The Patient Assessment Summary can be up to two pages and provides motivationally tailored messages based on the assessment and includes a personally tailored referral list for each substance class endorsed. Up to three Quick Fact Sheets are automatically generated based on the patient’s reports of specific substance use. They are each one-page and include information deemed most important by both providers and patients during the early end-user interviews, including definitions of addiction, health risks of using, benefits of quitting, a list of empirically based treatments, and additional educational resources. Finally, the one-page Motivation Review Worksheet is printed for all risky substance users and is comprised of a decisional balance exercise, a behavior change plan, and a cue-management exercise.

The Referral Provider Report, which is faxed to the “best match” referral provider for patients choosing a dynamic referral, is nearly identical to the Healthcare Provider Report. In addition to the summary of the assessment, it contains the patient’s contact information to allow the referral provider to contact the patient.

2.1.3. Referral generator

The referral generator draws upon an extensive, nationwide provider library of accredited substance abuse treatment facilities and independent practitioners maintained by Polaris. The dynamic referral library is generated using site-specific information created at the time of installation. For the purposes of the current study, we established agreements with seven facilities, which allowed for at least one dynamic referral option for every possible permutation in our algorithm. The participating dynamic referral providers agreed to contact the patient within five days of receiving the fax to complete an initial telephone screening, discuss treatment options, and, if appropriate, schedule an intake assessment. If patients were not appropriate, referral providers assisted them in accessing other treatment resources.

For every patient who reports risky substance use (see definition under Section 2.2.4.1), the referral generator automatically prints a tailored referral list matched to each substance use category reported, insurance status, geographic location, and level of care. The level of care algorithm is based on well-established criteria set forth by the American Society of Addiction Medicine (ASAM; American Society of Addiction Medicine, 2001), as well as direct input from substance abuse treatment providers during the design phase. For the purposes of our study, the treatment sites in the referral library were divided based on two meta-levels of care: inpatient and outpatient. The treatment sites preferred to differentiate more precisely for the patient the specific level of care that was most appropriate, such as traditional versus intensive outpatient. In order to designate the meta-level to which the patient should be referred (inpatient, outpatient), we used an algorithm that included type of substance used, severity of abuse, experience of withdrawal symptoms, and previous treatment experience.

Patients endorsing risky substance use are given the option to choose a dynamic referral. If this option is chosen, the DARSSA faxes a referral to a “best match” provider that is selected using the indicators described above. The referral consists of three pages: (1) a HIPAA-compliant cover sheet, (2) an electronically signed release of information, and (3) the Referral Provider Report, which includes the patient’s personal contact information.

2.2. Methods

2.2.1. Procedure

Once the prototype of the DARSSA was available for use, we evaluated it during Alpha testing with 35 patients recruited from an urban ED and an inpatient Progressive Care Unit (PCU).³ The DARSSA assessment and the evaluation interviews were completed in English. Using the data derived from the Alpha test, changes were made in the DARSSA to rectify minor problems noted with the branching logic, referral algorithm, Healthcare Provider Report format, and assessment text formatting.

Using the modified version, we enrolled a second sample of patients recruited from the same two settings during the Beta test. To gain experience with the two different implementation models suggested by the focus groups, we began with universal screening, which involved administering the DARSSA to all consenting participants. We then reverted to a clinically indicated approach, which involved pre-screening patients verbally for substance use and administering the DARSSA exclusively to patients who screened positive for risky use of at least one substance. Importantly, both methods adhered to universal screening principles; they simply differed in the method used to carry out the initial screening (computer vs. verbal). Figure 1 provides an example of the clinical pathway for the DARSSA in the ED setting. We discontinued enrollment for the Beta test once we accrued a combined total of 20 dynamic referrals using participants enrolled through both methods of implementation.

Figure 1.

Clinical pathway for patients in the emergency department. This figure is an example of the clinical flow pathway for the ED. In the PCU, the DARSSA was administered after the patient was stabilized and conversational but prior to the patient’s discharge. This model is a suggestion for the ED which can also be modified for other clinical settings. Screening with the DARSSA could occur before the primary medical assessment.

In all cases, the DARSSA was self-administered using a tablet personal computer (PC) mounted to a rolling cart and outfitted with wireless Internet access. This addressed the logistical barrier posed by inconsistent computer access in patient care areas. Several nurses who were interested in taking a more direct role in the actual implementation of the DARSSA were identified at both sites. These nurses underwent a 15 minute training on how to introduce and initiate the DARSSA assessment. When clinical demand permitted, the patient’s nurse, rather than the research staff, introduced the DARSSA, answered questions that arose, and used the Patient Feedback Report to help counsel participants with positive findings. Although we did not expect to enroll all participants in this manner because of the competing clinical demands and the investigational nature of the program, we believed that even a modest portion of nurse-guided administrations would enrich our evaluation with first-hand provider critiques. Research staff closely observed all participants throughout the process and encouraged them to point out confusing items and to describe difficulties navigating the interface. After the participant completed the assessment and reviewed his/her Patient Feedback Report, research staff used semi-structured interviews and standardized satisfaction ratings to assess the participant’s impressions. Additionally, research staff had the participant’s primary nurse and treating physician evaluate the Healthcare Provider Report.

A trained research assistant, blinded to the baseline information, telephoned all participants who reported any risky substance use two weeks after the healthcare visit to assess treatment initiation and to re-assess substance use. For all participants who selected a dynamic referral, research staff contacted the substance abuse treatment provider to which the patient was referred. This allowed us to verify the participant’s self-reported treatment initiation, as well as to identify problems with the dynamic referral process and the referral provider’s satisfaction with the Referral Provider Report.

2.2.2. Setting and population

The DARSSA was implemented in an urban ED and PCU. The ED is an academic, Level I trauma center serving a catchment area of approximately two million people. The annual census is approximately 47,000 visits, 20% of which are admitted to the hospital. The PCU cares for patients transferred from the coronary care unit and low-to-moderate risk cardiac patients admitted directly from the ED. These patients have a wide range of cardiovascular diseases, most prominently coronary artery disease, congestive heart failure, atrial fibrillation, and other rhythm disturbances. The annual PCU census is approximately 6,000 visits, with an average length of stay of three days.

We chose these two settings for three primary reasons. First, substance use and its medical consequences are prevalent within these two settings. Second, both settings represent demanding clinical environments. A computerized SBIRT system that can be integrated into ED and PCU settings will likely be feasible in most medical settings. Finally, the ED and PCU are very different in their clinical procedures, staffing, medical focus, and patient characteristics, allowing us to examine how the DARSSA functioned across two diverse settings.

2.2.3. Participant selection

Enrollment occurred Monday through Friday, during the hours of 9:00 AM to 5:00 PM. In both settings, adult patients were approached at the bedside by the research assistant after the patient had been evaluated by a physician and was medically stable. To avoid selection bias, a standard enrollment script was used which reassured patients that prior computer experience was not needed to participate. Exclusion criteria for both settings included severe illness or distress (e.g., intubation, vomiting, pain), cognitive insufficiency (e.g., dementia, psychosis, altered consciousness), and insurmountable language barriers.

2.2.4. Measures

2.2.4.1. DARSSA

The DARSSA assessment was described in Section 2.1.1. Using the assessment data, a dichotomous variable representing any risky substance use was defined as: (1) any tobacco use in the past 30 days, (2) alcohol consumption greater than the National Institute of Alcohol Abuse and Alcoholism (NIAAA; National Institute of Alcohol Abuse and Alcoholism, 2003) threshold for typical adults of 7 drinks per week or 3 drinks on any one occasion for females and 14 drinks per week or 4 drinks on any one occasion for males, or (3) non-medical use of any illicit or licit drugs in the past 30 days. We also created two other variables reflecting polysubstance use. The first was a sum of all the different substances reported in the past 30 days, and the second was a categorical variable: no risky substance use, monosubstance use, or polysubstance use. The DARSSA’s automatic time stamps were used to calculate assessment completion times in minutes.

2.2.4.2. Satisfaction assessments

Satisfaction assessments were completed with patients, healthcare providers, and treatment referral providers. These assessments consisted of semi-structured interviews which assessed the positive and negative impressions of the program, reports, and referrals, and obtained suggestions for improvement. Patients, healthcare providers, and treatment referral providers also rated their satisfaction across a variety of domains using a five-point scale (1 = Very Poor, 2 = Poor, 3 = Fair/Average, 4 = Good, 5 = Excellent). Sample items for the patient satisfaction assessment included: clarity of instructions; ability to read the words on the screen; ability to understand the assessment items; length of assessment; comfort in answering honestly; ability to understand the report; usefulness of the report; effect of materials on motivation to change; and usefulness of referral information. Sample items for the healthcare provider satisfaction assessment included: usefulness, understandability, and length of the report and appropriateness of the referral. Also, providers were asked about whether the Healthcare Provider Report added important information to their clinical assessment and whether it impacted their management of the patient. Healthcare providers indicated whether the DARSSA had generated referrals that would otherwise not have been given during the course of routine clinical care. Finally, substance abuse treatment providers who received a dynamic referral provided satisfaction ratings. Sample items for the treatment referral provider satisfaction assessment included: usefulness, understandability, and length of the report; effect on their ability to counsel the patient; user-friendliness; appropriateness of the referral; and accuracy of contact information.

2.2.4.3. Research assistant process log

The research assistant completed a process log for each participant, documenting any problems with executing critical tasks, the solutions applied, and the outcome.

2.2.4.4. Follow-up assessment

A research assistant contacted all participants who reported risky use of at least one substance. Each participant was asked if he or she had contacted any providers on the referral lists and attended an initial assessment, along with the number of treatment sessions attended. For each substance reported at baseline, the research assistant assessed self-reported use, including abstinence since the medical visit, efforts to decrease use, and amount of use. Any participant that reported a pending appointment with a treatment provider was re-assessed two weeks later. Treatment initiation was defined as being admitted for inpatient drug or alcohol treatment or having an initial outpatient appointment followed by a second appointment within a 14-day period (Garnick et al., 2006; McCorry et al., 2000). Finally, a research assistant contacted the referral providers for each dynamic referral and assessed whether: (1) the provider had received the faxed referral, (2) the provider had contacted the participant, (3) the participant had scheduled an initial assessment, and (4) the number of sessions the participant attended.

2.2.5. Data analysis

Descriptive statistics, including proportions, means, standard deviations, medians, and interquartile ranges (IQR), were calculated for all variables. Our primary quantitative outcomes consisted of (1) end-user satisfaction ratings, (2) completion times for the assessment module, and (3) the proportion of patients with risky substance use that chose to receive a dynamic referral. A priori, we established a target mean satisfaction score of ≥ 4.00 on the 5-point scale for each item assessed by each end-user, a completion time of ≤ 15 minutes for 95% of patients, and a dynamic referral acceptance rate of ≥ 20% among risky users. We examined predictors of longer completion time using the independent samples t-tests and one-way ANOVAs. The completion time predictor variables included whether the assessment had been interrupted, polysubstance use status, and dynamic referral status. All statistical tests were 2-tailed and alpha was set at p < 0.05. Failure to meet our pre-established targets meant we would modify the DARSSA to address the identified weakness and conduct a re-assessment prior to efficacy testing.

Fostering treatment initiation and abstinence are important goals of the DARSSA. However, because this was a Phase 1 study designed to establish the DARSSA’s functionality, feasibility, and acceptability, and not to establish its efficacy, treatment initiation and substance use were considered secondary outcomes.

3. Results

Of the 348 patients approached by the research staff, 185 (53%) were too sick or cognitively insufficient to be enrolled, 26 (7%) had an insurmountable language barrier, 42 (12%) refused to participate, and 95 (27%) consented to participate. The 42 patients who refused to participate did so primarily because they felt ill or were otherwise engaged with visitors or medical procedures. Of the 95 patients who consented to participate, four were discharged, admitted, or taken for medical diagnostics before they could complete the assessment. Six were enrolled during the clinically indicated phase, pre-screened verbally, and not administered the DARSSA because they did not report any risky substance use. The remaining 85 participants completed the DARSSA. The flow of patients through the Beta testing is shown in Figure 2.

Figure 2.

Patient flow for the DARSSA Beta testing. This figure illustrates the flow of patients through the Beta testing. Each patient was only offered one dynamic referral to reduce completion time and avoid confusion that might arise from multiple dynamic referral offers. Consequently, only 14 of the 33 smokers were offered a dynamic referral for tobacco, because the remainder met the criteria for an offer of an alcohol or drug dynamic referral. Likewise, all of the 13 patients who abused both alcohol and drugs only received one offer for a dynamic referral.

Of the 85 participants who completed the DARSSA, 63 (75%) were enrolled in the ED and 22 (26%) were enrolled in the PCU. Clinical nursing staff, rather than research staff, introduced the DARSSA and counseled the participants using the Healthcare Provider Report for 22 (26%) participants. The average age for participants was 47 (SD = 15). Thirty-eight (45%) were male and 47 (55%) were female. The race/ethnicity of participants was: 38 (46%) White, non-Hispanic; 28 (33%) African American, non-Hispanic; 18 (21%) White, Hispanic; and 1 (1%) Other. Twenty-four (28%) participants had commercial insurance, 18 (21%) had Medicare, 29 (34%) had Medicaid, and 14 (17%) were uninsured. Forty-eight (56%) participants were identified as risky substance users, of which 27 (32%) were identified as polysubstance users.⁴

3.1. Satisfaction

3.1.1. Patient satisfaction

Figure 3 illustrates the satisfaction scores from the 85 participants that completed the DARSSA. Every item exceeded our acceptability criterion of a mean ≥ 4.00, or “Good” (mean rating: 4.66; SC = 0.63).

Figure 3.

Mean DARSSA Satisfaction Scores for Patients (n = 85). The target satisfaction score was 4.00.

3.1.2. Healthcare provider satisfaction

A total of 19 ED physicians, 15 ED nurses, 5 PCU physicians, and 5 PCU nurses evaluated at least one Healthcare Provider Report. Sixty-nine (81%) of the 85 evaluations given to physicians and 74 (87%) of the 85 evaluations given to nurses were completed, yielding a total of 143 (84%) out of 170 possible evaluations. Eighty-two (97%) participants had his or her report evaluated by at least one healthcare provider. Every item exceeded our acceptability criterion of a mean ≥ 4.00 (mean rating: 4.63; SD = 0.46) for both physicians and nurses. The scores ranged from 4.48 (usefulness of report) to 4.75 (appropriateness of referral).⁵ Moreover, for 43 (62%) participants, physicians stated that the Healthcare Provider Report contained important information that had not been assessed during their evaluation; for 8 (12%) participants, physicians stated that the new information actually changed the way they managed the patient. According to their treating physician, 34 (71%) of the 48 risky substance using patients received referral and educational materials that they likely would not have received under routine clinical care.

3.1.3. Substance abuse treatment provider satisfaction

All 7 (100%) of the possible dynamic referral treatment providers were contacted and provided feedback on a total of 20 dynamic referrals. Every item exceeded our acceptability criterion of a mean ≥ 4.00, (mean rating: 4.55; SD = 0.66). The scores ranged from 4.31 (accuracy of contact information) to 4.80 (effect on ability to counsel).⁶

3.2. Completion time

Fifty-four (64%) participants completed the DARSSA in less than 15 minutes, with a median time of 13 minutes (IQR: 7–19 minutes). This fell short of our goal of 95% completion in less than 15 minutes. As expected, interruptions, polysubstance use, and dynamic referrals led to longer completion times.⁷

3.3. Dynamic referrals

Of the 48 risky substance using participants, 20 (42%) chose to receive a dynamic referral. Figure 2 illustrates the proportions within each substance use category choosing to receive a dynamic referral. A greater proportion of patients reporting drug use chose to receive a dynamic referral. Thirteen (65%) of the 20 dynamic referrals were successfully contacted by the treatment program to which they were referred.

3.4. Treatment initiation

We obtained follow-up interviews with 36 (75%) of the 48 risky substance using participants and with treatment providers for all 20 (100%) of the dynamic referrals. A total of four (8%) participants initiated substance abuse treatment within two weeks of their healthcare visit (i.e., initial session with one additional session). One was a smoker who reported contacting a treatment program from her tailored referral list. One was an illicit drug user who reported entering a treatment facility from which he had received treatment in the past but which was not on his tailored referral list. Finally, two were illicit drug users who received dynamic referrals. Treatment initiation was verified by the treatment providers for these two individuals. Seventeen (35%) risky substance users reported having continued intentions to contact a treatment facility on their lists in the future.

3.5. Substance use at follow-up

At follow-up, 11 (33%) of the 33 smokers reported smoking fewer cigarettes per day than reported at baseline, and 8 (24%) reported no cigarette use in the past 14 days. Thirteen (46%) of the 28 risky alcohol users reported drinking fewer average drinks/week than reported at baseline, and 11 (39%) reported no alcohol use in the past 14 days. Eleven (58%) of the 19 drug users reportedly tried to reduce or quit their drug use since their visit, and 6 (32%) reported no drug use in the past 14 days. We considered comparing substance use for patients choosing a dynamic referral versus those who did not (i.e., those receiving only the tailored referral list). However, the findings would not be meaningful due to the small sample size.

4. Discussion

In designing the DARSSA, we relied heavily upon extensive early end-user input. Recommendations gleaned from our focus groups and in-depth interviews were incorporated into the DARSSA’s earliest design specifications and continued to influence further modifications throughout the field tests. The strengths identified during the field tests overlapped strongly with our 10 core design principles, suggesting the design principles were well-reflected in the DARSSA. The DARSSA demonstrated its functionality through the successful assessment and production of personalized reports for 85 patients within two demanding hospital settings. The impact of the DARSSA on identifying and assisting substance users was substantial. Based on the healthcare providers’ feedback, the Healthcare Provider Reports provided important information that had not been obtained during routine clinical assessment for 62% of the participants. Moreover, healthcare providers indicated that 71% of the 48 risky substance users would not have received referral and educational materials had it not been for the DARSSA.

Satisfaction ratings from each end-user group exceeded our target of a mean greater than “Good” for every item by a comfortable margin. These ratings suggest that the DARSSA is user-friendly and provides a service viewed as desirable by patients, healthcare providers, and substance abuse treatment providers alike. Healthcare providers generally acknowledged that SBIRT practices are important and want them integrated into clinical care. However, they often lack the tools, training, access, resources, and time to implement them properly. Automated systems such as the DARSSA have the potential to help address these barriers.

Despite the positive feedback for the DARSSA program, there were significant barriers to implementation. In the ED setting, the major barriers related to the heavy clinical demands, multiple healthcare providers, fast pace, and patient acuity. In the PCU setting, the major barriers related to the large proportion of elderly patients who had problems with vision, general literacy, and computer literacy. In both settings, interruptions of the assessment were common, though these interruptions rarely prevented completion of these assessments.⁸ Our qualitative evaluations from the majority of healthcare providers, both in the focus groups and during the field tests, suggested a clinically indicated model was preferred. While both methods actually applied universal screening principles, the initial verbal screening used during the clinically indicated approach was viewed as more efficient than having to introduce every patient to the DARSSA to be screened. Verbal screening followed by DARSSA administration only among those reporting risky substance use may help to accommodate some of the most important barriers of integrating automated assessment systems, including competing clinical demands. Providers acknowledged, however, that there were strengths associated with universal DARSSA screening, including improved standardization of screening across healthcare providers. Our findings pertaining to implementation preference were based primarily on qualitative data. The small sample size rendered comparisons of quantitative satisfaction ratings infeasible. Future studies should examine the relative effectiveness and barriers to implementation for both models.

The assessment took longer than anticipated for some patients. Sixty-four percent of our sample completed the assessment in less than 15 minutes rather than our goal of 95%. However, most patients reported being highly satisfied with the length of the assessment. The patients who took the longest to complete the assessment were the ones who were most in need of the DARSSA, including those who had polysubstance use and those who wanted dynamic referrals. Our original projections of completion time did not take into account the items added based on the healthcare providers’ recommendations. Providers desired a more general psychosocial assessment, rather than one focused exclusively on substance use, because this is what healthcare providers are asked to do for clinical, billing, and regulatory purposes. This increased breadth of content translated into a longer assessment than originally planned but a more useful healthcare provider report.

Considering the resource investment required to build and maintain an accurate, up-to-date dynamic referral provider library, we were particularly interested in how many patients with risky substance use would choose to receive a dynamic referral. The overall acceptance rate among risky substance users was 42%, ranging from 13% for risky alcohol drinkers to 67% for drug users. These figures confirm the early opinions of both healthcare providers and patients that the dynamic referral capacity is a valuable feature for an automated substance use screening system. However, the fact that only 65% of these patients were able to be reached by the facilities to which they were referred, and only two entered treatment, suggests that interest does not automatically translate into treatment initiation. A common barrier in contacting substance abusing patients is the lack of reliable phone service. To help account for this barrier, all dynamically referred individuals received the referral provider’s contact information on their Patient Assessment Summary with instructions to call the provider should they not hear from the program within five days. This allowed the individual patient to follow-up with the substance abuse treatment facility if he or she missed the call or had only intermittent access to a telephone.

Studies dating back to 1983 have consistently found that the assessment of substance use can be effectively computerized and that the psychometric characteristics of such programs are comparable to paper-and-pencil tests and clinical interviews (Butler et al., 2001; Kobak et al., 1996; Skinner and Allen, 1983). However, the use of technological means for assisting with referrals and linkage to specialized treatment services remains an emerging area of investigation. Considering the lack of convenient access to substance abuse counselors, social workers, or other mental health professionals in most medical settings, computerized assistance with identifying and linking patients to treatment holds promise for facilitating treatment initiation. The DARSSA treatment initiation rate of 8% within two weeks of the healthcare visit among the 48 risky substance users is difficult to evaluate. We do not know how this rate would compare to treatment-as-usual, and we did not have a control group. There are no published studies using comparable methods, monitoring periods, and samples. The closest published study is Bernstein and colleagues (1997). They reported that approximately 50% of 257 alcohol and drug abusing ED patients who were assessed by a peer-counselor reported keeping an appointment with a substance abuse treatment facility or attending a 12-step meeting after their ED visit. However, their sample and methods differed markedly from ours. The sample represented a narrow spectrum of patients who were highly motivated for treatment (i.e., only 8% of the total alcohol and drug abusing sample screened), were followed for up to 90 days, and were provided a facilitated referral. The peer-counselor called treatment facilities, scheduled appointments, and arranged for transportation. Consequently, evaluating how the DARSSA compares to treatment as usual or in-person referral facilitation awaits further study using comparable samples and study protocols.

A recent example of a successful computerized referral system is the Computer Assisted System for Patient Assessment and Referral (CASPAR; Carise et al., 2005; Gurel et al., 2005). The problems and needs of clients presenting for an admission assessment at a substance abuse treatment program were assessed using a computerized version of the ASI developed by the Drug Evaluation Network System (DENS ASI; Carise et al., 1999). Local resources pertinent to substance abusing clients were extracted from an extensive computerized list of human service resources, including domains such as psychological, legal, housing, financial, employment, medical, family/social services, and education. In a randomized trial, the clients in the CASPAR group had more individualized treatment plans, were provided with more appropriate services, and were more likely to complete treatment in comparison to the standard care group (Carise et al., 2005). The results support the idea that using computerized assessments and referral databases can provide patients with better matched service referrals than routine care.

Although the basic principle behind the DARSSA and the CASPAR are similar and both use technology to help link substance using patients to resources, there are important differences. Whereas the DARSSA is intended for use in general medical settings, the CASPAR is intended for use in substance treatment settings. The DARSSA can be viewed as a “front end” package, helping individuals access specialized substance abuse treatment services, while the CASPAR can be viewed as a “back end” package, helping individuals in specialized treatment settings access further community resources. The advantages of the DARSSA include an automated referral system, broader application to multiple settings (e.g., occupational and education), and shorter assessment time. The DARSSA takes a median time of 13 minutes to complete, while the DENS ASI takes approximately 50 to 60 minutes for the full administration (Carise et al., 1999).

Feedback from end-users and direct observation by research staff identified several enhancements which should be considered for computerized systems such as the DARSSA. The most important include: (1) creating an interface specifically designed for tablet PCs and touch screen use to reduce data entry errors; (2) creating versions of the assessment and patient reports in other languages, like Spanish; (3) making the suicide question an optional component, to be decided upon by each site; (4) creating a process that allows the patient to review his/her assessment summary prior to finalization to allow the corrections of errors; (5) creating a provider registration module to facilitate entry and updating of substance treatment provider profiles; and (6) designing an interface that will allow the assessment results to be communicated directly to the patient’s electronic health record.

4.1. Limitations

Considering the fact that potential participants knew they were agreeing to complete a computerized assessment, patients who agreed to participate might have been more computer literate than those who refused. This selection bias would conceivably make the enrolled participants more favorably predisposed to the program than the population from which we sampled, inflating the satisfaction ratings. We attempted to counteract this by reassuring patients during our initial approach that computer knowledge was unnecessary. Based on our observations and participants’ comments, many of our participants had low or no computer literacy. However, it is possible that some residual bias remained. Our sample may have not represented the population in other ways as well. For example, because of our exclusion criteria, our sample was under-represented by very ill patients. The importance of this limitation is mitigated by the fact that the DARSSA is not likely to be used with such patients clinically. Unfortunately, we were unable to obtain data on patients who were missed, ineligible, or refused, preventing an analysis to compare our sample with our universe of patients. The demographic breakdown closely matches administrative statistics.

Another limitation of the DARSSA involved the tobacco assessment. It was patterned directly after the ASI questions for alcohol and drug use but was not formally validated. In our sample, patients did not report any difficulty with the tobacco items, and the resulting smoking prevalence rate of 39% was highly concordant with our previously reported prevalence rate of 40% in our ED patient population (Boudreaux et al., 2005).

It would have been ideal to have 100% of the DARSSA introductions and patient counseling led by the medical staff to more closely approximate the clinical environment. However, full clinical integration is not usually feasible or desirable in early evaluation research. Despite enthusiasm from most providers, the ED and PCU had not yet embraced the DARSSA as part of routine clinical care and viewed it (accurately) as an investigational procedure. Because of the investigational nature of the study, administering the DARSSA required more training and effort than would normally be required in the clinical setting. This created additional disincentives to integration into routine clinical care.

A longer follow-up window would have provided more information on the proportion of participants that entered treatment and on the proportion of those who reported abstinence that eventually relapsed. Over one-third (35%) of risky substance users reported continued intentions to follow up with referrals at a later date. Additional treatment services could have occurred after the 2-to-4 week follow-up window. Future studies should follow participants over a longer period to assess the short- and long-term impact on treatment involvement and abstinence.

Finally, we validated self-reported treatment initiation with the treatment facilities for dynamically referred patients, but we did not validate patient-reported abstinence with records from the treatment facilities or through biochemical means. This was not an efficacy trial, and substance use represented a secondary endpoint. Future efficacy trials should consider biovalidation of self-reported abstinence.

4.2. Conclusions

The DARSSA is an innovative program designed to assist in the identification, counseling, and connection of risky substance using individuals to appropriate treatment resources. Healthcare providers, patients, and substance abuse treatment providers exhibited strong satisfaction with the program. In particular the DARSSA helped to provide physicians with additional information not assessed in routine care and provided patients with referrals and educational materials they would not have otherwise received. Of the patients classified as risky substance users, 42% chose the dynamic referral option, suggesting that a notable portion of patients found this to be a valuable aspect of the program. Despite these positive findings, there are legitimate barriers to implementing such programs in the clinical setting that must be addressed proactively. Phase 2 of the DARSSA will add features to make it more easily integrated into clinical settings and more effective at fostering treatment initiation and abstinence.