Response: Assessing the Instruments

Jack Blaine: Except for the ASI, the instruments discussed in the article (Samet et al., 2007) were developed for research. The AUDADIS, as the authors note, was created for use in large-scale epidemiological research in the general population. It’s a very fine, fully structured interview. A research assistant with no clinical training can make valid diagnoses. The SCID and the PRISM were developed to be administered by experienced clinicians. These interviews take longer, though, so they’re not practical for larger-scale research. The CIDI was also developed for epidemiological research, though we use it in clinical settings.

Dace Svikis: In research, we use these formal structured assessment instruments to characterize our population. As the authors illustrate with their example of depression studies, we need to know what substance use and other psychiatric diagnoses participants have so that our analyses don’t miss any factors that affect outcomes.

Robert Forman: The use of formal structured assessment instruments is absolutely critical for research. Even in research, however, they must be used judiciously, because patients’ reactions to them can affect clinical outcomes and possibly confound the interpretation of results (e.g., Clifford, Maisto, and Davis, 2007). To me, the justification for using them in clinical practice hasn’t been established.

Blaine: Except for the ASI, the instruments discussed in the article take too much time to administer to be practical for routine clinical use. They also often require extensive training and monitoring for fidelity to ensure that clinicians continue to use the measures in line with the practice guidelines.

Data quality and therapeutic alliance

Svikis: The move toward the clinical use of standardized instruments was spearheaded by concern that some clinicians were not doing a thorough job of collecting patient information. Dr. A. Thomas McLellan has said that this was his original motivation for developing the ASI. He had noted that, for example, some clinicians were asking patients about depression, and others weren’t. Standardized instruments make sure that all clinicians obtain a uniform set of basic patient data. At the same time, we need to recognize that there are many really good drug counselors with excellent clinical skills. Those counselors can feel handcuffed by structured instruments.

Forman: You have to be extremely talented to go through a structured interview, even one that is semi-structured, and maintain a therapeutic alliance. Fully structured interviews have a tendency to become robotic, when what clients want is someone who’s going to listen and understand them—someone they can open up to.

Svikis: The way fully structured instruments work, if you follow the administration guidelines, when a patient doesn’t understand what you’re saying, you can only repeat the question verbatim. You can’t add information or paraphrase. That makes it hard to establish empathy. It is easier to establish a rapport with semi-structured instruments like the ASI.

Another feature of fully structured instruments that makes them aversive is that you have to repeat all the same questions for each potential drug of abuse. In a clinical setting where the average person uses six or seven substances, that makes for a very long and tedious interview. It’s quite different from using the instruments in an epidemiological context with a focus on the general population, where most people use one or two substances regularly or none at all.

Blaine: There is no question that it’s easier to establish a relationship with a patient with a less structured instrument than with a more rigid one. Still, the very act of gathering information shows that you are interested in finding out about a person’s problems. I think that builds rapport.

Forman: In a clinical situation, no matter what instrument or tool you use, patients may not be ready to be frank about all their issues in a first interview. Assessment is a process, not something that’s done once and for all at intake.

Svikis: One good example is the group of people who come to a program because of a legal issue, such as an arrest for driving while intoxicated. At intake, they tend to minimize and deny problems, to the point where their professed symptoms might not meet criteria for any substance use disorder. Two months into treatment, however, they might be more willing to provide more information about their alcohol and drug use.

Blaine: That’s where the rapport comes in.

Diagnosis and treatment planning

Blaine: The ASI, which is probably the most widely used assessment instrument, includes several domains besides alcohol and drugs that help determine what kinds of services the patient needs. All the other instruments discussed in the article really focus on making DSM-IV diagnoses of mental disorders. In a clinical setting, it makes more sense to start with a screener—say, the Hopkins Symptom Checklist 90 (Lipman, Covi, and Shapiro, 1979) or Brief Symptom Inventory (Derogatis and Spencer, 1982)—than with one of these highly complex instruments. Then, if you need to make a diagnosis after that, you can administer the appropriate section of one of these interviews.

Svikis: In today’s drug treatment settings, making a specific diagnosis of drug abuse or drug dependence doesn’t usually change what you’re going to do for a patient or how you’re going to do it. To date, we have little evidence that empirically based treatments produce different outcomes for patients with one of these diagnoses versus the other.

Forman: Most payers are interested only in knowing, does this person need to be hospitalized, and if not, does this person meet the criteria for dependence in very gross terms? For that, it’s sufficient to use a checklist of the seven DSM-IV criteria, such as the one developed by Dr. Robert Brooner (for more information, contact Dr. Brooner at RKBrooner@aol.com).

Blaine: Nevertheless, the information on severity that these instruments tap into is potentially useful. It is possible that people on the high end of the severity spectrum will benefit more from intense and different services than those on the low end. One reason we lack evidence for this is that research tends to focus, rightly, on the people in greater distress, those at the higher end.

Svikis: That’s true. When I ran a program for pregnant drug-dependent women, the women with lower severity indicators were the ones most likely to leave treatment against medical advice in the first couple of days. Our clinical impression was that they self-selected out because their addictions were less severe than those of their peers, and the program intensity level was too high for them. They took a look around and said, “I don’t need to be here, yet.”

Forman: I am sure the majority of treatment providers would agree and would like to match each patient’s level of care to the severity of his or her problem. However, in most cases, programs’ resources are too constrained to offer multiple levels of care. Usually, the only level of care decision that gets made is whether the patient should be treated in rehab, an outpatient clinic, or a hospital.

Blaine: The article raises an important treatment planning issue: that of distinguishing the mood symptoms of withdrawal from those of major depression. Programs often deal with that by waiting 2 weeks to a month, until the patient has finished the withdrawal process, and then administering an instrument like the Beck Depression Inventory to see if the symptoms are still there.

Svikis: The issue of withdrawal versus depression is very complex. Many programs ask counselors to make this distinction based solely on the ASI, which covers only symptoms in the past 30 days. I believe additional screening tools may improve the validity and reliability of such decisions. Even in research settings, we can’t always tease these two entities apart. Among the difficulties is the fact that the length of substance-related mood symptoms can vary by substance. Also, some outpatients continue to use drugs, which can lead to a recurrence or exacerbation of such symptoms.

ASI issues

Svikis: I think this would be a good time to evaluate how accurately the ASI is being administered and whether it is being used as intended when the field mandated its adoption. A primary reason for having a standardized tool across programs is to make it possible for payers to compare programs and see which ones have better retention rates, whether client characteristics at baseline are associated with different outcomes, and so forth. That all sounds really good in theory. In practice, as an ASI trainer, I’ve seen that in a typical class with 30 counselors, at least 7 or 8 will say they’ve been using the instrument for 6 months or more at their site, yet this is their first training. I’ve also been struck by how few supervisors attend training, and I wonder how they can monitor the aptness with which it is used for treatment planning without having been trained. I am not sure we do the field a service by telling people that you can compare data across programs, when we’re not implementing sufficient infrastructure to make sure the instrument is being used in a way that produces reliable and valid data.

Forman: It is not easy to become proficient with the ASI, and even under the best of circumstances, interviewers can “drift” from their training and begin to administer and score the measure in ways that were not intended. As well, people who use the ASI often don’t have time to review the results and make good clinical judgments around them. To address this problem, as the article mentions, NIDA’s Blending Initiative provides training materials for translating information acquired with the ASI into treatment plans.

Svikis: There is now a patients’ self-administered version of the ASI (www.asimv.com), and Tom McLellan and Deni Carise are currently testing a product that takes the ASI data input from clinicians and actually prints out a treatment plan. I think those are great steps in the right direction.

Forman: The pressures on clinicians’ time are so great that often the only way they can do something new is to stop doing something else. I think any tool that patients can self-administer has a much better chance of being integrated into clinical practice.

Svikis: In the big picture, notwithstanding the problems we’ve mentioned, the field has taken a significant step forward with the widespread adoption of the ASI. We now collect at least a minimum quantity of data on everybody concerning the domains that have been identified to be most relevant for treatment planning and treatment outcome. This is a big accomplishment and a big step forward.