The Treatment Effectiveness Assessment (TEA): an efficient, patient-centered instrument for evaluating progress in recovery from addiction

Abstract

The fields of addiction medicine and addiction research have long sought an efficient yet comprehensive instrument to assess patient progress in treatment and recovery. Traditional tools are expensive, time consuming, complex, and based on topics that clinicians or researchers think are important. Thus, they typically do not provide patient-centered information that is meaningful and relevant to the lives of patients with substance use disorders. To improve our ability to understand patients’ progress in treatment from their perspectives, the authors and colleagues developed a patient-oriented assessment instrument that has considerable advantages over existing instruments: brevity, simplicity, ease of administration, orientation to the patient, and cost (none). The resulting Treatment Effectiveness Assessment (TEA) elicits patient responses that help the patient and the clinician quickly gauge patient progress in treatment and in recovery, according to the patients’ sense of what is important within four domains established by prior research. Patients provide both numerical responses and representative details on their substance use, health, lifestyle, and community. No software is required for data entry or scoring, and no formal training is required to administer the TEA. This article describes the development of the TEA and the initial phases of its application in clinical practice and in research.

Introduction

The Affordable Care Act of 2010 in the United States reiterates and strengthens the stipulation of addiction as one of the essential care services to be covered at parity with other medical conditions. There is great anticipation of a new day dawning in addiction treatment, going hand in hand with the development of the electronic health records (EHR) system and brief interventions. The Affordable Care Act (ACA) recognizes the chronic nature of substance use disorders and emphasizes adoption of patient-centered or recovery-oriented, medical home–care models for behavioral health care.^1,2 Addiction is a chronic disease that requires use of a chronic care model and incorporation of patient-reported health outcome measures in order to monitor patient progress in a timely manner and to facilitate long-term recovery.

One issue confronting the recovery-oriented approach advanced by the ACA is how to measure the treatment outcome of these new interventions quickly and comprehensively, consistent with the evolving trend toward improved efficiency. Use of the EHR system has become an essential strategy in improving the efficiency of health care delivery, including routine assessments of patient status and progress. Patient-reported health outcome measures are needed to be implemented within the EHR system to facilitate routine assessments and monitoring of patient progress.³ Researchers in other nations have worked to create efficient outcome assessment tools, such as the Treatment Outcome Profile, which is in use in the United Kingdom for assessment of patients in treatment for substance use disorders.⁴

Traditional measures such as the Addiction Severity Index or the WHO Quality of Life instrument are not cheap (although many such assessments are available for free, the training and scoring software are not) or brief, and they don’t always give clinicians the kind of person-centered information that is meaningful and relevant to the lives of patients with a substance use disorder. The usual approach to measuring treatment outcomes is to construct lengthy questionnaires based on topics that clinicians or researchers think are important and then ask the patients to give answers to those questions. These questionnaires are long and complicated or, perhaps euphemistically, “comprehensive.” However, they may not capture the information that is important in the patient’s life or applicable to his/her life situation. Such traditional questionnaires also cannot be modified to fit each particular patient, and they often require the use of complex algorithms to determine the level of intervention needed.

The most common questionnaire used by addiction clinicians is the Addiction Severity Index (ASI).⁵ The baseline ASI contains 227 questions that query patients in seven functional domains: alcohol use, drug use, medical status, psychiatric health, employment/self-support, family relations, and illegal activity. It takes approximately 45–60 minutes to conduct the structured interview.^5,6 The shorter version, the ASI-Lite, contains 111 items and requires approximately 30–40 minutes to administer. ASI training typically takes two full days. However, many addiction treatment providers have difficulty training new staff (due to a high rate of staff turnover); after receiving the ASI’s intensive training, it is always questionable whether trainees are able to administer the ASI interview with clinical integrity.⁵ Therefore, with increasing pressure to offer brief screening, assessment, and treatment interventions as well as emphasis on adoption of a recovery-oriented care model, there is a clear need to develop an effective tool that is patient-centered and recovery-oriented, and that will efficiently and accurately assess the progress of patients in treatment and in recovery.

An optimum approach to assessing the progress of patients in treatment for substance use disorders would take a multidimensional, patient-centered, real-world perspective that allows patients to express their perceptions of changes in areas that are most pertinent and meaningful to them. An assessment tool capable of efficiently eliciting patients’ observations on their status and progress in treatment and during recovery is needed, especially in response to the calls for increased delivery of addiction medicine services in primary healthcare settings, and consistent with the Food and Drug Administration (FDA)’s current emphasis on patient-centered treatment outcomes in clinical research. Recognizing the growing need for such a tool, we have developed and used, in clinical practice and in research, a brief instrument – the Treatment Effectiveness Assessment (TEA) – to assess treatment progress and recovery, based on the patient’s perspective. The resulting easy-to-complete instrument yields results that are relevant to the patient and meaningful to both clinicians and patients.

We conducted preliminary evaluations to determine the feasibility and clinical utility of the TEA in practice and in clinical research. In this article we describe the development of the TEA and the early-phase application of the instrument in clinical settings. Subsequent manuscripts will describe the psychometric properties of the TEA as sufficient data become available, perhaps by late 2013.

The need to efficiently and comprehensively assess patients in treatment and in recovery

Documentation of patient progress is an essential part of clinical care. Over the years, various approaches have been used to assess outcomes of substance abuse treatment (eg, diagnosis, drug use and related problems, urine testing).⁷ The traditional approaches to assessing treatment progress require considerable time and trained staff to administer. Further, they may not entirely reflect the changes that are most important to patients, because the instruments rely on predetermined questions that may not be applicable to patients with diverse life experiences and cultural backgrounds. Thus, the resulting depiction of progress in treatment and recovery may be of uncertain relevance to clinical care. Moreover, the complexity and time-consuming characteristics of these instruments inhibit their utility in clinical settings, and as a result, the instruments are unevenly incorporated in program evaluation or treatment outcome assessment, if used at all. For example, the limited provider time and the pressure to screen or assess for multiple medical and psychiatric disorders in a busy clinical setting may hinder the adoption of a lengthy instrument for screening.^8,9

To date, the most widely used instrument for assessing treatment outcomes in addiction treatment settings is the ASI,^5,10 a lengthy questionnaire developed in the early 1980s and primarily based on male veterans. Some ASI questions may be outdated and uninformative, especially for persons with diverse racial or ethnic backgrounds.⁵ Clearly, clinicians and researchers need a way to find out how patients are doing without forcing patients and research participants to endure an exhaustive interview or complete a long questionnaire that may not be wholly informative to patient progress.

With an increased emphasis on the need for patient-centered treatment and support from the US federal governments, there is a need for additional outcome measures that are not only patient-centered and recovery-oriented, but that can also be adopted effectively by clinicians in clinical settings, to provide timely information about the patient recovery progress. To this end, we developed a brief, patient-oriented, easily administered instrument that is practical and, most important, relevant to patients in treatment and in recovery.

The concept of personal recovery

Recovery is not simply a return to premorbid functioning or remission of symptoms; instead, it is about finding purposes and meaning in life and engaging in worthwhile endeavors, despite one’s health condition.¹¹ Recovery is a personal and individual process of growth that unfolds along a continuum, and there are multiple pathways to recovery.¹² The meaning of recovery is related to both the illness condition (eg, diagnosis, severity) and personal perspectives on hope, confidence, or self-determination (such as taking responsibility for life, willingness to ask for help, and being connected to others).^13,14 Therefore, recovery from addiction takes a long-term perspective, involves multiple pathways, and involves processes that are unique in timing and characteristic for every individual.¹² In this regard, a person-centered approach to assessment and treatment for addiction is needed to understand a patient’s progress, needs, and status in recovery.

Consistent with this concept of personal recovery and need for establishing a recovery-oriented care system, a consensus panel convened by the renowned Betty Ford Center for Rehabilitation defined “recovery” as “a voluntarily maintained lifestyle characterized by sobriety, personal health, and citizenship.”¹⁵ Similarly, the Substance Abuse and Mental Health Services Administration defines recovery from a mental or substance use disorder as “a process of change through which individuals improve their health and wellness, live a self-directed life, and strive to reach their full potential.”¹⁶ The Substance Abuse and Mental Health Services Administration (SAMHSA), in 2012,¹⁶ delineated four major dimensions that support a life in recovery:

1. Health: overcoming or managing one’s disease(s) or symptoms – for example, abstaining from use of alcohol, illicit drugs, and nonprescribed medications if one has an addiction problem – and for everyone in recovery, making informed, healthy choices that support physical and emotional wellbeing;

2. Home: a stable and safe place to live;

3. Purpose: meaningful daily activities – such as a job, study, volunteerism, family caretaking, or creative endeavors – and the independence, income, and resources to participate in society; and

4. Community: relationships and social networks that provide support, friendship, love, and hope.

In a recovery-focused paradigm, a treated patient is neither “cured” if abstinent from drugs nor a “treatment failure” if there is a relapse to drug use. Rather, the patient is better regarded to be in a process of recovery, a continuum that may involve periods of abstinence and times of resumed drug use.^12,17 Routine monitoring of recovery over the long term, however, requires assessments that can be administered quickly and at low cost, in contrast to those currently in wide use, while still capturing the core elements of the recovery process.

Development of an efficient, patient-centered assessment instrument

Consistent with the existing literature on the concept of personal recovery,^11–14 we have developed a four-item, patient-centered and recovery-oriented instrument, the TEA, for evaluating the status of patient progress and recovery from addiction. The TEA directly assesses the patient’s personal perspectives on substance use, personal health, lifestyle, and responsibility to the community – common elements of recovery identified by prior studies, by the consensus panel of the Betty Ford Center, and by SAMHSA.^12–16 The TEA obtains information from patients about specific changes that are most salient to them (eg, housing, employment, and family relationships). Patients respond to the TEA questions providing both numerical responses and brief feedback about their situation along the lines of the four categories:

• Substance use (drugs, alcohol, tobacco);

• Health (eg, physical, emotional health);

• Lifestyle (eg, housing or living situation, family, employment, relationships);

• Community (eg, obeying laws and becoming a responsible member of society).

A version of the TEA is presented in Figure S1.

Treatment effectiveness assessment (TEA)

The TEA asks the patient to express the extent of changes for the better from his/her involvement in the program to the current point (or how things are at baseline, for the first TEA) in four areas: substance use, health, lifestyle, and community. For each area, the patient is asked to think about how things have become better and to circle the results on a 10-point scale, where 1 represents “not better at all” and 10 represents “very much better.” In each area, the patient is asked to write down the one or two changes that have the greatest personal importance and is encouraged to add details, explain remarks, and make comments.

An essential feature of the TEA is that it allows the patients themselves to assign their own “weights” to the conditions, perceptions, and behaviors that constitute changes in the four domains of recovery. This is consistent with another conclusion of the Betty Ford consensus panel, that “Individuals who are ‘in recovery’ know what it means to them and how important it is in their life. They do not need a formal definition.”¹⁵ Simply put, they don’t need clinicians or researchers to probe too far with a questionnaire; they just need to be queried simply, with a few questions to describe what changes have occurred that are important or meaningful to them. One patient might define lifestyle improvement as landing an entry-level job, another by resuming her law practice, and yet another by enrolling in college, which is completely in line with the concept of a personal or patient-centered view on progress in recovery.^11,12,14 Researchers and clinicians are not in the best position to know what changes may have occurred in the lives of patients, and this person-oriented assessment offers an additional option for clinicians and researchers to understand the recovery status from the patient’s personal perspective (ie, sensitivity to patient needs and changes in status).

Brevity and ease of administration

Brevity and ease of administration are important characteristics promoting adoption by clinicians in a busy treatment setting.^8,9 This patient-centered perspective has been embraced in the development of the TEA instrument, which also benefits from some important lessons learned over the past decade regarding instrument development. Although the field of psychometrics has long been dominated by the notion that “all other things being equal, a long test is a good test,”¹⁸ there is a growing body of literature suggesting that simpler, more concise measures of psychological constructs may be as good or better than more complex scales. The promise of using reduced sets of items is even greater when other advantages are taken into account, such as (1) greater statistical power in regressions, (2) reduced staff and respondent burden, and (3) reduced risk of common methods bias.¹⁹ Even with modest error term correlations between items in services research, the incremental information gained with each additional scale item is often extremely small, and the use of longer scales tends to inflate the across-item error terms (and annoy respondents).²⁰ Smith et al²¹ compared results of a single-item screen for drug use (“How many times in the past year have you used an illegal drug or used a prescription medication for nonmedical reasons?”) with the results of the ten-item Drug Abuse Screening Test (DAST-10). The single item was 100% sensitive and 74% specific for the detection of a drug use disorder (as determined by the Composite Diagnostic Interview Substance Abuse Module [CIDI-SAM]) – almost identical to that of the DAST-10. Citing the time constraints in primary care settings, Smith et al advised the expanded use of simpler, more straightforward approaches to assess patients.²¹

A study conducted by Farabee et al²² (the Correctional Offender Management Profiling for Alternative Sanctions [COMPAS]) indicates that certain measures can be shortened and/or simplified without any loss of information gained. The COMPAS study of needs among prison inmates revealed a close relationship between the COMPAS Substance Abuse scale (which is based on the Texas Christian University Drug Screen [TCUDS], an 18-item, self-administered assessment designed for evaluating prison-based treatment programs) and the inmates’ responses to a single “yes/no” item added to the interview. The investigators simply asked inmates whether they felt that they were in need of substance abuse treatment. The single-item measure accurately identified 70% of the inmates designated as “high need” offenders according to the COMPAS and 100% of those designated by the COMPAS as “low need.”²²

More recently, Wu et al^23,24 applied a series of psychometric methods to identify a core set of brief screeners for the detection of individuals with an alcohol or drug use disorder, in medical settings. The research was conducted on large samples of two different national, multisite studies and examined all available alcohol and drug use disorders in each study (alcohol, amphetamines, cannabis, cocaine, opioids, sedatives). Regardless of the sample characteristics and substances used, both studies consistently showed that use of two dependence items (instead of the seven questions designated for assessing a substance dependence disorder) can identify patients with an alcohol or drug use disorder.^23,24 Such psychometric analyses provide evidence that “more items is not better,” as some items may be uninformative or redundant, adding little information to the utility of an instrument.²⁵

Less can be more, and new instruments in the field are desirable for their simplicity, if their validity can be documented and shown to have advantages over current approaches. As that “simpler, more straightforward approach,” the TEA offers clinicians a means of efficiently and easily assessing individuals in treatment for substance use disorders, thereby enhancing the capacity of clinicians in mainstream medicine to monitor their progress. Furthermore, the TEA is responsive to the increased demand for electronic medical records and electronic data collection, given its simplicity and brevity, allowing computer-based and cell phone-based completion by patients in virtually any setting or location. The TEA offers a significant reduction in the time and cost required for patient assessment, facilitating its implementation. However, its seeming simplicity is deceptive; it takes advantage of the computing power of the patient’s brain to rapidly screen a host of life activities, emphasizing those most relevant to recovery, weighting their meaning and value and reducing them to a simple quantifiable TEA score.

Generation of quantitative data

The TEA yields a quantitative score that can be easily captured in the electronic medical record in a routine-care fashion to track the patient’s long-term progress and facilitate communications between patients and clinicians. The TEA is designed to help the patient and the clinician efficiently gauge treatment progress according to the patients’ sense of important areas of change and improvement. Patients do their own cognitive weightings before assigning a value of 1 (not improved or little improved) to 10 (very much improved) to each of four recovery-oriented domains: substance use (drugs, alcohol, and tobacco), health (physical and emotional), lifestyle (eg, housing/living situation, family, employment, relationships), and community (eg, obeying laws and becoming a responsible member of society). The total score is the sum of the responses to the four TEA domains, ranging from four (no measurable improvement or worse) to 40 (significantly improved), based on the changes that have taken place since entering the treatment program. The entire assessment takes only 2 to 3 minutes. In the course of treatment and during recovery, the TEA can be administered periodically in order to keep track of how things have or have not changed in some key areas of life since beginning treatment and to provide some specific examples of events that help the patients to assign the significance of such changes, and may help patients focus their efforts in recovery. Such a recovery-oriented care model has been increasingly recognized as an essential care model for persons with a chronic condition, like substance use disorder.^12,16

While data from our preliminary work with the TEA are insufficient for formal psychometric analysis, we were able to see a correlation between the TEA scores and the results of urine drug testing in that, patients whose urine results changed from “positive for illicit drug use” to “negative” over the treatment period, showed the greatest TEA change score in the drug use domain (N = 9).

In summary, the TEA has considerable advantages over existing assessment instruments, which are burdensome to researchers, clinicians, and patients. In addition to its brevity and ease of administration, a distinguishing feature of the TEA is its orientation to the patient. The TEA responses are portrayed as a simple, numerical tally that is useful in tracking progress over the course of treatment and in recovery. No software is required for data entry or scoring. The TEA process may also help patients recognize issues needing attention in order to achieve a sober, healthy, productive, and balanced life.

Discussion

The development of the TEA was based on decades of clinical practice, reflects the lessons learned from utilization of a wide array of assessment instruments in the field, and is well advised by the literature, especially some of the more recent material on the constructs of “recovery.”^15,16,26 We have been using and refining the TEA for several years, and we have shared the TEA with some local clinicians who have begun to use it in their practices. They find it easy to use and easy on their patients. Both physicians and patients like it because it is easy to understand and easy to do, and doing it pays off in ways they hadn’t thought of before. The TEA results give both the clinician and the patient specific benchmarks to track treatment progress, like recognizing familiar landmarks on a journey. For the clinicians, the TEA gives them specific areas in their patients’ lives that they know are important for their patients and therefore things to pay attention to in treatment. For patients, doing the TEA reminds them that their recovery has a lot to do with what they do or don’t do in life and that changes in their lives are actual, concrete, demonstrable items in their recovery.

We have evaluated a small series of patients with TEA, ASI, and urine analysis results; while the numbers are too small for statistical analysis, inspection of the raw data suggests a positive correlation between their respective results. One of us (DL) also collected TEA and urine data from a series of patients, and again, there is indication of a positive correlation, but since ASI is not part of her practice routine, we cannot perform statistical analysis of correlations of TEA results with ASI results.

More work is necessary to refine and validate the TEA (eg, factor and item response theory analysis, reliability and validity research) and to verify its appropriateness for various populations (eg, participants in trials of new outpatient pharmacotherapies, groups of distinct ethnic/racial characteristics, non-primary-English speakers, parolees or probationers, adolescents, and women, etc). In addition, although we examined the relationship between the TEA drug score and urine test results in the two pilot studies, similar comparisons are also needed for the other domains. For example, future research should compare results from selected domains of the ASI-Lite (health, financial stability, and legal problems) and from the other three TEA domains (health, lifestyle, and community) with external criteria (eg, physical exams, the Short Form (36) Health Survey [SF-36], employment records, and arrests) to assess criterion-related validity.

We also intend to assess the feasibility of implementing the TEA in a broader array of clinical settings, including primary care clinics. Given its brevity and promising preliminary results, we believe that the TEA will be well accepted by patients and clinicians and will be especially appealing to primary care physicians who will soon be tasked with addressing screening and treatment for substance use disorders while maintaining larger caseloads.²⁷ The specific events given as key changes underlying the change scores also provide data for verification, if desired, and for future research. For instance, improvement in drug use can be verified by urine drug screen; health status can be checked against medical records and data from health care utilization; lifestyle changes can be verified by employment records, such as pay stubs and tax returns; and community responsibility can be checked against arrest records.

We decided to more broadly disseminate the instrument by taking advantage of an open access journal so that the TEA becomes easily available to practitioners and other readers. We think it is timely because this issue confronts everyone who has given even a passing thought to what will be needed when health reform comes. And since the instrument is so simple, requires no formal training, and takes just a few minutes to learn and use, we believe many clinicians will find it easy to adopt into their practice. Those interested in instruments may find ways to validate the TEA – much like the psychometric research that has been conducted on the Clinical Opioid Withdrawal Score (COWS) – especially in settings where their clinical work requires them to perform some version of the ASI.

Looking toward a future quantitative examination of the TEA, there is an ongoing study that includes the TEA, the ASI Lite, and urine toxicology, but it will be 2 years before the study is complete and even longer before the results are available for our purpose. We decided to share the TEA with the treatment community, especially as healthcare reforms are imminent; we believe there is little to lose and much to gain by making available this efficient, useful instrument now.