Characteristics of Empirically Supported Treatments

Abstract

This study presents a survey of general characteristics of empirically supported treatments (ESTs) identified by the American Psychological Association Division 12 Task Force on the Promotion and Dissemination of Psychological Procedures. Results indicate that the ESTs share the following characteristics: they involve skill building, have a specific problem focus, incorporate continuous assessment of client progress, and involve brief treatment contact, requiring 20 or fewer sessions. Traditional assessment methods, such as intelligence testing, projectives, and objective personality tests such as the MMPI-2, are rarely used in these treatments. Although it is recognized that these findings are in part an artifact of sociological factors present in contemporary psychotherapy development and research, the findings may also serve as a heuristic aid in the development of therapies.

In 1995 the American Psychological Association Division 12 Task Force on the Promotion and Dissemination of Psychological Procedures was formed and charged with identifying psychotherapies with proven efficacy. The rationale was partly to educate those who might become involved (e.g., potential clients, third-party payors, and treatment providers) about the benefits that could be derived from psychotherapy. The task force included members with a variety of theoretical preferences in an attempt to minimize the intrusion of antecedent theoretical commitments. The committee agreed on standards for the adequacy of research evidence to warrant the positive judgment that the treatment was “empirically validated” or “empirically supported.” The 1995 task force identified 18 well-established empirically supported treatments (ESTs) and 7 probably efficacious treatments.¹ The 1998 task force updated this list and found 16 well-established treatments and 56 probably efficacious treatments.²

Although at each step the task force has attracted a great deal of controversy, its findings serve an important purpose because it is the first time in the history of psychotherapy that some consensus has emerged across psychologists with differing theoretical orientations regarding what psychotherapies have evidential support. This list of ESTs is an interesting data set in itself. It can be examined in an attempt to identify trends or commonalities among these treatments. Modern psychotherapies range widely in their assumptions on issues such as the importance of the psychotherapeutic relationship in the change process, the exact role of assessment and psychodiagnostics, the length of therapy, and the roles of skill building versus personality restructuring, among other issues. Trends in these relatively successful therapies might be useful in evaluating assumptions about the nature of effective therapy as well as a heuristic aid in the discovery of other effective therapies.

METHODS

Participants

Participants in this study were authors of studies cited as supporting the inclusion of the treatment as a “well-established” empirically supported treatment or a “probable” EST in the APA Division 12 Task Force reports on empirically supported treatments.^1,2 For each EST, the Task Force cited either one or two articles that were used to support the conclusion regarding efficacy of the treatment. In cases where one study was cited, attempts were made to locate both the first and the second author. If this could not be done, additional authors (third, fourth, etc.) were contacted. In cases where two studies were cited, attempts were made to locate the first author of each study. If these individuals could not be located, additional authors were contacted, with the goal of contacting one author from each study. As can be seen in Table 1, a total of 76 distinct well-established and probably efficacious ESTs were identified, 4 from the 1995 Task Force report and 72 from the 1998 report. Overall, we distributed in this way 120 of the questionnaires described below.

TABLE 1. Empirically supported treatments for which data were requested (data were not received from those in bold).

Procedures

An 11-item questionnaire designed to assess various characteristics of ESTs was mailed to each participant. Specific questions can be found in Table 2. The response format for eight questions was “yes-no-other.” However, three questions (#1, #5, and #8) required respondents to give a specific quantified response. The questionnaire included a cover letter explaining the purpose of the study, instructions concerning how to complete the questionnaire, and the name of the EST. After approximately 6 weeks, if at least one response had not been received concerning a particular EST, another questionnaire was sent to the author(s).

TABLE 2. Questionnaire items and responses.

Our goal was to obtain one completed questionnaire for each full and probable EST, although often two authors for a particular EST were sent questionnaires in order to increase the likelihood of obtaining a completed questionnaire. If two completed questionnaires were received for an EST, one questionnaire was randomly chosen and that one was used to gather data; this occurred 11 times. Questions #1, #5, and #8 required respondents to provide quantitative answers. When these answers included a range of numbers, the median of these two numbers was the datum used in the results.

Reliability

Fortuitously, two sets of data were collected for a total of 11 ESTs, and interrater reliability was calculated by using percentage agreement [agreements/(agreements+disagreements)]. Interrater reliability for this data set was 76%.

When examining the reliability data further, we found that a greater number of disagreements occurred for questions #6, #8, and #11 (each of these having nearly equal numbers of agreements and disagreements). Question #6 concerned the importance of the therapeutic relationship, question #8 concerned the frequency of assessment, and question #11 concerned the importance of knowing a client's history for treatment planning. It is impossible to know exactly why more disagreements occurred for these questions, but it could be due to the ambiguity of the question, different interpretations of the question, or simple disagreements among professionals.

RESULTS AND DISCUSSION

Overall, the response rate was 80% (61 of 76 ESTs). Responses for questions with a “yes-no-other” format are presented in Table 2. Specific trends derived from these data indicate that ESTs, in general, share certain characteristics. For instance, responses to question #3 indicate that homework is a component of most ESTs (85%). In addition, responses to question #4 show that traditional assessment devices (e.g., the Minnesota Multiphasic Personality Inventory [MMPI] or the Rorschach test) are generally not used because they do not aid in treatment planning; 92% reported not using these devices.

Other clear trends that emerged from the data show that ESTs generally 1) focus on skill building, not insight or catharsis (#7; 85%); 2) involve continuous assessment to monitor a client's progress (#9; 77%); and 3) are problem-focused (#10; 90%).

Trends were less clear concerning three questions. First, the necessity of knowing the history of a client's presenting problem for treatment planning showed no clear trend (#11). More specifically, 54% of respondents reported that knowing a client's history was not essential for treatment planning, whereas the other 46% either indicated that knowledge of the history was important or provided an “other” response. The necessity of making a DSM-IV diagnosis for treatment planning (question #2) is also somewhat less clear because responses were more evenly divided between “yes” and “no” responses. The majority of responses obtained (61%) indicated that diagnosis was not essential for making treatment decisions, but it appears that for at least a substantial minority of ESTs (38%), a DSM-IV diagnosis is an important aspect of treatment planning. Finally, 54% of respondents indicated that the formation of the therapeutic relationship was an important mechanism of change (#6), whereas 46% responded either “no” or “other.”

Responses for question #1, concerning the total number of sessions required for the treatment, are presented in Figure 1. The vast majority of ESTs (80%) require 20 or fewer sessions. Responses of “other” to this question indicated that certain ESTs require very large amounts of time so a specific number could not be given (e.g., behavior modification for developmentally disabled individuals).

FIGURE 1.

Total number of sessions required for empirically supported treatments (N=58)

Figure 2 shows data regarding the number of hours a client must devote to treatment both in and outside of the session per week. It was found that all ESTs require some out-of-session work, with a large percentage requiring between 2 and 5 hours per week (59%). “Other” responses to this question usually indicated that it would be difficult to calculate the actual number of hours because of large between-subject variability (as in token economy programs or interpersonal therapy for bulimia).

FIGURE 2.

Number of total hours clients must devote to therapy in and out of session per week (N=60)

Responses to question #8, concerning the frequency of assessment during treatment, are summarized in Table 3. A variety of responses were given to this question. In general, though, it is evident that most ESTs involve periodic assessment throughout the treatment process. This finding contradicts the common notion among the general public that assessment is a process that occurs prior to treatment and then stops. It cannot be concluded, however, that continuous assessment is essential for positive treatment outcomes; it may be the case that continuous assessment is part of the research protocol used to evaluate many ESTs.

TABLE 3. Responses to question #8 (frequency of assessment).

The response rate to the questionnaire was 80%, which leaves 20% of ESTs unaccounted for. As is always the case, it could be argued that had data been collected for these particular ESTs, the results of this study might have been somewhat different. Table 1 lists in boldface the ESTs for which no data were collected. These ESTs do not appear to differ in any important way from those for which responses were obtained. For instance, the majority of these therapies are behavioral or cognitive-behavioral, and they address a number of different problems (e.g., opiate dependence, obesity, sexual dysfunction). Therefore, we conclude that the data reported here are representative of ESTs in general.

Trends could, in theory, be different for the well-established ESTs versus the probably efficacious ESTs. To test this hypothesis, data for well-established and probably efficacious ESTs were compared for each of the “yes-no-other” questions. As can be seen in Table 4, the trends discussed above for the entire data set seem to apply equally well to the well-established ESTs and the probably efficacious ESTs. Furthermore, when data for particular questions are compared among the two sets of data, no appreciable differences emerge. Overall, the two sets of data show similar trends.

TABLE 4. Data for well-established versus probably efficacious empirically supported treatments.

The trends found thus far in these relatively effective therapies tend to vindicate assumptions that an effective therapy is one in which the treatment is short-term; the emphasis is present- and problem-focused; skill building is stressed; the therapeutic relationship is considered to be important; homework is assigned; assessment is periodic; and traditional psychological tests such as intelligence testing, the MMPI-2, and the Rorschach are superfluous. However, there was some variation to this general pattern.

Philosophers of science have suggested that there is no logic of discovery; that is, in the context of discovery, “anything goes.”³ We do not disagree with this. However, we believe that the pattern of similarities that we have identified across therapies for a wide variety of problems (e.g., enuresis, depression, coping with stressors, bulimia, oppositional children, headache, and marital problems) is important to acknowledge because it can serve as a guide in the numerous choice points the researcher faces in therapy development.

We also want to offer one caveat. This trend may be partly an artifact of certain sociological factors that pertain to contemporary psychotherapy development and research. That is, the individuals who are meeting the research burden indicated by the Task Force are largely cognitive and behavioral researchers who generally adhere to the trends we have identified. Moreover, contemporary evaluative methods fit better with some types of therapies than others (e.g., short-term therapies). Still, this is an interesting sociological sketch regarding this discipline at this point in time. The future will show whether those with other, radically different positions regarding the nature of effective psychotherapy can vindicate those positions in reasonably designed outcome research.