Internet-Based Approaches to Collaborative Therapeutic Assessment: New Opportunities for Professional Psychologists

Abstract

Collaborative (or therapeutic) assessment is an empirically supported procedure that involves the client as an active participant in the assessment process. Clients discuss data they provide with the assessor in a collaborative manner designed to provide insights and assist in setting mutually agreeable treatment goals. Internet-based procedures allow for ongoing (including daily) tracking of psychological variables and provision of immediate graphic feedback to therapists, clients, and clinical supervisors. As an example, we describe one such system: Evidence-Based Assessment System for Clinicians (EAS-C) that contains more than 30 brief and empirically validated assessment instruments that can be completed via the internet or smartphone. We also provide examples from a stress management intervention demonstrating how single-client data from a web-based daily stress and coping diary tied to the EAS-C were utilized to provide clients with individualized feedback, assess progress, identify idiographic patterns of cognitions, affect, and coping strategies, and test clinical hypotheses. Internet- and computer-based technological advances can improve service delivery and help bridge the gap that currently exists between science and practice.

Professional psychologists strive to provide the best therapeutic services as efficiently as possible. Few would question the value of systematic diagnostic assessment, collaborative goal-setting and ongoing progress assessment. Recent work has highlighted how systematic evaluation of client progress and collaborative discussion of assessment data can help shape treatment plans and improve therapeutic outcomes (Finn, 2007; Lambert, 2010). Tracking client progress helps reduce dropouts, prevent treatment failures, shorten length of treatment and improve outcomes (Reese, Norsworthy & Rowlands, 2009; Shimokawa, Lambert, & Smart, 2010). Ongoing collaborative assessment appears to be a particularly valuable therapeutic tool. In this article, we discuss how the use of new technologies can facilitate collaborative assessment in a way that is clinically useful, user-friendly, and cost-effective.

For most practitioners, the routine collection of client progress data is not feasible for practical reasons, particularly limitations in time and resources. However, internet and smartphone technologies provide therapists with the opportunity to easily and inexpensively track client responses at daily or weekly intervals, thereby yielding repeated measures of variables of interest. Therapeutically meaningful clinical data can be collected from clients using reliable, valid, HIPAA-compliant, and secure means. Confidentiality can be protected by using password-protected procedures and randomly-generated code numbers rather than names or other identifying information. De-identified data may be encrypted and stored in secure, designated servers.

In professional psychology, computerized tracking of client progress is already being accomplished using global measures such as the Outcome Questionnaire-45 (OQ-45; Lambert, 2010). Problem-specific and individualized assessment approaches are also being implemented. To illustrate this latter approach, we describe a psychological assessment package (Evidence-Based Assessment System for Clinicians; EAS-C) that provides individualized tracking of measures specifically matched to each client’s problems. The system can also be adapted to the measurement requirements of particular interventions. To illustrate this flexibility and how assessment data can be used to facilitate the therapeutic process for individual clients, we describe a customized daily diary program for tracking stress and coping developed as part of a manualized stress management intervention.

A Collaborative Approach to Assessment

For more than a century, professional psychologists of all theoretical persuasions have used behavioral, interview, and psychological test data for diagnostic purposes, case conceptualization, treatment planning, and outcome monitoring. In treatment process and outcome research, psychological test data have served as measures of client-selection, moderator, mediator, and outcome variables. Without reliable and valid assessment measures, it would not be possible to identify empirically-supported principles and treatments (APA Task Force on Evidence-Based Practice, 2006; Kazdin, 2008).

Beyond traditional diagnostic purposes, assessment data can also be used for therapeutic purposes, serving as an active component of treatment. One approach to assessment, termed collaborative assessment (CA; Fischer, 1994) or therapeutic assessment (TA; Finn, 2007), involves the client as an active participant in the assessment process. This approach is consistent with the ethical code of the American Psychological Association, which now requires psychologists to communicate to clients the information acquired in an assessment (APA Ethical Guidelines, 2002, Standard 9.10). In collaborative assessment (CA), clients are actively involved in defining assessment goals. They are invited to pose specific questions they would like answered by assessment results, and to reflect on the accuracy and personal relevance of the feedback provided. The goal is to use the assessment as a therapeutic intervention in which clients are invited to explore the meaning of the results and to gain new understandings and evaluations of themselves. Although nomothetically-derived test norms provide the basis for feedback, CA is inherently an idiographic enterprise focusing on the individual client.

Several systematic approaches to developing a collaborative client-assessor relationship and providing personally-meaningful feedback have been advanced (e.g., Finn, 2007; Fischer, 1994). Although early applications of CA were generally limited to one or two feedback sessions, an increasing emphasis is being placed on providing feedback throughout the course of therapy (e.g., Lambert, 2010). In this article we will focus on this latter approach to CA.

An emerging body of evidence supports the potential therapeutic effects of CA, whether provided during a single feedback session or on multiple occasions. In the empirically supported motivational interviewing approach to addictive behaviors (Miller & Rollnick, 2002), nonjudgmental feedback regarding self-reported substance abuse has been used to promote positive motivational and behavioral change. Other experimental studies involving the provision of personality and aptitude test feedback have also demonstrated positive effects on a wide range of outcome variables. Compared with control groups, individuals from a variety of populations (college undergraduates, counseling center clients, children and their parents, traumatic brain injury patients, and psychiatric inpatients) provided with feedback have reported more positive evaluations of the assessor and assessment experience (Allen, Montgomery, Tubman, Frazier, & Escovar, 2003; Hilsenroth, Ackerman, Clemence, & Strassle, 2002), as well as higher satisfaction with subsequent treatment (Pegg, Auerbach, Seel, Buenaver, Kiesler, & Plybon, 2005). Salutary changes on measures of psychological well-being, including increased self-esteem and optimism and reduced symptomatology have also been reported (Finn & Tonsager, 1992; Newman & Greenway, 1997), as well as a significant reduction in the number of psychotherapy clients who terminated treatment against medical advice (Ackerman, Hilsenroth, Baity, & Blagys, 2000). Poston and Hanson (2010) conducted a meta-analysis of 17 studies in which CA procedures were compared with control or comparison groups. On average, the analysis revealed a strong effect size (Cohen, 1977) of 1.12 on treatment process variables, such as client-rated helpfulness of and satisfaction with feedback, and a moderate effect size of 0.367 on outcome variables such as self-understanding and symptomatology. Overall, the mean effect size combining process and outcome variables was 0.547. Given that the studies typically involved only one or two sessions, no dose-response relation could be computed, so the potentially enhanced effects of repeated assessment and feedback over the entire course of treatment deserve additional empirical attention.

At a practical level, the issue is how assessment data used for CA purposes can be collected across a range of professional psychology practice settings. Many practitioners simply do not have the time or resources to collect ongoing assessment data or to score and then review multiple records returned by clients at each treatment session. Moreover, many popular assessment measures must be purchased from different publishing companies and must either be hand-scored or computer scored using proprietary software specific to each measure. However, web-based technology can facilitate the use of CA, and computer-based assessments are proving to be useful for both treatment providers and clinical researchers (Barak, Hen, Boniel-Nissim, & Shapira, 2008). For example, computerized daily assessment of smoking behavior is an important feature of an internationally-disseminated Web-based smoking cessation program (Muñoz, Lenert, Delucchi, Stoddard, Perez, Penilla, & Pérez-Stable, 2006). Smartphones are being employed to track behaviors and provide feedback to bulimic clients (Bauer, Percevic, Okron, Meermann, & Kordy, 2003), to assess daily mood levels in coping skills training, (Morris, Kathawaka, Leen, Gorenstein, Guilak, Labhard, & Deleeuw 2010), and to allow dialectical behavior therapy (DBT) clients and therapists to track clients’ daily use of DBT skills and provide therapeutic suggestions (Dimeff, Paves, Skutch, & Woodcock,, 2011). Ongoing CA is best accomplished with brief measures that are not burdensome to the client, together with a means of instantly scoring and displaying feedback to the client. Computers and smartphones help satisfy both of these provisos, drastically reducing assessment burden on both the practitioner and the researcher.

An Illustrative Web-Based Assessment System

Although clients may value ongoing progress assessment, they may not appreciate repeatedly filling out lengthy paper-and-pencil measures, especially if, as is typically the case, those measures are not immediately scored and integrated into treatment. In this regard, computerized assessments represent an improvement over paper-and-pencil measures insofar as they provide the client and clinician with instant feedback. HIPAA-compliant, password-protected, web-based systems allow clients to complete measures using any Internet-enabled device and to securely transmit their data from any location, thereby eliminating the need for clients to remember to bring in their homework or to be physically present in their practitioner’s office when completing measures. The importance of brief measures is underscored by the results of several recent studies. Miller, Duncan, Brown, Sparks, and Claud (2003) found a sharp decline over time in both clinicians’ and clients’ compliance in completing the widely-used 45-item OQ-45. In contrast, compliance remained high for brief single-item analogue measures of the OQ subscale variables. Likewise, Brown, Dreis, and Nace (1999) reported that a majority of clinicians they surveyed considered any measure or combination of measures that took more than 5 minutes to complete, score, and interpret to be impractical for ongoing assessment.

Even when measures are brief and can be instantly scored, clients still need to understand the assessment results in context. This can present a challenge for individual practitioners who may know what the scores represent on a given measure but may not have normative comparison data readily available to share with clients. Similarly, general measures of overall functioning, such as the MMPI-2 or the Rorschach, while clinically useful, may nevertheless lack face validity for clients presenting with specific problems. And finally, for clients with comorbid problems, it may be particularly useful to view the results of more than one problem-specific measure simultaneously (e.g. depression and alcohol use). Separate proprietary scoring systems for individual measures decreases the likelihood that practitioners will be able to quickly and easily score and display more than one outcome measure at a time with their clients.

EAS-C’s Progress Tracking Module

To facilitate both the tracking of treatment progress and the use of CA as a therapeutic component, we describe the EAS-C, a web-based assessment and outcome tracking system with applicability to the wide range of clinical problems likely to be seen by clinicians in mental health settings. EAS-C provides clinicians with an integrated practice management system that includes a web-based Progress Tracking Module. The module features an extensive electronic library of more than 30 psychometrically-sound and evidence-based measures for assessing the multitude of problems most likely to be seen by clinicians in outpatient settings (e.g., substance use, depression, etc.). The measures are suitable for use with a wide range of clients from elementary school age through adulthood (see Table 1). All measures are in the public domain or authored by individuals who have given permission for their measures to be scored and graphed by the EAS-C’s software. The EAS-C also contains measures of positive mental health, including self-esteem, happiness and life satisfaction, and global self-efficacy. These measures allow clinicians to track psychological resources and resiliency factors as well as current problems and mental health issues. The measures were selected not only on the basis of brevity but also on their empirically-demonstrated reliability and validity for the assessment of a given problem. For example, EAS-C utilizes the Personal Health Questionnaire- 9 (PHQ-9; Kroenke & Spitzer, 2002) for the assessment of depression. The PHQ-9 is a widely-used 9-item depression measure that correlates highly with the Beck Depression Inventory. This brief scale can be used to track depressive symptoms on a weekly or even daily basis if desired.

Table 1.

Clinical problems for which empirically-supported brief measures are currently available in the EAS-C library

Problem areas assessed for adults only	Borderline Personality Disorder
	Couples Issues
	Gambling
	Sport Performance Anxiety
Problem areas assessed for children only	Anxiety (Separation Anxiety, School Avoidance, Phobias)
	Conduct Disorder (CD) and Oppositional Defiant Disorder (ODD)
	Elimination Disorders
Problem areas assessed for both adults and children	Attention Deficit Hyperactivity Disorder (ADHD)
	Alcohol Use
	Anxiety Disorders (Generalized Anxiety Disorder, Obsessive Compulsive Disorder, Panic Disorder, Posttraumatic Stress Disorder, Social Phobia)
	Bipolar Disorder
	Depression
	Drug/Substance Use
	Eating Disorders
	Trichotillomania

Note. Other measures chosen by the counselor/clinician can be inserted into the EAS-C library, as long as a scoring key is also provided, allowing the system to score and graph the responses.

The EAS-C system scores and graphs client responses instantly in a user-friendly way (see Figure 1 for an example from the PHQ-9). The graphs provide a cumulative display of the client’s responses over time, as well as published norms (where available) and local norms that may be derived from the practitioner or center’s accumulated database. The graphs also present levels of severity and clinical cut-offs based on previous research. The visual representation of progress over time facilitates a collaborative discussion between clinician and client and a continuous monitoring of therapeutic goals and progress (e.g., “Notice how you’ve moved from the moderately depressed to the mildly depressed range after you started participating in your favorite hobbies again.”). Where such data are available, graphs may also display normative data on typical progress for a given problem (e.g., normative progress for outpatients being treated with cognitive-behavioral therapy for depression). Practitioners using EAS-C can also accumulate data on their own patients’ normative progress for a given problem over time and display the aggregated data graphically for feedback, quality assurance, or research purposes.

Figure 1.

Hypothetical EAS-C graphic output derived from a fictional client’s weekly responses to the PHC-9 depression scale, seen in relation to published and local (where available) norms.

Use of the EAS-C typically begins with the first telephone contact between clinician and prospective client. Clinicians may use (or train office staff to use) EAS-C’s structured telephone interview to screen prospective clients for a wide range of problems typically seen in outpatient mental health settings. The structured telephone screen consists of three major sections: demographic information, a brief description on the client’s presenting problem (an open-ended response), and a series of diagnostic screening questions for common mental health problems (e.g., depression, anxiety, substance use, conduct disorders for children). Each time the client answers “yes” to a screening question, a series of follow-up questions appear on the display to assess the severity and duration of these problems. The phone screen is not a diagnostic tool, but it does provide the clinician with some working hypotheses about the client’s issues and treatment needs prior to the intake session.

On the basis of clients’ phone screen responses, EAS-C automatically selects a set of baseline assessment measures tailored to each prospective client’s individual concerns (e.g., only those clients who endorsed hair-pulling on the phone screen would receive the EAS-C’s trichotillomania measure). Responses to the intake and baseline assessment measures automatically generate an individualized set of weekly progress assessment measures and a final set of termination measures. Practitioners who do not use a telephone screen may bypass that function and use EAS-C’s generic set of measures for new clients. At each step, practitioners can override the automatically selected measures by deleting, adding or substituting measures as needed.

Baseline assessment measures (BAM)

Prior to the first (intake) interview, EAS-C assigns a unique client identifier and an associated set of baseline assessment measures (BAM) from the EAS-C library to each individual based on the telephone screening interview and the client’s stated concerns. These measures are both global and problem-specific. As noted above, the PHQ-9 is used to assess depression in adults. The Comprehensive Drinking and Drug-taking Record (Brown, Meyer, Lippke, Tapert, Stewart, & Vik, 1998) assesses alcohol and drug use in adults, and and the Eating Attitudes Test (EAT-26; Garner, Olmsted, Bohr, Y., & Garfinkel, 1982) is used to assess eating disordered behavior in clients between 12 and 17 years of age. The Social Provisions Scale (SPS; Cutrona & Russell, 1987) is used to assess current social support. Clients can complete the measures on a laptop or other computerized device in the waiting or therapy room, or remotely on a secure online site. Meanwhile, practitioners can instantly review clients’ BAM responses, allowing them to make informed decisions about how to best structure the intake interview. Practitioners may choose to share the scored BAM results with the client during the intake, or they may wait to provide feedback until the first treatment session.

Progress assessment measures (PAM)

EAS-C automatically selects a set of brief problem-specific measures (PAM) on the basis of the BAM responses. The therapist can add to or substitute for these measures based on the subsequent interview and thereafter. The range of adult and child problems for which measures are available is shown in Table 1. PAM measures typically take no more than 5–6 minutes to complete and can be done on any Internet-enabled device, either remotely by computer or smartphone, or on site. An example is the Panic Disorder Severity Scale (PDSS; Shear, Rucci, Williams, Frank, Grochocinski, Bilt, & Wang, 2001), a 7-item scale used to assess panic attacks. Process measures, such as the Working Alliance Inventory-Short Form (Tracey & Kokotovic, 1989), are also administered periodically. Scores can be instantly graphed and viewed by both practitioner and client and used to collaboratively evaluate the effectiveness of the treatment and possibly alter treatment plans.

Termination assessment measures (TAM)

Clients who terminate therapy as planned receive a set of selected BAM measures, plus treatment-process and consumer satisfaction measures, prior to their last session. Clients who terminate unexpectedly may still complete these measures from home or any other remote location. Even for clients who terminate prematurely and therefore fail to complete the TAM measures, the frequent assessments during treatment allow for evaluation of the impact of treatment, given a sufficient number of data points.

Custom Measurement Creation Module

EAS-C allows practitioners and researchers to develop or insert their own measures and scoring criteria through the use of its custom measurement creation module. Designed to greatly simplify the creation of measures and attendant scoring criteria, this feature allows practitioners maximum flexibility in tracking outcomes as they are not limited to the preselected collection of measures in the EAS-C library. Practitioners can easily track variables of interest, such as nail-biting or other habits that may not be covered in the measurement library.

HIPAA compliance

The web-based feature allows practitioners or supervisors to remotely access client progress data and to write notes electronically. Consistent with HIPAA regulations, all patient health information is de-identified and encrypted prior to electronic transmission. Data are stored on designated servers with appropriate firewalls and other security features.

Overall, EAS-C allows practitioners and clients to easily track client progress over the course of treatment, instantly view the graphed results of single or multiple measures and to use these assessment results in a collaborative fashion. A randomized clinical trial is currently being planned, but preliminary usability evaluations indicate that both clients and therapists find routine computerized assessment to be valuable for both assessment and treatment purposes.

A Customized Stress and Coping Diary

In response to empirical findings as well as managed care dictates, many practitioners are choosing to utilize brief, empirically supported treatments developed for specific problems, such as anger or stress. EAS-C allows professional psychologists to utilize or even create computerized assessment tools that are customized to each client or to a given intervention, focusing on the key issues of therapeutic significance. As an example, to replace paper-and-pencil measures that have been employed for nearly three decades in a stress management training program, we developed an online EAS-C module for daily stress and coping assessment. This measurement system provides an illustration of how data can be utilized in CA to track progress, provide continuous feedback to both therapist and client, and to assess relevant psychological processes and test theoretical hypotheses.

The Cognitive-Affective Stress Management Training intervention (Smith & Rohsenow, 1987) is a brief (6 weekly or bi-weekly sessions) manualized coping skills intervention. Based on a person-situation transactional model of stress (Lazarus & Folkman, 1984), and consistent with the Cognitive-Affective Processing System (CAPS) model of personality (Mischel & Shoda, 1995), this empirically-supported intervention combines a number of empirically-supported clinical techniques into an educational program for self-regulation of emotional responses. As an important aspect of the program, clients self-monitor relations among the stressful situations in their lives, their appraisals of those situations and of themselves, their physiological reactions, of the coping strategies they employ, and their perceived effectiveness. Ideally, this is done on a daily basis so that important patterns of situations, responses, and consequences can be identified with minimal retrospective distortion. During the skill acquisition phase, clients learn relaxation and meditation skills to control physiological arousal, and they develop cognitive coping responses through cognitive restructuring (Ellis, 1962) and self-instructional training (Meichenbaum, 1985). The latter methods help the clients to create stress-reducing/preventing alternative self-statements with the assistance of written resource materials and discussion with the therapist. During the skills-rehearsal phase of treatment, an induced affect procedure (Smith & Ascough, 1984) is used to generate high levels of affective arousal to imagined situations based on the diary entries. The client practices “turning off” the intense arousal using the coping responses they have acquired during the course of their treatment (Smith & Nye, 1989).

The Stress and Coping Diary

We used the customization feature of the EAS-C to create a computerized diary system tailored to the stress management program and integral to the CA aspect of the intervention. The goal is to identify highly individualized patterns of relations between situational triggers of maladaptive processes, as well as appraisals, emotional responses, and suboptimal coping strategies. Sharing and discussing these findings with clients in a collaborative fashion, particularly when the description of the situations and stress-producing self-statements are based on the client’s own words, helps them develop insights into common triggers of stress across nominally different situations. Such insight helps clients to pair adaptive coping responses (e.g., cognitive restructuring and relaxation techniques) to the specific situational triggers and to target patterns of cognitions and coping strategies that are of particular importance to them. In addition, an individualized assessment allows for systematically analyzing the effectiveness of each coping strategy within the client’s particular life circumstances and personal dispositions. Furthermore, when implemented on an on-going basis, tracking the use of adaptive appraisals and coping responses may serve to prompt clients to utilize adaptive strategies between sessions. Finally, by collecting data on a daily basis, we sought to minimize inaccurate retrospective recall, which has been demonstrated to be a significant problem in the stress and coping literature (e.g., Smith, Leffingwell, & Ptacek, 1999).

To utilize the online system, clients log in to a website using an experimenter-generated ID number and a participant-generated password. In the current version of the diary, clients are first asked to rate the overall stressfulness of their life since the last entry, and they also rate themselves on 14 positive and negative affect terms. They are then asked to describe verbatim a stressful event in a text box and to rate the intensity of their emotional reaction to it and the degree of control they felt over the situation and over their emotional reactions to it. Next, they complete text boxes on how “automatic thoughts” about the situation may have contributed to their stress response and also what they might have told themselves instead in order to reduce or prevent their emotional response. The clients then rate the extent to which they employed the following 12 coping strategies (defined by means of brief paragraphs containing exemplars): cognitive restructuring, relaxation/breathing techniques, seeking social support, problem-focused coping, blaming self, blaming others, wishful thinking, avoidance, distraction, counting one’s blessings, physical activity, and acceptance. They also rate the effectiveness of each technique they used in producing or preventing stress, as well as their post-coping stress level. Finally, they rate their level of general life satisfaction. Screen shots of several portions of the online diary are shown in Figure 2. Clients report that it takes them between 5 and 20 minutes (M = 10 min.) to complete each diary entry, depending on the amount of detail they provide in the text boxes. To facilitate compliance with the assessment procedure, clients receive a daily e-mail reminder with a direct web link to the online diary. The date and time of each diary entry is automatically recorded when the data are submitted by the client to monitor compliance.

Figure 2.

Screen shots from the internet-based stress and coping diary used in the stress management training program.

This web-based diary program is designed to be customizable by non-programmers, as the content and response options (which can be formatted as a checklist, a rating scale, or free text responses) can be easily changed. It is also possible to customize the basic diary program content for each participant, or for a group of clients. For example, during the course of the program, one client and her therapist agreed that it would be beneficial to add three questions to her diary program based on the client’s discovery that recurrent resentment and hostility were related to her perceptions of unfairness. Another client wished to monitor on a daily basis one of her stress-related symptoms, namely, severe heartburn. Both of these changes were easily implemented by using a text editor to create the new questions. Once submitted, diary data can be viewed by the therapist or by a clinical supervisor through a web-based portal linked to the client’s web-based data folder.

Examples of Therapeutic Uses of Diary-Produced Information

Five clients (hereafter referred to as A, B, C, D, and E) participated in a 6-week feasibility study of the web-based stress and coping diary. Each client was treated by an advanced clinical psychology doctoral student who followed a detailed session-by-session treatment manual and provided the client with supplementary reading materials designed to foster the development of cognitive and affect-control skills (Smith & Rohsenow, 1987).

One immediately evident benefit of the online diary system is that it appears to increase treatment compliance through greater accessibility and user-friendliness. In contrast to the 6–12 reports that previous clients have typically completed when using a paper-and-pencil “analyzing thoughts and feelings” homework assignment (Smith & Nye, 1989), these clients provided a mean of 26.6 entries (SD = 8.1). The increased data density for each client was critical in allowing for the discovery of systematic patterns among the clients’ stress responses. These comparisons revealed highly individualized patterns of relations between stressors, appraisals, emotional responses, and coping strategies. Therapists found that these relations provided important information they could discuss in a collaborative fashion with their clients during the course of treatment.

Stressor characteristics

The web-based diary entries were helpful in identifying the nature of the stress-triggering situations for each client. For example, consistent with the stress and coping research literature (Aldwin, 2007; Lazarus & Folkman, 1984), self-reported stress levels for clients A, D, and E were clearly related to their appraisal that they had little personal control over situations they encountered in their lives. The therapists found this information helpful for encouraging these clients to reflect on whether they were accurately appraising the amount of control they had in the problematic situations and whether they might be able to employ coping behaviors (such as problem-focused coping) that would enhance personal control of stressful situations. Alternatively, when the situation was in fact beyond the control of the client, the client could be coached to utilize emotion-focused coping strategies, including acceptance. In addition, for client C, discussion of the diary entries revealed that many of her stressful situations involved self-blame about not having achieved more at this stage in her life. The client reported not being aware of the extent that self-blame helped to exacerbate her stress responses and contributed to an unrealistically negative self-image.

Through the use of the web-based daily diary, clients reported becoming more aware of the common “active ingredients” across nominally different stressful situations (e. g., cues indicating disapproval or exploitation). Identifying key stress triggers may allow clients to anticipate stress before it occurs and thereby prompt engagement in preemptive coping strategies. This identification was further enhanced by the ability of the client and the therapist to examine the client’s diary text entries in session and thereby engage in CA.

Cognition-affect relations

As a part of the web-based daily diary entries, clients are asked what they must have told themselves about the situation (or themselves) in order to trigger their stress response, and what they could have told themselves instead that would have reduced or prevented their stress response. This exercise yields highly individualized patterns of cognition-affect relations that begin to converge on core beliefs and automatic appraisals that can be the target of cognitive restructuring. Such cognition-affect relations tend to become more evident to both the client and the therapist as the diary is completed over multiple occasions. For example, Client A reported on a 2-month follow-up questionnaire that what continued to help her the most is “recognizing that (my) catastrophic thoughts are not reality.” The earlier such awareness occurs in the therapeutic process, the more time the client and therapist can spend developing alternative stress-reducing cognitive responses involving rational reappraisals or self-instructional self-statements (Meichenbaum, 1985). Further, identifying relations between maladaptive cognitions and affect can also facilitate other adaptive coping behaviors. For example, client D reported that utilizing cognitive restructuring enabled him to change the situation that was a major stressor for him, namely, time pressures associated with graduating from his program. He decided to delay his graduation date, which substantially decreased his stress and made it more plausible for him to accomplish other important things on his agenda. He reported being able to engage in this problem-focused coping because, through cognitive restructuring, he began to challenge his belief that not graduating at the designated time would mark him as a personal failure.

Stress-coping relations

The online daily diary data also allows us to identify important relations between stress and the 12 coping strategies mentioned previously. For example, one client tended to react to high initial stress levels with wishful thinking, but another client responded to stressful situations with a greater use of avoidance. Because clients report the level of stress experienced at the time of the event and after coping with the event, therapists are able to identify effective or ineffective coping strategies. For client B, avoidance (which he used with high frequency) was often associated with increased post-coping stress, indicating that this coping strategy was ineffective for him. In contrast, some coping strategies seemed to reduce stress for some clients. For instance, for client E, problem-focused coping in situations he rated as personally controllable was often associated with reduced post-coping stress, suggesting that this was an effective strategy for this particular client. Most important for the process of CA, these results could be clearly communicated (both graphically and verbally) to the clients, facilitating insight into their maladaptive and adaptive coping responses. In instances where typically-adaptive coping strategies seemed not to work, the therapist and client could focus on the issue of how well the strategy was actually executed by the client, resulting in discussions of how the strategy could be most effectively applied. In these ways, the daily diary program allowed therapists to provide feedback to clients concerning relations between coping strategy use and corresponding changes in the experience of stress that came directly from the data provided by the client.

In addition, the detailed information in the diary assisted in the implementation of treatment interventions. For example, therapists are able to use the client’s own stress-producing self-statements to increase the potency of the induced affect procedure during the skills-rehearsal phase of the intervention. Most importantly, the therapists can understand and communicate the individual tendencies of each client based on the client’s own words. This promotes insight and facilitated the collaborative targeting of particular patterns of cognitions and coping strategies. Finally, use of the graphic output of stress responses was useful for both client and therapist in tracking rate of progress.

Cognitive-behavioral treatments often utilize written homework materials. However, paper-and-pencil daily records are not always conducive to an active use of that information because the records are not available until the clients bring them into the session. Moreover, thorough review of the records can use up precious session time. The therapists, who were already well-trained in cognitive-behavioral treatments and the use of written homework, noted several advantages to the online diary method. As discussed earlier, more data were obtained and compliance was greater due to the ease with which clients could access and respond to the diary measure. Therapists also were able to view the daily diary entries prior to each treatment session and could therefore plan to highlight important findings in discussions with the clients.

Anonymous feedback obtained from the clients post-treatment revealed a generally positive response to the diary system. Clients reported that it was easy to access and complete, even with their busy, stressful schedules. When asked following treatment whether they would have been willing to continue the procedure if the intervention had been extended for more sessions, clients indicated a willingness to do so for periods as long as six additional weeks. In an ongoing replication with four new clients, we are again achieving high levels of compliance with the online diary submissions.

Tracking change during treatment

The collection of diary data provides important information on treatment response throughout the course of treatment. Although a major function of the daily diary was to facilitate the acquisition of effective coping strategies, the availability of the repeated diary measures allowed the therapists to monitor changes in the utilization of specific coping strategies. An example is shown in Figure 3. At the beginning of treatment, Client B’s favored coping strategy of avoidance seemed to exacerbate his interpersonal distress, causing him to deeply resent a working situation in which he saw himself as being exploited by coworkers who foisted their work onto the client. Feedback and discussion of this clearly apparent pattern resulted in a progressive decrease in his use of avoidance and an accompanying increase in a composite measure of adaptive coping strategies, including relaxation, cognitive restructuring, and problem-focused coping (in his case becoming more assertive in requesting that coworkers do their share of the work). These coping strategies were associated with lowered situational and personal stress.

Figure 3.

A cumulative plot of statistically significant changes in Client B’s use of maladaptive (avoidance) and adaptive (relaxation, cognitive restructuring, problem-focused) coping strategies to deal with stressful situations over the course of stress management training.

The repeated collection of questionnaire data also helps to address the important issue of nonresponse in treatment by allowing the therapist to closely monitor clients’ progress and detect changes on the measures that might not be reflected in a client’s verbal reports. We discovered that several clients had not responded optimally to the 6-session intervention and would require additional sessions to improve coping skill acquisition. In addition to those clients who do not benefit from treatment, there is evidence that 5–10% of therapy clients actually show deterioration (Bergin, 1971; Lambert & Ogles, 2004). Although we did not detect deterioration effects in the diary data of the five stress management clients we treated, it would clearly be important to detect clients whose stress responses seemed to be increasing in frequency and/or in intensity. Doing so during treatment allows therapists to alter the form of the treatment to address this issue and, hopefully, to enhance treatment effects.

Conclusions, Issues, and Limitations

CA is a promising supplementary approach to traditional treatment. Although additional research is needed, particularly randomized controlled trials of the use of CA over the course of long-term therapy, the widening range of application of this approach is clearly evident. Although CA can be successful with a one-time assessment followed by one or more feedback sessions (Finn, 1997; Poston & Hanson, 2010), we believe that it is equally adaptable to longer-term treatment with session-by-session (or even more frequent) assessment and tracking of progress in therapy. Extended application may well enhance its positive impact on both process and outcome when clients and therapists review and discuss the data on a continuing basis. We also believe that computer-assisted CA is applicable to any form of treatment, whether psychodynamic, humanistic, or behavioral. It is simply a matter of choosing appropriate instruments to measure the constructs relevant to the treatment of choice and utilizing the information in a manner consistent with the therapeutic approach.

We should expect to see the development of other systems like the EAS-C as internet-based technology continues to develop. To an increasing degree, web-based technology is being used to collect clinically-relevant data. In recent reviews of mental health delivery, Kazdin and Blasé (2011) and Newham and Page (2011) have cited this technical advance as an important development in both assessment and service delivery. The use of computers and smartphones to collect data, particularly when combined with e-mailed reminders, can enhance compliance if the measures are brief and seen as relevant by the client. The ease with which such data can be displayed and aggregated not only allows individual practitioners to introduce a scientific approach into their practice, but can also help reduce the gap that frequently exists between clinical science and practice (Borkovec, 2004; Kazdin, 2008). The ongoing collection, review, and client-therapist discussions of data over the course of therapy permit the examination of possible mechanisms of change, the testing of clinical hypotheses with single cases, and high-quality treatment delivery (Persons, 2007). As Barkham and colleagues (2010, p. 214) point out, "using data routinely to improve practice needs to be rooted in daily practice so that it can then filter upwards to contribute not only to improving practice but also to enhancing the scientific evidence for the effectiveness of psychological therapies.”

Understandable, most practitioners have neither the time nor the inclination to apply traditional statistical procedures to analysis of measures they might collect using a system such as the EAS-C. However, for those who might be interested in examining single-case data for quality assurance purposes or who are part of practice-research networks, recently developed and user-friendly time-series analytic approaches provide an opportunity to do so. One example is a freeware time-series analysis program (Simulation Modeling Analysis, or SMA) described by Borckardt, Nash, Murphy, Moore and O'Neil (2008). The user-friendly SMA procedure requires repeated measures over a designated baseline period and a subsequent treatment period (or designated phases of treatment). All that is required of the therapist is to enter the data points for a particular measure in tabular columns that represent the phases of treatment. Use of the procedure requires no other data analytic skills. The SMA program graphs the data points and computes autocorrelation-corrected significance tests of changes in data points over phases. Use of the SMA program to analyze the coping strategy data shown in Figure 3 revealed that both the use of avoidance coping and the use of adaptive coping strategies changed significantly from baseline (reports 1–9) through treatment (reports 10–27).

Issues in Collaborative Assessment

We have highlighted the advantages of systematic tracking of treatment data in collaborative assessment. We should acknowledge, however, that like any other clinical activity, limitations and contraindications arise. One issue concerns the nature and extent of information that should be conveyed. As the traditional issue of “depth of interpretation” would imply, it is important to tailor feedback to what the client can accept and tolerate. The use of diagnostic labels, which can have a pejorative quality, is generally contraindicated, particularly early in treatment. Although the “symptom-specific” face-valid measures of the EAS-C help to avoid this potential problem, clinical sensitivity and skill are required to effectively conduct CA (Finn, 2007; Fischer, 2004). Haydel, Mercer, and Rosenblatt (2011) and Hilsenroth, Ackerman, Clemence, and Strassle, (2002) have emphasized and demonstrated the importance of training in collaborative assessment to enhance practitioner effectiveness.

Among the issues that may arise during CA are indications of lack of progress or even deterioration effects and resulting client demoralization or resentment. A qualitative study by Ward (2008) revealed a number of issues related to CA experienced by both practitioners and clients. Practitioners experienced concerns that feedback might distress or damage the client, as well as challenges in responding to clients’ causal inquiries and issues that arose when the feedback either conflicted with the client’s preexisting conceptions or served as affirmation for a self-deprecating or fatalistic construal of self. Clients expressed similar concerns, as well as anxiety about discovering additional negative aspects of themselves and concerns about how the assessor would regard them. Clearly, these are important issues to keep in mind when employing collaborative assessment. However, both the practitioners and the clients in Ward’s study reported many positive consequences of the CA, such as those cited earlier.

Issues in Computerized Data Collection

Issues also arise in the use of computers and smartphones to collect assessment data. Of obvious importance are methods to ensure the confidentiality and security of the data that are transmitted and stored. Another issue involves client and therapist receptiveness to ongoing collection of clinical data in this manner, for which we do have some initial impressions to report. When we first began using routine (non-web-based) computerized assessment in our clinical setting (by using laptops in the therapy rooms), we were concerned that resistance from clients and/or therapists might make implementation difficult and thereby undermine any potential clinical utility. Specifically, we were concerned that groups historically associated with less computer usage (e.g., clients from lower income brackets, lower levels of education, and the elderly) would have difficulty using the computers or be less likely to agree to the routine assessments. We also were concerned that therapists (trainees in a clinical psychology graduate training program), though computer-savvy and philosophically open to integrating data into clinical practice, would find the routine use of computerized assessment to be too cumbersome or time-consuming. However, we found that both practitioners and clients found the computerized assessments to be useful and user-friendly. Qualitative responses indicated that problems occurred most often with the hardware (e.g., the laptop froze) rather than with the assessment software itself.

Utilizing repeated or continuous computer-based assessment requires some practical steps for the practitioner. Prospective clients should be informed that ongoing assessment is considered a routine part of treatment and given an opportunity to ask questions and discuss possible concerns. Informed consent or office policy statements should include a rationale for computerized assessment as well as a description of the assessments. These statements and policies have proven useful in our use of computerized assessment. If e-mail reminders are to be utilized, clients must also be informed of the security limitations of e-mail.

Since the Fall of 2008, 72 of 75 new clients in our outpatient clinic have agreed to the ongoing assessment procedure. On a post-treatment consumer satisfaction questionnaire, all of our clients (most of whom are low income or students) have expressed their willingness to participate in the routine assessments and indicated comfort in using laptops in our clinic to do so. It therefore appears that clients are generally receptive to the approach.

More usability information is needed, particularly for web-based completion of the EAS-C scales on computers or smartphones, and with a wider range of client populations. Clearly, some contraindications may exist. For example, not all people in lower economic groups have access to computers or smartphones, which would require data collection to occur on laptops at the treatment setting. Another contraindication applies to clients who are not fluent in English or have poor literacy. At the present time, all of the scales in the EAS-C are in English. These issues notwithstanding, evidence exists that remote online assessments achieve reasonably good compliance within a wide variety of socioeconomic and client populations, including smokers, eating-disordered clients, and clients with anxiety and mood disorders and personality disorders (Barak et al., 2008; Bauer et al., 2003; Dimeff et al., 2010; Munoz et al., 2006).

In conclusion, although more research is needed on such issues as dose-response relations and assessment compliance, existing research supports the potential efficacy of both CA and the ongoing collection of outcome and process measures during the course of treatment. We believe that in addition to enhancing therapeutic process and outcome, ongoing assessment of client progress increases transparency and accountability in health care, and fulfills the responsibility of practitioners to assess their effectiveness and monitor the well-being of their clients. Moreover, the development of practice-research networks (Borcovec, 2004) requires that practitioners have a means of collecting single-case data that can be contributed to a collective data bank where they can be analyzed by researchers. All of these functions are enhanced by computerized assessment, which facilitates the efficient collection and productive use of clinical data.

Biographies

RONALD E. SMITH received his PhD in clinical psychology from Southern Illinois University. He is Director of Clinical Psychology Training at the University of Washington. His research is in the areas of sport psychology, stress and coping, and personality.

COREY FAGAN received her PhD in clinical psychology from the University of Massachusetts. She is director of the Psychological Services and Training Center at the University of Washington. She specializes in practicum training, and her major interest is in the integration of science into practice through evidence-based assessment and treatment.

NICOLE L. WILSON received her PhD in Social Psychology and Personality from the University of Washington, where she is currently a Postdoctoral Research Associate. Her major interests are in stress and coping, behavioral signatures, and individual differences.

JESSICA CHEN, M.S., is a doctoral student in clinical psychology at the University of Washington. Her research interests are in personality variables in psychopathology and individual differenes in reactivity to stress.

MARISSA CORONA, M.S., is a doctoral student in child clinical psychology at the University of Washington. Her research interests are in understanding risk for depression among ethnic minority youth and co-occurring health conditions, such as youth obesity.

HONG NGUYEN, M. S., is a doctoral student in clinical psychology at the University of Washington. Her research focuses on cultural issues relating to sexual behaviors and substance abuse, including the influence of culture on sexuality, sexual risk taking, and coping with sexual assault victimization among Asian Americans. Her research also focuses on health disparities and resource utilization among ethnic minorities.

SARAH RACZ, M.S., is a doctoral student in child clinical psychology at the University of Washington. Her major interests include the family processes, specifically bidirectional associations between child and parent behavior that contribute to the development of aggression, oppositionality, and other behavior problems in children and adolescents.

YUICHI SHODA received his PhD in Psychology from Columbia University with a specialization in personality. His major interests are in analysis of within-person patterns of situation-behavior relations and translational research on the application of social cognitive personality theory to clinical assessment and behavior change.