Treatment Progress Indicator: Application of a New Assessment Tool to Objectively Monitor the Therapeutic Progress of Patients With Depression, Anxiety, or Behavioral Health Impairment

Abstract

Depression, anxiety, and behavioral health impairment are common in the US. Efforts to treat patients with depression, anxiety, and chemical dependency are surpassed by the great demand for psychiatrist and therapist appointments. Unlike other specialties, psychiatry lacks a vital sign or tests (eg, blood pressure for hypertension and hemoglobin A_1c for diabetes) to objectively measure a patient’s response to therapy. A new tool called the Treatment Progress Indicator (TPI) appears to offer clinicians an objective approach to assess patients and to monitor therapy over time. The TPI report shows an Expected Treatment Response that allows clinicians to compare their patients’ responses to therapy with those of a cohort of patients with a similar diagnosis and severity. The TPI allows for objective population-level management of disease. This should result in sharing of best practices that will lead to more effective and efficient care plans. Widespread implementation of the TPI has the potential to improve the time required to diagnose conditions and triage patients. As a result, implementation of the tool has the potential to improve behavioral health access and affordability.

Introduction

Disorders such as depression, anxiety, and substance abuse are among the most common and disabling health conditions in the US.¹ They often co-occur with chronic medical conditions, and untreated, they can worsen associated health care outcomes.² When behavioral health problems are not effectively treated they can be associated with poor health outcomes, increased mortality, decreased work productivity, and an increase in health care costs.^3–6

According to Unützer et al,⁷ most medical patients with depression, anxiety, and/or chemical dependency may not receive adequate therapy. Efforts to improve the treatment of these disorders in primary care initially focused on screening, education of physicians, development of treatment guidelines, and referral to psychiatry. Although these interventions have not been shown to independently improve care, each area has the potential to contribute to improved mental health outcomes.⁸

Every day clinicians must make decisions about patient care. They must decide whether a patient’s condition warrants treatment, referral, or watchful waiting. If treatment or referral seems appropriate for a patient, the clinician determines if the patient will be treated with medication or individual, group, or combination psychotherapy. After treatment begins, the physician must evaluate whether the patient is responding satisfactorily. All of these decisions pose unique challenges for behavioral health conditions because there are no vital signs, radiologic studies, or blood tests to rely on for diagnosis, treatment planning, and monitoring of the underlying disorder.

Some psychiatrists use one or more psychological tests to assess severity of the disorder and the patient’s progress (eg, Patient Health Questionnaire-9, Beck Depression Inventory, Symptom Checklist 90, Generalized Anxiety Disorder scale). These tests can be very useful but are typically narrowly focused on a single disorder (eg, depression or anxiety). Patients in specialty care commonly report symptoms of multiple disorders. Attempts to establish a broadly acceptable battery of measures have been unsuccessful, owing to differing views of which measures to include, and the length of time required for patients to complete the battery.

Currently, patients with diagnosed depression, anxiety, and/or chemical dependency may be treated in primary care or referred for psychotherapy or to psychiatry. However, most primary care physicians do not have an effective tool to objectively determine if a patient could benefit from a primary care intervention such as a health education class or an appointment with a case manager versus an appointment with a psychiatrist. The patient may wait several days to see a psychiatrist whereas intervention with a frontline case manager or social worker would be immediately available. This delay in care is concerning to patients and may prevent early interventions with the potential to stabilize an acute disorder.

Medical patients are considered to have a positive response to treatment when their symptoms remediate, their life functioning improves, and they feel better. A universally accepted mental health services screening and treatment progress indicator tool may be helpful in providing similar indicators of patient response to treatment for mental health conditions. It could be helpful for visualizing trends in improved functioning and symptom remission over the course of therapy, and in promoting multi-disciplinary collaboration for improved mental health outcomes.

Treatment Progress Indicator

The Treatment Progress Indicator (TPI) was created to improve access to care and help clinicians get a better assessment of a patient’s overall behavioral health impairment (BHI) and monitor their response to treatment. TPI is a Web-based self-report assessment tool (see Sidebar: Treatment Progress Indicator Self-Assessment Tool). It screens for the most common behavior disorders and measures the severity of the patient’s depression, anxiety, global symptoms, social functioning status, global functioning status, and subjective well-being. These data are combined to derive a global BHI score useful for tracking response to treatment and for risk stratification. The TPI alerts the clinician to severity and presence or absence of indicators of substance abuse, obsessive-compulsive disorder, bipolar disorder, posttraumatic stress disorder, and panic disorder. It includes items for treatment history, use of psychoactive medications, and resilience. It does not assess psychosis, schizophrenia, phobia, or personality disorders.

Treatment Progress Indicator Self-Assessment Tool.

How well have you been getting along emotionally and psychologically?

1. Quite poorly, can barely manage to deal with things

2. Fairly poorly, life gets pretty tough for me at times

3. So-so, manage to keep going with some effort

4. Fairly well, have my ups and downs

5. Quite well, no important complaints

6. Very well, much the way I would like to

During the past two weeks, how much have you had to cut down on the amount of time you spend on work or other activities as a result of any emotional problems (such as feeling depressed or anxious)?

1. Not at all

2. Cut down a little

3. Cut down a lot

Are you taking medications for a psychological problem?

1. Yes

2. No, but I have been advised to

3. No

During the last two weeks, how often have you been bothered by having little interest or pleasure in doing things?

1. Not at all

2. Several days

3. More than half the days

4. Nearly every day

During the last two weeks how often have you been bothered by feeling down, depressed, or hopeless?

1. Not at all

2. Several days

3. More than half the days

4. Nearly every day

During the last two weeks, how often have you been bothered by trouble falling or staying asleep, or sleeping too much?

1. Not at all

2. Several days

3. More than half the days

4. Nearly every day

During the last two weeks, how often have you been bothered by feeling tired or having little energy?

1. Not at all

2. Several days

3. More than half the days

4. Nearly every day

During the last two weeks, how often have you been bothered by poor appetite or overeating?

1. Not at all

2. Several days

3. More than half the days

4. Nearly every day

During the last two weeks, how often have you been bothered by feeling bad about yourself—or that you are a failure or have let yourself or your family down?

1. Not at all

2. Several days

3. More than half the days

4. Nearly every day

During the last two weeks, how often have you been bothered by trouble concentrating on things, such as reading the newspaper or watching television?

1. Not at all

2. Several days

3. More than half the days

4. Nearly every day

During the last two weeks, how often have you been bothered by moving or speaking so slowly that other people could have noticed? Or the opposite—being so fidgety or restless that you have been moving around a lot more than usual?

1. Not at all

2. Several days

3. More than half the days

4. Nearly every day

During the last two weeks, how often have you had thoughts that you would be better off dead, or of hurting yourself in some way?

1. Not at all

2. Several days

3. More than half the days

4. Nearly every day

During the last two weeks, how often have you had thoughts of harming someone else?

1. Never

2. Sometimes

3. Often

In the past two weeks, how often have you felt hopeless or pessimistic about the future?

1. Never or rarely

2. Some of the time

3. Often

4. All or almost all of the time

In the past two weeks, how often have you felt tense or anxious?

1. Never or rarely

2. Some of the time

3. Often

4. All or almost all of the time

In the past two weeks, how often have you worried too much about things?

1. Never or rarely

2. Some of the time

3. Often

4. All or almost all of the time

In the past two weeks, how often have you been in places or situations that you fear?

1. Never or rarely

2. Some of the time

3. Often

4. All or almost all of the time

In the past two weeks, how often have you had repeated thoughts or images that wouldn’t go away?

1. Never or rarely

2. Some of the time

3. Often

4. All or almost all of the time

In the past two weeks, how often have you had repeated disturbing memories, thoughts, or images of a frightening past experience?

1. Never or rarely

2. Some of the time

3. Often

4. All or almost all of the time

In the past two weeks, how often have you been satisfied with your relationships with others?

1. Never or rarely

2. Some of the time

3. Often

4. All or almost all of the time

In the past two weeks, how often have you felt guilty about your alcohol or nonmedical drug use, or that you should cut back?

1. Never or rarely

2. Some of the time

3. Often

4. All or almost all of the time

5. No use

In the past two weeks, how often have you felt that your health, work, or home life was affected by drinking or nonmedical drug use?

1. Never or rarely

2. Some of the time

3. Often

4. All or almost all of the time

5. No use

In the past two weeks, how often have you had severe mood swings (highs and lows)?

1. Never or rarely

2. Some of the time

3. Often

4. All or almost all of the time

In the past two weeks, how well have you been able to manage your day-to-day life?

1. Very poorly

2. Fairly poorly

3. Fairly well

4. Very well

In the past two weeks, how well have you been able to get along with friends?

1. Very poorly

2. Fairly poorly

3. Fairly well

4. Very well

In the past two weeks, how well have you been able to perform work/school/household tasks?

1. Very poorly

2. Fairly poorly

3. Fairly well

4. Very well

In the past two weeks, how well have you been able to participate in your usual social activities?

1. Very poorly

2. Fairly poorly

3. Fairly well

4. Very well

I am able to bounce back when things go wrong (eg, divorce, death of a friend, loss of job, etc).

1. Strongly disagree

2. Disagree

3. Slightly disagree

4. Slightly agree

5. Agree

6. Strongly agree

When I have problems, I go to people (eg, clergy, helpful family members, close friends, etc) who can help me.

1. Strongly disagree

2. Disagree

3. Slightly disagree

4. Slightly agree

5. Agree

6. Strongly agree

I am confident that treatment can help me.

1. Strongly disagree

2. Disagree

3. Slightly disagree

4. Slightly agree

5. Agree

6. Strongly agree

How long have you had the problem(s) for which you are now seeking treatment?

1. Less than two months

2. Two to three months

3. Four to six months

4. More than six months

How many times have you been in counseling or psychotherapy before now?

1. Never

2. Once

3. Two or three times

4. Four or more times

Have you ever been hospitalized for a psychological or emotional problem?

1. Never

2. Once

3. Two or more times

The TPI system was developed over a decade by Polaris Health Directions (www.polarishealth.com) with extensive involvement of researchers, experts in assessment of mental health treatment outcomes, clinicians, and information technology specialists. The work was sponsored by the National Institute of Mental Health and grounded in a decade of basic research.^8–13 Measures of the severity of multiple mental health conditions were developed and validated in both medical and mental health settings.^8,9,14 Concurrent validation of the depression scale versus clinician evaluations has been established using patient samples from private and public clinics.^14,15 Foundational research for the TPI also included a series of studies relating “doses” (sessions) of treatment to improvement in behavioral health symptoms and functioning, and in the global BHI score.^16–20 The TPI was developed using item reduction analysis to reduce the length of the assessment to 29 questions by deleting items that had minimal impact on the reliability of the scales or the correlations of the shorter with the longer versions of the same scale. Internal consistency reliability is acceptable for all 6 TPI scales and good or excellent for 5 of them (Table 1).

Table 1.

Treatment Progress Indicator internal consistency reliability

Scale	Reliability
Depression	0.82
Anxiety	0.82
Symptoms	0.91
Social impairment	0.77
Functional impairment	0.85
Behavioral health impairment	0.86

The TPI can be administered using a computer or fax form, typically upon admission to treatment and at clinic-specified intervals throughout an episode of care. The interval between assessments is at the discretion of the clinic. Most commonly it is administered immediately before every treatment session, or at two-week intervals. Assessment reports are available for clinician review immediately upon completion of the computer assessment and within 24 hours for the fax version. Excerpts of assessment reports are shown in Table 2 and Figure 1.

Table 2.

Assessment results for symptom and functioning scales in 2011^a

Symptoms	Percentile
	July	August	September
Overall behavior impairment	77	57	10
All symptoms	48	53	27
Depression (PHQ-9 raw score)	59 (11)	59 (11)	38 (7)
Anxiety	54	82	39
Overall functional impairment	80	49	19
Social functional impairment	45	45	45

^aLow scores are less severe.

PHQ = Patient Health Questionnaire.

Figure 1.

Tracking of actual treatment response vs expected treatment response

Table 2 shows the percentile score for the overall BHI and subscale percentile scores for depression, anxiety, and functional impairment. In this particular case the data show that the patient improved on all metrics except the social functional impairment metric. Despite this observation the overall Behavioral Impairment (BHI score) percentile improved markedly. Of note is the finding that the BHI score went from 80% to 19% despite only moderate improvement in depression and anxiety scores.

Expected Treatment Response

In addition to providing data for an individual, the TPI chart shown in Figure 1 also provides a graph showing the patient’s actual treatment response compared to the Expected Treatment Response (ETR) for BHI. The ETR algorithm was constructed from TPI data for thousands of patients presenting for treatment at Kaiser Permanente (KP) Southern California psychiatry clinics. The ETR curve was derived through hierarchical linear modeling of change in the patient’s global behavioral health condition as a function of initial severity and other prognostic factors, including treatment history, patient strengths, and expectations.^13,15,18,19 Therefore, the ETR provides a standardized, databased method for monitoring response to therapy for an individual patient compared to similar patients in the database at specific time intervals. The clinician’s decision concerning intervention is guided by projections of the likely course. If the clinician decides to treat, the actual treatment response can be compared with the expected treatment response to determine whether the patient is making adequate progress.

Although few patient trend scores will precisely track the ETR, a series of two or more scores that are substantially less favorable than the ETR may indicate that the patient is not responding as well as expected to therapy (see Figure 1, second assessment actual BHI score). However with treatment adjustment, the actual treatment response may improve more than the ETR for a similar patient profile (see Figure 1, third assessment actual BHI score), indicating that response to therapy is better than expected. This is potentially a very powerful tool as it may help guide therapy and may open the door to identify and spread best practices across the continuum of care.

Treatment Progress Indicator Benefits

By providing a global measure of behavioral health status as well as measures for common disorders and comorbidities, TPI may offer an efficient method for helping clinicians decide whether to treat. If the decision is to treat, TPI can help to determine whether the patient is making satisfactory progress in relation to an evidence-based expectation of progress. Monitoring treatment response may enable clinicians to identify patients who need more intensive therapy, or those that can be safely discharged or stepped down. Treatment approaches that consistently achieve unusually positive outcomes compared to ETR may be identified as best practices and shared.

Traditional evidence-based medicine supports treatment practices that have been shown to improve care. However, no behavioral health treatment is effective for all patients. The TPI enables us to determine whether a patient is receiving the “right” treatment based not upon a specific evidence-supported approach but based upon the answer to the question, “Is the patient making the progress that would be expected on the basis of their initial severity and other factors known to impact the rate and amount of patient improvement?” In clinical practice it is less important to know whether a treatment works in general than to know whether it is working for the patient under care.

The TPI can also be used to identify patients who present in psychiatry clinics who may be candidates for other care pathways. Those with mild, uncomplicated presentations may be candidates for Wellness Coaching or Health Education. Patients with unipolar depression and no significant comorbidities may be candidates for Depression Care Management. Analyses of data for 11,700 patients from 9 KP psychiatry clinics indicates that as many as 15% may be candidates for these less intensive interventions. In addition, having a way to objectively measure BHI sets the stage for triage to occur upstream, even in the primary care setting. Some patients’ care might be shifted to other departments including health education, frontline depression case managers, and social workers in primary care. This has the potential to improve access in psychiatry for patients who would most benefit from specialty care (eg, patients with severe bipolar disorders or schizophrenia, and patients at risk for self-harm). The TPI may allow us to assess which patients need to be referred to the Psychiatry Department and “with further research” which patients could be managed in primary care. The TPI’s evidence-based tracking of patients’ responses to therapy should help clinicians identify patients who can safely be returned to primary care. By reducing the need for follow-up appointments in Psychiatry, this could further increase access to new referrals.

The Future for the Treatment Progress Indicator

Every day thousands of patients are treated for depression, anxiety, and functional impairment. Each patient represents potential for learning and performance improvement. Our goal as clinicians is to better understand which types of interventions are most effective for which types of patients. Because there is currently no objective way to measure a patient’s emotional vital sign in the behavioral health setting, there is no optimal way to identify and to share standardized best practices among clinicians. The TPI is designed to harvest data to allow clinicians to better understand what works and for whom.

The utility of ETR curves will continually increase as new patient data are added to the TPI database. The capacity to routinely capture and make use of the treatment histories of real patients constitutes a learning system and a potential milestone for behavioral health. Large patient volume gives organizations like KP a unique opportunity to use the TPI to establish and to continually refine ETRs for all treatment modalities. ETRs can be established for group psychotherapy, individual psychotherapy, medication management, and combinations of the three (eg, medication and individual psychotherapy).

Numerous measures exist for detecting individual disorders such as depression, but there are very few instruments that measure all components assessed by the TPI or do so efficiently. TPI compares well with alternative tools when practical issues such as the number of items, time required for completion, mode of administration, complexity of scoring, customizability, and special training requirements are considered. The scope of the TPI is important because patients often report multiple problem areas. The ability of the TPI to monitor treatment response for patients with multiple problems is therefore important, and could be further strengthened when ETRs are derived for specific treatment modalities.

Detecting depression, anxiety, chemical dependency, suicidality, and functional status would be helpful both for the behavioral health specialists as well as primary care physicians. In each setting, determination of appropriate care pathways is best for the patient and for improving access to the most intensive treatments. In both settings, evidence-based tracking of patient response to treatment is critical. If a patient is responding to therapy, improvement can be documented using the TPI instead of relying on subjective, nonquantitative documentation in the patient chart. The TPI gives us the ability to identify best treatment strategies that could be shared with other physicians. The TPI would allow identification of psychiatry clinic patients who improve with therapy and who could return to their primary care physician. If symptoms recur, the TPI could be administered to determine if the patient needs to be referred back to behavioral health medicine. Return to the primary care physician will help increase access to behavioral health medicine.

Finally, the TPI can be used to quickly identify patients who are not responding to therapy. As a result, treatment plans can be reviewed and adjusted (eg, check for adherence to medication prescription, appropriate dosage of medication, or intensity of psychotherapy). A simple self-administered measure to complete either in the clinic, by secure messaging, online, or by telephone administration would be an efficient means to assess the nature and severity of conditions screened for by the TPI.

Conclusion

Physicians prefer to quantify a disorder when possible. The TPI might be considered a type of laboratory test or an emotional vital sign. Like blood glucose readings, the TPI may serve as an entry point for patients with behavioral issues and their physicians to communicate about disease control, decide among treatment options (including “watch and wait”), monitor progress, and, when appropriate, adjust therapy.

Multisymptom screening together with brevity, reliability, and construct and criterion validity makes the TPI tool an attractive dual-purpose instrument for making diagnoses and assessing severity of depression, anxiety, and chemical dependency disorders, particularly in the busy setting of clinical practice.

Some of All

Is the clinician a biochemist, a biophysicist, a biologist, a pathologist, a psychologist, a psychiatrist, a social scientist, a statistician? In my view, he is none of these and at the same time he must be something of all of them.

— John Romano, MD, 1908–1994, American physician, psychiatrist, and educator