Clinicians’ predictions of patient response to psychotropic medications

Abstract

Clinicians prescribe a medication when they assume that there is a reasonable probability of its success. There are many studies on the predictive value of social or clinical information, but these studies do not include the prognosis made by psychiatrists before treatment. These studies indicate that a small to moderate proportion of the total variance of outcome can be predicted from social or clinical information. It is peculiar that there are very few studies on the accuracy of psychiatrists’ “bets” about the effects of psychotropic drugs when they use the clinical characteristics of patients as predictors, considering the practical relevance of predicting the outcome of a psychiatric treatment. The absence of studies on the accuracy of clinicians’ bets or predictions in psychiatry is unfortunate.

The clinician proposes a pharmacological treatment to patients when he or she concludes that such a treatment is warranted: the clinician should, ideally, write prescriptions on the basis of the prediction of the benefits and risks of the drug (or psychotherapeutic, or social) treatment for each given patient. The actual evolution once the patient has been taking the treatment should correspond as closely as possible to the previously made prediction. In statistical terms, the clinician should predict much of the variance of drug response; and should achieve a high predictive accuracy (defined by the sum of the correct predictions divided by the total number of predictions). In colloquial terms, the goal is to know with little doubt that one is “betting a horse that will be the winner―or among the winners.”

The prediction of outcome is a prerequisite to personalized therapy, ie, a treatment chosen on the basis of the patient characteristics. Several steps precede this choice of therapy. A diseased state has to be recognized as such by both the patient and the physician; a diagnosis should be made in accordance with the profile of complaints and symptoms, as well as with classification criteria; the severity of the disorder should be correctly assessed; and a prognosis should be made. Misunderstanding, ignorance, or error can occur at each of these steps, leading to a decreased accuracy of the predictions of outcome, as well as to a decrease in the usefulness of therapy. These issues are the domains of studies on the rate of recognition of diagnoses, the concordance (or discordance) between structured interviews and therapists' evaluation, the usefulness of asking for a second opinion, interobserver reliability, and the test/retest reliability in scales scoring.

In order to tailor therapy to the individual, the clinician has information that can be classified into three different sets. The first set consists of clinical trials findings where patients were included in trials according to their clinical characteristics and then randomized into treatment subgroups on the basis of demographic or social data and scores on clinical scales. Results from clinical trials are average results, giving an overall probability of favorable response in a predefined patient population. Controlled clinical trials are the basis for evidence-based medicine, a method that is progressively being applied in psychiatry.

The second set of information consists of local or national opinions, or habits about the prescription of medication. Clinical guidelines are an illustration of such information; they combine information from evidence-based medicine and expert consensus statements based on clinical experience. The therapeutic image of psychotropic medications is another illustration of this second set: depending on the region, the period of time, or the marketing activity of pharmaceutical industries, physicians consider a molecule as having a given profile of effects that might not correspond strictly to the facts from clinical trials and evidence-based medicine. Many years ago, we showed that even medical students can have an image of a drug that does not correspond to the known therapeutic efficacy¹: 80% considered that cotrimoxazole (rather than ampicillin) should be used to treat acute urinary tract infection, while 80% considered that ampicillin (rather than cotrimoxazole) should be used to treat surinfection of chronic bronchitis―an unwarranted distinction at that time. For the treatment of insomnia, the prevalent choice among benzodiazepines (ie, the molecules selected most often by the responders) was the following: chlordiazepoxide for a chronic alcoholic patient, nitrazepam for a healthy young man, and oxazepam for an anxious menopausal woman. These answers were in accordance with the local image of the drug, or local belief.¹

The third set of information consists of personal opinions derived from treating patients, from talking to colleagues, and sometimes from wishful thinking or idiosyncratic hypotheses. These three sets of information are used in the decision process; this means that this process has rational and irrational, and explicit and implicit components.

Psychiatric disorders can be grouped into three categories on the basis of their evolution. Several run a chronic course, with aggravation or little improvement in core symptoms. This is the case for dementias, mental deficiency syndromes, and schizophrenia. Fibromyalgia is another disorder with an evolution characterized by unremitting symptoms over years. For these types of disorders, the prognosis, in the etymological sense of “knowing the future,” can be made with facility so long as one considers aspects of the disease for which no treatment has ever been shown to influence the course. Obviously, these aspects are unlikely to change, a situation that does not ask for competence in prediction. However, there are accompanying symptoms and comorbid disorders that can respond to treatment. For example, antidementia or other drugs can modify behavioral dyscontrol in dementia patients; agitation can be favorably managed in mental retardation patients; and fibromyalgia patients can make adaptations that improve their quality of life. It is within the framework of these improvements that the clinician should make a prediction of treatment usefulness. Other psychiatric disorders tend to run a favorable course, for example, many of the acute stress reactions, when the stressor is not too intense and when the threshold of severity remains below the criteria for posttraumatic stress disorder or adjustment disorder. For these types of disorders or situations, the challenge is to identify patients whose evolution might not be spontaneously favorable. Between these two categories of disorders―those showing little improvement over time and those improving spontaneously―there are disorders such as mood or anxiety disorders that have an evolution that differs from patient to patient, even when the baseline severity of symptoms, before treatment, is the same. In these cases, the range of possible outcomes is wide: from complete, rapid, and even spontaneous resolution to death. Table I indicates general aspects of outcome for which a prediction should be systematically made in everyday practice for all treatments and all patients. Patients wish to be informed about the nature of these outcomes, together with clinical comments on their probability.

Table I. Aspects of outcome prediction.

Response versus nonresponse under treatment or no treatment

Comparative efficacy of different treatments

Consequences of response or nonresponse on quality of life

Short-term risk of recurrence versus no recurrence under treatment or no treatment

Long-term risk of recurrence versus no recurrence under treatment or no treatment

Risk of occurrence of medication or psychotherapy side effects

Consequences of medication or psychotherapy side effects

Studies on predictive clinical variables

Variables can be measured at baseline, and then during or at the end of the study, to explore how they relate to outcome. Many prospective studies have been published on the predictive value of clinical variables in psychiatry. A selection of clinical variables included in these studies is presented in Table II.

Table II. Clinical variables included in outcome prediction studies. DSM, Diagnostic and Statistical Manual of Mental Disorders.

DSM diagnosis, symptomatic aspects (configuration of symptoms, severity of symptoms, score on clinical scales)

DSM diagnosis, temporal aspects (age at onset, duration of the disorder, number of recurrences, delay until initiation of treatment)

Comorbidity DSM-III or DSM-lV Axis I or Axis II, eg, personality disorders)

Demographic characteristics (age, gender, race)

Social variables (marital status, profession, income, social support)

Psychological characteristics (intelligence, years of education, insight, health belief model, etc)

Family variables (hereditary disorders, psychiatric or physical disorders in parents, expressed emotion)

Treatment-related variables (primary care or referral services, out- or inpatients, number of prior treatments, patients’ or therapists’ compliance, nature of treatment, dosage, and duration of treatment)

Studies on predictive variables establish to what extent the outcome of a patient is strictly dependent on the application of treatment, or whether, and to what extent, patient-related characteristics influence outcome under treatment. In statistical terms, the goal is to explore what proportion of the variance of the dependent variable (eg, clinical outcome) is explained by independent variables (eg, sex, age, neuropsychological tests results, and comorbidity). Some of the studies were on the relationship of single outcome measures with single predictors. For example, pretreatment cognitive deficits signal an unfavorable outcome of anorexia nervosa.² Other studies used elaborate models, from general linear models to artificial neural networks,³ or complex models that combine multivariate parametric statistics, artificial intelligence, and linguistic qualitative judgments. A few predictive studies in the fields of anxiety and mood disorders are summarized below.

Anxiety disorders

For anxiety disorders, comorbidity with personality disorders appears to predict a lesser response or nonresponsc to treatment.^4-6 In a 5-year follow-up study of patients suffering from anxiety disorders, 182 out of 210 of those initially randomized to drug treatment, cognitive and behavior therapy, self-help, or placebo were evaluated. Sixty percent had a good outcome. Interestingly, clinical evaluation of symptoms 10 weeks after the beginning of treatment was among the strong predictors of outcome 5 years later, whatever the treatment was (even with placebo). In this study, comorbid personality disorders predicted a worse outcome.⁷ Presence of hypochondriacal personality disorder (a personality disorder that is not listed in the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition [DSM-IV] classification system) in 17 of the patients was particularly predictive of a worse evolution of generalized anxiety, panic, or dysthymic disorder at 5 years.⁸ Comorbidity of two anxiety disorders can decrease the rate of response to treatment; this was the case, for example, for posttraumatic stress disorder and obsessive-compulsive disorder.⁹

Mood disorders

In mood disorders, several clinical variables intuitively expected to be predictors of evolution have not been confirmed as such. This is particularly striking for personality disorders, which seem to have no predictive value for outcome in several studies on antidepressant treatments.^10-12 In fact, in these studies, the proportion of patients who responded to the criteria of one or more personality disorders decreased over the duration of treatment, in line with what is known about the pharmacological treatment of Axis II personality disorders.^13,14

However, not all studies led to the conclusion that personality disorders do not influence the evolution of mood disorders. Several studies indicate that personality disorders do play a role; for example, the response to nortriptyline was less in cases of avoidant personality disorder,¹⁵ and bipolar patients with an Axis II comorbid personality disorder tended to keep residual symptoms of depression after remission.⁹ These differences might be explained by the medications used 30 years ago comparative to the present, or by the duration of follow-up, or by changes in populations of patients included in the clinical trials. In a 5-ycar, follow-up study on 86 outpatients, the outcome of dysthymic disorder was dependent on many clinical variables, such as Axis I or Axis II comorbidity and social variables, such as early stressful events.¹⁶

Studies on physicians’ predictions

In these studies, physicians indicate their prediction about the outcome of individual patients and the accuracy of the prediction is tested against the actual clinical evolution. Our search for such studies in the medical literature was a saddening experience: there are almost no studies on therapists’ prediction in psychiatry! We did find six studies.

In the first study, published more than 20 years ago, it was stated that the evolution of 73 nonpsychotic patients receiving psychoanalytically oriented psychotherapy could not be predicted by the therapist.¹⁷ The second study concerned the comparative efficacy of psychotherapy, relaxation, behavior therapy, and amitriptyline in 155 patients followed for 3 months. The pretreatment prediction of outcome by psychiatrists did not correlate to patient outcome, particularly in the recovered or the unremitted groups.¹⁸ In the third study, nurses and psychiatrists rated the likelihood of 308 hospitalized patients of becoming violent. Both professional groups achieved a good total predictive accuracy, with a proportion of cases correctly predicted of 82% to 84%. ¹⁹ The fourth study was on the specific issue of whether clinicians or patients could predict, or rather guess, whether an active medication or a placebo was given. This was a study of depression in child and adolescent outpatients; it showed that clinicians, patients, or parents could not adequately guess whether the patient had received fluoxetine or placebo. This was a study on a conditioned probability, since the subjects knew the quality of the clinical response at the time of their guess about treatment. When the analysis was done taking into account the clinical response at the end of 8 weeks, nonresponse was a strong predictor that clinicians, patients, or parents stated that patient was receiving placebo, while response was a strong predictor that clinicians, patients, or parents guessed that the patient was receiving the active compound. The clinicians predicted medication for the responders, ie, 27 out of 31 patients and placebo for the nonresponders, ie, 26 out of 35 patients. These predictions were not correct.²⁰

The other studies were on the prognostication of 100 alcoholics²¹ and on the course of hospitalization of 62 psychiatric patients.²²

The very limited number of studies in which the direct prediction of clinicians was measured cannot be explained by methodological problems, since there are studies during which it would have been easy to add an initial evaluation of patients’ future outcome by psychiatrists. Such data would not have been difficult to gather: patient outcome could be expressed in simple terms, for example, describing improvement on a 7-point scale such as the clinical global impression (CGI) scale. A more complete methodology would be to ask clinicians to list several outcomes for each patient, and associate probabilities to each of these outcomes.

We will describe a few studies on the variables influencing outcome to demonstrate how unfortunate it is that the simple issue of the quality of physicians’ bets quality was not included in protocols. Fichter et al²³ studied 196 women with bulimia nervosa purging type, 103 women with anorexia nervosa, and 68 women with binge eating disorder. They used path analysis with 14 factors and found many correlations, but only a few of these were statistically significant and related to the outcome of the patients after 6 years. It might have been interesting to compare this multifactorial statistical approach with the performance of clinicians in predicting evolution. Gabriels et al²⁴studied 17 children with a diagnosis of autism and organized a follow-up evaluation at a mean duration of 40 months of treatment. The outcome was not related to treatment; however, pretrcatment developmental intelligence was higher in those with a better outcome. Here also, it would have been easy to explore whether clinicians could have made such a prediction.

Pyne ct al²⁵ studied 59 inpatients with major depressive disorder. They measured a series of variables chosen from the literature for prediction of evolution in mood disorders and, depending on the variables included, obtained an accuracy value of 69% to 86%. Which accuracy levels would clinicians have reached, using clinical information and intuition, had they been tested on this question? Mulder et al¹¹ reported on 183 depressed outpatients who completed a personality disorder assessment and showed that 45% of them had at least one comorbid personality disorder, but that this comorbidity did not influence outcome. One exception was that patients with a cluster C personality disorder responded less well to nortriptyline than to fluoxetine.

Another example is the study by Denys et al²⁶ on the development of a scale for early prediction of obsessive-compulsive disorder response to treatment. The accuracy of the scale was reasonable, with an area under the receiver operator characteristic (ROC) curve of 0.71. Here again, no attempt was made to obtain a prediction from the clinicians. Clinical trials represent a valuable source of information concerning predictors of outcome. For example, the retrospective analysis of 1839 patients in five placebo-controlled studies of venlafaxine prescribed for general anxiety disorders showed that sleep disturbance predicted positive response, while restlessness predicted poor response. Some variables, such as difficulty in concentrating or substance abuse history, predicted positive response to the placebo.²⁷ The predictive variables measured in the above studies have an obvious clinical nature, and the absence of evaluation of clinicians’ performance in predicting outcome represents an unfortunate missing aspect of these protocols.

Biological predictors of outcome

Physiological measures (eg, sleep architecture), pharmacological challenges (eg, the administration of psychostimulants), neuroendocrine baseline values (eg, monoamines, metabolites in plasma, urine, or spinal fluid), neuroendocrine challenge studies (eg,dexamethasone or corticotropin-releasing factor [CRF] tests) have been studied in biological psychiatry research studies for decades. Several predictors of evolution have been identified in these studies, and in a few cases, these predictors explained one-quarter to one-half of the variance of outcome. More recently, developments in pharmacogenomics have opened new avenues for applying predictive medicine techniques to psychiatric disorders. These biological predictive variables are described elsewhere in this issue of Dialogues in Clinical Neuroscience.

Discussion

The concordance between psychiatrists’ predictions, based on clinical impression and intuition, and the actual outcome of psychiatric patients has not been studied correctly. Our search of the medical literature databases (Medline, Excerpta Medica, and Psyclit) may have been incomplete since it was limited to journal articles and did not include chapters in books, but we doubt that this was important. Thus, the publication by Luborsky et al¹⁷ on direct the prediction of psychotherapy outcome by therapists (and patients, as well as clinical observers) and the few other studies have not convinced researchers of the value of knowing how clinicians forecast the evolution of patients who receive psychiatric treatment, and of exploring whether experienced clinicians are better at this than beginners. In contrast, there are many prospective or retrospective studies where the major goal was to find predictors of response in psychiatric patients. None of these included clinicians“ bets, and this is unfortunate. Two major reviews on prognostic methods and outcome prediction^28,29 contained no mention of the issue of physicians’ individual bets on the basis of clinical data. These bets were also not included in the development of an artificial intelligence neural network to predict the need for hospitalization of patients in 658 psychiatric emergency room visits.³⁰

The lack of interest in clinicians’ direct predictions of patient outcome in psychiatry is not found in internal medicine, traumatology, oncology, or a few other medical specialties. We summarize a few studies to illustrate their relevance to clinical practice. An early study by Biorck and collaborators³¹ on the prediction of outcome of 100 consecutive myocardial infarction patients showed that the prediction was quite accurate for those who had a good prognosis or a bad one, but far from accurate for those who had an intermediate risk; experienced physicians did not make more accurate predictions. Another study on a similar question indicated that physicians’ experience played little role in the accuracy of 3-year survival prediction after myocardial infarction, and that mathematical models could surpass the physicians’ performance.³²

In an evaluation of 402 internal medicine patients, 6 physicians achieved predictions of patients remaining alive 5 years later with a sensitivity greater than 0.8, indicating that more than 80% of those surviving more than 5 years were correctly identified as such at the time of hospital discharge. The specificity was 0.6 to 0.8 depending on the physician, indicating that nonsurvivors were identified as such in 60% to 80% of cases.³³

Clinicians have a good capacity to predict patients’ survival during intensive care unit hospitalization, with ROC curve areas of 0.85.³⁴ However, in another study on 713 estimates made by 51 physicians, the prediction of survival after cardiac arrest and cardiopulmonary resuscitation was no better than chance level.³⁵ Also, physicians or nurses could not predict the quality of life in 521 patients interviewed 6 months after hospitalization in an intensive care unit.³⁶ It was also difficult for physicians to predict survival in cases of acute congestive heart failure.³⁷ The capacity of outcome prediction by internists, surgeons, and neurologists has also been studied in cases of patients having undergone severe traumas or burns. These studies were motivated by the need to assess triage decisions, in particular to identify patients too severely ill to survive (and then restrict intensive care unit hospitalization or withdraw treatment to these patients). These vital decisions are based on physicians’ predictions, and it is fortunate that the accuracy of these decisions has been a domain of research in intensive care medicine.

Clinicians have a synthetic and intuitive approach to the prognosis of their patients, but there are no data from which to decide whether to praise or criticize the quality of psychiatrists’ predictions about outcome. The lack of interest in the accuracy of psychiatrists’ direct predictions could be seen as benign neglect: clinicians’ bets are only one among the number of social, psych odynamic, or biological variables that could be included in research protocols on outcome prediction. We propose that this is not benign neglect, for several reasons. First, doctors often wonder about their capacity to predict patients’ evolution in the context of routine treatment, and they would be interested in reading research findings on this issue. Second, the absence of studies on the accuracy of psychiatrists’ predictions of patients’ outcome could be interpreted as a refusal to look into the important question of whether doctors have any idea about the consequence of their prescriptions. Such studies would answer the following question: do we, as clinicians, have any competence in prognostication that is better than chance level? Is it more comfortable to keep ignoring the answer than to confront ourselves with the possible conclusion that we are no good at predicting the clinical outcome of patients? Third, there is the obvious fact that psychiatrists are confronted daily with demands of predicting the risk that patients might not respond to treatment, might need to be hospitalized, might become violent, or might commit suicide. Complex multivariate predictive models including clinical and biological variables are being studied and will become available to psychiatrists in everyday practice. A better capacity and confidence in prognostication in the practice of psychiatry will represent a significant change, and help us forget that we have been working for decades not knowing the accuracy of our direct clinical predictions of patient outcome.