Use of Meta-analytic Results to Facilitate Shared Decision Making

Abstract

Objectives: Describe and evaluate an Internet-based approach to patient decision support using mathematical models that predict the probability of successful treatment on the basis of meta-analytic summaries of the mean and standard deviation of symptom response.

Design: An Internet-based decision support tool was developed to help patients with benign prostatic hypertrophy (BPH) determine whether they wanted to use alpha blockers. The Internet site incorporates a meta-analytic model of the results of randomized trials of the alpha blocker terazosin. The site describes alternative treatments for BPH and potential adverse effects of alpha blockers. The site then measures patients' current symptoms and desired level of symptom reduction. In response, the site computes and displays the probability of a patient's achieving his objective by means of terazosin or placebo treatment.

Setting: Self-identified BPH patients accessing the site over the Internet.

Main Outcome Measures: Patients' perceptions of the usefulness of information.

Results: Over a three-month period, 191 patients who were over 50 years of age and who reported that they have BPH used the decision support tool. Respondents had a mean American Urological Association (AUA) score of 18.8 and a desired drop in symptoms of 10.1 AUA points. Patients had a 40 percent chance of achieving treatment goals with terazosin and a 20 percent chance with placebo. Patients found the information useful (93 percent), and most (71 percent) believed this type of information should be discussed before prescribing medications.

Conclusions: Interactive meta-analytic summary models of the effects of pharmacologic treatments can help patients determine whether a treatment offers sufficient benefits to offset its risks.

Up to now, meta-analysis has been a tool used by medical professionals mainly to summarize evidence of the efficacy of therapies. In this paper, we describe the application of meta-analysis to facilitate shared decision making between patients and physicians over the Internet.

Background

Clinical Application Area

Benign prostatic hyperplasia (BPH) is a highly prevalent disorder in aging men. More than 400,000 transurethral resections of the prostate are performed annually in the United States to treat BPH.¹ Nonsurgical therapy for BPH has focused on alpha adrenergic receptor antagonists and androgen-blocking agents. The first successful treatment of BPH with nonselective alpha antagonists was reported in 1976.² Since then there have been many trials of both nonselective and more selective alpha blockers, including prazosin (Minipress), terazosin (Hytrin), doxazosin, alfusozin, indoramin, and others. The proposed mechanism of action of alpha blockers is relaxation of smooth muscle in the bladder neck, allowing increased flow of urine during micturition.

Most studies of alpha blockers show that treatment yields statistically significant drops in BPH symptom scores and improvements in urinary flow parameters. The primary issue for terazosin, thus, is not whether the drug is efficacious but whether the patient feels that the effectiveness of the drug warrants the expense and inconvenience of a therapeutic trial as well as a delay in obtaining a more definitive treatment. Other, more effective treatments for BPH exist, including transurethral resection of the prostate, laser prostatectomy, transurethral incision of the prostate, and numerous other “minimally invasive” surgical therapies.¹ Although these alternative treatments have greater risks, they also have greater benefits.

Shared Decision Making Using Meta-analytic Models

When several treatments exist for a particular condition, some of which offer more efficacy but with higher risks and costs, much debate ensues regarding how best to determine whether an individual patient should undergo such therapy. Many health professionals believe that such a decision should be shared with the patient. Shared decision making requires mutually agreed-on treatment goals as well as a mutual understanding of the prospects for success. One approach to implementing this ideal is to measure patients' treatment goals using a validated psychometric scale, and compare these goals to what can realistically be expected, based on available scientific information.

We propose to extend this concept by quantifying the gap between a patient's symptoms and the patient's goal, using the mean and standard deviation of the treatment's actual effect as observed in clinical trials. So calibrated, it is then possible to calculate the probability that an individual patient will achieve this goal (or a greater reduction) in symptoms. ▶ illustrates the principles behind this calculation (see the Methods section for a mathematical description). The approach creates an infrastructure that fosters shared decision making by allowing each patient to determine the specific desired level of reduction in symptoms and by allowing the patient to know the precise probability of success, as defined by the patient's own goals of treatment.

Figure 1.

Graphic representation of the probability that an individual patient will achieve his or her treatment goals. Using meta-analysis of clinical trials, the expected mean percentage change in symptom score (and standard deviation of that change) can be calculated. With this information, the probability that a patient will achieve at least the desired drop in score can be estimated using the normal distribution.

In the past, shared decision making occurred only in close conjunction with health professionals. Even using computer-based tools, the presence and the advice of a health professional have generally been required. The growth, popularity, and ever-improving technology of the Internet have made it possible for patients to directly access customized medical information that, in theory, should empower them to participate in decision making. In the remainder of this paper, we describe an application of our model to shared decision making via interactive access to meta-analytic models, the implementation of this approach in an Internet site, and patients' evaluations of the usefulness of the site.

Methods

Meta-analytic Model

First we developed a meta-analytic model of the effects of terazosin on symptoms of BPH. To identify relevant articles, we performed Medline and BIOSIS literature searches using the MeSH headings hyperplasia and adrenergic antagonists and keywords hyperplasia, adrenergic, prostate, and terazosin. We examined the references from each paper to discover additional relevant trials. Only randomized clinical trials of male patients with symptomatic BPH treated with terazosin or placebo were included in our analysis. A trial was excluded from symptom score analysis if it did not report symptom scores or if the main outcome was not BPH symptom relief.

Several scoring systems have been used for measuring the severity of BPH symptoms. Most trials of terazosin have used the Boyarsky et al. (FDA)³ or American Urological Association (AUA)⁴ instruments. Both instruments use several dimensions of urinary symptoms (nine in Boyarsky et al., seven in AUA), which are scored on an ordinal scale (from 0 to 3 or 0 to 5, respectively). Higher scores indicate more severe symptoms; a fall in score with treatment indicates a positive response. The reliability and validity of the AUA score have been studied in detail.^4,5,6

For each trial identified, we extracted symptom scores at baseline and at the end of the study, the reported change in symptom score, and the measure of variation (standard deviation or standard error) of the change in score. In some trials doses were increased incrementally to a specified level. Because terazosin dosing can be rapidly increased in clinical practice, we used the results only from the highest administered dose.

We modeled the effect of terazosin as producing a constant proportional (or percentage) decrease in symptom score across different levels of symptoms. According to this model, patients with more symptoms might expect larger effects (in terms of the raw drop in score) than patients with fewer symptoms. This model makes better clinical sense than a “fixed” drop across the AUA symptom scale, is consistent with the results reported in two of the larger clinical trials,^7,8 and has been used in other meta-analytic studies of the effects of other alpha blockers.⁹

We estimated the summary mean effect size across all randomized trials using the inverse-variance meta-analysis method and a random effects model.¹⁰ We summarized the measure of variability of change in symptom score by calculating the average observed standard deviation across all trials. When the standard deviation or standard error was not stated, we estimated it from the report's graph or from the standard deviation of the end-of-study mean symptom score, which tends to overestimate inter-individual variation in drug effect.

Calculation of the Probability of Achieving Treatment Goals

We estimated the distribution of outcomes for an individual patient from the meta-analysis results calculated above. This model assumes that the expected change in patient's symptom score is estimated by the observed effect in a population of treated patients. As noted, we modeled the percentage change in the symptom score for an individual patient after treatment with either terazosin or watchful waiting as being distributed approximately normally. The probability that a patient will experience a percentage change in symptom score (ΔS_%) that reduces his symptom score beyond some specified threshold level of reduction (ΔT_%) is

which is the proportion of the normal distribution (Norm), with meta-analytic mean change, μ, and standard deviation, both expressed on the percentage scale. For example, suppose a patient's current level of symptoms was 20 AUA points, his target level of symptoms was 10 AUA points (a 50 percent drop), the expected drop in symptom score with terazosin treatment was 40 percent, and the standard deviation in the expected drop was 30 percent. The probability that this patient would achieve the degree of relief he desired, or more, is 1 - Norm_X≥50% (40%, 30%).

Delivery over the Internet

One limitation of previously developed decision support tools has been their unavailability to patients and physicians at the point of care. To overcome this difficulty, we developed an Internet site (http://prefdev.ucsd.edu/bph/calculator.html) to present background information on treatments of BPH and to perform the calculations described above. The site was designed for patients to use either at home or in their physician's office. Users of the site are first presented with a brief overview of common symptoms of BPH and potential treatment options. They then can access, via hyperlinks, other Internet-based educational resources for BPH. Users are then required to digitally sign an online consent form. Participants enter their name into an electronic version of a standard consent form. They then receive an alias (e.g., a “username”) and a password with which to log into a protected section of the site. They then can browse information on BPH treatments.

When they are finished, patients complete an online questionnaire, recording whether they believe they have BPH (“yes,” “no,” and “maybe” responses allowed), whether they are receiving treatment for BPH symptoms, and the type of treatment they are receiving. They then respond to an online version of the AUA questionnaire. After submitting this data, participants complete a second, modified version of the AUA questionnaire. This version asks patients to characterize, for each item on the AUA scale, the maximum level of symptoms that they are willing to tolerate on a long-term basis. Patients then submit this second form.

Using the difference between a patient's current symptom score and his maximum tolerable score, the site computer calculates the patient's probability of achieving his threshold for relief. It then generates a Web page for the patient, displaying in graphic format the probability of achieving the patient's specified level of relief with either terazosin or placebo (▶). The patient is then shown additional data on the risks of adverse effects with terazosin and is urged to discuss this material with his physician.

Figure 2.

Display of individualized estimates of the probability of achieving specified treatment goals by terazosin treatment and watchful waiting. After a patient enters his current symptom score and the maximal level of symptoms he would be willing to tolerate, using the standard and a modified AUA symptom score questionnaire, the site calculates a patient-specific probability of achieving this treatment goal by either terazosin treatment or watchful waiting and displays this probability graphically.

After the evaluation, participants complete online response forms evaluating their experience using the site. This questionnaire explores patients' perceptions of the usefulness of the information and asks whether they believe physicians should be required to present this information to patients before prescribing medication for BPH. Participants could also send comments to the study center by typing them into a text field on the electronic form.

Subject Recruitment

The Internet site actively enrolled participants for six months. To help subjects find our site, we posted descriptions of the site to major commercial Internet index companies (Yahoo, Excite, Lycos, etc.).

Computer Software

Meta-analyses of symptom scores were performed in Microsoft Excel 7.0. The Internet site was implemented on a Macintosh computer using the Starnine 2.1 server (Starnine Software, Marina Del Ray, California). Interactive site programming and recording of patients' responses in the online study were performed using FileMaker Pro 3.0 (Claris Software, Cupertino, California). Communication between FileMaker Pro and the Web server was managed by the Web FM 2.0 common gateway interface program (Web Broadcasting, Palo Alto, California).

Results

Development of the Meta-analytic Model

Medline and BIOSIS searches revealed 110 potentially relevant studies, of which eight^{7,11,12,13,14,15,16,17} were randomized, double-blind clinical trials comparing the effects of terazosin and placebo. One trial¹² was excluded from the analysis because only changes in obstructive, but not total, symptom scores were reported. Standard deviations (or estimates of them) for the changes in symptom scores were available for all but one trial.

▶ shows characteristics of the terazosin trials included in the meta-analysis. The random effects model estimates of the percentage change in symptom score was 39 percent (95% CI for random effects, 35 to 43 percent) for the terazosin group (N = 1,562 patients) and 19 percent (95% CI, 15 to 23 percent) for the placebo group (N = 1,555 patients).

Table 1.

Summary of Randomized Clinical Trials of Terazosin for Treatment of Benign Prostatic Hyperplasia

Study	Max. Dose	N	Baseline Score	Drop in Score	% Change
Brawer et al.11
Terazosin	10	73	10.9	4.6	42
Placebo	-	74	10.4	1.1	11
Elhilali et al.13
Terazosin	10	66	11	3	27
Placebo	-	66	11	1.35	12
Lepor et al.14
Terazosin	10	54	10.1	4.5	45
Placebo	-	55	9.7	2.3	24
Lepor et al.15*
Terazosin	10	275	16.2	6.1	38
Placebo	-	264	15.8	2.6	16
Lloyd et al.16
Terazosin	10	22	11.8	5.3	45
Placebo	-	20	11.2	2.5	22
Roehrborn et al.7*
Terazosin	10	976	20.1	7.6	38
Placebo	-	973	20.1	3.7	18
Soloway et al.17
Terazosin	20	96	NR	NR	43
Placebo	-	103	NR	NR	30
Notes: Max. indicates maximum; N, number of patients in efficacy analysis; NR, not reported.

^*Studies used American Urological Association scoring system.

Inter-individual variability in symptom response was similar for terazosin and placebo. The standard deviation for the percentage drop in symptom score was 38 percent with terazosin treatment and 41 percent with placebo treatment. We used these figures to calculate the probabilities of clinical success (symptom score drops to post-treatment scores lower than individually determined thresholds) for patients accessing our Internet site.

Evaluation Studies

Over a 12-month period, 633 patients with unique first and last names accessed the site and enrolled in the study. The characteristics of enrollees are shown in ▶. Two hundred twenty-eight enrollees indicated that they had BPH. Of these, 191 were 50 years of age or older. These subjects' responses are reported below.

Table 2.

Characteristics of Study Enrollees

Characteristic	Number of Subjects	% Enrolled
Signed consent form	633	100
Used decision support tool	384	61
Provided a clinical history and demographic data	307	48
Reported that they were sure they had diagnosis of BPH	228	36
Older than 50 years of age	191	30

The group was well educated (81 percent reported at least some college education). There were no obvious pseudonyms; 71 percent of patients provided electronic mail addresses where they could be contacted if further information was needed.

Seventy-six percent indicated that they were receiving some treatment for BPH. Of these, 83 percent indicated that they were currently receiving medical therapy for BPH. The mean level of current symptoms reported by patients was 19.3 AUA points (95% CI for mean, 18.2 to 20.4 points). The mean maximum tolerable level of symptoms was 9.5 points (95% CI, 8.6 to 10.4 points). The average desired drop in symptoms was 10.1 AUA points (95% CI, 8.6 to 10.2 points). There was a strong correlation (r = 0.67) between the desired drop in patients' symptom scores and their current level of symptoms (▶).

Figure 3.

Association between the desired level of reduction in symptom score and a patient's current symptom score (r = 0.67). A few patients were satisfied with their current level of symptoms and indicated that the maximal tolerable level of symptoms was higher than their current score. These patients have negative values for their desired reduction in symptom score.

The average calculated probability that a patient would achieve his treatment goal with placebo therapy was small (20.4 percent; 95% CI for mean probability, 14.5 to 24.4 percent). The average probability that a patient would achieve his treatment goal with terazosin was substantially higher (40.6 percent; 95% CI, 34 to 46 percent). Only about 25 percent of patients had better than a 50 percent chance of achieving their treatment goals with terazosin.

Patients found this information useful (95 percent) and helpful for future decision making (92 percent). Seventy-two percent of patients agreed with the statement that “a doctor should require his/her patients to view this kind of electronic information prior to prescribing medicines to his/her patients.” Comments from patients using the site strongly suggested that they were, in fact, BPH patients. About 50 percent of the target group provided open-ended comments on the study. Many patients felt that this was precisely the type of information they wanted to receive in interactions with their physicians. A few patients expressed concern about the validity of the predictions; they were not satisfied with the level of symptoms yet had a high predicted probability of success. Sample comments from patients who used the site can be viewed on the Internet at http://prefdev.ucsd.edu/bph/comments.html.

Discussion

One way to encourage patients to participate in medical decision making is to allow them to determine the threshold for symptom reduction that they feel is minimally acceptable. A patient's current and target level of symptoms can be used to estimate the probability of a successful response to therapy using evidenced-based methods. This probability can then be used to guide an informed conversation between physician and patient about the risks and benefits of therapy. If a patient can comprehend simple concepts of probability, it is reasonable to present predictions of treatment outcomes, allowing the patient to decide whether a therapy affords sufficient benefit to outweigh its potential risks and inconvenience.

Treatment of BPH with terazosin is a good example of shared decision making. Terazosin is an efficacious treatment for BPH, in that nearly every trial of terazosin resulted in “statistically significant” drops in symptom scores. But how clinically important are the effects of this drug? The premise of shared decision making is that individual patients with BPH can determine this themselves. Many studies of trials of medical therapies of BPH have used as a threshold a drop in symptom scores of more than 30 percent to define “clinical success.” However, any given patient may or may not consider a 30 percent reduction in symptoms a successful outcome. Barry et al.¹⁸ have shown that patients who were satisfied with treatment had a wide range of reductions in symptom scores.

Our meta-analysis shows that in placebo-controlled clinical trials, terazosin treatment reduced symptom scores by a mean of about 39 percent and that placebo treatment (which we used to estimate the effects of the “watchful waiting” strategy) reduced scores by about 19 percent. The estimate for terazosin treatment is similar to estimates of alpha-blocker effects reported in the Agency for Health Care Policy and Research guidelines for BPH.¹⁹ Specific estimates for terazosin treatment may vary because of differences in meta-analytic methods and because results from more recently published studies were included in our analysis.

To allow patients to compare the effects of terazosin treatment with expectant observation (sometimes called watchful waiting), we estimated the average reduction in symptoms reported for the placebo arm of clinical trials. Because of the placebo effect commonly observed in clinical trials, this method may actually overestimate the effect of watchful waiting in clinical practice. However, our mean estimate of effect size is similar to that reported for watchful waiting in other observational studies.²⁰

By estimating the percentage drop in symptom score as well as the variation inherent in the distribution of that drop, it is simple to calculate the probability of moving from one level of symptoms to another. If the gap between a patient's current level and desired level of symptoms is small, terazosin treatment offers a good chance of producing results consistent with the patient's treatment goals. For patients with high symptom levels who want near-complete relief from symptoms, terazosin treatment is not likely be effective. A patient with such treatment goals should be informed of this low chance prior to starting treatment.

To study patients' preferences for the level of desired relief and to demonstrate the feasibility of our model in automated patient decision support systems, we developed an interactive Internet site designed for direct and unsupervised patient use. This site, consistent with our model, measured a patient's current level of symptoms and maximum tolerable level of symptoms. Results suggested that patients had symptoms that are similar to those reported for patients in clinical trials (mean AUA score, 18.2 points). The approach of having patients use the AUA scale to report their current symptoms and their maximum tolerable level of symptoms works for the majority of patients. The average minimum acceptable decrease in symptoms was 10 AUA points—a value consistent with the average reduction in symptoms reported by patients who were “satisfied” with the outcome of treatment.¹⁸ The distribution of desired levels of relief was large. However, patients, on average, had only about a 40 percent chance of achieving their desired level of reduction in symptoms.

The majority of patients using the site found the information on the effectiveness of terazosin treatment helpful, and useful for guiding their own medical decisions. More than 70 percent believed that physicians should be required to present similar information to patients when prescribing medications. These results demonstrate the feasibility and relevance of this approach to shared decision making.

Limitations

One limitation of our approach is the relatively strong set of assumptions required to calculate, from metaanalytic models, the probability of response in an individual patient. This approach assumes that 1) the percentage change in a symptom score is approximately normally distributed across all levels of symptoms and 2) the change in symptom score (as well as the variance of that change) is proportional to the baseline symptom score. Although the literature provides substantial evidence that these assumptions are appropriate for calculating the probable effects of terazosin treatment on BPH, they may not hold true in meta-analyses of other treatments and should be validated. On the other hand, assumptions about normality are inherent in all meta-analytic approaches to combining information from different sources.

A second limitation of this approach is that the AUA score weights each type of symptom equally. Some patients may place greater importance on certain types of symptoms than others (for example, the frequency of nocturia may be more important than hesitancy or other symptoms). This could lead to over-estimation of the probability of achieving the desired level of relief.

We evaluated our method using subjects recruited over the Internet. Not all users of the site were BPH patients. To increase the validity of the study, we restricted our analyses to patients who identified themselves as having BPH. There were no incentives for other persons to misrepresent themselves as patients. Free-text comments confirmed that many subjects were patients by providing details of their clinical history. Consistent with the demographics of Internet use, patients using our site are likely to be better educated than the population as a whole. However, it is clear that patients looking for information on the Internet find this sort of information useful and that access to this type of information should be facilitated. The primary outcome measures of the study were the percentage of patients who found the advice useful and the percentage who believe the information in the decision support system should be offered to all patients. These outcome measures were appropriate for a formative study. Additional studies are needed to assess the degree to which patients understand the concepts presented in the displays, the effect of the decision support on the patient's decision, and its effect on doctor-patient communication and patient satisfaction with the decision.

Conclusions

One potential way for patients to share in clinical decision making is to allow them to set their own treatment goals and judge for themselves whether treatments offer sufficient prospects of success. The probability of individual patients' achieving their therapeutic goals can be calculated from each patient's current level and target level of symptoms and the meta-analytic summaries of the expected distribution of outcomes of treatment. Application of this approach to terazosin treatment of BPH reveals that relatively few patients have a realistic chance of achieving their treatment goals with this drug. Patients recruited over the Internet who used this decision support tool found this information useful, and most believe that physicians should be required to discuss the likelihood of a successful outcome of medical treatment prior to prescription of a medication.