The Risks of Multimedia Methods

Abstract

Objective: While the use of multimedia methods in medical education and decision support can facilitate learning, it also has certain hazards. One potential hazard is the inadvertent triggering of racial and gender bias by the appearance of actors or patients in presentations. The authors hypothesized that race and gender affect preferences. To explore this issue they studied the effects of actors' race and gender on preference ratings for health states that include symptoms of schizophrenia.

Design: A convenience sample of patients with schizophrenia, family members of patients, and health professionals was used. Participants were randomly assigned to rate two health states, one portrayed by either a man of mixed race (Hispanic-black) or a white man and the second portrayed by either a white woman or a white man.

Measurements: Visual analog scale (VAS) and standard gamble ratings of health state preferences for health states that include symptoms of mild and moderate schizophrenia.

Results: Studies of the effects of the race of the actor (n = 114) revealed that racial mismatch between the actor and the participant affected the participant's preferences for health states. Ratings were lower when racial groups differed (mean difference, 0.098 for visual analog scale ratings and 0.053 lower in standard gamble, P = 0.006 for interactions between the race of the subject and the actor). In studies of the effects of a female actress on ratings (n = 117), we found no evidence of a corresponding interaction between the gender of the actor and the study participant. Rather, an interaction between actor's gender and method of assessment was observed. Standard gamble ratings (difference between means, 0.151), but not visual analog scale ratings (difference, 0.005), were markedly higher when the state was portrayed by the actress (P = 0.003 for interactions between actor's gender and method of preference assessment). Differential effects on standard gamble ratings suggest than an actor's gender may influence the willingness of viewers to gamble to gain health benefits (or risk attitude).

Conclusions: Educators and researchers considering the use of multimedia methods for decision support need to be aware of the potential for the race and gender of patients or actors to influence preferences for health states and thus, potentially, medical decisions.

There is a growing movement toward the use of video materials and multimedia in patient education materials and patient decision support tools.¹ One common component of multimedia presentations is a testimonial, or soliloquy, by a patient describing what life is like with the health condition or how he or she came to make a particular medical decision. Many applications use this vehicle to inform patients about what life may be like with a disease or side effect of treatment.^2,3,4,5 The Shared Decision Making videodisk series extends this concept by having patients recount how they balanced the risks and benefits of different procedures to reach a decision about a treatment alternative they prefer.^6,7,8 Videotape and multimedia are powerful tools that improve learning and recall,^2,9,10 especially with subjects who have lower levels of education.¹¹ Compared with text-based interventions, videotape presentations also appear to improve motivation to act.

This study examines potential pitfalls of the use of video materials in decision support systems for patients with mental illnesses. A critical decision for many patients with mental illness is whether to accept treatment with antipsychotic drugs. Antipsychotic drugs pose significant risks of both reversible and life-long side effects. There may be significant differences between patients and health professionals in beliefs about the importance of these side effects.^12,13 Therefore, participation of patients in therapeutic decision making, or of an appropriate proxy such as a family member,¹³ is a matter of utmost importance. Furthermore, informed consent for the use of antipsychotic drugs is poorly documented,^14,15 suggesting that there may less than full participation of patients in treatment decisions.¹⁶ It may not always be possible for patients to assent to therapy, because of the severity of their mental illness.¹⁷ However, one approach to improve patient participation in decision making might be to use multimedia or video materials to demonstrate potential adverse effects of these medications as part of the process of obtaining assent. Some researchers have expressed concerns that education about drug side effects might lead to noncompliance; however, there is no evidence to suggest that this is true.¹⁷ Because patients with mental illnesses often have cognitive impairments that impair concentration, the use of multimedia methods may be particularly important in educational materials that are oriented toward increasing patient participation in this critical decision.

Risks may accompany the benefits of the use of multimedia methods in decision support tools in any health domain. Visual and auditory aspects of multimedia can result in the inclusion of unintentional stimuli that may affect the ultimate action taken by those who use a multimedia decision support tool. Unintentional stimuli in multimedia may take many forms. Among the most important influences on decision making is whether the actor presenting information appears to be of the same cultural background and gender as the viewer. Other unintentional stimuli might include the attractiveness of the actor^18,19 and the apparent socioeconomic status of the actor.^20,21

An extensive literature describes the effects of race on the perception of acts, persons, and judgment.^22,23,24 In the health education context, previous work suggests that a message is perceived more favorably if the narrator has the same ethnic background as the viewer.²⁵ Ethnic mismatch also reduces apparent response rates to messages.³ Similarly, materials seem to be more effective if the actor or narrator has the same gender as the viewer.^26,27

The effect of race and gender on simulated physician decision making has also been studied using video methods with actors portraying patients.^28,29 Work by Schulman et al.²⁹ suggests an interaction between race and gender in cardiologists' recommendations for diagnostic tests. Female African American patient-actors were referred for cardiac catheterization slightly less frequently (88%) than Caucasian male, Caucasian female, and African American male patient-actors (94%).^30,31 Other work employing patient-actors has identified complex effects of race, gender, age, and socioeconomic status on diagnostic and therapeutic decision making.^20,28,32

This study examines attempts to understand the origins of effects of race and gender in videotape and multimedia through the study of the effects on utilities for health states (sometimes referred to as health values). Utilities, in this context, are the subjective weights in decision making assigned to each potential health outcome. The purpose of measurement of utilities is to isolate the role that values play in medical decisions (as opposed to the effects of perceived probabilities of outcomes²⁶ and other factors). Do people value states less highly when there is mismatch between their race or gender and an actor's race or gender? Or do the effects of race and gender manifest themselves in other ways?

▶ shows a model of how racial and gender stimuli in multimedia presentations might affect the decision-making process by altering preferences. Unintentional stimuli influence subjects' perceptions of a health condition under study as well as how highly, in decision making, they value the prospect of life with that condition. Perceptions are determined largely by facts, whereas the interaction between facts and a subject's cultural beliefs about what is desirable governs the subject's values. These influences may not be conscious and may not influence perceptions of facts, but rather how we weigh facts and exercise judgment in decision making.

Figure 1.

Model for effects of video depictions of medical decisions.

Widely used approaches for measurements of utilities include the standard gamble, time tradeoff, and visual analog scale.³³ Each method captures different aspects of preferences.³⁴ Analog scale methods capture primarily beliefs about the severity of a health condition. Time tradeoff and standard gamble approaches measure preferences for outcomes. By this we mean that when subjects use either of the two methods, they explicitly judge which of two options they prefer. In the gamble, they determine whether they prfer life with the burdens posed by a health condition or assumption of a risk of death to live free of those burdens. Therefore, the gamble metric captures both preferences for quality of life and the willingness to assume risk to gain health benefits (risk attitude). In the time tradeoff, they judge whether they prefer life with the burdens of the disease or a reduction in life expectancy. Accordingly, this measure captures both preferences for quality of life and longevity.³⁴

Utility assessment is typically done in the measurement of preferences for hypothetic conditions or health states.³³ Descriptions of these states are often text based. However, as has been shown in the case of patient education literature, description of states using multimedia methods results in higher recall and better recognition. Furthermore, subjects who use multimedia tools may be more certain about the preferences that they express.³⁵

Some evidence suggests that race and gender influence measurements of utilities. A study by Rathore et al.³⁶ indentified differences in medical students' health values for states with coronary disease associated with the race and gender of the actor describing the state.³⁶ Using the same patient materials used in the study by Schulman et al.,²⁹ Rathore et al. found that medical students valued cases portrayed by a black female actor less highly than those portrayed by a white male actor. However, little other work on this issue has been performed.

To test the hypothesis that the “unintentional stimulus” of the race and gender of “patients” in testimonials affects how subjects value a health state, we created multimedia descriptions of health states in schizophrenia and videotaped the performances of actors of different races and genders. We presented these descriptions to a panel of patients, family members of patients, and health care providers. We measured their preferences for these states using standard gamble and analog scale methods and then tested for effects of an actor's race and gender on these measurements.

Methods

This paper is the second in a series of papers examining the results of a multimedia computer study of preferences for outcomes in schizophrenia. The first paper¹³ describes differences in preferences between patients, family members and healthcare providers. In this paper, we present the results of a planned substudy in which we examined the effects of the race and gender of actors presenting health state information on preference ratings in a random sample of study participants.

The main study was carried out between May and Oct 1997 at four sites—the Corrigan State Mental Health Center, Fall River, Massachusetts; Cleveland Department of Veterans Affairs Medical Center, Cleveland, Ohio; the University of Missuori, Kansas City, Missouri; and the Affiliated Research Institute, San Diego, California. A convenience sample of 338 subjects was recruited to participate in a multimedia computer survey. All subjects were compensated for their participation. The sample included 99 health care professionals, 148 patients with schizophrenia, and 91 family members of patients with schizophrenia. Health care professionals were eligible for the study if they were registered nurses, psychologists, social workers with at least master's level training, doctors of pharmacy, or physicians. To be eligible as a family member, a potential subject had to be either the spouse or a first-degree relative of a person with schizophrenia. In addition, family members could not have a diagnosis of schizophrenia themselves. To be eligible as a patient, participants had to have a diagnosis of schizophrenia and be receiving or have previously received a neuroleptic agent. Participants could be, but were not necessarily, related one to another.

Study participants provided ratings of two health states with two levels—mild and moderate—of symptoms of schizophrenia. Three variants of mild conditions and three variants of moderate conditions were evaluated. For one of the three mild variants and one of the three moderate variants, we prepared matched videotape descriptions to test effects of race and gender. For the selected mild state, we developed videotapes with portrayals of the state by both a white male and an ethnic minority male actor. For the selected moderate state, we developed videotapes with portrayals of the state by a white male and a white female actor. We used these dual descriptions to test for effects of race and gender of actors on preferences.

Each substudy was powered to detect about a 0.1 difference in utilities across the race or gender of actors. Given a standard deviation in preference measurements of 0.2,³⁷ a sample with 50 participants in each group (e.g., total study size of 100) would have a power of approximately 0.8 (two-sample t test) to detect differences of this magnitude.

Of the 338 subjects in the main study, 114 were randomly assigned to participate in a substudy on the effect of race on ratings. In this group were 48 patients, 33 health care providers, and 33 family members. Approximately 70 percent of subjects were white, 54 percent were female, and 65 percent had some college education. Ages ranged from 19 to more than 71 years, with a median age category of 41 to 50 years. Randomization was successful, and there were no statistically significant differences in the demographics (age, educational level, and subject type) of groups randomized to the white actor and the minority actor. One hundred seventeen subjects participated in the experiment on the effects of actor's sex on preferences for moderate schizophrenia. In this group were 52 patients, 33 health care professionals, and 32 family members. Randomization was again successful, with no statistically significant differences in the demographic features of participants randomized to the male and the female actor.

Health State Descriptions

Effects of schizophrenia on quality of life were described in brief video presentations. The text of the video presentations was developed from analyses of patterns of health impairments in schizophrenia observed in a large randomized trial.³⁸ Details of the process are found in the report of the pilot study of the instrument. The effects of schizophrenia were detailed using a model with four aspects of mental health (positive symptoms of schizophrenia, negative symptoms, hostility, and anxiety/depression.) Material on symptoms of schizophrenia was supplemented with descriptions of patients' living situations and their functioning in a community. The texts of scripts were reviewed by several disease experts and refined. Finally, actors were hired to perform each script.

Depictions were brief, lasting between two and four minutes, depending on the number of symptoms a patient had. Each actor, in a soliloquy, described the effects of schizophrenia on his or her life. Scripts were followed word for word, and no deviation from the scripts was allowed. Actors were trained under the supervision of the psychiatrist who wrote the scripts.

When performing, actors wore identical clothing and were videotaped in a studio against a dark background. The upper portion of ▶ shows the white actor on the left and the minority actor on the right. The minority actor was chosen specifically to be of ambiguous ethnic background, possibly consistent with either Hispanic or African American heritage. The lower portion of ▶ shows the male and female actors. Both of these actors were white.

Figure 2.

Racial stimuli (top row) and gender stimuli (bottom row) for preference ratings.

Video materials were converted to QuickTime format and incorporated into the instrument. Subjects viewed each presentation in its entirety before assigning a value rating for the health state. The text of materials and video presentations can be viewed on the World Wide Web at http://preferences.ucsd.edu/schizophrenia/healthstates.html . After completion, we tested video materials in a pilot study to ensure that subjects could understand their content and use the materials to provide preference ratings for states.¹²

Survey Design

Preferences were measured using two methods—a visual analog scale and a standard gamble technique. Two different measures were used because, as described above, these methods capture different aspects of preferences. The primary difference between the two methods is the inclusion of risk attitude in gamble measurements. Previous work suggests that analog scale and gamble measurements have a fair degree of independence when analog scale measurements are performed first in a sequence employing the two methods.³⁹ This independence can be used to improve the precision of estimates of preferences.⁴⁰

Subjects viewed a picture of the actor during the rating process and could review portions of the video as they wished. After using a visual analog scale to rate the desirability of a health state, subjects rated the same state with the standard gamble. In measurement tasks, subjects were instructed to imagine that they had the condition themselves and to rate their imagined quality of life. Analog scale measurements were performed using an animated visual scale anchored on perfect health and death with 20 intervals. The scale was interactive and implemented in JavaScript. By clicking along its length, subjects received feedback on their rating. Standard gamble measurements were performed using a matching approach. Subjects determined the maximum risk of death they would accept not to have to live the remainder of their lives with the health condition. A graphical display depicted the probability of death using a random pattern of shaded face along with a numeric display of the chances of death and of healthy life. The utility of the state was estimated to be 1 minus the maximum acceptable risk of death not to have to live with the depicted condition. Approaches used for measurement of preferences had been previously validated.⁴⁰ The study was conducted using an instrument developed in HTML. The instrument was designed for administration over the World Wide Web. After obtaining consent for participation from subjects, study coordinators at each of the four “nationwide” sites used an online HTML form to enroll participants. At enrollment, participants were automatically randomized and assigned a password that also identified their session to the study's Web server. The survey used the combination of Web pages dynamically generated by the server and multimedia files cached on a local hard disk drive to implement the survey. A common gateway interface (CGI) program connected the Web server to a database that recorded subjects' responses. The database used rules to monitor subjects' progress through the survey. As they progressed through the survey, subjects were informed of inconsistencies in their preference ratings and received computer assistance with resolution of these inconsistencies using dynamic HTML pages that were generated on the basis of analyses of contents of the database.

All subjects were interviewed at the facilities listed above. They operated the computer by themselves when this was possible. Surveys were administered under the supervision of a trained research associate who assisted participants only when necessary and followed a rehearsed script. Research associates logged each episode of assistance they provided to subjects.

Effects of race and gender were studied initially using ANOVA for multiple observations, simultaneously examining visual analog scale and standard gamble ratings for each state. In this analysis, analog scale and standard gamble measurements were treated as independent assessments of preferences. We tested for main effects of race (or gender) of the subject, the actor, and the assessment method. We also tested for effects from the three potential two-way interactions between these factors and from the three-way interaction. To examine interactions between an actor's race and the participant's race, subjects were classified as minority group members if they reported their race as African American or nonwhite Hispanic. Because initial analyses of data from the study¹³ showed significant differences between subject groups in preferences, we performed sensitivity analyses to examine the effects of an actor's race and gender within each subject group. To study the potential sensitivity of findings to inclusion of any group, we repeated analysis of study data on a group-by-group basis, looking for trends in analyses identified in the complete data set.

Results

There were no statistically significant associations between the age, educational level, or marital status of subjects and their preferences. The results of ANOVA analyses for effects of race on preference ratings are reported in ▶. Analyses showed, as expected, a significant difference between rating methods in mean preference ratings for states. There were no significant main or fixed effects from the actor's race or subject's race on preferences. However, there was a significant interaction between the race of the actor and the race of the participant (P = 0.0063). As shown in ▶, both white and minority group members considered states to be less preferable when portrayed by actors of a different racial group. On average, standard gamble utilities were 0.053 units higher and visual analog scale measurements were 0.098 units higher when the race of the participant matched that of the actor. Evidence of the interaction between the race of the subject and the race of the actor were present in both minority group member subjects and white subjects in sensitivity analyses.

Table 1.

ANOVA Results for Effects of Race on Subjects' Standard Gamble and Visual Analog Scale Preference Ratings

	DF	Sum of Squares	Mean Square	F Value	P Value
Actor's race	1	487.896	487.896	0.817	0.3681
Participant's race	1	1186.997	1186.997	1.987	0.1615
Actor's race · Participant's race	1	4634.568	4634.568	7.757	0.0063
Participant	110	65721.523	597.468
Assessment method	1	13577.811	13577.811	45.632	<0.0001
Assessment method · Actor's race	1	977.936	977.936	3.287	0.0726
Assessment method · Participant's race	1	608.148	608.148	2.044	0.1557
Assessment method · Actor's race · Participant's race	1	858.148	858.148	2.884	0.0923
Assessment method · Participant	110	32730.179	297.547	—	—

Figure 3.

Effects of match and mismatch between race of the actor and race of the participant on standard gamble and analog scale preference ratings. Darker shading indicates matched race; lighter shading, mismatched race.

▶ shows the results of ANOVA for effects of the actor's sex on preference ratings. As expected, there were, again, significant differences in the values obtained by different measurement methods, with standard gamble measurements having a mean greater than that of analog scale measurements. Results suggested both a weak effect of actor gender (P = 0.031) and a strong effect of the interaction between rating method and actor gender (P = 0.0033) on preference ratings. ▶ shows a comparison of effects on standard gamble and analog scale ratings. Standard gamble ratings for states were markedly higher when states were portrayed by the female actor. The difference in mean standard gamble ratings was 0.151 units. Analog scale ratings were largely unaffected (mean difference, 0.007 units). In sensitivity analyses, similar effects of an actor's gender were seen among health care providers and other subject groups.

Table 2.

ANOVA Results for Effects of Gender on Subjects' Standard Gamble and Visual Analog Scale Preference Ratings

	DF	Sum of Squares	Mean Square	F Value	P Value
Subject's gender	1	319.57	319.57	0.44	0.5070
Actor's gender	1	3395.28	3395.28	4.71	0.0322
Subject's gender · Actor's gender	1	63.41	63.41	0.09	0.7674
Subject	111	80056.85	721.23	—	—
Assessment method	1	15677.95	15677.95	50.81	<0.0001
Assessment method · Subject's gender	1	32.63	32.63	0.11	0.7456
Assessment method · Actor's gender	1	2776.47	2776.47	9.00	0.0033
Assessment method · Subject's gender · Actor's gender	1	9.49	9.49	0.03	0.8611
Assessment method · Subject	111	34248.30	308.54	—	—

Figure 4.

Differential of actor's gender on standard gamble and analog scale preference ratings. Darker shading indicates male actor; lighter shading, female actor.

Discussion

In two randomized trials conducted in overlapping populations, we compared the effects of the portrayal of the health effects of schizophrenia by a white actor and by a minority-group actor and then compared the effects of portrayal of a health state by a male actor and by a female actor. Other aspects of the actors' appearance and speech were highly controlled. Effects were studied using two well-established methods for measuring preferences for health states—a global visual analog scale rating and the standard reference gamble. All subjects rated what they believed their own health would be if they had the condition. Thus, the race or gender of the actor portraying a condition should have been irrelevant to the judgment. However, results showed moderate effects (between one third and two thirds of the standard deviation in preference measurements), presumably due to race and gender, on preferences for the states. To understand the significance of effects of race, the average difference between states with mild and moderate symptoms of schizophrenia (described in first paper in this series¹³) was 0.047 for visual analog scale ratings and 0.050 for standard gamble ratings. Thus, the effects of the race of the actor were equal to or greater in magnitude than differences in utility between mild and moderate symptom levels.

The effect of the race of the actor depended on the race of the study participant. Participants viewing health states portrayed by an actor from a different racial group valued health states less highly than subjects viewing states depicted by actors from the same racial group. Effects were seen in both white and minority group member subjects and were at least as large as differences in utility between states with mild and moderate symptoms.

We also observed effects from use of a female actor to describe states on preferences. Even though there was statistical evidence of a “main” or fixed effect of gender on ratings, examination separately of analog scale and standard gamble results suggested that effects were seen primarily on standard gamble measurements. The absence of effects on analog scale ratings and the large effects on the gamble ratings are not contradictory results. Rather, the results suggest an effect on risk attitude—an attribute measured by the gamble but not the analog scale. The result is consistent with participants' viewing states as being of similar severity, but being much less willing to take risks to gain benefits (e.g., becoming risk adverse) when the state was portrayed by the female actress. Differences of the sex of the actor on standard gamble utilities were three times greater than the average difference in utility between mild and moderate health states.

The results confirm the model presented in ▶. While ethnic and gender diversity among actors or patients used in video presentations and decision support software is socially desirable, it may have unintended effects. The effects of these stimuli are relatively large—at least as large as effects of altering the severity of illness depicted in this experiment. Racial mismatch between the actor and the patient viewing the presentation may ultimately influence patients' decisions. Direct evidence of the effects of actors' race and gender on patients' decisions will require further work. However, since effects of actors' race and gender on physicians' decision making have been observed,^{20,28,29,32,41,42} it is likely that studies of patients' decision making will show similar results.

The results suggest that developers of educational videotapes and software for patients may be faced with a difficult and potentially distasteful choice. One option to reduce potential effects from gender or racial bias might be to use actors (or patients) of one consistent race and sex and abandon aspirations of cultural diversity in video and multimedia decision support tools. While use of actors of a consistent race and sex might bias preferences, the effects would be similar and the net impact on patient decision making would be minimal. The net result, however, is likely to be that the tools will be less well received by whatever racial groups are not represented in the production.²⁵ The other option for developers is to match the race of actors in presentations to the race of patients viewing the presentations. This approach might require producing several versions of video materials or software tools. Producing “separate but equal” materials might be prohibitively expensive and politically unacceptable in many settings.

Limitations

The primary limitation of this study is that the effects of race and gender were studied using a single actor of each race and gender. Thus, results could represent the effect of the actor's performance rather than an effect of race or gender. Even if effects were due to the “personality features” of actors rather than to their race or gender, this would still suggest that great care is necessary in the production of video materials to control for such influences.

This study examined preferences of patients, family members, and health professionals for health states caused by schizophrenia. While the results confirm and expand the findings of the study of Rathore et al.³⁶ (which demonstrated effects of an actor's race and gender on medical students' preferences for health states in coronary disease), the reader should not necessarily assume that these effects extend to other medical problem areas. The use of patients with schizophrenia may affect the ability to generalize results of this study. However, findings were observed in all subgroups in the study, suggesting that effects are not unique to these patients.

Conclusions

Producers of decision support software and educational video materials should exercise care in the choice of actors to portray model patients. The race and sex of an actor may alter how favorably viewers perceive one health state relative to another or may influence their willingness to accept risk to gain benefit. Haphazardly varying the race and gender of patients in multimedia materials for the admirable purpose of presenting a multicultural perspective may inadvertently bias viewers' subsequent decision making.