Measuring Exposure to Direct-to-Consumer Advertising—A Validation Study in the Context of Cancer-Related Treatment Advertising

Abstract

This research examines two recurrent conceptual issues of measuring media exposure in survey research in the context of cancer-related direct-to-consumer advertising (CR-DTCA)—the level of content specificity of survey items and the benefits of providing exemplars to aid recall. We evaluated three candidate measures of cancer patients’ self-reported exposure to CR-DTCA; these measures varied in content specificity and provision of ad exemplars. Using data from two distinct population-based surveys, we assessed the performance of each measure based on several reliability and validity criteria. Results across both surveys indicate that all three measures performed equally well in terms of internal consistency, convergent, nomological, and discriminant validity with a few minor differences between these measures. Increased content specificity or inclusion of ad exemplars did not result in better performance of the exposure measures. Participants were able to extrapolate from ad exemplars to report their exposure to broad categories of CR-DTCA. The briefest of the three measures posed the lowest level of survey costs among the three measures and was deployed successfully for both mailed and internet-based survey administration. We discussed future directions for application of these findings in DTCA research for other illness and for media exposure research more generally.

INTRODUCTION

A central task in mass communication research is to measure media exposure of specific content accurately (e.g. violent media, health information, or political news). This ability to obtain reliable and valid exposure measures is crucial to studying media effects because it directly impacts the validity of an investigator’s inferences about the presence or absence of media effects on an outcome of interest (e.g., aggression, health behaviors, or political socialization). Accordingly, a special issue in this journal was devoted to interrogate conceptual challenges of exposure measurement. In that issue, the editors remarked, “(d)espite the centrality of the exposure variable to communication research, systematic research about measuring exposure is remarkably thin…little is actually known about how best to measure exposure in order to assess the effects of that exposure on outcome variables” (Fishbein & Hornik, 2008). They further called for an expansion in the literature to address different aspects of methodological concerns in assessing media exposure.

The purpose of this present research is twofold—to contribute to the systematic research on exposure measurement in mass communication research by examining two recurring conceptual issues associated with measuring media exposure in survey research; and to offer as an example, a measurement validation approach that encompasses complementary study populations, research designs, modes of data collection, and a battery of validation criteria. The data we present here pertains to one specific media content—cancer-related direct-to-consumer advertising or CR-DTCA—but we hope to draw key lessons from this approach that would advance media exposure research in other domains beyond this particular context.

The studies reported here address two parallel conceptual issues: all self-reported measures ask respondents to generalize from the content in front of them to a larger category of exposure. If we ask: “how often have you seen or heard advertisements concerning each of the following? … Hospitals or doctors offering services for cancer?” we are asking respondents implicitly to consider, among other things, various sources of exposure to such ads (TV, radio, print, internet) and various types of specific services (radiation therapy, chemotherapy, etc.) that might have been advertised. The first conceptual issue we address then is whether that process of mental summarizing is likely to go better if questions include more of that detail (e.g. “Sometimes hospitals or doctors advertise their services (radiation therapy, chemotherapy, or comprehensive treatment) for treating patients with cancer. These advertisements may appear in the mass media (e.g., television, radio, newspapers, magazines, billboards, or the internet). Since your diagnosis, how often have you seen or heard advertisements concerning hospitals or doctors offering services for cancer?”). We recognize that the additional detail likely comes at the cost of complexity in language, and more time per question. The second issue is whether the abstractness of the simple question undermines quality of responses compared to questions which provide concrete examples, showing ‘representative’ video or print ads. Examples make it easy for respondents to grasp what the question is about, but they also may risk drawing attention to and producing estimates of recall to those specific ads rather than the category of ads which is of interest, and they also add time to read each question. The essential conceptual issues for this study are whether adding more detail and adding examples improves the quality of exposure estimates enough to justify their likely additional demands. We look at one example here, but the issue applies quite broadly to the measurement of exposure.

We first begin with an overview of approaches to measure media exposure, provide a background to CR-DTCA, and describe the rationale and key conceptual questions in this study. Next, we outline the validation criteria and methods utilized here, followed by findings from two population-based surveys. Finally, we discuss the implications of the findings and recommendations for future research in exposure measurement.

MEASURING MEDIA EXPOSURE

There are various approaches to measurement of exposure including experimental manipulation, external estimates of exposure (e.g. gross rating points), aggregated self-reports at the level of geographic or time period units, as well as individual self-reports (Fishbein & Hornik, 2008). Each of these approaches presents researchers with certain strengths and limitations in terms of internal and external validity. Of relevance here, effect estimates based on individual self-reports may suffer from lack of validity of those reports (e.g., declining accuracy in older populations), could be confounded by the influence of third variables (e.g., underlying interest or selective exposure), and the causal direction is often unclear (Slater, 2004; Southwell & Langteau, 2008). Nevertheless, this approach will continue to be the predominant mode of exposure measurement in communication research in the foreseeable future because of the relative ease of assessing media exposure directly among large population samples (Hayes, 2005). Quality of self-report measurement is the focus of the current study.

THE CURRENT STUDY—MEASURING CR-DTCA EXPOSURE

Direct-to-consumer advertising (DTCA) for prescription medications and healthcare services is controversial in the United States and subject to much debate (Bonaccorso & Sturchio, 2002; Fox & Ward, 2008; Frosch, Grande, Tarn, & Kravitz, 2010; Hoen, 1998; Hollon, 1999, 2005; Holmer, 1999, 2002; Rosenthal, 2010). Cancer-related DTCA (or CR-DTCA), a form of “subspecialty DTCA” which is targeted at patients with cancer, is of greater concern because of the limited options of highly effective interventions and potentially higher risks associated with specialized treatments (Abel, Lee, & Weeks, 2007). To better understand the extent of cancer patients’ perceptions of CR-DTCA—which we define as “promotional efforts by a pharmaceutical company, healthcare provider, or medical facility to present information about medications, medical devices, or medical services for patients diagnosed with cancer in the lay media environment” (adapted from Wilkes, Bell, & Kravitz, 2000)—and the potential impact of exposure to this advertising on various treatment-related behaviors or outcomes, it is essential to be able to measure patients’ exposure to CR-DTCA adequately.

Drawing from earlier validation research of incidental or scanned media exposure to cancer-related health information (Kelly et al., 2010; Kelly, Niederdeppe, & Hornik, 2009; Romantan, Hornik, Price, Cappella, & Viswanath, 2008; Shim, Kelly, & Hornik, 2006), we are interested in assessing exposure to cancer treatment ads that has been encoded in respondents’ memory and not merely exposure due to being present in the audience. Encoded exposure is described as the minimal trace in individuals’ memory of having seen or heard specific media content which would be accessible for later recall. This is the minimum that would have to occur to reasonably expect media exposure to produce content-related effects on various outcomes (Southwell, Barmada, Hornik, & Maklan, 2002). We hypothesize that cancer patients see ads that are about their cancer treatment as belonging to a common category and as such encode them in memory as linked. When asked about any one specific type of treatment ad, they are expected to call on the encoded and linked memories about this common category (i.e., cancer treatment ads) in responding to each question. Because an individual’s encoded exposure is not directly observable, we assume this would manifest as self-reports of encountering various CR-DTCA. In other words, we expect patients’ encoded memory about coming across all ads concerning their cancer treatment to influence reports of how often they saw or heard CR-DTCA promoting all aspects of their treatments. For instance, reporting frequent exposure to ads from cancer centers is presumed to be correlated with reporting greater exposure to ads about medications because these reflect an individual’s memory of encountering ads concerning one’s cancer treatment. We therefore hypothesize that these self-report measures are “effect indicators” (Bollen, 1989) and constitute a scale that represents the underlying latent construct of encoded exposure to cancer treatment ads (DeVellis, 2011).

RATIONALE FOR STUDY AND CONCEPTUAL ISSUES OF MEASURING CR-DTCA EXPOSURE

The impetus for this present validation research stemmed from a practical need to assess the reliability and validity of existing survey items of CR-DTCA exposure that we hypothesized as predictors of cancer patients’ communication behaviors in separate analyses not presented here. These exposure items were part of a survey among a sample of cancer patients in Pennsylvania (to be described shortly). Participants were asked “Since your diagnosis, how often have you seen or heard advertisements concerning each of the following?” for three categories of ads including “treatment alternatives for your cancer”, “dealing with side effects of treatment”, and “hospitals or doctors offering services for cancer”. Responses were measured along a 5-point scale (1 = Never to 5 = Almost every day). Lower scores represented lower frequencies of exposure to these types of ads (see Appendix A). The items were designed and refined following literature review, extended patient interviews, and expert consultation among oncologists.

While these three survey items were conceptually novel, we argue that they would be preferable as measures of CR-DTCA exposure compared with prior exposure measures described in the literature for the following reasons. First, the individual items focused on three broad categories of cancer treatment ads that were more comprehensive than measures in previous literature. Specifically, prior studies about DTCA exposure focused solely on measuring exposure to prescription drug ads (Abel, Burstein, Hevelone, & Weeks, 2009; Aikin, Swasy, & Braman, 2004; Bell, Kravitz, & Wilkes, 1999; Deshpande, Menon, Perri III, & Zinkhan, 2004; Frosch, May, Tietbohl, & Pagán, 2011; Martinez & Lewis, 2009; Murray, Lo, Pollack, Donelan, & Lee, 2004; Sumpradit, Fors, & McCormick, 2002; Weissman et al., 2003; Wilkes et al., 2000). However, reports show that hospital advertising spending is substantial and growing steadily from $493 million to $810 million between 2001 to 2005 (American Medical Association, 2006; Newman, 2011). Content analyses also indicate that advanced procedures (e.g., robotic surgery) and hospital or provider cancer treatment services are frequently promoted in hospital websites and newspaper ads (Jin et al., 2011; Larson, Schwartz, Woloshin, & Welch, 2005). The exposure measures described here would better match these various types of cancer treatment advertising beyond prescription drugs alone.

Second, the survey items in our research asked respondents about the frequency of encountering cancer treatment ads, which contrasted with measures from other studies that asked about awareness (i.e., whether respondents had seen or heard prescription drug advertisements (yes/no)) (Aikin et al., 2004). An important limitation of binary responses is that items have minimal variability (DeVellis, 2011). Conversely, the advantages for measuring frequency are greater variability and also the ability to assess if there is a dose-response relationship between CR-DTCA exposure and outcomes. Establishing such a relationship would provide additional evidence for inferring effects of CR-DTCA on various outcomes.

However, we recognize that these CR-DTCA measures pose a few conceptual limitations. First, the survey items in our study are brief in describing the content of treatment ads; they do not elaborate with examples or define the individual categories of ads for the respondents. The concern with this brevity is the inherent assumption that respondents could identify the named categories and report their exposure to ads according to those categories accurately. For instance, the survey item on the category of “hospitals of doctors offering services for cancer” could conceivably trigger recalling different types of ads in different respondents. In comparison, Abel and colleagues (2009) provided a list of 24 cancer-related brand name medications that appeared in print advertisements to prompt patients’ responses about their awareness of ads for each of these medications. Furthermore, the three survey items in our study may be limited by the absence of prompts to recall ad exposure from various sources, which may lead to under-reporting exposures. This differed from measures used by studies to assess respondents’ exposure to prescription drug ads across a variety of sources including television, radio, newspapers or magazines, or on the internet (Aikin et al., 2004; Martinez & Lewis, 2009).

Related to these limitations, we are interested in examining two recurrent conceptual issues in operationalizing reliable and valid self-report measures of media exposure and aimed to address these in this research. The first conceptual issue deals with selecting the optimal content specificity or generality of self-reported survey items (DeVellis, 2011; Romantan et al., 2008). For instance, survey items could have no mention of ad content on one end of the spectrum of specificity (e.g., general television viewing) to being highly content-specific on the opposite end (e.g., recall an ad from Memorial Sloan Kettering Cancer Center on television). General measures with low specificity could be flawed in terms of content validity and are less convincing measures that individuals were actually exposed to the media content under study. However, high content-specific exposure measures could be confounded by other constructs beyond exposure (e.g., pre-existing knowledge) and are associated with higher survey cost (Romantan et al., 2008). Items that are too specific and share similar content other than the intended construct (e.g., items about exposure to ads of brand name drugs for breast cancer treatment) could risk pulling away these items as a set from the underlying latent construct and undermine the performance of the instrument (DeVellis, 2011).

The second conceptual issue has to do with the lack of consensus about whether measures of self-reported ad exposure should be prompted with ad exemplars. The use of print or video exemplars to elicit cued recall of ads is well documented in the literature (e.g., exposure to health campaign messages or exposure to tobacco and alcohol ads) (Southwell et al., 2002; Unger, Johnson, & Rohrbach, 1995; Unger, Schuster, Zogg, Dent, & Stacy, 2003). An obvious advantage is that exemplars help define certain types of ads unambiguously for respondents. And if it is exposure to those specific ads which is of interest it is easy to justify their inclusion. However, there are limits to the number of exemplars that can be shown in a survey that would represent the breadth of ads of a category of specific content. Additionally, there is a potential threat that respondents could be reporting exposure to the specific exemplar shown rather than extrapolating their exposure to a broader class of content that is of interest.

The above conceptual issues motivated a separate survey to assess if modified versions of the existing CR-DTCA survey items aimed at addressing these issues (content specificity and use of exemplars) would perform better than the existing brief version. Specifically, in one modified version, we designed survey items that involved longer verbal descriptions of each of the ad categories and listed media sources to prompt respondents about their recall of cancer treatment ads from these sources to increase content specificity of the items. For instance, respondents were asked “Sometimes hospitals or doctors advertise their services (radiation therapy, chemotherapy, or comprehensive treatment) for treating patients with cancer. These advertisements may appear in the mass media (e.g., television, radio, newspapers, magazines, billboards, or the internet). Since your diagnosis, how often have you seen or heard advertisements concerning hospitals or doctors offering services for cancer?” (Appendix A). We designed a second set of alternative survey items to assess the impact of including ad exemplars on reliability and validity of the measures and to assess the threat of whether respondents were reporting recall of exemplars or if they could extrapolate their exposure to a class of ads from exemplars. These modified items incorporate longer descriptions of each ad category, list various media sources, and provide two exemplars of ads that are representative of each category to illustrate what the ad categories mean for respondents (Appendices A and B).

VALIDATION APPROACH

The approach of this study was adapted from related validation frameworks for assessing the performance of alternative measures of exposure to cancer information (Romantan et al., 2008), scanned information exposure (Kelly et al., 2009), and contradictory health messages (Nagler & Hornik, 2012). In this study, we examined how well each of the three candidate measures of CR-DTCA exposure performed based on the following criteria that were most relevant:

1. Internal consistency (a form of reliability measure: how well multiple items used to measure a single concept are associated with one another) (Schutt, 2011)

2. Convergent validity (how well one measure of a concept is associated with different types of measures of the same concept) (Schutt, 2011)

3. Nomological validity (a form of construct validity: how well measures are associated with other variables that they should logically be related to) (Cronbach & Meehl, 1955)

4. Discriminant validity (a complementary approach to construct validity: the measure to be validated is related most strongly to its comparison measure and less so to the measures of other concepts) (Schutt, 2011)

5. Test-retest reliability (how well measures of a phenomenon at two points in time are correlated if the phenomenon has not changed) (Schutt, 2011)

6. Survey costs (length of measures and survey response time)

The operationalization of these criteria is explicated in Table 1. To assess the performance of the candidate measures of exposure to CR-DTCA among cancer patients, we relied on two data sources—the Pennsylvania Cancer Patient Survey and an online survey of cancer patients across the U.S. We describe the methods and findings from these two data sources separately, followed by an overall discussion.

TABLE 1.

Operationalization of Validation Criteria

Criteria	Definition
Internal consistency	Moderate to strong inter-item correlations (Pearson’s correlation r ≥ 0.35), Items load on a unidimensional construct, and Internal consistency based on reliability measures (Cronbach’s alpha≥0.70).
Convergent validity	Correlations between across candidate measures of CR-DTCA exposure Correlations between scales derived from candidate measures of CR-DTCA exposure
Nomological validity	Correlations between patients’ exposure to CR-DTCA with variables that would be expected to predict CR-DTCA exposure as well as those that would be affected by exposure to CR-DTCA: General media usage, Health media exposure, Scanning of treatment information, and Discussing with physicians about information from lay media sources (where CR-DTCA would be encountered).
Discriminant validity	Candidate measures of CR-DTCA exposure would be more strongly associated with one another than with the variables assessed for nomological validity.
Test-retest reliability	Correlations between repeated measures of exposure to CR-DTCA over time.
Survey costs	Length of the measures in terms of word count, Number of responses required, Response times for completing the entire survey.

STUDY 1—PENNSYLVANIA CANCER PATIENT SURVEY

METHOD

Sample

Between 2006 and 2008, we conducted annual surveys among a probability sample of patients who were diagnosed with breast, prostate, or colorectal cancers and were reported, as legally required, to the Pennsylvanian Cancer Registry (PCR) in 2005. The data collection and survey instrument development procedures are detailed elsewhere (Nagler et al., 2010). The University of Pennsylvania Institutional Review Board approved the study. In this present research, we focused on the surveys in 2006 and 2007 (Rounds 1 and 2) that collected data about cancer patients’ exposure to CR-DTCA. In Round 1, 2013 participants completed the survey (American Association for Public Opinion Research response rate 4 was 64%) (American Association for Public Opinion Research, 2006). Briefly, response rate 4 (RR4) takes into account the proportion of cases of unknown eligibility that were actually eligible and includes partial interviews as respondents. Among 1758 respondents who agreed to be re-contacted, 1293 (74%) completed the Round 2 survey.

Measures

CR-DTCA exposure (Measure I)

As described earlier, three items asked respondents about their frequency of seeing or hearing about advertisements concerning “treatment alternatives for your cancer”, “dealing with side effects of treatment”, and “hospitals or doctors offering services for cancer”, measured on a 5-point scale (1 = Never to 5 = Almost every day) (see Appendix A). The numerical scores assigned to responses to these three items were treated as interval and averaged to form the measure of CR-DTCA exposure (Measure I) (Round 1: M = 2.41, SD = 1.02; Round 2: M = 2.21, SD = 0.95).¹

Health media exposure

We expected participants’ exposure to health information from media sources to be positively associated with exposure to CR-DTCA. Participants reported how frequently they had found out about health information in the preceding 30 days from five media sources (i.e., newspapers or general magazines, medical magazines or newsletters, health segments on television news, television programs other than news, and the internet). Responses for these items ranged on a 4-point scale from ‘not at all’ to ‘two or more times per week’ and were summed into an index of health media exposure (Round 1: M = 10.43, SD = 3.59; Round 2: M = 10.48, SD = 3.47; Pearson’s r between Round 1 and 2=.598, p<.0005)².

Scanning about treatment information

We further expected that encountering cancer treatment information during routine use of media sources would be positively associated with CR-DTCA exposure. Respondents were asked “What information have you come across about your cancer from media sources (television, radio, newspapers, magazines, Internet) when you were not looking for it since your diagnosis?” and were able to select if they had come across information about “what treatments were the best for my cancer”, “which doctors of hospitals would be the best for me”, and “how to manage side effects of treatments”. These binary responses (yes or no) were summed into a 0–5 point index of treatment-related information scanning (Round 1: M = 1.05, SD = 1.03; Round 2: M = 0.56, SD = 0.86; Pearson’s r between Round 1 and 2=.268, p<.0005)².

Discussion with treating doctors

Prior research suggests that exposure to prescription drug DTCA prompted patients to discuss the advertised drug with their physicians (Aikin et al., 2004; Murray et al., 2004). We considered that CR-DTCA exposure would be associated with discussion with one’s physicians about cancer-related information. Four survey items asked if respondents had discussed information they had gotten from media sources (i.e., television or radio; books, brochures, or pamphlets; newspapers or magazines; and internet excluding personal emails) with their treating doctors since the cancer diagnosis. These are media sources through which CR-DTCA tend to be noticed by cancer patients (Abel et al., 2009). The items were summed into an index (range of 0–4) of discussion with treating doctors (Round 1: M = 0.78, SD = 1.07; Round 2: M = 0.59, SD = 1.01 Pearson’s r between Round 1 and 2=.341, p<.0005)².

Analyses

We conducted analyses for assessing the performance of Measure I using the Stata release 11 statistical package (StataCorp, 2009). To assess internal consistency, we performed correlations between Measure I items, computed the Cronbach’s alpha statistic, and conducted principal component analysis to test the measurement hypothesis that the set of items reflected a single common dimension. Nomological validity was assessed with correlations between the CR-DTCA exposure scale with health media exposure, scanning about treatment information, and discussion with physicians at Rounds 1 and 2. Test-retest reliability was assessed with the correlation between CR-DTCA exposures in Rounds 1 and 2.

RESULTS

At Round 1, half of the study sample was female (51%) and the average age was 63 years. Other participant characteristics are reported in Table 2. The distributions (mean and standard deviation) of individual Measure I items in both rounds and the resulting CR-DTCA exposure scales are summarized in Table 3.

TABLE 2.

Participant Characteristics

	Study 1 N=2013			Study 2 N=363
Characteristics	M	SD	%	M	SD	%
Age	66	12		53	16
Gender – Female			51			52
Education
Some high school and below			16			2
High school			41			23
Some college or two-year degree			22			36
4-year college degree or higher			22			39
Race/ ethnicity
White			83			89
Black			13			6
Other			4			6
Cancer type
Colon cancer			34			10
Breast cancer			34			21
Prostate cancer			32			16
Other			0			43a

Note.

^aExamples of other cancer diagnoses were skin cancers (12%), endometrial cancer (4%), lymphoma (3%), and leukemia (3%).

TABLE 3.

Distributions of Items and Scales of Candidate Measures of CR-DTCA Exposure

	Study 1				Study 2
	Measure I Round 1a N=2013		Measure I Round 2b N=1293		Measure I N=363		Measure II N=216		Measure III N=147
	M	SD	M	SD	M	SD	M	SD	M	SD
Individual Items
Treatment alternatives	2.22	1.23	2.04	1.15	2.61	1.51	2.64	1.44	2.74	1.47
Dealing with side- effects	2.22	1.30	1.92	1.10	2.45	1.43	2.60	1.37	2.48	1.41
Hospital or doctors offering services	2.87	1.34	2.75	1.33	3.24	1.36	3.26	1.37	3.27	1.31
Combined scale
CR-DTCA exposure	2.41	1.02	2.21	0.95	2.77	1.22	2.83	1.19	2.83	1.14

Note. Means are based on a five-point scale (1 = Never and 5 = Almost everyday).

^aThe number of cases with missing values for the individual items ranged from 519 to 597; these were predominantly due to 369 cases who were randomized to receive a shortened form of the questionnaire in Round 1 that excluded these CR-DTCA items. The remainder was due to item non-response or multiple responses to an item.

^bThe number of cases with missing values for the individual items ranged from 129 to 211 in Round 2; these were due to item non-response or multiple responses to an item.

Internal consistency

In this study population, Measure I items were moderately correlated with one another (inter-item Pearson’s r ranged from 0.420 to 0.477 in Round 1 and from 0.461 to 0.474 in Round 2) (Table 4). In addition, the three items demonstrated reasonable internal consistency (Cronbach’s alpha=0.724 in Round 1 and 0.742 in Round 2). Principal component analyses showed that the items loaded on a single component with an eigenvalue greater than one in both Rounds 1 and 2, accounting for 64% and 66% of the total variances respectively.

TABLE 4.

Inter-Item Pearson’s Correlations Between Measure I Items (Study 1)

	Round 1			Round 2
	1	2	3	1	2	3
1. Ads concerning treatment alternatives for cancer	-	0.420	0.420	-	0.474	0.465
2. Ads concerning dealing with side effects of treatment		-	0.477		-	0.461
3. Ads concerning hospitals or doctors offering services for cancer			-			-
Cronbach’s alpha			0.724			0.742

Note. All Ps <0.0005.

Nomological validity

The CR-DTCA exposure scale derived using Measure I items was correlated with health media exposure, information scanning about cancer treatment from media sources, and discussion with physicians about information from lay media sources across both Rounds 1 and 2 (Table 5).

TABLE 5.

Pearson’s Correlations Between CR-DTCA Exposure Scale and Health Media Exposure, Scanning for Treatment Information, and Discussion with Physicians (Study 1)

	Pearson’s r
CR-DTCA exposure in Round 1 by Health media exposure Round 1	0.360
CR-DTCA exposure in Round 1 by Scanning for treatment information in Round 1	0.309
CR-DTCA exposure in Round 1 by Discuss lay media information with treating doctor in Round 1	0.194
CR-DTCA exposure in Round 2 by Health media exposure Round 2	0.224
CR-DTCA exposure in Round 2 by Scanning for treatment information in Round 2	0.270
CR-DTCA exposure in Round 2 by Discuss lay media information with treating doctor in Round 2	0.179

Note. All Ps <0.0005

Test-retest reliability

The correlation of CR-DTCA exposure scales between the two rounds was moderate (Pearson’s r = 0.492) (Table 5).

Survey costs

Measure I items consisted of 52 words and three distinct responses, suggesting that survey costs would be low to moderate.

To summarize, based on this study among a probability sample of Pennsylvanian patients diagnosed with breast, colorectal, or prostate cancers, Measure I items could be considered as reliable and valid means of assessing exposure to CR-DTCA. These items demonstrated reasonable internal consistency, nomological validity against selected criterion measures, and test-retest reliability between repeated measures over time. Measure I items are likely to impose low to moderate survey costs. One limitation of Study 1 was the sample consisted of only Pennsylvanian patients diagnosed with three types of cancers. This may limit the generalizability of the validity and reliability of Measure I, although we have no a priori reasons to expect substantial differences in other patient populations with different cancer types. We attempted to address these limitations by comparing Measure I with more detailed alternate measures (II and III) in Study 2 among a sample of patients diagnosed with different cancers across the U.S.

STUDY 2—ONLINE PANEL CANCER PATIENT SURVEY

METHOD

Sample

The data for this study was from a self-administered web-based survey between March and May 2012 among 363 adults who reported they had been diagnosed with cancer; these participants were recruited from an existing national opt-in panel maintained by Survey Sampling International (SSI) comprising persons who had a history of cancer. The criteria for inclusion to the study were having any cancer diagnosis within the past two years (from January 2010 onwards) and being aged 21 years or older. The criterion to include only patients who were recently diagnosed was necessary as it was assumed CR-DTCA would be more salient for these patients compared to those who had been diagnosed and treated many years ago. Because prior experience of online surveys indicated an under-representation of male participants, we conducted quota sampling to ensure approximately half of the study participants would be male. The University of Pennsylvania IRB approved this validation study as research that qualified for exemption from IRB review.

Measures

CR-DTCA Exposure (Measures I to III)

Measure I items were identical to those described in Study 1. These items were averaged into a scale of CR-DTCA exposure (M = 2.77, SD = 1.22). All the participants in this study were asked the Measure I items.

Measure II comprised three items that showed respondents descriptions of the content of each category of CR-DTCA (i.e., treatment alternatives, dealing with treatment side-effects, and hospitals or doctors offering services) and two exemplars of CR-DTCA collated from television and print advertising sources. Participants were then asked to recall how frequently they saw each type of ads (Appendices A and B). To illustrate, respondents first read a description for ads about hospitals and doctors. Next, participants viewed two examples representing this category of ads (e.g., one print ad and one video ad showing hospitals and doctors providing cancer treatment services randomly chosen from a pool of four print ads and four video ads respectively). They were then asked how frequently they recalled encountering similar ads about hospitals and doctors offering treatment services since their cancer diagnosis. The three items were averaged into an alternate scale of CR-DTCA exposure (M = 2.83, SD = 1.19). We randomly assigned approximately two-thirds of the study population to respond to Measure II items.

Measure III was identical to Measure II with the exception that the items did not provide exemplars of ads (see Appendix A). The average of the three Measure III items formed another alternative CR-DTCA exposure scale (M = 2.83, SD = 1.14). One-third of the study population was randomly assigned to answer Measure III items.

Nomological Criterion Variables

We included items for health media exposure (M = 11.64, SD = 4.82), scanning for treatment information (M = 0.93, SD = 1.07), and discussion with physicians (M = 0.77, SD = 1.03) in the questionnaire that were identical to those described in Study 1. Respondents also reported on their frequency of use of eight media channels (i.e., newspaper, magazine, national and local news, television programs, radio, email, and the internet) in the past 7 days (between 0 to 7 times). Responses to these items were summed to create an index of general media use (M = 36.88, SD = 11.52).

Analyses

The validation analyses proceeded in a parallel manner to Study 1. We first assessed the performance of Measures I to III using the criteria laid out in the earlier section on validation approach except for the test-retest reliability criterion because we did not collect repeat measures of CR-DTCA exposure using these items in Study 2. To examine discriminant validity of Measures I to III, we used the ‘BOOTCOR’ program in Stata (based on 1000 bootstrap replications) to compare the correlations between the candidate measures (i.e., between Measure I and II) with the correlations of each candidate measure with the nomological criterion variables (e.g., Measure I with general media exposure). This program computes two correlation coefficients being compared, performs r-to-z transformation for both coefficients, and calculates a difference score for each resample. A z-test is used to infer if there is difference between the two correlations based on the distribution of difference scores (Neal, 2000). The logic behind this approach is consistent with the expectation that discriminant validity is achieved if the measure to be validated is related more strongly to its comparison measure and less so to the measures of other concepts (Schutt, 2011).

Additionally, we assessed the threat that respondents assigned to Measure II items would report their exposure to specific exemplars that they viewed rather than their recall of a general class of ads represented by the exemplars. To illustrate, if the mean recall for having seen or heard ads about medications to deal with treatment side-effects varied significantly between participants who were shown an ad for Procrit (erythropoietin alfa) compared to those who were shown an ad for Neulasta (pegfilgrastim), this could signal respondents were not recalling their overall exposure to ads for dealing with treatment side-effects. Instead, respondents may be recalling their exposure to the individual ads that they viewed. To assess this possible threat, we conducted one-way ANOVA tests to compare the means of each of the Measure II items with the ad shown as the between subjects factor³.

Another concern was the potential threat of ordering effects in the way the candidate measures were presented in the survey. We attempted to minimize this threat by separating Measure I items from Measures II/III items with other survey questions (e.g., nomological criterion measures). We assessed the potential of ordering effects by randomly assigning half the participants to receive Measure I first while the remainder received Measure II or III first. The above validation analyses were repeated to detect the presence of any substantive differences in the findings due to the order of measures in the questionnaire.

RESULTS

Approximately 52% of the respondents were female and the mean age was 53 years. Other participant characteristics are summarized in Table 2. This study population tended to be younger, have higher education levels, and was less ethnically diverse compared to the Pennsylvanian sample of cancer patients in Study 1. The distributions (mean and standard deviation) of individual items of Measures I – III and the resulting CR-DTCA exposure scales are summarized in Table 3. The means of the corresponding exposure items did not differ appreciably between Measures I – III, leading to a first conclusion that there was no substantial under-reporting of CR-DTCA exposure based on Measure I items compared to Measures II or III.

Internal consistency

In Study 2, the items in all three candidate measures of CR-DTCA demonstrated moderate to strong inter-item correlations within the respective measures (inter-item Pearson’s r = 0.427 to 0.693). Furthermore, the items in these three measures were internally consistent (Cronbach’s alpha=0.743 to 0.814) (Table 6). Principal component analysis extracted one single component with an eigenvalue greater than one in all three measures (accounting for 73%, 71%, and 62% of the total variances in Measures I, II, and III respectively).

TABLE 6.

Inter-Item Pearson’s Correlations Within Alternative Measures of Exposure To CR-DTCA (Study 2)

	Measure I			Measure II			Measure III
	1	2	3	1	2	3	1	2	3
1. Treatment alternatives	-	0.693	0.589	-	0.618	0.585	-	0.556	0.487
2. Dealing with side- effects		-	0.489		-	0.578		-	0.427
3. Hospital or doctors offering services			-			-			-
Cronbach’s alpha			0.813			0.814			0.743

Note. All Ps <0.0005.

Convergent validity

We found that topic-matching items from Measures I and II tended to be more strongly correlated than non-matching items; a similar pattern was observed for correlations between matching items from Measures I and III (Table 7). The summed CR-DTCA exposure scales derived from Measures I and II were strongly correlated (Pearson’s r = 0.713; 95% CI = 0.637 to 0.776), as were the scales from Measures I and III (Pearson’s r = 0.765; 95% CI = 0.674 to 0.837).

TABLE 7.

Inter-Item Pearson’s Correlations Between Alternative Measures of Exposure To CR-DTCA (Study 2)

Measure I	Measure II			Measure III
	1	2	3	1	2	3
1 Treatment alternatives	0.588	0.498	0.433	0.718	0.478	0.449
2 Dealing with side-effects	0.528	0.583	0.407	0.563	0.662	0.367
3 Hospital or doctors offering services	0.516	0.470	0.580	0.594	0.384	0.640

Note. All Ps <0.0005. Matching items across measures (in bold) tended to be more strongly correlated than non-matching items.

Nomological validity

The analysis showed that all three exposure scales using Measure I – III items were significantly correlated with the nomological criterion variables of general media use, health media exposure, treatment information scanning, and discussion with physicians about cancer information from lay media sources (Pearson’s r = 0.298 to 0.632) (Tables 8 and 9).

TABLE 8.

Pearson’s Correlations Between CR-DTCA Exposure Measures I and II and Nomological Criterion Measures (N=216)

	Measure I				Measure II
	r	95% CI	z	p	r	95% CI	z	p
1. General media use	0.224	0.088 – 0.355	7.382	<0.0005	0.299	0.177 – 0.414	6.527	<0.0005
2. Health media exposure	0.591	0.495 – 0.673	2.533	0.011	0.564	0.458 – 0.656	3.315	0.001
3. Scanning for treatment information	0.502	0.392 – 0.602	3.630	<.00005	0.410	0.292 – 0.517	5.546	<0.0005
4. Discuss lay media information with treating doctor	0.422	0.308 – 0.525	4.952	<.00005	0.343	0.226 – 0.454	6.234	<0.0005

Notes. r denotes the observed Pearson’s correlation. 95% confidence intervals, z-scores, and p-values are derived using the ‘BOOTCOR’ program (based on 1000 bootstrap replications) comparing each of these correlations with the correlation between Measure I and II.

TABLE 9.

Pearson’s Correlations Between CR-DTCA Exposure Measures I and III and Nomological Criterion Measures (N=147)

	Measure I				Measure III
	r	95% CI	z	p	r	95% CI	z	p
1. General media use	0.382	0.234 – 0.514	5.015	<0.0005	0.375	0.213 – 0.516	5.018	<0.0005
2. Health media exposure	0.674	0.560 – 0.764	1.589	0.112	0.627	0.524 – 0.714	3.151	0.002
3. Scanning for treatment information	0.511	0.384 – 0.623	4.035	<.00005	0.491	0.381 – 0.588	4.968	<0.0005
4. Discuss lay media information with treating doctor	0.347	0.172 – 0.503	5.645	<.00005	0.373	0.236 – 0.496	5.525	<0.0005

Notes. r denotes the observed Pearson’s correlation. 95% confidence intervals, z-scores, and p-values are derived using the ‘BOOTCOR’ program (based on 1000 bootstrap replications) comparing each of these correlations with the correlation between Measure I and III.

Discriminant validity

The correlations between Measures I and II CR-DTCA exposure scales and the correlations between Measures I and III exposure scales were largely stronger compared to the correlations between each measure and all of the nomological criterion variables (comparisons with z-tests were significant in 15 out of 16 tests) (Tables 8 and 9). We conclude that the three candidate CR-DTCA exposure scales were distinct from general media use, treatment information scanning, and discussion with physicians. However, we note that the correlation between Measure I and health media exposure was not significantly different from the correlation between Measure I and III, suggesting that Measure I may not discriminate so well from health media exposure. Alternatively, this may be due to the lower precision of these correlation estimates because a smaller sample was assigned to receiving Measure III.

Survey costs

As described above, Measure I items consisted of 52 words and three distinct responses. In contrast, Measure II items consisted of 269 words while Measure III had 161 words, approximately three to five times as many words as Measure I. The median survey duration of respondents who received Measures I and II (11.6 minutes) was significantly higher than those who received Measures I and III (6.7 minutes) based on the Mann-Whitney test (p<0.0005)⁴. This comparison suggests that Measure II required higher survey costs than Measure III.

Based on the ANOVA tests, mean CR-DTCA exposure as reported with Measure II items did not differ significantly among ad exemplars of the three categories of CR-DTCA shown. The eta-squared values for these tests ranged from 0.002 to 0.015, which indicated the variance in CR-DTCA exposure explained by the ad exemplars viewed was minimal (Table 10). Finally, we did not detect any evidence of ordering effects; comparing the above validation analyses between participants who answered Measure I items first and those who received the Measure I items later did not reveal any substantive differences.

TABLE 10.

One-Way Between-Participants Analysis of Variance of Effect of Ads Shown on Measure II Items in Study 2

Category of ads and specific ads	M	SD	F(3, 212)	P	Eta-squared
Alternative treatment print ads			0.249	0.862	0.004
1. Altoona Regional Radiosurgery (A)	2.50	1.414
2. Altoona Regional Radiosurgery (B)	2.70	1.488
3. Las Vegas Cyberknife at Summerlin	2.69	1.342
4. St. Peter’s University Hospital Cyberknife	2.69	1.514
Alternative treatment video ads			0.148	0.931	0.002
1. Memorial Cancer Institute Cyberknife	2.65	1.507
2. Fox Chase Cancer Center Minimally Invasive Surgery	2.64	1.471
3. Novalis TX at St Vincent’s Medical Center	2.53	1.424
4. Phoenix Cyberknife	2.72	1.374
Treatment side effects print ads			0.463	0.708	0.007
1. Zuplenz (A)	2.46	1.410
2. Aloxi	2.78	1.388
3. Zuplenz (B)	2.63	1.369
4. Zometa	2.56	1.350
Treatment side effects video ads			0.873	0.456	0.012
1. Neulasta	2.65	1.388
2. Procrit (A)	2.36	1.266
3. Procrit (B)	2.77	1.417
4. Procrit (C)	2.67	1.438
Doctor and hospital print ads			1.066	0.364	0.015
1. Sylvester Comprehensive Cancer Center (A)	3.12	1.508
2. Sylvester Comprehensive Cancer Center (B)	3.20	1.265
3. Sylvester Comprehensive Cancer Center (C)	3.20	1.379
4. Sylvester Comprehensive Cancer Center (D)	3.56	1.270
Doctor and hospital video ads			0.150	0.930	0.002
1. UNC Cancer Center	3.16	1.347
2. Carle Cancer Center	3.33	1.476
3. Hudson Valley Hospital Center	3.29	1.390
4. Terrebonne General Medical Center – Mary Bird Perkins Cancer Center	3.28	1.294

Note. The panel of video and print ads is included in Appendix B.

To summarize Study 2 findings, this web-based survey among an opt-in sample of patients across the U.S. diagnosed with a wide variety of cancers compared the performance of three candidate measures of CR-DTCA exposure. Measures I to III displayed reasonable internal consistency, convergent validity and nomological validity. Discriminant validity was largely supported; the majority (15 out of 16 comparisons) of the correlations between candidate CR-DTCA measures was significantly stronger than the correlations between each measure and four other concepts. Measure I was judged to incur the lowest survey costs while Measure II required higher survey costs than Measure III.

DISCUSSION

In this validation study, we assessed two conceptual issues of the level of content specificity and implications of using of ad exemplars for aided recall in designing exposure measures. We presented findings in the specific context of comparing the performance of three alternative measures of CR-DTCA exposure that varied in these two dimensions. The study utilized a comprehensive set of criteria across two distinct study populations of cancer patients and involved different data collection designs. Study 1 involved repeated mailed questionnaires among a probability sample of patients from Pennsylvania who were diagnosed with breast, colorectal, or prostate cancers. In comparison, Study 2 involved a cross-sectional web-based survey among an opt-in sample of patients across the U.S. who were diagnosed with a variety of cancers.

Due to the multiple reliability and validity criteria in this research, we summarized the findings of the performance of candidate measures of CR-DTCA exposure based on Studies 1 and 2 for comparison in Table 11 (adapted from Nagler & Hornik (2012) and Romantan et al. (2008)). Across the criteria of internal consistency, convergent, nomological, and discriminant validity, Measures I – III performed equally well for the most part. All three measures demonstrated adequate levels of internal consistency and unidimensionality. Measure 1 was further evaluated for test-retest reliability over two rounds of survey in Study 1. The three measures differed in terms of survey costs: Measure II was the most costly while Measure I was least costly.

TABLE 11.

Summary Analysis Across Validity Criteria

Candidate Exposure Measures	Internal Consistency	Convergent validity	Nomological validity	Discriminant validity	Test-retest reliability	Survey costs
Measure I	4	4	4	3	4	4
Measure II	4	4	4	4	n/a	1
Measure III	4	4	4	4	n/a	3

Note. 1 = worst performance; 4 = best performance

Discrimination between CR-DTCA measures and three distinct concepts (i.e., general media exposure, treatment information scanning, or discussion with physicians) was largely supported. However, we found one comparison where Measure I, the briefest of three measures in Study 2, could not be unequivocally discriminated from general health media exposure. This finding could imply that Measure I items lack the optimal level of content specificity and could be tapping the general concept of media exposure instead of CR-DTCA. If so, this would mean that adding content specific cues including ad exemplars or detailed descriptions would be preferable to strengthen the validity of these CR-DTCA measures. We note here that this particular comparison trended in the expected direction (Pearson’s r between Measure I and III was 0.765 vs. Pearson’s r between Measure I and health media exposure was 0.674) consistent with the other 15 comparisons. Due to the lower precision involved in this analysis, we suggest a replication involving a larger sample to ascertain if Measure I could be clearly discriminated from health media exposure.

Regarding the first conceptual issue of optimal content specificity for survey measures of ad exposure, the findings comparing Measure I with more elaborate measures (II and III) did not suggest that the brevity of Measure I items led to systematic under-reporting of exposure to CR-DTCA. One interpretation from this finding is that Measure I, while being the briefest measure, may have contained sufficiently content-specific cues for the study population of cancer patients to accurately report their exposure to CR-DTCA such that any more detailed elaborations about ad categories, media channels, or exemplars yielded minimal incremental recall about ad exposure. Conversely, the increased content specificity in item phrasing in Measures II and III or having exemplars in Measure II did not appear to result in respondents reporting higher exposures systematically, which would have raised concerns that the increased content specificity was tapping into other constructs including selective exposure, underlying interest, or prior knowledge. The implication for media exposure measurement research more generally is that more content specificity may not always be necessary.

For the second conceptual concern about the utility of including exemplars, there was no clear advantage that Measure II was superior to Measures I or III in terms of reliability and validity criteria. The obvious disadvantage was increased survey costs and the added complexity of data collection methods requiring audiovisual technologies for Measure II whereas Measures I and III could be administered more easily. Another notable finding that would interest researchers beyond this domain of CR-DTCA is that respondents were able to extrapolate their exposure to categories of CR-DTCA from viewing two exemplars of each ad category using Measure II items; their responses did not suggest they were merely recalling exposure to specific exemplars that they viewed. This study offers empirical evidence in the specific context of CR-DTCA that respondents could understand and process the exemplars as triggers to recall their overall CR-DTCA exposure.

There is a practical implication for this research for subsequent efforts to understand the impact of health information exposure through DTCA on health outcomes particularly in the context of cancer treatment advertising. The replication of validity testing for Measure I across these two studies and exploration of alternative measures (II and III) in Study 2 provides increased confidence in the performance of Measure I to assess CR-DTCA exposure among a diverse population of cancer patients. The representative sample of cancer patients in Study 1, although from a single state, could be viewed as an improvement from studies that are limited to convenience samples of cancer patients or those that involve patients from a single health system. While the study population in Study 2 was not a representative sample of cancer patients across the U.S., we argue that Study 2 complemented the findings from Study 1 through the inclusion of a more diverse population of cancer patients (i.e., broader geographic locations and cancer diagnoses). Moreover, Study 2 involved a web-based survey that enabled us to test exposure measures that included audiovisual exemplars of CR-DTCA; this would be precluded by mailed questionnaires or phone interviews. This validation approach involving a combination of study populations, survey designs, and modes of data collection described here may serve as an illustrative example for future research aimed at developing and validating self-reported measures of exposure to health-related information more broadly.

We should caution that the observed similarity in performance across brief and more specific measures could be unique because the ad content in question—CR-DTCA—is highly salient for the study population of cancer patients. This may not apply in another context where the ad content is less salient for respondents (e.g., asking about exposure to ads about heart disease treatment in healthy adult populations). More content specificity and exemplars may elicit better self-reports of exposure in those contexts. We encourage future investigations to consider adapting the different versions of exposure measures described in this current study for other ad content and to systematically study the minimum level of content specificity sufficient to elicit reliable and valid self-reported media exposure across other domains. Although substantial modification may be needed for Measure II because of the content-specific exemplars, Measures I and III may be more easily adapted into survey items that measure media exposure across diverse types of media content (e.g., DTCA for other illnesses or health campaign ads).

The study was limited by the reliance on self-reported and close-ended measures of exposure to CR-DTCA. Recall biases are a threat to inferences about whether self-reported CR-DTCA exposure accurately reflects their memory of encoded exposure to such advertising in the media (Southwell et al., 2002; Southwell & Langteau, 2011; Southwell, 2005). Consequently, population-level measures of exposures through media market gross rating points of televised health campaign advertising or news reporting have been increasingly implemented as predictors of behaviors instead of individual-level self-reported media exposure measures (Farrelly, Davis, Haviland, Messeri, & Healton, 2005; Hwang & Southwell, 2009; Wakefield et al., 2008). Closed-ended survey questions that specify particular content may contain researchers’ biases and may miss the content that is most meaningful for the study population compared to open-ended questions that permit more in-depth assessment of exposures that are most interest to the target population (Clarke & Kline, 1974). Nevertheless, these alternatives to self-reported and close-ended measures are not without their limitations. Media-market gross rating points represent environmental availability of media messages and therefore reflect the opportunity to be exposed to media messages and not individuals’ exposure (Slater, 2004). Responses to open-ended questions about exposure may reflect one’s knowledge about a health topic rather than merely exposure (Salmon, 1986) and are often more costly and complex to collect and analyze (Romantan et al., 2008) than closed-ended items.

Despite these limitations, this validation study offers systematic evidence on the recurring conceptual concerns of optimizing survey items’ content specificity and decisions about including exemplars to aid recall of a class of media content. The validation approach encompassing complementary study populations, designs, modes of data collection, and comprehensive criteria may serve as a model for future research aimed at systematic comparisons of candidate measures of media exposure that vary in these dimensions.

APPENDIX A. Survey Items of Alternative Measures of Exposure to CR-DTCA

Measure I

Since your diagnosis, how often have you seen or heard advertisements concerning each of the following? Check all that apply:

	Never	Less than every month	About twice a month	About once a week	Almost every day
Treatment alternatives for your cancer	1	2	3	4	5
Dealing with side effects of treatment	1	2	3	4	5
Hospitals or doctors offering services for cancer	1	2	3	4	5

Measure II

The following questions ask about advertisements that you may or may not have come across in the mass media (e.g. television, radio, newspapers, magazines, billboards, or the internet.)

Sometimes advertisements about treatment alternatives for your cancer appear in the mass media (e.g., television, radio, newspapers, magazines, billboards, or the internet).

Please view the following two examples of advertisements concerning treatment alternatives for your cancer.

[Respondent was shown two randomly selected ads (one print ad and one video) of advanced technology or alternatives for cancer treatment.]

Since your diagnosis, how often have you seen or heard advertisements concerning treatment alternatives for your cancer?

Never	Less than every month	About twice a month	About once a week	Almost every day
1	2	3	4	5

Sometimes advertisements about dealing with side effects of cancer treatment appear in the mass media (e.g., television, radio, newspapers, magazines, billboards, or the internet). Please view the following two examples of advertisements concerning dealing with side effects of treatment.

[Respondent was shown two randomly selected ads (one print ad and one video) about dealing with side effects of treatment.]

Since your diagnosis, how often have you seen or heard advertisements concerning dealing with treatment side effects?

Never	Less than every month	About twice a month	About once a week	Almost every day
1	2	3	4	5

Sometimes hospitals or doctors advertise their services (radiation therapy, chemotherapy, or comprehensive treatment) for treating patients with cancer. These advertisements may appear in the mass media (e.g., television, radio, newspapers, magazines, billboards, or the internet).

Please view the following two examples of advertisements concerning hospitals or doctors offering services for cancer.

[Respondent was shown two randomly selected ads (one print ad and one video) of hospitals or doctors offering services for cancer.]

Since your diagnosis, how often have you seen or heard advertisements concerning hospitals or doctors offering services for cancer?

Never	Less than every month	About twice a month	About once a week	Almost every day
1	2	3	4	5

Measure III

The following questions ask about advertisements that you may or may not have come across in the mass media (e.g. television, radio, newspapers, magazines, billboards, or the internet.)

Sometimes advertisements about treatment alternatives for your cancer appear in the mass media (e.g., television, radio, newspapers, magazines, billboards, or the internet). Since your diagnosis, how often have you seen or heard advertisements concerning treatment alternatives for your cancer?

Never	Less than every month	About twice a month	About once a week	Almost every day
1	2	3	4	5

Sometimes advertisements about dealing with side effects of cancer treatment appear in the mass media (e.g., television, radio, newspapers, magazines, billboards, or the internet). Since your diagnosis, how often have you seen or heard advertisements concerning dealing with treatment side effects?

Never	Less than every month	About twice a month	About once a week	Almost every day
1	2	3	4	5

Sometimes hospitals or doctors advertise their services (radiation therapy, chemotherapy, or comprehensive treatment) for treating patients with cancer. These advertisements may appear in the mass media (e.g., television, radio, newspapers, magazines, billboards, or the internet). Since your diagnosis, how often have you seen or heard advertisements concerning hospitals or doctors offering services for cancer?

Never	Less than every month	About twice a month	About once a week	Almost every day
1	2	3	4	5

APPENDIX B. Panel of Video and Print Ads Displayed for Measure II Items (Study 2)

1. Print ads for hospitals or doctors offering services

   Print ad 1	Print ad 2	Print ad 3	Print ad 4

   Note. All four print ads are from U Miami Sylvester Comprehensive Cancer Center

2. Video ads for hospitals or doctors offering services

   Video ad 1	Video ad 2	Video ad 3	Video ad 4
   UNC Cancer Care	Carle Cancer Center	Hudson Valley Hospital Center	Terrebonne General Medical Center – Mary Bird Perkins Cancer Center

3. Print ads for treatment side effects

   Print ad 1	Print ad 2	Print ad 3	Print ad 4
   Zuplenz (anti-nausea)	Aloxi (anti-nausea)	Zuplenz (anti-nausea)	Zometa (prevent skeletal complications)

4. Video ads for treatment side effects

   Video ad 1	Video ad 2	Video ad 3	Video ad 4
   Neulasta (increase immune cell count)	Procrit (increase red blood cells)	Procrit	Procrit

5. Print ads for treatment alternatives

   Print ad 1	Print ad 2	Print ad 3	Print ad 4
   Altoona Regional Radiosurgery	Altoona Regional Radiosurgery	Las Vegas Cyberknife at Summerlin	St. Peter’s University Hospital Cyberknife

6. Video ads for treatment alternatives

   Video ad 1	Video ad 2	Video ad 3	Video ad 4
   Memorial Cancer Institute Cyberknife	Fox Chase Cancer Center Minimally Invasive Surgery	Novalis TX at St Vincent’s Medical Center	Phoenix Cyberknife