What’s the Appeal? Testing Public Service Advertisements to Raise Awareness About Gynecologic Cancer

Abstract

In 2013, the Centers for Disease Control and Prevention’s (CDC) Inside Knowledge: Get the Facts About Gynecologic Cancer campaign tested creative concepts for English- and Spanish-language video advertisements (for use on television and the Internet) with women aged 35–64 years. Sixteen English and nine Spanish focus groups were conducted in four U.S. cities. CDC used animatics (a series of photographs edited together with a sound track) to simulate produced advertisements, without having to incur the high cost of filming and production. Advertisement concepts consistently resonating with participants featured cancer survivors, were straightforward, included information about cancer symptoms, displayed Inside Knowledge educational materials, and featured diverse women. In the general population focus groups, a primacy testing order effect was observed in which the concept tested first tended to be the most favorably received. Varying the order in which concepts were tested and considering testing order when interpreting results was critical.

Background

In January 2007, the Gynecologic Cancer Education and Awareness Act of 2005¹ was signed into law, authorizing the Centers for Disease Control and Prevention (CDC), in collaboration with the U.S. Department of Health and Human Services’ Office on Women’s Health, to develop the Inside Knowledge: Get the Facts About Gynecologic Cancer campaign (www.cdc.gov/cancer/knowledge). This national multimedia initiative raises awareness among women and health care providers about the signs, symptoms, risk factors, and prevention strategies related to the five main gynecologic cancers—cervical, ovarian, uterine, vaginal, and vulvar. The messages of the Inside Knowledge campaign were developed based on an extensive review of the scientific literature, consultation with gynecologic cancer experts, and quantitative and qualitative research with health care providers and women aged 35 years and older.² Inside Knowledge emphasizes six messages:

• ■Pay attention to your body and know what is normal for you. Gynecologic cancers have warning signs.
• ■When gynecologic cancers are found early, treatment can be most effective.
• ■If you have vaginal bleeding that is unusual for you because of when it occurs or how heavy it is, see a doctor right away.
• ■If you notice any other unexplained signs or symptoms that last for 2 weeks or longer, see a doctor.
• ■Get a Pap test regularly to screen for cervical cancer. Cervical cancer is the only gynecologic cancer for which screening is recommended.
• ■Consider getting the human papillomavirus (HPV) vaccine if you are in the age group for which it is recommended.

To disseminate these messages to women, the Inside Knowledge campaign has developed and released a variety of consumer education materials, as well as television, radio, and Internet advertisements. Inside Knowledge materials are developed using the four-stage Health Communication Process endorsed by the U.S. Department of Health and Human Services.³ In stage 1, intended audiences are identified and their relevant knowledge, attitudes, and behaviors are studied. In stage 2, communication concepts are developed and tested with intended audiences using qualitative research methods. In stage 3, the program is implemented and audience exposure is assessed. In stage 4, the impact of the program is evaluated and refinements are identified and incorporated into future campaign development. The Health Communication Process is not linear, but rather is a circular model in which stages are revisited in a continuous loop of planning, development, implementation, and refinement.

This report describes the most recent round of stage 2 development—concept testing of English- and Spanish-language video advertisements for television and Internet.

Concept-Testing Process

In 2013, CDC conducted 16 general population and nine Spanish focus groups: four general population groups per market were held in New York City, Atlanta, Chicago, and Los Angeles; two Spanish groups per market were held in New York City, Atlanta, and Chicago, and three Spanish groups were held in Los Angeles. Each focus group included seven to nine participants and lasted approximately 2 hours.

The cities where focus groups were held were selected based on their geographic diversity, the racial/ethnic diversity of their residents, and availability of quality focus group facilities. Participants were limited to women aged 35–64 years who had not had a hysterectomy and had never been diagnosed with gynecologic cancer. Participants were recruited using public information (e.g., telephone directory) and venues (e.g., city parks), as well as proprietary lists (e.g., lists maintained by focus group facilities). Participants in the general population and Spanish focus groups included a diverse mix of women (Table 1).

Table 1.

Participant Characteristics (%), Inside Knowledge Video Advertisement Concept Testing, 2013

Characteristic	General population focus groups (n = 104)	Spanish focus groups (n = 71)
Age
35–39 years	4.3	7.3
40–44 years	13.7	16.4
45–49 years	12.9	32.7
50–54 years	30.9	21.8
55–59 years	23.0	14.5
60–64 years	15.1	7.3
Race/ethnicity
Caucasian	42.4	0.0
African-American	38.8	0.0
Hispanic	12.2	100.0
Asian	3.6	0.0
Other	2.9	0.0
Educational attainment
< High school	0.7	9.1
High school graduate	34.5	40.0
Some college	34.5	30.9
College degree	25.9	18.2
Advanced degree	4.3	1.8
Income
< $25K	12.9	14.5
$25K–$39K	18.7	47.3
$40K–$64K	29.5	29.1
$65K–$99K	30.9	9.1
≥ $100K	7.9	0.0
Medical insurance coverage
Yes	82.7	60.0
No	17.3	40.0
Timing of last pap test
≤ 1 year	75.5	54.5
2–3 years	17.3	41.8
≥ 4 years	7.2	3.6
Country of origin
Mexico	Not assessed	40.0
Guatemala		10.9
Puerto Rico		9.1
Colombia		7.3
Dominican Republic		7.3
Ecuador		7.3
Peru		5.6
Honduras		3.6
Costa Rica		1.8
Cuba		1.8
El Salvador		1.8
Venezuela		1.8

Due to the expense of filming advertisements, the creative concepts tested were formatted as animatics, a series of photographs edited together on video, along with a sound-track that includes the ‘‘working’’ script read by non-actors. These simulated advertisements included effects seen in regular finished ads, such as zooms, pans, and dissolves from one scene to another. Prior to showing the animatics, the focus group moderator explained the unfinished nature of the advertisements and emphasized that any concept, if produced, would include moving video, as well as professional actors. Animatics more closely simulate actual video advertisements than storyboards (the format traditionally used in concept-testing studies) which show a series of printed photos or drawings representing the progression of an advertisement’s storyline.⁴

In total, 11 concepts were tested; however, no more than seven per language were tested in any single market (Table 2). The concepts tested included a variety of appeals—testimonial, informational, humorous, and metaphorical. Of the 11 concepts, seven were tested in both general population and Spanish groups; three were tested only in general population groups; and one was tested only in Spanish groups. When developing Spanish advertisements or adapting advertisement created in English for a Spanish-speaking audience, Inside Knowledge works with consultants who specialize in developing Spanish-language materials, and native Spanish speakers to ensure cultural appropriateness and translation quality.

Table 2.

Concepts Tested by Market, Inside Knowledge Video Advertisement Concept-Testing, 2013

		Audience		Revisions
Concept name	Type of appeal	General population	Spanish	Market 1 New York City	Market 2 Los Angeles	Market 3 Atlanta	Market 4 Chicago	Production viability
Body Words	Informational	X	X	Original version	Major	Major	Major	Viable
Three Survivors	Testimonial	X	X	Original version	Major	Minor	None	Viable
Busy Women	Testimonial	X	X	Original version	Minor	None	Minor	Not viable
Reunión	Testimonial		X	Original version	None	Major	Testing discontinued
Book Club	Testimonial	X		Original version	Minor	Testing discontinued
Drip, Drip	Metaphorical	X	X	Original version	Testing discontinued
Patterns	Metaphorical	X	X	Original version	Testing discontinued
Inner Voices	Humorous	X	X	Original version	Testing discontinued
Personal Story	Testimonial	X	X	Not yet developed	Original version	None	None	Viable
Shoes	Metaphorical	X		Not yet developed	Original version	Testing discontinued
Mix and Match	Metaphorical	X		Not yet developed		Original version	Testing discontinued

The order in which concepts were shown was varied in every focus group. Concepts were tested in two blocks of two to four concepts each. The concepts in each block were shown one by one. Immediately after seeing each concept, participants completed an evaluation form to capture their initial reactions prior to group discussion. Then, each concept shown in the block was discussed in turn. This process was repeated for the second block of concepts. After discussion of the final concept in the second block, participants completed a form in which they ranked all of the concepts shown, in order of preference. The ranks were tabulated on a flip chart and discussed to clarify why participants gave the rankings they did.

At least 11 days were allotted between each study market to allow time to revise existing concepts, prepare new concepts in response to participant feedback, and determine which, if any, concepts should be dropped from testing. Generally, fewer concepts were tested in each subsequent market. For instance, in the general population groups, seven concepts were tested in Market 1; six in Market 2; five in Market 3; and finally, only four were tested in Market 4. Over time and from one market to the next, less promising concepts were discarded and more discussion time was devoted to refining the most promising concepts. Concepts tested in multiple markets were typically revised at some point during the testing process. Only one concept (‘‘Personal Story’’) survived through Market 4 without revision. This concept was developed based directly on the feedback of Market 1 participants and proved to be viable for production. Two other concepts (‘‘Shoes’’ and ‘‘Mix and Match’’) were developed during testing in an attempt to find an acceptable metaphorical approach. Neither proved to be viable for production, due to negative reactions from participants.

All focus groups were monitored by at least three on-site observers. Additional observers watched the groups remotely via online video streaming. All observers recorded detailed notes. After each market, the lead investigator prepared a summary of results, which was revised until each observer validated the findings. After each of the first three markets, a debriefing meeting was held to discuss revising existing concepts, discontinuing testing of less promising concepts, and developing new concepts for testing in the next market.

Identification of Viable Concepts

Of the four concepts tested in the final market, three were determined to be viable for production. The remaining concept (‘‘Busy Women’’) was deemed unsuitable for production, as it generated confusion, despite being favorably received in both general population and Spanish focus groups. In its plotline, three women, each experiencing an especially busy and stressful time of life, described symptoms they were having. One was diagnosed with a gynecologic cancer. Many focus group participants interpreted the take-away message as a warning against having too much stress, rather than the intended message that no matter how busy you are, take time to notice symptoms and see a healthcare provider. In one focus group, this scenario was mistakenly interpreted as literal odds (i.e., one in three women who experience symptoms will be diagnosed with gynecologic cancer).

Several creative elements consistently resonated with participants (Table 3). Most participants reported that they were not familiar with many of the symptoms that can signal gynecologic cancers, and information about symptoms—included in all of the concepts—captured their attention. In addition, several concepts featured a gynecologic cancer symptoms diary developed by the Inside Knowledge campaign, and many participants said the diary caught their attention and expressed interest in obtaining a copy. In most groups, participants responded favorably to concepts in which there was racial/ethnic diversity. Participants also wanted to see ‘‘real’’ women with diverse body shapes, and many commented at length that they appreciated the inclusion of full-figured women in particular. Finally, concepts featuring cancer survivors were generally very well received.

Table 3.

Creative Elements that Did and Did Not Resonate, Inside Knowledge Video Advertisement Concept-Testing, 2013

Creative elements that resonated	Cancer survivors	Testimonial appeals featuring cancer survivors were preferred by participants. Many commented that featuring gynecologic cancer survivors communicated hope and drew their attention.
	Symptom information	Many participants said that the symptoms shown or mentioned in the advertisements drew their attention, particularly symptoms that they were not aware could signal gynecologic cancer, such as bloating and frequent urination.
	Display of Inside Knowledge consumer education materials	Participants consistently said that the Inside Knowledge gynecologic cancer symptoms diary featured in several concepts caught their attention; many participants said they would like to obtain a copy for personal use.
	Racial/ethnic diversity	Many participants responded favorably to the racial/ethnic diversity of the women featured in concepts.
	Full-figured women	Generally, participants reacted very positively to the full-figured women featured in concepts. Participants said these women were especially relatable because they looked “real.”
Creative elements that did not resonate	Humor	Participants consistently reacted negatively to the use of humor, stating it was not appropriate given the seriousness of the topic.
	Metaphors	Participants preferred straightforward appeals, and generally reacted negatively to appeals with abstract imagery and analogies between a woman’s body and inanimate objects.

Creative elements that did not test well included humor and metaphors. Participants consistently reacted negatively to a concept that featured a body ‘‘talking’’ to its owner in a humorous voice, stating that this approach was not appropriate, given the seriousness of the topic. A variety of concepts with metaphors were tested, and none performed well. For example, one metaphorical concept (‘‘Mix and Match’’) showed women wearing mismatched clothing, while the announcer explained, ‘‘You know when something doesn’t feel right,’’ and urged women to take note when something feels ‘‘off.’’ Participants generally found this concept to be overly ‘‘busy,’’ and disliked the abstract imagery. Another metaphorical concept (‘‘Drip, Drip’’) likened a dripping faucet to an ongoing gynecologic cancer symptom, and some participants reacted very negatively to this analogy.

The results of the general population and Spanish focus groups were notably consistent. However, a strong primacy testing order effect was observed in general population focus groups, but not in Spanish focus groups. The concept tested first in general population groups was consistently the highest ranked (the most preferred) (Table 4). It is possible that this effect was due to the similarity of information included in all of the concepts tested, and the substantial interest in the subject matter exhibited by many participants. Thus, the novelty of first exposure to the information may have produced a ‘‘halo effect’’ of sorts around the first concept tested. In any case, it was critical to vary the order in which concepts were tested in the general population focus groups and to consider testing order when interpreting results. Equally important was the need to pay close attention to what participants said about each concept, as opposed to how they rated or ranked each concept.

Table 4.

Ranks^* of Top Four General Population Concepts by Testing Order, Inside Knowledge Video Advertisement Concept-Testing, 2013

		Testing order
Concept name	Market	First	Not first
Body Words	Market 1: New York City	1.0	2.0
	Market 2: Los Angeles	2.0	2.7
	Market 3: Atlanta	1.0	3.7
	Market 4: Chicago	1.7	3.0
Busy Women	Market 1: New York City	Not tested first in Market 1
	Market 2: Los Angeles	Not tested first in Market 2
	Market 3: Atlanta	1.0	3.7
	Market 4: Chicago	Not tested first in Market 4
Three Survivors	Market 1: New York City	3.0	2.7
	Market 2: Los Angeles	1.0	3.3
	Market 3: Atlanta	1.0	2.3
	Market 4: Chicago	Not tested first in Market 4
Personal Story	Market 1: New York City	Not yet developed
	Market 2: Los Angeles	1.0	3.3
	Market 3: Atlanta	1.0	2.3
	Market 4: Chicago	1.0	2.3

^*Reported ranks were established by consensus at the end of each focus group (rank 1 = most preferred concept). Each concept was tested first in no more than one group per market with the exception of Body Words in Market 4. The mean rank is reported when the testing order designation applies to more than one group. The higher rank in each row is shown in bold.

Production Considerations

The Inside Knowledge campaign is committed to producing high-quality materials in both English and Spanish. Inside Knowledge video advertisements are developed with film production professionals and meet the highest technical quality standards—produced in high definition and letterbox format, typically in lengths from 15 to 60 seconds.

Each of the three viable concepts will be produced and disseminated as funding allows. All were tested in both English and Spanish and will likely be filmed in both languages. Generally, there are cost savings associated with this type of dual production.

It is anticipated that women’s preferences related to video advertisements promoting gynecologic cancer awareness may evolve over time. However, the results of the current study are generally consistent with a 2009 Inside Knowledge concept-testing study,⁵ which also found that women preferred straightforward and testimonial approaches featuring ‘‘real’’ and diverse women, and they reacted negatively to the use of humor and metaphor in advertising promoting gynecologic cancer awareness.