Cognitive interviews to improve a patient-centered contraceptive effectiveness poster

Abstract

Objectives:

To refine by qualitatively comparing a patient-centered contraceptive effectiveness poster to a Centers for Disease Control and Prevention (CDC) poster based on comprehension, relevance, acceptability, design, and overall preference.

Study Design:

We conducted cognitive interviews with 26 women aged 18–44 living in North Carolina who spoke and read English and had ever had sex. We interviewed women about both a CDC and a patient-centered poster in alternating order. Participants were contraceptive users and non-users that we selected purposively to have a range of characteristics that might influence their perspective: age, race/ethnicity, previous births and pregnancies, contraceptive method(s) used in the past three months, pregnancy intentions, and numeracy. The initial response rate for participants was 55%. We coded the interviews for comprehension, relevance, and acceptability as defined in cognitive theory, as well as design and overall preference. We structured the 26 interviews into four rounds and revised the patient-centered poster after each round to improve these measures.

Results:

By the final round, 83% of women preferred the patient-centered poster overall, and it was preferred by the majority of women in terms of comprehension (86%), relevance (86%), and design (100%). Women raised few concerns about the acceptability of the final version of the patient-centered poster. Women identified many issues with both posters that the researchers did not anticipate, highlighting the value of patient-centered design approaches to educational materials.

Conclusions:

This study refined a patient-centered poster so that its language is clear and it addresses the informational needs of its target audience.

Implications:

The Centers for Disease Control and Prevention and the Office of Population Affairs recommend that clinicians educate women about contraception. This study developed a poster that could help clinicians follow this recommendation. Before widespread implementation, more research is needed to evaluate the poster’s impact on contraceptive knowledge and behaviors.

1. Introduction

A Cochrane review of the effect of decision aids on helping people make health treatment and screening decisions found that decision aids like posters can improve knowledge and decision-making [1, 2]. A second systematic review of contraceptive education interventions, such as written materials, found that the majority improve contraceptive knowledge, and many can also significantly increase comfort with the decision making process and improve contraceptive intentions [3]. Some interventions have also demonstrated impacts on contraceptive use and pregnancy outcomes [3]. The Centers for Disease Control and Prevention (CDC) adapted an existing [4] contraceptive education poster (Figure 1) to help providers adhere to the CDC/Office of Population Affairs (OPA) recommendation that reproductive health counseling include contraceptive education [2]. However, while posters such as the CDC’s are thoughtfully developed and scientifically accurate, they may not be designed with patients’ input, as is recommended by the CDC and OPA [5]. When we involve patients in design, it helps make educational materials understandable [6], especially for the half of Americans with low health literacy [7]. Making health materials understandable is both a national priority, according to the Institute of Medicine [7], and important in sexual education specifically [5] because low literacy/numeracy have been associated with poor contraceptive knowledge and use [8]. We used cognitive interviews to design a patient-centered poster that educates women about contraceptive effectiveness and their risk of pregnancy with unprotected sex; we tested this poster against the CDC’s contraceptive effectiveness poster.

Fig. 1.

CDC-adapted contraceptive effectiveness poster.

2. Material and Methods

2.1. Participants

We conducted in-person cognitive interviews with 26 women in four unequal-sized waves (N=4, 7, 8, and 7). Women were age 18–44, had ever had vaginal intercourse, and were able to speak and read English. We recruited women through a Family Medicine clinic, a University email list, and a women’s health research center. We used purposive sampling [9] to recruit women with a range of characteristics that might influence their perspective on comprehension, relevance, and acceptability. These characteristics were age, race/ethnicity, previous births and pregnancies, contraceptive method(s) used in the past three months, motivation to avoid pregnancy, and numeracy (Berlin single item scale [10]). We did not provide compensation for participation. The University of North Carolina at Chapel Hill institutional review board approved this study (IRB #17–1246).

2.2. Cognitive Interviewing Procedure

Cognitive interviewing is a method for studying how people process and feel about written and graphical materials [11]. It is iterative, with rounds of interviews followed by refinements to the material being tested [11]. Cognitive interviewing has been used to test comprehension of educational materials [12–14] and contraceptive information [15].

We presented the posters in alternating order by interview. We used the combined method of cognitive interviewing, which begins with the participant “thinking-aloud” [16] as they process a poster, and then exploring their cognitive processing further in a structured interview [16, 17].

2.3. Measures

We measured comprehension, relevance, and acceptability, the three key components of cognitive processing identified by Tourangeau’s cognitive theory [18] as adapted by Vreeman and Choi [19, 20]. We also measured design and preference.

Comprehension is a measure with two constructs: (1) intent, whether the respondent understands the information presented, and (2) meaning, whether the respondent understands the specific words and phrases used. We assessed intent by probing whether participants could use the poster to correctly answer questions about contraception. We assessed meaning by probing whether the participant understood the key terms on each poster.

Relevance (Tourangeau’s “response mapping”) is a measure of whether the information applies to the respondent. We assessed relevance by probing whether the participant found the information on the poster useful.

Acceptability (Tourangeau’s “sensitivity”) is a measure of whether the information seems truthful and inoffensive. We assessed acceptability by probing whether anything on the poster offended the participant and whether she believed the information on the poster.

Design was assessed by asking the participant to identify missing or extraneous information and propose changes to the poster’s appearance.

Preference was assessed by asking the participant to select which poster she preferred in terms of comprehension, relevance, design, and overall.

2.4. Analysis

We audio-recorded and transcribed each interview and developed a framework to code the interviews for our measures. Using NVivo 11, after each wave one interviewer (SA) coded all and a second researcher (MB) coded half of the interviews described. The kappa agreement scores for the two coders were 0.77 (range 0.41–0.94) for comprehension, 0.76 (0.44–0.99) for relevance, 0.77 (0.24–0.97) for acceptability, and 0.76 for design (0.55–0.89). The coders resolved discrepancies through discussion. We created an overview report for each poster documenting issues, the number of women mentioning an issue, and the number of mentions. After each round, we used the overview to revise the patient-centered poster. Our saturation criterion for each round was when an interview produced no new suggestions warranting serious modifications to the poster.

2.5. Derivation of Contraceptive Effectiveness Measures

The CDC poster expresses contraceptive effectiveness using the annual failure rate with typical use of contraception [5]. The first versions of the patient-centered poster instead used the theoretical average time-to-pregnancy for each contraceptive method. We derived these from the annual failure rates using the binomial probability function to solve for the time at which there is a 50% probability of at least one pregnancy having occurred. This measure estimates the average time women might expect to use a contraceptive method before they have an unplanned pregnancy, similar to how flood risks are commonly expressed [21]. This measure assumes that failure rates are constant over time, which may not be the case. Later versions of the poster used a “times more effective” measure of contraceptive effectiveness, which is calculated by dividing the probability of pregnancy for unprotected sex by the failure probabilities for each method. This measures gives how many times more effective at pregnancy prevention a contraceptive method is compared to having unprotected sex

3. Results

Of the 62 eligible women we invited to participate, 34 (55%) agreed, and 26 (42%) were interviewed before we reached saturation. Table 1 provides descriptive statistics for participating women on key characteristics. In general, the study sample was less likely to have children than US women generally, more likely to use highly effective methods of contraception, and less likely to be in the top 50% of the population for numeracy. The majority of women were trying to avoid pregnancy.

Table 1.

Characteristics of Interviewed Women by Interview Round

	Round 1 (N=4)	Round 2 (N=7)	Round 3 (N=8)	Round 4 (N=7)	Total (N=26)
Average Age (Years)	30.5	30.7	29.1	27.9	29.3
Race/Ethnicity
White	3	4	4	4	15
Black	1	2	3	0	6
Asian	0	1	1	2	4
American Indian/Alaskan	0	0	0	1	1
Native
Any Children
Yes	2	2	3	2	9
No	2	5	5	5	17
Ever Pregnant
Yes	2	2	4	2	10
No	2	5	4	5	16
Method(s) Used in Past 3 Months
Vasectomy	0	1	0	0	1
Sterilization	0	1	1	0	2
IUD	2	1	3	4	10
Implant	1	0	0	0	1
Injectable	0	0	1	0	1
Pills	3	4	4	4	15
Patch	1	0	2	0	3
Ring	0	0	1	0	1
Male Condom	3	3	4	5	15
Withdrawal	0	2	2	3	7
Diaphragm	0	0	0	0	0
Sponge	0	0	0	0	0
Spermicide	0	0	0	1	1
Other	0	0	1	0	1
No Method	0	0	1	0	1
Pregnancy Intentions
Trying to avoid pregnancy	2	5	5	5	17
Ambivalent	1	1	0	1	3
Trying to get pregnant	0	0	2	1	3
Don’t know	1	1	1	0	3
Numeracy
Top 50%	1	4	4	2	11
Bottom 50%	3	3	4	5	15

Round 1 identified simple mistakes in the patient-centered poster, leading us to reach saturation quickly (N=4). In the other rounds, we reached saturation within 5–10 interviews, as is typical in cognitive interviewing studies [16]. In Round 4, participants made few suggestions for changes and preferred the patient-centered poster overall, leading us to conclude the study. Tables 2 and 3 show abbreviated versions of the overview report for each poster by round. Figure 2 shows the final iteration of the patient-centered poster.

Table 2.

Abbreviated Overview Report for the CDC Poster by Domain

Round Problem	1 (N=4) n*	2 (N=7) n*	3 (N=8) n*	4 (N=7) n*	Total (N=26) n* (% of N)	How Addressed by Patient-Centered Poster?
COMPREHENSION: Information and words/phrases on the poster are clear
Clinical terms (LAM, LNG IUD, nulli/parous, methods’ clinical names) are unclear.	1	7	7	6	16 (62%)	All removed and replaced with suggested non-clinical terms.
Asterisk is not seen before the percentages, making the percentages confusing.	3	2	5	2	12 (46%)	Percentage legend added at the top of the column.
“Typical use” is unclear.	3	4	3	2	12 (46%)	Term not used.
Pictures unclear for some methods.	2	2	4	2	10 (38%)	Suggested change.
“Permanent” is confusing when there is a small risk of pregnancy.	1	1			2 (8%)	“Surgical” used instead.
RELEVANCE: Information on poster applies to the participant and is useful for her
Missing how each method works and how long it lasts.	4	5	7	7	23 (88%)	Added information.
Missing side effects/contraindications.	1	2	3	5	11 (42%)	Suggests 2nd poster.
Women only interested in methods that are commonly used, relatively effective, and easy to obtain.		2	4	3	9 (35%)	Only shows commonly used methods.
“Family Planning” not inclusive and not equivalent to “birth control.”		3	2	2	7 (27%)	“Birth control” used.
Missing unprotected sex.	1		4	1	6 (23%)	Added this information.
ACCEPTABILITY: Information on poster seems truthful and inoffensive
Withdrawal picture is confusing or offensive.	1	3	4	2	10 (38%)	New withdrawal image.
Pictures of sterilization offensive.	1		1	2	4 (15%)	Suggested change.
Add number to call or website.		1	1	2	4 (15%)	Suggested change for sponsored projects with sponsor.
Add “Talk to your doctor.”		1	2	1	4 (15%)	Added.
Picture of injectable is off-putting.		1	1		2 (8%)	Removed needle.
DESIGN: Poster is attractive and well-organized
No color.	3	6	6	7	22 (85%)	In color.
Too much text/“looks” like a lot of text.	3	5	2	1	11 (42%)	Text is in separate column.
Not enough of a visual emphasis on STI prevention.	2	3	4	2	11 (42%)	Bolded and in main text.
Titles and headers are not noticed first.	1	3	3	2	9 (35%)	Title is in large font.
The order in which the information on the sheet should	1	3	2	3	9 (35%)	Order to read information is signaled

Table 3:

Abbreviated Overview Report for Patient-Centered Poster by Domain

	Round 1 (N = 4)		Round 2 (N = 7)		Round 3 (N = 8)		Round 4 (N = 7)
Problem	n*	Change	n*	Change	n*	Change	n*	Change
COMPREHENSION: Information and words/phrases on the poster are clear
Pregnancy risk measure confusing.	4	Title changed.	7	Added first-year failure percentages.	5	Replaced with “times better” scale.	3	No change; more prefer alternative.
Pictures unclear for some methods.	2	No change.	2	No change.	4	No change.	2	Suggested change.
Unclear what colors convey.			2	“No method” in different color from other methods.	2	Tried to reduce number of colors and intensity.	1	Surgical methods now same color.
Unclear what numbers mean.			2	Added explanation.			6	Header added.
Unclear reference/scale for arrow.			2	Added time for most effective.			4	Switched scale label to “times more effective”.
RELEVANCE: Information on poster applies to the participant and is useful for her
Missing how each method works and how long it lasts.	2	No change.	5	Done for all but least effective methods.	4	Added information on least effective methods.	3	Updated IUD information.
Missing side effects/contraindications.	1	No change.	3	No change.	3	No change.	5	Suggests 2nd poster.
Missing brand names.	1	No change.			2	Added.	2	Added.
Women only interested in methods that are commonly used, relatively effective, and easy to obtain.			1	Retained all contraceptive options from CDC poster.	4	Removed sponge, spermicide, and female condom.	1	Removed diaphragm.
Missing LAM information.			1	More prefer removal.	1	More prefer removal.	1	More prefer removal.
ACCEPTABILITY: Information on poster seems truthful and inoffensive
Feels less believable due to no logo, short citation, or poor design.	3	Added logo and citation.	3	Revised design.			1	No space for long citation.
“Advertising” most effective methods.	1	No change.	3	No red, yellow, green.			1	All methods in one color.
Add “Talk to your doctor.”			1	No change.	2	No change.	1	Done.
Pictures of sterilization offensive.	1	No change.			1	No change.	2	Suggested change.
Add number to call or website.			1	No change.	1	No change.	2	Suggested change.
DESIGN: Poster is attractive and well-organized
The order in which the sheet should be read is unclear/sheet is crowded.			3	Decreased font size in order of importance.	7	New layout. Fewer numbers.	1	Removed diaphragm and technical names.
More visual weight on STI prevention.	1	No change.	4	Bolded STI box.	3	Bolded in text.
Titles and headers are not noticed first.	2	No change.	2	Larger title font.			1	Headers added in blue.
More emphasis on surgical methods.			3	Retained box.	4	Separated methods.
Too much text.			1	No change.			4	Removed surgical text.

Only the top five most mentioned issues for each category are included.

^*n represents the number of women who brought up that issue.

Fig. 2:

Final iteration of the patient-centered contraceptive effectiveness poster.

In the interviews, the CDC logo was used in the Logo Space to ensure that the two posters were comparable. Recent research suggests that some long-acting reversible methods can be used for longer than they were initially approved for in some groups of women [22]; however, we use lengths of time for which these devices were approved for use by the U.S. Food and Drug Administration.

3.1. Comprehension

Technical language was the most common issue with comprehension. Women often did not know the meanings of lactational amenorrhea (62%), parous/nulliparous (54%), typical use (46%), the LNG IUD (23%), and hysteroscopic/laparascopic/abdominal sterilization (19%). As one woman said:

I think that some of it can read jargony. So it could be unapproachable. Not necessarily lying, but that it’s just too much.

Women often assumed that the information was not relevant to them if they did not understand a word, and few asked for clarification. This finding led us to incorporate women’s preferred terminology into the patient-centered poster. For example, women suggested adding brand names and using “having tubes tied” instead of “female sterilization.” As one woman said:

[The poster] broke down the names, the actual names that the doctor will usually use… When you’re not in the medical profession, and you say, “Can I get the injectable?” and the doctor says, “Depo,” and you’re like, “Is that same thing?”… You feel a distance between who you’re talking to. You want to feel the same. You don’t want to feel as though the doctor is superior.

When the poster used women’s terminology it not only reduced confusion, but also increased the relevance of the information.

Another consistent comprehension problem was confusion about the patient-centered poster’s average time-to-pregnancy measure. Women found it difficult to understand because it was unfamiliar and not how they thought about risk.

I think about birth control in a very immediate way. So it’s kind of hard to pick [a contraceptive method] out in terms of, 10 years, when would I want an unplanned pregnancy? That I find a little confusing when I think about this.

As a result, in the fourth round we transitioned to contextualizing the percent failure rate with a “times more effective” scale. Women preferred this scale to the CDC’s use of a percent failure rate alone and were confident and accurate when interpreting it in their own words. Of the five women with low numeracy in the fourth round, three preferred the “times more effective” scale, and one did not prefer either scale. In the final round, six out of seven women interviewed said the patient-centered poster was easier to understand than the CDC poster.

3.2. Relevance

An unexpected finding was that 27% of women felt the term “family planning” excluded single women or that it did not reflect their attitude when they chose contraception. One younger woman said:

I don’t think it should be “family planning” unless it was maybe for a couple.

Women preferred “birth control” or “contraception,” so we use these in the patient-centered poster.

To increase relevance, 35% of women suggested removing uncommonly used methods from the poster. One woman observed:

I don’t think I have ever met anybody who uses a sponge or a diaphragm.

In the final poster, we removed methods used by less than 0.2% of contraceptive users: the female condom, diaphragm, sponge, and spermicide [23].

When the poster provided information relevant to women’s options, it increased women’s perceived ability to process their contraceptive choices. As one woman said in Round 3:

[The patient-centered poster] is just more useful to me because it gives me more of what I need in terms of being able to make a decision about birth control. How often do I have to take it? What’s the length of time that it will be effective for me in terms of not getting pregnant? … Does it have hormones? … And how it’s used. So I think that information is presented here in a much easier digestible frame. It’s easier for me to understand.

In each round, an increasing proportion of women said the patient-centered poster provided more relevant information than the CDC poster. By the final round, six out of seven women said that it was the more useful and relevant poster.

3.3. Acceptability

Some women found the withdrawal (38%) and sterilization (15%) pictures unacceptable. One woman (W) brought her sister (S) to the interview (I), and they discussed the issue:

I: I noticed you laughed at the withdrawal picture.

W: Yes. Because I had to really look at it to see what was going on there. [laughs]

I: What do you think about that one?

W: It’s really detailed. Tell you the truth I don’t think it should be in there.

S: Some people don’t know. Especially if they’re young. And maybe their parents don’t talk about sex at all.

W: But my child. I’m thinking about my child.

S: It’s not for kids.

W: But if she be in the room with me, she’s going to see that.

Another participant said she cannot depict human genitalia when she teaches sexual education at her church. In response, we commissioned an artist to draw a less graphic image for withdrawal, which was preferred by the majority of women and raised no acceptability concerns.

Another acceptability problem for the patient-centered poster was women’s perception that it was advertising highly effective methods. Black women were especially concerned about this, due to the history of forced sterilization in their community. As one Black woman said:

I do get some under-the-current, subliminal messages in that the [least effective methods] are in red, and then the middle one is in yellow… I’ve done more study on forced sterilization, stuff like that with certain populations. It gives me the subliminal message that women like me shouldn’t look at these [least effective] methods. And it makes me feel like, would the doctor forced sterilize me? Or offer me that option versus these other options?

We reduced these concerns by replacing the term “sterilization” and putting all the contraceptive methods in the same color.

3.4. Design

In every round, women said the patient-centered poster was more attractive than the CDC poster because it used color. One woman said:

I would probably be more likely to read the colorful one [the patient-centered poster]. So I would like [the doctor] to have that because it’s going to draw my attention. I like that they are making this a priority and aren’t trying to just put the information out. They want you to look at it; they want you to pay attention to it. To me, it would tell me that my doctor cares about these things.

The colors also helped women digest the information. According to a woman in Round 4:

I like this because I like color-coding it this way. That way I’m actually looking at the entire row. I’m looking at the picture, the percentage, and then the information next to it… I like colors, and I feel they help direct the eyes.

Over all the rounds, 85% of women asked for the CDC sheet to be more colorful, and all seven women in the final round said the patient-centered poster was more attractive than the CDC poster.

3.5. Preference

Table 4 shows the preferences of women by comprehension, relevance, attractiveness/design, and overall preference. Women preferred the patient-centered poster overall compared to the CDC poster and rated it as being more comprehensible, relevant, and attractive. By the final round, the only unaddressed acceptability issue with the patient-centered poster is that it shows male genitalia.

Table 4:

Women’s Choice of Preferred Poster by Round for Each Domain

	Round 1 (N=4)		Round 2 (N=7)		Round 3 (N=8)		Round 4 (N=7)
	CDC	Patient-centered	CDC	Patient-centered	CDC	Patient-centered	CDC	Patient-centered
Comprehensible	3	1	3	4	5	3	1	6
Relevant	4	0	7	0	3	5	1	6
Attractive/Design	1	3	1	6	3	5	0	7
Overall Preference	3	1	4	3	3	5	1	5
Column Sum	11	5	15	13	14	18	3	24
Percent of Round’s Total	11/16 = 69%	5/16 = 31%	54%	46%	44%	56%	11%	89%

Row totals within rounds do not always sum to the same number because women sometimes refused to voice a preference.

4. Discussion

Both written materials and graphical aids can provide effective contraceptive education [3, 4], and there are many contraceptive posters already available [24, 25]. However, to our knowledge, there are no studies documenting the process of developing these posters. The unique contribution of our study is creating a contraceptive poster using a structured process for incorporating women’s feedback to ensure the poster is comprehensible, relevant, and acceptable for the majority of women. This process is in alignment with CDC and OPA guidelines for developing evidence-based educational materials [2].

Women preferred the final version of the patient-centered poster overall and on the dimensions that we measured: comprehension, relevance, and design. It also raised few remaining acceptability concerns, especially in comparison to the CDC poster. We had a number of findings that may be relevant to future projects designing reproductive educational materials. For example, we found that women are unfamiliar with several contraceptive terms, such as lactational amenorrhea and LNG IUD. Testing educational posters with the intended audience would likely uncover technical jargon and give designers the opportunity to clarify their language. We also found that women are more comfortable with educational materials that do not depict genitalia because they worried that children might see them before their parents are ready to discuss sex and sexuality with them. While it is important not to compromise educational value, posters may be disseminated more broadly if they have child-friendly images. We also found that color should not be underrated as a tool to help women process information and draw their attention. Practically all of the women we interviewed strongly preferred that posters be presented in color. Finally, we found that some women were uncomfortable when a poster seemed to “advertise” highly effective contraception. Women preferred a neutral approach when being given educational information.

This study has several limitations. First, our study may have limited generalizability to US women because all interviews were conducted in North Carolina. Second, many of our participants used highly effective methods of contraception, which may bias our findings. However, we intentionally included participants that were users of less effective contraceptive methods, racially and ethnically diverse, and low-scoring in numeracy.

This study highlights a number of areas for future research. Our results suggest that low-numeracy women may prefer a “times more effective” scale for communicating contraceptive effectiveness. This finding should be quantitatively tested in larger, nationally representative samples of women. Future studies might also incorporate the preferences of health care providers into educational posters, who have important insight about women’s potential misunderstandings and gaps in knowledge. Before implementing this poster in practice, future studies should also evaluate the impact of the patient-centered poster on contraceptive knowledge and reproductive health outcomes to ensure that the poster is accomplishing its intended goal. The poster should eventually be studied in a clinical setting, where it would actually be distributed.

When women underestimate the effectiveness of contraception or their risk of pregnancy with unprotected sex, they are less likely to use contraception [26–29]. Therefore, posters that clearly communicate contraceptive information could be a valuable tool to help achieve the Healthy People 2020 goal of reducing unplanned pregnancies [30]. The women in our study valued information about contraception and spoke highly of doctors and organizations increasing access to such knowledge.