Improving Patients’ Understanding of Terms and Phrases Commonly Used in Self-Reported Measures of Sexual Function

Abstract

Introduction

There is a significant gap in research regarding the readability and comprehension of existing sexual function measures. Patient-reported outcome measures may use terms not well understood by respondents with low literacy.

Aim

To test comprehension of words and phrases typically used in sexual function measures to improve validity for all individuals, including those with low literacy.

Methods

We recruited 20 men and 28 women for cognitive interviews on version 2.0 of the PROMIS Sexual Function and Satisfaction measures. We assessed participants’ reading level using the word reading subtest of the Wide Range Achievement Test (WRAT). Sixteen participants were classified as having low literacy.

Main Outcome Measures

In the first round of cognitive interviews, each survey item was reviewed by 5 or more people, at least 2 of whom had lower than a ninth-grade reading level (low literacy). Patient feedback was incorporated into a revised version of the items. In the second round of interviews, an additional 3 or more people (at least 1 with low literacy) reviewed each revised item.

Results

Participants with low literacy had difficulty comprehending terms such as aroused, orgasm, erection, ejaculation, incontinence, and vaginal penetration. Women across a range of literacy levels had difficulty with clinical terms like labia and clitoris. We modified unclear terms to include parenthetical descriptors or slang equivalents, which generally improved comprehension.

Conclusions

Common words and phrases used across measures of self-reported sexual function are not universally understood. Researchers should appreciate these misunderstandings as a potential source of error in studies using self-reported measures of sexual function.

Introduction

Low literacy is a widespread but neglected problem in the United States. The 2003 National Assessment of Adult Literacy found that almost half of the adult population experiences difficulty in using reading, speaking, writing, and computational skills in everyday life situations [1]. Health literacy, the ability of individuals to obtain and understand basic health information needed to make appropriate health decisions [2], may be more limited than functional literacy because of the unfamiliar context and vocabulary of the health care system [3]. Addressing health literacy is now recognized as critical to delivering person-centered health care [4].

As health care and research become more patient-centered, the need to directly capture patient experiences and perspectives through the use of valid and reliable patient-reported outcome (PRO) measures increases. Many individuals with chronic health conditions, such as diabetes and heart failure, experience sexual dysfunction as a side effect of the disease or its treatments [5]. In order to provide accurate, timely support for these health outcomes, it is imperative that tools are available that accurately assess this health domain. However, PRO measures may use terms not well understood by respondents with low literacy. The National Institute of Health's Patient-Reported Outcome Measurement Information System^® (PROMIS^®) was established to develop and validate measures of self-reported health across chronic conditions [6]. PROMIS guidelines specify that items must be written in a manner that accommodates participants with low literacy.

Except for recently developed measures such as the British National Survey of Sexual Attitudes and Lifestyles, limited research has been conducted examining the readability level and comprehension of sexual function measures [7]. Two studies of underserved patients with prostate cancer treated at clinics for low-wealth patients found that more than 50% were unable to understand the terms erection, vaginal intercourse, impotence, and sexual drive [8,9]. Results from one of these studies showed that understanding of these terms was significantly correlated with literacy level [9]. There is a significant gap in research regarding the readability and comprehension of existing sexual function measures.

We addressed this gap as part of the development of the PROMIS Sexual Function and Satisfaction measure (PROMIS SexFS). This measure includes multiple instruments covering different domains of sexual function; some of these are gender-specific (eg, Erectile Function and Vaginal Discomfort) while others are gender-neutral (eg, Interest in Sexual Activity). Modern measure development includes an explicit step to assess the readability and acceptability of items under consideration [10, 11]. We conducted cognitive interviews to evaluate candidate items for version 2.0 of the PROMIS SexFS that were generated on the basis of a conceptual measurement model [12] and focus groups with patients with a variety of chronic diseases [13]. Because understanding among persons with lower literacy was a concern, we included such persons in accordance with the PROMIS protocol for cognitive interviewing [11].

Methods

Item Development

The development of version 1.0 of the PROMIS SexFS measures has been described in detail elsewhere [12]. Version 1.0 focuses on cancer populations. Version 2.0 expands the measures to include other targeted health groups, such as patients with heart disease, diabetes, anxiety, and/or depression. The development of the PROMIS SexFS measures adhered to the guidelines established for all PROMIS measures.

PROMIS SexFS Measures

The candidate domains for version 2.0 of the PROMIS SexFS measures consisted of nearly 200 items divided into 14 domains. For this study, we focused on difficulties in 4 core domains: vaginal discomfort, vulvar discomfort, erectile function, and orgasm. The vaginal discomfort domain consists of items that measure the degree of physical discomfort of the vagina during sexual activity. The vulvar discomfort domain measures the degree of physical discomfort of the labia and clitoris during sexual activity. Items in the erectile function domain measure the frequency and quality of achieving and maintaining an erection for sexual activity. The orgasm domain measures the person's experience of climax (ie, frequency, timing, and/or quality) with and without a partner. One additional item for men asks about pain or burning during or after ejaculation.

Cognitive Interview Methods

Measures of sexual function, including the PROMIS measure, contain clinical terminology such as vaginal penetration, erection, orgasm, and labia. We tested these terms in cognitive interviews to assess how respondents comprehend survey questions and formulate responses.

Cognitive interviewing is a technique derived from the cognitive aspects of survey methodology, which describes the human cognitive response process as a 4-stage model: comprehension of the question, retrieval from memory of relevant information, judgment/estimation processes, and response processes [14, 15]. This qualitative method is useful to assess the face validity of item content and instructions [15]. We developed the guide used during the cognitive interviews by reviewing each candidate item using a modified version of the Question Appraisal System, a checklist-based review system that reflects potential sources of error that can occur in the administration of and response to a survey item [14]. We developed item-specific probes to test the occurrence of such errors.

For example, a female-specific item asks, “How often have you had difficulty with sexual activity because of discomfort in your vagina?” This item might be a case of Question Appraisal System category 4a, “inappropriate assumptions,” as an assumption is being made that the female respondent uses her vagina during sexual activity. An item-specific verbal probe for this question would ask the respondent, “How well does this question apply to you?” or “Do you use your vagina during sexual activity?” In addition to item-specific probes, interviewers were trained to incorporate unscripted verbal probes when appropriate.

Study Population

Participants in the cognitive interviews were recruited via flyers posted in outpatient clinics of the Duke University Health System and on the university's clinical trials website. Potential participants took part in a 10-minute telephone screening to confirm eligibility. Eligible participants were 18 years or older and able to speak and understand English. We aimed for diversity with respect to race/ethnicity, age, sexual orientation, and diagnosis of common diseases, including heart failure, diabetes, mood and anxiety disorders, and cancer. In keeping with PROMIS guidelines [11], at least 40% of participants had lower than a ninth-grade reading level as measured by educational attainment or the word reading subtest of the Wide Range Achievement Test (WRAT-4) [16]. (The WRAT-4 was administered at the end of the cognitive interview; participants were not excluded based on their presumed literacy level.) The institutional review board of the Duke University Health System approved the study.

Procedures

Each interview was conducted in person by a trained interviewer who was gender-matched to the participant. During the audio-recorded interviews, which lasted 45 to 60 minutes, participants gave informed consent and reviewed 16 to 32 items. For each item, participants read the item and selected the most applicable response option. Interviewers were trained to use concurrent verbal probing (ie, probing conducted immediately after an item is answered) to elicit from participants verbal feedback on each item regarding understanding, assumptions, sensitivity, and recall. See Table 1 for examples of item-specific probes.

Table 1.

Examples of Item-Specific Verbal Probes

Item	Domain	Understanding	Response Construction	Assumptions	Sensitivity
How often have you felt aroused (turned on) when thinking about or looking at erotic scenes or pictures or when reading erotic stories?	Subjective arousal	Check participant's understating of the words “erotic” and “materials”	If participant chose response options “0” or “1” find out if they didn't have opportunity or aren't aroused by these materials	[If older or if female] How well does this question apply to them?
How often have you had difficulty with sexual activity because of pain in your vagina?	Vaginal discomfort			For women who do not use their vagina to participate in sexual activity, does “never” apply, or do none of the response options apply
How much have memories of traumatic experiences affected your satisfaction with your sex life?	Interfering factors	Check participant's understanding of the term “traumatic experience”	How did participant come up with their response?		Is it okay to ask this question on a survey?
When you had sexual activity with a partner, how often have you felt that both of you and your partner were satisfied?”	Global satisfaction with sex life	Check participant understanding of the word “satisfied”.	The question asks about a partner's satisfaction. Is this an important part of their satisfaction?
How often have you had discomfort in your clitoris during sexual activity?	Vulvar discomfort	Check participant's understanding of the word “clitoris”
How difficult has it been for you to get an erection when you wanted to?	Erectile function		Check whether participant had the same experience each time they got an erection or if their experience varied each time? [If varied] how did they decide which answer to choose?

At least 5 participants reviewed each item, and each item was reviewed by at least 1 person per targeted health group and at least 2 people with low literacy. After an initial analysis of participant comments and item revisions (described below), a second round of interviews was conducted with new participants to evaluate revised items from the first round. Revised items were reviewed by a minimum of 3 additional participants including at least 1 with low literacy.

Analysis

The research team analyzed participant comments for each item set (question and response options) and used these comments to address issues with the item stem and response options. Upon an initial review, items were rated as acceptable, minor revision, substantially revise, or eliminate. If minor revisions were needed, they were made to incorporate the information gleaned in the cognitive interview without additional testing. For items needing substantial revision, the item set was revised and reviewed again in the second round of cognitive interviews.

Results

Of the 133 candidates screened between July 2011 and April 2012, 48 participants were selected purposively to reflect a diversity of chronic conditions, literacy levels, demographic characteristics (ie, sex, age, race, and ethnicity), and self-identified sexual orientation. Table 2 shows the demographic and clinical characteristics of study participants. Twenty men and 28 women ranged in age from 21 to 70 years. Half were of non-white race, 3 were of Hispanic or Latino ethnicity, and 6 self-identified as gay, lesbian, or bisexual. Sixteen participants were classified as having low literacy by WRAT score, which did not necessarily correspond to having lower than a ninth-grade education (see Table 3).

Table 2.

Sociodemographic and Clinical Characteristics of the Study Population

Characteristic	All Participants (n = 48)	Men (n = 20)	Women (n = 28)
Age, mean (range)	43.3 (21-70)	42.3 (21-65)	43.3 (24-70)
WRAT score, mean (range)	56.5 (39-70)	53.8 (29-69)	58.5 (39-70)
Low literacya, No. (%)	16 (33.3)	8 (40)	8 (28.6)
Education, No. (%)
Less than high school	12 (25)	7 (35)	5 (17.8)
High school diploma	7 (14.5)	2 (10)	5 (17.8)
Some college	8 (16.6)	3 (15)	5 (17.8)
Bachelor's degree	10 (20.8)	5 (25)	5 (17.8)
Advanced degree	11 (22.9)	3 (15)	8 (28.5)
Race, No. (%)
African American	23 (47.9)	11 (55)	12 (42.9)
Asian	1 (2.1)	0	1 (3.6)
White	22 (45.8)	7 (35)	15 (53.6)
Unknown	2 (4.2)	2 (10)	0
Lesbian, gay, or bisexual, No. (%)	6 (12.5)	3 (15)	3 (10.7)
Not sexually active, No. (%)	14 (29.2)	6 (30)	8 (28.6)
Health conditionb, No. (%)
Diabetes	12 (25)	8 (40)	4 (14.3)
Heart failure	5 (10.4)	3 (15)	2 (7.1)
Mood/anxiety disorders	29 (60.4)	11 (55)	18 (64.3)
Cancer	7 (14.5)	1 (5)	6 (21.4)
Other	8 (16.7)	2 (10)	3 (10.7)
None	6 (12.5)	5 (25)	1 (3.6)

^aParticipants with WRAT scores ≤ 54 are considered low literacy.

^bParticipants may have more than 1 health condition.

Table 3.

Educational Attainment by Literacy Level (n = 48)

Educational Attainment	WRAT Score, mean	Low Literacya, No. (%)
Less than high school	42.3	10 (83.3)
High school diploma	51.1	3 (42.9)
Some college	58.6	1 (12.5)
Bachelor's degree	64.3	1 (8.3)
Advanced degree	67.4	0 (0)

^aParticipants with WRAT scores ≤ 54 are considered low literacy.

High-Literacy Vocabulary

The term vaginal penetration is commonly found in measures of sexual function, especially in the domain of vaginal discomfort. One representative item asks, “In the past 30 days, how often did you have discomfort or pain after vaginal penetration?” Two participants with low literacy who reviewed this item incorrectly described vaginal penetration to mean “difficult” or “discharge.” This item was revised to include a parenthetical descriptor that stated that “vaginal penetration is when something is put inside your vagina.” A similar revised item was retested in an additional round of interviews, where 4 participants, including 1 with low literacy, demonstrated a clear understanding of the item.

The terms labia and clitoris occur frequently throughout the items in the PROMIS vulvar discomfort domain. One item asks, “In the past 30 days, how often have you had discomfort in your labia?” Participants with low literacy (n = 2), as well as participants with higher literacy (n = 5), expressed either a complete lack of knowledge or only slight familiarity with the term labia. Upon reading the item, 1 woman with higher literacy did not immediately recognize the term but knew what it referred to once explained by the interviewer. A participant with low literacy also needed the term explained by the interviewer and stated that she thought the term was referring to the anus. As a result of these comments, items using this term were revised and retested. The revised item included the parenthetical descriptor “lips around the opening of the vagina.” Participants who reviewed the revised item (n = 3) did not demonstrate any difficulty with understanding the term. Likewise, difficulties with the term clitoris were resolved by adding the parenthetical descriptor “clit.”

Several participants were observed to have difficulties comprehending items in the erectile function domain that contained the word erection. One representative item asks, “How difficult has it been for you to get an erection when you wanted to?” Eight men reviewed this item, 5 of whom had low literacy. Of the participants with low literacy, 1 understood the term erection to mean ejaculation, and 2 participants only understood the term when it was read to them by the interviewer. This item and other items that contained the term erection were revised to include forms of colloquial phrases (ie, “get hard,” “stay hard,” “been hard”).

The terms orgasm and climax appear together throughout the orgasm domain based on cognitive interview results from our earlier work suggesting both terms were needed to maximize comprehension for diverse participants [17]. However, difficulties in comprehension remained among several participants with low literacy. Items representative of this domain ask, “How often have your orgasms or climaxes been satisfying?” and “How often did it take too long to have an orgasm/climax?” Of the 3 participants with low literacy, 1 understood the terms orgasm and climax to mean getting aroused by their own thinking or masturbation. One male participant with low literacy thought orgasm/climax applied only to women. Table 34 describes additional examples.

Table 4.

Terms Commonly Found in Self-Reported Measures of Sexual Function

Term	Example Item	Participant Responses	Revised Item
Labia	How often have you had numbness of your labia during sexual activity?	“It [labia] is a medical term. I've never seen it before. I thought they were talking about the anus.”a	How often have you had discomfort in your labia (lips around the opening of the vagina) during sexual activity?
Vaginal penetration	How would you rate your level (degree) of discomfort or pain during or after vaginal penetration?	“‘Vaginal penetration,’ probably some type of discharge.”a	How would you rate your level (degree) of discomfort or pain during or after vaginal penetration? (Vaginal penetration is when something is put inside your vagina.)
Erection	How difficult has it been for you to get an erection when you wanted to?	Participant is misunderstanding erection to mean ejaculation.a	How difficult has it been for you to get an erection (get hard) when you wanted to? If you use any aids to help you get an erection (eg, pills, injections, or a penis pump), please answer this question thinking about the times that you used these aids.
Orgasm/climax	When you have tried to have an orgasm/climax, how often did it take too long?	Participant thinks orgasm/climax refers to only women.a	— b
Genitals	How often have you touched a partner's genitals?	Participant has to have the word genitals read to him to understand.	How often have you touched a partner's genitals (private parts)?
Hormone treatments	How often have you used hormone treatments for sexual activity? Do not include hormones used for contraceptive purposes (ie, birth control pill).	Hormone treatments are things that get you into the mood like a false penis, toy.a	— b

^aResponse is from a participant determined to have low literacy based on their WRAT-4 reading score.

^bItem remained unclear to at least 1 respondent after revision.

Disease-Specific Items

Several items in the measure were relevant to participants who had specific issues but were confusing to those who had not experienced those problems. For example, one item read, “How often has your vagina felt too small or too short during sexual activity (for example, feelings of stretching or pulling)?” Among women with heart disease, diabetes, depression, or anxiety, none who reviewed this item (n = 4, l low literacy) demonstrated a clear understanding of what was meant by their vagina feeling too “small” or “short.” However, among women with gynecologic cancer, the treatments for which can cause vaginal atrophy [18] or stenosis [19], each participant (n = 3) understood what the question was asking, with 2 participants having experienced the symptoms. Due to the small numbers of participants in each disease group in our sample, this issue warrants further investigation in other samples.

Frequency Items

During data collection, we observed problems with responses to frequency-type questions. These items queried participants on the frequency of specific symptoms experienced in the past 30 days. In one example, an item asks, “In the past 30 days, how often have you had any of the following during sexual activity: numbness, pain, irritation, or other discomfort in your clitoris (clit) or labia (lips around the opening of the vagina)?” Response options for this item included: have not had sexual activity in the past 30 days, never, rarely, sometimes, often, and always. One participant (not low literacy) experienced difficulties during response selection: “This question is hard to answer because I've only masturbated once in the past 30 days, but I guess I would have to say ‘rarely,’ which seems kind of strange since I've only done it [masturbated] once.” Since the item context is “in the past 30 days” and the frequency of her sexual activity was once during this period of time, the more accurate answer (if at least 1 symptom was experienced) would be “always.” However, if the participant is referencing sexual activity outside of the item context, in which symptoms are generally not experienced, then the participant may consider “rarely” to be the most accurate answer. To address this challenge would require the addition of items with a yes/no response set for participants who indicate a single sexual activity or encounter during the past 30 days. This strategy could work for electronic administration of items but would be impractical for paper administration and would complicate scoring.

Discussion

Data collected in this study show how participants with low literacy experience difficulties with vocabulary commonly used in measures of sexual function. Most participants had a good understanding of items throughout the 14 domains. Out of a total tested items, 112 items (62%) required no revisions and were understood by all participants. Fifty-nine items required revision and retesting to address misunderstanding exhibited by participants. Participants with low literacy had the greatest amount of difficulty with understanding terms in the interfering factors and therapeutic aids domains. In domains that included sexual function terms, participants with lower literacy demonstrated difficulties in understanding the vaginal discomfort, erectile function, and orgasm domains. Participants of all literacy levels demonstrated difficulties in understanding terms in the vulvar discomfort domain. Items were revised using colloquial terms and parenthetical descriptors to aid in comprehension across literacy levels. Comprehension generally improved once these terms and descriptors were provided.

Although revised items yielded a higher overall understanding of items, some misunderstanding still remained among participants with low literacy. The purpose of item revision and retesting is to yield the highest overall understanding across a wide range of literacy levels. However, if misunderstanding still exists after revisions, respondents may require more elaborate efforts such as the use of graphics, illustrations, or three-dimensional models, when feasible.

Results from this study support the recommendations for providers found in clinical practice guidelines to meet patient needs [20], such as speaking with patients using clear, plain terms and checking patients’ understanding of the terms used (ie, the teach-back method [21]). Providers should also be prepared to provide additional information that may improve clarity if necessary. The research implications of this study suggest the importance of including participants from a range of educational and literacy levels during the development of PRO measures. Furthermore, researchers should ascertain whether individuals with low literacy are able to understand and respond accurately to items in self-report measures.

Our study has limitations. First, the study population is limited in its representation due to its small sample size. Second, all participants were from a single geographic region (Durham, North Carolina). Future studies should be conducted using samples with greater geographic diversity.

Conclusions

The use of cognitive interviewing was a critical step in the development and refinement of the PROMIS SexFS, providing evidence for the content validity of these self-reported measures. Attention to literacy demands is not routinely considered during PRO item development, yet is important to consider for both research and clinical reasons. Failure to include participants with low literacy in the measure development process could ultimately result in measurement error and/or missing data. The use of colloquial terms and parenthetical citations can successfully aid respondents in the comprehension of common terms used in measures of sexual function but may not eliminate all difficulties associated with low literacy.