Initial Development and Content Validation of a Health-Related Symptom Index for Persons either Treated or Monitored for Anal High-Grade Squamous Intraepithelial Lesions (HSIL): AMC-A02

Abstract

Objectives

Anal cancer, caused by oncogenic types of human papillomavirus, is a growing problem in the United States. A key focus of anal cancer prevention has been screening for and treating precancerous high-grade squamous intraepithelial anal lesions (HSIL). Since anal HSIL and its treatment may negatively impact health-related quality of life (HRQoL), and no HRQoL measure specific to this condition and treatment currently exists, we used qualitative techniques to develop such an instrument.

Methods

Expert consultation was used to guide one-on-one concept elicitation interviews with participants to identify HRQoL aspects they attribute to their anal HSIL and its treatment. This resulted in a draft instrument, which was administered to an independent participant sample, where cognitive interview techniques assessed comprehension.

Results

Eighteen anal HSIL-related concepts were identified by the expert panel. Across the 41 concept elicitation interviews, 23 items representing physical symptoms, physical impacts, and psychological symptoms were identified to comprise the initial measure, which was then evaluated during three rounds of cognitive interviews (n=45). Several questionnaire aspects were refined based on participant input, with three additional items added per expert/participant recommendation. One item was removed due to poor comprehension, resulting in a 25-item measure.

Conclusions

Using state-of-the-art qualitative methodology, we have established the content validity of this new instrument, the ANCHOR Anal HSIL Health-Related Symptom Index (A-HRSI). Quantitative validation efforts are currently underway. The participant-driven process of developing this tool will facilitate a participant-centered evaluation of the impact on morbidity for treatment of anal HSIL or observation without treatment.

Introduction

Anal cancer is a growing problem in the United States (1), with the incidence rising in the most common type of anal cancer, squamous cell carcinoma, from 1992-2011 (2). In the U.S. general population, the incidence of anal cancer from 2009-2013 was 1.8/100,000 among men and women (3), but there is a markedly higher incidence among subpopulations. The cumulative incidence of anal cancer among HIV-infected adults was reported to be 1.5% by age 75, compared with 0.5% among HIV-uninfected adults (4). Human papillomavirus (HPV) infection is causally associated with the development of anal cancer (2). Anal high-grade squamous intraepithelial lesions (HSIL), the precursor lesion to anal cancer (5), are also associated with persistent HPV infection (6).

HPV vaccination as primary prevention may reduce anal cancer in the long term, but this approach is limited by suboptimal uptake of vaccination, nonadherence to vaccination series among targeted populations, and the large number of persons already infected with HPV (7). Similar to programs to reduce the incidence of cervical cancer, secondary prevention of anal cancer includes screening for and treating anal HSIL prior to progression to anal cancer (8). High resolution anoscopy (HRA) is used to visually identify areas of possible anal HSIL and allow for target biopsy to confirm the lesion histologically. Treatments for anal HSIL include ablative procedures such as infrared coagulation, electrocautery and laser, surgical excision under anesthesia, and the use of topical antineoplastic agents, such as 85% trichloroacetic acid and 5% fluorouracil cream (9-14).

Treatment of anal HSIL is increasingly being performed to reduce the risk of anal cancer, even as the effectiveness of doing so is not yet known. It is important to assess physical symptoms and concerns related to health-related quality of life (HRQoL) in those diagnosed with and either in surveillance or treated for anal HSIL, as these may be important considerations in treatment decision-making for participants and their care providers. There is a paucity of data on symptoms and concerns related to the HRQoL of persons treated for anal HSIL, and the published studies are descriptive and without control groups. In one longitudinal study, 37 persons diagnosed with anal HSIL were treated with surgical excision and followed every 3-6 months to examine the safety and efficacy of HRA and surgical treatment (15). The findings highlighted the importance of assessing pain associated with bowel movements and anal sexual functioning.

A second study examined psychological symptoms associated with anal cancer screening via HRA in 104 HIV-positive men who have sex with men in primary care and HIV clinics in Canada (16). Younger participants and those who reported more symptoms and higher psychological distress scores at baseline were more likely to experience higher levels of negative psychological impact from screening. Finally, a third study showed that different anal HSIL treatments (topical imiquimod, topical fluorouracil, or electrocautery) produced different patterns of symptoms lasting different lengths of time (17). Participants in the electrocautery group were more likely to report anxiety or depression and were less satisfied with their overall sex life at week 16 than participants in the topical treatment groups.

These studies used diverse measures to assess HRQoL, ranging from study-specific ad hoc items (15), symptom instruments designed for HIV-positive persons but not specific to anal HSIL (16), and generic HRQoL measures (17), supplemented with validated sexual functioning items (16), or psychological distress measures (16). Although many of these measures showed sensitivity to the impact of anal cancer screening or treatment, the variety of tools used and lack of specificity to anal HSIL diagnosis and treatment indicate the need for a rigorously developed and validated instrument to provide reliable, comparable data across anal HSIL populations and treatments. As different treatments are used for anal HSIL, it is important to develop a HRQoL measure that can capture the array and severity of symptoms from diverse treatments and including active surveillance without treatment.

In 2015, the United States National Cancer Institute funded the Anal Cancer/HSIL Outcomes Research (ANCHOR) study, a phase III clinical trial conducted within the AIDS Malignancy Consortium (AMC) to determine if the treatment of anal HSIL, compared with active monitoring, can prevent the development of anal cancer in persons living with HIV infection. Given the absence of a HRQoL measure specific to anal HSIL, the ANCHOR study presents a unique opportunity to understand the symptoms and experiences of persons being treated or actively monitored for anal HSIL.

The purpose of the present study is to describe the initial steps of developing a health-related symptom index for HSIL for use in the ANCHOR trial (ANCHOR HRSI; A-HRSI), including three separate phases: 1) solicitation of disease-related concepts from an expert panel of clinicians in the AMC who have worked extensively with HSIL in this participant population; 2) qualitative elicitation of disease-related concepts from participants diagnosed with anal HSIL and either in surveillance or being treated for anal HSIL to inform development of a draft version of the A-HRSI; and 3) the use of one-on-one cognitive interviewing techniques in a separate cohort of participants in surveillance or being treated for anal HSIL to establish the content validity of the draft A-HRSI.

Methods

Participants

Six members of the AMC who are also investigators and clinicians in the ANCHOR trial, each with a minimum of a decade of experience diagnosing and treating anal HSIL, were asked to comprise an expert panel for this study. A cohort of HIV-positive participants who were diagnosed with anal HSIL in the last nine months and either treated or actively monitored were recruited from the ANCHOR/AMC sites (Laser Surgery Care Center, Montefiore Medical Center, and Weill Cornell Medicine [New York, NY], Anal Dysplasia Clinic-Midwest [Chicago, IL], and the University of California-San Francisco). Providers from these sites provided a study information sheet and encouraged potential participants to contact research staff at Memorial Sloan Kettering Cancer Center. This referral included a referral identification number and coded information on the type of treatment being used (or surveillance) and the volume of disease. For all participants who contacted study staff and arranged for either an in-person or telephone-based interview, the Research Study Assistant (RSA) reviewed the study, solicited questions, and confirmed willingness to participate; written or verbal informed consent was not required or collected. For those who agreed to participate, the RSA collected age, gender, education, race, and ethnicity information for description of the study sample. Study RSAs were trained to ensure interview consistency across participants. In consultation with the AMC’s expert clinicians who diagnose and treat anal HSIL, we learned that not only treatment status (treated or monitored) and modality of treatment can impact symptom report, but volume of disease can as well. A larger volume of disease may mean more extensive treatment and recovery process, leading to prolonged or more severe physical and psychological symptoms. As such, we incorporated low versus high volume of disease as a parameter of recruitment of the participants for this study to ensure that both levels of disease volume were represented. Independent sample sizes of at least 40 participants were planned for both the concept eliciation and cognitive interviewing phases to adequately achieve saturation of concept (i.e., the point at which no novel information was being captured from participants) (18, 19). Because no protected health information was collected at referring sites, the study was reviewed and deemed as exempt research by the NCI’s Cancer Therapy Evaluation Program and the Institutional Review Boards at each study site.

Procedure – Phase I (Expert Consultation)

Members of the expert panel were provided with a preliminary list of commonly reported symptoms, concerns and other HRQoL areas derived from an earlier study that utilized qualitative interviews with persons diagnosed with and treated for anal HSIL (20). On the basis of their clinical experience, each expert was asked via email to review the list independently and comment on, edit, add or remove any symptoms, concerns and/or HRQoL areas that were clinically relevant and important to participants with anal HSIL. They were also asked to use 0-10 numeric rating scales (NRS) to indicate the importance of the symptom or concern for HRQoL (0=Not at all important, 10=Extremely important), how frequently they observed the symptom or concern (0= Rarely/not at all, 5=About half the time, 10=Most or all of the time), and observed duration of the symptom or concern (0=Less than one week, 5=1-4 weeks, 10=1 month or more). Two conference calls were planned for discussion of the aggregated expert panel feedback to establish consensus.

Phase II (Concept Elicitation)

To facilitate recruitment, eligible participants were interviewed either in-person or via telephone to establish participant-elicited concepts with respect to which symptoms, concerns and HRQoL impacts related to anal HSIL treatment or surveillance had the most subjective relevance and importance. Initially, the RSA asked participants to spontaneously name any symptoms, concerns or HRQoL impacts that they attributed to treatment or surveillance. The RSA then asked participants whether they experienced any of the symptoms or HRQoL impacts from the expert list that were not mentioned in the initial portion of the interview. Lastly, participants were asked to rate the degree to which a particular concept was bothersome or difficult using a 0-10 NRS. Data from phases I and II were used to develop a draft version of the A-HRSI that was presented to the expert panel on a conference call for consensus approval prior to the start of Phase III.

Phase III (Cognitive Interviewing)

Cognitive interviews were conducted in-person or via telephone in an independent cohort of eligible participants to document how participants diagnosed with anal HSIL and either in surveilance only or treated comprehended the concepts included in the draft A-HRSI, and thus establish content validity. The interviews also helped assess the overall feasibility of the instrument, including adequacy of instructions and the response scale (21). Cognitive interviewing is an iterative process with questionnaire modifications made when they are identified as problematic by three or more participants and then tested in an indpendent sample from the same cohort.

For each interview, the participant was asked to complete the draft A-HRSI while being observed by the RSA, but without assistance from the RSA or anyone else. For those interviews that took place via telephone, the participant was sent the draft A-HRSI at least one week in advance of the interview in a sealed packet and was asked to not open the questionnaire until they were instructed via telephone. Participants were instructed to indicate whether any of the items were difficult to understand by selecting a checkbox to the right of the corresponding item. For the in-person interviews, the RSA made note of participant behavior while completing the draft A-HRSI, such as additional time spent on a given item. For telephone interviews, the RSA instructed the participant to indicate when they completed each of the three A-HRSI sections and noted overall time spent completing the instrument. The RSA reviewed the completed responses and then initiated a semi-structured interview that included a discussion of any items that the participant indicated as being difficult to understand, as well as any items the RSA observed the participant to have taken additional time to complete. The participant was then asked specific questions raised by the expert panel during the measure drafting phase, followed by probes about clarity of instructions and the response scale, and general comprehensiveness.

Analytic Approach

All interviews were audio-recorded to assist the RSA with completing summary reports. Summary forms from the participants interviewed in the concept elicitation phase were coded to determine which symptoms, concerns, or HRQoL impacts were attributed by participants to anal HSIL treatment or surveillance. Total types of participant expression of each concept, total numbers of unique patient concept expressions within a given domain, number of participants expressing a concept spontaneously without being probed from the expert-derived list, as well as 0-10 NRS participant ratings of how bothersome or difficult the concept was perceived to be were included in the summary report. This list was then reviewed by the expert clinician panel to finalize the content for the initial draft of the A-HRSI. Any outstanding questions to be asked of participants during the cognitive interviews were then identified, along with appropriate question format, response options, and item recall period.

Summary reports from each cognitive interview were coded to determine instances of participant problems and/or difficulties. For each aspect of the measure that had at least three instances of being identified by participants as being problematic, that aspect was to be revised for testing in the second round of cognitive interviewing. This process was repeated after the second round of cognitive interviewing, with continuing problematic items or aspects of the measure to be re-tested in a third and final round of cognitive interviewing. If any aspect or item of the measure had at least three instances of being identified as problematic by participants after the third round of cognitive interviewing the item was removed from the final instrument.

Results

Phase I (Expert Consultation)

The six AMC members who were originally approached agreed to serve as members of our expert panel. They were asked via email to review the initial list of 13 items identified by persons diagnosed with or treated for anal HSIL (20). Consensus was reached during two conference calls to select a total of 19 items representing three domains: physical symptoms, physical impacts, psychological symptoms.

Phase II (Concept Elicitation)

Using the information from the expert consultation in Phase I, an independent cohort of 41 participants (Table 1; mean age = 49.2 years, 12% female, 68% non-white, 27% Hispanic, 29% high school or less education) being monitored without treatment for diagnosed anal HSIL or treated for anal HSIL were interviewed either in-person (n=20) or via telephone (n=21). Participant-elicited concepts were coded into the three domains established during Phase I (i.e., physical symptoms, physical impacts and psychological symptoms (Table 2)). No differences were found between concepts identified by participants with respect to their treatment modality, (i.e., monitoring without treatment or treatment by ablation, topical medication, or surgery).

Table 1.

Participant Demographic and Clinical Characteristics by Study Phase

	Concept Elicitation		Cognitive Interview (Phase 2)						Total
	(Phase I) n = 41		Round 1 n = 15		Round 2 n = 12		Round 3 n = 17		N = 85
Age
Mean (SD)	49.2 (11.3)		44.2 (7.9)		49.0 (7.6)		52.8 (8.4)		49.1 (10.0)
Median (Range)	50 (22–66)		47 (29–54)		50 (35–59)		52 (42–68)		50 (22–68)
Gender
Female - Cisgender	5	12.2%	1	6.7%	2	16.7%	3	17.6%	11	12.9%
Female - Transgender	0	0.0%	1	6.7%	1	8.3%	0	0.0%	2	2.4%
Male - Cisgender	36	87.8%	13	86.7%	9	75.0%	14	82.4%	72	84.7%
Male - Transgender	0	0.0%	0	0.0%	0	0.0%	0	0.0%	0	0.0%
Ethnicity
Hispanic/Latino/Spanish	11	26.8%	2	13.3%	3	25.0%	4	23.5%	20	23.5%
Other	28	68.3%	11	73.3%	9	75.0%	13	76.5%	61	71.8%
No Answer	2	4.9%	2	13.3%	0	0.0%	0	0.0%	4	4.7%
Race
White	13	31.7%	4	26.7%	2	16.7%	4	23.5%	23	27.1%
Black or  African American	16	39.0%	7	46.7%	8	66.7%	8	47.1%	39	45.9%
Other	7	17.1%	2	13.3%	2	16.6%	5	29.4%	16	18.8%
Declined to answer	5	12.2%	2	13.3%	0	0.0%	0	0.0%	7	8.2%
Education
High School or Less	12	29.2%	5	33.3%	6	50.0%	8	47.1%	31	36.4%
Some College	11	26.8%	8	53.3%	3	25.0%	6	35.3%	28	32.9%
College Graduate	11	26.8%	0	0.0%	1	8.3%	2	11.8%	14	16.5%
Graduate/Professional	6	14.6%	1	6.7%	2	16.7%	1	5.9%	10	11.8%
Declined to answer	1	2.4%	1	6.7%	0	0.0%	0	0.0%	2	2.4%
Treatment Modality
Observation	5	12.2%	2	13.3%	2	16.7%	4	23.5%	13	15.3%
Ablation	27	65.9%	13	86.7%	10	83.3%	4	23.5%	54	63.5%
Topical	5	12.2%	0	0.0%	0	0.0%	5	29.4%	10	11.8%
Surgery	4	9.8%	0	0.0%	0	0.0%	4	23.5%	8	9.4%
Lesion Volume
High Grade (>50%)	8	19.5%	2	13.3%	2	16.7%	4	23.5%	16	18.8%
Low Grade (≤50%)	33	80.5%	13	86.7%	10	83.3%	13	76.5%	69	81.2%
Mode of Interview
In-person	20	48.8%	4	26.7%	2	16.7%	2	11.8%	28	32.9%
Telephone	21	51.2%	11	73.3%	10	83.3%	15	88.2%	57	67.1%

Table 2.

Concepts Identified via Concept Elicitation Interviews by Domain

Domain	Concept	Predominance in Coding			Bother/Difficulty
		Total Types of Patient Expressions of Concept	Total # of Patient Concept Expressions	% of Patients Expressing Concept Spontaneously	Mean Bothersome Rating (0–10)	Mean Difficulty Rating (0–10)
Physical Symptoms
Anal pain and discomfort
	Anal pain from bowel movements	2	34	65.9%	7.7	–
	Anal pain	4	26	48.8%	7.4	–
	Anal pain from sexual activity	4	12	14.6%	7.3	–
	Anal discomfort	6	6	14.6%	7.3	–
Non-anal pain and discomfort
	Pain other than anal pain	4	3	7.3%	8.3	–
	Discomfort other than anal discomfort	3	4	4.9%	5.8	–
Bowel and Bladder Symptoms:
	Problems with bowels	3	13	7.3%	7.2	–
	Constipation	4	21	14.6%	6.4	–
	Problems with bladder	1	1	2.4%	10.0	–
Bleeding from anus
	Bleeding from anus	3	30	5.3%	7.5	–
	Bleeding from bowel movements	4	10	9.8%	7.4	–
Anal itching and dryness
	Itching on or around anus	3	19	4.9%	6.2	–
	Dryness on or around anus	2	1	2.4%	10.0	–
Burning and stinging
	Burning sensations in anal area	5	8	19.5%	6.8	–
Discharge/wetness
	Discharge/wetness in anal area	4	8	17.1%	6.4	–
	Discharge/wetness from fissure	2	2	4.9%	8.5	–
External tissue features
	External tissue features	8	11	17.1%	6.4	–
OTHER Physical Symptoms
	GI symptoms	4	4	9.8%	5.3	–
	Loss of energy	3	3	7.3%	8.7	–
	Muscle contractions	2	2	4.9%	7.5	–
	Other physical symptoms	3	3	7.3%	7.3	–
Physical Impacts
Problems moving around
	Problems with moving around	4	16	12.2%	–	7.3
	Problems with exercise activities	4	10	2.4%	–	7.8
Problems with sitting
	Problems with sitting	2	2	4.9%	–	9.5
Problems with daily household chores
	Problems with household chores	5	18	9.8%	–	6.8
	Problems completing chores independently	1	1	2.4%	–	10.0
	Problems with caretaking	1	1	2.4%	–	10.0
Problems keeping a healthy lifestyle
	Problems with self-care	2	11	4.9%	–	7.5
	Problems with diet/nutrition	3	3	7.3%	–	7.3
	Problems with sleeping	2	2	4.9%	–	8.0
Social activities
	Problems with social activities	5	23	22.0%	–	6.8
Work productivity
	Problems with work productivity	5	14	9.8%	–	6.4
Leisure activities
	Problems with leisure activities	2	15	9.8%	–	6.7
Psychological Symptoms
Difficulty concentrating
	Difficulty concentrating	1	8	2.4%	–	6.4
	Confusion	1	1	2.4%	–	8.0
Problems with intimate relationships
	Problems with intimate relationships	9	23	22.0%	–	7.7
	Problems with sex life	4	12	9.8%	–	6.8
Problems with anal sexual activity
	Decreased enjoyment of anal sex	2	8	2.4%	–	8.0
	Decreased desire for anal sex	2	15	2.4%	–	7.7
	Problems with anal sex	2	2	0.0%	–	8.0
	Decreased enjoyment of forms of sexual activity other than anal sexual activity	1	6	0.0%	–	6.7
	Decreased desire for forms of sexual activity other than anal sexual activity	1	10	0.0%	–	7.0
Anxiety and worry
	Anxiety	7	18	14.6%	–	7.7
	Anxiety about condition	5	5	12.2%	–	9.7
Depression
	Depression	3	14	9.8%	–	7.9
Emotional impact
	Emotional impact	2	28	19.5%	–	7.8

Note: Total types of patient expression of concept represents the number of unique instances in which a patient referred to a given concept. Total number of patient concept expressions refers to the actual total number of instances that a given concept was expressed by patients.

The Physical Symptoms domain comprised nine conceptual clusters (i.e., anal pain and discomfort, non-anal pain and discomfort, bowel and bladder symptoms, bleeding from anus, anal itching and dryness, burning and stinging, discharge/wetness, external tissue features, and other physical symptoms). Of these, anal pain from bowel movements (65.9%), anal pain (48.8%), burning sensations in the anal area (19.5%), discharge or wetness in the anal area (17.1%) and external tissue features (17.1%) were the concepts most frequently mentioned spontaneously by participants. When asked to rate the degree to which they were bothered by each identified concept on a 0-10 NRS, participants were most bothered by itching on or around the anus (M=10.0), problems with their bladder (M=10.0), loss of energy (M=8.7), discharge or wetness from the anal fissure (M=8.5), and pain other than anal pain (M=8.3).

A total of seven conceptual clusters comprised the Physical Impacts domain (i.e., moving around, sitting, daily household chores, keeping a healthy lifestyle, social activities, work productivity, leisure activities). Participants spontaneously offered being impacted by problems with social activities (22.0%), moving around (12.2%), completing household chores (9.8%), work productivity (9.8%), and with leisure activities (9.8%). Participants were asked to rate the degree to which any of the identified physical impacts made their life difficult on a 0-10 NRS, with problems completing chores (M=10), taking care of themselves (M=10), sitting (M=9.5), sleeping (M=8.0), and exercise activities (M=7.8) identified as being most problematic.

The Psychological Symptoms domain comprised seven conceptual clusters: difficulty concentrating, problems with intimate relationships, problems with desire or enjoyment of anal or other forms of sexual activity, anxiety and worry, depression, and emotional impact. When asked to spontaneously identify concepts, participants most frequently identified problems with intimate relationships (22.0%), emotional impact (19.5%), anxiety (14.6%), anxiety about their condition (12.2%), problems with their sex life (9.8%) and depression (9.8%). Participant ratings of conceptual impact on a 0-10 NRS indicated that anxiety about condition (M=9.7), decreased enjoyment of anal sex (M=8.0), problems with anal sex (M=8.0), confusion (M=8.0), and depression (M=7.9) were most problematic.

Concepts within the three domains were used to develop 23 unique items, which were then reviewed by the expert panel of clinicians to finalize content, question format, response option, and item recall period. This process resulted in a draft version of the A-HRSI that included 10 items to represent physical symptoms, six items indicative of physical impacts, and seven items to capture psychological symptoms. Through consutlation with the expert panel, a 7-day recall period was selected for all items to best approximate symptoms or impacts experienced within a given week. Item response scale was selected based to mimic the Functional Assessment of Cancer Therapy – General (FACT-G (22)), with the following NRS options: 0 (not at all), 1 (a little bit), 2 (somewhat), 3 (quite a bit), and 4(very much).

Phase III (Cognitive Interviewing)

Three rounds of cognitive interviews were completed with an independent cohort of 44 participants (Table 1; data for all rounds: mean age = 48.7 years, 18% female, 77% non-white, 20% Hispanic, 43% high school or less education) actively monitored for or having been treated for anal HSIL (23). A total of 15 participants completed the initial round of cognitive interviewing in-person (n=4) or via telephone (n=11), with specific problems identified regarding the meaning of several items (Table 3).

Table 3.

Summary Tracking Matrix Showing A-HRSI Modified Terms and Reasons for Modifications

Domain	Initial version	Current version	Reason(s) for the modification
Physical Symptoms
	I have pain	I have pain other than anal pain	Contextual information added per participant recommendation in Round 1 to differentiate general pain from anal pain.
	I have itching on or around the anus	I have itching in or around the anus	Participants suggested wording change during Round 1 to improve comprehension.
	I have external tags outside my anus	–	Item dropped from A-HRSI due to poor comprehension across three rounds of cognitive interviewing. Concept not amenable to participant reporting.
	I have discharge/seeping/wetness/secretion in my anal area	I have discharge (wetness) in my anal area	Participants preferred discharge (wetness) for this concept when probed during Round 1.
Physical Impacts
	I have problems completing daily chores at home (cleaning, cooking, laundry, house maintenance)	I have problems completing daily household chores (e.g., cleaning, cooking, laundry, house maintenance)	Participants preferred the use of “daily household chores” rather than “daily chores at home” for this concept in Round 1.
	I have problems participating in leisure activities (entertainment, relaxing)	I have problems participating in leisure activities (e.g., watching television, relaxing)	Participants preferred the use of “watching television” when describing leisure activities rather than the broad “entertainment” term during Round 1.
	–	I have problems with sitting	Item added after completion of Round 3 per patient recommendation.
Psychological Symptoms
	I enjoy sex less	I have a decreased enjoyment of anal sexual activity	Participants preferred the concept of “decreased enjoyment” during Round 1.
	–	I have a decreased enjoyment of forms of sexual activity other than anal sexual activity	Item added after completion of Round 1 per patient recommendation.
	I have a decreased desire for anal sex	I have a decreased desire for anal sexual activity	Participants preferred that the concept of “sex” be referred to as “sexual activity” during Round 1.
	I have decreased desire for forms of sexual activity (other than anal sex)	I have a decreased desire for forms of sexual activity other than anal sexual activity	Participants preferred that the concept of “sex” be referred to as “sexual activity” during Round 1.
	I have problems with my romantic relationships	I have problems with my intimate relationships	Participants preferred the use of “intimate” rather than “romantic” to describe their relationships as part of Round 1.
	–	I am worried about my condition getting worse	Item added after completion of Round 3 per expert clinician panel recommendation.

The item regarding “external skin tags outside my anus,” was especially problematic. First, participants preferrred the use of “skin tags outside my anus” for this concept. Additionally, participants indicated difficulty in comprehending the concept of “skin tags,” confusing them with bumps or marks, anal warts, moles, or hemorrhoids. The discharge/seeping/wetness/secretion item was changed to “discharge (wetness)” based on participant preference. Participants also suggested the addition of a “decreased enjoyment of forms of sexual activity other than anal sexual activity” item and recommended that the “I have pain,” item be changed to “I have pain other than anal pain,” to eliminate confusion between the two pain sources. With respect to formatting of the A-HRSI, participants preferred that the “Check if Not Applicable,” column was greyed out, with the exception of the “work productivity,” and “sexual activity” items.

Changes were made to the A-HRSI based on participant-identified concerns from round 1, and a revised A-HRSI was administered as part of a second round of cognitive interviewing with 12 participants. Four participants were unclear about the term “anal HSIL,” in the instructions for the A-HRSI. This concept was amended to include a “pre-cancer lesion in the anus,” definition to improve participant understanding. Participant comprehension problems persisted for the “skin tags outside my anus” concept; this item was again amended to “skin tags (small flap of tissue that hangs off the skin – not a hemorrhoid or wart) outside my anus,” to help clarify this concept for participants.

Changes from round 2 were tested in a third and final round of cognitive interviewing (n=17), where it was determined that the “skin tags outside my anus” item would be removed due to an overall lack of participant understanding across all three rounds of testing. During this third round, participants suggested the inclusion of a “I have problems with sitting” item. Across all rounds, there were no participant reports of issues with the 7-day recall period or the response scale. As with Phase II, no differences in comprehension were found between cognitive interview participants with respect to their treatment modality.

At the conclusion of round 3, the clinician expert panel reviewed the modified A-HRSI and added an “I am worried about my condition getting worse” item to the Psychological Symptoms scale. This resulted in a final, content valid A-HRSI comprised of 25 items: 9 items to assess physical symptoms, 7 items to assess physical impacts, and 9 items to assess psychological symptoms (Table 4).

Table 4.

Final Stem Wording for A-HRSI Items by Domain

Physical Symptoms	Psychological Symptoms
Bleeding from the anus	Anxiety
Burning sensations in the anal area Constipation	Decreased desire for anal sexual activity Decreased desire for forms of sexual activity other than anal sexual activity
Discharge (wetness) in my anal area Itching in or around the anus	Decreased enjoyment of anal sexual activity Decreased enjoyment of forms of sexual activity other than anal sexual activity
Pain (Anal)	Depression
Pain (During bowel movements)	Difficulty Concentrating
Pain (Other than anal pain)	Problems with intimate relationships
Urgency for bowel movements	Worried about condition getting worse
Physical Impacts
Problems with completing daily household chores (e.g., cleaning, cooking, laundry, house maintenance)
Problems participating in leisure activities (e.g., watching television, relaxing)
Problems participating in social activities (e.g., going out to eat, visiting friends)
Problems taking care of myself (e.g., bathing, dressing, shaving)
Problems with my physical ability to move around
Problems with sitting
Problems with work productivity

Discussion

While a number of established methods exist for treating or actively monitoring anal HSIL, there is currently no valid instrument to accurately capture the quality of life impact of these methods from patients. Consistent with best practices in measure development (23, 24), we have used patient-centered methods in a multi-step process to develop a content-valid health-related symptom index for patients diagnosed with and either treated or monitored for anal HSIL.

Capturing the patient voice via qualitative techniques is critical in the development of novel instruments to assess health-related symptom impacts. Regardless of the level of expertise that a clinical team may have, patients have firsthand knowledge of disease-related symptoms and impacts; this information should be elicited directly to be fully understood as part of the complete picture of the patient experience. Our study provided several examples of the importance of qualitatively capturing the patient experience. The expert panel had posited that “external skin tags outside my anus” would be an important concept to capture from patients; however, despite changes being made across three rounds of cognitive interviews, this item was determined to be not amenable to patient reporting. Additionally, patients indicated that it was important to consider pain other than anal pain, enjoyment of forms of sexual activity other than anal sexual activity, as well as problems with sitting as important to their anal HSIL experience.

Two items, “I have problems with sitting” and “I am worried about my condition getting worse” were not cognitively tested, but were ultimately included in the current version of A-HRSI. While “problems with sitting” was indicated as being problematic during concept elicitation interviews, the impact itself received low endorsement (i.e., spontaneously mentioned by two participants). However, when asked during cognitive interviews if there were any other aspects that were related to anal HSIL that were not included on the A-HRSI, “problems with sitting” was volunteered by an additional five participants. We decided that this participant input warranted the inclusion of this particular item. The item “I am worried about my condition getting worse” was added based on expert clinician feedback that those not treated but actively monitored may experience this concern more intensely. The item was adapted from a similar item in the FACT-G (22). Both items will undergo additional testing during the next phase of psychometric validation.

Although A-HRSI development used best practice methodology for devising a new measure, limitations of the study include a lower percent of female-identified participants (15.3%) than one might expect, given that women make up about 25% of the population of persons living with HIV/AIDS in the U.S. (25). Recent work has demonstrated that the prevalence of anal HSIL in HIV-infected women ranges from 18-25% (26). As such, we are aiming to target a higher percentage of females during the quantitative validation phase. Those whose primary language is non-English were excluded for this phase of measurement development, but future studies should entail culturally-appropriate translations. To facilitate recruitment, we provided participants with the option to complete cognitive interviews via telephone, with the A-HRSI sent to them a week in advance via mail. While participants were asked to not review the A-HRSI prior to being prompted via telephone, we did not probe to determine whether they adhered to these instructions. Additionally, our RSA was not able to observe subtle participant reactions when completing the questionnaire for telephone-based interviews that would be otherwise probed during an in-person interview. However, we feel that having participants indicate that a question was difficult to understand, followed by probing of those problems would encompass any issues experienced for a given item.

The present study made use of state-of-the-art techniques to establish the content validity of a measure that will be used to provide patients diagnosed with and/or monitored for anal HSIL with an opportunity to report their health-related symptoms and impacts. Currently underway are steps to demonstrate other psychometric aspects of the A-HRSI validity, including test-retest reliability, discriminant and convergent validity, and the clinical responsiveness of score changes in those recently diagnosed and in surveillance and those recently treated. Once these measurement validation steps are completed, the measure will be implemented in the ANCHOR study to capture longitudinal HRQoL outcomes in both study arms, with this information ultimately being used to better meet the needs of these patients and inform clinical decision-making.

ARTICLE SUMMARY.

We report the development of a health-related symptom index to assess the impact of anal cancer screening and treatment or active monitoring of pre-cancerous lesions.