Theoretical to Tangible: Creating a Measure of Self-Advocacy for Female Cancer Survivors

Abstract

Background and Purpose

Self-advocacy is well appreciated but poorly defined concept in oncology.

Methods

The development of the Female Self-Advocacy in Cancer Survivorship (FSACS) Scale’s theoretical underpinnings and item development led to evaluations of the measure’s content validity and reliability.

Results

The construct of self-advocacy contains 3 subdimensions with 57 Likert-type self-report items. Content validity results (S-CVI = 0.81 and S-CVI/UA = 0.83) indicated strong relevancy of items. Reliability results for each of the 3 subdimensions supported the consistency of the FSACS Scale scores, with strong internal consistency (Cronbach’s alphas = .88, .81, and .90) and test–retest reliability (PPMC r = .85, .97, and .88).

Conclusions

The FSACS Scale captures the construct of self-advocacy, and its scores show strong reliability.

Self-advocacy has been identified as a critical component of improving health outcomes because it underlies individuals’ ability to understand and lead their own care (Clark & Stovall, 1996; Ferrell, McCabe, & Levit, 2013; Walsh-Burke & Marcusen, 1999). A patient’s ability to advocate for her health, social, and personal needs not only has the potential to make her a proactive and engaged health care consumer but also affords her the ability to ensure her health and care promote her self-worth and identity. Yet, a dearth of research leaves researchers and clinicians unable to measure survivors’ ability to self-advocate and therefore provide evidence-based interventions (Hermansen-Kobulnicky, 2008). Moreover, differences between and within genders are known to exist in how cancer survivors self-advocate (Sinding et al., 2010; Wiltshire, Cronin, Sarto, & Brown, 2006). In order for self-advocacy to be an accurate, effective variable for use in research and practice, a new measurement tool must be created specific to the phenomena of self-advocacy in this population.

The purpose of this study is to describe our research team’s multistep process of establishing content validity and reliability of a measurement of self-advocacy among female cancer survivors, the Female Self-Advocacy in Cancer Survivorship (FSACS) Scale. We aim to (a) report the conceptual and empirical steps included in our process and (b) evaluate the initial content validity and reliability of the FSACS Scale. Conceptual challenges and key decisions will be discussed along with their implications for clinical and research use of the final instruments. The process of compiling quantitative (deductive) and qualitative (inductive) sources of information to form the domains of a construct, developing items within each domain, and then testing the content validity and reliability of the instrument will be described.

PROCEDURES FOR INSTRUMENT DEVELOPMENT

Concepts and constructs are the basic building blocks of scientific theory and represent our verbal representations of “real world” phenomena (Watt & van den Berg, 1995). Some of these phenomena are directly observable; others are more abstract. Regardless of the level of abstraction, concepts and constructs must be unambiguously defined to operationalize the construct into reliable and valid instruments. As Shadish, Cook, and Campbell (2001) caution, “To the extent that experiments contain construct errors, they risk misleading both theory and practice” (p. 65). Ensuring that measures of these subjective phenomena accurately and meaningfully detect the presence of these concepts allows inferences to be made regarding the presence of the construct based on the measurement.

The process of creating valid and reliable measures of abstract constructs such as behaviors, attitudes, personal characteristics, and quality of life is a complex, time consuming but essential task in behavioral research. Few articles provide practical advice and methodological directions about how to operationalize abstract behavioral concepts into psychometrically sound, parsimonious instruments. Similarly, most manuals and textbooks provide broad guidelines without describing the contextual nuances of item and scale development. This article, which describes the progression from concept to tool can provide an exemplar for researchers endeavoring to develop measures that accurately operationalize abstract constructs.

Establishing the initial FSACS Scale was performed in multiple steps using established methods for psychometric and instrument development (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 1999; U.S. Food and Drug Administration, 2009): (a) Developing the initial instrument: Quantitative and qualitative findings from the literature and qualitative data from focus groups was analyzed and synthesized into a construct definition, three dimensions, and 57 items. (b) Content validity testing: The initial instrument was tested for face value approval among a group of professional and lay experts in self-advocacy to evaluate relevancy and clarity. (c) Reliability testing: A revised instrument was tested among a pilot sample of 40 adult female cancer survivors for consistency in item responses over a 2-week time period.

Intent

The first step in instrument development is defining the intent of the instrument. We wanted to measure adult female cancer survivor’s abilities to advocate for their health, well-being, and self-worth during their cancer journey. Self-advocacy was assumed to be a state, or transient characteristic, rather than a trait, or enduring, characteristic.

Construct Definition

Constructs link theories to experiments, and therefore how we define constructs is crucial to ensuring that inferences from measurement to theory can be made within and across concepts, theories, and uses (Shadish et al., 2002). During a 2-year period, we collected and analyzed multiple sources of evidence and reduced all of our data into clearly defined, essential attributes of self-advocacy. The goal was to operationalize the concept of self-advocacy using its distinguishing features as revealed by previous research and patient experience. This iterative process included several revisions before deciding on a set of items. Conceptual domains were created using both deductive and inductive methods.

Deductive methods included a concept analysis and literature review of self-advocacy within cancer survivorship (Hagan & Donovan, 2013b). An in-depth review of the literature and patient experiences were absolutely critical to uncovering the full range and breadth of the phenomena. Research was included from oncology, HIV/AIDS, mental health, disability, empowerment, engagement, and female health psychology fields. After reviewing this broad literature, theoretical clarity came through conceptually differentiating the predictors, outcomes, and defining characteristics of self-advocacy within oncology according to Walker and Avant’s methodology (2005).

During the literature review, an existing measure of self-advocacy, the Patient Self-Advocacy Scale (PSAS; Brashers, Haas, & Neidig, 1999), was identified. Although both the PSAS and the FSACS Scale attempt to capture the construct of “self-advocacy,” the populations of interest are categorically distinct. The PSAS was developed among individuals with HIV and a primarily male population. Because of the significant disease and gender differences and existing evidence for inadequacy of the PSAS when tested among cancer survivors (Hermansen-Kobulnicky, 2008), we chose not to adapt the previous measure but to create a new measure with consideration for the behavioral aspects that may overlap between patient and gender populations.

Inductive methods included a focus group of cancer survivors (n = 14) to elicit patient perspectives on and experiences of self-advocacy (Hagan & Donovan, 2013a). The results not only corroborated several of the findings from the concept analysis but also revealed new attitudes about self-advocacy particular to the female cancer population that had not yet been discussed in the literature.

Construct

The construct of self-advocacy was ultimately defined as how patients stand up for themselves during their cancer experience. When faced with any of the myriad challenges that cancer diagnosis, treatment, and survivorship presents, how is a woman able to get her needs, priorities, and preferences met? Beyond just being proactive and engaged, self-advocacy defines the ability to face problems that come as a result of the cancer. Many of these problems may concern treatment and working with the medical team, but other problems may concern accessing and using information or maintaining relationships with family members, friends, and other cancer survivors.

Although historically advocacy has been used as a means of addressing social inequalities and power hegemony within the medical institution (Brashers, Haas, Neidig, & Rintamaki, 2002), the construct of self-advocacy does not promote adversarial relationships between patients and their health care providers or institutions. Rather, the degree to which a patient is able to self-advocate is the degree to which she can productively participate as an equal member of her health care team and social support network even in the face of difficult situations.

Challenges

Conceptual challenges arose while developing the definition of self-advocacy. This has consequences not only for the measurement tool but also in how the tool (a) will be applied to and translated into intervention research and (b) can support broader theory development. As discovered while conducting the concept analysis (deductive method), theories of self-advocacy within cancer are largely immature and nonparsimonious, creating confusion regarding the defining features of self-advocacy and differentiation with its antecedents or consequences. Definitions encompassed both attitudinal and behavioral components; inter- and intrapersonal applications; and situational and policy spheres of action. Different disciplines made different assumptions about the intent of advocacy varying from collective groups changing national or state policies to groups of individual survivors for improving their personal health, well-being, and autonomy. Ultimately, we decided to define self-advocacy in terms most relevant to cancer survivors which had scientific evidence in the literature and would ensure that the application of the tool would be meaningful given the needs of this population.

Difficulties also occurred during the inductive derivation of item content. The focus groups uncovered the real-life behaviors and attitudes of how women define “self-advocacy” in their contextually situated cancer experiences. The initial analysis of the results provided rich descriptive data identifying a multitude of themes and subthemes. However, consistent with traditional focus group analysis, analyses focused on reaching theoretical saturation and not generalizability to other populations. In order for instrument development to encompass the full breadth of the participants’ conversations, a fresh reading of the focus group transcripts was required to assure all possible behavioral indicators of self-advocacy would be captured in the measurement’s items and dimensions.

Combining the results from the use of the deductive and inductive methods was a third challenge and the most conceptually demanding. Our aim was to have subscales that were similar enough to have a shared relationship with the overall construct yet different enough to provide unique information about the full conceptual breadth of self-advocacy. After consulting with instrumentation experts, it was decided to focus on three dimensions of self-advocacy supported by both deductive and inductive methods, which together would be able to define self-advocacy, discriminate between women who do and do not self-advocate, and potentially be the most modifiable characteristics. This meant that some findings from the concept analysis and focus group study were not directly included in the scale’s dimensions. These findings were considered outside the scope of the measurement model and scale building because they did not focus on actions but remain significant aspects of self-advocacy to be integrated into future research. Figure 1 illustrates the synthesis and refinement of the focus group and concept analysis results into the dimensions of the FSACS Scale.

Figure 1.

Derivation of female self-advocacy in Cancer Survivorship Scale’s dimensions based on deductive and inductive pilot work. FSACS Scale = Female Self-Advocacy in Cancer Survivorship Scale.

DESCRIPTION, ADMINISTRATION, AND SCORING OF THE INSTRUMENT

Operationalization and Item Development

After each of the three dimensions was defined, self-report items were created according to the domain sampling model (DeVellis, 2012). An exhaustive list of items from a hypothetical universe of items relating to the defining characteristics of self-advocacy among female cancer survivors resulting in 20–30 items for each dimension with 71 items. This was considered enough to provide sufficient breadth of items per domain. Redundancy was encouraged at this stage with the goal of capturing the full breadth of the construct.

Response options included a 6-point Likert-type (1 = strongly disagree and 6 = strongly agree) response scale to avoid overselection of a neutral response option that is often observed in response bipolar scales with a midpoint. Positive and negative stems were included. Varying levels of intensity or difficulty were included to ensure response variance and scale sensitivity to differences in self-advocacy.

Central to our consideration of writing items was ensuring that the items were specific and concrete enough to inform future research to support women who struggle to self-advocate. By selecting reflective indicators of the construct as opposed to unobservable precursors that give rise to self-advocacy, items are more likely to be modifiable and translated into intervention. Items asking for respondents to indicate their beliefs or react to hypothetical situations were therefore avoided and preference given to actions and behaviors.

Administration and Scoring

The FSACS Scale is intended to be a self-administered measurement tool to be used in both research and practice settings as a means of identifying female cancer survivors who struggle to advocate for themselves. A total score will be used to measure a woman’s overall ability to self-advocate, whereas subdimension scale scores will provide more specific information about areas in which they may struggle.

Content Validity Testing and Results

Content validity is “the degree to which elements of an assessment instrument are relevant to and representative of the targeted construct for a particular assessment purpose” (Haynes, Richard, & Kubany, 1995, p. 239). Testing content validity assesses the degree to which the content of the instrument reflects the construct of self-advocacy among female cancer survivors.

Employing both population and content experts was critical to establishing a valid measure (Vogt, King, & King, 2004). The expert panel, described in Table 1, included nine representatives (N = 9): three females with cancer, one nurse practitioner, one physician, two researchers, one social worker, and one patient advocate. One of the survivors was an African American female. All professionals on the panel specialized in working with female cancer survivors. Qualifications of experts were evaluated by their professional and personal experience with female cancer survivors and identification by peers as promoting patient self-advocacy (Grant & Davis, 1997).

TABLE 1.

Expert Panelists for Content Validity

Member	Number	Gender	Years in Role	Cancer Survivor
Cancer survivor	3	Female	9, 10, and 18	Breast, ovarian, and brain
Nurse practitioner	1	Female	18	No
Physician	1	Male	30	No
Researcher	2	Female	14, 25, and 35	No
Social worker	1	Female	30	No
Patient advocate	1	Male	10	Melanoma

We personally contacted each panel member to explain the instrument development process, the definition of self-advocacy, and the value of their experiential expertise in self-advocacy to critiquing our instrument. All contacted experts agreed to participate. Each expert panel member was sent the initial FSACS Scale, a written introduction to the concept of self-advocacy, a content validity questionnaire, and a copy of the preliminary FSACS Scale. The experts reviewed each item and the scale as a whole. Each item was rated separately for relevancy and clarity using 4-point Likert-type scales ranging from 1 (not relevant/clear) to 4 (very relevant/clear) and space was given for comments. We also asked questions regarding the overall scale, suggestions for additions and revisions to the measure, and any other comments. Completed packets were returned and analyzed.

Lynn’s (1986) methodology was used to calculate the content validity index (CVI) for each scale item and the entire scale. Based on Lynn’s criteria, a CVI of 0.78 (or seven of the nine panelists) indicates adequate endorsement of an item or the instrument beyond the α = .05 level of significance.

In total, 14 items were deleted because of low CVI, redundancy of items, or for being outcomes rather than behaviors of self-advocacy. Thirteen items were reworded, mainly to soften any confrontational language, improve specificity, and include action orientation. Based on panelist feedback, 22 items had ≥3 panelists rating its relevancy as a 1 or 2. Five of the 12 items from the previous self-advocacy measure received low rankings from the panelists. Of note, 3 items were preserved despite low ratings because of discrepancies between researchers’ and survivors’ ratings with preference given to survivors’ ratings. For example, the item “I don’t know enough to make decisions about my cancer and treatment” (reverse scored) was endorsed by survivors but not researchers.

Content validity statistics of the 57-item scale are reported in Table 2. The Average Scale-CVI (S-CVI/Ave) was calculated by averaging the proportion of experts rating each individual item as relevant (rating of 3 or 4). In other words, the S-CVI/Ave is equal to the average of each of the individual item CVIs (I-CVI). The Scale-CVI/Universal Agreement (S-CVI/UA) was calculated as the proportion of items rated as relevant (rating of 3 or 4) by all 8 experts (Polit & Beck, 2006). Both the S-CVI/Ave (0.81) and S-CVI/UA (0.83) were above the recommended cutoff level of 0.78.

TABLE 2.

Content Validity Statistics

Type of CVI	Statistic	Calculation
Average Scale-CVI (S-CVI/Ave)	0.81	Average proportion experts rating each item as relevant (rating of 3 or 4)
Scale-CVI Universal Agreement (S-CVI/UA)	0.83	Proportion of total items judged by all experts as relevant (rating of 3 or 4)

Note. CVI = content validity index.

Reliability Testing and Results

Reliability testing was performed to (a) evaluate internal consistency (the degree of consistency among items in the instrument) and test-retest reliability (the stability of the observed scores over time) of the FSACS Scale; (b) evaluate the feasibility of assessing self-advocacy and related measures necessary for instrument validation; and (c) produce a parsimonious scale to be validated in a large sample study. Note that prior to reliability testing, pilot testing of the final 57 items was conducted with a group of masters’-prepared nurses, research staff, and staff at the University of Pittsburgh School of Nursing as final check for readability, grammar, spelling, and formatting.

Evaluation was completed with a sample of female adult cancer survivors. To estimate test-retest reliability, a repeated measure design with baseline and 2-week measures was used. Time points were chosen to test the scale’s consistency within a time period in which little variation is expected (DeVillis, 2012).

Recruitment of N = 40 participants was conducted at four cancer and advocacy organizations and three cancer clinics in Pittsburgh, Pennsylvania. Inclusion criteria included (a) female, (b) ≥18 years old, (c) history of a cancer diagnosis at ≥18 years old, and (d) a cancer diagnosis other than basal cell carcinoma or cervical intraepithelial neoplasia stage 1 (these survivors have inherently different treatment and symptom experiences than other cancer survivors). Equal numbers of participants were recruited through clinics and organizations. Directors at clinics and organizations identified potential participants and introduced the study to them. Potential subjects who expressed interest in participating were introduced to the researcher (TH) who then carried out screening and informed consent procedures.

On Day 1, the researcher screened potential subjects for eligibility. If the potential participant met the inclusion criteria and was interested in participating in the study, written consent was obtained. Participants completed all baseline study measures in a private room at the clinic or office. Individual measures were randomly ordered to avoid an order effect. The researcher (TH) handed all measures to the participant, instructed the participant to complete the survey independently, asking the student for clarifications or assistance as needed. Following completion of the FSACS Scale, the researcher conducted cognitive interviews with the participant to review any problems or issues with the items or survey and to assess the feasibility of completing the survey.

After all surveys were complete, the researcher provided the participant with an envelope containing the follow-up survey, instructions to complete the survey in 2 weeks (14 days), and a preaddressed, prestamped envelope to mail the survey to the student. The researcher called each participant 2 days before the follow-up survey was to be completed and sent reminder postcards 3 and 5 days after the due date if the survey was not received. Participants were sent thank you cards with a $20 prepaid debit card after receipt of the follow-up survey.

Sample characteristics and health histories for the reliability study are described in Table 3 and Figure 2. The sample had a mean age of 57.28 years (SD = 13.16 years, observed range = 25–89 years), and 53.8% were married. Women represented a diverse population with 12.8% of participants identifying as Black or African American and one identifying as Hispanic. Twenty-one percent of women had a high school degree or less, and only 35.9% were working full-time. About 20% (n = 8) of women had a household gross annual income less than $30,000. Most women (n = 17, 42.5%) had a diagnosis of ovarian cancer, followed by breast cancer (n = 15, 37.5%). Almost half of the women were within 1 year of their cancer diagnosis (n = 19, 47.5%). The remaining women were either between 1 and 5 years since diagnosis (n = 11, 27.5%) or greater than 5 years since diagnosis (n = 10, 25%). Most women (n = 26, 65%) were receiving treatment at the time of survey completion. Seven (12.5%) women had experienced at least one recurrence.

TABLE 3.

Demographic and Health Information for Reliability Study Sample

Demographic Information	n	%	M	SD
Age			57.5	13.1
Years of education			14.6	4.2
Race and ethnicity
White	32	84.2
African American/Black	5	13.2
Other	1	2.6
Latina	1	2.6
Employment status
Working ≥35 hr/week	14	35.9
Retired	10	25.6
Working <35 hr/week	9	23.1
Disabled	4	10.3
Laid off/unemployed	1	2.6
Relationship status
Currently married	21	53.8
Divorced/separated	7	18.0
Never married	4	10.3
Widowed	4	10.3
Living with partner	3	7.7
Household annual income
<$30,000	7	20.6
$30,000–$59,999	8	23.5
$60,000–$99,999	9	26.5
≥$100,000	9	26.5
Cancer History
Time since diagnosis
<1 year	19	47.5
1–5 years	11	27.5
>5 years	10	25.0
Cancer recurrence (M and SD if any recurrence)	7	17.5	3.4	4.4
Multiple cancer diagnoses	5	12.5
Currently on treatment	26	65.0

Figure 2.

Cancer diagnoses of reliability study sample.

Two measures of reliability were estimated: (a) internal consistency as measured by Cronbach’s alphas based on N = 40 and (b) test-retest reliability or item response stability as measured by Pearson’s product-moment correlations based on n = 39. Participant feedback was reviewed using content analysis to identify recurring themes. Internal consistency for the FSACS was strong (alpha = .92). The three dimensions had Cronbach’s alpha’s of .88, .81, and .90, respectively.

Preliminary evaluation of test-retest reliability indicates that the scale is highly stable across time points (Pearson product-moment correlation of r = .94). The three dimensions also showed strong test-retest reliability (Application of Information: r = .85; Leading My Health Care: r = .97; Connected Strength: r = .88). Feasibility and acceptability of completing the FSACS Scale was reported to be high by participants although several participants noted that the number of items in the FSACS Scale would need to be greatly reduced to reduce response burden.

Cognitive interviews identified participant concerns about specific questionnaire items. The most common concerns reported by participants included difficulty responding to items that implied that they have control over their cancer, have an adversarial relationship with their provider, or suggested a need to actively reach out to other survivors or tell their story to others. Future iterations of the survey instructions and final item selection will consider these patient concerns.

Figure 3 illustrates the FSACS Scale used during reliability testing. Indications are made to highlight items that performed poorly during content validity testing, reliability testing, and cognitive interviewing. Items listed in this figure do not represent the final FSACS Scale; construct validity testing will further test this measurement model and result in a parsimonious scale with a significantly reduced number of items. Figure 3 is meant to be a useful template for instrument developers who are looking to take a construct and clarify its dimensions, subdimensions (if any), and items while ensuring consistency between these varying levels of abstraction.

Figure 3.

Initial female self-advocacy in cancer survivorship measurement model with results of content validity, reliability, and cognitive interview testing. P = in previous measure of self-advocacy for HIV/AIDS population; R = reverse-scored item; + = cognitive interviews: negative verbal feedback from at least three participants; * = CVI score: low score measured by at least three content experts rating item a 1 or 2 for relevancy or clarity; # = Reliability: low internal consistency measure by item-total correlations <.30; Italicized = Considered for deletion in future testing based on ≥1 negative reliability and content validity testing (+, *, or #).

DISCUSSION

Self-advocacy is a process that precedes and promotes many positive health care behaviors and attitudes. Only by measuring self-advocacy through scientifically discovered indicators of the construct can we as researchers and practitioners aim to improve it. The FSACS Scale, which operationalizes the latent variable of self-advocacy, should provide the means by which to measure self-advocacy. In this study, we moved the construct of “self-advocacy” from a well-appreciated but underdefined part of the cancer lexicon into a concrete, measurable construct. Using both deductive and inductive reasoning in a conjoint, purposeful manner, we formed and tested the FSACS Scale for content validity and reliability. By rigorously developing and testing the initial content validity and reliability of the measure, users can have increased confidence in the ability to make inferences about patients’ abilities to self-advocate based on their FSACS Scale scores.

An S-CVI of 0.81 and S-CVI/UA of 0.83 and strong reliability results provide promising evidence that the content is (a) representative of the targeted construct, (b) item responses are stable over time, and (c) items have a high degree of consistency with each other. Cronbach’s alpha is a function of scale length and may be overinflated, so internal consistency will be retested in future construct validity testing with the final set of scale items.

Although the methodology of clarifying the dimensions of self-advocacy was time consuming and required reading multiple sources of literature to understand the historical uses and intentions of self-advocacy, we believe this detailed attention resulted in a clear tool capable of being retested and reapplied in future patient populations. By paying attention to previous research, patient experiences, expert opinion, and thorough analysis of the initial tool, future applications of the tool should be easier to understand and interpret because it reflects the best understanding of the phenomena of self-advocacy among female cancer survivors.

The version of the FSACS Scale discussed in this analysis is not the final scale. Currently, in-depth construct validity testing of the FSACS Scale is underway in a large sample study. Hypotheses will be tested related to the extent to which the factor structure of the scale is congruent with the conceptual dimensions of self-advocacy and how the FSACS Scale scores should compare to scores on other measures, including (a) predictors and outcomes of self-advocacy and (b) constructs that are similar but distinct from self-advocacy (e.g., PSAS, patient activation). Our goal is to validate a parsimonious measure of self-advocacy that is specific to self-advocacy in female cancer survivors, sensitive to changes in self-advocacy over time, and specific enough to distinguish between women who struggle to self-advocate and those who do not.

Finally, this study has limitations. Survey research inherently includes errors of observation and nonobservation that impact the precision of a measure. Participant responses may not be accurate (observational error), the sample may not represent the population to which the instrument is to be made generalizable (coverage error), and people who are approached do not always complete the study (response error). Self-reported administration methods introduce respondent recall, bias error, and social desirability error, which can impact external validity of score interpretations. Concerns of observation and bias were addressed by instructing participants that little is known about the benefits or harms of self-advocacy, including a large number of items, and targeting behaviors varying in level of difficulty. Despite these limitations, the initial FSACS Scale meets most of Lohr and colleagues’ (1996) attributes for high-quality, health outcome measurement tools.