Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2014 May 6.
Published in final edited form as: J Nurs Meas. 2013;21(2):320–334. doi: 10.1891/1061-3749.21.2.320

Psychometric Analysis of the Patient Assessment of Own Functioning Inventory in Women With Breast Cancer

Mandy J Bell 1, Lauren Terhorst 2, Catherine M Bender 3
PMCID: PMC4011069  NIHMSID: NIHMS572635  PMID: 24053059

Abstract

Background and Purpose

The purpose of this secondary analysis was to examine the reliability and validity of the Patient Assessment of Own Functioning Inventory (PAOFI) in postmenopausal women with early-stage breast cancer before adjuvant therapy.

Methods

Data from 259 postmenopausal women with early-stage breast cancer before adjuvant therapy were analyzed. Exploratory factor analysis was used to uncover the PAOFI’s underlying factor structure and reliability coefficients were computed for each subscale.

Results

5 factors measuring perceived cognitive functioning had eigenvalues >1 and accounted for 54% of the extracted variance. Subscale reliability coefficients ranged from .572 to .883.

Conclusions

Psychometric evaluation of the PAOFI provided evidence of reliability and construct validity in this population. Additional studies are needed to confirm the 5-factor structure.

Keywords: breast cancer, psychometrics, neuropsychological tests, postmenopausal

Changes in cognitive function with systemic adjuvant therapy have been well-documented in women with breast cancer (Bender et al., 2006; Bender et al., 2007; Castellon et al., 2004; Shilling, Jenkins, Fallowfield, & Howell, 2003). As many as 71% of women with breast cancer experience some type of change in cognitive function as a consequence of their disease and treatment (Wefel, Saleeba, Buzdar, & Meyers, 2010). The biological basis for these changes in cognitive function is likely to be multifactorial and may include changes in blood–brain barrier integrity, DNA damage, cytokine deregulation, genetic susceptibility, and reduced estrogen as a consequence of cancer therapy (Ahles & Saykin, 2007). It is also now clear that some women with breast cancer perform more poorly on objective measures of cognitive function prior to the initiation of systemic adjuvant therapy than healthy age- and education-matched women (Ahles et al., 2008; Wefel et al., 2004). Despite the standard use of objective neuropsychological tests to measure cognitive function, the ecological validity of these tests, and how test performance translates to the execution of daily functional activities, is not clear (Chaytor & Schmitter-Edgecombe, 2003). Because subjective measures may reveal how cognitive impairments affect an individual’s ability to carry on their daily functional activities, such measures have been incorporated into comprehensive neuropsychological assessments when assessing cognitive function (Bender et al., 2008). Self-reports of cognitive problems are more commonly associated with depression, anxiety, and fatigue than they are with scores on objective measures of cognitive function (Poppelreuter et al., 2004). Thus, identification of a reliable and valid self-report measure of subjective cognitive function can be useful clinically to direct nurses to assess whether women with breast cancer who report cognitive problems are also experiencing depression or anxiety, and it can provide valuable information regarding the functional impact of cancer and cancer treatment in this patient population.

Prior to the incorporation of a subjective measure into research that assesses cognitive function, the measure’s psychometric properties should be analyzed to ensure its reliability and validity in the population of interest. The Patient Assessment of Own Functioning Inventory (PAOFI), a subjective measure of cognitive function consisting of five dimensions, has been validated in a sample of patients referred for neuropsychological testing; however, the reliability and validity of the PAOFI have not been examined in different populations. According to Streiner and Norman (1995), validity of measurement is contingent on the population, and when an instrument is used in a different population, validity must be reestablished. The objective of this secondary analysis was to establish the reliability and validity of the PAOFI in a sample of postmenopausal women with early-stage breast cancer prior to adjuvant therapy. To examine the underlying factor structure of the PAOFI, exploratory factor analysis (EFA) was performed. The expectation was that the five dimensions of the PAOFI would be uncovered; however, there was no predetermined hypothesis for item composition of each subscale. Construct validity was examined by correlating PAOFI scores to scores obtained from the Profile of Moods States (POMS) and the Medical Outcome Study Short Form-36 (SF-36). The following hypotheses related to construct validity were tested:

  1. A decrease in perceived cognitive functioning will be associated with a decline in physical and psychological functioning. Because high PAOFI scores indicate poor perceived cognitive functioning and high SF-36 scores indicate better functioning, a negative relationship is expected.

  2. Based on prior studies of the PAOFI (Chelune & Lehman, 1986), we expect that we will find subscales that measure both intellectual and sensorimotor functioning, as well as additional subscales. Therefore, we hypothesize that the correlations of any emergent PAOFI subscales that assess cognitive or intellectual functioning will relate moderately to strongly to the psychological subscale of the SF-36, whereas PAOFI subscales that assess sensorimotor functioning will be moderately to strongly related to the SF-36 physical functioning subscale.

  3. A decrease in perceived cognitive functioning (high PAOFI score) will be related to an increase in fatigue (high POMS score). This hypothesis was based on previous findings (Iconomou, Mega, Koutras, Iconomou, & Kalofonos, 2004; Kibiger, Kirsh, Wall, & Passik, 2003; Poppelreuter et al., 2004) that reported that a decline in cognitive functioning is associated with symptoms such as fatigue.

BACKGROUND OF THE PATIENT ASSESSMENT OF OWN FUNCTIONING INVENTORY (PAOFI)

Conceptual Background

The PAOFI was “designed to elicit patients’ self-perceptions regarding the adequacy of their functioning in various everyday tasks and activities,” and it is described as an “expanded Halstead-Reitan battery” (Chelune & Lehman, 1986, p. 96). A full description of the history of the Halstead-Reitan battery can be found in Reitan and Wolfson (2004). In addition to the lists of abilities adapted from the Halstead-Reitan battery, PAOFI item development was based on previous patient complaints. Items of the PAOFI measure the frequency of everyday difficulties using a 6-point scale ranging from almost never to almost always. As noted by Chelune and Lehman (1986), the underlying conceptual framework for the instrument was based on the possibility that an individual’s perceptions of his/her abilities and disabilities could provide insight into his/her underlying condition.

Structure of PAOFI

In total, the multidimensional PAOFI comprises 47 items that were logically grouped, with the first 33 items in five subscales, and three additional narrative subscales (Chelune & Lehman, 1986). The first five subscales consist of items rated on a 6-point scale (almost never to almost always) that assess daily functional abilities/activities related to

  • memory (Subscale 1; 10 items), comprising items that measure how often participants forget events, people, or things they were supposed to do (e.g., “How often do you forget something that has been told [sic] you within the last day or two?”; Chelune & Lehman, 1986, p. 119).

  • language and communication (Subscale 2; 9 items), which elicits information about participants understanding of verbal and written information and difficulty with word-finding (e.g., “How often do you have difficulties understanding what is said to you?”; Chelune & Lehman, 1986, p. 121).

  • use of hands (Subscale 3; 2 items), comprising two items that measure difficulty performing tasks with the right and left hands (e.g., “How often do you have difficulty performing tasks with your right hand?”; Chelune & Lehman, 1986, p. 123).

  • sensory-perceptual (Subscale 4; 3 items), made up of items that measure the senses of touch and vision (e.g., “How often do you have difficulty feeling things with your right hand?”; Chelune & Lehman, 1986, p. 123).

  • higher level cognitive and intellectual functions (Subscale 5; 9 items), comprising items that measure participants’ perceptions of whether their thoughts seem confused, they are distracted from what they are doing, or have more difficulty planning activities or solving problems (e.g., “How often do your thoughts seem confused or illogical?”; Chelune & Lehman, 1986, p. 124).

The three narrative subscales measure information related to a respondent’s daily activities and the presence/absence of any perceived impairments (Chelune & Lehman, 1986). Please refer to Chelune and Lehman (1986) for the complete published version of the PAOFI (first five subscales).

Previous Psychometric Analyses

The PAOFI has been used to assess perceived cognitive function in diverse populations, including breast cancer (Bender et al., 2008), HIV (Rourke, Halman, & Bassel, 1999), bipolar disorder (Burdick, Endick, & Goldberg, 2005), chronic kidney disease (Jassal, Devins, Chan, Bozanovic, & Rourke, 2006), and substance abuse (Richardson-Vejlgaard, Dawes, Heaton, & Bell, 2009; Roehrich & Goldman, 1993); however, the reliability and validity of the PAOFI has not been established in these populations. Evaluation of the PAOFI’s psychometric properties has been limited to two studies.

In the study conducted by Chelune and Lehman (1986), the factor structure of the PAOFI was assessed in a sample of subjects referred for neuropsychological evaluation on clinical grounds (N = 598) and normal controls (N = 105). Principal components analysis (PCA) with varimax rotation was employed to assess the underlying factor structure of 32 of the 33 items. One item of the Memory Scale, which addresses forgetfulness of meeting people (“How often do you forget people whom you knew or met a year or more ago?”; Chelune & Lehman, 1986, p. 120), was not found to be significantly different between those referred for neuropsychological evaluation and those classified as normal controls, and it was omitted from the factor analysis. Five factors with eigenvalues > 1 were extracted. These five factors accounted for 57.5% of the extracted variance.

Next, the resultant structure was compared to the logically grouped PAOFI subscales. Although the memory factor was identified as unidimensional in the logically grouped structure, the PCA revealed items from the memory subscale loaded under two separate factors, suggesting that the memory subscale was multidimensional. In addition, items from the Use of Hands and Sensory-Perceptual subscales loaded under one factor, indicating these two factors could be combined to create a more generalized sensorimotor factor. Items from the language and communication subscale loaded together under one factor as previously defined in the logical grouping, whereas items from higher level cognitive and intellectual functions subscale also formed a separate factor that replicated the prior logical grouping. Thus, the following five factors were extracted from the PCA: (1) Memory 1; (2) Memory 2; (3) Sensorimotor; (4) Language and Communication; and (5) Higher-Level Cognitive and Intellectual Functions. Based on these results, Chelune and Lehman (1986) concluded that the PAOFI items within each subscale “represent meaningful clusters of everyday difficulties with central themes (i.e., memory, language and communication, sensorimotor functioning, and cognitive/intellectual skills)” (p. 103).

In the study by Richardson-Vejlgaard et al. (2009), construct validity of the memory, language and communication, and the higher level cognitive/intellectual functions subscales was assessed in a sample of male veterans enrolled in a daily substance abuse treatment program (N = 74) and a sample of nonclinical, English speaking males (N = 150). Comparison of PAOFI scores between subjects enrolled in the daily substance abuse treatment program and nonclinical subjects demonstrated discriminant validity. Subjects in the substance abuse group reported twice as many PAOFI complaints overall compared to the subjects in the nonclinical group (p < .001). However, evaluation of Pearson correlations between PAOFI scores and factor scores of the neuropsychological test battery failed to demonstrate convergent validity. Examination of the relationship between PAOFI scores and cognitive decline also failed to demonstrate convergent validity. Although the number of subjects with clinically meaningful cognitive decline was significantly greater in the substance abuse group, (χ2[1] = 25.0, p < .0001), results did not demonstrate that those with clinically meaningful cognitive decline reported more PAOFI complaints (t[131] = 0.98, p > .05). This phenomenon of lack of correlation between objectively measured cognitive function and subjectively measured reports of cognitive problems is well documented (Iconomou et al., 2004; Kibiger et al., 2003; Poppelreuter et al., 2004). More often, subjectively measured cognitive problems are related to mood, and other self-reported symptoms such as fatigue.

The study of Richardson-Vejlgaard et al. (2009) revealed that PAOFI responses differed in different populations, with the nonclinical group reporting less difficulty than the substance abuse group. This indicates that patients in diverse populations may have different item response patterns, which may lead to a different factor structure than the result reported in the Chelune and Lehman (1986) study. Postmenopausal women with early-stage breast cancer are likely to have a differing set of cognitive difficulties as compared to the clinical populations of the prior empirical research; therefore, although the underlying framework of five dimensions of perceived cognitive functioning is likely to hold, items may cluster differently or may be unwarranted in this population. Therefore, the purpose of the current investigation was to perform a secondary data analysis using scores from the PAOFI to ascertain evidence of reliability and construct validity in a sample of postmenopausal women with early-stage breast cancer prior to adjuvant therapy.

METHODS

This study was a secondary data analysis of baseline data obtained from the Anastrozole Use in Menopausal Women (AIM) Study (R01 CA107408). Using a prospective, quasi-experimental, nonequivalent, four-group, repeated measures design, the AIM study examined the impact of anastrozole on cognitive function in postmenopausal women with early-stage breast cancer. Women eligible for the AIM study were postmenopausal, 18–75 years of age, able to speak and read English, and had completed at least 8 years of education. Women with a self-reported history of hospitalization for psychiatric illness within 2 years of study enrollment or a prior diagnosis of neurological disease (e.g., stroke, dementia syndrome) were excluded from study participation. Women with clinical evidence of distant metastasis or a prior diagnosis of cancer were excluded. Seven domains of cognitive function including attention, learning, memory, psychomotor speed, mental flexibility, executive function, and visuospatial ability were evaluated with a comprehensive, objective, neuropsychological test battery, which included but was not limited to the PAOFI, POMS, and the SF-36.

Sample

Data for the current secondary analysis were restricted to baseline PAOFI assessment of postmenopausal women who had undergone surgery for early-stage breast cancer (N = 259). Baseline data collection occurred after primary breast surgery and before the initiation of any adjuvant therapy. Given that women with breast cancer, prior to systemic adjuvant therapy, perform more poorly on objective measures of cognitive function than their healthy, age and education matched counterparts (Ahles et al., 2008; Wefel et al., 2004), healthy controls were not included in this secondary analysis. Baseline data from the SF-36 and the Fatigue/Inertia subscale of the POMS were also used in this secondary analysis to further assess construct validity between the PAOFI and measures of functional ability and fatigue. Approval for this secondary analysis was granted by the University of Pittsburgh Institutional Review Board.

Measures

PAOFI

As described earlier, the PAOFI measures subjects’ perceptions of functioning when performing everyday tasks and activities that reflect cognitive strengths and weaknesses. Subjects rate each item on a scale ranging from 0 (almost never) to 5 (almost always). The total score is the sum of the responses to each item (score of 0 to 160); subscale scores are the sum of responses to items in each subscale. High scores on the PAOFI subscales are indicative of poor perceived cognitive functioning.

SF-36 and POMS

Data from the SF-36 and the Fatigue/Inertia subscale of the POMS were also examined to establish external validity for the PAOFI. The SF-36 is a 36-item self-report measure of functional ability organized into dimensions for physical and psychological functioning. Dimension scores are standardized, ranging from 0 to 100, with higher scores indicating better functioning. RAND 36-Item Health Survey procedures were used to calculate the summary scores for physical and psychological functioning (McHorney, Ware, & Raczek, 1993). The POMS Fatigue-Inertia subscale is a 7-item self-report subscale in which adjectives are rated on a 5-point Likert scale. The score is the sum of responses for items (McNair, Lorr, & Droppleman, 1981), with high scores indicating increased fatigue.

Analysis

Data Screening

Prior to the evaluation of the PAOFI’s psychometric properties, data screening was performed on the PAOFI items (n = 32) used to assess perceived cognitive functioning. Table 1 illustrates the original item composition of the PAOFI administered to AIM study participants. Language and Communication item #13a was dropped from further analysis because it was not applicable to all subjects. Item frequencies and distributions were examined for potential floor or ceiling effects and univariate outliers and data were screened for missing data. Because of the small amount of missing data across the sample, the data were assumed to be missing at random. One subject had missing values for two PAOFI items and five subjects each had a missing value for one PAOFI item. In addition to graphical assessment (e.g., histograms and box plots), the Shapiro-Wilk test of normality was used to assess each item’s univariate distribution. All Shapiro-Wilk tests were found to be significant, indicating nonnormal item distributions. To assess multivariate normality, Mahalanobis distance and leverage values were generated. Leverage values greater than .05 were considered influential. One subject was identified as an influential multivariate outlier and was excluded from further analysis. Exclusion of the subjects with missing data and the score that was identified as an outlier resulted in 259 scores for use in analyses.

TABLE 1.

Item Composition of the PAOFI Administered to AIM Study Participants

PAOFI Subscale Item Numbers Number of Items
Scale 1: Memory 1,2,3,4,5,6,7,8,9 9
Scale 2: Language and Communication 10,11,12,13,13a,14,15,16,17 9
Scale 3: Use of Hands 18,19 2
Scale 4: Sensory-Perceptual 20,21,22 3
Scale 5: Higher Level Cognitive and Intellectual Functions 23,24,25,26,27,28,29,30,31 9
Total Items 32
Open in a new tab

Note. The original PAOFI measure is published in Chelune and Lehman (1986). It is important to note that the item numbers associated with each question may vary between Chelune and Lehman and this study. PAOFI = Patient Assessment of Own Functioning Inventory; AIM = Anastrozole Use in Menopausal Women.

After data screening, PAOFI item correlations were examined and sampling adequacy was assessed via the Kaiser-Meyer-Olkin (KMO) test and Bartlett’s test of sphericity. Item correlations ranged from −.002 to .715. The KMO was equal to .882. Bartlett’s test of sphericity was significant (χ2(465) = 3676.983, p <.001). Because the KMO statistic was greater than .600 (Kaiser, 1974) and the Bartlett’s test of sphericity was significant (Bartlett, 1950), the item correlation matrix was determined to be factorable.

Exploratory Factor Analysis

The dimensionality of the PAOFI was evaluated using EFA methods with SPSS software. According to previous studies, the subjects-to-variable ratio for an adequate EFA sample size should be not lower than 5 (Hatcher, 1994; Bryant & Yarnold, 1995). With 33 variables in the current analysis, the sample size of 259 was more than sufficient to support an EFA.

EFA was chosen because of its ability to parse out the shared and unique item variance to form the underlying structure (Costello & Osborne, 2005). The EFA method is considered superior to PCA because of its ability to minimize the inflation of variance estimates (Gorsuch, 1990). PCA can be a useful data reduction method; however, its inability to separate shared and unique variance produces overestimates of variance accounted for by the extracted structure and limits the extraction of a structure based on the underlying latent variables (Costello & Osborne, 2005). Because of the nonnormal distributions of the items, Principal Axis Factoring (PAF) was selected as the EFA method (Fabrigar, Wegener, MacCallum, & Strahan, 1999). Varimax rotation, which does not assume that factors are correlated and provides cleaner factor loadings, was performed to simplify factor interpretation (Cureton & Mulaik, 1975).

A forced five factor structure was selected based on the structure of the results of the study by Chelune and Lehman (1986). Eigenvalues and percentage of variance extracted by the five factors were inspected. The following criteria were assessed to determine if the five factors should be retained: (a) if all five factors possessed eigenvalues greater than 1 and (b) if all five factors accounted for at least 50%–60% of the extracted variance. According to Pett, Lackey, and Sullivan (2003), extracted factors should explain around 50%–60% of the variance for an adequate solution. To form factors, items were required to have loadings ≥0.40, which was a slightly more conservative cutoff than the .32 recommendation of Tabachnick and Fidell (2006). Cross-loading items were identified as those with loadings ≥0.40 on two or more factors.

Reliability and Construct Validity

The reliability of the emergent PAOFI subscales was also assessed. Final item composition of the subscales was based on the evaluation and refinement of the results generated by the factor analysis. Item-total correlations and inter-item correlations were inspected. Separate Cronbach’s alpha (Cronbach, 1951) coefficients were computed for each subscale. Data were not available for a gold standard measure of cognitive functioning; therefore, as mentioned in the introduction, we tested several hypotheses that were derived a priori to provide evidence of validity (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 1999). This method of validity testing has been conducted in prior psychometric studies (Sepucha et al., 2012). PAOFI subscale scores were correlated with the fatigue subscale of the POMS and with the physical and mental subscales of the SF-36 using Pearson’s correlation. Correlations with p values less than .05 were considered significant. The direction and magnitude of the correlations were examined to draw conclusions regarding the hypotheses.

RESULTS

Sample Demographics

Data from the baseline assessment included N = 259 postmenopausal women with early-stage breast cancer prior to adjuvant therapy. This well-educated sample ranged in age from 44 to 75 years with an overall mean age of 60.44 (SD = 6.365) years. The 95% of the sample was White, 63% were currently married, and 68% were diagnosed with stage 1 breast cancer. Adjuvant treatment regimens initiated after primary breast surgery and baseline data collection included chemotherapy and anastrazole (33.2% of women), chemotherapy only (10.4% of women), and anastrazole only (56.4% of women).

Exploratory Factor Analysis

PAF using a forced five-factor structure and varimax rotation revealed that five factors had eigenvalues > 1 and accounted for 54% of the extracted variance. Relevant factor loadings (≥0.40) are presented in Table 2. Item communalities ranged from .157 to .699. The five factors were designated by the following names: Higher Level Cognitive & Intellectual Function (Factor 1); Language & Communication (Factor 2); Memory 1 (Factor 3); Sensorimotor (Factor 4); and Memory 2 (Factor 5). These factor/subscale names were based on item content and previous subscale names applied by Chelune and Lehman (1986).

TABLE 2.

Varimax Rotated Factor Loadings for PAOFI Items With Factor Loadings .0.4

PAOFI Item Factor 1 Factor 2 Factor 3 Factor 4 Factor 5
Memory 1
 #1 Memory .590
 #2 Memory .556
 #3 Memory .428
 #4 Memory .517
 #5 Memory .425
 #14 Language & Communication .592
 #15 Language & Communication .549
Memory 2
 #6 Memory .465
 #7 Memory .664
 #8 Memory .646
 #9 Memory .488
Language & Communication
 #11 Language & Communication .666
 #12 Language & Communication .640
 #16 Language & Communication .562
 #17 Language & Communication .669
Sensorimotor
 #18 Use of Hands .416
 #19 Use of Handsa .338
 #20 Sensory-Perceptual .764
 #21 Sensory-Perceptual .785
 #22 Sensory-Perceptual .400
Cognitive/Intellectual Functions
 #23 Higher Level Cognitive & Intellectual Functions .512
 #24 Higher Level Cognitive & Intellectual Functions .499
 #26 Higher Level Cognitive & Intellectual Functions .440
 #27 Higher Level Cognitive & Intellectual Functions .410
 #28 Higher Level Cognitive & Intellectual Functions .652
 #29 Higher Level Cognitive & Intellectual Functions .751
 #30 Higher Level Cognitive & Intellectual Functions .659
 #31 Higher Level Cognitive & Intellectual Functions .751
Total Variance Explained (54.0%) 31.6% 7.2% 5.9% 4.9% 4.4%
Open in a new tab

Note. PAOFI = Patient Assessment of Own Functioning Inventory.

a

Item #19 (Use of Hands) is included with the other sensorimotor items loading onto Factor 4 because of the importance of Item #19’s content area in this population. Cross-loaded and dropped items: #10, #13, and #25.

All items previously identified by Chelune and Lehman (1986) as measures of higher level cognitive and intellectual functions loaded under Factor 1 with the exception of item #25. Item #25 failed to load under any factor based on the criterion that factor loadings should be higher than .40 to be considered relevant. As a result, item #25 was omitted from the final factor structure. Thus, the higher level cognitive and intellectual factor of this study contained one less item than the factor identified by Chelune and Lehman.

Items that formed the language and communication subscale of Chelune and Lehman (1986) loaded on two separate factors in this analysis. Although four language and communication items loaded together under Factor 2, three different language and communication items (#10, #14, and #15) loaded with memory items under Factor 3. In addition, language and communication item #13 failed to load under any factor (no factor loadings ≥0.40) and was therefore eliminated from the final factor structure. Evaluation of item content for the language and communication items (#10, #14, and #15) loading under Factor 3 suggested that items #14 and #15 were conceptually related to the memory domain. Therefore, items #14 and #15 were retained under Factor 3. In contrast, item #10 was omitted from the final factor structure.

Items from the original memory subscale also loaded on two separate factors, suggesting that the memory items may address different memory domains/dimensions. Five of the memory items loaded under Factor 3 (along with the three language and communication items) and four different memory items loaded under Factor 5. Although one memory item (#3) cross-loaded onto Factor 2, evaluation of item #3’s content resulted in the placement of this item with the other memory items under Factor 3. The split of the memory items into two separate factors is consistent with the finding of Chelune and Lehman (1986).

The use of hands items and the sensory-perceptual items loaded together under Factor 4. This finding is again consistent with Chelune and Lehman (1986), who also found that the hands and sensorimotor items clustered together to form one factor. Although #19 (left handed tasks) had a factor loading less than 0.40, it was included in the final factor structure. Given that cancer treatment with taxane therapy can cause peripheral neuropathies, evaluation of item #19 is important in this patient population.

In summation, the expectation of finding a five-factor structure was upheld; however, items did not load on the factors in the same manner as the Chelune and Lehman (1986) study. Results that were dissimilar to the Chelune and Lehman study included dropping Item #25 from the higher level cognitive and intellectual factor, and reducing the language and communication factor to four items. The remaining language and communication items were placed on one of the two memory factors. Results that were consistent with Chelune and Lehman included the split of the memory items into two factors, and the combination of sensorimotor and hands items into one factor.

Reliability and Construct Validity Assessment Results

Reliability

PAOFI subscale reliabilities, as computed using Cronbach’s alpha, were α = .818 (Memory 1), α = .744 (Memory 2), α =.792 (Language & Communication), α = .572 (Sensorimotor), and α = .883 (Cognitive/Intellectual Functions). According to Nunnally and Bernstein (1994), internal consistencies should reach .800 to be considered adequate. All subscales with the exception of Sensorimotor were near or more than the .8 threshold.

Construct Validity

Table 3 reports the correlations of the PAOFI subscales extracted from the current EFA with the SF-36 and POMS fatigue subscales, respectively. These correlational results provided evidence to determine the accuracy of our hypotheses. The results are as follows:

TABLE 3.

Correlations of PAOFI Subscales With POMS Fatigue and SF36 Subscales

Subscale Memory 1 Memory 2 Language & Communication Sensorimotor Cognitive/Intellectual Functioning
POMS fatigue .243** .237** .196** .283** .386**
SF36 Physical −.210** −.239** −.233** −.323** −.249**
SF36 Mental −.299** −.307** −.202** −.257** −.394**
Open in a new tab

Note. PAOFI = Patient Assessment of Own Functioning Inventory; POMS = Profile of Moods States; SF36 = Medical Outcome Study Short Form-36.

**

p < .001.

  1. A decrease in perceived cognitive functioning will be associated with a decline in physical and psychological functioning. An examination of PAOFI subscale correlations indicated that the Cognitive/Intellectual Functioning subscale was negatively correlated with the Mental Health component of the SF-36 (r = −.394), indicating high scores on the PAOFI (low cognitive functioning) were associated with low on the SF-36. This finding supports our hypothesis.

  2. The correlations of any emergent PAOFI subscales that assess cognitive or intellectual functioning will relate moderately or strongly to the psychological subscale of the SF-36, whereas PAOFI subscales that assess sensorimotor functioning will be moderately or strongly related to the SF-36 physical functioning subscale. According to Table 3, the Sensorimotor subscale was significantly correlated with SF-36 Physical component (r = −.323). Although this correlation was lower than we expected, we noticed that it was higher than the correlation between the Sensorimotor subscale and the SF-36 Mental component (r = −.257). The strongest correlation occurred between the Cognitive/Intellectual Functioning subscale and the Mental Health component of the SF-36 (r = −.394), which was supportive of the hypothesis.

  3. A decrease in perceived cognitive functioning (high PAOFI score) will be related to an increase in fatigue (high POMS score). All correlations between the PAOFI and POMS fatigue subscale were positive, indicating that an increase in fatigue led to a decrease in perceived cognitive functioning. This finding supported our hypothesis and was consistent with previous findings (Iconomou et al., 2004; Kibiger et al., 2003; Poppelreuter et al., 2004).

DISCUSSION

Assessment of perceived cognitive function in women with early-stage breast cancer prior to adjuvant therapy, as measured by the PAOFI, may provide valuable information regarding the impact of cancer and cancer treatment in this patient population. A reliable and valid self-report measure of cognitive function will provide important information about the scope of cognitive problems experienced by women with breast cancer, and provide valuable information about the functional impact of cancer and cancer treatment in this patient population.

Prior to this study, the reliability and validity of the PAOFI in women with breast cancer was not established. Thus, the purpose of this secondary analysis was to examine the psychometric properties of the PAOFI in a sample of postmenopausal women with early-stage breast cancer prior to adjuvant therapy.

Comparison of Results to Previous Psychometric Analyses in Other Populations

A factor analysis of the PAOFI items revealed an underlying factor structure that was similar to the underlying factor structure reported by Chelune and Lehman (1986). Results from both studies suggest that the memory domain is multidimensional, the higher level cognitive and intellectual functions (executive function) domain is unidimensional, and the general sensorimotor domain reflects both the use of hands items and sensory-perceptual items. In both studies, items from the rationally grouped memory subscale loaded under two separate factors. As noted by Chelune and Lehman, the multidimensional memory domain includes information related to general memory/orientation and memory for specific information.

Although most of the PAOFI’s underlying factor structure was similar between the sample of women with breast cancer in the AIM study and Chelune and Lehman’s (1986) sample, the clustering of the language and communication items differed between the samples. In the study by Chelune and Lehman, the language and communication items loaded under one factor. In the current investigation, three language and communication items loaded with other memory items. Cognitive domains are integrated with one another so that decline in function in one domain can affect the function of another domain. For example, if one has difficulty paying attention to information presented to them, it is likely that they will also have difficulty remembering that information. The two original language and communication items that loaded onto the memory factor and were retained both have a memory component to them (e.g., thinking of names of things). Moreover, verbal memory is a cognitive domain that is highly sensitive to the hormonal changes that women with breast cancer experience as a consequence of therapy (Bender, Paraska, Sereika, Ryan, & Berga, 2001). Additional differences were found in the particular items that formed the five factor solution of the current investigation versus the prior findings of Chelune and Lehman. The factor structure uncovered in this study revealed that some items were not clustering within factors and could potentially be dropped from the scale. Item 13 (indistinct or improper pronunciation), which was originally placed in the language and communication scale, did not load on any factor in this analysis. Based on consultation with an expert in breast cancer research, it was determined unlikely that women with early-stage breast cancer would have difficulty with speech; therefore, this item was dropped from the scale. In addition, it was concluded that women with early-stage breast cancer would not experience cognitive issues at this level, and so Items 10 (difficulty understanding what is said) and 25 (confusion about whereabouts) were excluded from the final factor structure. Ultimately, differences between this study and the study conducted by Chelune and Lehman suggest that the same items may be measuring slightly different aspects of functioning in the two patient populations. We recommend that future researchers use discretion when deciding which items to include on the scales.

Reliability and Validity

Reliability

In addition to the evaluation of dimensionality, internal consistency of the PAOFI subscales was also assessed. Internal consistency reliability, as measured by Cronbach’s alpha, was >.74 in four of the five subscales. The internal consistency coefficient for the fifth subscale (sensorimotor) was <.60. The low reliability of the sensorimotor scale could possibly be attributed to Item #19 (left handed tasks), which had a low factor loading on the scale. Cronbach’s alpha is the degree to which items within a scale measure the same concept; therefore, if Item #19 is worded in a way that could indicate a different concept than the other items in the scale, it could decrease the internal consistency estimate. Because this item is important for this population, it should remain on the scale, but may need rewording.

Validity

Correlations of the emergent PAOFI subscales with the SF-36 were inversely related and weak to moderate, with stronger relationships found between the cognitive/intellectual functioning and SF-36 mental health and between the Sensorimotor and SF-36 physical component. These findings support hypotheses (a) and (b). The moderately strong relationships in the expected direction provide further evidence of validity for PAOFI scores. In addition, there was a positive, moderate significant relationship between the POMS fatigue and the PAOFI cognitive subscale, which aligns with findings of other studies and supports hypothesis (c). Similar findings of relationships between perceptions of cognitive problems and depression and anxiety and symptoms such as fatigue have been reported (Iconomou et al., 2004; Kibiger et al., 2003; Poppelreuter et al., 2004). In fact, it is common for patients’ perceptions of problems with cognitive function to be related to mood and fatigue rather than with scores on objective measures of cognitive function (Poppelreuter et al., 2004). These findings have potential implications for clinicians who may interpret perceived cognitive problems as indicative of impairments in cognitive function. But reported cognitive complaints from patients with cancer may point to other problems including depression, anxiety, or fatigue, and these problems should be pursued clinically as well.

Limitations

As with all research studies, several limitations were noted in this secondary analysis. First, several items failed to load on any factor. Although potential floor effects may have contributed to these findings, low item correlations may also explain the findings. Second, two PAOFI items (#3 and #10) had high factor loadings on more than one factor. Because ambiguous items may tap into more than one domain of cognitive function, evaluation of item content specificity may help to reduce cross-loading. Third, in an attempt to simplify factor interpretation, varimax was selected as the method of rotation. Because the domains of cognitive function are likely to be correlated, oblique rotation may have been a more appropriate method of factor rotation. Fourth, the use of a sample size less than 300 comprising mostly White subjects limited the potential to explore subgroups and confirm the factor structure. Finally, this study was limited to postmenopausal women. As such, results may not be generalizable to other breast cancer populations (e.g., younger premenopausal population), which warrants the need for assessment of the PAOFI’s validity and reliability in these populations.

Suggestions for Future Research

Although the sample size of the current investigation was adequate for EFA, it did not allow for the confirmation of the resulting factor structure. Future studies in larger, more diverse samples are needed to support these findings. Suggestions for methodological approaches include confirmatory factor analysis to confirm the factor structure and Item Response Theory to identify properties at the item level. An additional assessment of temporal stability is also needed to confirm that the PAOFI consistently measures perceived cognitive function over time. The results of the current investigation further suggest that future studies should tailor the PAOFI’s structure prior to its inclusion as a measure of perceived cognitive function in women with breast cancer.

CONCLUSIONS

The evaluation of the PAOFI in a sample of postmenopausal women with early-stage breast cancer prior to adjuvant therapy provided evidence of internal consistency reliability for four of the five subscales and construct validity for the PAOFI scores in this patient population. Ultimately, the inclusion of a valid and reliable subjective measure of cognition into nursing research and nursing practice could provide a more comprehensive assessment of the functional impact of cancer and cancer treatment in this population, potentially identifying underlying problems (e.g., depression, fatigue) that could be mitigated with further evaluation and treatment. Identification and treatment of such problems may improve the individual’s functional abilities and therefore increase their perceived quality of life.

Acknowledgments

NINR (T32NR009759; 1F31NR011379); National Cancer Institute (R01 CA107408).

Contributor Information

Mandy J. Bell, Magee-Womens Research Institute.

Lauren Terhorst, Community Care Behavioral Health Organization.

Catherine M. Bender, University of Pittsburgh School of Nursing.

References

  1. Ahles TA, Saykin AJ. Candidate mechanisms for chemotherapy-induced cognitive changes. Nature Reviews Cancer. 2007;7(3):192–201. doi: 10.1038/nrc2073. http://dx.doi.org/10.10381/nrc2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Ahles TA, Saykin AJ, McDonald BC, Furstenberg CT, Cole BF, Hanscom BS, Kaufman PA. Cognitive function in breast cancer patients prior to adjuvant treatment. Breast Cancer Research and Treatment. 2008;110(1):143–152. doi: 10.1007/s10549-007-9686-5. http://dx.doi.org/10.1007/s10549-007-9686-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. Standards for educational and psychological testing. Washington, DC: American Educational Research Association; 1999. [Google Scholar]
  4. Bartlett MS. Tests of significance in factor analysis. British Journal of Psychology (Statistical Section) 1950;3:77–86. [Google Scholar]
  5. Bender CM, Pacella ML, Sereika SM, Brufsky AM, Vogel VG, Rastogi P, Ryan CM. What do perceived cognitive problems reflect? The Journal of Supportive Oncology. 2008;6:238–242. [PMC free article] [PubMed] [Google Scholar]
  6. Bender CM, Paraska KK, Sereika SM, Ryan CM, Berga SL. Cognitive function and reproductive hormones in adjuvant therapy for breast cancer: A critical review. Journal of Pain & Symptom Management. 2001;21(5):407–424. doi: 10.1016/s0885-3924(01)00268-8. http://dx.doi.org/10.1016/S0885-3924(01)00268-8. [DOI] [PubMed] [Google Scholar]
  7. Bender CM, Sereika SM, Berga SL, Vogel VG, Brufsky AM, Paraska KK, Ryan CM. Cognitive impairment associated with adjuvant therapy in breast cancer. Psycho-Oncology. 2006;15:422–430. doi: 10.1002/pon.964. http://dx.doi.org/10.1002/pon.964. [DOI] [PubMed] [Google Scholar]
  8. Bender CM, Sereika SM, Brufsky AM, Ryan CM, Vogel VG, Rastogi P, Berga SL. Memory impairments with adjuvant anastrozole versus tamoxifen in women with early-stage breast cancer. Menopause. 2007;14(6):995–998. doi: 10.1097/gme.0b013e318148b28b. http://dx.doi.org/10.1097/gme.0b013e318148b28b. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Bryant FB, Yarnold PR. Principal components analysis and exploratory and confirmatory factor analysis. In: Grimm LG, Yarnold PR, editors. Reading and understanding multivariate analysis. Washington, DC: American Psychological Association Books; 1995. pp. 99–136. [Google Scholar]
  10. Burdick KE, Endick CJ, Goldberg JF. Assessing cognitive deficits in bipolar disorder: Are self reports valid? Psychiatric Research. 2005;136:43–50. doi: 10.1016/j.psychres.2004.12.009. http://dx.doi.org/10.1016/j.psychres.2004.12.009. [DOI] [PubMed] [Google Scholar]
  11. Castellon SA, Ganz PA, Bower JE, Petersen L, Abraham L, Greendale GA. Neurocognitive performance in breast cancer survivors exposed to adjuvant chemotherapy and tamoxifen. Journal of Clinical and Experimental Neuropsychology. 2004;26(7):955–969. doi: 10.1080/13803390490510905. http://dx.doi.org/10.1080/13803390490510905. [DOI] [PubMed] [Google Scholar]
  12. Chaytor N, Schmitter-Edgecombe M. The ecological validity of neuropsychological tests: A review of the literature on everyday cognitive skills. Neuropsychology Review. 2003;13(4):181–197. doi: 10.1023/b:nerv.0000009483.91468.fb. [DOI] [PubMed] [Google Scholar]
  13. Chelune GJ, Lehman RAW. Neuropsychological and personality correlates of patients’ complaints of disability. In: Goldstein G, Tarter RE, editors. Advances in clinical neuropsychology. New York, NY: Plenum Press; 1986. pp. 95–118. [Google Scholar]
  14. Costello AB, Osborne JW. Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research, and Evaluation. 2005;10(7):1–8. [Google Scholar]
  15. Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16(3):297–334. [Google Scholar]
  16. Cureton EE, Mulaik SA. The weighted varimax rotation and promax rotation. Psychometrika. 1975;40(2):183–195. [Google Scholar]
  17. Fabrigar LR, Wegener DT, MacCallum RC, Strahan EJ. Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods. 1999;4(3):272–299. [Google Scholar]
  18. Gorsuch RL. Common factor-analysis versus component analysis—Some well and little known facts. Multivariate Behavioral Research. 1990;25(1):33–39. doi: 10.1207/s15327906mbr2501_3. [DOI] [PubMed] [Google Scholar]
  19. Hatcher L. A step-by-step approach to using the SAS system for factor analysis and structural equation modeling. Cary, NC: SAS Institute; 1994. Focus on the CALIS procedure. [Google Scholar]
  20. Iconomou G, Mega V, Koutras A, Iconomou AV, Kalofonos HP. Prospective assessment of emotional distress, cognitive function, and quality of life in patients with cancer treated with chemotherapy. Cancer. 2004;101(2):404–411. doi: 10.1002/cncr.20385. [DOI] [PubMed] [Google Scholar]
  21. Jassal SV, Devins GM, Chan CT, Bozanovic R, Rourke S. Improvements in cognition in patients converting from thrice weekly hemodialysis to nocturnal hemodialysis: A longitudinal pilot study. International Society of Nephrology. 2006;70:956–962. doi: 10.1038/sj.ki.5001691. http://dx.doi.org/10.1038/sj.ki.5001691. [DOI] [PubMed] [Google Scholar]
  22. Kaiser HF. An index of factorial simplicity. Psychometrika. 1974;39:32–36. [Google Scholar]
  23. Kibiger G, Kirsh KL, Wall JR, Passik SD. My mind is as clear as it used to be: A pilot studies illustrating the difficulties of employing a single-item subjective screen to detect cognitive impairment in outpatients with cancer. Journal of Pain and Symptom Management. 2003;26(2):705–715. doi: 10.1016/s0885-3924(03)00237-9. http://dx.doi.org/10.1016/So885-3924(03)00237-9. [DOI] [PubMed] [Google Scholar]
  24. McHorney CA, Ware JE, Jr, Raczek AE. The MOS SF-36-item short form health survey (SF-36) II. Psychometric and clinical tests of validity in measuring physical and mental health constructs. Medical Care. 1993;31(3):247–263. doi: 10.1097/00005650-199303000-00006. [DOI] [PubMed] [Google Scholar]
  25. McNair D, Lorr M, Droppleman L. EITS Manual for the Profile of Mood States. San Diego, CA: Education and Industrial Testing Service; 1981. [Google Scholar]
  26. Nunnally JC, Bernstein IH. Psychometric theory. 3. New York, NY: McGraw-Hill; 1994. [Google Scholar]
  27. Pett MJ, Lackey NR, Sullivan JJ. Making sense of factor analysis: The use of factor analysis for instrument development in health care research. Thousand Oaks, CA: Sage Publications; 2003. [Google Scholar]
  28. Poppelreuter M, Weis J, Külz AK, Tucha O, Lange KW, Bartsch HH. Cognitive dysfunction and subjective complaints of cancer patients: A cross-sectional study in a cancer rehabilitation centre. European Journal of Cancer. 2004;40(1):43–49. doi: 10.1016/j.ejca.2003.08.001. http://dx.doi.org/10.1016/j.ejca.2003.08.001. [DOI] [PubMed] [Google Scholar]
  29. Reitan RM, Wolfson D. Theoretical, methodological, and validational bases of the Halstead-Reitan Neuropsychological Test Battery. In: Golden G, Beers S, editors. Comprehensive handbook of psychological assessment (Volume 1)—Intellectual and neuropsychological assessment. Hoboken, NJ: John Wiley and Sons; 2004. pp. 105–132. [Google Scholar]
  30. Richardson-Vejlgaard R, Dawes S, Heaton RK, Bell MD. Validity of cognitive complaints in substance-abusing patients and non-clinical controls: The patient’s assessment of own functioning inventory (PAOFI) Psychiatric Research. 2009;169:70–74. doi: 10.1016/j.psychres.2008.06.018. http://dx.doi.org/10.1016/j.psychres.2008.06.018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Roehrich L, Goldman MS. Experience-dependent neuropsychological recovery and the treatment of alcoholism. Journal of Consulting and Clinical Psychology. 1993;61(5):812–821. doi: 10.1037//0022-006x.61.5.812. [DOI] [PubMed] [Google Scholar]
  32. Rourke SB, Halman MH, Bassel C. Neuropsychiatric correlates of memory-metamemory dissociations in HIV-infection. Journal of Clinical and Experimental Neuropsychology. 1999;21(6):757–768. doi: 10.1076/jcen.21.6.757.852. [DOI] [PubMed] [Google Scholar]
  33. Sepucha KR, Belkora JK, Chang Y, Cosenza C, Levin CA, Moy B, Lee CN. Measuring decision quality: Psychometric evaluation of a new instrument for breast cancer surgery. BMC Medical Informatics and Decision Making. 2012;12:51–61. doi: 10.1186/1472-6947-12-51. http://dx.doi.org/10.1186/1472-6947-12-51. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Shilling V, Jenkins V, Fallowfield L, Howell T. The effects of hormone therapy on cognition in breast cancer. The Journal of Steroid Biochemistry & Molecular Biology. 2003;86:405–412. doi: 10.1016/j.jsbmb.2003.07.001. http://dx.doi.org/10.1016/j.jsbmb.2003.07.001. [DOI] [PubMed] [Google Scholar]
  35. Streiner DL, Norman GR. Health measurement scales: A practical guide to their development and use. 2. Oxford, United Kingdom: Oxford University Press; 1995. [Google Scholar]
  36. Tabachnik BG, Fiddell LS. Using multivariate statistics. 5. Boston, MA: Allyn and Bacon; 2006. [Google Scholar]
  37. Wefel JS, Lenzi R, Theriault R, Buzdar AU, Cruickshank S, Meyers CA. ‘Chemobrain’ in breast carcinoma? A prologue. Cancer. 2004;101(3):466–475. doi: 10.1002/cncr.20393. [DOI] [PubMed] [Google Scholar]
  38. Wefel JS, Saleeba AK, Buzdar AU, Meyers CA. Acute and late onset cognitive dysfunction associated with chemotherapy in women with breast cancer. Cancer. 2010;116(14):3348–3356. doi: 10.1002/cncr.25098. http://dx.doi.org/10.1002/cncr.25098. [DOI] [PubMed] [Google Scholar]

RESOURCES