Structural and Reliability Analysis of a Patient Satisfaction with Cancer-Related Care Measure: A Multi-Site Patient Navigation Research Program Study

Abstract

Background

Patient satisfaction (PS) is an important outcome measure of quality of cancer care and one of the four core study outcomes of the National Cancer Institute (NCI) sponsored Patient Navigation Research Program (PNRP) to reduce race/ethnicity-based disparities in cancer care. There is no existing PS measure that spans the spectrum of cancer-related care.

Objective

Develop a Patient Satisfaction with Cancer Care (PSCC) measure that is relevant to patients receiving diagnostic/therapeutic cancer-related care.

Methods

We developed a conceptual framework, an operational definition of PSCC, and an item pool based on literature review, expert feedback, group discussion and consensus. The 35-item PSCC measure was administered to 891 participants from the multi-site NCI-sponsored PNRP. Principal components analysis (PCA) was conducted for latent structure analysis. Internal consistency was assessed using Cronbach coefficient alpha (α). Divergent analysis was performed using correlation analyses between the PSCC, the Communication and Attitudinal Self-Efficacy (CASE-Cancer), and demographic variables.

Results

The PCA revealed a one-dimensional measure with items forming a coherent set explaining 62% of the variance in PS. Reliability assessment revealed high internal consistency (α ranging from 0.95 to 0.96). The PSCC demonstrated good face validity, convergent validity and divergent validity as indicated by moderate correlations with subscales of the CASE-Cancer (all p_s < 0.01) and non-significant correlations with age, primary language, marital status, and scores on the Rapid Estimate of Adult Literacy in Medicine (REALM) Long Form (all p_s > 0.05).

Conclusion

The PSCC is a valid tool for assessing satisfaction for cancer-related care for this sample.

INTRODUCTION

Patient satisfaction reflects a core dimension of health care quality and patient-centered care.^1-3 Patient satisfaction indicates the extent to which patients’ health care experiences match their expectations.^4,5 The construct of patient satisfaction has been linked to health status, quality of life, adherence to recommended treatment and medical advice including cancer treatment, initiation of complaints, and patient-healthcare provider communication in the clinical dyad.^6-16 Patient satisfaction with care represents an important outcome measure for health care in general and cancer care, in particular.¹⁷

Patient satisfaction is one of the primary study outcomes of the National Cancer Institute (NCI) supported Patient Navigation Research Program (PNRP) to reduce disparities in cancer care for individuals from racial/ethnic minorities and lower socioeconomic groups. The PRNP involves nine independent research programs operating under cooperative agreements with the NCI Center to Reduce Cancer Health Disparities to evaluate the impact of patient navigation on outcomes among patients with cancer screening abnormalities or diagnosed cancer.¹⁸

While there are numerous patient satisfaction measures, including a number of measures related to cancer treatment, none of these measures span the spectrum of cancer-related care from screening to treatment of diagnosed cancer.^19-25 For example, the widely used EORTC-INPATSAT32 is designed to assess satisfaction with the inpatient cancer care while the FAMCARE assesses satisfaction among those with advanced cancer.^{20, 24}

In the present study, we aimed to develop a Patient Satisfaction with Cancer-related Care measure that had: a) sufficient breadth (i.e., addressing satisfaction with care during evaluation of screening abnormalities and treatment), b) addressed many of the challenges confronted by poor and minority individuals receiving cancer-related care, and c) was relevant for evaluation of care among both navigated and un-navigated patients.

MATERIALS AND METHODS

Development of the Patient Satisfaction with Cancer-related Care (PSCC)

The scale development team included investigators from different PNRP sites with content and technical expertise in clinical care of patients from diverse cultural and socioeconomic backgrounds, as well as measurement development and psychometrics. The team reviewed existing patient satisfaction measures, considered various domains of satisfaction (access/logistical, interpersonal/relational, communicational/informational, and coordination of care), selected and modified existing items for inclusion in the new PSCC scale. One additional item was administered only to participants with a confirmed diagnosis of cancer: “My treatment was explained in a way I could understand”.

Response Options and Scoring

Patients responded to each scale item on a 5-point Likert scale (“1 = Strongly Agree” to “5 = Strongly Disagree). A total scale score was obtained by adding scores on all items, with lower scores indicating higher satisfaction with cancer care.

Participants

The PNRP methods have been previously published.¹⁸ Briefly, the PNRP is cooperative program funded by NCI and the ACS to rigorously evaluate the role and benefits of patient navigation among participants with abnormal cancer screening findings or diagnosed cancer - breast, cervical, colorectal or prostate cancer – within nine largely racial/ethnic minorities and low-income communities across the country. Study design and type of cancer differ by participating site.

The satisfaction items were administered to a subsample of the 8075 participants in the PNRP. In all, 891 English fluent participants from the multi-site NCI-sponsored PNRP who completed the PSCC measure. Survey participants were similar in age, but more likely to be female, minority, lower income and less educated.

Procedures

Medical staff at the PNRP recruiting sites (e.g., clinics or hospitals) was informed about the study and referred eligible patients to speak with a trained research assistant (RA) or patient navigator about participating in the study. To minimize possible effects of low literacy, surveys were read out loud to participants in English.

Eligibility and Exclusion Criteria

Eligibility for the present study included having an abnormal breast, cervical, colorectal and prostate cancer cancer test finding or a new diagnosis of these cancers without any prior history of cancer treatment other than non-melanoma skin cancer.¹⁸

Additional Measures

Demographic characteristics

These included age, sex, race, ethnicity, primary language, income, education, marital status, and whether the patient received care related to evaluation of cancer screening abnormalities or treatment of cancer, and type of cancer being evaluated or treated (breast, cervical, colorectal or prostate).

Communication and Attitudinal Self-Efficacy (CASE-Cancer)

The CASE-Cancer is a psychometrically validated multi-dimensional measure (i.e., Understanding and participating in care, maintaining a positive attitude, seeking and obtaining information) of communication and attitude. Structural analysis of the CASE-Cancer revealed high internal consistency and construct validity.²⁶ Given overlap in constructs, we expected that the PSCC would correlate with the CASE-Cancer.

Data Analysis

Dimensionality analysis of the PSCC

Latent structural and psychometric validation analyses were conducted using SPSS version 17.0 statistical software package for Microsoft Windows. Data from our multi-site sample were randomly divided into two separate datasets (Sample 1, N₁ = 453; Sample 2, N₂ = 438) using SPSS. One dataset was used to test the latent structure of the PSCC and the second dataset was used to validate the said structure. We had a very large sample that facilitated calculation of reliable correlation coefficients for the PSCC. This approach is in accordance with guides on sample sizes for factor analysis/principal components analysis.^27-28 Additionally, the PCA solutions include many high variables markers and therefore could have facilitated stable and reliable estimates of correlation coefficients with even a smaller sample size.²⁹ Prior to conducting the principal components analysis (PCA), suitability of the data for dimensionality analysis was assessed using various criteria (e.g., examination of the correlation matrix for correlations of .30 and above). The PCA was conducted to reduce the data to a few components that could be more easily described. We performed an initial PCA, using Sample 1 data, without rotation to facilitate extraction and examination of meaningful components, based on eigenvalues and screeplot criteria, that more accurately describe the latent structure of the PSCC. The Kaiser-Meyer-Olkin value (KMO), an index of sampling adequacy, was used to determine the suitability of the data for dimensionality analysis.^{30, 31} Additionally, we examined the screeplot of eigenvalues to help determine the number of components to retain. We subsequently rotated the initial factor solution using the VARIMAX technique. Items from Sample 2 were also subjected to a PCA in order to replicate and test the evidence of the structure of the PCA obtained from Sample 1 through successive unconstrained exploratory procedures. We conducted similar PCA for Sample 2 (N₂) as described above for Sample 1 (N₁).

Measurement reliability analysis

Scale reliability assessment was conducted to determine the degree to which items of the PSCC represent a coherent set that measures the same underlying construct. Cronbach coefficient alpha was used as an index of internal consistency of the PSCC. Measurement reliability analysis was conducted separately for each Sample 1 and Sample 2.

RESULTS

The mean age of the analytic sample was 51 years (range 18 - 98 years). Most of the sample was female (approximately >80%) and included participants from diverse racial/ethnic backgrounds including White (43%), Black (32%), Hispanic/Latino (23%), Asian (1%), American Indian/Alaska Native (0.5%) and other (0.5%). Half of the sample reported only a high school education or less. Participants presented with abnormal test findings or diagnosis from various types of cancer, including approximately 64% breast, 11% cervix, 12% colorectal, 13% prostate and 0.5% multiple concurrent cancer sites. Detailed demographic and clinical characteristics of study participants are provided in Table 1. All participants provided informed consent for participation. The Institutional Review Board of all participating institutions approved this study.

Table 1.

Demographic and Clinical Characteristics of 891 Participants

	n	Mean (Std)
Age	843	51.43 (13.77)
	n	Percent
Cancer Site
Breast	572	64.2
Cervix	96	10.77
Colorectal	107	12.01
Prostate	112	12.57
Multiple concurrent cancer sites	4	0.45
Gender
Female	686	81.28
Male	158	18.72
Race/Ethnicity
White	360	43.22
Black/African American	266	31.93
Asian	9	1.08
American Indian/Alaska Native	4	0.48
Hispanic or Latino	190	22.81
Other	4	0.48
Primary Language
English	740	87.78
Spanish	87	10.32
Other	16	1.9
Birth Country
US	647	82.32
Other	139	17.68
Marital Status
Single/Never married	256	30.51
Married/living as married	339	40.41
Divorced/separated	190	22.65
Widowed	54	6.44
Education
8th grade or less	69	8.93
Some high school	106	13.71
High school diploma (including equivalency)	196	25.36
Some college/vocational after high school	182	23.54
Associate degree	58	7.5
College graduate	100	12.94
Graduate or professional degree	62	8.02
Household Income
Less than $10,000	219	30.85
$10,000 to $19,999	134	18.87
$20,000 to $29,999	88	12.39
$30,000 to $39,999	69	9.72
$40,000 to $49,999	38	5.35
$50,000 or more	162	22.82
Employment Status
No current employment	443	56.58
Part-time employment	106	13.54
Full-time employment	234	29.89
Health Insurance Coverage
Yes	681	83.15
No	138	16.85

Sample 1, N₁ – Testing of PSCC latent structure

Suitability for Factor Analysis (Sample 1, N₁)

Examination of the items correlation matrix revealed the presence of many correlation coefficients of .30 and higher. In addition, the KMO value was 0.95, exceeding the recommended value of 0.60.^{30, 31} The Bartlett's Test of Sphericity also reached statistical significance (χ² (378) = 7850.920; p = 0.001), which also supported the appropriateness of dimensionality analyses of the correlation matrix.³² Values were skewed toward favorable ratings mean coefficient of skewness, 1.45 (-2.2- -0.5).

Construct Validity (Sample 1, N₁)

The initial unrotated PCA revealed the presence of five components with eigenvalues exceeding one (λ > 1): 12.698, 1.734, 1.383, 1.087, and 1.081, which explained 45.35%, 6.19%, 4.94%, 3.88%, and 3.86% of the total cumulative variance (64.22 %) respectively. Inspection of the screeplot revealed a clear break after the second component. Catell's (1966) screeplot test and the eigenvalues criteria suggested that two components could be retained for further investigation.³³ The components matrix showed that approximately 82% of the items (the first 23 items) loaded on the first component, with factor or components loadings ranging from 0.51 to 0.86. Of these 23 items, five loaded on factors 3 to 5, with components loadings ranging from -0.31 to 0.44. Another set of five additional items loaded moderately to strongly on factors 2 to 4, with components loadings ranging from 0.33 to 0.92. These second set of five items seem related primarily to time waiting at the hospital, transportation and money concerns, and explication of medical tests and health condition.

Subsequently, we removed items with moderate loadings on multiple components because of plausible overlapping contributions. We also decided to not include components defined by just one or two variables since such components are unstable, generally account for a very small percentage of the variance, and are difficult to correctly interpret.³⁴ Based on these criteria, we ended up with a one-dimensional 18-item PSCC measure as indicated by a single-component structure with items forming a coherent set that explained 62% of the variance in patient satisfaction with cancer-related care (Table 2). The results of our psychometric analyses support the validity of PSCC for this sample.^{34, 35}

Table 2.

Component Loadings for Sample 1 (N₁ = 453) and Sample 2 (N₂ = 438): Correlations between Individual items and the Underlying Component.

Patient satisfaction with Cancer Care	Component Loadings
Scale Items	Eigenvalue (λ) 14.58	Eigenvalue (λ) 15.25
	Sample 1	Sample 2
1. I felt that my health concerns were understood.	.782	.756
2. I felt that I was treated with courtesy and respect.	.762	.739
3. I felt included in decisions about my health.	.816	.751
4. I was told how to take care of myself.	.741	.725
5. I felt encouraged to talk about my personal health concerns.	.758	.715
6. I felt I had enough time with my doctor.	.774	.790
7. My questions were answered to my satisfaction.	.805	.815
8. Making an appointment was easy.	.549	.577
9. I knew what the next step in my care would be.	.670	.745
10. I feel confident in how I deal with the health care system.	.744	.791
11. I was able to get the advice I needed about my health issues.	.817	.851
12. I knew who to contact when I had a question.	.695	.747
13. I received all the services I needed.	.798	.780
14. I am satisfied with the care I received.	.855	.829
15. The doctors seemed to communicate well about my care.	.830	.792
16. I received high quality care from my regular doctor.	.723	.752
17. I received high quality care from my specialists.	.811	.803
18. My regular doctor was informed about the results of the tests I got.	.541	.630

Extraction Method: Principal Components Analysis

Sample 2, N₂ – Validation of PSCC latent structure

Suitability for Factor Analysis (Sample 2, N₂)

We tested the emergent structure of the data in Sample 1 by conducting another PCA on data from Sample 2. This approach is based on the notion that successful replication through successive unconstrained exploratory procedures will substantiate the underlying structure of the PSCC beyond any constrained confirmatory procedure. Similar to Sample 1, examination of the correlation matrix for Sample 2 revealed the presence of many correlation coefficients of .30 and higher. In addition, the KMO value was 0.95, exceeding the recommended value of 0.6.^{30, 31} The Bartlett's Test of Sphericity also reached statistical significance (χ² (378) = 7853.56; p = 0.001), supporting the appropriateness of dimensionality analyses of the correlation matrix.³²

Construct Validity (Sample 2, N₂)

The initial unrotated PCA revealed the presence of five components with eigenvalues exceeding one (λ > 1): 13.12, 1.76, 1.39, 1.20, and 1.03, which explained 46.87 %, 6.31 %, 4.96%, 4.28%, and 3.66% of the total cumulative variance (66.09%), respectively. Inspection of the screeplot revealed a clear break after the second component. Catell's (1966) screeplot test and the eigenvalues criteria supported the retention of two components for further investigation.³³ Similar to the PCA for Sample 1, the components matrix showed that approximately 82% of the items (the first 23 items) loaded on the first component, with factor or components loadings ranging from 0.48 to 0.86. Of these 23 items, eight loaded on factors 2, 4, and 5, with components loadings ranging from -0.41 to 0.47. Another set of five additional items loaded moderately to strongly on factors 2 to 5, with components loadings ranging from -0.62 to 0.68. Similar to the structure of the PSCC in Sample 1, the second set of five items in Sample 2 seemed to involve time waiting at the hospital, transportation and money concerns, and explication of medical tests and health condition. As previously described for Sample 1, we removed items with moderate loadings on multiple components (two or more) because of issues related to overlapping contribution in Sample 2. Just as in sample 1, we did not include components defined by just one or two variables since such components are unstable, account for a small percentage of the variance, and are difficult to reliably interpret.³⁴ Based on these criteria, we also ended up with an 18-item one-dimensional measure for Sample 2 as indicated by a one-component structure (Tables 2). Results of our structural analyses supported the use of the PSCC as a valid measure for this sample and, more importantly confirmed the underlying structure of the PSCC through successive unconstrained exploratory procedures.^{34, 35}

PSCC Reliability and Convergent and Divergent Validity

Scale reliability assessment was conducted for the 18-item PSCC

Internal consistency – degree to which items that make up this scale represent a coherent set that measures the same underlying construct – was evaluated using Cronbach's coefficient alpha. The results showed Cronbach coefficients alphas of approximately 0.95 and 0.96 based on standardized items for the PSCC for Sample 1 and Sample 2, respectively. The scale reliability assessment supported the use of the PSCC as a reliable tool of satisfaction with cancer care for this sample.³⁶

Convergent and Divergent Validity

The PSCC total score for Sample 1 (N₁ = 453) correlated with subscales of the CASE-Cancer (Understanding and participate in care (r = 0.40, p = .001), and Seek and obtain information (r = 0.32, p = .004)). The results, however, did not reveal any statistically significant correlation between the PSCC total score and age, primary language, marital status, and scores on the REALM long form (all p-values > .05). Likewise, the PSCC total score for Sample 2 (N₂ = 438) positively correlated with subscales of the CASE-Cancer: Understanding and participate in care (r = 0.51, p = .001), Maintain a positive attitude (r = .30, p = .01), and Seek and obtain information (r = 0.39, p = .001). Again, the analysis revealed no statistically significant correlation between the PSCC total score and age, primary language, or marital status (all p-values > .05). Convergent and divergent validity analyses examined the degree to which the PSCC correlates with measures that assess related constructs (e.g., the “Understanding and participate in care” and the “”Seek and obtain information subscales of the CASE-Cancer) and differ from measures or indices of other unrelated constructs (e.g., age, primary language, or marital status), hence confirming that the items of the PSCC formed a coherent set that assess the specific construct of patient satisfaction with the cancer-related care they received.

DISCUSSION

We designed the PSCC to be a simple and easy to administer tool to assess satisfaction with cancer-related care for individuals from diverse cultural and socioeconomic populations. An important goal for developing the PSCC was to ensure the measure assesses experiences common to all patients regardless of whether or not they were navigated. This approach is expected to ensure the applicability and relevance of this measure to people from comparable racial, ethno-cultural and socioeconomic backgrounds.

The results of our structural analysis and psychometric validation revealed a parsimonious and reliable one-component solution for the PSCC. This measure provides a milieu-specific patient-oriented approach for assessing perceived relevance and satisfaction with cancer care for individuals from diverse racial, ethno-cultural and socioeconomic backgrounds. The PSCC demonstrates high construct validity. The degree to which the items of the PSCC constitutes a coherent set that assess the underlying construct of patient satisfaction with cancer care was demonstrated by high indices of internal consistency and reliability.

The PSCC differs from previous generic scales in that it focuses on satisfaction with cancer-related care rather than the broader concept of health care in general or the narrower concept of cancer treatment for a particular, cancer, disease stage or location (hospital or ambulatory).^37-40 The PSCC addresses the broad domain of cancer-related care including diagnostic testing in addition to treatment rather than focusing on particular or specific aspects of cancer care.^41-43

The limitations of these findings merit comment. First, we adapted and modified items from existing instruments, but we did not conduct cognitive interviewing.⁴⁴ However, a pilot study of the questionnaire revealed no problem that would have indicated a need to modify questionnaire items to help improve participants’ understanding or interpretation of the items. In addition, the PSCC scale was administered orally in order to minimize effects of low literacy; therefore, it is not certain that similar results would be obtained from participants who self-administer the scale.

Second, consistent with previous satisfaction measures, we observed significant skewing or tendency towards the higher end of satisfaction.⁴⁵ Whether this represents truly favorable experiences or reflects low expectations is unknown.³ We did not specifically query patients about expectations. For many patients, their abnormal screening/diagnosis may have been their first experience with cancer-related care. Thus, they may have used a priori general healthcare experiences to form their expectations, which could explain the trend towards the higher end of reported satisfaction. This could also be representing a social desirability response bias related to interview format. ⁴⁶ Further studies are needed to help determined if this finding will remain if patients responded anonymously and whether this ceiling effect will affect the sensitivity of the scale.

Furthermore, about 80 percent of the sample were women. Further studies are needed to confirm generalizability of the PSCC to men. Also, the PSCC accounted for 60 percent of the variance in patient satisfaction. Follow-up studies are needed to identify plausible factors that could account for the unexplained portion of this variance.

Lastly, we did not assess the responsiveness of the measure to change and/or how well it matches clinical impression. That is, we do not know how well the PSCC will capture differences in health care processes. Some aspects of care such as interpersonal processes may have a much greater impact on satisfaction than technical aspects.^46-48

The strengths of the study include psychometric assessment of the PSCC measure with medically underserved and underrepresented individuals from racial/ethnic minorities and lower socioeconomic populations across different types of health care systems (e.g. community health centers, Veteran Administration, and University and community-based oncology practices). The development of the PSCC represents an initial attempt to develop and assess the validity and reliability of a context-specific measure of satisfaction with cancer-related care that is applicable to underserved and traditionally underrepresented racial/ethnic minorities and lower income individuals who face a variety of barriers to cancer care.

Validation of this PSCC measure will facilitate examination of the impact of patient navigation on cancer-related care.¹² Further studies should examine the predictive validity of the PSCC for treatment-related outcomes within longitudinal research settings. Our analyses showed divergent and convergent capabilities of the PSCC. Additional studies that examine divergent and convergent characteristics of the PSCC with other relevant psychometrically valid and reliable health measures will provide evidence of the strength of the PSCC and further inform the underlying structure and validity of this measure for cancer patients. This scale, the PSCC, should prove useful for evaluation of PN not only in the participating nine sites of the NCI funded Patient Navigation Research Program, but in other cancer navigation programs as well.