Psychometric Validation and Reliability Analysis of a Spanish version of the Patient Satisfaction with Cancer-Related Care Measure: A Patient Navigation Research Program Study

Abstract

Background

Patient satisfaction (PS), a key measure of quality of cancer care, is a core study outcome of the multi-site National Cancer Institute (NCI) Patient Navigation Research Program (PNRP). Despite large numbers of underserved monolingual Spanish speakers (MSS) residing in the United States, there is no validated Spanish measure of PS that spans the whole spectrum of cancer-related care. The present study reports on the cross-validation of the Patient Satisfaction with Cancer Care (PSCC) measure for Spanish (PSCC-Sp) speakers receiving diagnostic and therapeutic cancer-related care.

Methods

Original PSCC items were professionally translated and back translated to ensure cultural appropriateness, meaningfulness and equivalence. Then, the resulting 18-item PSCC-Sp measure was administered to 285 MSS. We evaluated latent structure and internal consistency of the PSCC-Sp using principal components analysis (PCA) and Cronbach coefficient alpha (α). We used correlation analyses to demonstrate divergence and convergence of the PSCC-Sp with a Spanish version of the Patient Satisfaction with Interpersonal Relationship with Navigator (PSN-I-Sp) measure and patients’ demographics.

Results

The PCA revealed a coherent set of items that explicates 47% of the variance in PS. Reliability assessment demonstrated that the PSCC-Sp had high internal consistency (α = .92). The PSCC-Sp demonstrated good face validity, and convergent and divergent validities as indicated by moderate correlations with the PSN-I-Sp (p = 0.003) and non-significant correlations with marital status and household income (all p_s > 0.05).

Conclusion

The PSCC-Sp is a valid and reliable measure of PS and should be tested in other MSS populations.

INTRODUCTION

Patient satisfaction (PS) is an essential element of health care quality and patient-centered care.^1–3 PS is often used to systematically measure the extent to which patients’ health care experiences match their expectations.^4–5 Previous studies have reported significant relationships between PS and health status, quality of life, adherence to recommended medical advise and treatment, initiation of complaints, and patient-healthcare provider communication.^6–16 PS is a critical outcome measure for health care in general, and holds a particular promise for cancer care.¹⁷

Patient satisfaction is one of the four core study outcomes of the Patient Navigation Research Program (PNRP) to reduce cancer disparities for individuals from underserved racial-ethnic minorities and lower socioeconomic populations. The PNRP is a collaborative research involving nine independent research programs operating under cooperative agreements with the National Cancer Institute (NCI) Center to Reduce Cancer Health Disparities and the American Cancer Society (ACS). The NCI funded eight PNRP sites and ACS funded one additional PNRP site (i.e., The Ohio State University). The overarching goal of the PRNP was to evaluate the impact of patient navigation on cancer care outcomes among patients with cancer screening abnormalities or diagnosed cancer.¹⁸ We carefully examined available patient satisfaction measures from the medical literature and found no reliable measure that covers cancer-related care on a continuum ranging from screening to definitive diagnostic resolution and completion of the cancer treatment process. We subsequently developed the Patient Satisfaction with Cancer-related Care (PSCC) and this Spanish version (PSCC-Sp) to span the whole spectrum of cancer-related care for individuals from underserved racial-ethnic minority and lower socioeconomic populations who had either an abnormal cancer screening or a definitive cancer diagnosis.^19–25 The Hispanic population in the United States is rapidly growing, accounting for a significantly larger percentage of the total populations of major cities in the United States. The Hispanic population in the United States has increased by 57.9%; from 22.4 million in 1990 to 35.3 million in 2000.²⁶ This rapid growth is expected to continue and the number of individuals of Hispanics heritage is expected to reach or surpass 102,560 million (i.e., approximately 24.4% of the total projected United States population) by the year 2050.²⁷ Hispanics also constitute one of the largest underserved and underrepresented racial-ethnic minority groups in the United States, and generally struggled with various linguistic, cultural, and socioeconomics barriers to timely access equitably beneficial and reliable health care.

Despite the growing presence of Hispanics in major cities across the United States, culturally appropriate measures that are developed and validated for Spanish speakers are lacking. This dearth of psychometrically validated Spanish measures is especially noticeable in key areas of cancer prevention and control research and in cancer clinical care settings. To address the paucity of psychometrically validated and reliable evaluation tools for Spanish speakers, we developed and assess the latent structure and validity of Spanish PS measures for the PNRP.

This manuscript reports the development and validation of the PSCC-Sp to address satisfaction with cancer care for Spanish speaking patients who had either an abnormal cancer test screenings or a definitive cancer diagnosis. Similarly to the English version (e.g. PSCC), we designed the PSCC-Sp to address satisfaction with care and many of the challenges confronted by individuals from underserved racial-ethnic minorities and lower socioeconomic groups seeking and receiving cancer-related care. We specifically developed the PSCC-Sp to be relevant to both navigated and non-navigated patients.

MATERIALS AND METHODS

Participants

A total of 285 patients with verbal fluency in Spanish from the multi-site NCI-sponsored PNRP completed the PSCC-Sp and other study measures in Spanish. The purpose of the PNRP was to rigorously examine the effect of patient navigation on diagnostic or therapeutic care for breast, cervical, colorectal or prostate cancer within a largely racial-ethnic minorities and lower income populations.¹² Detailed socio-demographic characteristics of the sample are provided in Table 1. Race and ethnicity were self-reported by study participants.

Table 1.

Participants’ Demographic and Clinical Characteristics

	n	Mean (Std)
Age	285	41.0 (11.8)
	n	Percent
Cancer Site
Breast	208	73.0%
Cervix	68	23.9%
Colorectal	9	3.1%
Gender
Female	285	100.0%
Male	0	0.0%
Race/Ethnicity
White Hispanic	278	97.5%
Black Hispanic	5	1.8%
Other Hispanic	2	0.7%
Primary Language
English	0	0.0%
Spanish	276	96.8%
Other	9	3.2%
Birth Country
US	10	3.9%
Other	246	96.1%
Marital Status
Single/Never married	79	28.3%
Married/living as married	161	57.7%
Divorced/separated	33	11.8%
Widowed	6	2.2%
Education
Less than high school	160	64.8%
High school diploma (including  equivalency)	46	18.6%
More than high school	41	16.6%
Household Income
Less than $10,000	100	38.6%
$10,000 to $19,999	98	37.8%
$20,000 to $29,999	45	17.4%
$30,000 to $39,999	10	3.9%
$40,000 to $49,999	5	1.9%
$50,000 or more	1	0.4%
Employment Status
No current employment	135	49.5%
Part-time employment	62	22.7%
Full-time employment	76	27.8%
Health Insurance Coverage
Yes	148	52.7%
No	133	47.3%

Inclusion and Exclusion Criteria

Participants included in this study had an abnormal breast, cervical, colorectal and prostate cancer test finding or a new histologically confirmed diagnosis of one of the aforementioned types of cancers. Patients were eligible to participate in the study from their initial abnormal screening test finding until a definitive diagnosis or 365 days. Participants were excluded from the PNRP study if they had any history of treatment for primary cancer of the breast, cervical, colorectal or prostate, or if they had a history of prior patient navigation experience. This exclusion criterion was implemented to help prevent possible effects of confounders that could lead to an erroneous conclusion of the effects of the patient navigation intervention.

Procedures

Similarly to the PSCC, medical staff at the PNRP recruiting sites (e.g., clinics or hospitals) was informed about the study and referred eligible Spanish speaking patients to meet with a trained research assistant (RA) to determine eligibility to participate in the study. Only participants from the navigation arm of the PNRP study completed the PSN-I-Sp. Some study sites such as Boston, Massachusetts consented patients via telephone. To minimize possible effects of low literacy, surveys were read out loud to participants in Spanish. Study questionnaires were administered either in person or over the telephone. Overall, participants complete the PSCC-Sp in about 10 minutes. On average, participants also received $25 in cash of in gift card(s) to help defray their cost of participation in the study.

Validation of the Patient Satisfaction with Cancer-related Care-Spanish (PSCC-Sp)

A multidisciplinary measurement development team, consisting of members from each PNRP site, completed the latent structure and reliability analysis of the PSCC-Sp. The measurement development team of the PSCC and PSCC-Sp included investigators from different PNRP sites with content and technical expertise in psychometrics, and in research and clinical care of patients from diverse multicultural and socioeconomic backgrounds. The team reviewed existing patient satisfaction measures, considered various domains of satisfaction (i.e., access/logistical, interpersonal/relational, communicational/informational, and coordination of care) and selected, at times modifying, existing items for inclusion in the new PSCC scale. One additional item was administered only to participants with a confirmed diagnosis of cancer: “My treatment was explained in a way I could understand”. Items of the PSCC were professionally translated into Spanish and then back translated into English to ensure that the resulting PSCC-Sp measure was culturally appropriate, relevant and semantically equivalent to the English PSCC measure.

Previous studies reported that the PSCC is one-dimensional measure with items forming a coherent that explained 62% of the variance in patient satisfaction with cancer care, high internal consistency as indicated by Cronbach alphas α ranging from 0.95 to 0.96, and appropriate face validity, and adequate convergent and divergent validities.²⁸

Response Options and Scoring

Similarly to the PSCC, Patients responded to each item of the PSCC-Sp on a 5-point Likert scale (“1 = Strongly Agree” to “5 = Strongly Disagree”). We added scores on all items of the PSCC-Sp to obtain a total scale score. Lower scores on the PSCC-Sp indicate higher satisfaction with cancer care.

Additional Measures

Demographic and clinical characteristics

Demographic characteristics included age, sex, race, ethnicity, primary language, income, education, and marital status. Clinical characteristics included data on whether the patient received care related to evaluation of cancer screening abnormalities or treatment of cancer, and type of cancer being evaluated or treated (breast, cervical, colorectal or prostate).

Patient Satisfaction with Interpersonal Relationship with Navigator (PSN-I-Sp)

The 9-item PSN-I-Sp is a Spanish version of the PSN-I designed to assess patients’ perception of their relationships with their navigators. The PSN-I is a one-dimensional measure with very good psychometric properties as indicated by internal consistency index of Cronbach alpha = 0.98. Principal components analysis of the PSN-I showed that the PSN-I-Sp measure explained 85.0% of the variance in patient satisfaction with navigator.²⁹

Data Analysis

Dimensionality analysis of the PSCC-Sp

We evaluate the latent structure and psychometric properties of the PSCC-Sp using SPSS version 17.0 statistical software package for Microsoft Windows (SPSS Inc., Chicago, Il). We assessed the suitability of the data for dimension analysis using various criteria (e.g., examination of the correlation matrix for correlations of .30 and above). We subsequently conducted a principal components analysis (PCA) to reduce the data to a minimum number of components that could facilitate more accurate interpretation.

We initially conducted the PCA without any rotation to facilitate extraction and examination of meaningful components, based on eigenvalues and screeplot criteria that more accurately describe the latent structure of the PSCC-Sp. We used the Kaiser-Meyer-Olkin value (KMO), an index of sampling adequacy, in determining the suitability of the data for dimensionality analysis.^29–30 Then, we evaluated the screeplot of eigenvalues to help determine the number of components that could be retained. Afterward, we conducted a VARIMAX rotation of the initial factor solution.

Measurement reliability analysis of the PSCC-Cp

Scale reliability assessment was conducted to determine the degree to which items of the PSCC-Sp represent a coherent set that assess the same underlying construct. Cronbach coefficient alpha was used as an index of the internal consistency of the PSCC-Sp.

RESULTS

The mean age of the analytic sample was 41 years (Standard deviation = 11.8). The total sample was female (100.0%). A large proportion (83.4%) of the sample reported completion of only a high school education or less. Each participant presented with either an abnormal test finding or a definitive diagnosis of cancer including breast (73.0% breast), cervix (23.9%), and colorectal (3.1%). Table 1 presents more detailed demographics and clinical characteristics of study participants. Each participant included in the present study provided a signed informed consent. The Institutional Review Boards of all the PNRP participating institutions approved this study.

Testing of the PSCC-Sp latent structure

Suitability for Factor Analysis

We examined the items correlation matrix of the PSCC-SP and found many correlation coefficients of .30 and higher, as well as a KMO value was 0.93 that exceeded the recommended KMO value of 0.60.^31–32 Additionally, the Bartlett’s Test of Sphericity was statistical significant (χ² (378) = 7850.920; p = 0.001), which confirmed the appropriateness of dimensionality analyses of the correlation matrix.³³

PSCC-Sp Construct Validity

The initial unrotated PCA revealed the presence of 3 components with eigenvalues exceeding one (λ > 1): 8.51, 1.17 and 1.02, which explained 47.3 %, 6.5 % and 5.7 % of the total cumulative variance (59.5 %) respectively. Inspection of the screeplot revealed a clear break after the second component. Catell’s (1966) screeplot test and the eigenvalues criteria suggested that two components could likely be retained for further investigation.³⁴

Similarly to the PSCC, we removed items with moderate loadings on multiple components because of plausible overlapping contributions. We also exclude components defined by just a single or two variables because these components are generally unstable, usually account for a small percentage of the variance, and are often difficult to reliably interpret.³⁵ Based on these criteria, we also ended up with a one-dimensional 18-item PSCC-Sp measure, as indicated by a single-component structure that explained 47.21% of the variance (Table 2). Our analyses supported the equivalence and cross-cultural validation of the PSCC-Sp for this sample.^35–36

Table 2.

Component Matrix for Sample 1 and Sample 2

Patient satisfaction with Cancer Care	Component Loading
Scale Items	λ= 8.5 N = 285
1. I felt that my health concerns were understood.  Sentí que mis preocupaciones de salud fueron entendidas.	.613
2. I felt that I was treated with courtesy and respect.  Sentí que fui tratado(a) con cortesía y respeto.	.660
3. I felt included in decisions about my health.  Me sentí incluido(a) en decisiones sobre mi salud.	.617
4. I was told how to take care of myself.  Fui informado(a) sobre cómo cuidarme.	.642
5. I felt encouraged to talk about my personal health concerns.  Me sentí alentado(a) a hablar sobre mis preocupaciones con respecto a mi salud personal.	.616
6. I felt I had enough time with my doctor.  Sentí que el medicó me dedicó suficiente tiempo.	.688
7. My questions were answered to my satisfaction.  Mis preguntas fueron contestadas satisfactoriamente.	.739
8. Making an appointment was easy.  Pedir una cita fue fácil.	.415
9. I knew what the next step in my care would be.  Sabía cuál sería el próximo paso en mi tratamiento.	.503
10. I feel confident in how I deal with the health care system.  Me siento confiado(a) en cómo manejar el sistema de cuidados de la salud.	.607
11. I was able to get the advice I needed about my health issues.  Pude conseguir el consejo que necesitaba acerca de mis problemas de salud.	.728
12. I knew who to contact when I had a question.  Sabía a quién contactar cuando tenía una pregunta.	.693
13. I received all the services I needed.  Recibí todos los servicios que necesitaba.	.793
14. I am satisfied with the care I received.  Estoy satisfecho(a) con los cuidados que recibí.	.825
15. The doctors seemed to communicate well about my care.  Los médicos parecían comunicarse bien con respecto a mis cuidados.	.835
16. I received high quality care from my regular doctor.  Recibí cuidados de alta calidad de mi médico habitual.	.794
17. I received high quality care from my specialists.  Recibí cuidados de alta calidad de los especialistas.	.763
18. My regular doctor was informed about the results of the tests I got.  Mi médico habitual fue informado de los resultados de las pruebas que se me hicieron.	.696

Extraction Method: Principal Components Analysis

PSCC Reliability and Convergent and Divergent Validity

Scale reliability assessment was conducted for the 18-item PSCC

Internal consistency – degree to which items that make up this scale represent a coherent set that measures the same underlying construct –was evaluated using Cronbach coefficient alpha. The results showed a Cronbach coefficient alpha of 0.92 based on standardized items for the PSCC-Sp. Findings of the scale reliability assessment supported the use of the PSCC as a reliable tool to assess satisfaction with cancer care for the present sample.³⁷

Convergent and Divergent Validity

The PSCC-Sp total score moderately correlated with the PSN-I-Sp (r = 0.27, p = .003). The results, however, did not reveal any statistically significant correlation between the PSCC-Sp total score and participants’ primary language, marital status, and household income (all p-values > .05).

PSCC-Sp Scale

The 18-item PSCC-Sp was scored on a 5-point scale (“1 = Strongly Agree” to “5 = Strongly Disagree”). As seen with most patient satisfaction measures values were skewed toward favorable ratings with mean coefficient of skewness equal to 1.05. The total scale score for the PSCC-Sp include a summation of scores on all the items, with lower scores indicating higher satisfaction with cancer care.

DISCUSSION

Similarly to the English version (PSCC), we developed the PSCC-Sp to be a brief measure that can easily be used to assess satisfaction with cancer-related care among underserved and lower socioeconomic Spanish speaking individuals who participated in the NCI sponsored PNRP. We professionally translated the English version of the PSCC into Spanish (PSCC-Sp) and then we back-translated the PSCC-Sp into English in order to ensure that we ended up with a culturally meaningful, balanced and equivalent measure for Spanish-speaking participants from the PNRP.

Our analysis revealed a psychometrically valid one-dimensional measure (i.e., PSCC-Sp) that provides a milieu-specific and patient-oriented assessment of perceived relevance and satisfaction with cancer-related care. Our analysis also revealed that the PSCC-Sp has very good construct validity and that items of the PSCC-Sp form a coherent that assess the construct of interest: patients’ satisfaction with the cancer-related care they received.

Similarly to the PSCC, the PSCC-Sp differs from previous generic scales in that it focuses on satisfaction with cancer-related care rather than the broader concept of health care in general or the narrower concept of cancer treatment for a particular, cancer, disease stage or location (hospital or ambulatory).^38–41 The PSCC-Sp was designed to address the broad domain of cancer-related care ranging from diagnostic testing to treatment process and completion.^42–44

Despite its psychometric validity and reliability, the PSCC-Sp has certain limitations that warrant discussion. First, similar to the English version (i.e., PSCC) we did not conduct cognitive interviewing for the PSCC-Sp.⁴⁵ Nonetheless, we conducted a pilot that revealed no problem related to participants’ understanding of the PSCC-Sp items. The PSCC-Sp was administered to participants orally in an attempt to eliminate issues related to the effects of low literacy. Second, consistent with previous satisfaction measures, we observed a tendency towards the higher end of satisfaction in the PSCC-Sp. More specifically, the results were skewed towards favorable rating of satisfaction. However, we were unsure whether this skewness represents truly favorable experiences or reflects low expectations since we did not specifically query patients about expectations.³ It is likely that many patients’ abnormal screening or diagnosis may have been their first experience with cancer-related care. Consequently, they may have used a priori general healthcare experiences that influenced the observed trend towards the higher end of reported satisfaction. Additionally, the observed trend in reported satisfaction could also be representing a social desirability response bias related to the interview format. Further, the present sample included only females. Further studies need to determine the applicability of the PSCC-Sp to male Spanish speaking cancer patients. Finally, we did not assess the degree to which the PSCC-Sp matched clinical impression across different health care processes. This is important since certain aspects of healthcare such as the interpersonal processes of care may have a much greater impact on satisfaction than technical aspects.^46–48

Despite the aforementioned limitations, the PSCC-Sp is a strong and psychometrically valid measure for Spanish speakers, especially those from lower socioeconomic populations across different types of health care systems (e.g. community health centers, Veteran Administration, and University and community-based oncology practices). To our knowledge, the PSCC-Sp represents a first attempt to develop and assess the validity and reliability of a context-specific measure of satisfaction with cancer-related care for underserved Hispanics from lower socioeconomic status facing a variety of barriers to effectively access and use cancer care.

The PSCC-Sp would facilitate the evaluation of the effect of patient navigation on cancer-related care.¹² Results of the psychometric and reliability analysis demonstrated preliminary divergent and convergent validities of the PSCC-Sp. Future studies should also focus on examine the predictive validity of the PSCC for cancer treatment-related outcomes within longitudinal research settings, as well as in relation to other psychometrically valid measures that assess similar construct for cancer patients. The PSCC-Sp holds promise for to the evaluation of patient navigation across different research and clinical service navigation to reduce cancer disparities.