Psychometric Development and Reliability Analysis of a Patient Satisfaction with Interpersonal Relationship with Navigator Measure: A Multi-Site Patient Navigation Research Program Study

Abstract

Background

Patient Navigation (PN) is a method for addressing racial-ethnic and socioeconomically based disparities in cancer-related care. Patient navigators provide logistic and emotional support to underserved patients to facilitate successful completion of diagnostic and treatment care. Yet, little is known about patient satisfaction with the relationship with a navigator due to a dearth of instruments measuring satisfaction.

Objective

To validate the Patient Satisfaction with Interpersonal Relationship with Navigator (PSN-I) measure for patients undergoing diagnostic and/or therapeutic cancer care.

Methods

We administered the PSN-I to 783 participants from the nine different sites of the National Cancer Institute sponsored Patient Navigation Research Program. We evaluated the latent structure and internal consistency of the PSN-I using principal components analysis (PCA) and Cronbach coefficient alpha (α) respectively. We used correlation analyses to examine divergence and convergence of the PSN-I with the Patient Satisfaction with Cancer-Related Care (PSCC), the Rapid Estimate of Adult Literacy in Medicine (REALM) Long Form, and patients’ demographics.

Results

The PCA revealed a coherent set of items that explicates 76.6% of the variance in PSN-I. Reliability assessment revealed high internal consistency (α ranging from 0.95 to 0.96). The PSN-I had good face validity, as well as convergent and divergent validities as indicated by moderate correlations with score on the PSCC (all p_s < 0.0001) and non-significant correlations with primary language, marital status, and scores on the REALM-Long Form (all p_s > 0.05).

Conclusion

The PSN-I is a valid and suitable measure of satisfaction with a patient navigator for the present sample.

INTRODUCTION

Patient Navigation (PN) represents a potentially effective approach for addressing racial, ethnic and socioeconomic disparities across the continuum of cancer care [1]. PN is defined as the logistic, educational and emotional support needed to help individuals from underserved racial/ethnic minorities and lower income successfully complete timely diagnostic and treatment care [2].

PN may be provided by either a lay health worker (i.e., trained paraprofessional) or by a health professional, such as a nurse or social worker [3–6]. Support provided by patient navigators can be categorized as emotional (e.g., direct emotional such as accompanying patients to visits or indirect emotional such as through referral to a support group or mental health treatment)or instrumental/technical (e.g., assistance with insurance, arranging transportation, assisting with financial aid), and educational (e.g., sharing approved information, explanation of medical terms, and coaching) [2, 7–8].

Interest in PN has grown considerably over the past five years. This growth has been aided by funding through the National Cancer Institute (NCI), Centers for Medicare and Medicaid Services, the U.S. Department of Health and Human Services, Health Resources and Services Administration, the American Cancer Society (ACS), and other foundations [9–10]. Given this rapidly growing interest in PN, there is a pressing need for a validated measure to assess the patient’s satisfaction with navigation; such a measure is needed for PN research as well as for internal program evaluation [11].

Many patient satisfaction scales focus on satisfaction with physicians and fewer are devoted to satisfaction with nurses [12–20]. Most of these patient satisfaction measures reflect at least two dimensions: interpersonal/affective and technical/competence [13, 21]. Satisfaction with the first dimension, interpersonal care, is often a key driver underlying overall patient satisfaction, including satisfaction among cancer patients [12–18]. In this manuscript, we report on the development and validation of a scale to assess patient satisfaction with the interpersonal relationship with their navigator (PSN-I).

MATERIALS AND METHODS

Development of the Patient Satisfaction with Interpersonal Relationship with Navigator

The development of the PSN-I involved a multidisciplinary team of investigators and research staff from the nine geographically different sites of the NCI-sponsored PNRP. Members of the PSN-I measurement development team had experiences in clinical research and practice with individuals from diverse racial/ethnic, socio-cultural and economic backgrounds, as well as expertise in psychometric development and validation. Through an iterative process involving multiple revisions, the PSN-I development team carefully generated items thought to reflect key interpersonal aspects of PN. The resulting 9-item PSN-I (Table 2) assesses adequacy of time spent with patient, patient’s level of comfort, perceived navigator’s dependability, courtesy and respect, and listening ability, as well as ease of patient-navigator communication, perception of a caring relationship, navigator’s problem solving, and accessibility of the navigator.

Table 2.

Socio-Demographic Characteristics of Patients Navigators.

	N	Percent
Sex
Female	60	89.55
Male	7	10.45
Race/Ethnicity
White	18	26.87
Black/African American	19	28.36
Hispanic or Latino	23	34.33
Other	5	7.46
No Answer	2	2.99
Primary Language
English	48	71.64
Spanish	16	23.88
Other	2	2.99
No Answer	1	1.49
Birth Country
United States	46	68.66
Other Countries	20	29.85
No Answer	1	1.49
Education
8th grade or less	1	1.49
High school diploma (including equivalency)	5	7.46
Some college/vocational school after HS	15	22.39
Associate degree	4	5.97
College graduate	23	34.33
Graduate or professional degree	18	26.87
No Answer	1	1.49
Household Income
$10,000 to $19,999	2	2.99
$20,000 to $29,999	16	23.88
$30,000 to $39,999	6	8.96
$40,000 to $49,999	8	11.94
$50,000 or more	18	26.87
No Answer	17	25.37

Response Options and Scoring of the PSN-I

The 9-item PSN-I was designed to be administered orally. The following instructions were read to study participants: “The next nine statements are related to your satisfaction with the interpersonal relationship with your navigator(s). For these next statements, I will read the statement to you and I want you to tell me if you “strongly disagree,” “disagree,” “are undecided,” “agree,” or “strongly agree” with the statement I am going to read.” We summed responses to all 9 items of the PSN-I to obtain a total scale score for each participant. A higher score on the PSN-I indicates higher satisfaction with their interpersonal relationship with the patient navigator. The PSN-I was administered by Research Assistants and not by Patient Navigators to avoid any issue related to social desirability biases (e.g., response biases).

Study Participants

The PNRP is a cooperative program funded by NCI (eight sites) and the ACS (ninth site) to rigorously evaluate the role and benefits of a PN interventions to reduce disparities in cancer-related care for individuals from racial-ethnic minorities and lower socio-economic groups with abnormal cancer screening findings or diagnosed cancer – breast, cervical, colorectal or prostate cancer – within nine largely underserved and poor communities across the United States [2].

A total of 783 English fluent participants completed the PSN-I measure (See Table 1). These participants were recruited from the multi-site NCI-sponsored PNRP collaborative study designed to systematically assess and describe the impact of navigating patients on diagnostic or therapeutic care for breast, cervical, colorectal or prostate cancer as described above.

Table 1.

Demographic and Clinical Characteristics of 783 participants

	N	Mean (Standard Deviation)
Age	772	50.7 (13.8)
	n	Percent
Cancer Site
Breast	510	65.6
Cervix	99	12.7
Colorectal	68	8.7
Prostate	98	12.6
Multiple concurrent cancer sites	2	0.26
Gender
Female	645	82.2
Male	130	16.8
Race/Ethnicity
White	43	32.1
Black/African American	28	20.9
Asian	2	1.49
American Indian/Alaska Native	1	0.75
Hispanic or Latino	60	44.8
Other	0	0.00
Primary Language
English	697	90.3
Spanish	63	8.2
Other	12	1.55
Birth Country
US	650	87.1
Other	96	12.9
Marital Status
Single/Never married	227	29.4
Married/living as married	319	41.4
Divorced/separated	175	22.7
Widowed	50	6.5
Education
8th grade or less	52	6.9
Some high school	84	11.1
High school diploma (including equivalency)	181	23.9
Some college/vocational after high school	193	25.5
Associate degree	50	6.6
College graduate	121	16.0
Graduate or professional degree	76	10.0
Household Income
Less than $10,000	176	25.7
$10,000 to $19,999	122	17.8
$20,000 to $29,999	85	12.4
$30,000 to $39,999	70	10.2
$40,000 to $49,999	45	6.6
$50,000 or more	186	27.2
Employment Status
No current employment	427	56.0
Part-time employment	98	12.9
Full-time employment	237	31.1
Health Insurance Status
Yes	641	82.8
No	133	17.2

Procedures

The institutional Review Board of all participating institutions approved this study. Medical staff and clinicians at each of the nine PNRP recruiting sites (e.g., clinics or hospitals) were informed about the study and subsequently referred eligible patients. Referred participants met with either a trained research assistant (RA) or a patient navigator who confirmed their eligibility to participate in the study and consented participants who agreed to be in the study. Some sites consented patients over the telephone. In an effort to minimize possible effects of low literacy, surveys were read out loud to participants in English. The PSN-I was administered within three months of receiving an abnormal cancer test screening or a definitive diagnosis of cancer. The maximum duration of patient navigation was twelve months.

Inclusion and Exclusion Criteria

Inclusion criteria for this study included the presence of an abnormal breast, cervical, colorectal and prostate cancer test finding or a new histologically confirmed diagnosis of any one of the above mentioned carcinomas. Exclusion criteria included any prior history of treatment for the breast, cervical, colorectal or prostate cancer or receipt of previous patient navigation. The Patient Navigation Research Study was designed for participants who were navigation naïve and who had no prior experience with cancer-related care. Prior cancer treatment or navigation experience could mitigate the effects of patient navigation. Additionally, individuals who are familiar with the cancer-related care system might be less likely to need assistance from navigator.

Additional Measures

Demographic characteristics and cancer variables

Demographic indices were obtained by participant report or through chart abstraction. They included age, sex, race, ethnicity, primary language, income, education, and marital status. Data collected also included the type of cancer being evaluated or treated (i.e., breast, cervical, colorectal or prostate) and screening test and outcome.

Socio-Demographic Characteristics of Patients Navigators

PN completed self-reported questionnaire items that assess their socio-demographic characteristics including age, sex, race, ethnicity, primary language, country of birth, education, an income.

Patient Satisfaction with Cancer-Related Care (PSCC)

The PSCC is an 18-item measure that assesses patient satisfaction (PS) with the care they received. The PSCC has been shown to explain 62% of the variance in PS, with high internal consistency reliability as indicated by Cronbach’s coefficient alphas ranging from 0.95 to 0.96. The psychometric validation of the one-dimension PSCC has demonstrated that it is a reliable and valid tool for assessing satisfaction for cancer-related care across diverse racial-ethnic and socioeconomic populations.²²

Rapid Estimate of Adult Literacy in Medicine Long Form (REALM-LF)

The REALM-LF is a 125-item word recognition measure developed as a quick screening tool to help physicians identify patients with limited reading skills and estimate patients’ health literacy in primary care, patient education, and medical research. The REALM-LF has good psychometric properties, as indicate by Cronbach alpha of 0.91, and can be administered and scored in three to five minutes [23].

Data Analysis

Dimensionality analysis of the PSN-I

We conducted latent structural analysis, psychometric validation, and reliability of the PSN-I using IBM PASW^® (SPSS) version 18.0 statistical software package for Microsoft Windows. We randomly divided data from our multi-center sample into two separate subsamples or datasets (Sample 1, N₁ = 391; Sample 2, N₂ = 392). We used the first dataset (Sample 1) to identify and evaluate the underlying latent structure of the PSN-I and we used the second dataset (Sample 2) to replicate and validate its underlying structure. Prior to conducting the principal components analysis (PCA), we examined the suitability of the data for dimensionality analysis using various criteria including examination of the correlation matrix for correlations of .30 or greater. We had an adequate size sample that supported calculation of reliable correlation coefficients for the PSN-I in accordance with guides on sample sizes for factor analysis and principal components analysis [24–25]. Nonetheless, the PCA solutions also contained many high variables markers that could support stable and reliable estimates of correlation coefficients with an even smaller sample size [26].

We conducted the PCA to reduce our data to a few components that facilitate comprehension and explication. We performed an initial non-rotated PCA, using Sample 1 data to allow automatic extraction and examination of all meaningful components, based on eigenvalues and screeplot criteria, that would more adequately describe the latent structure of the PSN-I. We also used the Kaiser-Meyer-Olkin value (KMO), an index of sampling adequacy, to confirm suitability of the data for dimensionality analysis [27–29]. Additionally, we examined the screeplot of eigenvalues to help determine the number of retainable components based on the above-mentioned criteria. Items from Sample 2 were also subjected to a PCA to replicate and test the evidence of the structure of the PCA obtained from Sample 1 through successive unconstrained exploratory procedures [22]. We conducted similar PCA for Sample 2 (N₂) as described above for Sample 1 (N₁).

Measurement reliability analysis

We conducted a scale reliability assessment to determine to what degree items of the PSN-I represent a coherent set that measures the same underlying construct. We used Cronbach’s coefficient alpha as an index of internal consistency of the PSN-I. Measurement of reliability analysis for Sample 1 and Sample 2 was conducted separately.

RESULTS

Our sample included individuals ranging in age from 18 to 92 years, with a mean age of 50.7. The majority of the sample was female (82%) and reported diverse racial-ethnic backgrounds including White (32%), Black (21%), Hispanic/Latino (44%), Asian (1.5%), and American Indian/Alaska Native (0.7%). Approximately 18% of the sample reported a high school education or less. Cancer screening abnormality/cancer diagnosis were as follows: 65% breast, 13% cervix, 9% colorectal, 13% prostate and 0.26% multiple concurrent cancer sites. Detailed demographic and clinical characteristics of study participants are provided in Table 1. The findings did not reveal any significant differences between Sample 1 and Sample 2. All participants provided informed consent for participation.

Socio-Demographic Characteristics of Patients Navigators

The present study included 67 PNs across all nine sites of the PNRP. The majority of the PNs were females (n = 60) from various racial-ethnic, educational, and socioeconomic backgrounds (Table 2). Navigators typically spent about 367 minutes with each patient over an average of 150 days.

The overall focus of PNs included helping participants identify barriers to accessing timely and beneficial cancer-related care and findings ways to overcome these barriers. Some of the daily activities of PNs involved facilitating referrals/direct contacts between patients and cancer care resources, accompanying patients to medical appointments, providing emotional support to patients, managing navigation related records/record keeping, providing and discussing cancer-related educational materials to patients, scheduling appointments, contacting patients’ families, helping patients and their families find needed resources, and facilitating timely follow-up to medical appointments.

Sample 1, N₁ – Testing of PSN-I latent structure

Suitability for Factor Analysis (Sample 1, N₁)

The items correlation matrix included many correlation coefficients of .30 and higher. In addition, the data also revealed a Keiser-Meyer-Olkin (KMO) value of 0.95, exceeding the recommended value of 0.60 [28–29]. The Bartlett’s Test of Sphericity also reached statistical significance (χ² (36) = 3742.35; p = 0.001) [30]. These aforementioned criteria supported the appropriateness of dimensionality analyses of the correlation matrix [26].

Construct Validity (Sample 1, N₁)

The PCA revealed a single component with one eigenvalue exceeding one (λ > 1 = 6.90), which explained 77% of the total cumulative variance. The screeplot test and eigenvalue criteria indicated that only one component could be retained for further investigation [27]. Consequently, the data indicated that the 9-item PSN-I constitutes a one-dimension measure (Table 2). Results of the PCA supported the use of the PSN-I as a valid and reliable measure for our sample [28–30]. Similar findings were observed among the subsample of patients with diagnosed cancer.

Sample 2, N₂ – Validation of PSN-I latent structure

Suitability for Factor Analysis (Sample 2, N₂)

We used data from Sample 2 to confirm the underlying structure of the PSN-I that emerged from our analysis with data from Sample 1. This approach was based on the notion that successful replication through successive unconstrained exploratory procedures provides a stronger confirmation of the underlying structure of the PSN-I beyond any constrained confirmatory procedure [22]. Similar to Sample 1, examination of the correlation matrix for Sample 2 revealed the presence of many correlation coefficients of .30 and higher. In addition, the KMO value was 0.95, exceeding the recommended value of 0.6 [28–29]. The Bartlett’s Test of Sphericity reached statistical significance (χ² (36) = 3631.99; p = 0.001), supporting the appropriateness of dimensionality analyses of the correlation matrix [30]. Thus, the data supported a 9-item one-dimensional PSN-I measure (Table 3). The findings of our psychometric development and internal consistency analyses supported the use of the PSN-I as a valid and reliable measure for our sample.

Table 3.

Component Loadings for Sample 1 (N1 = 391) and Sample 2 (N2 = 392): Correlations between Individual Items and the Underlying Component.

Satisfaction with Interpersonal Relationships with Navigator	Component Loadings
Scale Items	λ= 6.90	λ= 6.79
	N1 = 391	N2 = 392
My navigator gives me enough time	.931	.906
My navigator makes me feel comfortable	.922	.908
My navigator is dependable	.906	.920
My navigator is courteous and respectful to me	.890	.866
My navigator listens to my problems	.884	.882
My navigator is easy to talk to	.873	.869
My navigator cares about me personally	.852	.865
My navigator figures out the important issues in my health care	.813	.832
My navigator is easy for me to reach	.797	.799

Construct Validity (Sample 2, N₂)

The PCA for Sample 2 revealed the presence of a single component with one eigenvalue exceeding one (λ > 1 = 6.79), which explained 75% of the total cumulative variance. The screeplot test and the eigenvalues criteria supported the retention of two components for further investigation [27–28].

PSN-I Reliability and Convergent and Divergent Validity

Scale reliability assessment was conducted for the 9-item PSN-I

Internal consistency – degree to which items that make up this scale represent a coherent set that measures the same underlying construct – was evaluated using Cronbach’s coefficient alpha. The results showed Cronbach’s coefficients alphas of 0.95 and 0.96 based on standardized items for the PSN-I for Sample 1 and Sample 2, respectively. The scale reliability assessment supported the use of the PSN-I as a reliable tool for measuring patients’ satisfaction with their interpersonal relationships with their navigators [31–32].

Convergent and Divergent Validity

The PSN-I total score for Sample 1 (N₁ = 391) correlated positively with the total score on the PSCC (Spearman r = 0.37, p < 0.001), indicating convergence of the PSN-I with the PSCC, a measure of satisfaction with cancer-related care. The results, however, did not reveal any statistically significant correlation between the PSN-I total score and patients’ primary language, marital status, and scores on the REALM long form (all p- values > 0.05). Likewise, the PSN-I total score for Sample 2 (N₂ = 392) positively correlated with the total score on the PSCC (Spearman r = 0.45, p < 0.0001), confirming the convergence of the PSN-I with the PSCC. Again, analysis of Sample 2 (N₂ = 392) revealed no statistically significant correlation between the PSN-I and patients’ primary language, marital status, and scores on the REALM long form (all p-values > 0.050). Similar findings were observed among the subsample of patients with diagnosed cancer.

PSN-I Scale

Patient scores on the 9-item PSN-I ranged from 9 to 45 with a mean of 41. As seen with most patient satisfaction measures values were skewed toward favorable ratings with mean coefficient of skewness equal to 2.17 [33]. PSN-I mean scores by geographic site (all nine sites) of navigation ranged from 38 to 43, and by navigators 32 to 45 (although some of these ranges were based on a very small numbers). Of the 783 participants included in this study, 592 had abnormal cancer screening results and 191 had histologically confirmed cancer diagnoses. Participants with a cancer diagnosis scored slightly higher (M = 41.66, SD = 5.35) than those with abnormal screening results (M = 40.47, SD = 5.9) on the PSN-I. However, this difference was not statistically significant (p = 0.20). This pattern of response could be due to the fact that participants with cancer generally spend more time with their navigators compared to those with abnormal cancer screening results.

DISCUSSION

PN is increasingly being incorporated into cancer-related care. The integration of PN into the cancer-related care process underscores a corresponding need for valid and reliable tools to assess the quality of PN. We developed the PSN-I to assess a key dimension of PN: patient perceptions of the interpersonal/relational aspects of navigation. The development of the PSN-I involved an iterative process. We incorporated items with high face validity designed to reflect patient perceptions of key interpersonal characteristics of navigation. The results of our structural analysis and psychometric validation revealed an internally consistent one-dimensional scale, the 9-item PSN-I, with high construct validity. As predicted, the PSN-I scale showed a statistically significant, but moderate, correlation with the Patient Satisfaction with Cancer-Related Care (a milieu-specific patient-oriented measure of perceived relevance and satisfaction with cancer care) [22] and no statistically significant correlation with patients’ primary language, marital status and score on the REALM-LF; indicating adequate convergence and divergence of the PSN-I.

To our knowledge, the PSN-I represents the first validated scale for patient satisfaction with their navigator. Currently, there are no “gold standard” measures to assess the patient-navigator relationship. The PSN-I provides a convenient way for both research programs and community navigator programs to assess patient satisfaction with the interpersonal process of navigation. The 9-items PSN-I assesses key aspects of navigator performance including time spent, dependability and accessibility, in addition to more affective-laden aspects such as listening, respect and caring. As navigation becomes increasingly integrated into cancer care, it is critical to have valid measures for its assessment. Navigation programs could utilize the PSN-I to evaluate the quality of the interpersonal dimension of their navigation services and identify areas for improvement. In addition, the PSN-I could be used to assess differences in patients’ satisfaction with different models of navigation (i.e. telephone-based versus face-to-face navigation).

Certain limitations of the PSN-I warrant comment. The study sample was based primarily on female patients undergoing diagnostic testing following a cancer screening abnormality, rather than on patients with diagnosed cancer. However, analysis of this subsample of patients with diagnosed cancer revealed similar performance. In addition, patients who lacked primary English language proficiency were excluded. Further, it is not yet known whether this aspect of PN (satisfaction with the patient-navigator interpersonal relationship) is associated with other PN outcomes such as timely receipt of care, patient adherence, or improvements in health status. The PSN-I assesses only one dimension of navigation, interpersonal relationship. Lastly, like most patient satisfaction measures, the scale is skewed towards favorable ratings of navigators.³³ However, we observed significant variation in PSN-I scores by site and by navigator.

In summary, the 9-item PSN-I scale reliably assesses patients’ perceptions of their interpersonal relationship with their navigator. Further study is required to assess whether patients’ satisfaction with their navigator is associated with improved outcomes. Additionally, the PSN-I can be used in addition to other measures such as the PSCC to evaluate the benefits of PN programs across the cancer care continuum.