Patient Satisfaction with Navigator Interpersonal Relationship (PSN-I): item-level psychometrics using IRT analysis

Abstract

Background

Patient navigation (PN) is a promising intervention to eliminate cancer health inequities. Patient navigators play a critical role in the navigation process. Patients’ satisfaction with navigators is important in determining the effectiveness of PN programs. We applied item response theory (IRT) analysis to establish item-level psychometric properties for the Patient Satisfaction with Interpersonal Relationship with Navigators (PSN-I).

Methods

We conducted a confirmatory factor analysis (CFA) to establish unidimensionality of the 9-item PSN-I in 751 cancer patients (68% female) between 18 and 86 years old. We fitted unidimensional IRT models—unconstrained graded response model (GRM) and Rasch model—to PSN-I data, and compared model fit using likelihood ratio (LR) test and information criteria. We obtained item parameter estimates (IPEs), item category/operating characteristic curves, and item/test information curves for the better fitting model.

Results

CFA with diagonally weighted least squares confirmed that the one-factor model fit the data (RMSEA = 0.047, 95% CI = 0.033–0.060, and CFI ≈ 1). Responses to PSN-I items clustered into the 4th and 5th categories. We aggregated the first three response categories to provide stable parameter estimates for both IRT models. The GRM fit the data significantly better than the Rasch model (LR = 80.659, df = 8, p < 0.001). Akaike’s information coefficient (6384.978 vs. 6320.319) and Bayesian information coefficient (6471.851 vs. 6443.771) were lower for the GRM. IPEs showed substantial variation in items’ discriminating power (1.80–3.35) for GRM.

Conclusions

This IRT analysis confirms the latent structure of the PSN-I and supports its use as a valid and reliable measure of latent satisfaction with PN.

Introduction

Patient navigation (PN) is a promising program to eliminate cancer health inequities and premature death by facilitating timely cancer screening, definitive diagnostic resolution, and treatment initiation and completion for medically underserved racial-ethnic minority and lower socioeconomic populations [1–9]. Dr. Harold Freeman first conceptualized and implemented PN to eliminate cancer disparities in 1990 in Harlem, New York [2, 3, 10]. Funded by the US Federal government and private foundations (e.g., American Cancer Society), PN programs have been implemented across various medical settings.

Specifically, PN involves pairing patients in need of cancer-related care (e.g., definitive diagnosis or treatment) with patient navigators (e.g., lay community health workers/trained paraprofessionals, or other healthcare professionals including nurses and social workers) [3]. Patient navigators play a critical role in the PN process to eliminate or reduce cancer health inequities, and provide logistical (e.g., assistance with health insurance, transportation to medical appointments, and financial support), educational (e.g., sharing of approved information, explication of medical terms and coaching), and emotional support (e.g. caring attitudes, accompanying patients to medical appointments, referral to community support group, and psychosocial resources) to patients [11]. Patient navigators can help reduce cancer health disparities and inequities by assisting patients from medically underserved populations in identifying and removing barriers to timely access to equitably beneficial cancer care [12, 13]. PN strategies that help patients overcome healthcare system barriers to receiving timely and optimal cancer-related care are likely to influence these patients’ perceptions of the quality of care they received. Specifically, patient navigators who work directly with cancer patients to achieve positive health outcomes through the PN programs potentially improve patients’ satisfaction with cancer care.

Satisfaction is an important dimension of perceived quality of care that is often used to quantify the extent to which a patient’s health care experience matches the level and quality of expected care [11]. Increasing use of PN program to help reduce cancer disparities and inequities underscores the importance of validated and reliable measures to systematically quantify patients’ satisfaction with important components of PN programs, especially the navigators who provide direct services to these patients. Most satisfaction measures focus on satisfaction with doctors, and fewer measures focus on satisfaction with nurses and other healthcare personnel such as patient navigators [14–22]. Patient satisfaction measures typically reflect two broad dimensions: (1) interpersonal/affective (perception of social and relational skills of health care providers) and (2) technical/competence (perception of task behaviors such as practical, technical, and medical skills of health care providers, e.g., physicians) [15, 23]. The interpersonal/affective dimension of patient satisfaction is generally more readily observable by patients than the technical/competence dimension of patient satisfaction. Specifically, patients judge their health care team including their navigator based on the person’s perceived warmth, caring, and demeanor. Patients often lack the knowledge and relevant information necessary to judge the technical competence of team members. For example, a patient is better able to judge if their navigator is empathetic (interpersonal dimension) than whether their navigator is knowledgeable about every relevant community resource (technical competence). In addition, higher levels of interpersonal/affective behaviors of health care providers (e.g., physicians) are associated with higher patient satisfaction [15–20]. The technical/competence dimension of patient satisfaction may not yet be well understood.

We previously developed and validated a Patient Satisfaction with Navigator Interpersonal Relationships (PSN-I) measure using classical test theory (CTT) method [24]. In the present paper, we are reporting outcomes of an item response theory (IRT) analysis that provides item-level psychometric properties and confirms the latent structure of the PSN-I, which can be utilized to help integrate concerns and viewpoints of medically underserved patients into their cancer care. We conducted a confirmatory factor analysis (CFA) and applied IRT analysis to each item of the PSN-I. We calculated item parameter estimates (IPEs) (e.g., difficulty and discrimination/slope parameters based on item characteristic curves) to facilitate reliable assessment of satisfaction and evaluation of scores from different subsets of PSN-I items that are put on the same scale. The IRT-based IPEs from this large sample can be adopted in studies with small sample sizes that may not support computation of IRT item-level estimates. The IPEs from the present IRT analysis enhance our previous CTT outcomes and provide more comprehensive information about item-level psychometric properties and item difficulties for the PSN-I, which will facilitate tailoring subset of items to specific patients. The latent PSN-I score obtained from this IRT analysis accounts for measurement error, focuses on item-level analytics, discriminates high versus low patient satisfaction with navigators, and facilitates determination of conditional measurement precision.

Methods

Participants

We collected data from 1296 participants who participated in the multisite National Cancer Institute (NCI) and American Cancer Society (ACS) sponsored PatientNavigation Research Program (PNRP) collaborative study to reduce cancer disparities. Of those, 545 did not provide a response to any PSN-I item, with frequency and rate of missing data ranging from 545 (42.05%) to 560 (43.21%). This large number of (PSN-I) missing data is due to the number of non-navigated patients (i.e., control group) who did not have to complete the PSN-I. Descriptive statistics showed that there were no differences in available characteristics between non-navigated and navigated patients in the present sample. We analyzed data from the other 751 participants, age 18 to 86 years, who completed the PSN-I. This socioeconomically diverse sample consisted of 13.6% Whites, 18.9% Blacks/African Americans, 24.6% Asians/Asian-Americans, 0.1% American Indians or Alaska Natives, 10.8% Hispanics/Latinos, 1.3% other races/multiple races, and 30.7% missing. Eligibility criteria included having an abnormal cancer screening result or a definitive cancer (i.e., breast, cervical, colorectal, or prostate) diagnosis [11]. The cancer sites (i.e., breast, cervical, prostate, and colorectal) were determined in the request for application by the National Institutes of Health for the purpose of the PNRP cooperative agreement grant because of the greatest disparity in screening and follow-up of these types of cancer. Institutional Review Boards of participating institutions approved this study. All participants provided signed informed consent for this study (Table 1).

Table 1.

Sample socio-demographic and clinical characteristics (N = 751)

Characteristic	Number of participants
		Mean (SD)
Age (range, 18–86 years)	619	50.6 (14.3)
Missing	132
		%
Cancer
Breast	500	66.6
Cervix	89	11.9
Colorectal	77	10.3
Prostate	81	10.8
Multiple concurrent cancer sites	3	0.4
Missing	1	0.1
Sex
Female	513	68.3
Male	109	14.5
Missing	129	17.2
Race/ethnicity
White	102	13.6
Black/African American	142	18.9
Asian	185	24.6
American Indian/Alaska Native	1	0.1
Hispanic or Latino	81	10.8
Other	10	1.3
Missing	230	30.7
Primary language
English	515	68.6
Spanish	96	12.8
Other	10	1.3
Missing	130	17.3
Birth country
USA	476	63.4
Other	129	17.2
Missing	146	19.4
Marital status
Single/never married	193	25.7
Married/living as married	254	33.8
Divorced/separated	136	18.1
Widowed	37	4.9
Missing	131	17.5
Education
8th grade or less	58	7.7
Some high school	76	10.1
High school diploma/GED	141	18.8
Some college/vocational school	166	22.1
Associate degree	40	5.3
College graduate	87	5.3
Graduate or professional degree	46	6.1
Missing	137	18.3
Household income
Less than $10,000	155	20.6
$10,000 to $19,999	110	14.6
$20,000 to $29,999	78	10.4
$30,000 to $39,999	58	7.7
$40,000 to $49,999	35	4.7
$50,000 or more	116	15.4
Missing	199	26.6
Employment status
No current employment	338	45.0
Part-time employment	92	12.3
Full-time employment	187	24.9
Missing	134	17.8
Health insurance coverage
Yes	499	66.4
No	119	15.8
Missing	133	17.8

Not all of the 751 participants have responded to each of the socio-demographic and clinical characteristic questions. For example, only 619 participants reported their age, and 622 participants reported their sex

Procedures

Staff at collaborating clinics and hospitals identified and/or referred participants to the PNRP. Participants were assigned to receive PN according to each specific site’s protocol and contacted to complete assessment questionnaires within 3 months of initiating cancer treatment. Study team members read the PSN-I items aloud in English or Spanish to participants to reduce low literacy bias. The PSN-I was translated (i.e., English to Spanish) and back-translated (i.e., Spanish to English) by professional translators and expert native speakers familiar with the concepts and terminology of this measure. This translation and back-translation approach is commonly used in cross-cultural measurement psychometric validation to help achieve cultural and conceptual equivalence. Participants completed the PSN-I and other measures in English or Spanish based on linguistic proficiency.

Measures

Demographic and clinical characteristics

Research assistants assessed patients’ demographic and clinical characteristics (e.g., age, sex, race-ethnicity, primary language, income, education, marital status, and cancer types [i.e., breast, cervical, colorectal, or prostate]), using paper-pencil questionnaires and data from medical records.

Patient satisfaction with interpersonal relationships with navigators

The development of the PSN-I is previously described [24]. Briefly, the measurement development team discussed domains of patient satisfaction relevant to the PNRP, reviewed existing satisfaction measures/scales, and selected, modified, and developed new PSN-I items. The relevance of each item was discussed with experts and patients during the item pool development and pre-pilot evaluation ofthe PSN-I. The PSN-I items were phrased to develop a coherent set of items to assess the underlying construct of satisfaction with interpersonal relationship with navigators. Psychometric validation of the PSN-I using CTT and principal components analysis (PCA) revealed a unidimensional structure accounting for 76.6% of the variance, and high internal consistency reliability (i.e., Cronbach alphas = 0.95–0.96). The latent structure of the PSN-I was replicated across different samples and time points. Face validity and discriminant and convergent characteristics were also established [24].

PSN-I response option

Participants were asked to respond to each PSN-I item on a 5-point Likert-type scale: “1 = Strongly Agree,” “2 = Agree,” “3=Neutral,” “4=Disagree,” and “5=Strongly Disagree.” Agreement or disagreement with PSN-I items connotes participants’ level of satisfaction or dissatisfaction. A summative total scale score was calculated for each study participant. Lower PSN-I total score indicates higher satisfaction.

Data analysis

Confirmatory factor analysis

We performed a CFA using diagonally weighted least squares (a default of lavaan for categorical data, an R statistical package for fitting structural equation models), which further indicated model fit and supported the application of unidimensional IRT analysis. Detailed CFA results are provided in the results section.

IRT analysis

Two one-dimensional IRT models were fitted to PSN-I data from 751 patients: unconstrained graded response model (GRM), and constrained GRM (Rasch model) with discrimination parameters across items set to be equal [25]. The PSN-I response scale ranged from k = 1 to m, where m = 5 for the PSN-I 5-point Likert-type scale. According to the GRM, the probability of scoring at or above category k on item j at a given level of latent trait θ is as follows:

$$P_{jk}^{*}\left(\theta \right)=P\left(X_j\ge k|\theta \right)=\frac{\exp \left({\alpha }_j\left(\theta -{\beta }_{j\left(k-1\right)}\right)\right)}{1+\exp \left({\alpha }_j\left(\theta -{\beta }_{j\left(k-1\right)}\right)\right)},$$

where k = 1, 2, …, m; X_j represents a participant’s response to item j; α_j is the discrimination parameter; and β_jk are the threshold parameters. It should be noted that for an item with m response categories, m − 1 threshold parameters are estimated. In this case for item j, there are four threshold parameters, β_j1 to β_j4. In our PSN-I IRT analysis, the discrimination parameter indicates how much an item can discriminate between patients with different latent satisfaction levels. The threshold parameter of β_jk represents the “hurdle” (i.e., the point where the latent trait level leads to an equal probability of endorsing either one of the two adjacent response categories) on the latent satisfaction continuum between score category k and (k + 1). The probability of scoring exactly k, denoted by P_jk(θ), therefore is $P_{jk}^{\ast }\left(\theta \right)-P_{j\left(k+1\right)}^{\ast }\left(\theta \right)$. For instance, for the 5-point Likert scale of the PSN-I,

$$P_{j2}\left(\theta \right)=P_{j2}^{\ast }\left(\theta \right)-P_{j3}^{\ast }\left(\theta \right)=\frac{\exp \left({\alpha }_j\left(\theta -{\beta }_{j1}\right)\right)}{1+\exp \left({\alpha }_j\left(\theta -{\beta }_{j1}\right)\right)}-\frac{\exp \left({\alpha }_j\left(\theta -{\beta }_{j2}\right)\right)}{1+\exp \left({\alpha }_j\left(\theta -{\beta }_{j2}\right)\right)}$$

By definition, $P_{j1}^{\ast }\left(\theta \right)=1$ and $P_{jm}\left(\theta \right)=P_{jm}^{\ast }\left(\theta \right)$.

Overall, the GRM is a widely used IRT model for Likert-type data from unidimensional measures [26]. The Rasch model, however, is a more parsimonious model that necessitates the estimation of fewer parameters. The Rasch model is a reasonable alternative when it offers comparable fit to the data. In the present analysis, we obtained model fit indices, including log likelihood, Akaike’s information coefficient (AIC), and Bayesian information coefficient (BIC) for both the GRM and the Rasch model. We conducted a likelihood ratio (LR) test to compare the fit of GRM and Rasch model. We calculated item parameter estimates and latent trait parameter estimates (i.e., latent patient satisfaction with navigators), category characteristic curves, and item and test information curves for the better fitting model. We used the lavann package in R for CFA and PARSCALE for the IRT analysis [27–29].

Results

Factor analysis

The CFA revealed a one-factor model with factor loadings ranging from 0.881 to 0.958, small standard errors (0.009–0.010), and adequate model fit (χ² = 69.064, RMSEA = 0.047, 95% confidence interval of RMSEA [0.033, 0.060], and CFI = 0.99). The small RMSEA (i.e., 0.047) and high CFI indicate that the model fit the data very well. A statistically significant χ² was expected for this large sample and the skewed/non-normal data (i.e., most of the responses to PSN-I items clustered into response option categories 4 and 5). Statistical significance of test results are dependent on sample size. Thus, small effects may reach statistical significance with a large sample. Given the sample size of this study, we could not depend only on statistical significance test for interpretation of model fit. Consequently, we also examined the RMSEA and CFI which revealed that the one-factor model fit very well and that fitting models with additional factors was not needed. Additionally, we assessed the latent structure and internal consistency of the PSN-I using principal components analysis (PCA) and Cronbach coefficient alpha (α) in a previous CTT analysis [24]. The PCA revealed that items of the PSN-I formed a coherent set that explicates approximately 76.6% of the total cumulative variance in patient satisfaction with interpersonal relationships with navigators. PCA with a second sample also revealed that the 9 items of the PSN-I loaded on a single component with one eigenvalue exceeding one (λ > 1 = 6.79) that explained 75% of the total cumulative variance. Screeplot tests and eigenvalue criteria for both these PCAs also revealed a one-dimensional measure. The reliability assessment revealed high internal consistency based on Cronbach coefficient α ranging from 0.95 to 0.96. Outcomes of both the previous CTT analysis and the present study suggest that the items of the PSN-I constitute a one-dimensional and parsimonious model that fits the data adequately. In the present analysis, we used diagonal weighted least squares to fit the factor analysis model instead of ML-based methods that assume normality and were not suitable for the data. Overall, the results supported the use of a unidimensional IRT model to determine item-level psychometric properties, and facilitate the development of PSN-I short forms.

IRT model comparison

We found a Cronbach’s alpha of 0.978, which could be increased to 0.981 if item “1” (i.e., “My navigator gives me enough time”) of the PSN-I was deleted. We should also note that while having a high Cronbach alpha suggests high internal consistency of the scale, it may also indicate possible redundancy among the items. Additionally, in certain applications, future users of the PSN-I may be able to delete items with information curves that appear to provide no new information in order to achieve a more parsimonious model. In such cases, proper psychometric validation and structural analyses of this possible shorter PSN-I scale should be completed.

The lower end of the PSN-I (i.e., Strongly Disagree) was endorsed less frequently, a response pattern typical of satisfaction measures. Most of the responses to the PSN-I items clustered into categories 4 and 5 (Table 2). Coupled with the large sample size, this possibly explains why the χ² for the CFA was statistically significant. Therefore, we considered the RMSEA (0.047) and the CIF (≈1) instead of the χ² value in evaluating the CFA model fit, which supported the application of unidimensional IRT models to PSN-I data. Additionally, with fewer patients (less than 5%) endorsing response option categories 1, 2, or 3 for PSN-I items 1, 2, 3, 6, 7, and 9, it was difficult to obtain precise parameter estimates for both the Rasch model and GRM, which may have also caused convergence issues. Thus, we aggregated response categories 1, 2, and 3 and assessed the fit of both the GRM and Rasch model. The endorsement rate of each response category after aggregation is presented in Table 2. The new category 1, which combines the original categories 1–3, now has over 5% endorsement for most of the items. The new categories 2 and 3 are the original categories 4 and 5. Using the GRM, three item parameters were estimated for each item: a discrimination parameter, and two threshold parameters that divided response categories 1 and 2 (β_i1), and 2 and 3 (β_i2), respectively. Using the Rasch model, all items share the same discrimination parameter, but two threshold parameters β_i1 and β_i2) were still estimated for each item.

Table 2.

Patient Satisfaction with Interpersonal Relationships with Navigator Scale (PSN), and percentage of responses for each category (%)

PSN-I items	Original categories					Aggregated categories
	1	2	3	4	5	1	2	3
1. My navigator gives me enough time	1.3	1.1	1.1	27.7	68.8	3.5	27.7	68.8
2. My navigator makes me feel comfortable	1.3	1.6	3.6	29.7	63.7	6.6	29.7	63.7
3. My navigator is dependable	1.2	0.9	4.2	30.9	62.8	6.3	30.9	62.8
4. My navigator is courteous and respectful to me	1.5	2.2	5.8	32.3	58.2	9.5	32.3	58.2
5. My navigator listens to my problems	1.4	1.8	6.6	33.4	56.8	9.8	33.4	56.8
6. My navigator is easy to talk to	0.9	0.9	0.8	27.6	69.7	2.7	27.6	69.7
7. My navigator cares about me personally	1.3	0.9	3.2	32.3	62.2	5.5	32.3	62.2
8. My navigator figures out the important issues in my health care	1.6	2.0	9.4	31.9	55.0	13.1	31.9	55.0
9. My navigator is easy for me to reach	1.2	1.3	1.5	30.6	65.4	4.0	30.6	65.4

Rasch model

Using the Rasch model with the discrimination parameters set to be equal, we obtained a log likelihood = − 3173.489, AIC = 6384.978, and BIC = 6471.851. We obtained the item parameter estimates (IPEs) by using the marginal maximum likelihood (MML) and constraining the discrimination parameter (α) to be equal for all the PSN-I items. Our analysis revealed that the β_is were smaller than “0” (Table 3).

Table 3.

Item parameter estimates and the corresponding standard errors for the Rasch model and unconstrained grade response models of the PSN-I

Rasch model				Unconstrained graded response model
Item	α	βi1	βi2	α	βi1	βi2
1	2.579 (.088)	− 1.808 (.075)	− .582 (.041)	2.191 (.235)	− 1.969 (.083)	− .625 (.047)
2	2.579 (.088)	− 1.482 (.055)	− .432 (.041)	2.274 (.228)	− 1.610 (.061)	− .464 (.045)
3	2.579 (.088)	− 1.506 (.057)	− .396 (.041)	2.998 (.329)	− 1.530 (.052)	− .428 (.040)
4	2.579 (.088)	− 1.304 (.047)	− .288 (.040)	1.845 (.174)	− 1.510 (.062)	− .310 (.050)
5	2.579 (.088)	− 1.295 (.046)	− .221 (.040)	2.161 (.212)	− 1.425 (.055)	− .241 (.046)
6	2.579 (.088)	− 1.954 (.087)	− .602 (.041)	2.642 (.314)	− 2.030 (.087)	− .648 (.042)
7	2.579 (.088)	− 1.577 (.060)	− .377 (.041)	3.330 (.384)	− 1.566 (.052)	− .410 (.038)
8	2.579 (.088)	− 1.150 (.042)	− .188 (.039)	1.798 (.169)	− 1.330 (.057)	− .204 (.050)
9	2.579 (.088)	− 1.737 (.071)	− .471 (.041)	3.352 (.400)	− 1.724 (.063)	− .514 (.038)

Unconstrained graded response model

By fitting the unconstrained GRM to the PSN-I data, we obtained a log likelihood = − 3133.160, AIC = 6320.319, and BIC = 6443.771. When comparing/selecting models, the model with smaller AIC and BIC is deemed better. We found that both AIC and BIC clearly favored the GRM. We then conducted a likelihood ratio (LR) test between the two models, and found LR = 80.659, df = 8, and p < .001, which indicated that the GRM fit the PSN-I data significantly better than the constrained Rasch model. This finding is supported by parameter estimates using marginal maximum likelihood (MML) that revealed discrimination parameter (α) ranging from “1.7” to “3.4” (Table 3). The range in the discrimination parameter estimates was large (items 4 and 8 had relatively small estimates whereas items 7 and 9 had relatively large estimates) and therefore supported the selection of the GRM instead of the Rasch model. Consequently, we considered the GRM the better fitting model for the PSN-I data.

Item and test characteristics based on unconstrained GRM

Category response curves

We plotted category response curves for each item of the PSN-I based on the unconstrained GRM (Fig. 1). Each curve within each plot showed the probability of endorsing a certain response category. For example, the black curve of item 1 revealed how the probability of endorsing the second response category (i.e., “agree”) changed as a function of the latent satisfaction level. The curve had a bell shape, indicating that patients who were neither very satisfied nor dissatisfied with their navigators were more inclined to endorse the middle response category for the item. As satisfaction increased, the probability of endorsing the first category (i.e., “dissatisfied or neutral”) represented by the blue curve decreased, and the probability of endorsing the third response category (i.e., “strongly agree”) represented by the red curve increased.

Fig. 1.

Item Response Category Characteristic Curves (CCC) for each PSN-I item

Item information curves

Item information was generally higher when the latent trait was around −2 and −0.5 for most of the items (Fig. 2). The fact that item information peaked at negative values is consistent with negative β_i estimates—the items are generally “easy,” meaning that respondents tend to score high (i.e., report higher satisfaction). “Easy” items provided more information about participants at the low end of “ability” (i.e., low latent satisfaction with patient navigators), and not much information about participants at the higher end of the latent trait of satisfaction with their navigators. Additionally, PSN-I items 4 and 8 afforded relatively lower information the other items. Therefore, items 4 and 8 can be modified. Further, the item information curves revealed little information beyond latent trait level of “0.” This finding from the item information curves suggests that adding items that provide information at the higher end of the satisfaction continuum may be beneficial. This information could enhance researchers’ understanding of the latent trait (e.g., satisfaction level), which will inform the development of reliable methods to more accurately capture the essence of patients’ satisfaction with navigators who helped them overcome barriers to receiving timely, optimal, and beneficial care.

Fig. 2.

Item Information and Test Information Curves for the PSN-I Scale items. The test information of the short form test is obtained after removing items 4 and 8

Test information curve

Test information curve analysis revealed that test information was higher when the latent trait was between − 1.7 and − 0.5 (Fig. 2). Hence, the PSN-I is most reliable when the latent trait is around − 1.7 and − 0.5. Overall, the items of the PSN-I were very good and formed a coherent measure to assess cancer patient satisfaction with interpersonal characteristics of navigators.

Discussion

PN programs are being implemented in many hospitals and clinics throughout the USA to reduce inequities in cancer-related care and premature mortality for individuals from medically underserved racial-ethnic minority and lower socioeconomic populations. Patient navigators play a critical role in the overall care experience and treatment outcomes for many cancer patients. Patient navigators provide multidimensional (e.g., logistical, educational, and emotional) support to cancer patients. The challenges inherent in facilitating timely access to optimal cancer care for patients, the stress related to dealing with the cancer diagnosis and treatment process, and the interpersonal and/or intrapersonal characteristics of both navigators and patients are likely to influence their relationships. Understanding the dynamics of the patient-navigator relationship is essential to the development of strategies to improve procedures and outcomes of PN.

Reliable assessment of patient satisfaction is important in evaluating quality of cancer care. This is especially true and challenging in the context of patient satisfaction with the navigators who worked closely with cancer patients and provide direct services to them during a stressful life experience (e.g., cancer diagnosis and treatment processes). We previously developed and validated the PSN-I using classical test theory methods to facilitate the evaluation of the PN experience. In the present study, we applied IRT—a modern paradigm for the design, psychometric validation, and scoring of tests, questionnaires and related scales to measure latent constructs—to better understand the psychometric properties of the PSN-I and possibly fine-tune it. This application of IRT analysis to PSN-I precedes further testing of differential item functioning and measurement equivalence/invariance, which will help establish similarity and quantitative comparison of scores across different subsets of items for specific participants (i.e., between-group analysis) or across time for individual participants in repeated measurement conditions (i.e., within-group analysis). Overall, we obtained IRT parameter estimates that can be used to facilitate reliable evaluation of satisfaction with navigators in other representative samples, especially in cases where only a subset of PSN-I items (e.g., PSN-I short form) is completed.

Additionally, results of the IRT analysis confirm previous findings of the CTT based PSN-I psychometric validation [24]. The unconstrained GRM fits the PSN-I data well and reveals important ways in which we can improve the PSN-I and its application in PN programs designed to reduce cancer health inequities for the poor and medically underserved populations in the USA. For example, we can modify the PSN-I by aggregating the low-end response categories. Also, items 4 and 8 have relatively lower information and thus can be revised or omitted, to achieve a short form for the PSN-I, with minimum loss of test information. Figure 2 shows test information before and after plausible deletion of items 4 and 8 that supports the possibility of revising/omitting these 2 items. Furthermore, the general shape of the curve is largely maintained even though the test information curve of the short form is slightly lower than that of the long form. Nevertheless, we should carefully review these items (i.e., 4 and 8) to ensure that their deletion will not compromise content validity. Overall, the IRT analysis provides a detailed description of the psychometric properties of the PSN-I through item parameter estimates, and item category and information curves.

The item parameter estimates derived from this large, multicultural, and socioeconomically diverse calibration sample are accurate and can be used in prospective studies of patient satisfaction with navigators. The PSN-I IRT analysis provides item-level information that complement scale-level information obtained from the previous CTT analysis, which can be useful in future studies on the effectiveness of tailored PN programs to reduce health inequities, alleviate cancer burden, and improve treatment outcomes for medically underserved cancer patients and survivors. These IRT parameters can be useful in the development and testing of a computer adaptive testing (CAT) system. The advantage of a CAT version may be modest given the length of the PSN-I. With more items, however, a PSN-I CAT system could facilitate tailoring the PSN-I scale to specific individuals’ latent trait. Adding items with less ceiling effect and less skewed distribution in future revisions might enhance the PSN-I. However, any addition of scale items will require a reevaluation of the psychometric validation of the measure that will results in a new version of the PSN-I (e.g., PSN-I version 2—PSN-I₂). The PSN-I can also be used to promptly determine patient satisfaction, which can be shared with care providers and PN personnel to enable process-monitoring evaluation of strategies to eliminate cancer health disparities.

Additionally, this IRT analysis provides important information to facilitate optimal selections of items to estimate latent satisfaction level, and to help researchers determine comparable scores from different subsets of items. However, the findings of the present study should be interpreted cautiously. First, the observed racial-ethnic distribution of the present study sample is different than the distribution of breast cancer patients and the larger US population. This is probably because this PNRP study focused primarily on recruiting individuals from under-served racial-ethnic minority and lower income populations. Further, we did not examine issues related to measurement invariance across subpopulations (e.g., race, ethnicity, sex, and age) in the present analysis. Future studies should address these patient distribution and plausible measurement invariance issues.

Overall, the PSN-I is a psychometrically valid and reliable unidimensional measure that can be utilized to help improve patient-centered care by identifying interpersonal attributes of patients and navigators that may influence the navigation experience. The IRT-based PSN-I statistics obtained from the present analysis can also be applied in future studies to enhance patient-navigator communication, and to integrate culturally-based perspectives of cancer patients from traditionally underrepresented and medically underserved racial-ethnic minority and lower socioeconomic backgrounds into the cancer care process. Such an application of the PSN-I will facilitate better understanding of patients’ health care expectations and satisfaction levels across the cancer care continuum throughout the navigation process.