Measuring a Patient’s Understanding of their Prognosis: An exploratory analysis

Abstract

Objective:

In patients with cancer across the illness trajectory, treatment decisions are often influenced by one’s perception of their prognosis (i.e., curability of disease, life expectancy, quality of life). However, research on how patients understand their prognosis (i.e., prognostic understanding) has been limited by simplistic measurement tools that fail to capture the complexity of the construct. This study describes the initial development of a measure of prognostic understanding: the Prognostic Understanding Perceptions Scale (PUPS) for use in patients with advanced cancer.

Method:

An initial pool of 16 candidate items were developed through semi-structured interviews with 15 experts (oncology, psycho-oncology and palliative care professionals) and 30 advanced cancer patients. We investigated the dimensionality, internal item structure, item difficulty and item discrimination of the item pool using exploratory factor analysis (EFA), classical test theory (CTT) and item response theory (IRT) analyses. Convergent and divergent validity were based on correlations between PUPS, terminal illness acknowledgement, self-report measures of depression, anxiety, hopelessness, and death acceptance.

Results:

The final measure was comprised of nine items encompassing three factors (perceived curability, illness trajectory, treatment options), yielding strong psychometric properties.

Conclusion:

These results provide preliminarily support for PUPS as a multifaceted measure of prognostic understanding developed for use in patients with advanced cancer. Preliminary findings also highlight the potential utility of the PUPS for clinical settings, as a means of enhancing communication between patients and physicians.

Background

Medically ill adults with advanced or terminal illness face challenging decisions: Should I pursue life sustaining interventions like artificial hydration or nutrition? Should I discontinue “curative” treatments because the side effects outweigh the potential for extending my life? These and other medical decisions are heavily influenced by a patient’s prognosis – and by extension, the patient’s understanding of their prognosis. Unfortunately, research has frequently documented the failure of medically ill patients to accurately understand essential elements of their prognosis. For example, a recent meta-analysis of research with advanced or terminally cancer patients estimated that only about half of all patients accurately understood their prognosis.¹

Understanding how a patient’s perception of their prognosis impacts medical decision making has been hampered by reliance on simplistic measurement tools. Applebaum and colleagues found that most studies simply asked patients with terminal illness whether they perceived the illness to be terminal or curable, or if the goal of treatment was palliative or curative.² Researchers have typically characterized a terminally ill patient as lacking prognostic awareness if they rate themselves as “relatively healthy” or perceive the goal of treatment to be curative; those who perceive themselves “seriously but not terminally ill” may be categorized as having partial awareness. Although there is some appeal to simple methods for rating prognostic awareness (e.g., quick, easily analyzed), this approach ignores the complexity involved in understanding one’s prognosis.

A recent qualitative study identified multiple components encompassed by the term prognosis, all of which form a patient’s prognostic understanding more generally.³ These domains include the current disease state and its likely curability, the availability of treatment options, anticipated life expectancy, and the likely trajectory of symptoms and/or functioning (i.e., deterioration). Furthermore, each element exists along a continuum; an individual may be more or less accurate in their understanding of any domain depending on their cancer stage. While some conditions, such as cognitive impairment or extreme denial, may cause difficulties across multiple domains, partial awareness might reflect a failure to understand one or more aspects of one’s prognosis. Hence, measurement of a patient’s perception of their illness across the spectrum of beliefs and capacities encompassed under the umbrella of prognosis is critical for advanced care planning and assisting health care professionals in understanding the information needs of patients they treat.

The purpose of this study was to develop and evaluate a multifaceted measure of prognostic understanding intended for use in patients with cancer across the illness trajectory. We hypothesized that the developed measure would be significantly correlated with a simplistic measure of prognostic understanding and expected that in those patients with advanced disease, higher scores on the developed measure would be associated with greater endorsement of theoretically relevant psychological symptoms.

Method

Procedures and Participants

Patients with cancer were recruited from the outpatient clinics of a tertiary care cancer hospital and individuals participating in a medical case management program for immigrants and medically underserved populations through the same institution. The latter site was included to oversample individuals with cancer from diverse racial and ethnic backgrounds and maximize generalizability. Recruitment relied on direct solicitation by clinical research coordinators during outpatient chemotherapy visits, case management appointments or by telephone as well as posted flyers. Inclusion criteria were age over 18 years old and English or Spanish speaking with a cancer diagnosis. We used a professional service to translate the draft measure into Spanish and back-translate it to English, with discrepancies resolved by a bilingual clinical psychologist with extensive experience in psycho-oncology.

Although the construct of prognostic understanding applies to individuals at all disease stages, we originally included only hospital outpatients with stage III or IV (i.e., metastatic) disease, since scores on the measure are more directly interpretable (i.e., as accurate or inaccurate) in patients with advanced disease; patients of any disease stage were recruited from the case management program for item-level analyses (where inter-item and item-latent construct associations are central). However, we expanded our data collection from the hospital during the last 6 months of recruitment to increase our sample size for item-level (rather than scale-level) analyses. Prospective participants were excluded if they exhibited severe cognitive impairment that, in the opinion of study personnel, impaired the participant’s ability to give informed consent or complete study questionnaires. After providing written informed consent, participants were administered a battery of self-report questionnaires (described below). The study was approved by the Institutional Review Boards of Memorial Sloan Kettering Cancer Center (protocol numbers i14-BR–151 and 16–316, respectively).

The sample was comprised of 281 adults with cancer (155 hospital outpatients and 126 from the case management program) who were invited to participate in a study inquiring about “how patients understand their disease.” Nine individuals omitted all or most questionnaire items and were excluded from all analyses, resulting in a sample of 272: 175 (64.3%) women and 91 (33.5%) men (gender was missing for 6 participants). The average age was 61.1 years old (range: 34 to 88), and included 138 (50.7%) White, non-Latinx participants, 70 (25.7%) Black participants, 44 (16.2%) Latinx participants and 20 (7.4%) who identified as another race/ethnicity (or left this item blank); 22 individuals completed study measures in Spanish. The sample was highly educated, with 106 (37.7%) individuals who completed high school (or equivalent), 64 (38.1%) who completed a college degree and 57 (20.3%) who had a graduate or professional degree. Most had Stage III (n=43; 20.7%) or Stage IV (n=125; 59.6%) disease, whereas 41 (19.7%) participants had Stage I or II disease; disease stage was missing for 64 participants (23.5%) recruited through the case management program (many were unsure of their disease stage).

Measures

To explore the construct of prognostic understanding, we conducted semi-structured interviews with 15 oncology, psycho-oncology, and palliative care professionals with expertise in physician-patient communication³ and 30 interviews with advanced cancer patients. Interviews were transcribed and analyzed using thematic content analysis to illicit core elements of the construct. These themes were used to create an initial pool of 16 items that addressed the domains identified, including the current state of the disease, anticipated trajectory of the illness, and treatment options for their illness. We opted for a relatively small pool of candidate items to decrease patient burden and asked the original pool of experts and several cancer patients to review the items to insure comprehensiveness and comprehensibility. Items were generated using two different formats, a 5-point ordinal response format and a true/false format, to ascertain the best format for this population.

In addition to the draft measure of prognostic understanding (hereafter referred to as the Prognostic Understanding Perceptions Scale; PUPS), participants recruited from hospital outpatient clinics (N=155, 96.75% of whom had advanced disease) were administered self-report measures of psychological distress. Terminal Illness Awareness (TIA)⁴ was measured using a single item asking patients to describe their health as “relatively healthy,” “seriously, but not terminally ill” or “seriously and terminally ill.” In terminally ill patients, acknowledgement of serious illness indicates a higher degree of awareness. Distress was measured using the Hospital Anxiety and Depression Scale (HADS),⁵ a 14-item measure with subscales measuring anxiety and depression symptom severity. The Hopelessness Assessment in Illness questionnaire (HAI)⁶ is an 8-item measure that has demonstrated strong reliability and temporal stability in advanced cancer patients.⁷ Patients also completed the Death Acceptance (DA) subscale of the Life Attitude Profile,⁸ an 8-item scale where higher scores indicate greater acceptance of death as a natural part of life. Perceived social support was assessed with the 14-item Duke/UNC Functional Social Support Questionnaire (FSSQ).⁹ Finally, demographic and medical information was elicited via a self-report questionnaire, including age, sex, race/ethnicity, preferred language for healthcare information, country of birth, length of time in the United States (for those born outside the U.S.), marital status, and years of education. Medical variables included site and stage of cancer.

Statistical Analysis

We evaluated the psychometric properties of each PUPS item to develop a final set of items that would reflect the breadth and structure of the underlying construct while minimizing redundancy and patient burden. Preliminary analyses involved item-total correlations and Cronbach’s α, associations between the PUPS items and demographic variables (gender, race, disease stage). We then conducted exploratory factor analysis to explore dimensionality and item response patterns, using parallel analysis¹⁰ to test the dimensionality (retaining factors if the observed eigenvalue exceeded the simulated eigenvalue), and fitted EFA models with weighted least square estimation with mean and variance adjusted χ² (WLSMV)¹¹ and geomin rotated solution.¹²

Based on the preliminary analyses and EFA, we conducted IRT analysis, testing dichotomous items using a 2-PL model and polytomous items using the graded response model (GRM).¹³ Problematic items were identified based on item difficulty (item thresholds) and item discrimination, as well as item characteristic curves and item information functions. We removed items based on these statistics and item content and compared the models across iterations using global model fit indices: Akaike information criterion (AIC),¹⁴ Bayesian information criterion (BIC),¹⁵ root mean square error of approximation (RMSEA)¹⁶ and comparative fit index (CFI)¹⁶. A model with RMSEA below .06 and CFI greater than .90 was identified as satisfactory;¹⁶ better-fitting models were based on lower AIC, BIC, and RMSEA, and higher CFI. The results of each iteration are presented in online supplements. To examine convergent and discriminant validity of the PUPS, we calculated correlations between PUPS total and factor scores (based on the final measure in patients from the tertiary cancer hospital) with measures administered concurrently: TIA, HADS, HAI, DA, and FSSQ. For these correlational analyses, we included only participants who had stage III or IV disease and had completed the “supplemental” self-report questionnaires, resulting in a sample of 116.

Results

Preliminary Analysis

Descriptive statistics for the initial 16 PUPS items are provided in Table 1. Response option 5 for item 5 was identified as invalid because of its compound nature, hence this response was treated as a missing value in all analyses. Item 14 had a very low mean as very few people (2%) chose “True” for this item. Cronbach’s α was .79 for the 16-item measure. Item-total correlations indicated item 8 was uncorrelated with total score (r=−.02) and increased α to .80 if deleted. The average inter-item correlation was .20 (Supplemental Table 1) with three items below r=.15.

Table 1.

Descriptive Statistics of Original PUPS Items

	Item	M	SD	N	Item-total r	Cronbach’s α if deleted
5-Point Scale Items
	1	2.93	1.30	271	.65	.759
*	2	2.59	1.32	267	.56	.771
	3	1.62	0.81	261	.44	.780
*	4	4.01	1.02	258	.31	.793
	5	1.98	0.89	266	.55	.770
*	6	3.96	0.95	269	.57	.767
	7	1.52	0.89	267	.63	.762
True/False (Dichotomous) Items
	8	0.51	0.50	266	−.02	.804
	9	0.27	0.45	266	.67	.773
	10	0.06	0.23	264	.29	.793
*	11	0.73	0.44	258	.17	.797
*	12	0.81	0.39	261	.59	.778
	13	0.35	0.48	263	.15	.796
	14	0.02	0.14	263	.12	.797
	15	0.42	0.49	260	.18	.795
*	16	0.84	0.37	264	.57	.780

^*indicates reverse-coded item.

Associations between PUPS items and demographic variables (Table 2) indicated no significant differences in item endorsement based on gender, but nine items differed by race/ethnicity (White, Black, Latinx). White participants were more likely to acknowledge negative prognostic indicators than those identifying as Black or Latinx). Ten items were significantly associated with cancer stage, indicating that participants with Stage IV disease rated the items as indicative of a more negative prognosis than those with Stage I/II or Stage III disease.

Table 2.

Univariate Associations Between PUPS Items and Gender, Race, and Stage of Cancer

Item	Gender			Race/Ethnicity			Stage of Cancer
	t	df	p	F	df	p	F	df	p
1	−0.31	251	.76	22.32	(2, 246)	< .001*	31.44	(2, 197)	< .001*
2	1.48	235	.14	7.17	(2, 231)	< .001*	15.28	(2, 185)	< .001*
3	−0.52	232	.61	2.58	(2, 229)	.08	6.27	(2, 184)	.002*
4	−1.81	225	.07	0.60	(2, 223)	.55	0.58	(2, 176)	.56
5	−1.04	247	.30	17.18	(2, 242)	< .001*	31.37	(2, 193)	< .001*
6	2.40	257	.02	17.34	(2, 252)	< .001*	14.74	(2, 200)	< .001*
7	−0.07	255	.94	2.33	(2, 250)	.10	11.31	(2, 198)	< .001*
	χ 2	df	p	χ 2	df	p	χ 2	df	p
8	4.13	1	.04	24.81	2	< .001*	6.25	2	.04
9	0.10	1	.75	30.57	2	< .001*	31.02	2	< .001*
10	0.86	1	.35	9.50	2	.01	1.01	2	.60
11	0.81	1	.37	10.31	2	.01	2.96	2	.23
12	0.08	1	.78	17.63	2	< .001*	26.28	2	< .001*
13	1.56	1	.21	15.49	2	< .001*	15.31	2	< .001*
14	0.05	1	.83	4.98	2	.29	0.16	2	.92
15	1.82	1	.18	3.89	2	.42	1.76	2	.41
16	0.14	1	.71	18.09	2	< .001*	15.80	2	< .001*

^*= p < .003 (Bonferroni corrected α = .05/16 = .003).

Note: Race/ethnicity was analyzed as a three-category variable contrasting White, Black, and other (primarily Latinx). Stage of cancer was also analyzed as a three-category variable contrasting Stages I/ II, Stage III and Stage IV.

Exploratory Factor Analysis

Factor analysis (Supplemental Table 2) indicated five factors with eigenvalues that exceeded the mean of the simulated values derived from parallel analysis. However, the scree plot (Supplemental Figure 1) supported 2, 3, or 4 factor solutions. Guided by these results, we examined factor loadings for 2- through 5-factor models (Supplemental Table 3). The 5-factor model failed to converge due to negative residual variance for item 11. We used a cutoff of .32 for satisfactory factor loadings (roughly 10% overlap between the factor and item).¹⁷ The 2-, 3-, 4-factor models explained 30.2%, 34.1%, and 37.2% of the variance, respectively. However, the 4-factor model had many double-loaded items and the 2-factor model had two items with loadings below .32. Based on these results, we determined that a 3-factor model best fit the data.

Factor 1 (Perceived Curability) included items that focused primarily on the patient’s perception of illness severity and anticipated curability (e.g., The chances of going into remission are…). Factor 2 (Treatment Options) contained items focused on the availability of treatments (e.g., At this point…). Factor 3 (Illness Trajectory) contained items focused on the perceived pace of the illness and/or its sequelae (e.g., My cancer is progressing…).

Item Response Theory Analysis

We tested an IRT model based on the EFA results. Supplemental Table 4 provides the factor loadings and IRT parameters for all 16 PUPS items. Consistent with EFA, items 8, 11 and 13 had low factor loadings and low item differentiation on factor 3. These items also had low item-total correlations (r = −.02, .17, and .15) and item information curves that were flat across the values of factor 3 (Supplemental Figure 1), indicating that the probability of endorsing the item was insensitive to the respondent’s true perception. These findings, coupled with some deliberate redundancy in item content with other items and the extremely low endorsement rate for Item 14, led us to remove four items.

A second model without items 8, 11, 13 and 14 had fit indices (AIC=5087.42, BIC=5267.72, RMSEA=.041, CFI=.987) that improved upon the first model (AIC=6097.35, BIC=6299.27, RMSEA=.046, CFI=.965). In this model, all but one item had discrimination parameters exceeding 1.0; item 15 had lower discrimination (0.69) but was retained due to its unique and important item content. All items had moderate to very high discrimination parameters (moderate: 0.35–0.64; high: 1.35–1.69; very high: ≥1.70 and above).¹⁸ The item content for items 7, 9 and 12 overlapped with other items that had stronger discrimination and therefore were removed.

In the third model (items 7, 8, 9, 11, 12, 13 and 14 removed), the model fit indices (AIC=4190.91, BIC=4327.93, RMSEA=.031, CFI=.992) were better than the second model. All item discrimination parameters were satisfactory except item 15, which had item discrimination < 1.0. Item difficulties and item characteristic curves suggested reducing the 5-point scale items to a 3-point scale with negligible loss of information (Supplemental Figure 3). We contrasted analyses conducted with the original scale to those based on a 3-point rating scale, either by collapsing the lowest categories or two middle categories (depending on item content and the distribution of responses). The model fit statistics with collapsed 3-point scales were equivocal, as decreased AIC and BIC statistics suggested that the new model better fit the data than the previous model despite higher RMSEA and lower CFI in the new model (AIC=3006.92, BIC=3111.49, RMSEA=.052, CFI=.949). The parameter estimates, item information functions, and item characteristic curves showed that this model had satisfactory model fit. All items had loadings between .48 to .89 and discrimination parameters larger than 1.0. Supplemental Figure 4 shows the test information function of the three factors. The test information for Factor 1 peaked when the factor score was approximately 0.5; Factor 2 had two peaks, at approximately 0 and 2.3 and Factor 3 was relatively flat, reaching its peak when the factor score was approximately 0.8.

Convergent and Discriminant Validity

Table 4 presents the correlations between the PUPS total score (sum of the final item pool with collapsed item responses, see Appendix A), the three factor scores, and the criterion measures: TIA, HADS, HAI, DA, and FSSQ. Because only patients with stage III or IV cancer (n=116) were included in these analyses, higher scores represented greater prognostic understanding (i.e., participants perceived the illness as more serious across the three factors). Total PUPS scores were positively correlated with TIA, r=.45, p < .001, as were the three PUPS factors, Perceived Curability r=.41, Treatment Options r=.31, and Illness Trajectory r=.44 (all ps < .001). Participants who perceived their illness as more serious on the TIA also endorsed PUPS items indicative of a poorer prognosis. PUPS total scores were positively correlated with HAI, r=.39, p < .001, indicating that perception of a poorer prognosis was associated with greater levels of hopelessness. There was also a significant negative correlation between PUPS total scores and the DA, r=−.29, p < .001, indicating lower levels of death acceptance among those with greater understanding of the severity and nature of their prognosis. Although no significant association was observed between total scores on the PUPS and the HADS total score, r=.15, p=.15, a significant association between was observed with the Illness Trajectory factor, r=.27, p=.004; no significant correlation was found between the PUPS total or factor scores with a measure of social support (Table 4).

Table 4.

Correlations between PUPS and Criterion Measures for Stage III and Stage IV Patients

Variable	PUPS	F1	F2	F3	TIA	HAI	HADS	DA	FSSQ
PUPS	1.00
F1	.95***	1.00
F2	.74***	.73***	1.00
F3	.68***	.54***	.32***	1.00
TIA	.45***	.39***	.28**	.42***	1.00
HAI	.39***	.30**	.09	.45***	.38***	1.00
HADS	.15	.05	.−01	.27***	.25***	.75***	1.00
DA	−.29**	−.16	−.06	−.21*	−.26**	−.27**	−.32***	1.00
FSSQ	−.13	−.15	−.16	−.15	−.14	−.32***	−.29**	.00	1.00

Note.

^*p < .05

^**p < .01

^***p < .001.

PUPS: Sum of the PUPS items based on the final subset of 9 items (using collapsed item responses); F1: Perceived Curability factor score); F2: Treatment Options factor score; F3 Illness Trajectory factor score; ACP: Advance Care Planning Checklist; HADS: Hospital Anxiety and Depression Scale; HAI: Hopelessness Assessment in Illness Scale; DA: Life Attitude Profile – Revised, Death Acceptance Subscale; FSSQ: Duke Functional Social Support Questionnaire.

Discussion

Despite the importance of prognosis in patients with cancer, little research has attempted to develop a multifaceted measure of prognostic understanding. Previous research on prognostic understanding has generally utilized single items to measure one aspect of prognosis, often with little or no evidence to support the validity of these single-item rating systems. Because patients may have an accurate understanding of some aspects of their prognosis yet be unrealistic in their perceptions of others, we sought to develop a multifaceted tool that focused not only the perceived severity of the disease but incorporated perceived curability, the availability of treatment options, and the anticipated trajectory of the illness. These domains were identified in our team’s qualitative study delineating the domains that are encompassed by the term “prognosis.”³

Our analysis of the 16-item pool found that a 3-factor model provided the best fit to the data, generating a good model fit. An analysis of item difficulty and discrimination, item content and iterative factor solutions resulted in a 9-item scale with three response options (rather than the 5-point and dichotomous items that had been initially developed and evaluated). The final scale included two of the dichotomous items (requiring development of a third response option that had not previously been evaluated), resulting in a final scale that deviates slightly from the instrument used to establish convergent and discriminant validity (Table 4). Although two of these factors had only two items each, and hence have lower reliability, the importance of scale brevity for patients with cancer outweighs the benefits of a longer scale that included potentially redundant items.

Preliminary evidence of construct validity was also demonstrated by moderate positive correlations between a single-item measure of terminal illness acknowledgement (TIA) and a measure of hopelessness specifically developed for patients with a terminal illness (HAI). These associations are consistent with expectations, given that acknowledgement of one’s illness is a core component, particularly of factor one of the PUPS. Likewise, while hopelessness is strongly associated with (and can reflect an element of) psychological distress, in the context of terminal illness this measure can also represent a rational appraisal of one’s illness.⁶ However, the moderate associations (r’s between .4 and .5 with TIA and HAI) support the expectation that the PUPS is not simply measuring acknowledgement of illness severity or the perception that treatments are futile, but reflects a broader construct related to multiple aspects of one’s prognosis. For example, patients may acknowledge that the illness is terminal but not perceive treatment to be futile (factor 2) or anticipate a continued decline in their functioning (factor 3). Indeed, the correlations within and between the three factors further support these interpretations, as the correlations between TIA and HAI were stronger for factor 1 (Perceived Curability) and factor 3 (Illness Trajectory) than for factor 2 (Treatment Options). Previous research also corroborates the correlation between hopelessness and perceived curability in a sample of stage IV cancer patients.¹⁹

A weak and non-significant correlation was observed with a measure of general psychological (HADS), although the association with the Illness Trajectory factor was significant. Early research on prognostic understanding using a single-item rating of prognostic awareness based on a semi-structured interview found that patients with more depressive symptoms were more likely to deny or minimize the severity of a terminal illness.²⁰ However, another study of patients starting treatment for gastrointestinal cancer found no significant difference in depression severity between participants who acknowledged their terminal cancer diagnosis and those that did not.²¹ Thus, the association between PUPS and depression may have been attenuated by using a broad-based measure of prognostic understanding, as only some aspects of prognostic understand may engender (or depend on) distress. While this explanation is speculative, particularly given the preliminary nature of these analyses, it supports continued research exploring the different components of “prognosis” to better understand the interactions between emotions and illness information processing.

The modest negative association between death acceptance and PUPS total scores indicates that, at lease for some patients, understanding one’s prognosis may cause them to “struggle” with acceptance of their eventual death. This finding is consistent with research demonstrating that patients with stage IV cancer participating in a psychotherapy research trial who perceived their cancer as unlikely curable or incurable had less death acceptance than those who perceived the cancer as curable or likely curable.¹⁹ In short, acceptance of death may be somewhat easier for those who do not perceive death to be as imminent. Clearly, further research is clearly necessary to understand the relationship between these constructs, as well as the extent to which psychotherapeutic interventions may influence these associations.

Clinical Implications

Although our rationale for developing the PUPS was driven by the challenges we have encountered in researching (and understanding) decisions made by patients with advanced and terminal illness, we anticipate that the tool will be useful in many clinical settings. How a patient perceives their illness may impact a wide range of treatment decisions. Particularly given the relationship between prognostic understanding and engagement in advance care planning,²² a systematic assessment of the patient’s understanding may help clinicians identify gaps or inaccuracies, as research has demonstrated that health care providers often overestimate the extent to which their patients understand prognostic information.¹ Regardless of cancer site and stage, the PUPS allows patients and their healthcare team to reflect on how they understand their current illness. Hence, the PUPS could help facilitate discussions between patients, their family caregivers, and health care providers, bolstering the ability of patients to make informed treatment decisions that are consistent with their values and preferences.

There are a number of important strengths that support the present study findings, beginning with the rigorous methodology used to develop scale items. A qualitative study was conducted with internationally recognized experts in the field and a large sample of patients with advanced or terminal cancer to identify key aspects of prognostic understanding to be represented in measure items. Moreover, both the qualitative study and the present analyses involved diverse samples, as we purposely oversampled immigrant and traditionally underserved populations (including monolingual Spanish speakers) to maximize the generalizability of the PUPS. We also utilized EFA and IRT analyses to compare models and item formats to identify the model and items that best fit the data. Further, this brief measure (nine items) places relatively little burden on patients, which is particularly important given its presumptive use with seriously ill patients.

Study Limitations

There are also limitations in the study, beginning with the absence of data that utilized only the final nine items with three response options. Because recruitment was halted by the Covid-19 pandemic, we could not collect a second sample utilizing the final instrument. Although there is no reason to expect marked differences in item or scale properties (e.g., the 3-factor model), additional research is needed to further explore the psychometric properties of the PUPS. Likewise, our sample size was not sufficient to carve out a validation sample, which would have permitted a more rigorous analysis of the initial item pool. The language used in the PUPS items may have also been confusing for some participants, however patients who reviewed the initial draft questionnaire raised no such concerns and study personnel rarely received requests for term clarification during the court of the study. Additionally, participants who completed the full study battery were recruited from the outpatient oncology clinics at a state-of-the-art cancer treatment center. Associations with psychological variables such as hopelessness, depression, or anxiety may be stronger in samples drawn from treatment settings that have fewer resources or less ability to treat advanced cancer. Significant associations with demographic variables such as education may also have emerged in more diverse samples, as our sample was more highly educated than a typical medical population.

The reliance on a voluntary sample, while obviously unavoidable, may have also under-represented individuals who found the subject matter distressing, potentially injecting a source of bias into the study results. Further, our study methodology did not permit collection of additional study variables (e.g., TIA, HAI, HADS) from the case management program, resulting in less diversity for the subsample used in these analyses and precluding an analysis of whether racial/ethnic or linguistic differences impacted the associations between the PUPS and other variables. Additionally, many participants from the case management program did not provide their disease stage, and were not necessarily receiving their cancer care at the study institution, we could not cross-reference medical records to obtain this information. Due to the relatively small sample sizes of patients with stage III or stage IV cancer recruited from the outpatient clinic, we could not test measurement invariance across disease stage, race or language of administration. Finally, while the PUPS was developed for use with cancer patients, its applicability to other serious or life-limiting illnesses is not known.

Conclusions

Notwithstanding these limitations, these results provide preliminary support for a new, multifaceted measure of patient prognostic understanding. The 3-factor model allows for a more detailed and nuanced examination of this construct and may help illuminate more subtle influences on the cognitions and decision-making processes of terminally ill patients. Perhaps most importantly, by helping clinicians identify gaps in their patients’ understanding, the PUPS can provide an opportunity to enhance the dialog between patient and health care providers about important issues related to the illness and its prognosis.

Table 3.

Standardized Factor Loadings, Factor Correlations, and IRT Item Parameters of the Final Item Pool

Original Item (Final Item)	Factor Loadings			IRT Item Parameters
	Factor 1	Factor 2	Factor 3	Discrimination	Difficulty1	Difficulty2
1 (1)	.81			2.50	1.14	0.40
2 (2)	.67			1.64	0.14	−1.60
3 (3)		.89		3.55	7.74	0.37
4 (4)			.68	1.68	3.52	1.35
5 (5)	.55			1.18	4.44	1.20
6 (6)	.64			1.53	3.03	1.59
10 (7)		.61		1.40	3.60	N/A
15 (8)			.48	1.00	0.42	N/A
16 (9)	.76			2.12	2.70	N/A
	Factor Correlations
	Factor 1	Factor 2	Factor 3
Factor 1	1.00
Factor 2	.76***	1.00
Factor 3	.52**	.30	1.00

^*p < .05

^**p < .01

^***p < .001.

Note: Items 7, 8, 9, 11, 12, and 13 were removed; Items 1 to 6 were collapsed to a 3-point scale.

Appendix A: Final PUPS Questionnaire

1. The doctors think my illness is (factor 1)

   1. Serious but probably treatable
   2. Uncertain
   3. Not likely to be curable
2. The chances of my illness going into remission are (factor 1)

   1. Very low
   2. Moderate
   3. High or almost certain
3. Based on my conversations with the doctors (factor 2)

   1. There are well established treatments left to try
   2. There are some treatment options remaining but they are largely experimental
   3. There are no treatments left that will significantly extend my life
4. My cancer is progressing (factor 3)

   1. Quickly
   2. At an average pace
   3. Slowly (or not at all)
5. Which statement best describes the goal of your current treatment? (factor 1)

   1. To cure or stop the progression of the cancer
   2. To slow, but not stop the progression of the cancer
   3. To alleviate symptoms of a cancer that can no longer be treated
6. My ability to carry out day-to-day tasks (factor 1)

   1. Is getting worse and will probably worsen more quickly in the near future
   2. Is relatively stable
   3. Is improving a bit (or I expect to improve in the coming weeks)
7. At this point, (factor 2)

   1. There are really no treatments left to try
   2. There may be some treatments left to try but they are probably experimental
   3. There are definitely more treatments to try
8. Based on what my doctors have told me about how my illness will progress (factor 3)

   1. I see no reason to make changes in my plans for the future
   2. I may need to make some changes in my plans for the future
   3. I have already begun to make changes in my plans for the future
9. My conversations with my doctors (factor 1)

   1. Have not given me hope for improvement in my illness
   2. Have given me a little hope for improvement in my illness
   3. Have given me hope for improvement in my illness