Evaluating the measurement properties of the 26-item Expanded Prostate Cancer Index Composite (EPIC-26) with a multicenter cohort

Abstract

Introduction

We aimed to evaluate the psychometric properties of the 26-item Expanded Prostate Cancer Index Composite (EPIC-26) for measuring the quality of life in patients treated for localized prostate cancer. The EPIC-26 is a patient-reported outcome instrument recommended for use with patients treated for localized prostate cancer.

Methods

This study is based on data collected prospectively between September 2014 and February 2017 in Alberta, Canada. Men were treated with either radical prostatectomy or radiation therapy and administered the EPIC-26. Responses to the EPIC-26 were the primary outcome. Construct validity was measured using confirmatory factor analysis. Reliability was measured using Chronbach’s alpha and item-total correlation. Ceiling and floor effects were also investigated.

Results

EPIC-26 response data from 205 participants (prostatectomy =138; radiation=60; both=7) were used in this analysis. The EPIC-26 was administered an average of 33.8 weeks after treatment. The confirmatory factor analysis model did not meet the threshold for adequate fit. Several items had near-zero factor loadings and were non-significant. Four out of the EPIC-26’s five domains met the acceptable reliability threshold based on Cronbach’s alpha. Ceiling effects were observed in four out of five domains.

Conclusions

The EPIC-26 demonstrated poor construct validity, adequate reliability, and large ceiling effects. Several issues were observed, suggesting that the instrument’s five domains were not well-defined by their respective items. The original EPIC’s conceptual framework should be reviewed and the shortened instrument revised to improve its performance for measuring post-treatment quality of life.

Introduction

The Expanded Prostate Cancer Index Composite 26 (EPIC-26) is the most commonly used patient-reported outcome (PRO) instrument with men treated for localized prostate cancer.¹ the International Consortium for Health Outcomes recommends the EPIC-26 in their standardized set of outcomes for men with localized prostate cancer.²

Despite its popularity, there has been very little study into EPIC-26’s psychometric properties (i.e., reliability, validity, and feasible). A recent review by Axcrona et al found that only 13 studies (8% of those published using the EPIC-26) reported any kind of psychometric results.³ Summarizing the results from these studies, the review found that the EPIC-26 had good feasibility, internal consistency, and test-retest reliability. The EPIC-26 demonstrated good convergent and criteria validity but weakened content validity from the original EPIC. There was insufficient evidence with respect to the construct and predictive validity of the EPIC-26 to draw any firm conclusions.

The Axcrona et al review also provided a good history of the EPIC-26. The original EPIC was a 50-item instrument, four expanding upon the 20-item University of California Los Angeles Prostate Cancer Index (UCLA-PCI) by adding items on irritative (storage) symptoms and hormonal therapy.⁴ Axcrona et al observed how the original conceptual framework for the UCLA-PCI has been mutated through the various iterations of the EPIC and EPIC-26 instruments, specifically how the distinction between concepts of bother and function have been blurred. The authors also note that there have been very few evaluations of the EPIC-26’s domains.

The purpose of this study is to evaluate the psychometric properties of the EPIC-26 for measuring the quality of life in patients treated for localized prostate cancer. The results from this study will advance our understanding of the EPIC-26 so that clinicians can determine the appropriate use and interpretation of this instrument.

Methods

Sample

This study is based on a secondary analysis of data prospectively collected from participants enrolled in the Alberta Prostate Cancer Research Initiative (APCaRI) study.⁵ The APCaRI study is enrolling the population of men undergoing diagnosis for prostate cancer in Calgary and Edmonton, Canada. These men were referred for a diagnostic biopsy based on conventional clinical guidelines (e.g., elevated prostate-specific antigen [PSA] and/or abnormal digital rectal examination). To be eligible for the APCaRI study, men must be over 18 years of age, speak English or have a translator available, and not have had a prior prostate cancer diagnosis.

Participants in the APCaRI study consent to having their data used for secondary studies. Only participants who provided this consent were included in the analytic data set used for this study. To be eligible for this study, participants had to have been diagnosed with prostate cancer, been treated either by radical prostatectomy or radiation (i.e., external beam or brachytherapy), and completed the EPIC-26 12 months after being enrolled in the APCaRI study. Participants were excluded if they had received adjuvant androgen-deprivation therapy, as the EPIC-26 has not been validated for this treatment group. The analytic data set was comprised of EPIC-26 response data collected between September 2014 and February 2017. This study was approved by the Conjoint Health Research Ethics Board at the University of Calgary.

Data collection

Participants’ demographic characteristics were collected during the in-person interview prior to biopsy. Trained data collectors abstracted participants’ clinical data from laboratory, pathology, or treatment reports. All data were entered into a REDCap database.⁶ The APCaRI study database was de-identified and provided to our research team for analysis.

Twelve months after being enrolled in the APCaRI study, the EPIC-26 was administered either in-person or online. It was included as part of a larger battery of surveys, similar to that recommended by the International Consortium for Health Outcomes Measurement.²

The EPIC-26’s items are responded to using either a four-or five-item Likert scale. Responses are transformed to a 0–100 scale, with higher scores representing less symptom severity. Using the available scoring instructions, five items are grouped into one of five domains: urinary incontinence (four items), urinary irritative/obstructive (four items), bowel (six items), sexual (six items), and hormonal (five items). One item (i.e., “Overall, how big a problem has your urinary function been for you during the last four weeks?”) is not included in any domain. For each domain, item scores are averaged to calculate the domain summary score. Participants completed the EPIC-26 either in-person (if the participant is willing to come into the clinic), over a secure website, or by post.

Data analysis

The EPIC-26’s construct validity was assessed using a confirmatory factor analysis (CFA) with robust maximum likelihood estimation to evaluate each of the EPIC-26’s five domains. This model assumes that participants’ responses to the EPIC-26’s items can be modelled as the result of one or more latent variables. Given that each of the EPIC-26’s domain is scored separately, we assumed that they each measured their own latent trait.

Two aspects of the CFA model are of interest: overall fit and factor loadings. With respect to the overall fit of the model, five latent variables representing each of the EPIC-26’s five domains were fit, with each item loading on its respective domain. The absolute fit of the CFA model was assessed using the Satorra-Bentler corrected Chi-square test.⁷ The model demonstrates adequate fit if the p-value for this test is greater than 0.05. The Root Mean Square Error of Approximation (RMSEA), Standardized Root Mean Residual (SRMR), and the relative fit of the specified CFA model was assessed with the Comparative Fit Index (CFI) and Tucker-Lewis Index (TLI). For these statistics, adequate fit is indicated when the RMSEA and SRMR are less than 0.05, and the CFI and TLI are greater than 0.90.⁷

CFA factor loadings indicate how strongly the EPIC-26’s items measure the latent variables. These statistics can be interpreted similarly to a regression coefficient. Item intercepts represent the mean response to that item among participants. For a PRO questionnaire to be appropriate for use in a given sample and to avoid floor or ceiling effects, these intercepts should be close to the middle of the response scale,⁸ which for the EPIC-26 is 50. If the mean item response is close to 0 or 100, that item is unlikely to be able to differentiate between individuals with different levels of symptom severity.

For ease of estimation, a single imputation of missing values was generated prior to CFA using demographic, clinical, and EPIC-26 variables.⁹ If the CFA model fails to fit the data, exploratory alterations will be made to improve the model fit.⁷

The reliability of the EPIC-26’s five domains were assessed using Cronbach’s alpha. Alpha greater than 0.7 is considered acceptable, and values greater than 0.9 are considered excellent reliability.¹⁰ Item-total correlation corrected for item overlap was used to assess how strongly a given item measures the domain of interest.¹⁰ Domain correlation values were also calculated.

Ceiling effects were examined for each domain by calculating the percent of the sample that answered all items within a domain at the highest level (i.e., scores 100 out of 100). Floor effects were examined in a similar manner, calculated as the percentage of the sample that answered all a domain’s items at the lowest level (i.e., scores 0 out of 100). Domains with >15% of respondents answering at the highest or lowest level were considered to have large ceiling or floor effects, respectively.¹¹

Results

The analytic data set provided by the APCaRI included 2331 participants who had no previous diagnosis of prostate cancer. Of those, 36% (n=839) were diagnosed with prostate cancer, as defined by a total Gleason score greater than 5 whose cancer was not classified as benign. Among these participants, 33% (n=281) had been treated either by radical prostatectomy or radiation therapy at the 12-month followup, and 73% (n=205) had completed some or all of the EPIC-26. These 205 participants were included for analysis.

Missing data was minimal in this sample. No item had more than 3% missing responses, and 88% (n=180) of the study sample answered all the items. A few participants skipped entire domains: six participants did not answer any of the sexual domain items and three participants did not answer any of the hormone domain items.

The sample’s demographic and clinical characteristics collected at study enrollment are provided in Table 1. Most participants (n=188; 92%) identified themselves as being white. Most of the sample (n=89; 43%) were between the ages of 61 and 70. Most of the participants had Gleason grade of 7 (n=137; 67%) and underwent a radical prostatectomy (n=138; 67%). The mean time from treatment to completing the EPIC-26 was 33.8 weeks.

Table 1.

The sample’s demographic and clinical characteristics at the time of study enrollment

Variable	Count	Percent
Total, n	205	100.0%
Race
White	188	91.7%
Non-white	9	4.4%
Missing	8	3.9%
Age (years)
30–60	51	24.9%
61–70	89	43.4%
70 or older	27	13.2%
Missing	38	18.5%
BMI
<18.5 (underweight)	2	1.0%
18.5–25 (normal)	36	17.6%
25–30 (overweight)	76	37.1%
30–35 (obese)	26	12.7%
>35 (very obese)	15	7.3%
Missing	50	24.4%
Total Gleason grade
6	33	16.1%
7	137	66.8%
8	12	5.9%
9	22	10.7%
Missing	1	0.5%
Treatment received
Radical prostatectomy only	138	67.3%
Radiation therapy (external beam)	30	14.6%
Radiation therapy (brachytherapy)	30	14.6%
Radical prostatectomy and radiation	7	3.4%
Mean time between treatment and EPIC-26 (standard deviation)	33.8 (11.7)
Mean 12-month post-enrollment EPIC-26 domain scores (standard deviation)
Urinary incontinence	77.4 (24.7)
Urinary obstructive/irritative	89.2 (13.6)
Bowel	93.5 (12.5)
Sexual	31.7 (25.6)
Hormonal	86.4 (16.8)

EPIC: Expanded Prostate Cancer Index Composite.

The CFA model failed to demonstrate adequate overall fit (Table 2). The test of model fit using the Satorra-Bentler corrected Chi-square test was rejected (p<0.001). The values of both the RMSEA (0.068) and the SRMR (0.078) both indicate a poor model fit. The CFI statistic just met the recommended threshold of 0.90 and the TLI (0.89) was slightly below this threshold.

Table 2.

Results of confirmatory factor analysis for EPIC-26 as originally scored

Model fit statistics
Chi-squared	451.3
DF	265
Satorra-Bentler correction	1.361
Robust CFI	0.900
Robust TLI	0.887
Robust RMSEA	0.068
SRMR	0.078
Domain	Item	Loading	SE	p	Intercept	SE	p
Urinary incontinence
	1	31.6	1.4	<0.001	65.9	2.8	<0.001
	2	20.4	1.6	<0.001	78.8	1.6	<0.001
	3	18.4	2.3	<0.001	86.7	1.7	<0.001
	4	22.5	1.6	<0.001	78.3	1.7	<0.001
Urinary obstructive/irritative
	5	7.6	2.8	0.007	94.9	1.2	<0.001
	6	1.1	1.1	0.310	99.5	0.4	<0.001
	7	17.6	2.3	<0.001	86.0	1.7	<0.001
	8	19.3	2.4	<0.001	75.5	2.0	<0.001
Bowel
	10	20.8	2.0	<0.001	89.5	1.5	<0.001
	11	15.8	1.9	<0.001	92.2	1.3	<0.001
	12	10.3	2.4	<0.001	95.7	1.1	<0.001
	13	0.9	0.6	0.150	97.8	0.6	<0.001
	14	5.2	2.2	0.018	94.6	1.2	<0.001
	15	16.5	2.0	<0.001	91.2	1.4	<0.001
Sexual
	16	25.4	1.4	<0.001	19.6	1.9	<0.001
	17	22.4	1.7	<0.001	34.9	2.4	<0.001
	18	32.8	1.6	<0.001	29.9	2.5	<0.001
	19	30.8	1.8	<0.001	24.5	2.4	<0.001
	20	23.9	1.6	<0.001	24.3	2.0	<0.001
	21	7.9	2.3	0.001	55.4	2.6	<0.001
Hormonal
	22	15.9	2.8	<0.001	86.8	1.9	<0.001
	23	2.7	1.2	0.019	98.4	0.6	<0.001
	24	14.0	2.6	<0.001	87.4	1.6	<0.001
	25	23.0	2.1	<0.001	75.2	2.1	<0.001
	26	20.2	2.6	<0.001	82.8	1.9	<0.001

CFI: Comparative Fit Index; EPIC: Expanded Prostate Cancer Index Composite; RMSEA: Root Mean Square Error of Approximation; SE: standard error; SRMR: Standardized Root Mean Residual; TLI: Tucker-Lewis Index.

Examination of the items’ factor loadings on the five domains and their p-values revealed problems with several items. Item 6 (“bleeding with urination”) in the urinary obstructive/irritative domain, and item 13 (“bloody stools”) in the bowel domain had near-zero factor loadings and were non-significant. Item 14 (“abdominal/rectal/pelvic pain”) from the bowel domain, item 21 (“how big of a problem has your sexual function … been for you during the last four weeks”) from the sexual domain, and item 23 (“breast tenderness/enlargement”) in the hormonal domain also had low factor loadings.

The latent variables defined by the five domains of the CFA model exhibited a range of correlations (Table 3). Most domains had moderate, statistically significant correlations. The bowel and sexual domains had near-zero and non-significant correlation, while the urinary obstructive/irritative and bowel domains had a significant correlation of 0.57.

Table 3.

Domain correlation values

Domain		Correlation	p
Urinary incontinence	Urinary obstructive	0.22	0.030
	Bowel	0.10	0.230
	Sexual	0.17	0.003
	Hormonal	0.08	0.319
Urinary obstructive/irritative	Bowel	0.57	<0.001
	Sexual	−0.11	0.249
	Hormonal	0.31	0.005
Bowel	Sexual	−0.01	0.936
	Hormonal	0.35	<0.001
Sexual	Hormonal	0.31	<0.001

To determine if simple alterations would improve model fit, several options were considered. First, we removed items 6 and 13. Removing item 6 had no effect on model fit. Removing item 13 improved model fit, and has been recommended previously.¹² Second, we added item 9 separately to the urinary incontinence domain and urinary obstructive domain, and then to both domains together. While item 9 had the highest factor loading on the urinary obstructive/irritative domain, it had the largest impact on model fit when allowed to cross-load on both urinary domains. The bowel and sexual domains, and the sexual and urinary obstructive/irritative domains were also constrained to be orthogonal.

While the Satorra-Bentler corrected Chi-squared test of model fit remained significant (p<0.001), CFI and TLI increased only slightly to 0.91 and 0.90, respectively. RMSEA stayed at 0.068 and SRMR decreased slightly to 0.076. Item factor loadings were statistically significantly different from 0, with the exception of item 6, although several items had low factor loadings. Many items had high intercepts (indicating high mean response), including all the items in the bowel domain.

All domains had acceptable reliability except for urinary obstructive/irritative (α=0.58); none of the domains exceeded an alpha of 0.9 (i.e., excellent reliability) (Table 4). We observed a wide range of item-total correlations within each domain: urinary incontinence (0.79–0.89), urinary obstructive/irritative (0.40–0.60), bowel (0.18–0.90), sexual (0.22–0.93), and hormonal (0.34–0.74).

Table 4.

Item-total correlations, domain reliability, and ceiling effects

Domain	Item	Text	Item-total correlation	Reliability α (95% CI)	Ceiling effect n (%)
Urinary incontinence	1	How often have you leaked urine?	0.80	0.89 (0.86, 0.91)	78 (38%)
	2	Which of the following best describes your urinary control?	0.88
	3	How many pads or adult diapers per day did you usually use?	0.79
	4	Dripping or leaking urine	0.89
Urinary obstructive/irritative	5	Pain or burning on urination	0.60	0.58 (0.50, 0.65)	73 (36%)
	6	Bleeding with urination	0.40
	7	Weak urine stream or incomplete emptying	0.60
	8	Need to urinate frequently during the day	0.51
(None)	9	Overall, how big a problem has your urinary function been for you?
Bowel	10	Urgency to have a bowel movement	0.90	0.81 (0.78, 0.84)	128 (62%)
	11	Increased frequency of bowel movements	0.80
	12	Losing control of your stools	0.69
	13	Bloody stools	0.18
	14	Abdominal/pelvic/rectal pain	0.31
	15	Overall, how big a problem have your bowel habits been for you?	0.87
Sexual	16	Your ability to have an erection?	0.93	0.86 (0.83, 0.89)	0 (0%)
	17	Your ability to reach orgasm (climax)?	0.69
	18	How would you describe the usual quality of your erections?	0.88
	19	How would you describe the frequency of your erections?	0.88
	20	Overall how would you rate your ability to function sexually?	0.88
	21	Overall, how big a problem has your sexual function been?	0.22
Hormonal	22	Hot flashes	0.58	0.72 (0.67, 0.77)	74 (36%)
	23	Breast tenderness/enlargement	0.34
	24	Feeling depressed	0.53
	25	Lack of energy	0.70
	26	Change in body weight	0.74

CI: confidence interval.

Ceiling effects were observed in four of the five domains (Table 4). Approximately a third of the sample answered all the items at the highest level for the urinary incontinence, urinary obstructive/irritative, and hormonal domains. The bowel domain had the largest ceiling effects, with 62% (n=128) of participants responding to all its items at the highest level. Conversely, no ceiling effects were observed for the sexual domain, although a small floor effect was observed with 7% of responses (n=15). These ceiling effects persisted even when examined by treatment subgroups (Table 5).

Table 5.

Ceiling effects by treatment subgroups

	Radical prostatectomy		External beam radiation		LDR brachytherapy
	n=132		n=30		n=30
Domain	Mean	Ceiling effect*	Mean	Ceiling effect*	Mean	Ceiling effect*
Urinary incontinence	73.3	30.3%	95.9	50.0%	90.2	63.3%
Urinary obstructive/irritative	91.8	39.4%	86.9	26.7%	80.6	30.0%
Bowel	95.7	71.2%	91.1	50.0%	89.6	46.7%
Sexual	31.4	0.0%	17.8	0.0%	52.6	0.0%
Hormonal	90.7	42.4%	74.8	10.0%	86.2	40.0%

^*Ceiling effect is the percent with a perfect score of 100 on the given domain.

Discussion

The EPIC-26 has been used in major prostate cancer studies, such as the Comparative Effectiveness Analysis of Surgery and Radiation study¹³ and the TrueNTH Global Registry.¹⁴ It is also recommended by an international consortium for use in clinical practice for men with localized prostate cancer.¹⁵ Despite the wide use, the instrument has undergone very little evaluation of its psychometric properties. This study aimed to address this issue by assessing the validity, reliability, and ceiling effects of the EPIC-26 using data from a multicenter study of men with prostate cancer undergoing radical prostatectomy or radiation therapy.

From our results, we observed that the EPIC-26 did not meet the threshold of adequate fit in our CFA model. Even altering the model by dropping item 13 and cross-loading item 9 had very little impact on the model’s fit. One other study has conducted an exploratory and confirmatory factor analysis of the EPIC-26 using data from 651 Norwegian patients undergoing radical prostatectomy.³ Their results were largely consistent with those from this study. From the exploratory factor analysis, the authors found the best fit using a six-factor model, with the hormonal domain breaking down into two domains. When considered with the results from our CFA, it suggests that the EPIC-26’s domains are poorly structured, with their respective items not all measuring the same trait.

We also observed that the EPIC-26 has adequate reliability in four of five domains. Cronbach’s alpha values were strongest for the urinary incontinence and sexual domains, approaching the excellent threshold of 0.9. These observations are similar to those published previously, which demonstrated strong correlations between these two domains and two condition-specific PRO instruments.¹⁶ There were, however, a wide range of item-total correlations observed in all domains, with the exception of urinary incontinence. Items with low factor loadings or item-total correlation should be replaced with those that better measure the domain of interest.

We observed that the urinary obstructive/irritative domain had poor reliability and poor item-total correlation, with no item exceeding 0.6. These observations suggest that this domain is not well-defined and needs improvement.

The response format used by the EPIC-26 resulted in large ceiling effects for four domains. The bowel domain had the highest ceiling effect, with two out of every three participants responding using the highest category (i.e., no symptom bother). One-third of participants used the highest category in the urinary incontinence, urinary obstructive/irritative, and hormonal domains. As these ceiling effects have been observed in several other studies,^3,17,18 we conclude that the EPIC-26 poorly measures those patients with only slight or mild symptom severity.

One final observation is with respect to item 9 (i.e., “Overall, how big of a problem has your urinary function been during the last four weeks”), which is not scored in any domain and not accounted for in the instrument’s scoring manual. It is not clear why this item is included in the instrument, or what it adds beyond what is covered by the two urinary domains. This underscores larger problems with how the EPIC-26 was shortened from the larger EPIC instrument, an observation made by others.³

Based on our results, we recommend that the EPIC-26 undergo significant revision, starting with a review of its conceptual framework. The original EPIC distinguished between bother (i.e., symptoms) and quality of life. This distinction has been lost in the EPIC-26. Consequently, items measuring either traits are lumped together resulting in an instrument that measures neither trait very well. Future researchers should investigate creating two, distinct short-form instruments for measuring 1) prostate cancer-related quality of life; and 2) symptom severity to ensure that results are not misinterpreted and that patients are being asked questions appropriate for their situation. The timing with respect to when the EPIC-26 should be administered post-treatment also needs further investigation and empirical testing. This study has several limitations that may affect its generalizability. First, it is based on data drawn from a population of men living in Alberta, Canada. While the sample is representative of the prostate cancer population in this province, it is comprised of mainly white participants, a quarter of whom are 60 years of age or younger. While the EPIC-26 scores are similar to those reported by other studies,^13,19 the lack of diversity of this sample might limit its generalizability.

The second limitation is that this study only included those participants treated with radical prostatectomy or radiation therapy. It excluded participants opting for active surveillance or other interventional treatments (e.g., chemotherapy). Thus, the results from this study may not apply to those undergoing these treatment modalities. However, including these patients would have likely increased the observed ceiling effects because urinary, bowel, and sexual function are not usually affected by these treatment modalities. It is also important to highlight that the EPIC-26 has not been validated with these other treatments.

Conclusions

The EPIC-26 demonstrates poor construct validity, suggesting that the instrument’s five domains were not well-defined by their respective items. New items should be developed, and items should be assigned to domains based on both theoretical and empirical testing. The instrument demonstrates acceptable reliability, although this varied between domains. Large ceiling effects were also observed for most of the domains. The EPIC-26 should be revised to improve its performance for measuring post-treatment quality of life.