Exploring The Optimal Erectile Function Domain Score Cut-Off that Defines Sexual Satisfaction After Radical Prostatectomy?

Abstract

Introduction:

The International Index of Erectile Function (IIEF) is the gold standard validated instrument for defining erectile function (EF) and its response to treatment. The Erectile Function Domain (EFD) contains 6 questions and is a sensitive and specific measurement of treatment-related changes in EF. EFD has been widely used as a primary assessment endpoint for clinical trials of erectile function recovery following radical prostatectomy (RP). Various EFD scores have been used to define functional erections. Recently an EFD score of ≥22 has been used as a threshold in major post-RP penile rehabilitation studies.

Aims

To define the EFD score that optimally defines “functional” erections following RP.

Methods:

We assessed men 24 months post-RP using the IIEF and specifically analyzed the scores of the EFD and Intercourse Satisfaction Domain (ISD). We used two questions (Q) from the ISD to classify intercourse satisfaction (IS; range 0–10), Q1 on satisfaction (0–5) and Q2 on enjoyment (0–5). We tested the following classifications of IS: IS =10; IS ≥8; sum of both questions ≥4. We used the classification, which produced the greatest area under the curve (AUC) using a receiver operating characteristic (ROC) curve. We then used a three step process to determine the optimal EFD score cutoff using sensitivity and specificity analysis.

Results:

178 men had an average age at RP of 58±7 years and a 24m EFD score of 20±9. 64% had complete nerve sparing surgery, 35% had partial nerve sparing surgery, while 1% had both nerves fully resected. Concerning types of prostatectomies, 33% were laparoscopic radical prostatectomies, and 67% were open radical prostatectomies. The ROC curves produced the following areas under curve (AUC): IS =10, AUC=0.80, IS ≥8, AUC=0.85, both IS Qs ≥4, AUC=0.86 (all p<0.001). Using the IS criterion of both ISD Qs ≥4 (highest AUC), the sensitivity/specificity values for EFD scores were: 22=0.89/0.66, 23=0.78/0.71, 24=0.78/0.80, 25=0.77/0.82, 26=0.73/0.85. The scores of 24 and 25 met the criteria outlined in the first two steps of analysis. The score of 24 was selected as the cutoff using face valid judgment and previous literature.

Conclusions:

These data support an EFD score of 24 as a valid cutoff defining “functional” erection in men with ED following RP. These data are important for clinicians in counseling patients and to researchers to define inclusion criteria and treatment end-points for post-RP ED trials.

INTRODUCTION

Prostate cancer is the most frequently diagnosed solid tumor and the second most common cause of malignancy death among men in the US[1]. Radical prostatectomy (RP) is a gold standard curative treatment for early and localized prostate cancer. RP has been found to decrease the risk of prostate cancer death versus watchful waiting in men younger than 65 years old at 10 years of follow-up[2]. Erectile dysfunction (ED) is a common side effect of RP. Despite the development of nerve sparing RP, only 16% of men undergoing an RP will regain their presurgery level of erectile functioning[3,4]. This is important as ED can have a negative effect on quality of life and can cause significant distress in men and their partners. The burden of sexual bother following RP persists at significantly high levels for at least 2 years post surgery[5] and the link between ED and depressive symptoms is now well established[6].

The Erectile Function Domain (EFD) of the International Index of Erectile Function (IIEF) has been widely used as a primary assessment endpoint for clinical trials of erectile function (EF) recovery following RP. Various EFD scores have been used to define functional erections. Normally, absence of ED is defined by a EFD score ≥ 26[7], however this cutoff was developed in men with general ED as opposed to men with ED following prostate cancer treatment. Briganti et al. presented data on an EFD cutoff score in men following RP and demonstrated that patient sexual satisfaction following RP was equivalent for patients with mild ED (EFD score of 22–25) and those men with no ED (EFD score ≥26) [8]. As such, these authors concluded that an EFD score cutoff of 22 could be used to define post operative EF recovery. Subsequently this cut-off has been used as threshold in a major erectile-rehabilitation study (REINVENT AND REACCT), following radical prostatectomy[9,10].

While the Briganti study helped provide the first data quantitatively defining EF recovery with the EFD following RP, this has never been replicated. There is concern that a score of 22 might be too low to represent “true” EF recovery, and this low score may overestimate the percentage of men who “recovered” erection. Also, the method of dividing EFD scores into specific severity groups, and comparing those groups raises some methodological questions[11]. For example, if different group ranges were selected (e.g., a range of mild ED of 20–25), the authors may have easily defined a different cutoff score.

The aim of this study was to define an optimal EFD cutoff score following radical prostatectomy to define “functional” erections.

METHODS

Patient Population:

This study is part of a larger prospective quality of life study conducted with early stage prostate cancer patients at our institution. The study was approved by the Institutional Review Board and complied with the ICH Good Clinical Practice Guidelines founded on the Declaration of Helsinki. The subjects eligible for this study were men diagnosed with localized prostate cancer who were undergoing an RP and had the ability to speak English. Patients were recruited consecutively in our clinics prior to RP, and once consented completed the Prostate-Health Related Quality-of-Life Questionnaire[12]. The subjects completed this questionnaire presurgery and then every three months for two years following surgery. Since nerve healing and recovery are considered to be complete by 24 months following RP[13], the analysis in this paper will use data from the men who completed the 24 month assessments.

Outcome Measures:

The Prostate-Health Related Quality-of-Life Questionnaire is a psychometrically validated, patient self-report questionnaire which contains 63 disease-specific items that measure 11 domains: urinary, sexual, and bowel function; associated bother and role limitations domains, and cancer worry, treatment satisfaction, and regret[12]. This study utilized the Sexual Function subscale from this QOL instrument, which is represented by the IIEF[14]. The IIEF contains five domains, and this study will analyze the six questions that comprise the EFD (maximum score 30) and two of the three questions of the Intercourse Satisfaction Domain (ISD; maximum score 10). The EFD is considered the gold standard assessment of EF and is a sensitive and specific measurement of treatment-related changes in EF[15]. We included only those men who were sexually active (EFD ≥ 6) at 24 months post RP. The ISD asks about frequency, satisfaction, and enjoyment of sexual intercourse. We used only the satisfaction and enjoyment questions of the ISD. We removed the intercourse frequency question from the ISD for this analysis since physical and psychological factors related to sexual functioning after RP may impact sexual frequency and confound ISD results. Higher scores indicated better functioning in these domains.

Statistical Analysis:

All analyses were run on IBM SPSS v24 (Armonk, NY, USA). Descriptive statistics are provided to characterize the sample. We used a receiver operating characteristic (ROC) curve to determine the optimal EFD score cutoff that identifies intercourse satisfaction. The ISD as used in the analysis has two questions that have response options that range from 1 to 5 on a Likert scale. Since there is not a standard or defined cutoff for the ISD, we defined a priori three “face valid” classifications of “intercourse satisfaction” based on subjects’ scores on the ISD. The three classifications were: ISD=10 (very highly satisfying, highest score possible); ISD≥8 (fairly to very highly satisfying); or both questions of ISD≥4 (highly to very highly satisfying). We ran a separate ROC curve for each of these three definitions of intercourse satisfaction and used the one that produced the greatest Area under the Curve (AUC) as primary definition for the analysis. An AUC=1 represents perfect classification, and an AUC=0.5 indicates no better than chance. Once we determined the ISD criterion which produced the highest AUC, we then analyzed the sensitivity and specificity for each EFD score on this criterion. Sensitivity is the measure of the proportion of “positives” who are correctly identified by the cutoff score. (i.e., the proportion of men correctly identified who met the ISD criterion of sexual satisfaction). Specificity measures the proportion of “negatives” who are correctly identified by the cutoff scores (i.e., the proportion of men correctly identified who did not meet the ISD criterion of sexual satisfaction). Since there is no standard definition of intercourse satisfaction, we also repeated this analysis for the other two definitions of intercourse satisfaction to determine if there were any discrepancies between definitions. We include this as supplemental analyses to support the primary analysis.

There is a lack of established empirical guidelines to determine the optimal sensitivity and specificity[6,11]; as such, we used a three step process to define which EFD cutoff produces the optimal sensitivity and specificity in this study. First, we outlined acceptable levels of sensitivity and specificity for the purpose of determining the best cutoff score of the EFD[11]. We set the acceptable level of sensitivity and specificity at 75% as we felt this was the minimum level acceptable for this type of cutoff score. If these levels were met by more than one score, we then used the Youden Index (J) (J = maximum [Sensitivity + Specificity – 1]) to determine the highest combination of sensitivity and specificity for the remaining scores. The Youden index is recognized as one of the most reliable ways of determining an optimal cut-off score compared to visual inspection of ROC curves[16,17]. Lastly, if the criteria of sensitivity and specificity were met and Youden index was within 0.10 of the remaining possible cutoff scores, we then used a “face valid” assessment and/or previous research to determine the most reasonable cutoff.

RESULTS

433 men completed the baseline assessment of the larger QOL study and 232 men completed the 24 month assessment. Of these 232, 178 men were sexually active at the 24 month follow-up. This analysis will focus on these 178 men. 84% of men at baseline had penetration hardness erections (EFD ≥24), while 16% had EFD scores of ≤ 23. We included men with non-penetration hardness erections pre-surgery as we wanted these analyses to apply to all men who have a RP. See Table 1 for patient characteristics of the sample. All degrees of nerve sparing were included. 64% had complete nerve sparing surgery, 35% had partial nerve sparing surgery, while 1% had both nerves fully resected. Prior to surgery, 83% “never” used a PDE5i, 12% “sometimes” used a PDE5i, and 5% “always” used a PDE5i. At 24 months, 47% “never” used a PDE5i, 38% “sometimes” used a PDE5i, and 15% “always” used a PDE5i. Concerning types of prostatectomies, 33% were laparoscopic radical prostatectomies, and 67% were open radical prostatectomies.

Table 1.

Sample Characteristics

Sample characteristics
N	178
Mean Age	58 (SD=7)
Married/Partners	86%
Caucasian	94%
Pre-RP Mean EFD	27 (SD=6)
24m Mean EFD	20 (SD=9)
Hypercholesterolemia	36%
Hypertension	33%
Diabetes	2%
Smoking Status :
Current Smoker	4%
Former Smoker	14%
Never Smoker	27%
Unknown	55%

Defining the Intercourse Satisfaction Criterion:

The three a priori defined criteria for intercourse satisfaction produced a percentage of “satisfied” that ranged from 14% to 46% (See Table 2). These all produced a relatively high AUC that ranged from 0.80–0.86 and all were significant (p<0.001, See Table 2). The intercourse satisfaction criterion which produced the highest ROC of 0.86 was both questions of the ISD≥4 (Figure 1). We then used this criterion to determine the sensitivity and specificity of EFD scores.

Table 2.

AUC for Each ROC curve by IS classifications

Area Under Curve (AUC) for each ROC curve by Intercourse Satisfaction IS
IS Classification	AUC	% Reporting Sexual Satisfaction
IS=10	0.80	15%
IS≥8	0.85	46%
Both Q’s≥4	0.86	44%

Figure 1:

Diagonal segments are produced by ties.

Selecting EFD Cutoff Score

Step 1: Two potential cutoff scores met the criteria of both sensitivity and specificity > 75%. These cutoff scores were: 24 and 25 (see Table 3).

Table 3.

The sensitivity and specificity values for EFD Scores (Using IS criterion of both ISD Qs≥4)

Sensitivity and specificity
IIEF Score	Sensitivity	Specificity
22	0.89	0.66
23	0.78	0.71
24	0.78	0.80
25	0.77	0.82
26	0.73	0.85
27	0.65	0.87

Step 2: The Youden’s index for both of these scores were within 0.10 of each other. The Youden’s index (J) for each score was: 24, J=1.58; 25, J=1.60.

Step 3: We decided on the final cutoff score of 24. Since the Youden’s index for both remaining scores was within 0.10 of each other, we used two criteria to select 24. First, a score of 24 equals an average of four points (out of five points) of each of the six questions. A score of four represents an average response of “most of the time” for each question and would indicate very good erectile function. This is a “face valid” determination. Second, since previous research has suggested the cutoff score to be 22 for men following RP, this would suggest the lower score out of 24 and 25 may be the most appropriate cutoff score.

Supplementary Analysis:

In determining the intercourse satisfaction criterion, we selected the definition of intercourse satisfaction that produced the highest AUC. Since all three potential criteria produced significant AUC with values very close to each other (see Table 2), we ran the same three step process to select a cutoff for the other two potential criteria for intercourse satisfaction. The criterion of IS≥8 produced an area under the curve of 0.85. The cutoffs score of 24 and 25 met the criteria of both sensitivity and specificity > 75%, and the cutoff score of 24 produced the highest Youden’s index. The criterion of IS≥10 produced the lowest AUC of 0.80. For this criterion, there were no cutoff scores which met the criteria of both sensitivity and specificity >75%. Because this was the “strictest” criterion, it produced the lowest percent of sexually “satisfied” subjects of 14% (compared to 44% and 46% from the other two criteria). The lower sensitivity and specificity combined with the low level of satisfaction indicates this may not be an appropriate criterion. This supplemental analysis for IS≥8 and IS≥10 supports 24 as an appropriate cutoff.

DISCUSSION

In the present analysis, we found that the EFD of 24 produced the most reasonable EFD cutoff to indicated “functional” erections following prostatectomy. We used intercourse satisfaction as the criterion defined as men indicating at least a “4” on both questions of the ISD, and an ROC curve analysis identified an EFD of 24 as the optimal cutoff score. As such, we would suggest that an EFD of 24 be used to indicate men that have “functional” erections following radical prostatectomy in future clinical studies and outcomes studies where surgeons are defining the percent of patients who recover erectile function following surgery.

We used two questions from the ISD of the IIEF (Item 7 and 8 related to the satisfaction and enjoyment of intercourse) to define intercourse satisfaction. We selected the ISD because the domain has been used in analyses in previous studies to help clarify the meaning of EFD scores. The ISD has been used to supplement an ROC analyses in the development of cut points on the EFD in men with general ED[7], for the estimation of the minimal clinically important differences in the EFD[18], and for determining an EFD cutoff in men following prostate cancer surgery[8]. The National Institute of Health also considers these questions relevant to the definition of ED[19]. As stated in the methods section, we removed the intercourse frequency question from the ISD for this analysis since physical and psychological factors related to sexual functioning after RP may impact sexual frequency and confound the intercourse satisfaction results. Since there is no standard or defined cutoff for the ISD, we defined three “face” valid definitions of “satisfied” on the ISD. Although we selected one of these three definitions as the primary definition for the analysis, we ultimately ran the analysis for all three of these definitions. All three indicated 24 was the more appropriate cut-off score and the consistency among these three definitions supports the results of this study.

Our suggested cutoff score of 24 is between the cutoff of 26 outlined in the original IIEF validation study and the cutoff of 22 suggested by Briganti et al. for men following radical prostatectomy[7,20]. The differences between the methods and analyses presented in this paper and those of the original validation study are relatively clear. The studies which suggest a cutoff of 26 were conducted in men who had general ED as opposed to ED associated with prostate cancer treatment, and used a the criterion of men with ED vs. men without ED to anchor the ROC curve analysis.

Our paper and the Briganti et al. paper both investigate the cutoff score in men following radical prostatectomy and use intercourse satisfaction as the criterion variable (Briganti et al. also used “overall satisfaction” domain of the IIEF). This intercourse satisfaction criterion is used because it is difficult to define a group of men without ED following prostate cancer surgery. The vast majority of men, 85%, report at least some difficulty with erections following prostate cancer treatment[21], and only 16% of men will get back to their baseline erectile functioning pre-RP[4]. The primary difference between our study and the Briganti et al. study is the analytic methodology. These authors divided the men into three ED severity groups as outlined in the initial validation studies of the IIEF (moderate to mild ED: EFD score of 17–21; mild ED: EFD score of 22–25; and no ED: EFD score of 26–30)[8]. There was no difference in intercourse satisfaction between the mild ED and no ED group, and the moderate to mild ED group reported lower intercourse satisfaction scores. The authors then concluded that men who scored a 22 reached a threshold of intercourse satisfaction relatively equal to men who scored above 22. The method of dividing EFD scores into specific severity groups, and comparing those groups raises some methodological questions[11]. For example, if different group ranges were selected (e.g., a range of mild ED of 20–25), the authors may have easily defined a different cutoff score. Also, the group ranges that Briganti et al. used were the defined ranges of the EFD in men without prostate cancer and these ranges may not be appropriate to apply to men following RP. In fact, if they were appropriate, there would be no need to develop a different cutoff score. The study samples are also different. Briganti et al.’s sample excluded men who had ED prior to surgery and included only those who had bilateral nerves sparing surgery. Our study did not have inclusion/exclusion criteria related to erectile quality prior to surgery or type of nerve sparing surgery.

We believe the methodology used in our study is appropriate for selecting a cutoff score. First, we considered three different criteria for intercourse satisfaction. We then used an ROC curve analysis, which is a standard methodology used to develop cutoff scores. Finally, we used multiple criteria to weigh the sensitivity and specificity of the potential cutoff scores, and used the results presented by Briganti et al. as part the decision making process. Thus, we would argue that a score of 24 would be the most appropriate cutoff score. At 24 months the nerves are considered to be fully healed and any recovery that will take place is thought to be complete[13]. As such, we thought this was the most appropriate time point to use. The goal was to develop a cutoff that indicated “functional” erections following surgery. If someone reaches an EFD of 24 at 1 year, we believe these data would still apply and they would be considered “functional” erections.

Our report suggest that IS decrease following RP even in men with good erectile function score. Rossi el al reported that Overall Satisfaction (OS) significant decrease following RP even in men who report Back to Baseline (BTB) erections post surgery. among 383 patients who were able to achieve BTB erection only 26,9% were satisfied and OS was highly correlated with EF score and ability to achieve BTB erections[22].

While we believe our study has several strengths (large sample size, rigorous statistical analyses), there are also limitations. The subjects of the study were from one center only. Although there are multiple surgeons at our center, results may vary at other institutions. Second, intercourse satisfaction is to a great extent dependent on the partners, and there were no partner assessments included in the study. Third, these data were collected as part of a larger study, and the methods were not designed specifically to address the aims of this manuscript. Fourth, of the 438 who completed the baseline assessment of the larger QOL study, only about half (232 men) completed the 24 month follow-up and 178 men were sexually active at the 24 month follow-up. The fact that our cutoff differs from that of Briganti et al. suggests that future research is needed on the validity of our EFD cutoff score following RP. Results from multiple centers with potentially different methodology may help clarify which score is most appropriate.

CONCLUSIONS

An EFD score of 24 represents the optimal cut-off for prediction of intercourse satisfaction in men with ED after RP. These data may be important when defining an erectile function score that defines “functional” erection after treatments such as radical prostatectomy and for defining inclusion criteria and treatment end-points for post-RP ED trials.