Patient reported outcome measure‐haemorrhoidal impact and satisfaction score (PROM‐HISS): Development, reliability and construct validity

Abstract

Aim

Haemorrhoidal disease (HD) is a frequently occurring disorder with a significant negative impact on a patient's quality of life. Here, we describe the development and validation of the Dutch patient reported outcome measure‐haemorrhoidal impact and satisfaction score (PROM‐HISS).

Methods

The development of the PROM‐HISS followed recommended guidelines. Face and content validity, structural properties, reliability and construct validity were evaluated in a HD population. Reliability was tested by assessing the test‐retest reliability, defined by the intraclass correlation coefficient (ICC), and internal consistency measured with Cronbach's alpha. Construct validity was evaluated using confirmatory factor analysis (CFA) and hypotheses testing.

Results

The PROM‐HISS consists of three domains: (1) HD symptoms (blood loss; pain; prolapse; soiling; itching), (2) impact of symptoms on daily activities, and (3) satisfaction with treatment. The PROM‐HISS showed good face and content validity. The PROM‐HISS was completed by 102 patients (65% male), with a mean age of 58 years (23–81 years). The ICCs of the different items in the domain HD symptoms ranged between 0.56 and 0.79 and were interpreted as good. The Cronbach's alpha value was 0.80 and considered satisfactory. The CFA provided further evidence for construct validity with a good model fit. A high score on the symptoms of HD correlated with a high impact of HD on daily activities (Pearson's r = 0.632, p < 0.01) and a low degree of satisfaction (Pearson's r = 0.378, p < 0.01).

Conclusion

The PROM‐HISS is a reliable and valid instrument to evaluate symptoms of HD, impact on daily activities and satisfaction with treatment.

What does this paper add to the literature?

The patient reported outcome measure‐haemorrhoidal impact and satisfaction score (PROM‐HISS) evaluates the symptom burden of haemorrhoidal disease, impact on daily activities and satisfaction with haemorrhoid treatment. It shows sound structural properties, internal consistency and construct validity. We endorse the use of the PROM‐HISS in research settings and clinical practice.

INTRODUCTION

Haemorrhoidal disease (HD) is a frequently occurring disorder with a significant negative impact on a patient's quality of life [1]. HD affects a large number of people worldwide, with prevalence rates ranging between 4.4% and 36.4% in the general population [2, 3].

While there are ample clinical studies evaluating the effectiveness of varying HD treatment strategies, there is a lack of uniform outcome definition, measurement and reporting in the research publications. This limits research quality and complicates evidence synthesis [4, 5]. Hence, the European Society of Coloproctology (ESCP) has recently developed a core outcome set (COS) to achieve standardization of outcomes and outcome measurement in HD studies. The primary outcomes of the COS were symptoms of HD and the secondary outcome was treatment satisfaction [6, 7]. Symptoms and satisfaction should both be reported by patients and can be evaluated using a patient‐reported outcome measure (PROM). A PROM collects information directly from the patient without interpretation by a healthcare professional or others [8, 9, 10]. A recently published systematic review aimed to determine the most appropriate instruments that classify the severity of HD disease according to symptoms, identified five studies describing the development and validity of PROMs and scoring systems based on core symptoms reported by patients [11]. Nevertheless, these measures have several drawbacks and an established PROM for haemorrhoids is missing. In some cases, several psychometric properties of the PROM were not tested, for example, validation, responsiveness [12]. In others, the tools were not developed in our population of interest (patients with HD), and no input from this group was asked in the development process [13, 14]. Lastly, consensus‐based standards for designing and reporting validation research were not used [15, 16].

Hence, we decided to develop a PROM that specifically addressed the primary outcomes of the ESCP COS in strong collaboration with HD patients. In taking on this endeavour, we closely followed the Consensus‐Based Standards for the Selection of Health Measurement Instruments (COSMIN) guidelines development and testing [17]. The COS, which was dominated by a health care professional view, and the patient interviews, served as a fundament for the development of the patient reported outcome measure‐haemorrhoidal impact and satisfaction score (PROM‐HISS) for HD. This study describes the development and validation of the Dutch PROM‐HISS.

METHODS

The development of the PROM‐HISS consisted of several steps, following recommended guidelines for the development and validity of health status questionnaires [18, 19]. A panel of experts included a health outcomes researcher, a colorectal surgeon, two clinical statisticians, a health technology assessment expert and two researchers in the field of coloproctology.

Step I. Development of the PROM‐HISS

A first collection of items relevant for inclusion in the PROM‐HISS, was generated by combining topics and symptoms derived from literature review of existing measures (e.g., the Sodergren score [12], the haemorrhoidal symptom score [15]), individual patient interviews [20], and discussion with the panel of experts. The initial relevant items were grouped in an overarching framework of domains and further evaluated in a face‐to face meeting with the panel of experts. Several aspects of the PROM‐HISS concept were further elaborated on; that is, initial items and their instructions, minimum age, the variety of response options, and appropriate recall period. By combining the input from the panel of experts, a first version of the PROM‐HISS was developed. The process of assessing the items in a face‐to face meeting was repeated several times and alterations to the items were based on majority agreement of the panel of experts. Consensus on the first version of the PROM‐HISS was reached when no other adjustments were made. This version was used in the consecutive testing steps.

Step II. Evaluation of face and content validity

Individual patient interviews were conducted to assess the face and content validity and the time needed to fill out the questionnaire.

Both male and female adult patients (≥18 years old) diagnosed with HD grades I–IV were invited by their treating physician to participate in the face and content validity test. Patients were interviewed at the outpatient clinic, face‐to‐face, by one of two trained interviewers (SK and RT). A cognitive verbal probing technique was used [21]. After initial completion of the questionnaire the participant was asked the following questions for each item individually: (1) “did you understand the item?”, (2) “is the item relevant to you?” (3) “were you able to retrieve the information required?”, and (4) “were you able to make a judgement?”. Finally, participants were asked whether they felt important items or domains were missing.

Step III. Evaluation of structural properties, reliability and construct validity

The COSMIN‐methodology was used to assess the structural properties, reliability, and construct validity of the PROM‐HISS [22]. Data were collected prospectively between April 2020 and February 2021 from Dutch patients older than 18 years with HD grade I–IV. Patients were recruited from one hospital and were identified using hospital records. Patients who had visited the outpatient clinic in the previous year because of complaints of HD were invited by their treating physician via email. Permission was requested electronically and after obtaining written informed consent the PROM‐HISS and the EQ‐5D‐5L, a measure of health‐related quality of life (HRQoL), were sent to the patients by mail [23]. Data on age, sex, and grade of HD of each participant were retrieved from the EPF.

Structural properties

A stacked bar chart graphically presents the distribution of scores and percentage of missing values. Floor and ceiling effects were considered present when at least 15% of respondents scored the lowest or highest possible score, respectively [24].

Reliability

Reliability includes test‐retest reliability and internal consistency measures. For the test‐retest reliability, all participants were asked to complete the PROM‐HISS a second time, 1 week after initial completion. The intraclass correlation coefficient (ICC) was used as a measure of the reliability of the questionnaire [25]. ICC estimates and their 95% confident intervals (CI) were calculated for each item based on absolute agreement and a 2‐way mixed‐effects model and positively rated when at least 0.70 [18].

Internal consistency is the degree of the interrelatedness among items within a domain or construct. Internal consistency is relevant for the five items in the domain “symptoms”. It assesses whether the symptoms are correlated (homogeneous), thus measuring the same concept. The internal consistency was estimated by calculating the Cronbach's alpha. A low Cronbach's alpha indicates a lack of correlation between the items. The internal consistency was considered good when Cronbach's alpha was between 0.70 and 0.95 [18].

Construct validity

A confirmatory factor analysis (CFA) with robust maximum likelihood estimation was performed to test whether the data fit a premeditated factor structure [26]. The CFA test was used for the symptom domain consisting of five items. For the CFA test the Chi‐square, the root mean square error of approximation (RMSEA) and the comparative fit index (CFI) were taken into account. The Chi‐square index assesses the fit between the hypothesized model and data from a set of measurement items (the observed variables). The RMSEA is a measure for model fit. Furthermore, the CFI assesses the model fit by analysing the discrepancy between data and the hypothesized model. We hypothesize that all five symptoms form one model. Convergent validity was evaluated by means of the average variance extracted (AVE), which provides information about the amount of variance that is captured by the construct and should exceed 0.50 [27].

Upon findings of good internal consistency and model fit of the domain symptoms, a sum score can be calculated by coding responses for each item on a 1–5 scale and averaging responses. A higher sum score (range 1–5) represents a higher symptom burden. This sum score can then be used for additional assessment of the construct validity.

Construct validity was further evaluated by testing a set of hypotheses about expected relationships between the PROM‐HISS and another high‐quality comparator instrument, the EQ‐5D‐5L. The EQ‐5D‐5L examines patient HRQoL and consists of a descriptive system and visual analogue scale (VAS). The descriptive system comprises five dimensions: mobility, self‐care, usual activities, pain/discomfort, and anxiety/depression. The VAS records the patient's self‐rated health [23]. We hypothesize that a high score in the symptoms of HD correlates with a lower HRQoL score. Furthermore, we hypothesize that a high sum score on the symptoms of HD measured by the PROM‐HISS correlates with a high impact of HD on daily activities and a low degree of satisfaction of treatment, as previously reported in the literature [20].

Finally, we investigated whether scores on the domains of the PROM‐HISS (e.g., symptoms, impact on daily activities, and treatment satisfaction) varied between predefined subgroups of patients in a manner that is consistent with a priori hypotheses. We expected varying scores for the two different HD groups; low HD grade group:grade I and II HD, and high HD grade group:grade III and IV HD. More specifically, we tested if the higher grade HD group is associated with higher symptom scores (i.e., more severe symptoms) and a higher (negative) impact on daily activities by using an independent student's t‐test.

Statistical analyses were performed using the SPSS statistical package version 25 (IBM SPSS Statistics 25) and R version 3.6.1.

RESULTS

Step I. Development of the PROM‐HISS

Based on the patient interviews reported earlier and the meetings with the panel experts, the first version of the PROM‐HISS was drafted [20]. This version consisted of the following three domains: (1) HD symptoms, (2) impact of HD on daily activities, and (3) satisfaction with treatment. The first domain comprises of five symptom‐items including blood loss, pain, prolapse, soiling and itching. Both the second and the third domain contain one item; impact of HD on daily activities, and patient's satisfaction with treatment, respectively. The five symptom‐items of the first domain are graded using a 5‐point Likert scale, ranging from (1) “not at all”, (2) “a little”, (3) “reasonable”, (4) “a lot”, and (5) “very much”. The remaining two domains are each scored on a numeric rating scale from 0 to 10. Regarding the impact of symptoms, 0 correlates with “no impact at all” and 10 with “highly impacted on daily activities”. For patient satisfaction with treatment, this ranges between 0 “not at all satisfied” and 10 “very satisfied”. The recall period for the items in the PROM‐HISS is ‘in the past week’.

Step II. Evaluation of face and content validity

Ten patients, with a mean age of 56 years (37–73) and equal males and females, were interviewed. Six patients (60%) were diagnosed with Goligher grade II HD, three (30%) with grade III;, and one (10%) with grade IV. The majority (90%) had received a rubberband ligation treatment for HD. Items were generally well understood by patients and considered relevant. On average, the questionnaire was completed in 3 min (range 1–5 min).

Two respondents indicated that they missed an opportunity to vent their feelings of shame and embarrassment about HD. Expert panel agreement was reached that this aspect is considered outside of the scope of the PROM‐HISS that aimed to focus on HD specific symptoms and impact of symptoms on daily activities. Henceforth, the PROM‐HISS was considered suitable to take forward into further validity testing.

The PROM‐HISS was developed in Dutch (Appendix S1). See Table 1 for an overview of the provisionally translated domains and items of the PROM‐HISS.

TABLE 1.

Domains, items and response options of the PROM‐HISS

Domain	Items	Response options
Symptoms	Blood loss Pain Prolapse Itching Soiling	Likert scale – 1 (not at all) to 5 (very much)
Impact on daily activities	Impact on daily activities	Scale – 0 (no impact at all) to 10 (highly impacted on daily activities)
Satisfaction with treatment	Satisfaction with treatment	Scale – 0 (not at all satisfied) to 10 (very satisfied)

Step III. Evaluation of structural properties, reliability and construct validity

The PROM‐HISS was sent out to 236 patients, of which 102 patients completed the questionnaire at least once. This group consisted of 66 males (65%) and 36 females (35%), with a mean age of 58 years (23–81 years) and diagnosed with HD grade II (55%) or III (39%). Of the 102 patients, 91 (89%) had recently received a treatment for their haemorrhoids, either once (26%), twice (24%) or three or more times (40%). In most cases the treatment concerned rubber band ligation (88%), and in some, an operation (10%).

Demographic characteristics of the cohort are summarized in Table 2. A study flowchart of this process is shown in Figure 1.

TABLE 2.

Demographic characteristics

Characteristics	n = 102
Sex, n (%)
Women	36 (35%)
Men	66 (65%)
Age, mean, [range], year	58.4 [23–81]
Goligher's classification, n (%)
Grade I	2 (2%)
Grade II	56 (55%)
Grade III	40 (39%)
Grade IV	4 (4%)
Treatment, n (%)
Treatment(s) received	91 (89%)
If yes, treatment specified*
Rubber band ligation	90 (88%)
Operative treatment	10 (10%)

(n = number, y = years) ^*Patients could have received both RBL and an operation.

FIGURE 1.

Flowchart of the participating patients of the Evaluation of structural properties, reliability and construct validity

Structural properties

As the PROM‐HISS was sent out digitally and participants could not proceed to the next question without having completed the former, there were no missing data.

The distribution of response levels was satisfactory for all items of the domain “symptoms”, see Figure 2. All items showed responses across the full range of response options. The mean scores of the different items were 1.8 (SD 1.0) for blood loss, 2.0 (SD 1.1) for pain, 2.3 (SD 1.6) for prolapse, 1.9 (SD 1.0) for itching and 1.9 (SD 1.1) for soiling. Within the domain “impact on daily activities” the range of response options was used with a mean score of 2.6 (SD 3.2). The same trend was seen in the domain “satisfaction with treatment”, which was rated with a mean of 6.5 (SD 2.8). However, floor effects were identified in all items of the domain “symptoms”. Likewise, in the domain “impact on daily activities” a floor effect was seen, with 50% of respondents reporting a score below 1. No ceiling effects were identified in the three domains.

FIGURE 2.

Stacked bar chart of response distribution of the domain “symptoms” of the PROM‐HISS (n = 102)

Reliability

All patients received the PROM‐HISS and EQ‐5D‐5L twice, and eighty‐four patients (82%) completed the second questionnaire. The mean time between test and retest was 11 days (range 6–34). The ICCs (95% CI) of the different items in the domain “symptoms” were: blood loss 0.56 (0.39–0.69); pain 0.61 (0.46–0.73); prolapse 0.67 (0.53–0.77); itching 0.72 (0.60–0.81); and soiling 0.79 (0.69–0.86).

The Cronbach's alpha value was 0.80 and fell within the recommended range of 0.70–0.95, providing evidence for an internally consistent (homogeneous) scale for symptoms [18].

Construct validity

To assess the construct validity, a CFA of the domain “symptoms” was performed (see Figure 3). Chi‐square indicated that the covariance matrix derived from the model represents the population covariance (p = 0.67) and represents an acceptable model. The RMSEA was zero and the CFI value exceeded 0.95 (1,000), suggesting a model with satisfactory fit. Convergent validity was considered good, with moderate to large sizes of all factor loadings and an AVE above 0.50 (i.e., 0.54). Findings from the internal consistency testing and the CFA support the use of a sum score for the domain “symptoms”, in order to further assess construct validity.

FIGURE 3.

Depiction of the confirmatory factor analysis latent variable standardized solution model of the domain symptoms

As postulated, a high (sum) score on the symptoms of HD measured by the PROM‐HISS correlates with a high impact of HD on daily activities (Spearman's Rho = 0.668, p < 0.01, 2‐tailed), and a low degree of satisfaction (Pearson's r = 0.378, p < 0.01, 2‐tailed).

An overall high score of symptoms of HD on the PROM‐HISS was linked to a lower score of HRQoL as measured by the EQ‐5D (Pearson's r = 0.574, p < 0.01, 2‐tailed). Contrary to our expectations, we did not find a significant difference between low or high grade HD and the symptom score (2.0 ± 0.8 vs. 2.0 ± 0.8, respectively, p = 0.79, 2‐tailed) nor between low or high HD grade and impact on daily activities (2.5 ± 3.4 vs. 2.7 ± 3.1, respectively, p = 0.81, 2‐tailed).

DISCUSSION AND CONCLUSION

The present study details the development of the PROM‐HISS and reports on its psychometric properties. The PROM‐HISS evaluates the severity of HD symptoms, impact on daily activities and satisfaction with treatment.

Structural properties of the PROM‐HISS were satisfactory, with good response distribution. Overall, low categories were marked in the domain “symptoms” with floor effects in all items. Similarly, floor effects were seen in the domain “impact on daily activities”. This may be explained by the fact that our study population consisted of patients who had received a rubberband ligation treatment at the outpatient clinic in the last year. The timing in the treatment trajectory may have resulted in a reduced number and severity of HD symptoms and impact on daily activities. However, the PROM‐HISS is specifically developed to evaluate symptom burden and treatment satisfaction throughout the patient journey.

Test‐retest reliability measured by the ICC was high, exceeding the 0.70 bar in two out of five symptoms, indicating that the PROM‐HISS can yield consistent results on different timepoints. However, for the symptoms “blood loss”, “pain” and “prolapse” this benchmark was not reached. A probable explanation is that these symptoms are more prone to day‐to‐day fluctuations than “itching” and “soiling”.

Good model fit of the CFA suggests that the data adequately represents the underlying framework of the domain “symptoms”. Furthermore, high factor loadings support convergent validity of the measure. Overall, the measure has a good internal consistency and model fit. Its unidimensionallity allows for its items to be pooled into a summary score for the domain “symptoms”, facilitating comparisons between patients.

This study confirmed our hypothesis that if a patient suffers greatly from his or her symptoms of HD, this will translate in a high impact of HD on daily activities and low treatment satisfaction. Contrary to our expectations, we found that neither the score on the symtoms domain nor impact on daily activities differed significantly between a low or high grade of HD. This indicates that the experienced burden and severity of symptoms is not related to the grading of HD and therefore that the Goligher classification, that is used to assign the grades, may not include aspects of the disease relevant to the patient [28].

The strength of the PROM‐HISS is that it has the potential to support an evidence‐based approach to surgical data and that it provides an insight into patients’ experiences with HD in a quantitative and systematic manner. Compared to generic measures (such as the SF‐36), a condition‐specific PROM, like the PROM‐HISS, assesses a particular aspect of health or a condition, relevant to the patient, and is generally more sensitive to detect improvements (or deteriorations) in health or differences in effectiveness between treatments [29]. The PROM‐HISS is not the first of its kind. However, it is the first PROM for HD to be developed in conjunction with the HD patient population and closely following the COSMIN guideline for the development of health outcome measurement instruments [22].

There are some limitations that need to be addressed. First, most participants were Caucasian, and it concerned a Dutch population. To establish cross‐cultural validity, the PROM‐HISS should be translated and tested in different cultural environments. Second, the participants were mostly diagnosed with grade II or III HD, and had attended the outpatient clinic for treatment in the last year. Patients with grade I and IV HD were underrepresented in this study. Third, the psychometric aspect of responsiveness has not been addressed in this study. Longitudinal data are needed to assess responsiveness. However, this aperture will be tackled in the near future using PROM‐HISS data that are currently being collected in two large Dutch randomised controlled trials [30, 31].

During the face and content validity testing, it was mentioned that the psychological burden related to HD was missed in the PROM‐HISS. Yet, the focus of the PROM‐HISS lies primarily in the physical aspects of HD and its impact on daily activities. Despite not adding the concept of psychological burden of HD to the PROM‐HISS, we do recommend health care providers to take this aspect of HD into account during consultation.

The PROM‐HISS is a unique PROM evaluating the HD symptom burden, impact on daily activities and satisfaction with HD treatment. The PROM‐HISS shows sound structural properties, internal consistency and construct validity. We endorse the use of the PROM‐HISS in research settings and clinical practice. Further research is encouraged to examine the responsiveness of the PROM‐HISS and clinically relevant difference, and asses, next to its use in clinical trials, its merits to inform treatment decisions in clinical practice.

PATIENT CONSENT STATEMENT

All patients provided written informed consent to participate in the study.

CONFLICT OF INTEREST

The authors declare that they have no conflict of interest.

AUTHOR CONTRIBUTIONS

SZK, MLK and RRT conceived the study. SZK, MLK, RRT and SOB planned the study. SZK and MLK collected and prepared the data. SZK, MLK, SFW and SMJK made the statistical analysis, and all authors interpreted the data. SZK, MLK wrote the manuscript. SOB and CDD conducted critical revisions of the manuscript.

ETHICAL APPROVAL

Ethical clearance for this study was obtained from the Medical Ethics Committee of Maastricht, the Netherlands, with reference number 2018–0889.

Supporting information

Appendix S1

Kuiper SZ, Kimman ML, Van Tol RR, Waardenburg SF, Van Kuijk SMJ, Dirksen CD, et al. Patient reported outcome measure‐haemorrhoidal impact and satisfaction score (PROM‐HISS): Development, reliability and construct validity. Colorectal Dis. 2022;24:992–999. 10.1111/codi.16079

DATA AVAILABILITY STATEMENT

The data that support the findings of this study are available from the corresponding author upon reasonable request.