WEB VS PAPER-BASED COMPLETION OF THE EPIDEMIOLOGY OF PROLAPSE AND INCONTINENCE QUESTIONNAIRE (EPIQ)

Abstract

Objectives

To examine the validity and reliability of a web-based version of the Epidemiology of Prolapse and Incontinence Questionnaire (EPIQ).

Methods

Participants included 876 women ages 38 to 65 attending primary care clinics in the Salt Lake Valley. Women completed a single web or paper based version of the symptom bother questions from EPIQ, and a subset repeated the same or opposite method at 2 separate time points. To assess subscales for the web-based version factor analysis of the 22 EPIQ items related to pelvic floor disorder (PFD) symptoms was performed using principal components analysis and varimax rotation. Internal consistency was assessed using coefficient alpha. Test-retest and inter-method reliability were assessed using intraclass correlation coefficients (ICC) for domain scores. Correlations above 0.70 were considered acceptable.

Results

Overall, 384 and 492 women completed at least 1 web and 1 paper EPIQ and 93% were Caucasian with mean age of 50±7 years. Of these, 63 completed web-web, 57 web-paper, 47 paper-web and 109 paper-paper test-retest. Overall, factor analyses were consistent with the 7 domains of the original EPIQ. Cronbach’s alpha for the 4 symptomatic PFD domains and range of test-retest reliability for the various administration methods were similar to the original EPIQ instrument. Correlations for domain scores were above 0.70, except the anal incontinence scale (ICC=0.68.)

Conclusions

Web administration of the EPIQ has similar psychometric properties with comparable internal consistency and test-retest reliability when administered in the same modality. Reliability between both methods of administration is acceptable.

INTRODUCTION

The Epidemiology of Prolapse and Incontinence Questionnaire (EPIQ) is a self-administered instrument designed to identify prevalence and risk factors for bothersome female pelvic floor disorders (PFD) in population based research.¹ The entire questionnaire is a 16-page document that includes demographic, medical and obstetric history along with 22 stem questions related to urinary and bowel symptom bother as measured on 100 mm visual analog scales. Unlike most condition specific instruments used in pelvic medicine, it was specifically validated in a diverse population of women both seeking as well as not seeking care for PFD.¹ The EPIQ addresses a broad spectrum of symptoms related to pelvic floor, urinary and bowel function, with reported predictive values for identification of bothersome stress urinary incontinence (SUI), overactive bladder (OAB), anal incontinence (AI) and pelvic organ prolapse (POP).

While the EPIQ was developed and validated as a self-administered written instrument, there are limitations to this method of administration in the electronic age.² There may be several advantages to completing this type of questionnaire electronically in future research. For example, skip patterns and quality control parameters can be made automatic, data entry errors can be minimized, costs associated with printing, distributing and data entry are obviated, and data are rapidly available.³ A web-based entry system is ideal for population-based studies in which participants do not otherwise need to present to a clinical office for evaluation or form completion.

The Pelvic Floor Distress Inventory (PFDI) and the International Consultation on Incontinence Modular Questionnaire-Lower Urinary Tract Symptoms Quality of Life (ICIQ-LUTSqol), instruments that assess pelvic floor symptom burden in individuals presenting for care for pelvic floor disorders, have established reproducibility when administered by web and paper.^14,15 The aims of this study were to examine the stability, internal consistency, and test-retest reliability of a web-based version of the EPIQ; and to assess inter-method reliability compared to the standard paper-based administration, in a population of community-dwelling and primary care participants not seeking care for PFDs.

MATERIALS AND METHODS

This study was approved by the University of Utah Institutional Review Board, and all participants signed informed consent. English speaking participants were recruited from an ongoing case-control study whose aim is to estimate the association between lifetime physical activity and current pelvic organ prolapse and stress urinary incontinence.⁴ Women not seeking care for PFDs were recruited from over 20 primary care level gynecology and family medicine clinics located across the Salt Lake Valley and from community advertising. The exclusion criteria for the parent study included: current pregnancy or within six months postpartum, < 39 or > 65 years of age, body mass index (BMI) < 18.5 kg/m2 or ≥ 40 kg/m2, history of prior surgical treatment for prolapse or incontinence, predominant urgency incontinence as identified by the 3-IQ, inability to walk independently, major medical problems precluding exercise for the last 12 months, medical conditions associated with incontinence or low physical activity (including diabetes, neurologic disorders such as multiple sclerosis, spinal cord injury, or stroke; rheumatoid arthritis, blindness, radical hysterectomy or pelvic irradiation), currently undergoing treatment for cancer, or unable to complete questionnaires. Of those recruited to the parent study, 19 women had ages or BMI above or below the inclusion cut-point for the primary study; however, they were included in this EPIQ validation study. All women enrolled by February, 2012 who completed at least one version of the EPIQ were included in these analyses.

Participants in the parent study were given the option of participating in ancillary studies aimed at evaluating the reproducibility of web versions of previously validated questionnaires including the Lifetime Physical Activity Questionnaire (LPAQ)⁵ and the EPIQ¹. These participants completed the study instruments at home, either once by web; once by paper; twice by paper, twice by web, or once by web and once by paper. Written forms were returned by mail. For those who agreed to complete both web and paper instruments, the order of administration was randomized to web vs. paper first, by opening sealed opaque envelopes. Women completed the EPIQ only after administration of the LPAQ and the Occupation Questionnaire⁶ from a different historical activity instrument, which resulted in a lengthy battery of questions regarding leisure, household and outdoor activity over various age epochs. We did not assess time for completion of the series of instruments, however it was estimated that the LPAQ would take approximately 45 minutes to complete.

The aims of this study were to examine the stability, internal consistency, and test-retest reliability of the 22 items related to PFD in a web-based version of the EPIQ. Thus, we first determined whether the same set of domains, or underlying factors, were identified in this population as compared to the original population in which the instrument was validated. This was performed using principal components factor analysis with varimax rotation in SAS 9.3 on the 22 EPIQ items about presence and severity of various symptoms. Factor analysis was based on women who first or solely completed a web-based version of the EPIQ. When a woman answered negatively about the presence of a symptom, the accompanying visual analogue scale response about bother was set to zero as was established in the original validation study. In the original EPIQ validation, the following domains emerged: stress urinary incontinence (SUI), overactive bladder (OAB), anal incontinence (AI), and pelvic organ prolapse (POP); plus three additional factors for which criterion validity could not be established: incontinence-related quality of life (QOL), defecatory dysfunction (DD), and pain and difficult voiding (PDV). As in the original study, we identified the number of factors with eigenvalues greater than 1.0 and performed varimax rotation to determine whether the original subscales were reproduced for the web-based version. Factor loadings of at least 0.4 were considered meaningful.

Each subscale was then represented by an unweighted sum of its items for further analysis. We estimated internal consistency of the web-based version using alpha coefficients for each subscale, except the POP subscale, which has only one item. For women who completed one web-based and one paper-based version of the EPIQ, we tested whether the alpha coefficients differed between the web-based and paper-based versions.⁷ Criterion validity was not assessed in this population as women with PFD were not specifically included and definitions for identifying symptomatic PFD were not consistent with the original EPIQ description. As noted, the goal of the EPIQ is to identify women bothered enough by a PFD to seek treatment; in the original validation study, half of women were recruited from urogynecology clinics (i.e., seeking care for a pelvic floor disorder) and half from a general gynecology clinic. In our parent study, we specifically recruited women not seeking care and so would not expect to have the full spectrum of disease needed to assess criterion validity. We assumed that if the web version was found to be reproducible, maintained the same internal consistency and loaded on the same factors, there would be no need to re-assess criterion validity.

For women who completed two web versions of the EPIQ, we assessed test-retest reliability using intraclass correlation ICC. ⁸ For women who completed one web-based and one paper-based version of the EPIQ, we calculated the inter-method correlation as an intraclass correlation ICC for each subscale. We also scrutinized Bland-Altman plots for bias and outliers.⁹ The presence or absence of SUI, OAB, AI and POP was determined using formulae noted in the original validation paper. Demographics and subscales were compared across unmatched study groups by independent samples t-tests, Pearson’s chi-squared test, or Fisher’s exact test, as appropriate; and across the same study group on two occasions by paired t-tests or McNemar’s chi-squared test. Correlations were considered acceptable if above 0.70.¹⁰ Statistical tests were two-sided, at the conventional 5% significance level.

A priori sample size calculations estimated that the sample size would be adequate as follows: In the comparison of alpha coefficients for the web- versus paper-based EPIQ, 100 women who completed both versions would provide 80% power to detect Feldt’s delta=1.67, e.g., Cronbach’s alpha of 0.8 versus Cronbach’s alpha of 0.666.⁷ 51 women completing the web-based (or paper-based) EPIQ twice would provide 80% power to detect a single Pearson correlation of 0.8 versus 0.65; and similarly for web-based vs. paper-based comparisons. The final sample size was ultimately limited by constraints of the parent study.

RESULTS

Of the 1093 women enrolled into the parent study at the time of this data analysis, 876 completed at least 1 EPIQ survey adequately; 384 completed at least one web-based EPIQ survey and 492 at least one paper-based EPIQ. Of these 876 women, 496 agreed to complete a second EPIQ and 276 (55.6%) actually did so. Of the 496 that agreed to complete a second EPIQ, 213 were randomly assigned to complete paper-then-web surveys or web-then-paper surveys and 104 (49%) actually did both. Participant flow is summarized in Figure 1. As noted, withdrawal from the test-retest component of the study was common. This may be in part due to the length of the LPAQ survey studied in tandem with the EPIQ as 40 women who completed the physical activity survey then failed to complete the EPIQ. Overall, 63 women completed web twice, 57 web-then-paper versions, 47 paper-then-web versions, and 109 completed the paper version twice (Figure 1).

Figure 1.

Flow Diagram of Study Participants

Factor analyses of the web version of EPIQ were based on the 384 women who completed a web-based EPIQ as the first or only administration. Table 1 shows the baseline characteristics of the women who completed the web vs. paper based EPIQ first. Those who completed the second administration of the questionnaires were similar to the larger sample (data not shown). Most women were White (93%), with a mean age of roughly 50 years. Compared to women who completed the paper-based EPIQ first, those who completed the web-based EPIQ first were slightly younger (50 vs. 51 years, p=0.002), with more education (65% vs. 56% with some college, p=0.001) and had lower parity (21% vs. 19% with 0 births, p=0.019.) The average time between survey completion was approximately 2.5 months, and longer for women who took the paper-based EPIQ first early in the study (p=0.036.)

Table 1.

Demographic characteristics of subjects completing Web vs. Paper survey first

	Participants with web survey first: N=384 n (%) or Mean( SD)	Participants with paper survey first: N=492 n (%) or Mean( SD)
Age (38–65)	49.5(6.9)	50.9(6.7)***
<39	1(0.3%)	0(0%)**
39–50	217(56.5%)	241(49.0%)
51–65	166(43.2%)	251(51.0%)
>65	0(0%)	0(0%)
BMI	26.2(4.9)	26.1(5.3)
<18.5	0(0%)	4(0.8%)
18.5–29.9	300(78.3%)	383(78.2%)
30–40	81(21.2%)	95(19.4%)
>40	2(0.5%)	8(1.6%)
Parity (#vaginal+#caesarean)
Ever delivered	298(79.5%)	397(81.2%)
0 births	77(20.5%)	92(18.8%)**
1–2 births	177(47.2%)	194(39.7%)
3+ births	121(32.3%)	203(41.5%)
Ethnicity
Hispanic	12(3.1%)	24(4.9%)
Non-Hispanic	372(96.9%)	466(95.1%)
Unknown/missing/ declined	0	2
Race - multiple races may be checked
African American	2(0.5%)	7(1.4%)
American Indian	2(0.5%)	7(1.4%)
Asian American/Pacific Islander	8(2.1%)	20(4.8%)
White	368(95.8%)	446(91.0%)
Other/Unknown/ missing/declined	2	5
Education
Less than High School	0(0%)	5(1.0%)***
High School	26(6.8%)	71(14.4%)
Some College	119(31.0%)	147(29.9%)
College Graduate	129(33.6%)	134(27.2%)
Graduate/Professional	110(28.7%)	135(27.4%)
Administration interval in days for those in test retest groups (mean(SD), range, n)	55.7(51.2) (14–276) n=120	69.9(57.0)** (14–340) n=160
EPIQ DIAGNOSES-1st survey
POP	10(2.6%)	27(5.5%)**
SUI	44(11.5%)	62(12.6%)
OAB	22(5.7%)	42(8.5%)
AI	64(16.8%)	96(19.5%)

^*percents are out of nonmissing data;

^**p<0.05;

^***p<0.01.

SUI: Stress Urinary incontinence, OAB: Overactive bladder, AI: Anal incontinence, POP: Pelvic organ prolapse

Stability of construct validity in web-based EPIQ

In the present study, factor analyses supported several possible models; however, the seven-factor model was most clinically plausible and closely represented the original EPIQ factors. This model explained more than 70% of the total variance and other than having two urgency incontinence questions loading in the SUI domain, the models closely matched the original validation. Given the exclusion criteria set out for the parent study to minimize presence of bothersome UI, these findings were not unexpected. Table 2 displays factor loadings for the web version of each EPIQ domain along with internal consistency of the clinically relevant subscales: SUI, OAB, AI and POP for the web and paper based versions. Factor analyses for paper version of EPIQ are not shown, but were not significantly different. Internal consistency was high and comparable to the originally reported instrument regardless of method of administration (Table 2).

Table 2.

Factor loadings for first web survey (N=384) and internal consistency for all participants

Factor (domain):*	QOL	POP	SUI/UUI	OAB	AI
How bothered by…
Need to rush?			0.62
Day time frequency?				0.79
Night time frequency?				0.73
Frequent urination?				0.83
Leakage related to urgency?			0.67
Leakage related to Cough/sneeze?			0.87
Small urine leaks (drops)?			0.82
UI affects household chores?	0.82
UI affects recreation?	0.53		0.51
UI affects entertainment?	0.86
UI affects travel?	0.73
UI affects social activities?	0.79
UI affects emotional health?	0.76
UI affects feeling frustrated?	0.57		0.44
Bulge sensation?		0.85
Incontinent of flatus?					0.44
Incontinent of liquid stool?					0.8
Incontinent of solid stool?					0.68
Internal Consistency (Cronbach’s alpha)
	QOL	POP	SUI/UUI	OAB	AI
Original 2005 EPIQ 1 validation study	0.91	**	0.85	0.89	0.58
Web-first group (N=384)	0.89	**	0.81	0.84	0.51
Paper-first group (N=492)	0.91	**	0.84	0.84	0.61

^*Factors 5 & 6 Defacatory Dysfunction & Pain and Difficulty Voiding (data not shown)

^**Prolapse domain has only one item, thus Cronbach’s alpha cannot be calculated.

QOL: Quality of life, POP: Pelvic organ prolapse, SUI/UUI: Stress/Urinary incontinence, OAB: Overactive bladder, AI: Anal incontinence

Descriptive statistics for these subscales with mean (SD) VAS scores are demonstrated in table 3. Examination of the range of scores reported during web-based versus paper-based completion of the EPIQ revealed that the minimum score was 0 in every context, but the maximum score for a subscale tended to be slightly higher when reported on paper than when reported via web. Table 3 also shows that the mean subscale scores were 2–6 mm higher for paper completion as compared to web (p<0.05 by paired t-test for paper-first versus web-first, except AI.) Of note, when administering the web version of the VAS scale, the default score was set to zero and the subject was required to move the bar from that location to a higher degree of bother, possibly explaining these small differences in mean VAS scores. Bland-Altman plots revealed some outliers, in which a woman reported few symptoms on one occasion and many on the other. The POP scale revealed a very low range of responses in some of the reproducibility subgroups except the paper-paper and web-paper groups, where responses spanned the entire 0–100 mm VAS scale. Thus, reproducibility of the web version of the EPIQ could not be determined for POP.

Table 3.

Mean EPIQ Scale Scores by administration timing and method (mm on the VAS scale)

			Intra-method Test Retest				Inter-method Test Retest
			Web Twice		Paper Twice		Web then paper		Paper then web
Domain	Group with web survey first	Group with paper survey first	1st survey	2nd survey	1st survey	2nd survey	Web survey (1st)	Paper survey (2nd)	Paper survey (1st)	Web survey (2nd)
	n=384	n=492	n=63	n=63	n=109	n=109	n=57	n=57	n=47	n=47
POP mean VAS (SD) (range)	2.4(13.1) (0–100)	4.6(17.0) (0–100)	1.9(11.7) (0–88.0)	0.3(1.7) (0–12.0)	4.4(16.8) (0–100)	4.8(17.8) (0–97.0)	2.2(12.1) (0–88.0)	2.5(13.9) (0–100)	0.7(3.5) (0–21.0)	0.1(0.7) (0–5.0)
SUI mean VAS (SD) (range)	14.3(20.8 (0–91.0)	18.8(22.8) (0–96.9)	14.9(21.0) (0–73.8)	14.7(20.8) (0–73.8)	17.9(24.1) (0–91.8)	17.3(23.8) (0–94.5)	13.2(18.9) (0–65.5)	16.5(21.7) (0–84.5)	11.2(15.7) (0–63.0)	7.3(12.7) (0–53.8)
OAB mean VAS (SD) (range)	14.4(20.3) (0–88.4)	20.5(21.5) (0–93.6)	14.4(21.3) (0–72.8)	15.9(23.4) (0–95.0)	17.9(20.9) (0–86.9)	17.7(20.8) (0–90.0)	15.3(18.0) (0–68.8)	17.2(19.6) (0–70.3)	16.3(18.7) (0–68.8)	12.6(17.7) (0–63.8)
AI mean VAS (SD) (range)	9.4(16.5) (0–99.3)	11.3(19.4) (0–100)	10.3(17.0) (0–66.7)	9.1(15.9) (0–66.3)	11.6(19.8) (0–100)	10.8(17.9) (0–97.0)	9.1(14.1) (0–64.3)	9.5(16.0) (0–66.7)	5.7(10.7) (0–33.3)	4.4(8.8) (0–33.3)

VAS: visual analog scale, SD: standard deviation

POP: Pelvic Organ Prolapse; SUI: Stress Urinary Incontinence; OAB: Overactive Bladder; AI: Anal Incontinence

Test-retest and Inter-method reliability

Table 4 demonstrates acceptable intraclass correlations ranging from 0.68 to 0.91. However the reproducibility subgroups included too few bothersome POP cases to be evaluated. Table 4 also shows inter-method reliability above 0.8 for SUI, OAB, POP, and paper-based AI, which was similar to the original EPIQ.

Table 4.

Reliability Characteristics of EPIQ Subscales

	Intermethod Reliability (ICC)		Test-Retest Reliability (ICC)
Subscale	Web vs paper in the WP group N=57	Paper vs web in the PW group N=47	Web twice group N=63	Paper twice group N=109
POP	NR	NR	NR	0.85
SUI	0.79	0.81	0.88	0.89
OAB	0.82	0.90	0.85	0.90
AI	0.82	0.76	0.68	0.85

POP: Pelvic Organ Prolapse; SUI: Stress Urinary Incontinence; OAB: Overactive Bladder; AI: Anal Incontinence; NR: Not reported due to insufficient numbers (<5) for comparison

DISCUSSION

We endorse the use of the web-based EPIQ to identify bothersome PFDs among women not seeking care for PFDs. Overall, compared to paper administration, web administration of the EPIQ has similar psychometric properties with comparable internal consistency and test-retest reliability when administered in the same modality. Reliability between both methods of administration is acceptable for SUI, OAB and AI. Because very few women in the reliability studies had POP (as defined by the EPIQ), reliability of a bothersome POP diagnosis using web administration can’t be certain; however, based on the similar construct validity of the other domains, we would not expect major deviations from the rest of the instrument. Future validation in women with POP should be done to confirm this. Overall, the web-based administration of the EPIQ survey appears to be a valid method of administration in our population of women not seeking care for PFDs.

Lukacz et al. initially developed biologically valid definitions of PFDs and determined criterion validity, construct validity, internal consistency, and test-retest reliability of the paper-based EPIQ survey in a Kaiser population of women aged 25–84, 50% of whom had PFDs with the rest recruited from a general gynecology office.¹ In contrast, our study group was more nearly middle-aged (age 38–65) and 90% were educated beyond high school, compared to 80% in the Kaiser population. Despite the fact that our study population had less variability in terms of the range of PFDs, the stability of constructs (factors) was still comparable to the original EPIQ study. Only the SUI domain included some urge incontinence questions that loaded preferentially onto the SUI domain. We are not surprise by this finding due to the fact that women with urge urinary incontinence were specifically excluded from the parent study. Thus, those women with UI may have had stress predominant mixed UI, forcing the responses onto a “urinary incontinence” domain as opposed to an OAB one. Further, the fact that the test-retest reliability of the 2 questionnaire completions was good (regardless of method) is reassuring. In fact, our reliability estimates are likely underestimates given the relatively long inter-administration interval and the fact that POP and UI are known to wax and wane to some degree over time.^{11, 12}

While we found that web-based EPIQ subscale scores were lower, on average, than paper-based scores, the average variation between the two ranged from 2 to 6 mm. We suspect that this difference relates to the way in which people complete visual analogue scales (VAS). For the web-based version, the cursor for the VAS is set at zero and women had to manually move it to express bother related to a given symptom greater than none. For the paper-based version, women placed a mark near the zero line to indicate no bother and thus the mark could be several millimeters away from zero. With our large sample size, these differences were statistically significant but not likely clinically relevant (though the minimum clinical important differences for EPIQ subscales have not yet been established). Diagnosing conditions using the EPIQ requires VAS bother scores far greater than the differences seen between versions (eg.,>47.3 for SUI, >33 for POP, >22.8 for AI).¹

Strengths of this study include the rigorous psychometric statistical testing of a large number of subjects completing a web based version of previously validated instrument. Limitations of our study include the inability to ascertain criterion validity or discriminate validity due to the nature of the parent study; further, since our population was not seeking care for PFDs, we were able to examine construct validity only by concordance of factor analysis of the web-based EPIQ with the paper-based one and are not able to provide evidence that actual care-seeking behavior varies by web-based EPIQ scores. The study is also limited by the large withdrawal rate between the 1^st and 2^nd questionnaire administration. While there were statistically significant differences in some demographic variables between women that withdrew and women that completed the reproducibility study, these differences (such as a several month difference in age) were not clinically meaningful and are unlikely to have biased our results. As noted previously, we suspect the high withdrawal rate is due to the long physical activity questionnaire that preceded the EPIQ; very few women completed the former but not the EPIQ. Among completers, women who first completed the web-based EPIQ were slightly younger, more educated, and with lower parity than women who first completed a paper-based EPIQ, and their time interval between surveys was shorter. It is unlikely that these differences altered the internal validity of the test-retest and inter-method comparison, because we used an intra-class correlation which penalized for differences between groups.⁸ Therefore, the adequate to high reliabilities found in this study had to overcome any differences among groups and are thus conservative. Women were allowed to choose whether to complete the paper version twice, the web version twice, or each version once. It is likely that more experienced web users chose to do the web version; thus, we do not know whether our results would hold for less experienced web users. However, internet use continues to rise and as of April, 2012, 91%, 77% and 53% of Americans between ages 30–49, 50–64, and ≥ 65 years, respectively, reported using the internet at least occasionally and thus would be able to complete the web version.¹³ Finally, because women over 65 years were excluded from the parent study, we also do not know whether our results hold for older women; given the older age of many women with PFDs, future work confirming the acceptability of web-based instruments in this population is needed.

The EPIQ now joins the Pelvic Floor Distress Inventory (PFDI) and the International Consultation on Incontinence Modular Questionnaire-Lower Urinary Tract Symptoms Quality of Life (ICIQ-LUTSqol), as tools to assess PFDs with proven reproducibility when administered by web and paper.^14,15 In contrast to the PFDI, which was validated in women with PFDs, the EPIQ adds value to community based research targeting women not presenting for care. This study establishes internal consistency, test-retest reliability, and inter-method reliability of the web-based version of the EPIQ survey, among women in primary are and the community, aged 38–65, not seeking care for PFDs.