PAPER VERSUS WEB-BASED ADMINISTRATION OF THE PELVIC FLOOR DISTRESS INVENTORY-20 AND PELVIC FLOOR IMPACT QUESTIONNAIRE-7

Abstract

Introduction and hypothesis

Web-based questionnaires are increasingly employed for clinical research. To investigate whether web-based and paper versions of the Pelvic Floor Distress Inventory-20 (PFDI-20) and Pelvic Floor Impact Questionnaire-7 (PFIQ-7) yield similar results, we compared results obtained with these two modes of administration.

Methods

Women with pelvic floor disorders completed both versions of these questionnaires. Scores between modes of administration were compared using paired t-test and intraclass correlation coefficient (ICC).

Results

Among 52 participants, there were no significant differences in scores or scale scores between the web-based and paper questionnaires. The ICC was 0.91 for the PFDI-20 score and 0.81 for the PFIQ-7 score (p<0.001 for each). The web-based format was preferred by 22 participants (53%), 10 (24%) preferred the paper format, and 9 (21%) had no preference.

Conclusions

The acceptability and score equivalence recommend these web-based questionnaires as an alternative to paper questionnaires for clinical research.

Summary

Web-based administration of the Pelvic Floor Distress Inventory-20 and Pelvic Floor Impact Questionnaire-7 yields similar results to paper-and-pencil administration and is preferred by patients.

INTRODUCTION

In clinical and epidemiological studies of pelvic floor disorders, self-administered questionnaires have been used to identify pelvic floor disorders, to describe the severity of symptoms, and to assess impact on quality of life^1–4. The Pelvic Floor Distress Inventory (PFDI) and the Pelvic Floor Impact Questionnaire (PFIQ) are validated condition specific measures of pelvic floor symptom burden and impact². The PFDI-20 and PFIQ-7 are short forms of these questionnaires⁵.

In any clinical research involving questionnaire data, problems may result from incomplete or illegible questionnaires, as well as costs for data entry and for printing and distributing a paper questionnaire. In addition, the data entry process can introduce error. Electronic questionnaires ^6–10 give the investigator immediate and direct access to data entered by the subject, thus decreasing costs, permitting more rapid analysis of collected data, and, in theory, improving the accuracy of collected data. A web-based approach is appealing because, in addition to the advantages of electronic questionnaires, web-based questionnaire administration also allows data entry from any convenient internet-accessible location and at any time convenient to the respondent. This has obvious applications in multi-center clinical trials and epidemiological studies.

However, a potential disadvantage of electronic questionnaires is difficulty using the computer-based entry system, particularly among elderly or less educated patients. Several quality-of-life measures have exhibited stable psychometric characteristics across mode of administration,^7–11 but this may depend on the nature of the survey instrument. Studies of irritable bowel, reflux dyspepsia, chronic pain, and arthritis have found no difference between mode of administration,^{7, 8} but when the questionnaire addresses sensitive subjects or behaviors, such as sexual activity, mode of administration can significantly affect the nature and rate of response.^12–14 The purpose of this study is to evaluate the equivalence of the PFDI-20 and PFIQ-7 administered using an electronic, web-based format compared to standard pencil and paper administration.

MATERIALS AND METHODS

We obtained IRB approval for this research. We offered enrollment to all adult women scheduled for new-patient evaluations at any clinical site within Johns Hopkins Center for Pelvic Floor Health. When a new-patient appointment was scheduled, the research coordinator called the woman to invite her to participate in this study. The coordinator read a script describing the study and obtained oral consent to participate. Women who were willing to participate completed both versions of the research questionnaire: the paper-and-pencil version and the electronic web-based questionnaire (WBQ).

This was a randomized crossover study. Order of questionnaire completion was randomly assigned (through opaque randomization envelopes). If paper administration was the first mode assigned, the questionnaire was mailed to the participant. If the WBQ was the first mode assigned, the woman was provided with a link to the web site, a unique user name, and a unique password. Women were asked to complete the first questionnaire as soon as possible. The second questionnaire was completed upon arrival at the clinical site. The interval between completion of the first and second questionnaire varied, but was between 2 days and 6 weeks for all participants.

The WBQ and paper questionnaire were identical in content but there were three differences in format. First, subjects were not permitted to skip a question in the WBQ, although subjects could return to view and alter prior responses if desired. Second, the WBQ presented one item at a time. Finally, a branching feature was built in to the WBQ. Regarding the branching feature, women who did not endorse a symptom on the PFDI-20 were not asked to rate the severity of that symptom. In addition, participants who did not endorse any items for an individual scale of the PFDI-20 (e.g., the Urinary Distress Inventory-6) did not complete the corresponding scale of the PFIQ-7 (e.g., the Urinary Impact Questionnaire-7) and were assigned a score of 0 for this PFIQ-7 scale.

After submitting the second questionnaire, each participant completed a preference questionnaire. We surveyed women regarding their preference for mode of administration and we recorded descriptive information about computer and internet use for each subject. Additional demographic information was also obtained for each participant.

Overall summary score and individual scale scores were compared between the two modes of administration using the paired t-test and the intra-class correlation coefficient (ICC). The mean differences in scores (95% confidence interval) between the two modes of administration were also determined. Characteristics of women who preferred the paper and WBQ modes of administration were compared using chi square and fishers exact tests.

RESULTS

We invited 95 women to participate in this study. Of these, 33 (34.7%) declined. Sixty-two women agreed to participate. Order of completion was randomly assigned: 30 were assigned to complete the WBQ first and 32 were assigned to complete the paper questionnaire first. However, 10 of these women cancelled their new-patient clinic visits and did not provide any data for this research.

Of the 52 women who participated in this research, 24 were assigned to complete the WBQ first and 28 were assigned to complete the paper questionnaire first (table I). The completion rates were similar for the two modes of administration: 92% for the WBQ and 90% for the paper questionnaire (p=.94). Among the 43 women who completed both versions of the questionnaire, the mean age was 52±13 years. The majority was White (73%), had a computer at home (95%), and used the Internet (93%).

Table I.

Characteristics of participants, by initial mode of questionnaire administration.

	WBQa (n=24)	Paper (n=28)	P valueb
Age, in years (mean, SD)	52±12	52±14	.86
Race (n, %)
White	17 (74)	21 (75)	.36
Black	6 (26)	5 (18)
Asian	0 (0)	2 (8)
Education (n, %)
College or graduate degree	12 (52)	18 (64)	.37
Computer at homec (n, %)	19 (100)	26 (93)	.50
Computer at workc (n, % of those who work outside the home)	11 (58)	14 (52)	.51
Use the internet c (n, %)	18 (95)	25 (96)	.82
For Email	18 (95)	23 (88)
To pay bills	10 (52)	8 (31)
For shopping	15 (79)	14 (54)
For following investments	8 (42)	5 (19)
For games/entertainment	8 (42)	8 (31)

^aWBQ = web-based questionnaire

^bP value for unpaired t test or Fisher’s exact test

^c19 subjects in the WBQ-first group and 26 subjects in the paper-first group completed the questions about computer use.

Comparing the summary scores and individual scale scores between the two modes of administration (table II), we found no significant differences for the summary scores or any individual scales of the PFDI-20 or PFIQ-7. Post-hoc power analysis revealed that these data provide 80% power to exclude a difference in mean PFDI score of ±10 points. Also, we noted a high degree of correspondence between scores with the two modes of administration. Specifically, the ICC was 0.91 for the PFDI-20 summary score, 0.81 for the PFIQ-7 summary score, and 0.66 to 0.88 for each of the scales (p<.001 for each). No effect from the order of administration was detected.

Table II.

Correlation between web-based and paper versions of the PFDI-20 and PFIQ-7.

Scale or subscalea	Mean (SD) score		Mean differenceb (95% CI)	P Valuec	Interclass correlation coefficient (ICC)
	WBQ	Paper
PFDI-20 summary score	88 (56)	92(57)	−5 (−12 to +3)	.20	.91
UDI	43 (29)	43 (28)	0 (−4 to +4)	.91	.88
POPDI	24 (21)	27 (21)	−3 (−6 to +1)	.14	.87
CRADI	21 (20)	22 (22)	−1 (−5 to +2)	.14	.86
PFIQ-7 summary score	53 (67)	47 (55)	+5 (−6 to +17)	.37	.81
UIQ	26 (29)	29 (33)	−3 (−8 to +2)	.20	.86
POPIQ	18 (28)	13 (24)	+5 (−6 to +12)	.51	.87
CRAIQ	9 (20)	6 (11)	+3 (−1 to 8)	.09	.66

^aPFDI, Pelvic Floor Disorders Distress Inventory; UDI, Urinary Distress Inventory; POPDI, Pelvic Organ Prolapse Distress Inventory; CRADI, Colorectal anal Distress Inventory; PFIQ, Pelvic Floor Disorders Impact Questionnaire; UIQ, Urinary Impact Questionnaire; POPIQ, Pelvic Organ Prolapse Impact Questionnaire; CRAIQ, Colorectal anal Impact Questionnaire

^bMean differences for those who completed both questionnaires.

^cP value from paired t-test

There was no missing data from the WBQ because subjects were not allowed to skip items in this format. For the paper questionnaires, 2.8% of items were left blank or were illegible. Ten of the 47 paper questionnaires (21%) included at least one item that was left blank or was illegible. There was no discernible pattern of items left blank.

Of the 43 women who completed both questionnaires, 10 (22%) found the paper questionnaire easier to complete, 15 (34%) found the WBQ easier to complete, and 18 (42%) found the two versions equally easy. Regarding convenience, 8 (20%) found the paper questionnaire more convenient, 21 (51%) found the WBQ more convenient, and 12 (29%) found the two versions equally convenient. Finally, 22 participants (53%) preferred the web-based format, 10 (24%) preferred the paper format, and 9 (21%) had no preference. Women who preferred the WBQ were significantly younger than those who preferred the paper questionnaire (46±11 vs. 60±15, p<.01). The minority of subjects who did not use the internet (7%) universally preferred the paper format. There were no significant associations between questionnaire preference and demographic characteristics of the participants including race, and highest level of education.

DISCUSSION

Web-based administration of the PFDI-20 and PFIQ-7 was acceptable to the women participating in this research. Moreover, the majority of participants preferred this mode of administration. We suspect that the branching pattern, which shortened the questionnaire for some participants, contributed to the preferences voiced by the women in this study. Other researchers have found that research participants voice a preference for electronic questionnaire administration.^{8–10, 13} The acceptability to research subjects, in conjunction with the high degree of correspondence between scores obtained through WBQ and traditional pencil and paper administration, recommends web-based administration for future research protocols.

In research involving sensitive or embarrassing information, self- administered paper and pencil questionnaires or electronic questionnaires are preferred to direct interviews. Interviews are less sensitive than self-administered questionnaires because subjects are less likely to volunteer potentially-embarrassing information.^{13, 14} Moreover, when comparing modes of self-administration, prior research has shown that electronic questionnaires may be more sensitive than paper questionnaires,^{12, 14} possibly because patients respond to the apparent “legitimacy” of the format¹⁴ or because electronic questionnaires are perceived by research subjects as more anonymous.¹³ In this study, we did not find that the WBQ was more sensitive than the paper-and-pencil questionnaire. Scores and subscale scores were not significantly different between modes of administration. This is reassuring, since researchers can conclude that scores obtained through one mode of administration are equivalent to scores obtained with the other mode.

Electronic questionnaires have numerous benefits to researchers. These include the ability to incorporate complex branching patterns, reduction of errors (including multiple responses to single-option items), and elimination of the need for separate data entry.^11–13 While branching patterns can be incorporated into both paper questionnaires and electronic questionnaires, complex skip patterns or extensive customization may create confusion, leading to unnecessary responses to some items and/or incomplete responses to other items. Also, electronic questionnaires can be designed with automated date and time stamps,¹⁵ allowing accurate representation of when timed questionnaires or diaries are completed. The on-screen format can be designed to present questions individually, minimizing the impact of questionnaire length on subject response. We found that 21% of paper questionnaires had at least one missing or uninterpretable item, with less than 3% of items missing overall. Other researchers have reported 5–10% of missing items on self-administered paper questionnaires¹⁶ and an increasing proportion of missing items with increasing questionnaire length.¹⁷ Because subjects were not allowed to skip items on the WBQ, this format eliminated the problem of missing data for this mode of administration. With an electronic questionnaire, the problem of missing data can be eliminated if the electronic format either does not permit subjects to skip individual items or if skipped items are presented a second time at questionnaire completion.⁸ For research on sensitive topics, participants should be given an option to “decline” any specific item if skipped items are not allowed by the questionnaire format.

We speculated that age, education and computer experience might impact questionnaire results, but we were not able to identify any significant differences in scores or subscale scores. The relatively high level of educational attainment in this population may have limited our power to detect differences. However, other researchers have found similar results. Specifically, age, level of formal education, and familiarity with computers do not seem to significantly influence the accuracy of electronic questionnaire results.^{8, 11, 13}

A limitation of our research is the sample size. This may have limited the power of this study to detect differences in some of the scores considered. Also, the women who agreed to participate in this research may not reflect the population of women typically presenting for clinical care. Nevertheless, they are likely to reflect populations that participate in research and therefore we believe our results are generalizable to subjects enrolled in clinical research. Finally, the interval between completion of the first and second versions of the questionnaire may have been up to 6 weeks. It’s possible that symptoms may have changed over time. However, since the PFDI-20 and PFIQ-7 both specifically address symptoms “over the last 3 months”,⁵ we don’t believe an interval of 6 weeks would have a substantial impact on participants’ responses.

Approximately one-third of potential participants declined to join this study. We speculate that some of the women may have declined because of a lack of familiarity with computers or the internet. Indeed, the enrolled population was highly educated and familiar with computers. These observations suggest that researchers who plan to use a WBQ for data collection should plan an alternative mode of administration to facilitate participation by a broad cross-section of eligible women. Our data suggest that paper and WBQ versions of the PFDI-20 and PFIQ-7 can be used interchangeably in this situation.