Risk perception bias, self-reporting of illness, and the validity of reported results in an epidemiologic study of recreational water associated illnesses

Abstract

Epidemiologic studies of water associated illness often have to rely on self-reported symptoms of the outcome illness(es) under study. Individual participant’s perception of risk, in theory, can affect the validity of self-reported symptoms. The magnitude and effect of possible “risk perception bias” was evaluated as part of a series of randomized trials designed to assess infectious disease transmission via exposure to marine recreational waters with modest sewage contamination. All study subjects were blinded to both their individual indices of exposure and the outcome illnesses under study. Of the five outcome illnesses studied, the effect of “risk perception bias” only effected one: skin ailments. Although analysis of crude rates of skin ailments showed the exposed group (bathers) to be 3.5 times more likely to report skin ailments relative to the non-exposed (non-bathers), when the data was stratified by any perceived health risk of bathing in such waters, this association was shown to be spurious in nature. Bathers having pre-conceived notions of any health risk due to the exposure were 10.63 times more likely to report skin ailments relative to the unexposed (non-bathers) (95% CI 2.36–47.8, P = 0.0002), while bathers without any pre-conceived notion of risk were no more likely to report skin ailments relative to non-bathers (OR = 0.60, 95% CI 0.11–3.24, P = 0.71). Further stratification by exposure grouping showed bathers with pre-conceived notions of excess risk to be 4.78 times more likely to report skin ailments relative to bathers without any notion of excess risk (95% CI 1.04–21.86, P = 0.03), while among non-bathers those with pre-conceived notions of risk were 3.70 times less likely to report skin ailments relative to non-bathers without any pre-conceived notion of risk (95% CI 0.70–19.60, P = 0.10). This study shows that “risk perception bias” can be strong enough to lead to spurious associations in the presence of self-reported symptoms, and should be controlled for in future epidemiologic studies of recreational water associated illnesses and other water associated environmental exposures where the use of self-reported symptoms cannot be avoided.

1. Introduction

Epidemiologic studies of environmental exposures sometimes have to rely on self-reported symptoms of the outcome illness under study. Self-reporting of symptoms usually are necessary where the symptoms of the illness under study are subjective in nature, and where objective medical diagnostic procedures are not possible and/or prove either impractical or too costly. It is a well-known fact that the use of self-reported symptoms in prospective studies can be subject to bias caused by either differing participant perceptions of what constitutes a particular symptom or set of symptoms, failure to report symptoms, or various psychological characteristics of the study participant. Despite the inherent bias, the self-reporting of symptoms continues to be used in epidemiological studies of water associated environmental exposures (e.g., viral water-borne illnesses (Payment et al., 1991, 1994) recreational water-borne illnesses (Cabelli et al., 1982; Fleisher et al., 1993, 1996; Kay et al., 1994 etc.)).

In the environmental health setting where the exposure being studied is known or even vaguely perceived by individual participants, one psychological factor that can affect the validity of self-reported illness is the participant’s perceived perceptions of the risk of exposure. Blinding subjects to the nature of the illness being studied in prospective epidemiological study designs does not necessarily control for such “perception bias”. Health risks due to environmental exposures have received much media attention over the past decade or so. Such media attention can influence risk perception among individual study participants. Therefore, individual perception of risk can be either exposure-specific or nebulous in nature based on a small amount of knowledge, often inadequately or inappropriately reported by the news media. This sort of media exposure can influence individual study participant’s perception of the risk of any environmental exposure in a very general and broad manner. Therefore, blinding of study participants to the outcome illness under study would not necessarily control for any pre-conceived general or specific notions of risk attributable to environmental exposures in general, or if known, the environmental exposure being studied.

In the context of conducting perspective environmental epidemiologic studies, our study both identifies and quantifies this phenomenon, and will show that individual “risk perception bias” among study participants blinded to the outcome illnesses under study can be severe enough to produce spurious associations.

It must be emphasized that the concepts and results reported in this study are not restricted to the environmental exposure they were derived from, but are applicable to any environmental exposure where self-reported illness among study participants becomes integrated into the epidemiologic design of the study. The data used to explore the effect of what we label “risk perception bias” were derived from four previously reported randomized trials of infectious illnesses associated with exposure to marine recreational water contaminated with domestic sewage (Fleisher et al., 1993, 1996; Kay et al., 1994). A Brief description of study methods and results follow.

Four separate trials within England and Wales were conducted during the summers of 1989–1992 at four separate marine bathing locations (study sites A, B, C, D). Exposure to differing amounts of sewage contamination among the study participants exposed to marine recreational waters was measured using standard bacteriologic indices of domestic sewage contamination of marine waters. All study locations passed current European Community ‘Imperative’ criteria governing marine bathing waters (Council of the European Economic Communities, 1976) as well as current USEPA bacteriological criteria governing marine recreational waters (United States Environmental Protection Agency, 1986). The study locations were sufficiently distant from each other in order to identify possible site-specific differences in risk.

The study design called for the randomization of healthy volunteers into exposed (bather) and unexposed (non-bather) groups; the rigid control of the time and location of individual bather exposure; and the extensive monitoring of bacteriologic indices of sewage contamination (i.e., the actual measure of exposure). This study design allowed bacteriological indices of sewage exposure to be assigned to each individual bather within 15 min of exposure and within a maximum of approximately 10 m of the place of exposure; and the use of extensive pre- and post-trial-day structured interviews designed to identify and control for non-water-related risk factors and/or confounders for the outcome illnesses under study. Study participants were blinded to the outcome illnesses under study. Participants that were randomized into the exposed group (bathers) on each of the four trial days were blinded to their specific level of exposure (i.e., their level of exposure to bacteriological indices of sewage exposure and thus their implied level of exposure to domestic sewage).

Five outcome illnesses were studied (gastroenteritis, Acute Febrile Respiratory Illness-ICD 9–461-466; 480-(Benenson, 1990), eye, ear, and skin ailments). Bathers were found to be at excess risk of gastroenteritis, Acute Febrile Respiratory Illness, eye and ear ailments relative to non-bathers at all four study locations. In addition, clear dose–response relationships were demonstrated between bather exposure to increasing levels of bacteriological indices of sewage contamination, and increased risk of gastroenteritis, Acute Febrile Respiratory Illness, and ear infections. Interestingly, skin ailments were associated with the exposure (bathing) at only one of the four study locations (Site D), but not with exposure to increasing indices of sewage contamination at any of the study locations.

These series of trials, however, attracted considerable media attention and, thus, study participants were exposed to both electronic and print media reporting prior to the commencement of each of the trials. In order to assess for possible bias caused by media reporting, a series of ten questions designed to identify possible “perception bias” were dispersed throughout the final follow-up questionnaire. Nine of the 10 questions where “sham” questions with only question 3, answer 3 used to test for possible perception bias. The “sham” questions were included in order to “blind” the participant to the nature of our inquiry. This question was then used to assess any “risk perception bias” on the four outcome illnesses found to be associated with bathing in marine waters contaminated with domestic sewage. Of these illnesses, only skin ailments at Site D showed a strong association with this question used to identify possible “risk perception bias”. We now present the results of our analyses designed to identify and quantify any “perception bias” observed to be operating on the observed association between skin ailments and exposure to recreational waters contaminated with domestic sewage at this particular study location (Site D).

2. Methods

Site-specific incidence rates of skin ailments among 469 bathers and 595 non-bathers were assessed at all four study locations for bathers (exposed) and non-bathers (unexposed). The Pearson χ² statistic was used to assess any bather vs. non-bather differences in the incidence of skin ailments within individual study locations, while the Mantel–Haenszel summary statistic was used to assess statistically significant differences between the incidence of skin ailments among bathers vs. non-bathers at all study sites combined. To assess for risk at high levels of the measure of the exposure (i.e., exposure to high levels of bacteriological indices of sewage contamination), rates of reported skin ailments among non-bathers and bathers at quartiles of indicator organism exposure were assessed for all four study locations combined and for the one site showing bathers to be at higher risk for skin ailments relative to non-bathers (Site D). Chi-square analysis was then used to identify any statistically significant trends among the observed incidence of skin ailments among non-bathers vs. bathers at increasing quartiles of bacteriologic indices of sewage exposure at site D. A trend was considered significant only if the chi-square-for-trend statistic remained statistically significant, with and without inclusion of the non-bather (unexposed) group. This insures differences between the unexposed or reference group, and the lowest exposure grouping among bathers does not unduly influence the ξ²-for-trend statistic (Maclure and Greenland, 1992; Breslow and Day, 1980). Where no trend was identified, the rates of skin ailments reported by bathers at Site D exposed to the highest quartile of sewage exposure were compared to rates among bathers in the lowest quartile of sewage exposure using chi-square analysis.

Relationships between the excess risk of skin ailments reported at site D and the question used to identify possible “risk perception bias” was conducted using univariate chi-square analyses. Where a statistically significant association (P < 0.05) was observed with this questions, simple stratification methods were then used to explore the nature of the association.

3. Results

Table 1 shows the questions that were included in the follow-up instrument. This series of ten questions was primarily designed to assess an individual study participant’s perception of health risks associated with swimming without letting the participant know the exact nature of the possible risks, and thus not influence the participant’s responses in any manner. As such, all but 1 (question 3, answer 3) of the questions listed in Table 1 were designed to draw attention away from the issue of health risks, and are thus irrelevant to assessing possible “risk perception bias”. In addition, to further safeguard against any possible bias introduced by the process of inquiring about the participant’s perception of health risk, the questions listed in Table 1 were randomly inserted among the 76 additional questions that comprised the final follow-up questionnaire.

Table 1.

Questions used to assess the presence of possible perception bias in the reporting of symptoms among study participants

1. Do you consider water-related activities dangerous?

2. Have you heard anything regarding the way beaches are maintained in the UK?

3. Have you heard anything regarding the cleanliness of bathing waters in the UK?

What specific problems have you heard about? (can check more than 1)

Oil spills—Objects floating in water

Chemical pollution—Sewage pollution—N/A—

4. Have you ever refused to go bathing?

5. Have you ever become ill soon after bathing in waters in the UK?

6. Have you ever gone to the beach feeling ill?

7. Did feeling ill on these occasions prevent you from entering the water to go bathing?

8. When at the beach, do you bathe or enter the water?

9. How frequently do you immerse your head while bathing?

10. How often do you get sunburned while at the beach?

Table 2 shows site-specific crude rates of skin ailments among all bathers vs. all non-bathers. Inspection of Table 2 shows no statistically significant differences in reported skin ailments among bathers vs. non-bathers at three of the four study locations (sites A, B, C). Site D, however, showed bathers to have significantly higher rates of skin ailments relative to non-bathers (OR = 3.58, 95% CI 1.44–8.87, P = 0.004). No significant differences in the rates of skin ailments among bathers vs. non-bathers were observed when the data from the four study locations were combined (P = 0.23). Comparisons of the rates of skin ailments among all non-bathers vs. bathers (all study locations combined) at quartiles of sewage exposure showed no statistically significant trends in ailment rates vs. indices of sewage contamination, nor any statistically significant differences among bathers exposed to the highest quartile of indices of sewage contamination vs. those exposed to the lowest quartile (Table 3).

Table 2.

Site-specific rates of skin ailments

Site	Rate/100
	Bathers	N	Non-bathers	N	P
Site A	5.7	106	1.6	122	0.15
Site B	6.5	92	8.7	150	0.55
Site C	13.0	146	15.6	147	0.52
Site D	14.4	125	4.5	156	0.004
All sites combined	10.4	469	7.8	575	0.23a

^aP value from the Mantel–Haenszel summary statistic.

Table 3.

Skin ailments among non-bathers vs. bathers at quartiles of indices of sewage contamination (all sites combined)

Exposure status	N	Rate/100	P (trend)	P (Q1–Q4)a
Non-bathers	575	7.8	0.31	0.97
Bathers 0–14b	101	8.9
Bathers 15–27	122	13.9
Bathers 28–50	109	10.0
Bathers 51–158	137	8.8

^aTests for statistically significant differences among bathers in highest vs. lowest quartile of fecal streptococci exposure.

^bRange of indices of sewage contamination (per 100 ml of sample) comprising each quartile of exposure.

Table 4 shows the results of the analysis of reported skin ailments among non-bathers and bathers at Site D by quartiles of sewage exposure. Table 4 shows a statistically significant trend in reported skin ailments among non-bathers and bathers at quartiles of increasing sewage exposure (P trend = 0.001). However, when the non-bather group was removed from the analysis, there is little evidence of a trend in risk of skin ailments among bathers at increasing quartiles of sewage exposure (P = 0.17). Moreover, rates of reported skin ailments among bathers in the lowest quartile of sewage exposure did not differ significantly from those reported by bathers in the highest quartile of sewage exposure (P = 0.17). Therefore, it would seem that the observed excess risk of skin ailments among bathers at Site D was unrelated to indices of sewage exposure and thus unrelated to sewage pollution.

Table 4.

Skin ailments among non-bathers vs. bathers at quartiles of indices of sewage contamination Site D only

Exposure status	N	Rate/100	P (trend)	P (Q1–Q4)a
Non-bathers	156	4.5	0.001	0.17
Bathers 0–10b	49	8.2
Bathers 11–23	20	20.0
Bathers 24–33	25	16.0
Bathers 34–70	31	19.4
			Bathers only P (trend) = 0.17c

^aTests for statistically significant differences among bathers in highest vs. lowest quartile of fecal streptococci exposure.

^bRange of indices of sewage contamination (per 100 ml of sample) comprising each quartile of exposure.

^cComparison between bather groupings only.

Table 5 is a list of non-bathing-water-related risk factors used as potential confounders of the relationship between skin ailments and bather status observed at Site D. Chi-square analysis of the non-water-related risk factors shown in Table 5, when applied to the data obtained at Site D only, showed no statistically significant differences (P > 0.05) among bathers vs. non-bathers, or among bathers reporting skin ailments vs. bathers who did not, for any of the non-water-related risk factors listed in Table 5.

Table 5.

Non-water-related risk factors for skin ailments

A skin rash, ulcer or sore anywhere on the body persisting for more than 24 h within 3 weeks prior to initial interview

Itching and irritation anywhere on the body persisting for more than 24 h within 3 weeks prior to initial interview

Use of steroids within 4 weeks prior to initial interview

Bathing within 3 days prior to the trial day or bathing within 7 days subsequent to the trial day

Bathing within 3 weeks subsequent to the trial day

Usual frequency of sunburn

Skin ailments in household within 3 weeks subsequent to trial day that preceded illness in volunteer

Analysis of the question asked to assess any potential “risk perception bias” among bathers vs. non-bathers, or among ill vs. non-ill bathers, indicated that at Site D bathers who reported skin ailments were more likely to have heard that bathing in UK waters might present a possible health risk relative to bathers who did not report skin ailments (Table 1, Question 3) (OR = 4.78, 95% CI 1.04–21.86, P = 0.03). Table 6 shows an analysis of the effect of such knowledge when stratified by bather status (bathers or non-bathers). Inspection of Table 6 shows that bathers with general knowledge that bathing might cause health effects were more likely to report skin ailments relative to bathers without such knowledge (OR = 4.78, 95% CI 1.04–21.86, P = 0.03), while non-bathers with such knowledge were less likely to report skin ailments relative to non-bathers without such knowledge (OR = 0.27, 95% CI 0.05–1.44, P = 0.10). Table 7 shows the impact of this source of confounding in a slightly different perspective. Table 7 presents an analysis of reported skin ailments among bathers vs. non-bathers at Site D stratified by general knowledge of a bathing-related health risk. Inspection of Table 7 again shows strong association between such general knowledge and the reporting of skin ailments. Inspection of Table 7 shows that among study participants aware of possible bathing-associated health risk, bathers at Site D were almost eleven times more likely to report symptoms relative to non-bathers (OR = 10.63, 95% CI 2.36–47.81, P = 0.0002), while among study participants reporting no knowledge of health risks, bathers were again less likely to report skin ailments (OR = 0.60, 95% CI 0.11–3.24, P = 0.71).

Table 6.

Skin ailments at Site D stratified by any knowledge that bathing in the united kingdom might present a possible health risk

Knowledge status	N	% Reporting skin ailments	Odds ratio	95% Confidence interval
Bathers
Did not have knowledge	42	4.8	1.00	–
Had knowledgea	83	19.3b	4.78	1.04–21.86
Non-bathers
Did not have knowledge	65	7.7c	1.00	–
Had knowledgea	91	2.2	0.27	0.051–1.44

^aBased on the study participant indicating that they had “heard” that bathing in UK waters might present a “health risk”.

^bP = 0.03.

^cP = 0.10.

Table 7.

Skin ailments at Site D stratified by any knowledge that bathing waters in the united kingdom might present a possible risk to health^a

Exposure status	N	% Reporting skin ailments	Odds ratio	95% Confidence interval
Study participants who indicated they have not heard anything regarding the possibility that bathing in UK waters presents a possible risk to health
Non-bathers	65	7.7b	1.00	–
Bathers	42	4.8	0.60	0.11–3.24
Study participants who indicated they have heard that bathing in UK waters might present a possible risk to health
Non-bathers	91	2.2	1.00	–
Bathers	83	19.3c	10.63	2.36–47.81

^aBased on the study participant indicating that they had “heard” that bathing in UK waters might present a “health risk” (Bathers vs. non-bathers).

^bP = 0.71.

^cP = 0.0002.

4. Discussion

The analyses presented indicate that the observed difference in reported skin ailments among bathers vs. non-bathers observed at the Site D study location is subject to substantial amounts of reporting bias among study participants. The nature of this bias seems to be due to “risk perception bias” acting on bathers who had some form of broad knowledge that the exposure could cause health effects, and thus be more likely to find and report skin ailments. An alternative explanation is that this group of bathers exaggerated their symptoms, and thus reported conditions that others would not categorize as true skin ailments. The observation that non-bathers with knowledge about possible bathing-related health effects reported symptoms of skin ailments less frequently than non-bathers without such knowledge, coupled with the fact that no statistically significant difference between the reported incidence of skin ailments was observed among bathers vs. non-bathers who had no knowledge about possible bathing-associated health risks, provides strong evidence that the apparent association between skin ailments and exposure to marine waters contaminated with domestic sewage initially observed at Site D was a result of participant “perception bias” and thus spurious in nature. The fact that no statistically significant excess incidence of skin ailments was found among bathers at the three other study locations nor when the data from all four study locations were combined further supports the notion of a spurious association resulting from “perception bias” among study participants, and thus was not due to exposure-related effects.

The main purpose of this report is to both identify and quantify the effect of any pre-conceived notions of the risk an exposure can have on individual study participants and thus on the validity of reported results. The “risk perception bias” identified and quantified herein was based on a study of domestic sewage derived recreational water-associated infectious illnesses. However, the phenomenon described is by no means limited to the bathing water exposure in which it was described, but remains relevant to any prospective epidemiological water associated study where the exposure assessed is perceived to have an accompanying health risk (regardless of the validity of this assumption) among individual study participants when illness is self-reported. Therefore, “risk perception bias” can occur even where study participants only have the most rudimentary idea of the actual health effects of a particular environmental exposure. As such, the procedures described in this report should be implemented in epidemiological studies of water associated illnesses where self-reporting of symptoms cannot be avoided. The severity of the bias reported in this study shows that incorporation of methods to identify and quantify possible “risk perception bias” is clearly warranted in such situations.