Social Desirability Bias in Self-Reporting of Hearing Protector Use among Farm Operators

Abstract

Objective:

The purposes of this study were (i) to examine the relationship between reported hearing protector use and social desirability bias, and (ii) to compare results of the Marlowe-Crowne social desirability instrument when administered using two different methods (i.e. online and by telephone).

Methods:

A shortened version of the Marlowe-Crowne social desirability instrument, as well as a self-administered instrument measuring use of hearing protectors, was administered to 497 participants in a study of hearing protector use. The relationship between hearing protector use and social desirability bias was examined using regression analysis. The results of two methods of administration of the Marlowe-Crowne social desirability instrument were compared using t-tests and regression analysis.

Results:

Reliability (using Cronbach’s alpha) for the shortened seven-item scale for this sample was 0.58. There was no evidence of a relationship between reported hearing protector use and social desirability reporting bias, as measured by the shortened Marlowe-Crowne. The difference in results by method of administration (i.e. online, telephone) was very small.

Conclusions:

This is the first published study to measure social desirability bias in reporting of hearing protector use among farmers. Findings of this study do not support the presence of social desirability bias in farmers’ reporting of hearing protector use, lending support for the validity of self-report in hearing protector use in this population.

INTRODUCTION

Self-reported measures are one of the most common sources of information in research studies surrounding health behaviors, including seatbelt use (Ibrahimova et al., 2011; Ozkan et al., 2012), smoking (Gorber et al., 2009), alcohol consumption (Davidson, 1987), physical activity level (Slootmaker et al., 2009; Ainsworth et al., 2012), and nutrition (Johnson, 2012). However, self-reported measures, especially regarding sensitive issues, are often subject to inaccuracy and bias (Kretsch et al., 1999; Gorber et al., 2009; Howard et al., 2009; Krumpal, 2013). Such inaccuracy and bias are evident in various studies that have compared actual observed to self-reported protective behaviors (e.g. Eime et al., 2005). For example, in the study by Eime et al. (2005) about eyewear protection among squash players (n = 1219), participants’ rates of self-reported uses of protective eyewear [9.4%; 95% confidence interval (CI) = 7.8, 11.0] were significantly higher (1.5 times more) than the recordings of trained observers (5.9%; 95% CI = 4.6, 7.2).

Discrepancies among observed and self-reported findings stem from the need for social approval and/or avoidance of disapproval (Horner et al., 2002). The term socially desirable responding has been coined to describe this tendency to report socially desirable responses and favorable self-presentation (Crowne and Marlowe, 1960). Crowne and Marlowe (1960) developed a social desirability instrument to measure under- and/or over-reporting of behaviors in research. This 33-item true/false instrument has an internal consistency reliability (alpha) of 0.88 (Crowne and Marlowe, 1960). Additionally, shortened forms of the Marlowe-Crowne instrument have been developed, retaining much of the reliability, while reducing the number of items. These shortened versions have shown to be effective in measuring social desirability bias in self-reported measures of behaviors (Ray, 1984; Fischer and Fick, 1993; Loo and Thorpe, 2000).

Several research studies have utilized these instruments to show that socially desirable responding has significant impact on study results, thus influencing study validity (Tooze et al., 2004; Adams et al., 2005). For example, Adams et al. (2005) found a high correlation between social desirability and over-reporting of activity. Given the potential influence of social desirability bias on study results, researchers are cautioned to account and control for the impact of social desirability bias on the validity of their research (van de Mortel, 2008).

Self-reported hearing protector use among workers

Self-reported use of hearing protection devices (HPDs) to prevent noise induced hearing loss, like other self-reported behaviors, has potential for errors and biases (Melamed et al., 1996). Studies that explored HPD use among workers have described the discrepancy between self-report and actual use of HPDs (Griffin et al., 2009). Griffin et al. (2009) studied predictors of accuracy of self-reported HPD use associated with variations in noise exposure conditions (i.e. steady, variable, and unpredictable) among 58 workers (including warehouse, sheet metal, and construction workers). Using two measures of HPD use (i.e. daily activity cards and paper questionnaires), HPD use was more accurately reported on the daily activity cards than questionnaires completed on Day 1 at the fixed steady [odds ratio (OR) = 3.9, P = 0.08] and fixed variable sites (OR = 1.4, P = 0.6) and 2 weeks after the study began at the fixed steady (OR = 2.3, P = 0.2) and the fixed variable sites (OR = 0.9, P = 0.9). Workers in the fixed steady noise site were nearly 20 times more likely to report HPD use concordantly with observation than workers at the construction site (OR = 19.42, P = 0.02). Self-reported use of HPDs on questionnaires was higher than activity cards, but more accurately reported on activity cards for workers in all three settings.

In contrast, Lusk et al. (1995) evaluated HPD use among 48 blue collar workers using (i) trained observer report, (ii) supervisor report, and (iii) self-report. Self-report was highly correlated with trained observer report (0.89, P < 0.001; 0.84, P < 0.001; 0.69, P < 0.001). Supervisor report was poorly correlated with trained observer (0.47, P < 0.001; 0.42, P < 0.01; 0.33, P < 0.01) and self-report (0.50, P < 0.001; 0.49, P < 0.001; 0.38, P < 0.001). There were high correlations between the three recall time periods (1 week, 1 month, and 3 months) for both self-report (0.96–0.99) and supervisor report (0.91–0.96).

In a nationally representative study of the prevalence of workplace noise exposure by industry, Workers in agriculture have a high (43.3%) prevalence of exposure to hazardous workplace noise from farm equipment (Tak et al., 2009). Sample noise exposures include tractors (91 dB, Beckett et al., 2000), sheep confinement facilities (87 dB), and swine confinement facilities (90 dB, McBride et al., 2003; Humann et al., 2011). However, noise dose (i.e. time in hazardous noise) studies in agriculture are rare.

A number of investigators have used self-report as a measure of HPD use among farm workers (e.g. Carruth et al., 2007; Gates and Jones, 2007). Gates and Jones (2007) reported that 60 and 28% of their farm worker participants (n = 25) reported to have never used, and seldom used hearing protection, respectively; only one participant responded that they always used hearing protection while farming. In the 2007 study by Carruth et al., despite the high rate of left-sided hearing loss (F = 10.30, P =0.002), fewer than 10 of the 56 farmers reported regularly wearing hearing protection. Use of HPDs was rated lowest in agriculture among all industries, second only to construction (Tak et al., 2009). However, these studies failed to account for potential bias and error in their self-report measures. Among farm workers, little is known about the impact of social desirability on their reported HPD use. The potential for social desirability bias relating to self-report HPD use among farm workers remains unexplored.

The purposes of this study were to (i) examine the relationship between reported hearing protector use and social desirability response, (ii) compare the mean scores and strength of the relationship between the shortened Marlowe-Crowne and HPD use instruments when administered using different methods (online, telephone), and (iii) assess the reliability of the seven-item Marlowe-Crowne scale in this sample.

METHODS

Setting and sample

The study was approved by the investigators’ university institutional review board. The sample included participants in a trial of random assignment to three interventions designed to increase HPD use among noise-exposed, adult resident farmers who reported being active in agricultural production at least 20h per week. Participating farmers were from diverse geographic areas of the USA, representing 24 states.

Variables and instruments

Instruments used in the study included the farmers’ use of hearing protection (McCullagh et al., 2002) and Marlowe-Crowne Social Desirability Scale (short-version; Fischer and Fick, 1993).

Hearing protector use

Use of hearing protection was measured by the farmers’ use of hearing protection instrument, which was developed expressly to be used with farmers. The instrument consists of five items to reflect the percentage of time that individuals reported using hearing protection when they were exposed to high noise in occupational and recreational settings. As detailed elsewhere (McCullagh et al., 2002), development of the farmers’ hearing protection use instrument included pretesting, revisions, and review for content validity by an expert panel. The instrument defined high noise as present whenever an individual had to raise his or her voice to be heard by another person at a distance of three feet or less (Lusk et al., 1995). Hearing protection use was described as using hearing protectors, such as ear plugs or muffs. In prior studies (McCullagh et al., 2002, 2010), farmers reported a wide range of use of hearing protection and experienced no difficulty using the instrument. The instrument was scored as the average percentage of time HPDs were used among settings in which the worker reported being exposed to high noise.

Social desirability response

Social desirability responding was measured using a shortened version of the Marlowe-Crowne social desirability scale proposed by Strahan and Gerbasi (1972), and further shortened after confirmatory factor analysis by Fischer and Fick (1993). Compared with the original 33-item scale, the shortened scale includes seven items, but retained a reliability level (as measured by Cronbach’s alpha) of 0.79 in Fischer and Fick’s test sample, while maintaining high correlation with the original scale. The shortened scale includes true/false items describing ‘culturally approved but improbable behaviors’ (Strahan and Gerbasi, 1972). A sample item from this scale is, ‘I’m always willing to admit it when I make a mistake.’

Data collection procedures

Prospective study participants were contacted through farm organizations and invited to visit a website to enroll. Enrollment procedures (including eligibility screening, administration of informed consent, and administration of baseline data for the longitudinal study) and an intervention were delivered online. At the 6-month follow-up, participants were invited to re-visit the website to report post-intervention HPD use and to respond to a social desirability instrument. Study subjects who failed to complete the 6-month follow-up online were subsequently contacted by telephone, and invited to continue study participation online or by phone. As an incentive for participation, subjects were offered checks totaling $40, awarded in increments for each wave (i.e. baseline, 6-month, and 12-month) of data collection completed.

Data analysis

A cross-sectional design was used to examine relationships between HPD use and socially desirable responding. SPSS (version 22) was used to compute descriptive and correlational statistics.

As measured in this study, HPD use, with a mean of 48.6 and a standard deviation of 31.2, is close to being normally distributed given the reasonable sample size to be analyzed by linear model analyses (e.g. t-tests and linear regressions). The Marlowe-Crowne social desirability instrument was scored by taking the sum of the seven social desirability items. Some of the items were reverse coded before taking the sum so that a high score indicates high social desirability bias.

RESULTS

A total of 497 participants responded to the questionnaire; 445 reported their use or non-use of HPDs. Of these, 394 participated online, and 51 responded by telephone. Non-users of hearing protection did not differ from those who did report their use of HPDs on age (46.3 versus 45.3) [t (495) = 0.47, P = 0.64, 95% CI (−3.30, 5.36)], size of the farm (Fisher’s exact test P = 0.17), gender [χ² (1) = 2.01, P = 0.16] or race (Fisher’s exact test P = 0.076). A summary of the demographics of the study sample appears in Table 1.

Table 1.

Descriptive statistics of the demographics of the respondents at baseline (N = 497).

Characteristic	Mean	SD	N	%
Age at baseline (years)	46.22	15.01	497
Male gender			383	77.06
Race Caucasian/White			487	97.99
Ethnicity			497
Hispanic			6	1.21
Non-Hispanic			491	98.79
Years farming	25.90	15.59	495
Farm role			497
Operator			355	71.43
Paid employee			57	11.47
Non-paid farm worker			85	17.10

SD, standard deviation.

Relationship between reported hearing protector use and social desirability response

Results of a linear regression analysis predicting reported use of HPDs from social desirability score indicated that social desirability accounted for an R-square of 0.042 and an adjusted R-square of −0.001 [t (443) =0.88, P = 0.38; 95% CI (−0.94, 2.44); Table 2]. Thus, the magnitude of the association is too small to detect even with a sample size over 400. Thus, there is no evidence of a relationship between reported hearing protector use and results of the Marlowe-Crowne social desirability instrument.

Table 2.

Results from linear regression analysis to test the model effects.

Source	Parameter estimates
	Parameter estimate	SE	t Value	df	P
(Intercept)	55.01	8.81	6.25	441	<0.001
Method	−10.01	9.41	−1.06	441	0.29
Social desirability scale	−2.01	2.60	−0.77	441	0.44
Method × social desirability	3.11	2.76	1.13	441	0.26

Dependent variable: self-reported percentage of hearing protector use was treated as continuous measure. Model includes method, social desirability scale, and method × social desirability scale. SE, standard error of the parameter estimate; df, degrees of freedom.

Results of the instruments administered using different methods

Results of the t-test comparing two methods online and telephone for HPD use showed no difference [t (443) = −0.13, P = 0.90; 95% CI (−9.75, 8.54); Table 3]. There is also no difference in the participant’s social desirability score among online and telephone administration methods [t (454) = 0.87, P = 0.38; 95% CI (−0.28, 0.72); Table 3].

Table 3.

Comparison of mean hearing protection device use and social desirability score by administered method using t-test.

	Method	N	Mean	SE	t	df	P
Hearing protector use	Internet	394	48.51	1.59	−0.13	443	0.90
	Telephone	51	49.11	3.97
Social desirability score	Internet	405	3.16	0.09	0.87	454	0.38
	Telephone	51	2.94	0.24

SE, standard error of mean.

Table 4 summarizes the results of linear regression analysis using the dependent measure (use of HPDs) and predictors (social desirability score, method, and the interaction of social desirability score and method). The interaction effect between method and social desirability score is non-significant [t (441) = 1.13, P = 0.26; 95% CI (−2.30, 8.52)]. A reduced model with no interaction term was fitted (Model 2) and results are given in Table 4. Both main effects namely method and social desirability score are non-significant. Therefore, we conclude that there is no linear relationship present between use of HPDs and participant’s social desirability score [t (441) =0.88, P = 0.38; 95% CI (−0.93, 2.45); Table 4]. There is also no difference in the use of HPDs among two administered methods of assessment (telephone versus online) in this study [t (441) = −0.17, P = 0.87; 95% CI (−9.92, 8.35); Table 4].

Table 4.

Results from linear regression analysis to test the model effects.

Source	Parameter estimates
	Parameter	SE estimate	t value	df	P
Model 1
(Intercept)	46.20	3.09	14.95	443	<0.001
Social desirability scale	0.75	0.86	0.88	443	0.38
Model 2
(Intercept)	46.87	5.06	9.26	441	<0.001
Method	0.79	4.66	0.17	441	0.87
Social desirability scale	0.76	0.86	0.88	441	0.38

Reliability of the seven-item Marlowe-Crowne scale

Cronbach’s alpha for the seven-item scale for this sample was 0.58 (95% CI = 0.55, 0.61). Examination of alpha-if-item-deleted statistics showed that elimination of any item would not improve Cronbach’s alpha.

DISCUSSION

What have we learned?

Social desirability scores for farmers reporting HPD use were not different from scores for farmers reporting no HPD use. This finding suggests that there was no social desirability bias in reporting HPD use in this sample. This result supports the validity of self-report of hearing protector use in this sample of farm operators.

Although self-report of HPD use and other health behaviors is subject to social desirability bias, and studies using self-reported health behaviors are often criticized for use of self-report, this finding supports use of self-report of HPD use in this sample.

Comparison of instrument results when administered using different methods

The study compared data collection using alternative methods, online, and telephone. The anonymity of online reporting would be expected to result in lower social desirability bias in reporting hearing protector use when compared to reporting to a data collector by telephone. However, the results of the study reported here showed no difference in mean scores of hearing protector use between online and telephone administration. These results are consistent with the finding that there was no detectable social desirability bias reporting of HPD use in the study reported here. Based on these findings, data collection using online or telephone is appropriate for hearing protector use.

Reliability of the seven-item Marlowe-Crowne scale

Reliability of the shortened (seven-item) Marlowe-Crowne social desirability scale was lower (0.58) in the sample reported here than the 0.79 reported by Strahan and Gerbasi (1972). This lower Cronbach’s alpha may be explained by differences in samples, and reflects a need to explore alternative versions of a social desirability bias reporting instrument for use among farmers.

Comparison to previous findings

Unlike some previous studies (Tooze et al., 2004; Adams et al., 2005) that showed social desirability bias in reporting of health behaviors (e.g. activity, anxiety, depression), no social desirability bias was found in the sample reported here. This might be explained by the lack of regulation in the industry and lack of supervision of workers’ use of HPDs in the farm environment.

Study limitations

A limitation of this study is the homogeneity of the study participants. Thus, the findings may not be generalizable to other populations, such as minority groups, and those of different educational backgrounds. Furthermore, although the internal consistency reliability (alpha) of the scale was 0.78 in a sample by Crowne and Marlowe (1960), the alpha coefficient for the current sample was considerably lower, at 0.58. Although this level of alpha (0.58) may be considered poor by classical standards, the value of alpha in psychometrics has been challenged by some contemporary statisticians (Schmitt, 1996). Further research into the reliability of the shortened Marlowe-Crowne instrument is indicated.

The comparison of mean scores of telephone respondents and online respondents may be subject to selection bias. This is due to the fact that telephone respondents were those who did not respond to requests to complete the questionnaire online. Consequently, members of the two groups might differ in their HDP use or social desirability bias, rather than any differences being due to the mode of administration.

Comparison of mean scores for telephone versus online administration are subject to selection bias. The two groups (telephone and online respondents) are not comparable in that the telephone subjects did not respond to the online survey in a timely manner, so these respondents might differ in their actual HDP use or their actual social desirability bias, rather than any differences being due to the mode of administration.

Farmers are in a unique position of serving as their own safety specialists, determining when the noise level is high enough to justify use of hearing protection. The measure of noise exposure (i.e. have to raise your voice to be heard by someone three feet away) used in this study is subjective, but not dependent on the person reporting the noise exposure, as it measures the response of the receiver of the message. Consequently, the measure is not affected by the hearing ability of the respondent, and existing hearing loss of the respondent is not expected to affect their exposure to high noise. Indeed, the prevalence of self-reported hearing loss among study respondents was high, with only 69% of respondents reporting that their hearing was good or excellent.

We measured the influence of local (e.g. individual farm operation) culture on HPD use through the farmer’s norms of HPD use instrument. Results of these and other measures influencing HPD use are reported separately (McCullagh et al., 2002, 2010).

New questions, questions not answered, and future research

There is an intrinsic potential for inaccuracy and bias of in the use of self-report in measuring health behavior; measurement of social desirability bias in studies using self-reporting methods would provide some context for assessing the validity of the study’s findings. Furthermore, assessment of social desirability bias across many studies of health behaviors would help to provide insight into patterns of social desirability bias. An understanding of social desirability bias in self-reporting health behavior would guide researchers in designing future studies to reduce threats to internal validity.

CONCLUSIONS

Self-report is a commonly used method of measurement of use of hearing protection, as well as other health behaviors, and is frequently criticized for its vulnerability to social desirability bias. This is the first published study to measure social desirability bias in self-report among farmers regarding their hearing protector use. Self-reports of hearing protector use among farmers reflect their actual use of hearing protection. Findings of this study do not support the presence of social desirability bias in farmers’ reporting of hearing protector use, lending support for the validity of self-report in hearing protector use in this population.

FUNDING

Financial support for this publication was made possible by NIH/NIDCD R01DC010827 and training grant T42 OH008455 from the National Institute for Occupational Safety and Health, Centers for Disease Control and Prevention.