The behavioral immune system and vaccination intentions during the coronavirus pandemic

Abstract

The behavioral immune system is considered to be a psychological adaptation that decreases the risk of infection. Research suggests that, in the current environment, this system can produce attitudes with negative health consequences, such as increased vaccine hesitancy. In three studies, we investigated whether two facets of the behavioral immune system—germ aversion (i.e., aversion to potential pathogen transmission) and perceived infectability (i.e., perceived susceptibility to disease)—predicted intentions to accept COVID-19 and influenza vaccination during the pandemic. The behavioral immune system mechanisms were measured before the COVID-19 pandemic in one study, and during the pandemic in two. In contrast to previous research, those with higher germ aversion during the pandemic perceived vaccines to be safer and had higher intentions to accept vaccination. Germ aversion before the pandemic was not associated with vaccination intentions. Individuals who perceived themselves as more susceptible to disease were slightly more willing to accept vaccination. We conjecture that high disease threat reverses the relationship between the behavioral immune system response and vaccination. As the associations were weak, individual differences in germ aversion and perceived infectability are of little practical relevance for vaccine uptake.

1. Introduction

Less than one year from the pandemic outbreak, safe and effective vaccines against the SARS-CoV-2 virus started to become available. Although some of these vaccines have efficacies in excess of 90% (Baden et al., 2021; Polack et al., 2020), successful suppression of the pandemic ultimately depends on the public's acceptance of the vaccines. Research surveying acceptance rates before COVID-19 vaccinations began indicated that 5–26% of individuals in European countries intended to refuse vaccination, and an additional 12–28% were unsure about their vaccination decision (Detoc et al., 2020; Freeman et al., 2020; Neumann-Böhme et al., 2020; The COCONEL Group, 2020; Ward, Alleaume, Peretti-Watel, & the COCONEL Group, 2020). In the US and Canada, 14–25% indicated that they would refuse a COVID-19 vaccination (Reiter, Pennell, & Katz, 2020; Taylor et al., 2020), whereas 7% of individuals in Australia reported that they would definitely or probably not accept a COVID-19 vaccine (Faasse & Newby, 2020).

Deciding whether to take a vaccine is a complex psychological process driven by several factors. Research has demonstrated that attitudes towards the vaccine-preventable disease, as well as beliefs regarding the safety and effectiveness of the vaccines, play a key role in explaining vaccine hesitancy (Betsch et al., 2018; Brewer, Chapman, Rothman, Leask, & Kempe, 2017; Larson, Jarrett, Eckersberger, Smith, & Paterson, 2014; Thomson, Robinson, & Vallée-Tourangeau, 2016). This means that understanding the psychology behind these driving attitudes is of considerable importance for ensuring sufficient uptake of COVID-19 vaccines. In the present study, we investigated the degree to which dispositional differences related to psychological disease-avoidance mechanisms—the behavioral immune system—explain people's intentions to accept a COVID-19 vaccine as well as their concerns regarding its safety. The behavioral immune system includes cognitive, emotional, and behavioral mechanisms that during our evolutionary history have decreased the risk of infection.

1.1. Mechanisms of the behavioral immune system

The detrimental effects of infectious diseases on survival are assumed to have resulted in the selection of psychological mechanisms, across evolutionary time scales, that decreased the risk of pathogenic contamination. These psychological adaptations include cognitive, emotional, and behavioral mechanisms that have been labeled the behavioral immune system (Schaller, 2006; Schaller & Park, 2011). The behavioral immune system evolved to identify potential sources of pathogens in the environment and to motivate behaviors that decrease the risk of pathogen exposure. An emotion that is central to the behavioral immune system is disgust (Curtis, Barra, & Aunger, 2011; Oaten, Stevenson, & Case, 2009; Schaller & Park, 2011; Tybur, Lieberman, Kurzban, & DeScioli, 2013). Disgust motivates avoidance of sources identified as potentially containing pathogens. As most pathogens cannot be directly observed, the behavioral immune system responds to environmental cues that in our past correlated with the presence of pathogens. Examples of such cues include spoiled food, the smell of biological decay, bodily secretions (e.g., saliva, nasal secretion, mucus), and wounds (Oaten et al., 2009). Because pathogen cues are not perfectly correlated with the actual presence of pathogens, the system might erroneously activate when pathogens are not present or fail to activate when pathogens are present. As failing to react to existing pathogens tends to have more detrimental consequences than reacting in their absence, the system evolved to be hypersensitive (Ackerman, Hill, & Murray, 2018; Schaller & Park, 2011). The behavioral immune system therefore occasionally responds to cues that pose no actual infection threat. This hypersensitivity leads us to avoid healthy and nutritious food sources that smell like spoiled food (e.g., fermented fish, aged cheese) or avoid social contact with individuals who have deformities that are not contagious.

Avoiding potential sources of pathogens comes with both benefits (e.g., successful avoidance of contaminated food) and costs (e.g., missed opportunity to get nutrition). The cost-benefit ratio is a trade-off that the individual must balance. The optimal ratio depends on how vulnerable the individual is to infection, and, therefore, the behavioral immune system is assumed to be flexible. This means that when individuals perceive the situation to have high contamination risk, and perceive themselves to be more vulnerable, they are expected to be more attentive to pathogen cues and react more strongly to them than when risk and vulnerability is judged to be low (Ackerman et al., 2018; Schaller & Park, 2011).

1.2. The behavioral immune system and vaccination

The behavioral immune system has been found to be related to a wide range of attitudes and behaviors humans display in the modern environment. Unsurprisingly, the behavioral immune system has been shown to be involved in attitudes and behaviors that promote health (Gruijters, Tybur, Ruiter, & Massar, 2016; Murray & Schaller, 2016; Schaller, Murray, & Bangerter, 2015). However, the behavioral immune response may also have negative health consequences. For example, as many infectious diseases are transmitted through interpersonal contact, considering oneself vulnerable to infection may lead to reduced sociability (Murray & Schaller, 2016; Schaller et al., 2015). At the same time, social isolation and loneliness is associated with long-term, negative health outcomes (Hawkley & Cacioppo, 2010; Hawkley & Capitanio, 2015; Jaremka et al., 2013).

Another area where the behavioral immune response might have negative health consequences is in making decisions about vaccination. Because the purpose of vaccination is to protect against infectious diseases, individuals with higher propensity to experience disgust towards potential sources of pathogens (hereafter, pathogen disgust sensitivity) and who react more negatively to potential pathogen transmission (hereafter, germ aversion) could be expected to have more positive attitudes towards vaccination. Seemingly paradoxically, several studies show that individuals with higher pathogen disgust sensitivity and more germ aversion hold more negative vaccine attitudes (Clay, 2017; Clifford & Wendell, 2016; Luz, Brown, & Struchiner, 2019; Reuben, Aitken, Freedman, & Einstein, 2020). Luz et al. (2019) further investigated whether pathogen disgust sensitivity and germ aversion were related to past uptake of the influenza vaccine. They found a relationship between lower likelihood of having accepted the vaccine and higher pathogen disgust sensitivity and germ aversion that was mediated by more negative vaccine attitudes. In addition, there was a positive direct relationship between pathogen disgust sensitivity and vaccine uptake, indicating that those with higher propensity to experience pathogen disgust were more likely to have been vaccinated, when controlling for vaccine attitudes. This could indicate that pathogen disgust sensitivity and germ aversion negatively affect vaccine uptake only to the degree that it affects vaccine attitudes.

A proposed explanation for the surprising result that those with high pathogen disgust sensitivity and germ aversion have more negative vaccine attitudes is that vaccines are administered in ways that in and by themselves are cues to contamination, such as puncturing the skin, and inhalation or ingestion of a foreign substance (Clay, 2017). For this explanation to be supported, individuals who are more averse to pathogens and contamination should react to vaccination as a contamination cue. This reaction would, in turn, lead to more negative attitudes towards vaccines. This notion derives support from the finding that individuals with higher disgust towards needles and blood hold more negative attitudes to the safety and efficacy of vaccinations (Hornsey et al., 2018a). Individuals with injection fears have also been found less willing to accept vaccination against COVID-19 (Freeman et al., 2021). Furthermore, vaccination opponents commonly describe vaccines as ‘unnatural’, containing toxic substances, and causing illness (Kata, 2010; Moran, Lucas, Everhart, Morgan, & Prickett, 2016). Concerns about vaccine safety has been found to be among the most common reasons for unwillingness to accept COVID-19 vaccination (Karlsson et al., 2021; Neumann-Böhme et al., 2020; Taylor et al., 2020). In this manner, higher germ aversion could, ironically, increase reluctance to take a vaccine against COVID-19.

The theory of the behavioral immune system also suggests that perceiving oneself as particularly susceptible to infectious diseases (hereafter, perceived infectability) should further promote behaviors that protect against disease. Research on the association between perceived infectability and vaccination has produced somewhat conflicting results. Although some studies show that perceived infectability is unrelated to vaccine attitudes (Clay, 2017; Luz et al., 2019), the likelihood of accepting the influenza vaccine has been shown to be higher in individuals with higher perceived infectability (Luz et al., 2019). Taken together, this indicates that perceiving oneself at a higher risk of catching infections might motivate people to get vaccinated regardless of their vaccine attitudes.

1.3. The behavioral immune system and the COVID-19 pandemic

As mentioned above, the theory of the behavioral immune system stipulates that situational factors, such as the degree of disease threat, should affect the aversive response (Ackerman et al., 2018; Oaten et al., 2009). The hypothesized effect of increased disease threat on behavioral immune system activation has, however, been investigated only in a few studies. A study in an Australian sample of university students surveyed during the lockdown in spring 2020 found higher disgust sensitivity and germ aversion than in comparable samples collected before the COVID-19 pandemic (Stevenson, Saluja, & Case, 2021). There was no difference in perceived infectability between the pre- and mid-pandemic samples. Studies also indicate that individuals with higher pathogen disgust sensitivity and germ aversion react more strongly to the pandemic threat, as they consider government measures more necessary, find it more important to engage in preventive behaviors such as handwashing and social distancing, and report higher compliance with recommendations (De Coninck, D'Haenens, & Matthijs, 2020; Díaz & Cova, 2020; Makhanova & Shepherd, 2020; Shook, Sevi, Lee, Oosterhoff, & Fitzgerald, 2020). In contrast to vaccination, these non-pharmaceutical interventions and behaviors include no contamination cues.

Previous research on the behavioral immune system and vaccination has been conducted before the pandemic, during lower disease threat. We know of no previous studies that have investigated if people's willingness to take the COVID-19 vaccine—a higher disease-threat situation—is related to the behavioral immune system. If there is a relationship between germ aversion, perceived infectability, and COVID-19 vaccination intentions, this information can be used when developing efficient health-communication strategies.

1.4. The current study

We examined the relationship between the behavioral immune system and intentions to accept vaccination against COVID-19 and perceptions about the safety of a COVID-19 vaccine. More specifically, we investigated whether germ aversion and perceived infectability predicted vaccination intentions and vaccine safety perceptions in three studies in Finland. Based on the previous literature, we predicted that 1) individuals with more (vs. less) germ aversion would have lower vaccination intentions, that 2) individuals with more (vs. less) germ aversion would perceive vaccines as less safe, and that 3) individuals with high (vs. low) perceived infectability would have higher vaccination intentions. Previous research on the topic has investigated general vaccine attitudes and attitudes towards childhood vaccines (Clay, 2017; Clifford & Wendell, 2016; Hornsey et al., 2018b; Luz et al., 2019; Reuben et al., 2020) and past influenza vaccinations (Luz et al., 2019). To facilitate comparison with previous research, we also measured intentions to get vaccinated against influenza. In Finland, the influenza vaccines are administered free of charge to risk groups (e.g., older individuals and young children) and health professionals (Finnish Institute for Health and Welfare, 2021b). Some employers also offer free influenza vaccinations to their employees. In other cases, individuals are required to cover the costs themselves.

Study 1 was longitudinal and conducted in a sample of parents of young children. We measured the parents' germ aversion and perceived infectability before the pandemic outbreak, in 2018, and their vaccination intentions after the first peak of the coronavirus pandemic in Finland in 2020 (between May 4th and June 7th). By this time, the weekly incidence of COVID-19 cases per 100,000 inhabitants decreased from ~11 (calendar week 19) to ~3 (calendar week 23). Studies 2 and 3 were cross-sectional and carried out during the peak of the first pandemic wave in Finland (Study 2 between March 30th and April 12th, 2020, and Study 3 between April 3rd and 17th, 2020). During this time, the weekly incidence of COVID-19 cases was ~16 per 100,000 inhabitants. In Study 2, we surveyed people living in an area with suboptimal uptake of vaccines included in the national vaccination program, and, in Study 3, we surveyed a national sample of Facebook users.

To receive an estimate of the overall size of the associations between the behavioral immune system mechanisms and vaccination intentions, we combined the effect sizes obtained using meta-analysis.

2. Study 1: method

2.1. Respondents and procedure

We sourced respondents from the FinnBrain Birth Cohort Study (L. Karlsson et al., 2018). The FinnBrain study is a longitudinal project that investigates child development by collecting information from children and their parents in the Turku region and on the Åland islands. In May 2018, we invited all parents with at least one child younger than 4.5 years (N = 3401) to an online survey to investigate vaccination attitudes and behaviors. The survey included measures of behavioral immune system mechanisms. A total of 761 (22.4%) responded to the survey. In May 2020, all parents participating in the FinnBrain study (N = 5103) were invited to a survey concerning the coronavirus pandemic, which included measures of vaccination intentions. The survey was open for five weeks, and 856 (16.8%) parents responded. In the present study, we included the 294 parents who responded at both time points. The mean age of the sample was 38.09 years (SD = 4.76, range = 24–55; see Table 1 for descriptive statistics). We conducted comparisons between those individuals who responded to both timepoints and those who responded only in 2018 with regard to demographic variables, germ aversion, and perceived infectability. The results from these analyses are reported in the online supplementary material.

Table 1.

Description of the three included samples.

Variable	Study 1 (N = 294)		Study 2 (N = 200)		Study 3 (N = 1324)
	n	%	n	%	n	%
Agea
18–29	8	2.72	34	17.09	101	9.88
30–39	183	62.24	61	30.65	185	18.10
40–49	100	34.01	44	22.11	255	24.95
50–59	3	1.02	29	14.57	274	26.81
60–69	0	0.00	31	15.58	166	16.24
70+	0	0.00	0	0.00	41	4.01
Sexb
Female	233	79.25	136	68.00	1053	79.59
Male	61	20.75	63	31.50	253	19.12
Other	–	–	–	–	7	0.53
Do not want to report	–	–	1	0.50	10	0.76
Languageb
Finnish	245	83.33	34	17.00	1181	89.27
Swedish	49	16.67	166	83.00	142	10.73

Note. In Studies 1 and 3, we had information about the respondents' exact age in 2020. In Study 2, the respondents indicated the age range they belonged to in the 2019 data collection. A dash (–) indicates that the response alternative was not available.

^aInformation about one (0.5%) respondent missing in Study 2, and 302 (22.8%) in Study 3.

^bInformation about one (0.01%) respondent missing in Study 3.

Ethical approval for both data collections was obtained from the Ethics Committee of the Hospital District of Southwest Finland. The respondents gave their informed consent in the electronic survey. They did not receive any compensation for participation.

2.2. Measures

We administered the surveys in either Finnish or Swedish (the two official main languages in Finland) depending on the preference of the respondent.

2.2.1. Germ aversion and perceived infectability

In the 2018 data collection, we administered the Perceived Vulnerability to Disease (PVD) scale (Duncan, Schaller, & Park, 2009). The scale consists of 15-items divided into two subscales. One subscale (Germ aversion; 8 items) measures aversive response in relation to potential pathogen transmission (e.g., “I prefer to wash my hands pretty soon after shaking someone's hand”). The other subscale (Perceived infectability; 7 items) measures perceived susceptibility to infectious diseases in general (e.g., “I am more likely than the people around me to catch an infectious disease”). Item 15 (“I avoid using public telephones because of the risk that I may catch something from the previous user”), belonging to the subscale Germ aversion, was not administered due to the absence of public telephones in Finland. Respondents indicated how much they agreed with each statement on a scale from 1 (strongly disagree) to 7 (strongly agree). The scale was translated from English to Swedish by the authors of the present study (LCK, AS, JA). A professional translator then translated the Swedish version of the measure to Finnish. Translations were checked and discussed between authors.

2.2.2. Vaccination intentions

In the data collection in 2020, we measured intentions to accept a vaccine against COVID-19 with the question “How likely do you consider it to be that you would take a vaccine against COVID-19, if such a vaccine was available, free of charge, and recommended to everyone by the authorities?”. We also measured intentions to accept the influenza vaccine for oneself with the question “How likely do you consider it to be that you will take the influenza vaccine next season (season 2020–2021)?”. The respondents answered on a scale from 1 (very unlikely) to 5 (very likely).

2.3. Statistical analyses

We conducted a structural regression (SR) with the package lavaan (Rosseel, 2012) in R version 3.5.1 (R Core Team, 2018a, R Core Team, 2018b) to investigate whether the PVD subscales measured in 2018 predicted vaccination intentions in 2020. The reliability of the two PVD subscales were Cronbach's α = 0.68, 95% CI [0.61, 0.73] and Cronbach's α = 0.86, 95% CI [0.82, 0.89] for germ aversion and perceived infectability, respectively. To account for measurement error, germ aversion and perceived infectability were represented by latent factors (see Appendix Table A1 for the measurement model) in the analyses. The factors were specified to predict the two vaccination intention variables. We applied WLSMV estimation (factor indicators were specified as ordinal) with delta parameterization. Missing data was deleted pairwise. We also conducted control analyses including gender and age as covariates.

3. Study 1: results and discussion

Information on the parents' vaccination intentions are presented in Table 2 (vaccination intentions of the total 2020 sample are reported in the study by LC Karlsson et al., 2021, that investigated the relationship between risk perceptions and vaccination intentions using the same data-collections as presented here). Information on the parents' responses to the PVD scale is presented in Table A2. Zero-order correlations are shown in Table 3 .

Table 2.

Responses to vaccination-intention questions in the three current studies.

Variable	Study 1		Study 2		Study 3
	n	%	n	%	n	%
COVID-19 vaccine: test-phase
1 Very unlikely	–	–	66	34.02	407	30.97
2	–	–	37	19.07	277	17.28
3	–	–	57	29.38	282	21.46
4	–	–	25	12.89	249	18.95
5 Very likely	–	–	9	4.64	149	11.34
COVID-19 vaccine: recommended
1 Very unlikely	14	4.76	8	4.12	159	12.06
2	26	8.84	15	7.73	55	4.17
3	41	13.95	20	10.31	143	10.85
4	75	25.51	49	25.26	288	21.85
5 Very likely	138	46.94	102	52.58	673	51.06
Influenza vaccine: 5-point scale
1 Very unlikely	37	12.59	–	–	291	22.59
2	31	10.54	–	–	92	7.14
3	31	10.54	–	–	135	10.48
4	45	15.31	–	–	116	9.01
5 Very likely	150	51.02	–	–	654	50.78
Influenza vaccine: 3-point scale
No intention to vaccinate	–	–	80	40.00	–	–
Undecided	–	–	63	31.50	–	–
Intend to vaccinate	–	–	57	28.50	–	–

Note. Study 1: labels for the response alternatives 2 (not that likely), 3 (hard to say), and 4 (quite likely) were provided. Study 2: 6 (3.0%) responses missing concerning the COVID-19 vaccines. Study 3: 10 (0.8%) responses missing concerning the test-phase COVID-19 vaccine, 6 (0.5%) concerning the recommended COVID-19 vaccine, and 36 (2.7%) concerning the influenza vaccine.

Table 3.

Zero-order correlations between the measures included in Study 1.

Variable (year of data collection)	1	2	3	4
1. Germ aversion (2018)	–
2. Perceived infectability (2018)	0.38	–
3. COVID-19 vaccination intentions (2020)	−0.03	0.05	–
4. Influenza vaccination intentions (2020)	−0.01	0.15	0.57	–

Note. Bolded correlation coefficients p < 0.05.

The fit of the SR model was good, χ ²(97) = 137.78, CFI = 0.990, TLI  =  0.988, RMSEA =  0.038; 90% CI [0.022, 0.052], SRMR = 0.059. Neither germ aversion, nor perceived infectability significantly predicted intentions to accept a COVID-19 vaccine (Fig. 1 ; see also, Fig. 5 in the Meta-analysis section). Concerning the influenza vaccine, perceived infectability was a statistically significant predictor of vaccination intentions. Respondents who consider themselves more vulnerable to infections were more likely to intend to take the influenza vaccine the next season. This relationship was, however, weak. Germ aversion did not predict influenza vaccination intentions. Including gender and age as covariates resulted in the same conclusions (see, Table S1 in the online supplementary material).

Fig. 1.

Germ aversion and perceived infectability (measured in 2018) as predictors of vaccination intentions (measured in 2020) in Study 1. Factor indicators and covariance, as well as disturbance covariances between outcomes, are not shown in the figure. *p < 0.05, **p < 0.01, ***p < 0.001.

Fig. 5.

Standardized beta coefficients for the relationships between the PVD measures (germ aversion and perceived infectability) and intentions to accept a recommended COVID-19 vaccine and influenza vaccines in all three studies.

The results thus suggested that individuals with higher pre-pandemic germ aversion were slightly less willing to accept COVID-19 vaccination, but this relationship was not statistically significant. This was the case also for seasonal influenza. In contrast to the present study, previous research has investigated attitudes and not intentions (Clay, 2017; Clifford & Wendell, 2016; Reuben et al., 2020). In the study that found a relationship between germ aversion and past influenza vaccination behavior (Luz et al., 2019), the relationship was mediated by attitudes to vaccines. Potentially, aversive reactions to pathogens and contamination might co-vary with environmental threat (Ackerman et al., 2018; Oaten et al., 2009; Stevenson et al., 2021), and aversion measured in a lower threat context is thus a poor predictor of outcomes at higher threat.

Perceiving oneself more susceptible to infectious diseases was not significantly related to intentions to accept a COVID-19 vaccine, but individuals perceiving themselves as more susceptible to disease reported higher intentions to accept influenza vaccination the next season. Speculatively, an explanation for this could be that fewer individuals in Finland have suffered from COVID-19 compared to influenza. The respondents' history of catching influenza or influenza-like illnesses could thus play a bigger role in shaping their perceived infectability.

4. Study 2: method

In Study 2, we measured the same constructs as in Study 1. The PVD scale was administered in 2020 during the pandemic. In addition, we measured the respondents' willingness to accept a test-phase vaccine against COVID-19. A test-phase vaccine is likely perceived as less safe than an approved vaccine recommended by authorities.

4.1. Respondents and procedure

In April 2019, we invited 5000 18–65-year-old individuals living in the Finnish Pietarsaari region to an online survey on vaccine attitudes. The individuals were randomly drawn from the Population Information System, which is a national register holding information about all individuals living in Finland (Digital and Population Data Services Agency, 2021). The sample of invitees was stratified according to the age, gender, and language distribution (Finnish and Swedish) in the region. We targeted the Pietarsaari region because it has a suboptimal uptake of several vaccines in the national vaccination program. The survey was open for two weeks, and a total of 1139 (22.8%) individuals responded. On March 30th, 2020, those 335 (29.4%) who in the data collection in 2019 had consented to being contacted again and provided their e-mail address were invited to fill out an online survey related to the coronavirus pandemic. A total of 205 (61.2%) individuals replied and 200 (59.7%) had responded to the questions on germ aversion and perceived infectability. The sample is described in Table 1. We compared the demographics of the individuals who responded in 2020 to those of the original sample who did not. The results from these analysis can be found in the online supplementary material.

The Board for Research Ethics at Åbo Akademi University approved the study. Respondents gave their informed consent electronically and did not receive any compensation for their participation.

4.2. Measures

4.2.1. Germ aversion and perceived infectability

As in Study 1, we measured germ aversion and perceived infectability with the PVD scale (Duncan et al., 2009).

4.2.2. Vaccination intention

Concerning a recommended vaccine against COVID-19, we presented the question: “Imagine a hypothetical scenario where the authorities recommend a new vaccine against COVID-19 free of charge. How likely do you consider it to be that you would accept such a vaccine?”. Hence, the formulation of the question was slightly different from that of Study 1. We additionally measured willingness to take a test-phase vaccine against COVID-19 with the question: “Imagine a hypothetical scenario where you would be offered the possibility to participate in a trial of a new vaccine against COVID-19 that is under development. How likely do you consider it to be that you would accept such a vaccine?”. Response alternatives ranged from 1 (very unlikely) to 5 (very likely).

To measure intentions to accept the influenza vaccine, we presented the question “Will you take the influenza vaccine next season (season 2020–2021)?” with the response alternatives “no”, “yes”, or “I don't know”. We re-coded the responses as 0 (no intent to vaccinate or undecided) or 1 (intend to vaccinate).

4.3. Statistical analyses

The statistical procedures were the same as in Study 1 (the measurement model is presented in Appendix Table A3). The reliability of the PVD subscales were α = 0.55, 95% CI [0.43, 0.64] for germ aversion and α = 0.82, 95% CI [0.77, 0.86] for perceived infectability. The reliability of the germ aversion subscale was thus poor. Nevertheless, measurement error was taken into account in the analyses, as the subscales were represented by latent factors. The fit of the measurement model was good.

5. Study 2: results and discussion

The respondents' vaccination intentions are presented in Table 2. Although the respondents were recruited from an area with sub-optimal vaccine uptake, they did not report systematically lower intentions to be vaccinated than the other two samples. The sample might thus not be representative of the population in the region. Descriptive information on the responses to the PVD scale is presented in Table A2.

The fit of the SR model was good (χ ²[109] = 164.05, CFI = 0.963, TLI  =  0.954, RMSEA =  0.050; 90% CI [0.034, 0.066], SRMR = 0.081). The zero-order relationships indicated that higher germ aversion and perceived infectability were significantly related to higher intention to accept a recommended COVID-19 vaccine and the next influenza vaccine (Table 4 ). In the regression model (Fig. 2 ; see also, Fig. 5 in the Meta-analysis section), where the PVD measures controlled for each other, the only statistically significant relationship was between perceived infectability and influenza vaccination intention. However, also this relationship was non-significant in the analysis controlling for gender and age (the strength of the relationship was approximately the same but the uncertainty in the estimate was larger; Table S2).

Table 4.

Zero-order correlations among measures in Study 2.

Variable	1	2	3	4	5
1. Germ aversion	–
2. Perceived infectability	0.60	–
3. COVID-19 vaccination intentions: recommended	0.24	0.16	–
4. COVID-19 vaccination intentions: test-phase	0.14	0.11	0.50	–
5. Influenza vaccination intentions	0.51	0.44	0.53	0.35	–

Note. Bolded correlations p < 0.05. Because the influenza vaccination measure was dichotomous, the estimates related to influenza vaccination intentions are represented by zero-order probit regression coefficients.

Fig. 2.

Germ aversion and perceived infectability as predictors of vaccination intentions in Study 2. Factor indicators and covariance, as well as disturbance covariances between outcomes, are not shown in the figure. *p < 0.05, **p < 0.01, ***p < 0.001.

In contrast to Study 1 (where germ aversion was measured before the pandemic), the relationships between germ aversion and vaccination intentions in Study 2 (where mid-pandemic germ aversion was measured) were positive and slightly stronger, but the relationships did not reach statistical significance in the SR models, potentially due to the small sample size.

6. Study 3: method

Study 3 replicated the analyses from Study 2 in a larger sample. In Study 3, we investigated the issue of perceived vaccine safety further by asking explicitly about the perceived safety of a recommended COVID-19 vaccine and the influenza vaccines.

6.1. Respondents and procedure

Between the 3rd and 17th of April 2020 (two weeks), we marketed a Facebook post with a link to an online survey including questions related to the coronavirus pandemic. The post reached 97,408 Facebook users and 2233 (2.3%) started to fill out the questionnaire. We included the 1325 individuals who completed the questionnaire (i.e., had responded to questions on the last page of the survey) and who had answered an attention check question correctly. One individual, who had not answered any questions related to germ aversion or perceived infectability, was excluded, and the final sample size was 1324. The mean age of the sample was 47.71 years (SD = 13.11, range = 18–100, n = 1022; see, Table 1 for more information on the sample).

The Board for Research Ethics at Åbo Akademi University approved the study. Respondents gave their informed consent electronically and did not receive any compensation for their participation.

6.2. Measures

6.2.1. Germ aversion and perceived infectability

As in Studies 1 and 2, we used the PVD scale (Duncan et al., 2009) to measure germ aversion and perceived infectability.

6.2.2. Perceived vaccine safety

We measured how safe the respondents perceived a vaccine against COVID-19 would be with the statements “If a vaccine against COVID-19 became part of the recommended vaccines in Finland, I would trust that it is safe”. Furthermore, we measured the perceived safety of the influenza vaccines with the statement “The influenza vaccines are safe”. Respondents indicated whether they agree on a scale from 1 (completely disagree) to 5 (completely agree).

6.2.3. Vaccination intentions

We measured intentions to accept a recommended vaccine and a test-phase vaccine against COVID-19 with the same questions as in Study 2. Intentions to accept the influenza vaccine the next season was measured in the same way as in Study 1.

6.3. Statistical analyses

The reliability of the PVD scale was α = 0.67, 95% CI [0.63, 71] for germ aversion and α = 0.82, 95% CI [0.80, 84] for perceived infectability. The statistical procedure was the same as in Studies 1 and 2 (the measurement model is shown in Appendix Table A4). Additionally, we analyzed a second model with the perceived safety of a recommended COVID-19 vaccine and the influenza vaccines as outcomes.

7. Study 3: results and discussion

Table 2 shows the responses to the variables measuring vaccination intentions. The responses to the PVD scale are presented in Table A2. All zero-order correlations between the PVD measures and the vaccine related variables were statistically significant and positive (Table 5 ).

Table 5.

Zero-order correlations between measures in Study 3.

Variable	1	2	3	4	5	6	7
1. Germ aversion	–
2. Perceived infectability	0.66	–
3. COVID-19 vaccination intentions: recommended	0.38	0.43	–
4. COVID-19 vaccination intentions: test-phase	0.16	0.22	0.61	–
5. Influenza vaccination intentions	0.37	0.43	0.61	0.46	–
6. Perceived safety: COVID-19 vaccine	0.33	0.37	0.81	0.58	0.59	–
7. Perceived safety: influenza vaccine	0.28	0.32	0.66	0.49	0.63	0.74	–

Note. Bolded correlations p < 0.05.

The fit of the SR model on vaccination intentions was satisfactory (χ ²[108] = 537.96, CFI = 0.960, TLI  = 0.950, RMSEA =  0.055; 90% CI [0.050, 0.060], SRMR = 0.047), as was the fit of the vaccine safety model (χ ²[96] = 549.15, CFI = 0.958, TLI  =   0.948, RMSEA =  0.060; 90% CI [0.055, 0.065], SRMR = 0.051). Germ aversion was significantly related to intentions to accept a recommended COVID-19 vaccine and the influenza vaccine (Fig. 3 ; see also Fig. 5 in the Meta-analysis section). The relationships were in the opposite direction of what was hypothesized based on previous studies, as those with more germ aversion had higher intentions to vaccinate. Individuals with more germ aversion also perceived the COVID-19 vaccine as significantly safer than those with less germ aversion (Fig. 4 ). There was no relationship between germ aversion and willingness to take a test-phase vaccine. Potentially, higher uncertainty concerning the safety of such a vaccine leads more individuals—not only those high on germ aversion—to decline vaccination. The same logic could explain why germ aversion was unrelated to perception regarding the safety of influenza vaccines. The perceived safety of the influenza vaccines has been found to be lower than that of other vaccines (Karlsson et al., 2021). This might be because the Pandemrix vaccine administered against the H1N1 influenza during the pandemic in 2009–2010 was later found to increase the risk of narcolepsy (Sarkanen, Alakuijala, Julkunen, & Partinen, 2018). The association between the vaccine and narcolepsy was frequently covered by Finnish media. This event might have caused more widespread concerns about the safety of influenza vaccines, not only among those with higher germ aversion.

Fig. 3.

Germ aversion and perceived infectability as predictors of vaccination intentions in Study 3. Factor indicators and covariance, as well as disturbance covariances between outcomes, are not shown in the figure. *p < 0.05, **p < 0.01, ***p < 0.001.

Fig. 4.

Germ aversion and perceived infectability as predictors of perceived vaccine safety in Study 3. Factor indicators and covariance, as well as disturbance covariances between outcomes, are not shown in the figure. *p < 0.05, **p < 0.01, ***p < 0.001.

For perceived infectability, the SR results on influenza from Studies 1 and 2 were replicated, as individuals with higher perceived infectability reported higher intent to take the influenza vaccine the upcoming season. Contrary to Studies 1 and 2, perceived infectability was also significantly related to willingness to accept COVID-19 vaccination. Perceived infectability was more strongly related to vaccination intentions than germ aversion was (confidence intervals not overlapping). Those with higher perceived infectability also reported significantly greater trust in the safety of the vaccines.

The control analyses produced roughly the same results (Table S3). Some of the relationships were slightly weaker and uncertainty in the estimates was larger when gender and age were included in the models. Because the relationships were small to begin with, some became non-significant in the control model. Nevertheless, the general conclusion, that the associations including germ aversion were in the opposite of what was hypothesized, and that higher perceived infectability was related to higher vaccination intentions, did not change when controlling for gender and age.

8. Meta-analysis

To receive an estimate of the overall size of the relationships between the behavioral immune system components and vaccination intentions, we combined the effect sizes obtained across studies and vaccines. We conducted separate meta-analyses for germ aversion and perceived infectability (the effect sizes we combined were still retrieved from the models where germ aversion and perceived infectability were simultaneously included). We also conducted separate meta-analyses for the pre- and mid-pandemic measures. Hence, the effect sizes concerning pre-pandemic PVD included only Study 1, whereas the effect sizes related to mid-pandemic PVD included Studies 2 and 3. To account for dependency between measures obtained from the same participants, we applied a mixed-effects model. The β coefficients were treated as r coefficients, and z-transformed to obtain linearity. The results were transformed back to r coefficients and interpreted as β coefficients.

The results from the meta-analyses are shown in Fig. 5 . There was no significant relationship between pre-pandemic germ aversion and vaccination intentions (β = −0.07, 95% CI [−0.15, 0.01], p = 0.091), whereas those with higher pre-pandemic perceived infectability had slightly higher intentions to accept vaccines (β = 0.12, 95% CI [0.04, 0.20], p = 0.004). The relationships between the mid-pandemic PVD measures and vaccination intentions were weak but statistically significant (β = 0.15, 95% CI [0.11, 0.18], p < 0.001 and β = 0.28, 95% CI [0.15, 0.41], p ≤0.001, for germ aversion and perceived infectability, respectively). Individuals who reported higher germ aversion and perceived infectability during the pandemic had higher intentions to accept vaccines.

9. General discussion

In the present study, we investigated if individual differences in aspects of the behavioral immune system (germ aversion and perceived infectability) can explain people's intentions to vaccinate against COVID-19 and their trust in the safety of the vaccine. Based on previous research (Clay, 2017; Clifford & Wendell, 2016; Hornsey et al., 2018a; Luz et al., 2019; Reuben et al., 2020), we hypothesized that higher germ aversion would be related to hesitancy to accept COVID-19 vaccination and believing the vaccine to be unsafe. We further hypothesized that higher perceived infectability would be related to higher willingness to accept COVID-19 vaccination. We also measured the respondents' intentions to accept influenza vaccination in the upcoming influenza season.

9.1. Germ aversion

Contrary to our hypotheses, the combined results of the three studies indicated that individuals with more germ aversion did not report lower intentions to accept vaccines (Fig. 5). Instead, the results indicated that pre-pandemic germ aversion was unrelated to vaccination intentions during the pandemic, whereas those with more mid-pandemic germ aversion had slightly higher intentions to vaccinate. This relationship was, however, weak. Previous research has focused primarily on attitudes to vaccines (i.e., the safety and efficacy of vaccines) and found higher pathogen disgust sensitivity and germ aversion to be associated with more negative attitudes (Clay, 2017; Clifford & Wendell, 2016; Luz et al., 2019; Reuben et al., 2020). These findings have been interpreted as the behavioral immune system reacting to vaccination as a cue to contamination, leading to an aversive response to vaccines. Our results indicated that those with more germ aversion during the pandemic perceived vaccines as slightly safer, and they were not less willing to accept a test-phase COVID-19 vaccine—a vaccine likely perceived as less safe—compared to those with less germ aversion. These results do not support the notion that individuals respond to vaccination as a cue to contamination.

One possible explanation for our finding is, of course, that vaccination is not recognized as a contamination cue. This explanation is, however, in contradiction with previous results. An alternative explanation could be that the increased disease threat at the time of data collection overrides an aversive response to vaccines as contamination, and that this is particularly true for those with high germ aversion. For individuals with high germ aversion, the perceived pathogen threat posed by vaccines might pale in comparison to that posed by COVID-19. This is in line with the assumption that the behavioral immune system response should be heightened in situations that pose increased disease threat. More specifically, the results would indicate that the effect of individual differences in the behavioral immune system are moderated by situational factors such as level of threat. It is noteworthy that mid-pandemic germ aversion was positively related also to intentions to accept influenza vaccination, indicating that the effect of increased disease threat is generalized across vaccines and not only affects the vaccine preventing the disease that poses the current treat. As the present study did not examine the relationship between germ aversion and vaccination during lower threat circumstances, this explanation is, however, highly hypothetical.

9.2. Perceived infectability

The combined results of the three studies suggested that individuals with higher perceived infectability had higher vaccination intentions (Fig. 5). Although the relationships were weak, the results are in line with the theory of the behavioral immune system. When individuals perceive themselves as more vulnerable, they are expected to be more motivated to take preventive action. Interestingly, those with higher perceived infectability perceived the vaccines as safer. This indicates that perceived infectability is related not only to vaccination intentions but to perceptions of the vaccines.

There was, however, variation between the studies in the strength of the relationships, with perceived infectability showing the most consistent relationship with intention to accept influenza vaccination. Previous research has found higher perceived infectability to be related to having accepted past influenza vaccination (Luz et al., 2019), but unrelated to attitudes towards childhood vaccines and vaccines in general (Clay, 2017; Luz et al., 2019). Potentially, perceived infectability is more robustly related to influenza vaccination intentions, as the PVD subscale mainly probes the respondents' history of catching infectious diseases. Relative to influenza, few individuals in Finland have been infected with COVID-19 (Finnish Institute for Health and Welfare, 2020, Finnish Institute for Health and Welfare, 2021a), and responses may, therefore, to a greater degree be based on respondents' history of catching influenza or influenza-like illnesses.

9.3. Study limitations and recommendations for future research

All data in the present study were based on self-reports. Self-reports are vulnerable to biases related to, for example, social desirability. To decrease the risk that social desirability would affect the responses, all respondents were informed that the data would be handled without identifying information. Also, the responses relating to the mid-pandemic period, and especially those relating to germ aversion, might have been affected by compliance with the official recommendations concerning protective behaviors during the time of the data collection, and might not have been solely based on heightened levels of aversion. The reliability of the germ aversion subscale, represented by Cronbach's alpha, was low in the three samples (α = 0.68, α = 0.55, and α = 0.67, respectively). Speculatively, this could be related to issues with translation or to the pandemic context (other studies measuring germ aversion during the pandemic has received α estimates around 0.70; De Coninck et al., 2020; Makhanova & Shepherd, 2020; Shook et al., 2020). Nevertheless, latent variables were used in the analyses to mitigate the influence of measurement error on the results.

The fact that two of the samples were relatively small leads to large uncertainty in the estimates. Furthermore, there are sample-specific limitations to the generalizability of the results. First, the respondents in the sample of Study 1 are participating in a longitudinal study that includes multiple health-related surveys and medical examinations over several years. It is possible that these respondents hold more positive attitudes to medical treatments, including vaccination, compared to the general population. Second, according to national statistics, the Pietarsaari region has a lower uptake of childhood vaccines compared to other Finnish regions. The sample recruited from this region in Study 2, however, did not systematically report lower willingness to accept vaccines than the other two samples. The Pietarsaari sample might thus not be representative of the region. This might be a result of the recruitment process, and that those who had consented to being contacted again held more positive vaccine attitudes than those who did not. Third, self-selection bias might be especially relevant for Study 3. Whereas the respondents in Studies 1 and 2 received personal invitations to the surveys, the sample in Study 3 entered the survey by clicking an ad in their Facebook feed. Individuals who chose to interact with the ad might differ from those who did not. For example, those responding might have more extreme attitudes to vaccines.

The fact that the current studies were conducted during and directly after the first pandemic peak can be considered both a strength and a limitation. The strength lies in the fact that we had a unique opportunity to investigate behavioral immune system mechanisms during higher disease threat. However, the questions about COVID-19 vaccination were highly hypothetical, and there was large uncertainty regarding if and when such vaccines could be available. Knowledge about the virus was also limited. The vaccination intentions that individuals report in such circumstances might be very different from their actual vaccination behaviors when they are offered a vaccine.

Our speculation, that the increased disease threat explains why our results differed from those obtained in previous research, should be investigated in future studies. Our study also differs from previous research in other ways. For example, we mainly investigated individuals' intentions to vaccinate themselves, whereas previous research has investigated attitudes to vaccines (except for Luz et al., 2019 that investigated past influenza vaccination behavior). Furthermore, none of the previous studies investigated attitudes or intentions related to COVID-19 vaccines. Nevertheless, we find it unlikely that these differences between studies cause the opposing results.

9.4. Conclusions

In the current study, individuals with high germ aversion were not more likely to react to vaccination as a contamination cue. Instead, the combined results of the studies suggested that those with higher germ aversion during the pandemic were slightly more willing to accept vaccines, and, thus, to protect themselves against disease. Level of germ aversion before the pandemic did not predict vaccination intentions. It is possible that current high disease threat moderates the relationship between behavioral immune system response and vaccination. When it comes to perceived infectability, the results were in line with the theory of the behavioral immune system and suggested that individuals perceiving themselves as more susceptible to diseases were slightly more willing to accept vaccination.

In sum, the results of the present study indicated that behavioral immune system activation promotes acceptance of vaccines when disease threat is high, but the associations were weak and of little practical relevance for public campaigns aimed at increasing vaccine uptake.

Funding

This project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 964728 (JITSUVAX).

This work was supported by the doctoral network of Minority Research at Åbo Akademi University; the Academy of Finland [grant numbers 316004 and 316726]; and the Polin Institute.

The funding sources had no involvement in the study design, the data collection, the analysis and interpretation of data, the writing of the report, or the decision to submit the article for publication.

CRediT authorship contribution statement

Linda C. Karlsson: Conceptualization, Methodology, Formal analysis, Investigation, Data curation, Writing – original draft, Writing – review & editing, Visualization. Anna Soveri: Conceptualization, Methodology, Investigation, Writing – review & editing, Supervision, Project administration, Funding acquisition. Stephan Lewandowsky: Conceptualization, Methodology, Supervision, Writing – review & editing. Linnea Karlsson: Funding acquisition, Methodology, Resources, Writing – review & editing. Hasse Karlsson: Funding acquisition, Methodology, Resources, Writing – review & editing. Saara Nolvi: Investigation, Methodology, Writing – review & editing. Max Karukivi: Investigation, Methodology, Writing – review & editing. Mikael Lindfelt: Writing – review & editing, Funding acquisition, Project administration. Jan Antfolk: Conceptualization, Methodology, Formal analysis, Writing – review & editing, Supervision.

Declaration of competing interest

None.

Appendix A.

Table A1.

Factor loadings and variances from the measurement model in Study 1.

Factor	Unstandardized			Standardized
	Parameter	Estimate	SE	Estimate	SE	R2
Factor loadings
Germ aversion	Item 1	1.00	−	0.57	0.06	0.33
	Item 3	1.17	0.15	0.67	0.04	0.45
	Item 4	0.92	0.13	0.53	0.05	0.28
	Item 7	1.27	0.17	0.73	0.05	0.53
	Item 9	0.46	0.14	0.27	0.08	0.07
	Item 11	0.72	0.14	0.41	0.07	0.17
	Item 13	0.87	0.14	0.50	0.06	0.25
Perceived infectability	Item 2	1.00	–	0.74	0.02	0.55
	Item 5	0.75	0.06	0.55	0.04	0.31
	Item 6	1.16	0.05	0.86	0.02	0.74
	Item 8	1.31	0.05	0.97	0.01	0.95
	Item 10	0.91	0.04	0.67	0.03	0.45
	Item 12	0.71	0.07	0.53	0.05	0.28
	Item 14	0.65	0.07	0.48	0.05	0.23
Factor variances
Germ aversion		0.33	0.07	–	–	–
Perceived infectability		0.55	0.03	–	–	–

Note. Model fit: χ²(73) = 111.46, CFI = 0.991, TLI  =   0.988, RMSEA =  0.042; 90% CI [0.025, 0.058], SRMR = 0.061. Residual correlations between the reversed items of Perceived infectability were included (Item 5 and Item 2: r = 0.45, p < 0.001; Item 12 and Item 14: r = 0.35, p < 0.001; Item 5 and Item 14: r = 0.38, p < 0.001).

Table A2.

Responses to perceived vulnerability to disease in all studies.

	Study 1 (2018)		Study 2 (2020)		Study 3 (2020)
	M	SD	M	SD	M	SD
Germ aversion
Item 1	5.64	1.60	6.54	1.02	6.64	1.05
Item 3a	4.49	2.02	2.16	1.76	1.67	1.56
Item 4	3.54	1.89	5.89	1.77	6.03	1.89
Item 7	3.40	1.81	6.15	1.34	6.38	1.43
Item 9	2.22	1.59	3.65	2.37	4.04	2.43
Item 11a	3.41	1.98	2.98	2.23	2.53	2.15
Item 13a	4.81	1.60	3.50	2.07	2.69	2.17
Perceived infectability
Item 2	2.90	1.41	4.43	1.39	4.34	1.62
Item 5a	4.30	1.62	2.99	1.83	3.30	2.05
Item 6	2.99	1.61	2.93	1.82	3.84	1.83
Item 8	2.98	1.58	3.38	1.75	3.94	1.92
Item 10	2.40	1.57	2.66	1.71	3.77	2.01
Item 12a	4.06	1.60	2.92	1.76	2.94	1.84
Item 14a	4.18	1.54	3.07	1.63	2.67	1.79

^aReverse-scored item.

Table A3.

Factor loadings and variances from the measurement model in Study 2.

Factor	Unstandardized			Standardized
	Parameter	Estimate	SE	Estimate	SE	R2
Factor loadings
Germ aversion	Item 1	1.00	−	0.77	0.11	0.59
	Item 3	0.55	0.17	0.42	0.12	0.18
	Item 4	0.80	0.15	0.62	0.11	0.38
	Item 7	0.80	0.15	0.62	0.09	0.38
	Item 9	0.55	0.13	0.43	0.07	0.18
	Item 11	0.54	0.16	0.42	0.11	0.18
	Item 13	0.48	0.21	0.37	0.15	0.14
Perceived infectability	Item 2	1.00	–	0.64	0.04	0.40
	Item 5	0.60	0.15	0.38	0.10	0.15
	Item 6	1.39	0.10	0.88	0.03	0.78
	Item 8	1.40	0.10	0.89	0.03	0.79
	Item 10	1.13	0.09	0.72	0.04	0.52
	Item 12	0.77	0.11	0.49	0.06	0.24
	Item 14	0.77	0.09	0.49	0.05	0.24
Factor variances
Germ aversion		0.59	0.17	–	–	–
Perceived infectability		0.40	0.05	–	–	–

Note. Model fit: χ²(73) = 121.97, CFI = 0.967, TLI  =  0.958, RMSEA =  0.058; 90% CI [0.039, 0.076], SRMR = 0.083. The model included residual correlations between the reversed items of Perceived infectability (Item 5 and Item 12: r = 0.62, p < 0.001; Item 5 and Item 14: r = 0.45, p < 0.001; Item 12 and Item 14: r = 0.44, p < 0.001).

Table A4.

Factor loadings and variances from the measurement model in Study 3.

Factor	Unstandardized			Standardized
	Parameter	Estimate	SE	Estimate	SE	R2
Factor loadings
Germ aversion	Item 1	1.00	−	0.67	0.05	0.45
	Item 3	1.05	0.09	0.70	0.04	0.49
	Item 4	0.88	0.08	0.59	0.04	0.35
	Item 7	1.27	0.09	0.85	0.03	0.72
	Item 9	0.81	0.07	0.54	0.03	0.30
	Item 11	0.71	0.08	0.47	0.04	0.23
	Item 13	1.01	0.09	0.67	0.04	0.45
Perceived infectability	Item 2	1.00	–	0.79	0.02	0.63
	Item 5	0.54	0.05	0.43	0.04	0.18
	Item 6	0.87	0.04	0.69	0.02	0.47
	Item 8	0.94	0.04	0.74	0.02	0.55
	Item 10	0.72	0.04	0.57	0.02	0.33
	Item 12	0.79	0.04	0.63	0.03	0.40
	Item 14	0.94	0.04	0.74	0.02	0.55
Factor variances
Germ aversion		0.45	0.06	–	–	–
Perceived infectability		0.63	0.04	–	–	–

Note. Model fit: χ²(72) = 458.30, CFI = 0.966, TLI  =   0.957, RMSEA =  0.064; 90% CI [0.058, 0.069], SRMR = 0.053. The model included residual correlations between Item 6 and Item 8: r = 0.60, p < 0.001; Item 8 and Item 10: r = 0.36, p < 0.001; Item 6 and Item 10: r = 0.31, p < 0.001; and Item 5 and 12: r = 0.32, p < 0.001.

Appendix B. Supplementary data

Supplementary material