Validity and reliability of Turkish version of the Vaccine Hesitancy Scale

Abstract

Objective

This study was planned to adapt the Vaccine Hesitancy Scale to Turkish and evaluate its validity and reliability.

Material and Methods

This research was conducted methodologically. The sample of the study consisted of 317 individuals from the patient relatives in the waiting rooms, who have children between the ages 9–16, who applied to the Suleyman Demirel University Hospital policlinic in December 2019. A questionnaire developed by the researcher and the Vaccine Hesitancy Scale were used as data collection tools.

Results

Explanatory factor analysis and confirmatory factor analysis methods were used to evaluate the validity and reliability of the Vaccine Hesitancy Scale. In the Vaccine Hesitancy Scale, the Kaiser-Mayer-Olkin coefficient was 0.897 and the Bartlett test’s chi-square value was 1319.67 and the result was significant (P <0.001). Considering the factor loads of the items, it varies between 0.608 and 0.845. As a result of the explanatory factor analysis, it was seen that the scale consists of two factors. This finding is consistent with the original form of the scale. The internal consistency coefficient of the scale was found to be 0.86.

Conclusion

As a result of the validity and reliability analysis, the Vaccine Hesitancy Scale was found to be a valid and reliable scale for the Turkish society, which consists of 9 items and 2 sub-dimensions.

What is already known on this topic?

• Vaccine hesitancy is a global public health problem.

What this study adds on this topic?

• There is no scale developed in Turkish to measure vaccine hesitancy.

• Objectively determining vaccine hesitancy in our country with a standardized measurement tool will help to investigate the causes of this situation, to evaluate our international situation, to observe its change over the years, and to develop immunization policies towards the target.

Introduction

Obtaining artificial immunity by giving vaccines created from weakened forms of bacteria and viruses or their antigenic parts to the body is called vaccination (1). The main purpose here is to prevent vaccine-preventable diseases in society, especially in infants and children, and to prevent morbidity and mortality that may result from these diseases (2). Vaccine-preventable diseases cause deaths and complications that are difficult to treat, especially in developing countries (3). According to the December 2019 report of the World Health Organization (WHO), 2–3 million deaths are prevented annually with global immunization. Nevertheless, the same report emphasizes that 19.4 million children under the age of one have not received their childhood primary vaccinations, and the proportion of children who received recommended childhood vaccinations under global immunization has remained the same for the past few years (4). Due to the important role of vaccines in controlling infectious diseases, the increase of anti-vaccine groups and individuals may cause a global problem in terms of social immunity (3).

The basis of the concept of vaccine rejection in the world is in England. Mandatory vaccination practices during the smallpox epidemic in England in 1853 caused a public reaction. This reaction has decreased with the awareness of society and positive policies (5). In recent years, concerns about vaccination have increased again. The WHO formed the ‘Vaccine Hesitancy Working Group’ in 2012 to deal with this issue more closely (6). According to the report prepared as a result of the studies of this group, vaccine hesitancy and vaccine rejection were defined as different concepts. Delay in having the vaccine or rejection despite reaching the vaccine is defined as vaccine hesitancy, and not having any vaccination is defined as vaccine rejection (7). According to the reports of WHO, many factors prevent vaccination such as contextual effects, individual-group effects, and effects related to vaccine or vaccination (6). It is observed that anxiety, hesitancy, and false beliefs related to vaccines increase with the effect of these factors (8). The number of families who do not want to have their children vaccinated in our country was 183 in 2011, 980 in 2013, 5 thousand 400 in 2015, 12 thousand in 2016, and the number of cases related to vaccine rejection reached the level of twenty-three thousand as of 2018 (9). According to the Turkey Demographic and Health Survey (TDHS), 20 thousand cases were found in never unvaccinated children 13–26 months in 2008 (1.6%), while in 2013, around 37 thousand cases (2.9%) were found (10). In 2018 TDHS, evaluations were made on 12–23 months-old children due to changes in the vaccination schedule, and the status of being never been vaccinated was found to be two percent (11). Although it is considered that the TDHS data are also affected by other reasons and do not fully reflect vaccine hesitancy, it can be said that our country is also affected by this process when evaluated together with other data. Studies on this subject are also carried out in our country (12, 13).

Larson et al. (7) developed the Vaccine Hesitancy Scale to compare the relationships of vaccine hesitancy, the situation of vaccine hesitancy between countries and to evaluate changes in this situation over time. The validity and reliability study of the Vaccine Hesitancy Scale was conducted by Shapiro et al (14). There is no scale developed in Turkish to measure vaccine hesitancy. Objectively determining vaccine hesitancy in our country with a standardized measurement tool will help to investigate the causes of this situation, to evaluate our international situation, to observe its change over the years, and to develop immunization policies towards the target. For these reasons, this study was planned to adapt the Vaccine Hesitancy Scale (VHS) to Turkish and to evaluate its validity and reliability.

Material and Methods

The research is a methodological research conducted to adapt VHS into Turkish and evaluate its validity and reliability. To conduct the study, permission was obtained from the Ethics Committee of Suleyman Demirel University dated 29/11/2019 and numbered 303 and the study was carried out following the Helsinki Declaration Principles.

Population and Sample

The population of the study consists of the relatives of the patients who applied to all outpatient clinics other than pediatrics at Suleyman Demirel University Hospital for any reason in December 2019, who have children between the ages of 9–16, who do not have communication problems, and who are in waiting rooms. Patients and their relatives who admitted to the pediatrics outpatient clinic were not included in the study, considering that they may cause bias in the survey results. No sample was selected in the study. In scale development studies, the number of people to be reached was suggested to be between 5 and 30 per item (15). For this reason, it is aimed to reach 270 people for our scale consisting of 9 propositions. The study was completed with 317 people who voluntarily participated in the study.

Data Collection Tools, Data Collection and Evaluation of Data

The questionnaire prepared by the researchers was applied by face-to-face data collection method. The data collection tool consists of two parts. In the first part, similar to the research in which the scale was developed, the sociodemographic characteristics of the participants (age, gender, educational status, marital status, the existence of social health insurance) are questioned. In the second part, there is the original 9-item form of the Vaccine Hesitancy Scale developed by Shapiro et al (14). The Vaccine Hesitancy Scale is prepared in Likert type and each statement consists of 5 choices: I strongly disagree (1), disagree (2), I am undecided (3), I agree (4), I strongly agree (5), 7 positive statements (1,2,3, 4, 6, 7 and 8 questions) and 2 negative statements (5^th and 9^th questions). The total score range of the scale varies between 9–45 and to evaluate the scale over the total score is recommended. In the scoring of the scale, the scores in the Lack of Confidence (1, 2, 3, 4, 6, 7, 8) sub-dimension consisting of positive statements as in the original scale should be inverted and summed, and the Risks (5, 9) sub-dimension consisting of negative statements should be added directly. Thus, higher scores indicate that the person has more hesitancy. There is no cut-off point for the Vaccine Hesitancy Scale. The Vaccine Hesitancy Scale is shown in Annex 1.

Appendix 1.

Turkish version of the Vaccine Hesitancy Scale

	Kesinlikle katılmıyorum	Katılmıyorum	Kararsızım	Katılıyorum	Kesinlikle katılıyorum
1. Çocukluk çağı aşıları çocuğumun sağlığı için önemlidir	1	2	3	4	5
2. Çocukluk çağı aşıları etkilidir.	1	2	3	4	5
3. Çocuğuma aşı yaptırmak toplumumdaki diğer kişilerin sağlığı için önemlidir.	1	2	3	4	5
4. Toplumumuza devlet tarafından sunulan tüm çocukluk çağı aşıları faydalıdır.	1	2	3	4	5
5. Yeni aşılar eski aşılardan daha fazla risk taşır.	1	2	3	4	5
6. Aşı programından aşılarla ilgili aldığım bilgiler güvenilirdir.	1	2	3	4	5
7. Aşı yaptırmak çocuklarımı hastalıklardan korumak içi iyi bir yoldur.	1	2	3	4	5
8. Doktorum veya yardımcı sağlık personeli (ebe, hemşire vb.) çocuklarım için aşılar hakkında ne önerirlerse genellikle yaparım.	1	2	3	4	5
9. Aşıların ciddi yan etkileri hakkında endişeliyim	1	2	3	4	5

In the original scale, a two-factor structure, namely lack of confidence (Cronbach’s alpha = 0.92) and Risks (Cronbach’s alpha = 0.64), was introduced. Other fit indices of the original scale are given as The Root Mean Square Error of Approximation (RMSEA) 0.09; comparative fit index (CFI) 0.95; non-normed fit index (NNFI) 0.94. To adapt the scale to Turkish, permission was obtained by contacting one of the authors who developed the scale.

Statistical Analysis

The frequency, percentage and mean tests were used to describe the demographic characteristics of the participants, item analysis for the validity and reliability studies of the scale, Cronbach’s alpha reliability coefficient, exploratory factor analysis, and confirmatory factor analysis methods under structural equation modeling. Statistical Package for the Social Sciences (SPSS) version 22.0 (IBM SPSS Corp.; Armonk, NY, USA) was used for exploratory factor analysis in the analysis of research data, and AMOS 23 package program was used for confirmatory factor analysis.

Results

Sociodemographic Findings

Thirty percent of the study group were male (n=95). The mean age of the individuals was 37.6±7.2, 25.6% of them were primary school graduates and less, 15.8% were secondary school graduates, 31.5% were high school graduates, 27.1% had a university degree and a higher education status. 95.9% of the group was married, 90.5% had social health insurance.

Validity and Reliability of the Vaccine Hesitancy Scale

The findings regarding the validity of the Vaccine Hesitancy Scale are given as language validity of language, construct validity, and reliability.

Validity of Language

In the first phase of the research, studies were conducted for the language validity of the scale by obtaining permission from the responsible author, Gilla K. Shapiro, who developed the scale for adaptation to Turkish. The scale was translated from English to Turkish by three people who know Turkish and English well and whose native language is Turkish, and a common Turkish translation was created by the researchers. Later, this common form was translated from Turkish into English by a linguist, an academician (infectious diseases) who knew both languages well, and the differences between the translations and the original form of the scale were examined by a person who worked in the field of vaccine hesitancy. The scale form created after these evaluations was presented to the opinion of an 8-person expert group (working in the field of public health) to be evaluated in terms of understandability, suitability for purpose, compatibility with our culture, and language equivalence. Experts use expressions for the measurement degree of each item in the scales such as “Not suitable”: 1, “The item needs to be changed accordingly”: 2, “Appropriate but needs minor change”: 3, “Very suitable”: 4, and each item was asked to be scored between 1–4 (16). Then, the expert opinions were evaluated with the Kendall W test to determine content validity. Kendall’s was calculated as W = 0.096 and P=0.635. The fact that there is no statistically significant difference between the scores given by the experts to the items of the VHS indicates that there is no difference between their views and that there is “agreement between independent observers” reliability (17). It can be said with these results that the language and content validity criteria of the scale translated into Turkish are met.

Construct Validity and Reliability

After the scale was applied to the target group, exploratory and confirmatory factor analysis methods were used to evaluate the construct validity of the scale. Kaiser-Mayer-Olkin (KMO) and Bartlett tests were conducted to determine the suitability of the sample examined in factor analysis. The KMO coefficient of VHS was 0.897 and the chi-square value of the Bartlett test was found to be 1319.67, and the result was found to be significant (P<0.001). With these results, the VHS data set was found suitable for factor analysis at a “very good” level. Also, the suitability of each question for factor analysis was measured with anti-image correlation. If the anti-image correlation value is less than 0.50, excluding the relevant item from the analysis is recommended (18). Anti-image Correlation values of all items of the Vaccine Hesitancy Scale were examined and all of them were found above 0.5. For this reason, it was not planned to exclude any item from the analysis.

Factor loads of items, change in Cronbach’s alpha coefficient when the item was removed, item discrimination power index, common variance values in items, item-total correlations, and Cronbach’s alpha coefficients were examined for construct validity and reliability.

When the factor loads of the items in the Vaccine Hesitancy Scale were examined, it was found that they varied between 0.608 and 0.845 (Table 1). The propositions on the scale had a high level of factor load. These values were found to be sufficient as an indicator of the construct validity of the scale. The load distributions formed according to the factor analysis were examined, there was no item with a load value below 0.45 in the factor and the difference in load values in at least two factors less than 0.1, all items showed a high load distribution in a single factor. According to the factor analysis results, 9 items were collected under two factors with eigenvalues greater than 1.0. A two-factor structure was seen in the slope graph, where the point where the slope starts to disappear or the line showing the slope begins to flatten expresses the factor number (Figure 1). According to the analysis results, the total variance ratio explained by the two-factor structure was found to be 63.429%. The eigenvalue of the first factor was 4.642 and the variance it explained was 51.573%, the eigenvalue of the second factor was 1.067 and the variance it explained was 11.856%. When the Kaiser criterion (eigenvalue number greater than one) was taken as basis in determining the appropriate factor number and the slope graph of the analysis was examined, it was seen that the 9-item VHS, consisting of seven positive and two negative statements, was distributed to two factors, as in the original version of the scale. The factors were named as Factor 1: lack of confidence (1, 2, 3, 4, 6, 7, 8), Factor 2: risks (5,9), reflecting their content. The lack of confidence sub-dimension of the scale contained positive statements, while the risks sub-dimension contained negative statements. The items in these two sub-dimensions were evaluated as strongly disagree (1), disagree (2), undecided (3), agree (4), strongly agree (5) points in the original scale. In the scoring of the scale, seven items (lack of confidence sub-dimension) as in the original scale were reversed, thus high scores showed that all items experienced more hesitancy.

Table 1.

Results of item and reliability analysis of the vaccine hesitancy scale

	Item Number	Mean	Standard Deviation	Item-Total Correlation	Item Discrimination Index	Cronbach’s Alfa	Factor Loading	Rotated Factor Loading
FACTOR 1	1	1.45	0.66	0.732	13.984	0.889	0.845	0.855
	2	1.51	0.66	0.703	13.214		0.817	0.829
	3	1.59	0.82	0.551	11.434		0.678	0.716
	4	1.72	0.85	0.733	17.823		0.813	0.757
	6	1.90	0.94	0.605	13.424		0.710	0.680
	7	1.59	0.74	0.737	17.090		0.835	0.814
	8	1.51	0.68	0.637	12.652		0.738	0.701
FACTOR 2	5	2.92	0.80	0.449	7.482	0.467	0.753	0.830
	9	2.65	1.20	0.307	15.725		0.608	0.751

Figure 1.

Number of factors-eigenvalue slope plot

As a result of the exploratory factor analysis, the items were listed and load distributions, eigenvalues, and explained variance values were calculated. Among the exploratory factor analysis methods used in the original scale, the principal components method and the “varimax” rotation method, one of the orthogonal rotation methods, were preferred. The mean, standard deviation, item-total correlation, item discrimination index, factor analysis, rotated factor analysis, and Cronbach’s alpha values of the items in each factor are shown in Table 1.

If the Cronbach’s alpha coefficient increases more than 5% when the item is deleted from the scale, it is decided that that item will be eligible to be removed from the scale. An item (item 5) whose reliability coefficient increased with its removal was not removed from the scale as suggested, since it did not increase the reliability coefficient by more than 5%. To determine the item discrimination of the scale items, the method of the Top-Bottom 27% Groups Comparison was used and the item discrimination index was calculated. As a result of the analysis, the difference between the groups was found to be significant at the P<0.001 level for each item, and it was determined that the t value for each item was positively signed. Table 1 shows the t values for the item discrimination index analysis.

When the item-total correlations of VHS are examined, it is seen that it has medium, strong, very strong positive values between 0.307 and 0.737 (Table 1). In this scale used, the item with a negative item-total correlation coefficient and below +0.30 was not detected. The Cronbach’s alpha coefficient was used to determine the internal consistency since the items of VHS contain a Likert type evaluation. Cronbach’s alpha value of the scale was found to be highly reliable as 0.856. The Cronbach’s alpha value for the lack of confidence sub-dimension of the scale was calculated as 0.889, and the Cronbach’s alpha value for the Risks sub-dimension was calculated as 0.467 (Table 1). The Cronbach’s alpha value of the lack of confidence sub-dimension in the original scale was shown as 0,92, and the Cronbach’s alpha value of the Risks sub-dimension was shown as 0,64.

When the reliability analyses are examined in terms of item analysis and additivity of the scale, it has been observed that the items differ significantly (ANOVA, Tukey’s test for nonadditivity P<0.001). Nonadditivity probability was found to be P= 0.686 and P<0.001 in Hotelling’s t² test. These values show that the items in the scale measure different sub-dimensions and that the items of the scale are summable.

Confirmatory factor analysis was performed on the sample in which the exploratory factor analysis was performed, and the fit indices of the model were examined. The unweighted least squares method is used as the method. In the modification indices, covariances between 2 values e1-e2 with covariances MI value above 50 were created and the fit values were improved. Whether the models are suitable in the confirmatory factor analysis assessment of the scale used for the research and whether the factors are sufficient to explain the model were decided as a result of examining Chi-square (x²) value corrected with degrees of freedom (df), the values in the standardized residual covariance matrix and other goodness of fit indices (19). Confirmatory factor analysis compatibility values of the scale are shown in Table 2 and the path diagram for confirmatory factor analysis is shown in Figure 2.

Table 2.

Confirmatory factor analysis compatibility values of the Vaccine Hesitancy Scale

Model fit indexes	Good Fit	Acceptable Fit	Scale Values
NPAR			20
Chi-square (χ2)			38.382
P	0.05<P≤1	0.001<P≤0.05	0.042
Degrees of Freedom (DF)			25
Chi-square/Degrees of Freedom (χ2/DF)	0≤ χ2/sd ≤2	2<χ2/sd≤3	1.535
Root Mean Square Error of Approximation (RMSEA)	0≤RMSEA≤0.05	0.05<RMSEA≤1	0.041
Standardized Root Mean Square Residual (SRMR)	0≤SRMR≤0.05	0.05<SRMR≤1	0.032
Comparative Fit Index (CFI)	0.95≤CFI≤1	0.90≤CFI<0.95	0.928
Goodness of Fit Index (GFI)	0.95≤GFI≤1	0.90≤GFI<0.95	0.973
Adjusted Goodness of Fit Index (AGFI)	0.90≤AGFI ≤1	0.80≤AGFI<0.90	0.951
Normed Fit Index (NFI)	0.95≤NFI≤1	0.90≤NFI<0.95	0.901
Non-Normed Fit Index (NNFI) (TLI)	0.97≤NNFI≤1	0.95≤NFI<0.97	0.966

Figure 2.

Path diagram for confirmatory factor analysis

Discussion

The purpose of this research was to make the validity and reliability study of VHS in Turkish. In our country, 9-year-old children are children who have completed the vaccination schedule except for tetanus-diphtheria booster (it is done in the 4th grade of secondary school). The criterion of having a child between the ages of 9–16 was used both because of the original form of the scale and its compatibility with our vaccination schedule. Construct validity is defined as the degree to which a test, which measures a property that cannot be directly measured, can do this completely and objectively (15). Explanatory factor analysis and confirmatory factor analysis were performed to evaluate the construct validity of the Vaccine Hesitancy Scale. The suitability of the sample examined in factor analysis can be done by several methods (20). Kaiser-Mayer-Olkin and Bartlett tests were performed to determine this suitability. For the Kaiser-Meyer-Colin coefficient, the values are evaluated as; 0.90–1.00 excellent, 0.80–0.89 very good, 0.70–0.79 good, 0.60–0.69 medium, and 0,50–0.59 weak. Bartlett’s Test of Sphericity tests whether the correlation matrix is an identity matrix. If the p-value found as a result of Bartlett’s sphericity test is less than 0.05, it is said that the correlation matrix is suitable for factor analysis (21). The VHS data set was found to be “very good” for factor analysis with the results stated in the findings,

In determining the number of factors, it is recommended to have a factor as much as the number of eigenvalues with a value greater than 1, consider the eigenvalue number explaining at least 40% of the total variance, and examine the scree plot (22). As a result of the exploratory factor analysis, it was seen that the scale consists of two factors, with the results covering all three of these criteria. This finding is consistent with the study conducted with the original form of the scale. These two factors are named as “Lack of Confidence” and “Risks” similar to the original form study of the scale,

Our analyses show that the factor loads of the scale items vary between 0.608 and 0.845. The relationship of items with factors is explained by factor loads and, according to Büyüköztürk (23), factor load values of the scale being 0.45 and higher indicates that it is a good criterion for selection. Although studies are supporting this value, there are also studies in the literature stating that the 0.3 limit value can be used for factor loads (18, 24). Our results show that these criteria are met.

The significance of the ANOVA Tukey’s test for nonadditive value (P<0.05) indicates that the items are significantly different from each other and the items in the scale have a structure that can explain at least two different sub-dimensions. Scale can be summed if nonadditivity is unimportant (25). Nonadditivity probability of Vaccine Hesitancy Scale was determined as P=0.686, and according to this result, items in the scale can be summed up. Hotelling’s t² test tests whether the item means are equal to each other. If the item means differ significantly, the items in the scale measure different sub-dimensions (25).

Internal consistency coefficient (Cronbach’s alpha) is one of the methods used to determine reliability in scale validity and reliability studies. If the internal consistency coefficient is between 0.00 and 0.40, the scale is not reliable, between 0.40–0.60 lowly reliable, between 0.60–0.80 quite reliable, between 0.80–1.00 highly reliable (17). The Cronbach’s alpha value for the vaccine hesitancy scale was found to be highly reliable as 0.86. In the sub-dimensions of the scale, Cronbach’s alpha value was 0.889 for the ‘lack of confidence’ and was 0.467 for the ‘risks’. The reason for the low Cronbach’s alpha value for the risks sub-dimension is due to the low number of items in the sub-dimension. Therefore, it is recommended to use a total score.

One of the methods used to make the scale more reliable is to evaluate item-total correlations for each item. Items with total item correlation coefficients between 0.20 and 0.30 are items that need correction; between 0.30–0.40, are well discriminating; a level of 0.40 and higher are considered to be very good distinguishing items. Items with this coefficient lower than 0.20 should not be included in the scale even if they are statistically significant (26). While some researchers want this value to be greater than +0.25, some studies have stated that the limit value can be taken as 0.30 (25). When the item-total correlations of the Vaccine Hesitancy Scale are examined, it is seen that there is a medium, strong, and very strong positive direction between 0.307 and 0.737.

Confirmatory factor analysis is a verification and test method used to determine whether there is a sufficient level of correlation between the factors obtained from the explanatory factor analysis, which variables are associated with which factors, whether the factors are independent of each other, and whether the factors are sufficient to explain the model (27). As a result of the confirmatory factor analysis, the ratio of Chi-square value to the number of degrees of freedom below 3 indicates that the model is good and corresponds to a perfect fit (28). This ratio was calculated as 1.707 in our study. When the other fit indices are examined, the root mean square error of approximation (RMSEA) calculated as 0.047 in the analysis which is equal to or less than 0.05 indicates the perfect fit (29); Goodness of fit index (GFI) and comparative fit index (CFI) of 0.95 and above is a perfect fit; Normed fit index (NFI), adjusted goodness of fit index (AGFI), and non-normed fit index (NNFI) of 0.90 and above also indicate a good fit (28). The fit indexes show that VHS has a very good fit, and our study is at an acceptable level. Validity and reliability analyses also show that VHS is a valid and reliable scale.

Since our study was a VHS validity and reliability study, it was applied only to individuals with children between the ages of 9–16 to be faithful to the sample that the original study was conducted. Studies in different groups are needed to be generalized to society. Also, invariance such as test-retest and parallel forms reliability was not evaluated.

Conclusion

As a result, the Vaccine Hesitancy Scale consisting of 9 items and 2 sub-dimensions was seen as a valid and reliable scale for the Turkish society after the validity and reliability analyses.