Turkish middle school students’ evaluation of fallacious claims about vaccination

Abstract

In today’s world, there is too much information pollution and people circulate it without questioning, and the claims on controversial issues often contain fallacies and conspiracy theories. Considering this point of view, it is necessary to create citizens who critically evaluate information. In order to achieve this goal, science educators need to address students’ evaluation of fallacies on controversial issues. The aim of the present study, thus, is to explore eighth graders' evaluation of fallacies about vaccination. We used case study in the study in which 29 eighth grade students participated. We adapted a rubric that was developed by Lombardi et al. (Int J Sci Educ 38(8):1393–1414, 2016. https://doi.org/10.1080/09500693.2016.1193912) to assess students' evaluation levels between claims and evidence and used it to analyze students' evaluations of each fallacy in groups and individually. The findings of this study indicate that students were mostly unable to evaluate claims and evidence critically. We advocate that attempts should be made so that students cope with misinformation and disinformation, ensuring that they establish a consistent relationship between claim and the evidence, and the social and cultural factors that affect their evaluation of false claims should be identified. The implications of this study suggest to make deliberate attempts in order to enable middle school students to evaluate claims and evidence critically in various science topics, particularly in health issues, due to the COVID-19 pandemic. Implications of the present research include suggestion of a method including discussions of fallacies about controversial issues and utilizing additional data sources such as interviews to reveal student ideas in-depth and analyze students' decision-making skills.

Postmodernism is a movement, which argues that there is no objective truth and people who claim to know the truth actually are just trying to oppress people (McIntyre, 2018). Nowadays, with the impact of this movement, people tend to reject evidence-based information because they think that there is no objective truth and that evidence-based information cannot lead to the truth. Stuart Sim (2019) stated that each generation has its own particular spectre and post-truth is our spectre, which we need to struggle to respond. In this age, more concerning than lack of knowledge is individuals' attitudes and inappropriate beliefs which is not evidence based (Rizeq, Flora and Toplak 2021). Although the conflict of interest between scientists and industrial companies such as tobacco, lead and asbestos as well as the resistance to change occasionally makes science a conservative discipline (Krimsky, 2019), the growth of scientific knowledge includes a series of processes such as elaborative investigation, scrutinized research and peer review. However, scientific conclusions have also been extensively questioned by non-experts who disagree with these conclusions in the age of post-truth (McIntyre 2018).

Post-truth is a philosophical concept used to describe the disappearance of shared objective standards for truth and the undifferentiated blurring of facts, values, knowledge, opinion, and belief (Biesecker 2018). In the post-truth era, such kind of questioning is sometimes based on the biases, beliefs, ideology and prejudices of the questioners. As a result of these inquiries, personal beliefs override tested knowledge, science denial emerges, and due to the rapid development of communication technologies, all kinds of ideas can be circulated. Considering that social media is the primary news source for people in the age we live in (Bronstein et al. 2020), the sharing of untested information promotes the rapid circulation of misinformation and conspiracy theories on various issues. Such topics are ranging from anthropogenic climate change to vaccination as well as from genetically modified organisms to the shape of the earth.

Individuals begin to perceive the truth as a form of cultural cognition in harmony with the identity of the groups they belong (Lobato and Zimmerman 2019). Social media platforms keep their users in filter bubbles with their algorithmically personalized newsfeeds (Pariser 2011). In such a way, social media may restrict diverse perspectives and lead to the formation of echo chambers (Cinelli et al 2021) where like-minded people come together and possibly affect other individuals' decision-making processes as in the Cambridge Analytica scandal. In post-truth period, people may adopt the information that fits to them or their groups’ beliefs and prejudices and they publicize them without questioning. Some people tend to accept a claim as if it is true and tend to spread that claim even if they know that the claim is not valid since it is compatible with their beliefs and opinions (Rini 2017). If a claim does not fit their worldview, they prone to ignore that claim or assume it is false. Even in the scientific world, similar situations have occurred in the past. The history of science is full of examples of the mentioned bias. For example, two American scientists, namely Michelson and Morley, who believed space permeated with luminiferous ether tried to detect the existence of it. Despite the negative result, they insisted on their hypotheses and changed the experimental conditions. However, each condition changing showed that there is no such substance permeating the space. Mendel, on the other hand, preferred to ignore research results that did not meet his mathematically derived expectation. The 2016 Brexit referendum and the US presidential election are among the most recent events that can be an example of this situation, where the facts were obscured and evidential standards in reasoning were abandoned (McIntyre 2018). In addition to incompatibility with the common sense that leads to ignoring or distorting the truth, there may also be conflicts of interest. The private sector has made a number of attempts to distort the truth, using the names, titles and biased publications of respected scientists to protect their business interests. Commercial companies used a method called the "Tobacco Strategy" by Naomi Oreskes and Eric Conway (2010) that established a conflict of interest with scientists all served in high levels of science administration. These scientists blurred the relationship between smoking and cancer and claimed that volcanoes cause acid rain and ozone holes. They published and made statements, claiming that secondhand smoke and asbestos are not harmful and emphasizing that global warming is a natural phenomenon. At the same time, Emilio Lobato and Corinne Zimmerman (2019) reported that the role of factors such as group affiliation or ideology in science denial might depend on specific scientific issues rather than against the whole of science. Although there is a certain consensus in the scientific community on these issues, due to some social, ideological and religious reasons those issues are defined as controversial issues. From this point of view, it seems necessary to explore students’ evaluation of scientific knowledge and fallacious claims for an effective teaching of scientific knowledge about controversial issues. By controversial issue, we mean an issue which is understood differently by people because of its and people’s socio-political background and lack of knowledge resulting from pro-capitalist falsehoods and neoliberal manipulations.

Science learners often go through a reasoning process during discussing scientific arguments about scientific topics that were mentioned in science classes. However, in daily life or social media, individuals are mostly exposed to cases containing fallacies and they tend to adopt the claims containing fallacies although they completed their formal education including science courses in middle and high school levels, and even in higher education. Hence, we argue that the claims about controversial issues containing fallacies should take part in the instruction process for science learners. We argue in this paper that the exploration of students’ evaluation of fallacies about vaccination through evaluation of claims and evidence will bring new light to science education literature. The fallacies and fallacious reasoning were rarely and theoretically used in science education (see Saribas and Cetinkaya 2021; Jungwirth 1987; Zeidler, Lederman and Taylor 1992; Zeidler 1997). Thus, there is a need to discuss non-scientific claims and false reasoning in classroom practice. The goal of this study, therefore, was to explore eighth graders’ evaluation of fallacious claims about vaccination. From this point of view, we addressed the following research questions in the present study:

• How do eighth graders evaluate fallacious claims about vaccination?

• How do participant groups evaluate claims and evidence?

• How do individual students evaluate fallacious claims after group discussion?

Theoretical framework

Anti-vaccine movement

The first documented vaccination was conducted by a British physician called Edward Jenner who vaccinated his family in 1774 against smallpox using cowpox. It was also the first documented event of anti-vaccination at the same time. His neighbors condemned him to be "inhuman brute" (Barquet and Domingo 1997), and he was insulted whenever he attended markets in the neighborhood (Howard and Reiss 2018). Although the anti-vaccination movement was first embodied with the smallpox vaccine in the past, the recent anti-vaccine movement has been accelerated by an article, which was written by Andrew Wakefield et al. (1998) who linked MMR (measles, mumps, rubella) vaccine and autism. Although Andrew Wakefield et al. (1998)'s study did not have a control group and had a small sample size besides numerous ethical problems, some of the mass media circulated their claims. Media mostly focused on celebrities rather than science and scientific explanations and depicted Wakefield as a hero (White 2014). Subsequent studies with large number of children revealed that the rate of autism is independent of the MMR vaccine (Taylor, Swerdfeger and Eslick 2014). After these fallacies became widespread and an anti-vaccination movement occurred, a considerable number of parents believe that their children were harmed by vaccines, and they began to adopt alternative treatments and products to the vaccines. Non-vaccination has dramatic effects on public health such as the increase in the risk of children’s sickness, and widespread non-vaccination undermines herd immunity and leads to pandemics (Howard and Reiss 2018).

Vaccine rejection, and particularly hesitation to MMR vaccine, has led to a significant increase in cases of measles. In 2013 New York City measles outbreak, it was determined that 78% of 58 patients, most of whom were children, were not vaccinated with parental refusal, and more than half of the patients did not apply for medical health care and contacted thousands of people (Rosen et al. 2018). Similarly, according to Turkish Medical Association (2018), the number of families who refused to have their child vaccinated increased from 183 to 23,000 just in 7 years in Turkey, and in the same period measles cases also rose from 105 to 716 people. A recent study in Turkey showed half of the people stated that they would not get the vaccine if there is a vaccine against COVID-19 (“The rate of in Turkey” 2020). It is known that these results are not limited to several countries and vaccine rejection has become a global problem. Vaccine rejection might be due to a deficiency in reasoning in terms of fallacious claims about vaccines, which correspondingly might be related to lack of scientific literacy. It is extremely important to know and recognize the arguments of the anti-vaccine movement and to be aware of the fallacies that it contains in different cultures.

Mats Lundström, Margareta Ekborg and Malin Ideland (2012) explored how teenagers decided whether to vaccinate themselves or not and found that the students rarely used school science repertoire in their talk about vaccination. Instead, risk, solidarity, family and friends and the media repertoires were included in their talk. These researchers indicate the need to discuss the arguments from family and friends as well as media reports in dealing with scientific literacy. However, it is important to note that the students’ families and friends might also be fallacious. Thus, it seems necessary to let the students be aware of various types of fallacies and evaluate them critically.

Fallacies and type of fallacies

Fallacy is a type of argument that contains errors in reasoning (Copi, Cohen and McMahon 2014). Some people commit fallacies purposely, in order to convince someone in a debate. The arguments that are put forward by science deniers in their pseudoscientific claims often contain logical fallacies. According to Walton (2007), people see fallacies as tricky tactics to win a debate unfairly. Students’ fallacious reasoning damages the curricula goals, such as decision-making and scientific literacy (Zeidler, Lederman and Taylor 1992). Analyzing and evaluating arguments, identifying fallacies and judging the trustworthiness of the sources are the significant elements of scientific literacy and critical thinking (Vieira and Tenreiro-Vieira 2016). For this reason, students should be aware of the logical fallacies that are frequently committed in argumentation and able to recognize and critically evaluate the claims when they encounter logical fallacies. In order to fulfill this requirement, the researchers of this study used logical fallacies as a teaching tool in the context of anti-vaccination that the recently re-rising star of science denial. Table 1 illustrates the types, definition and examples of some common logical fallacies frequently encountered in everyday life.

Table 1.

Types of logical fallacies

Fallacy	Description	Example
Argumentum ad hominem	This fallacy occurs when someone attacks the claimant personality, rather than responding to the other party's argument	You say very easily that vaccines do not cause autism in children because you don't even have a child
Argumentum ad populum	This fallacy occurs when someone accept a claim as true because majority of people (Kelly 1990) or a particular group accept it (Bennett 2012)	Majority of women who have a baby think that vaccines are harmful, therefore vaccines can damage my baby
Argumentum ad verecundiam	A fallacy of an argument that the argument is valid, just because it raised by an authority without stressing that the statement is valid. This fallacy ignores the validity of the authority's statement	Vaccines are a product of big pharma. Nobel prize winner author explained this topic in depth
Argumentum ad passions	Such arguments (appeal to fear, loyalty, force, compassion or conscience) are based on validity of a claim by addressing someone's emotions	Today, the governments take away the right of parents to make decisions regarding their children. What's next? Our children?
Circulus in probando	This fallacy occurs when one of the premises of the argument is used to confirm the conclusion	The author says vaccines are not effective in preventing epidemics. He already writes that in his book
Post Hoc Ergo Propter Hoc	It is a kind of fallacy created by establishing an imaginary link between the premises and the conclusion (Hurley and Watson 2018)	My sister got autism after vaccination. So, vaccination causes autism
Secundum quid	Hasty generalization occurs when the particular sample is not representing the rest of the group (van Eemeren and Grootendorst 2004)	Wakefield et al. (1998) connected MMR vaccines with autism and his study performed with only 12 children, but generalizes this result to the whole society
Onus Probandi	This fallacy occurs when someone places the burden of proof on the wrong side of the argument	Mercury in vaccines causes autism. Show me if you have evidence which proves otherwise
Slippery slope	This fallacy occurs when there are very weak connections between the premises (Muniz 2019)	Today, the governments interfere with the MMR vaccine and tomorrow will interfere with all vaccines. If this continues, it will result in interference with fundamental rights and freedoms
Straw man fallacy	This fallacy is based on making argument easier to attack by distorting or exaggerating his/her argument to outwit the opponent	You talking about the benefits of flu vaccines, but you never mention animal experiments to develop these vaccines
False dichotomy fallacy	This fallacy occurs where people are forced to choose only one of two options although there are more than two possible options	You either get vaccinated and get autism, or you stay healthy without being vaccinated
Nirvana fallacy	It is a fallacy of comparing realistic things with unrealistic or idealized one	The vaccines provide 85% to 95% protection. Since vaccines are not 100% effective, vaccination is useless
Genetic fallacy	This fallacy occurs when someone advocates an argument just based on the origin of its premises or claims rather than its current context	You're not going to get vaccinated, are you? You know "vaccination" comes from "vacca" which means cow in Latin. I don't even want to think that the you would ignore the idea that vaccines would look like cows

Fallacies of anti-vaccine activists’ arguments and critical evaluation

Conspiracy theories are the main source of the anti-vaccine movement, which also contain logical errors, and number of conspiracy theories spread in the public by the anti-vaccine movement could be a book by itself (Howard and Reiss 2018). These conspiracy theories often seem to be designed to spread fear and keep people away from vaccinations. Thus, conspiracy theorists assert fallacious arguments. Some of these can be listed as follows: vaccines cause infertility (Feldman-Savelsberg, Ndonko and Schmidt-Ehry 2000), cancer (Chitukuta et al. 2019), disability and paralysis (Harmancı, Gürbüz, Torun, Tümerdem and Ertürk 2003). In addition, the doctors are concerned about the claims such as vaccine information concealed from the public (Kata 2010) and viruses including Ebola and COVID-19 are produced in laboratories to sell vaccines.

Facilitating critical evaluation of conspiracy theories and fallacies about vaccines necessitates evaluating claims based on evidence. In order to achieve these goals, the students need to decide whether evidence supports, contradicts or nothing to do with the claim. Science education, thus, requires developing methods and scaffolds how scientific knowledge is constructed and evaluated (Erduran and Dagher 2014). Improving students’ understanding of the development of scientific knowledge requires deliberate attempts to let them use reasoning to decide that scientific explanation is constructed by the lines of evidence obtained from trustworthy sources and through reliable processes. In order to promote such kind of reasoning, teachers and students need instructional scaffolds that enable students actively engage in critical evaluation of claims and evidence (Lombardi, Sinatra and Nussbaum 2013; Sinatra and Chinn 2011).

Critical evaluation of information

We live in a period when we are exposed to abundant information flows than ever before throughout history (Rizeq et al. 2021). In the past, the lack of information was thought to be the biggest barrier to science acceptance. However, easier access to information and having more information did not increase the public appreciation of science (Lobato and Zimmerman 2019) because science appreciation requires more than just more knowledge. There is a need for individuals who can identify and critically evaluate the high-quality information in this intensive flow of information, i.e., acquisition of scientific literacy.

Scientific literate citizens make informed decisions about health issues mostly through a thorough understanding of the world around them, critically analyze the claims and arguments, identify questions and draw conclusions based on evidence, and consequently make informed decisions about health issues (Rennie et al. 2001). Besides, science literate citizens sometimes may need to make decisions like scientists who make science with theoretical calculations and thought experiments. However, it is necessary to make a distinction between different versions of scientific literacy: Vision I, Vision II and Vision III. Vision I refers to the products and processes of science, while Vision II focuses on situations that learners encounter as citizens associate with the interrelationships of science and society, interrelationships of science and humanities, as well as the differences between science and technology (Roberts 2007). On the other hand, a more critically oriented form of Vision II, namely Vision III, aims at emancipation and societal participation and implies a politicized science education, including the aspects like socio- and eco-justice (Sjöström et al. 2017). This study took the position of Vision III by aiming at investigating students’ evaluation of scientific and fallacious claims about vaccination that might have arisen from their values, worldviews, and politicization.

Misinformation, disinformation and uncertainty in science

As we witness in the pandemic era, it became more difficult for people to evaluate claims and evidence because of the rapid spread of misinformation and disinformation about health issues in public. Thus, scientific literacy involves not only critical evaluation of claims and evidence, but also ability to dealing with uncertainty in science (Saribas and Cetinkaya 2021). The COVID-19 pandemic demonstrates that uncertainty is inherent in science, and it is inevitable and essential. Although most of our scientific understanding is robust, there are uncertainties in the details (Kampourakis 2018). While studies on the etiology of the novel coronavirus at the beginning of the pandemic showed that this virus jumped from a wild animal to a human, a certain judgment could not be reached about the type of this animal. Similarly, although vaccines developed against the disease are safe and effective, they do not develop immunity with 100% certainty. However, the citizens are exposed to various forms of information including scientific information, as well as disinformation and misinformation, and may not always make informed decisions. Because of limited scientific knowledge among public and the uncertainty of the novel coronavirus pandemic, the people cannot meet their need of certain information and advice. This situation sets the stage of fake news, unchecked claims and fallacious statements (Nguyen and Catalan-Matamoros 2020).

Considering this theoretical background, educators should protect public from post-truthism by creating scientific literate citizens who understands the uncertainty in science (Kienhues, Jucks and Bromme 2020). In order to achieve this aim, it may be suggested to examine students’ evaluation of various claims about vaccination. Such kind of exploration may create an opportunity to restructure science courses from very early grades to prevent students from accepting misinformation and disinformation.

Using controversies in science education

People are prone to accept misinformation and disinformation because of more than one reason. Cognitive biases are one of the reasons that people believe in information disorders about controversial issues (Bronstein et al. 2020). People develop cognitive filters for concepts they encounter for the first time, and these concepts help to confirm later similar concepts and make counterexamples harder to spot (Höttecke and Allchin 2020). The information disorders that occur in the form of confirming information that fits one's existing beliefs and expectations are called confirmation bias (Mercier 2017). Sarit Barzilai and Clark Chinn (2020) suggest that in order to overcome the problems, which arise from cognitive bias, education might sometimes be use as a tool by teaching how to cope with cognitive biases. The claims on controversies might provide an educational opportunity to explore and deal with cognitive biases. Another reason is that scientific documents use a probabilistic terminology, while anti-scientific documents use a certain language (Lobato and Zimmerman 2019). Individuals, on the other hand, tend to accept claims containing certainty if they have limited knowledge about controversial issues. While uncertainty in science is actually an important factor in advancing science (Kampourakis 2018), it is often brought up to question the credibility of science and to defense anti-scientific claims. (Oreskes 2015).

It is vital to critically evaluate claims and understand the uncertainty, which is intrinsic to science, especially on health-related issues, as individuals use these patterns when making their decisions. Mats Lundström, Margareta Ekborg and Malin Ideland (2012) stated that despite free vaccination offers during the swine flu outbreak in Sweden by government, an important part of Swedish people decided to not vaccinate themselves. According to the researchers, misinformation containing speculation and rumors that spread on social media was responsible for individuals to make this decision. Aviv Sharon and Ayelet Baram-Tsabari (2020) highlighted that scientific literacy is effective to identify such kind of misinformation. The researchers argued the analysis of pro-vaccination and anti-vaccination evidence texts including evaluation as an effective context for the promotion of scientific literacy.

Method

Participants

The participants of this study were 29 eighth grade students (15 males and 14 females) who study in the same class in a public middle school in Turkey. This school was located in a suburban area of a metropole, and the participants’ parents were mostly from lower middle class. According to Dennis Liu (2012), science teachers encounter a wide range of science denial in their classrooms. Based on the science denial that reaches young ages, students should evaluate the claims and evidence on controversial issues. Considering that controversial issues are included in the school curriculum of many countries (Oulton et al. 2004), it is clear that it is necessary to study with young students. Compulsory education system in Turkey is 12 grades in three parts consisting of four years each. It is called primary school from the first grade to the fourth grade, middle school from the fifth grade to the eighth grade, and high school from the ninth grade to the twelfth grade. Students are usually complete the mentioned four-year parts together in the same class. Science curriculum published by Ministry of National Education in Turkey (2018) suggests that vaccination, gene therapy, gene transferring, and cloning should be emphasized on eighth grade. Eighth graders have also basic skills about classification of living things, cell, reproduction, DNA and genetics as the prerequisites of vaccination. In addition, none of the participants in the present study had prior implementation related to the argumentation, critical thinking or claim and evidence structure. Thus, this study was conducted with the eighth graders.

Procedure

The study presented in this paper is a case study in order to achieve a deep understanding in which eighth grade students’ evaluation of fallacies about vaccination. A case study is an empirical inquiry that investigates a contemporary phenomenon within its real-life context, especially when the boundaries between phenomenon and context are not clearly evident (Yin 2008). For this study, the case is defined as the evaluation of fallacies about vaccination. The case study helped the researchers to understand the participant group and individual students’ evaluation of fallacies and their abilities to employ a relationship between claim and evidence. In recent years, vaccine rejection rates rising rapidly around Turkey as well as the world. Individuals who display vaccine refusal mostly live in the development regions and have high income and high educational levels (Özceylan, Toprak and Esen 2020). There are various views on the underlying reasons of vaccine refusal. While some individuals advocate vaccine rejection in a political context as a rebellion against the status quo, others advocate vaccine rejection in a social context that will identify with community belonging (Sobo 2016). In addition, vaccine rejection can also stem from public misunderstanding (or ignorance) of science (Goldenberg 2016) and lack of critical thinking on controversial issues. The researchers examined students’ evaluation of fallacies about vaccination by implementing group discussion and individual reflection of claims and evidence throughout for a total 16 h, 4 weeks, 4 h per weeks. This instruction included the presentations of the claims, distribution of the evidence texts, group discussions on the relationships between claims and evidence and finally individual reflection on each fallacy. The first author was the teacher of these students. He began lesson by listing the fallacies and concluded by presenting the scientific knowledge by critically examining each fallacy based on the protective function of vaccines as well as the trustworthiness of the sources of claims and evidence.

The lesson started with listing and discussing fallacies in the class. First, the types of fallacies listed in Table 1 including the following anti-vaccine claims were obtained from various mass media and listed on the board:

1. Vaccines may cause autism.

2. Vaccines cannot protect people against epidemic diseases.

3. Children’s immune system can be overloaded if the child receives multiple vaccines at once.

4. Diseases had already begun to disappear before vaccines are introduced.

5. The majority of people who get disease have been vaccinated.

6. Big medicine companies are marketing vaccines exclusively for their commercial interests rather than public health.

7. Since diseases like polio have disappeared from our country, it’s no longer necessary to vaccinate children against them.

8. Vaccines contain toxins.

9. Vaccine is a Western conspiracy.

10. We have to respect parents’ choice not to vaccinate their children.

Following the introduction of these fallacies, the teacher presented six evidence texts to the students, three of which contained anti-vaccine claims and three of which contained pro-vaccine claims. In the second part of the research, the teacher distributed evidence texts containing the anti-vaccine and pro-vaccine claims that the researchers previously obtained from various mass media such as newspapers, blogs and social media to the students. He asked them to read these texts. Sufficient time was given to read each text, and after students completed reading these texts, the teacher asked students to form groups of 4 and evaluate each text of evidence by a group task of evaluation which is presented in Table 3 in the context of the 10 fallacies previously presented. Since the evaluation skills of the students were the subject of the research rather than the academic achievements, student choice grouping style was used to group formation. The group members discussed the evidence texts given to them during the group evaluation, and they benefited from evidence texts during discussion while establishing the relationship between claim and evidence regarding each fallacy. The researchers of this study analyzed participant groups’ evaluation of claims and evidence about vaccination by using claim–evidence task, which they filled during the group discussion.

Table 3.

Group task of evaluation of claims and evidence

After completing these tasks, the teacher asked the students to evaluate each claim individually on an individual reflection task, which is presented in Table 4. In group task, they evaluated each claim and evidence. Differently from group task, individual reflections included the students’ individual judgments on each fallacy.

Table 4.

Individual reflection task

The controversies such as vaccines were used as a context to enable students critically evaluate each fallacy and evidence. However, in order to facilitate the evaluation of links between claims and evidence the students not only read evidence texts, but also discussed each of them in the whole class; afterward, the teacher introduced them scientific information about the vaccines by relating each piece of evidence and emphasizing the credibility of the sources. The students discussed credibility of sources by depending on the expertise of the author of the source and data gathered from research studies considering the functions of vaccines that they learned in sixth grade. At this point, the functions and the trustworthiness of the claims were emphasized rather than a deeper analysis of vaccines including side effects or correcting students’ misconceptions.

Data analysis

The researchers adapted a rubric developed by Doug Lombardi, Carol Brandt, Elliot Bickel and Colin Burg (2016) to analyze both participant groups’ evaluations of claims and evidence and individual evaluations of fallacies. Lombardi et al. (2016) asked seventh grade students to explain the connections between evidence and alternative explanations of the controversial topic, namely climate change, and determined four distinct categories of student evaluations of alternative explanations. These levels are (a) erroneous evaluation, (b) descriptive evaluation, (c) relational evaluation, and (d) critical evaluation. Erroneous evaluation refers to an explanation that includes incorrect connection between evidence and explanations. Descriptive evaluation addresses to an explanation involving a correct relationship, but lack of elaboration or correct interpretation of evidence without stating a relationship. Relational evaluation refers to an explanation that addresses text similarities and contains both specific evidence and a related explanation or reference to a model. Lastly, an explanation is considered as critical evaluation when it describes a causal relationship and/or meaning of a specific link between evidence and explanation. They analyzed students’ evaluations by using the rubric they developed based on this categorization.

The researchers of the study presented here analyzed students’ group and individual evaluations of evidence texts by using an adapted version of the rubric that Lombardi et al. (2016) developed. Differently from Lombardi et al. (2016), the researchers of this study explored not only the links between claims and evidence that students created, but also their evaluation of claims and evidence itself. In the adapted rubric, erroneous evaluation refers to an explanation that does not include any evidence at all or includes only some part of information presented in the text without considering the main idea of the text. Another different part of the adapted rubric from the original one is critical evaluation level. The critical evaluation level in the adapted rubric not only referred to the causal relationships between evidence and claims that students made, but also the use of rebuttal and mentioning the weaknesses of claims which they evaluated. Descriptive and relational evaluation levels of the adapted rubric are similar to those in the original one.

Lombardi et al. (2016) developed this rubric to analyze students’ evaluations of model and evidence in the MEL diagram including models and evidence. However, the study presented here did not aim to use such a diagram. Instead, the researchers aimed to probe into students’ evaluations of both claims and evidence. The teachers asked the students to decide whether evidence text supports, contradicts, or have nothing to do with each claim. The adapted rubric to analyze students’ evaluation level is presented in Table 2.

Table 2.

The students’ evaluation levels of claims and evidence

Category	Description
Erroneous evaluation	The explanation did not contain evidence at all, or it is inconsistent with the scientific knowledge
Descriptive evaluation	The explanation includes a correct judgment on the claim; however, it lacks elaboration. It does not clearly distinguish between evidence and claim or does not provide sufficient evidence
Relational evaluation	The explanation indicates a correct relation between claim and evidence; however, it seems to be addresses merely text similarities in evidence texts
Critical evaluation	The explanation correctly describes a relationship between claim and evidence by reflecting a deeper cognitive processing that elaborates on an evaluation of claim and evidence

Data sources

Claim–evidence link task

The researchers of this study analyzed students’ tasks that they filled in the phase of small group discussions during which they decided whether evidence text supports, contradicts or have nothing to do with each claim. These relationships that the students decided indicated their ability to relate claims and evidence about vaccination. Table 3 illustrates groups tasks of evaluation of each claim and evidence texts.

Individual reflections on claims

The students also reflected their individual evaluations on each fallacy in the phase of reflections on each fallacy. They explained whether they agree on each claim and justified their reasons. Individual reflection task is shown in Table 4.

Results

The eighth graders’ evaluation of fallacies about vaccination was examined throughout a 4-week implementation. The students evaluated each evidence text by relating them to each fallacy both in groups and individually. In this context, this section includes the students’ evaluation level of claims and evidence about vaccination and participant groups’ evaluation.

Participant groups’ evaluation level of claim and evidence

The researchers analyzed the group evaluations of claims and evidence about vaccination by using claim–evidence task which they filled during the group discussion. Table 5 illustrates a variety of samples of evaluation level of groups' evaluations of connections between each claim and evidence texts.

Table 5.

Group evaluations of connections between claims and evidence

Category	Evaluation
Erroneous evaluation	“Vaccines can spread infectious disease to children” “Everyone is the man of pharmaceutical companies, from doctors who advocate the vaccines to public health units of countries. They take bribes. They rub their hands to become richer even that babies die or become crippled” “Toxins like mercury slow the brains of children”
Descriptive evaluation	“According a study with 650,000 children, it was shown that the vaccine does not cause autism” “People who are not vaccinated infect the other people” “Vaccines save lives”
Relational evaluation	“Even in studies which aim to show that thimerosal substance in vaccines causes autism, it was found that vaccines do not cause autism” “The anti-vaccine advocators see the doctors who defend the vaccines as murderers, psychopaths and scrooges, and they have no evidence for this. Pharmaceutical companies profit from vaccines, but their profits do not change the fact that they produce the products that are beneficial to health” “Today, thanks to vaccines, approximately 3 million people per year are protected from death, and much more from tuberculosis, polio, diphtheria, pertussis, meningitis, or chronic illnesses”
Critical evaluation	“People have an immune system from birth. Babies and toddlers meet with 2000–6000 new antigens while playing games every day. This number is much higher than about 150 antigens given throughout the entire vaccination schedule. In other words, vaccines given to babies are not too much for them”

The explanations that did not mention the information in the evidence texts or the explanations that ignored the focus of the evidence text regarding the claim and mentioned only limited part of the information in these texts were categorized as erroneous evaluation. According to Table 5, a group stated that vaccines do not protect children against infectious diseases; on the contrary, vaccines can spread infectious diseases to children. One group claimed that vaccine supporters were agents of pharmaceutical companies, while another group claimed that toxins slow down children's brain functions. The evidence texts do not include the claims mentioned by the groups or the information that implies these claims. Although evidence texts include the information that a mercury compound, namely thimerosal (not mercury element), is used as a preservative in vaccines, students ignored this information. The groups committed ad hominem fallacies through attack against the personalities of the vaccine supporters by bringing up various conspiracy theories and claimed that this information is not based on any evidence.

The evaluations that create a coherent relationship between the claim and the evidence but did not share any details or shared unrelated examples were considered as descriptive evaluations. As shown in Table 5, the groups stated that vaccines save lives, unvaccinated people infect other people, and a study with a large sample revealed that there is no link between vaccine and autism. The groups benefited the information in the evidence texts during judging the claims, but mostly did not provide any details.

The evaluations that established a consistent relationship between the claims and the evidence and also referred to the evidence texts accepted as relational evaluation. The references to the explanations in the evidence texts in relational evaluation were generally based on text similarities and far from establishing a high-level of evaluation of this relationship. It is evident from Table 5 that the groups emphasized that thanks to vaccines millions of people were protected from various diseases and deaths in today’s world, and that pharmaceutical companies are profit organizations. However, the final aim of these organizations is saving lives and serving public health. The examples of this level correctly explained the commercial interests of companies in the evidence texts and relate this claim to the evidence, but this explanation lacked the critical evaluation of main aim of these organizations.

The explanations that the groups constituted a coherent relationship between the claims and the evidence, detailed this relationship, and referred to the evidence texts by establishing a causal relationship between claim and evidence were considered as critical evaluation. This level of evaluation includes rebuttals and weaknesses of the claims rather than text similarity. As illustrated in Table 5, one group referred to the functioning mechanism of the immune system by using evidence text and compared the number of antigens encountered in daily life with the immunization schedule in order to refute the claim that the children’s immune system can be overloaded if the child receives multiple vaccines at once.

The researchers analyzed the frequency of the participant groups’ evaluation levels of each claim and the evidence text as illustrated in Table 6. It is evident from this table that the groups rarely made critical evaluation of claims and evidence. They evaluated the claims and evidence mostly in descriptive level.

Table 6.

The frequency of participant groups’ evaluation levels of claims and evidence

	Erroneous evaluation		Descriptive evaluation		Relational evaluation		Critical evaluation
	f	%	f	%	f	%	f	%
Group 1	23	38.33	36	60.00	1	1.66	–	–
Group 2	15	25.00	37	61.66	7	11.66	1	1.66
Group 3	16	26.66	36	60.00	8	13.33	–	–
Group 4	14	23.33	36	60.00	10	15.00	–	–
Group 5	20	33.33	35	58.33	5	8.33	–	–
Group 6	23	38.33	32	53.33	5	8.33	–	–
Group 7	14	23.33	37	61.66	9	15.00	–	–
Group 8	18	30.00	34	56.66	8	13.33	–	–

As shown in Table 6, only one participant group critically evaluated only one of the links between claims and evidence. Relational evaluations were also found rarely in the participant groups’ explanations. The number of erroneous level of evaluations was relatively high. It is evident from this result that the students in the participant groups were challenged to evaluate claims and evidence not only critically, but also correctly. Even if they made a correct judgment on a claim, they mostly did not clearly demarcate between evidence and claim or did not provide sufficient evidence. They also had problems to make a correct judgment at least one fourth of their evaluations.

Individual evaluations of claim

The researchers analyzed students’ individual evaluations of each claim based on given evidence texts by using an adapted rubric. Table 7 shows examples of students’ evaluations for each category.

Table 7.

Examples of the students’ individual evaluations of claims for each level

Category	Description
Erroneous evaluation	“Vaccines can cause autism for some reason” “Poor people are not vaccinated, and they die”
Descriptive evaluation	“I think vaccination protect children against diseases” “The children who are not vaccinated can have serious epidemic diseases”
Relational evaluation	“The claim that vaccines are Western conspiracy is not true. The people in Western countries are also vaccinated” “If vaccines contain toxins why do people who are vaccinated are still alive and vaccines still protect people against diseases?”
Critical evaluation	“The children are exposed to 2000–6000 antigens during being fed or crawling. The number of antigens in vaccines is much less than that” “The people who are ignorant about the content of vaccines are afraid of them. So, they produced these conspiracy theories. If vaccines did not exist, much more people would have died because of epidemic diseases. Most of the diseases eradicated thanks to vaccines”

The erroneous explanation included the explanations that the students never mentioned the information presented as evidence or that they relate the claim to only a part of the information mentioned in the text and overlooked the main idea of the text. According to Table 7, a student claims that vaccines can cause autism for some reason. Although the student was provided with information about the relationship between vaccines and autism in the evidence texts, the student never mentioned this information. In addition, the aforementioned student could not make an appropriate inference using the scientific explanations and information in the evidence texts and reached a non-scientific conclusion. The fact that students made different choices on the same subject despite having sufficient knowledge to make inferences suggests that it is due to their socio-cultural or cognitive differences. For these reasons, this student’s explanation was accepted as erroneous evaluation.

The explanations where a consistent relationship was established between the claim and the evidence but given no details or given irrelevant examples were considered as descriptive evaluation. As shown in Table 7, one student stated that children who were not vaccinated would have serious epidemic diseases, while the other student stated he thought that vaccines protect children against diseases. These students used the information which contained in the evidence texts while forming their claims. However, the relationship between the students' claims and the evidence is not clearly distinct. The students who made descriptive evaluation did not make any further references to evidence text to justify their claims and did not establish a causal relationship between the claim and the evidence.

The relational evaluation consisted of the explanations that establish a consistent relationship between the claim and the evidence, and which referred to the evidence texts but explanations based only on the text similarities and did not establish a causal relationship. As indicated in Table 7, a student rejected the claim that vaccines are a Western conspiracy, by arguing that people in Western countries are also being vaccinated. This level of evaluation used the evidence texts to support the claim and the student not only provided an appropriate explanation of the fallacy, but also established a relationship between the claim and the evidence.

The explanations in which the students establish a consistent relationship between the claim and the evidence, elaborate this relationship, refer to the evidence texts, and sometimes use rebuttals by examining alternative claims, were accepted as critical evaluation. According to Table 7, one student not only stated that vaccines protect against epidemic diseases, but also emphasized that much more people would have died in the absence of vaccines. This student also evaluated critically that people who believe in such fallacy are not scientifically literate, they are ignorant about the content of vaccines, and they produce these complex conspiracy theories. This student made a deep reflection of the claims and evidence about vaccines by presenting a holistic picture of conspiracy theory about vaccines.

The researchers analyzed the frequency of the students’ individual evaluation levels of each claim based on their own judgment by using an adapted rubric. Table 8 illustrates the frequency of students’ evaluation level in each claim.

Table 8.

The frequency of individual students’ evaluation levels for each claim

	Erroneous evaluation		Descriptive evaluation		Relational evaluation		Critical evaluation
	f	%	f	%	f	%	f	%
Vaccines may cause autism	17	58.62	9	31.03	3	10.34	–
Vaccines cannot protect people against epidemic diseases	8	27.58	16	55.17	4	13.79	1	3.44
Children’s immune system can be overloaded if the child receives multiple vaccines at once	10	34.48	11	37.93	5	17.24	3	10.34
Diseases had already begun to disappear before vaccines are introduced	8	27.58	14	48.27	6	20.68	1	3.44
The majority of people who get disease have been vaccinated	7	24.13	15	51.72	7	24.13	–
Big medicine companies are marketing vaccines exclusively for their commercial interests rather than public health	20	68.96	6	20.68	2	6.89	1	3.44
Since diseases like polio have disappeared from our country, it’s no longer necessary to vaccinate children against them	4	13.79	17	58.62	8	27.58	–
Vaccines contain toxins	14	48.27	9	31.03	5	17.24	1	3.44
Vaccine is a Western conspiracy	10	34.48	8	27.58	8	27.58	3	10.34
We have to respect parents’ choice not to vaccinate their children	10	34.48	6	20.68	11	37.93	2	6.89

Table 8 indicates that students evaluated claims mostly in erroneous, descriptive and relational evaluation levels, however, rarely in critical evaluation level. As shown in Table 8, students substantially evaluated links between claims and evidence either erroneous or descriptive level. Furthermore, we found that they made erroneous and descriptive evaluation more than relational and critical evaluation for almost all the listed claims.

It is evident from Table 8 that more than half of the students have not been able to correctly evaluate the claims that “Vaccines may cause autism” and “Big medicine companies are marketing vaccines exclusively for their commercial interests rather than public health”. This result can be shown as evidence that students are challenged to establish an appropriate relationship between the claim and the evidence, based on the nature of the fallacy in the claim. In addition, students are unable to make critical evaluations about the following claims: “Vaccines may cause autism”, “The majority of people who get disease have been vaccinated” and “Since diseases like polio have disappeared from our country, it’s no longer necessary to vaccinate children against them”. It is also remarkable to note that a significant number of students were able to evaluate the claims that “Vaccine is a Western conspiracy” and that “We have to respect parents’ choice not to vaccinate their children” correctly, but usually in relational level (frequencies of 8 and 11, respectively).

Discussions and conclusions

The study presented here investigated eighth graders’ evaluation of fallacious claims about vaccination by implementing group discussion and individual reflection of claims and evidence in science class. The claim–evidence link task that the students filled both in groups and individually revealed that they evaluated the relationship between claims and evidence about vaccination mostly at the descriptive level. These tasks also showed that considerable number of their evaluation of claims and evidence was also incorrect. They also seemed to have been challenged even to distinguish claim and evidence clearly. Their individual evaluations of each claim and evidence were rarely at the critical level. This result is consistent with those of the earlier studies that indicate pre-service elementary teachers evaluated model-evidence relationships mostly at the descriptive and relational level and rarely at the critical level (Saribas and Akdemir 2019; Can and Saribas 2019). Since the grade level of the participants of this study was much lower than those of these earlier works, it was not surprising to get this result. One can conclude from this result that science educators and curriculum developers need to make deliberate attempts to enable middle school students evaluate claims and evidence in various science topics, especially controversial issues. In order to examine students’ evaluation of fallacious claims and evidence deeper, it is also necessary to explore their ability to build a consistent link between claim and evidence and deal with disinformation and misinformation. Considering that scientists do not only benefit from evidence while doing science and producing information, it is also necessary to bring to light the social and cultural factors that affect students' evaluation of fallacious claims.

Building a consistent link between claims and evidence

In general, the students have difficulties to make accurate judgement in general and few of the students can establish a high level of relationship between the claims and evidence. Students’ evaluation levels varied depending on the content of the claim. Although students had previous instruction in vaccination and immunity and they were given evidence texts with enough information about effect mechanism of vaccines as well as social and individual benefits of vaccination during the research, the majority of them were still prone to adopt anti-vaccination claims and did not provide adequate explanation for these claims. Some of them ignored the claims in the evidence texts during evaluating the claims they were given either. Consistent with the findings of this study, Yuchen Shi (2020) reported that students failed to build a coherent relationship between claim and evidence in the previous studies. On the other hand, Doug Lombardi, Elliot Bickel, Janelle Bailey and Shondricka Burrell (2018) argued that the activities in which students build connections between claims and evidence may develop their critical thinking skills by facilitating evaluation of explanations based on evidence. From this point of view, the current study is promising to promote middle school students’ critical thinking skills.

One of the possible reasons of students’ lack of competence to build a consistent relationship between claim and evidence is their inability to resolve uncertainty in science. Although uncertainty is intrinsic to science and is one of the significant elements that make science advance, students expect certain answers from science (Kampourakis 2018). For this reason, we argue that students prone to accept the claims that use certain expressions in evaluation process as true. Shiang-Yao Liu, Chuan-Shun Lin and Chin-Chung Tsai (2011) reported that students who understand uncertainty are more likely to recognize and accept unmanipulated evidence on controversies, even if it contradicts their own beliefs. From this point of view, understanding of science necessitates providing students to understand uncertainty in science (Kampourakis and McCain 2019). We argue that understanding uncertainty in science may enable them to evaluate claims more critically as well as build a consistent relationship between claims and evidence. On the other hand, scientists do not function in socio-political isolation and may be affected by cultural, social and economic factors. It is important for students to know factors that affect scientists and to understand the conflict interest between the scientific and industrial community. For mentioned reason, we suggest that further instruction attempts should also consider the reflections of neoliberal policies on education.

Dealing with disinformation and misinformation

The findings of this study revealed that Turkish middle school students were unable to detect some specific disinformation and misinformation in the evidence texts. For instance, neither of them was able to critically evaluate the claims that vaccines cause autism and vaccines are unnecessary. Although the students were taught about the vaccination and immune system in the sixth grade, they seem to have been challenged to detect the irrelevance of vaccination used for epidemic diseases and disorders such as autism. The erroneous evaluation of the claim that big medicine companies are marketing vaccines exclusively for their commercial interests rather than public health indicated their inability to deal with information disorders. Doug Lombardi, Benjamin Heddy and Ananya Matewos (2020) argued that values, beliefs and attitudes are integral to the learning process and multiple representations and multiple perspectives mediate the relations between values, attitudes and learning. They advocated that explicit learning of values by using scaffolds allowing students to engage in the evaluation of the links between lines of evidence and alternative models with multiple representations (e.g., different forms of evidentiary data) and multiple perspectives on the topic may influence students’ attitudes toward the utility of science and belief in the role of critical evaluation to solve the problems. From this point of view, it can be inferred that educators should create learning opportunities allowing students evaluate their own values, attitudes, and beliefs by using scaffolds enabling them critically evaluate claims and evidence to help them deal with information disorders.

Limitations and implications

It is necessary to create critical thinkers who critically evaluate claims and evidence when they are exposed to an information in this post-truth era, in which people spread fallacies rapidly. The study presented here depicts middle school students’ challenges during evaluating claims and evidence about vaccines. In the pandemic era that we are living, people need to critically evaluate information about health issues more than ever, because, with the spread of the post-truth movement and the acceleration of neoliberal policies in recent years, various health crises have been experienced due to the effects of powerful groups and individuals on health-related scientists/researchers and engineers. Such pressures have led many deaths from the overuse of opioid drugs. Similarly, the novel coronavirus vaccines produced as a result of medical research supported by public funds are sold to public authorities for tens of times more than their cost as a result of neoliberal policies. There is a necessity for individuals to evaluate such information and to relate claims and evidence to such information. Therefore, science educators and curriculum makers need to find ways to improve students’ ability to evaluate claims and evidence.

Jasmyne Sanderson and Ullrich Ecker (2020) argued that exposure to misleading and inaccurate information become more common with the pervasion of internet and social media throughout the world, and the quality and accuracy of information cannot be ensured enough. Moreover, in recent years, claims of the use of big data by data-driven approach with support of powerful people, foundations and political figures manipulating the users have become widespread. The Cambridge Analytica scandal, in which millions of Facebook user data were used and political preferences regarding the 2016 US presidential elections and the Brexit referendum were manipulated over social media (Bömelburg and Gassman 2021), can be given as an example of information manipulation carried out by the influence of the powerful people. Jasmyne Sanderson and Ullrich Ecker (2020) suggested improving individuals’ critical thinking skills and creating healthy skepticism. According to these researchers, an important aspect of information literacy is source evaluation and available technologies, such as fake news-detecting sites, must be used to avoid misinformation on the Internet. They also argued that critical thinking is not considered independently from factual knowledge and people need to acquire subject-relevant knowledge in order to evaluate new information critically. We argue here that it is crucial to create scientific literate citizens who critically evaluate information by relating claims and evidence and questioning the credibility of sources of evidence from very early grades. The results of the present study offer insight into how students can acquire new knowledge by reasoning on claims about a controversial topic and evaluate these claims based on evidence. Further studies explore the development of students’ scientific knowledge through discussions on controversial topics.

A possible way to promote students’ ability to evaluate information critically as well as their subject-relevant knowledge about vaccines is discussing both scientific information and fallacies about vaccines by evaluating claims and evidence as well as introducing content knowledge about the functions and protective feature of vaccines. This study presents a descriptive study which shows middle school students’ evaluation of fallacies by using students’ group discussions and individual reflections. Suggesting a method to eliminate misinformation and foster students’ critical evaluation is beyond the aims of this study. This is the main limitation of this study. Further research implementing an intervention and investigating middle school students’ evaluation skills on a controversial issue at the end of this intervention may bring new light to this issue. Chin and Osborne (2010) suggested supporting argumentation and conceptual change through student—generated questions. The results of the current study also suggests further investigations of students’ argumentation skills by encouraging them generate their own questions and create arguments on a controversial topic based on evidence in a discourse environment.

Controversies like vaccines require individuals make informed decisions based on scientific evidence by evaluating each claim and evidence. Lee Yeng Hong and Corrienna Abu Talib (2018) reported that scientific argumentation process involves the students to use critical thinking skills to make informed decisions. Discussing fallacies in a classroom practice in the context of a controversial issue may be helpful to enable students to make informed decisions as well as critical evaluations. This study aimed at exploring students’ evaluation of fallacies on vaccines. Investigating and promoting their decision-making skills was not the scope of this study. Further research including such investigations seems necessary. On the other hand, considering the point that technology has reached, we suggest a helpful education about the nature of science, technology and engineering, including the effects of powerful people and groups who play a role in processing big data and guiding people's choices.

The present study focuses on the evaluation of fallacious claims about vaccination, which is a controversial issue containing the uncertainty in science by middle school students. Pablo Antonio Archila et al. (2020) emphasized that controversial issues are significant pedagogical resources in order to enhance students' engagement to both small- and whole-class discussions. In addition, Ladislao Salmerón, Agnese Sampietro and Pablo Delgado (2020) argued that using controversial topics which are presented from different sources in classroom practice can improve students' evaluations and the use of sources of information. In this respect, it is clear that the discussion of vaccination, which is a controversial issue in the classroom, will improve students' evaluations and their thoughts about the use of sources. This research is limited to the vaccination in general as a controversial issue for students to evaluate the fallacies. We suggest further research including a specific context, such as vaccination against COVID-19 pandemic broaden our perspective associated with students’ mentioned skills.

The current study based on the Vision II of scientific literacy in which usefulness and meaningfulness of science are emphasized to prepare citizens for the technological society. From this point of view, this study focused on the students’ critical evaluation of claims to deal with uncertainty in science. However, Sjöström et al. (2017) went one step further and focused on emancipation and called it Vision III. This vision of scientific literacy emphasizes science for individual and societal actions and transformation by which a politicized science education is implied. Such values-driven science education aims at emancipation and societal participation including socio- and eco-justice. These researchers suggested to incorporating values, worldviews, politicizations, as well as actions into science and environmental education. Larry Bencze et al. (2020) suggested to provide students with insights from the three “SinC” (science-in-context) fields, i.e., science technology, society and environment (STSE) relationships, Socially-Acute Questions (SAQ—which deal with controversies that have high levels of public consciousness and debate) and socio-scientific issues. The values and acute questions were not the scope of this study. Overcoming the limitations of this study, further researches examining students’ decision-making skills including controversies and uncertain issues that are acute may bring new insight to this issue.

The data source used in this study was students’ evaluation task of claims and evidence. The students completed this task in groups and individually. All students must contribute to achieve prominent gains in the group work (Chang and Brickman 2018). However, various factors influence participation in group work and all students may not contribute the group works equally and may avoid sharing their own ideas. It is also necessary to examine the views of students who are less visible in group discussions, less involved, and do not contribute to the evaluation process. Although individual reflections enable students to evaluate the relationship between claim and evidence, further analysis by using different data sources, such as interviews, may be beneficial for in-depth analysis of individual students’ ideas.

Overcoming the limitation of this study necessitates examining middle school students’ decision-making skills about vaccines as well as implementing an intervention to promote their evaluation skills. Individual interviews with students as well as students’ small group and whole group discussions may broaden our perspective in terms of science curriculum development.

Biographies

Ertan Cetinkaya

is a teacher who works in a government school in İstanbul, Turkey. He holds a PhD in science education from Gazi University, Ankara, Turkey. His research interests include demarcation problem between science and pseudoscience, argumentation, nature of science, critical thinking and science fairs.

Deniz Saribas

is an associate professor in the Elementary Education Department at Istanbul Aydin University. She has a PhD in science education. Her research interests include pre-service teacher education, critical thinking and critical evaluation and scientific practices.