United States and South Korean citizens’ interpretation and assessment of COVID-19 quantitative data

Abstract

We investigate United States and South Korean citizens’ mathematical schemes and how these schemes supported or hindered their attempts to assess the severity of COVID-19. We selected web and media-based COVID-19 data representations that we hypothesized citizens would interpret differently depending on their mathematical schemes. We included items that we conjectured would be easier or more difficult to interpret with schemes that prior research had reported were more or less productive, respectively. We used the representations during clinical interviews with 25 United States and seven South Korean citizens. We illustrate that citizens’ mathematical schemes (as well as their beliefs) impacted how they assessed the severity of COVID-19. We present vignettes of citizens’ schemes that inhibited interpreting representations of COVID-19 in ways compatible with the displayed quantitative data, schemes that aided them in assessing the severity of COVID-19, and beliefs about the reliability of scientific data that overrode their mathematical conclusions.

1. Introduction

Early in 2020, governments, scientific organizations, and media used quantitative data to make arguments for and against taking disruptive measures to prevent the spread of COVID-19. The data representations varied (e.g., text, graphs, charts, tables) and relied on mathematical concepts including rate of change, comparisons of relative size, exponential growth, probability, accumulation, and mathematical modeling. Mathematics education researchers have created models of how students and teachers understand these representations and concepts (Behr, Harel, Post, & Lesh, 1992; Byerley & Thompson, 2014; Byerley, 2019; Castillo-Garsow, 2013; Ellis, Ozgur, Kulow, Dogan, & Amidon, 2016; Harel & Confrey, 1994; Johnson, 2012; Konold, 1989; Lesh & Lehrer, 2003; Lobato & Siebert, 2002; Moore, Stevens, Paoletti, Hobson, & Liang, 2019; Steffe & Olive, 2009; Steffe, Liss, & Lee, 2014; Thompson, 1994a, 1994b; Thompson, Hatfield, Yoon, Joshua, & Byerley, 2017), but independent of the COVID-19 pandemic and data. These same researchers have argued people’s schemes for these concepts and representations vary in their productivity, but again these claims have been independent of a major event like the COVID-19 pandemic. We thus extend this work to identify the extent these previously reported schemes are productive or unproductive 1 for a citizen interpreting COVID-19 data. Namely, we investigate the question: How do citizens’ mathematics support them in assessing the severity of COVID-19?

As an initial response to this question, we implemented task-based clinical interviews (Ginsburg, 1997; Goldin, 1997) with United States (US) and South Korean (SK) citizens during Spring 2020. Our interview tasks included a variety of COVID-19 data representations from the media that we hypothesized citizens would interpret differently depending on their mathematical schemes. In this manuscript, we narrow our focus to how mathematical schemes for comparing the relative sizes of quantities, rate of change, slope, and graph were either productive or unproductive for citizens’ attempts to interpret COVID-19 data. We explore how people use those schemes in a complex and time-sensitive situation. We also illustrate how citizens’ other knowledge and beliefs about news sources, uncertainty, and the reliability of scientific expertise and data collection interacted with their mathematical schemes as they interpreted COVID-19 data.

2. Background

2.1. Context for COVID-19

The first news about COVID-19 appeared late in 2019 when a cluster of pneumonia cases in the city of Wuhan in the Hubei Province of China were found to be caused by a novel coronavirus (Novel coronavirus situation report -1, 2020, January 21). This disease was designated coronavirus disease 2019 (COVID-19) ("Naming the Coronavirus Disease (COVID-19) and the Virus That Causes It,” 2020). COVID-19 spread swiftly, initially causing an epidemic in China before escalating over several months to a worldwide pandemic (Coronavirus disease (COVID-19) situation report -132, 2020, May 31). The understanding of COVID-19 evolved rapidly in early 2020 as the scientific and medical communities investigated virus and disease characteristics that are critical for personal health decisions and public health policies. This knowledge included information about transmissibility, infectivity, and immunogenicity of the virus as well as risk factors, clinical features, patient management, morbidity, and mortality of the disease.

Social Distancing was widely recommended by governments to limit face-to-face contact by keeping at least six feet of space between non-household members and avoiding gathering in groups (“Social Distancing, Quarantine, & Isolation,” 2020, May 6). Initially, there was no vaccine or effective therapies for COVID-19, so behavior modification was the most effective mitigation strategy. Requests and mandates for people to modify behaviors (or alternatively, arguments telling people to not modify behaviors) were often accompanied by quantitative data. This data, along with prospective epidemiological models, was presented to influence individuals’ choices regarding personal preventative measures. Furthermore, numerous websites were created in order to track and disseminate COVID-19 data to a broad population. In the US, most states relied on voluntary compliance with guidelines and asked individuals to make drastic shifts in their personal lives, professional settings, and social behaviors.

When we designed our study in March 2020, most of the scholarly work about COVID-19 was about epidemiological modeling and medicine. Since then, there has been more work in the areas of COVID-19 data literacy and COVID-19 data visualization that we briefly summarize. For example, Bloom, Fuentes, and Crocker (2020) started an important conversation about how citizens’ understanding of mathematical modeling impacts their perception of COVID-19 models. Bowe, Simmons, and Mattern (2020) wrote a reflective analysis of the wide variety of data visualizations that have been created for and by the public. Juergens (2020) wrote about principals of good design of choropleth maps that make it easier readers to understand the data as intended. Shelton (2020) analyzed examples of intentionally and unintentionally misleading COVID-19 data visualizations produced by governments. Nguyen (2020) wrote about the increasing need for global data literacy and the difficulty of making sense of the vast amount of COVID-19 data representations. Okan et al. (2020) defined health literacy in the context of the COVID-19 info-demic:

Health literacy can facilitate distinguishing between reliable information on COVID-19 and dis and misinformation on the topic, it helps navigating sources of health information and health services, and health literacy empowers people to make informed health decisions and to practice healthy and protective behaviors in the time of the coronavirus and COVID-19 pandemic [9,19, 20]. In general, health literacy is defined as the motivation, knowledge, and competence used to access, understand, appraise, and apply health information and make health-related decisions [23]. (p. 2)

In this study, we contribute empirical evidence to support arguments about the importance of health literacy in the COVID-19 pandemic.

2.2. A perspective on “productive” understandings and background research

The level of a society’s data literacy is known to directly influence people’s health behavior, decisions, and outcomes (Crusoe, 2016; Sun et al., 2013; Twidale, Blake, & Gant, 2013). Data literacy enables individuals to productively interpret, critically assess, and use data to solve everyday problems (Prado & Marzal, 2013). Such abilities are critical in responding to health threatening situations such as the COVID-19 pandemic, as these situations require the public to consume data presented in the media and make decisions regarding their health behavior. Reflecting the importance of data literacy, we define productive understandings of quantitative data to be those schemes that support citizens in interpreting COVID-19 data representations in accurate ways that afford their making informed decisions (Steen, 1999). As we identify in the following sections, productive schemes afford an individual conceiving equivalent mathematical properties across various representations despite differences in those representations (e.g., comparing a cases per day graph with an accumulation graph or comparing graphs using different scales). Productive schemes are generative in their capacity to accommodate to novel data representations due to their foregrounding quantities and their relationships (Ellis, 2007; Liang & Moore, 2020, online; Moore et al., 2019; Smith & Thompson, 2007; Thompson, 2013a).

We define unproductive or less productive understandings of quantitative data to be those schemes that limit citizens’ abilities to interpret data representations of COVID-19 in accurate ways. We define unproductive (or less productive) quantitative schemes as those that increase the liklihood that citizens make decisions that are inconsistent with that citizen’s personal risk tolerance and their personal views about community health. We strive to refrain from judging citizens’ decisions about risks they are willing to accept and instead focus on how they are interpreting the data with respect to its accuracy (or lack thereof). For example, one citizen we spoke to interpreted “stay at home” orders given in large cities to mean that he needed to stay indoors in his smaller town, and he reported missing the sun. His extreme caution with regard to COVID-19 transmission was inconsistent with how he thought about risk in other areas of life. On the other hand, other citizens who take a number of precautions in their lives to avoid risk made the mathematically incongruent decision that COVID-19 was not risky enough to avoid. We note that unproductive (or less productive) schemes for interpreting COVID-19 data can lead to accurate interpretations, but only in limited circumstances when compared to more productive schemes. As illustrated in this section, unproductive (or less productive) schemes are constrained in their applicability, and they restrict an individual’s ability to conceive equivalent mathematical properties across various representations due to either some limitation in the mathematical development of the scheme (e.g., additive versus multiplicative reasoning) or the absence of logico-mathematical operations from the scheme (e.g., indexical associations between slope and shape). Unproductive (or less productive) schemes can also lead to alternative interpretations inconsistent with the represented data, or they can lead to a perturbation and the need for assimilation via a different scheme (Moore, in press; Steffe & Olive, 2009).2

In this section, we discuss past research on students’ or teachers’ schemes used for interpreting quantitative data with an eye toward the extent those schemes may be productive for interpreting COVID-19 data representations. We concentrate the literature review on research on students’ (productive and unproductive) schemes for comparisons of relative size, rate of change, slope, and graph. We adopt Thompson’s definition of scheme “as an organization of actions, operations, images, or schemes—which can have many entry points that trigger action—and anticipations of outcomes of the organization’s activity” (Thompson, Carlson, Byerley, & Hatfield, 2014). For example, a person’s scheme for slope might focus on the steepness of a line or it might entail a multiplicative comparison of changes in quantities’ measures, and these schemes have very different consequences for how a person interprets a graph. Given the diversity of mathematical and social topics in our interviews, we recognize there are other useful analytical tools from mathematics education and other fields. Practically, we chose to focus on the following set of theoretical tools—a set that has proved viable when building models of individuals’ meanings—to collect and analyze data in a timely manner.

2.2.1. Schemes for average rate of change

The concept of average rate of change is an important idea for making sense of and comparing quantities in the COVID-19 pandemic. Discussing a teaching experiment on ideas of the Fundamental Theorem of Calculus with 19 senior and graduate mathematics students, Thompson (1994b) described a productive scheme for average rate of change:

[By “average rate of change”] we typically mean that if a quantity were to grow in measure at a constant rate of change with respect to a uniformly changing quantity, then we would end up with the same amount of change in the dependent quantity as actually occurred. An average speed of 55 km/hr on a trip means that if we were to repeat the trip traveling at a constant rate of 55 km/hr, then we would travel precisely the same amount of distance in precisely the same amount of time as had been the case originally (p. 50).

Thompson attended to the concept of average rate of change explicitly in his teaching experiment, yet he found that many students struggled to develop the scheme he designed the instruction to promote (p. 49).

Based on Thompson’s struggle to aid senior and graduate mathematics students in developing his intended schemes for average rate of change, we suspect that many citizens also struggled to develop these schemes in their own educations. In fact, there is evidence from a medium-sized, but not representative, sample of US secondary mathematics teachers suggesting that many teachers are not prepared by their college professors to teach average rate of change (Yoon, Byerley, & Thompson, 2015). Approximately half of 96 US secondary mathematics teachers revealed productive schemes on two items about average rate of change and in contrast over 90 percent of SK mathematics teachers revealed productive schemes. Many of the US responses involved teachers understanding average rate of change as one quantity that gave information about velocity but did not involve understanding that a velocity is a comparison of two quantities, a change in distance and a change in time. A person with this understanding might take the arithmetic mean of two velocities to find an average velocity even if the car was traveling at those velocities for different periods of time. Understanding an average rate of change as a comparison of changes in two quantities is essential in interpreting many COVID-19 data sources. For example, when considering the severity of COVID-19 it is important to attend to not just overall deaths, but also deaths per unit of time, so that comparisons can be made between various causes of death that are reported in different time intervals.

In addition to the tendency to think of an average rate of change as one quantity, Byerley and Thompson (2014) identified that mathematics teachers have the propensity to make additive comparisons when multiplicative comparisons are more appropriate, a finding that reflects the documented difficulty of developing multiplicative reasoning (Steffe & Olive, 2009). Because the development of multiplicative reasoning is difficult, we hypothesized that many citizens might make additive comparisons of COVID-19 data. For example, we expected some citizens to make additive comparisons when asked to compare the severity of COVID-19 and the flu3 given numbers of confirmed cases and time periods. As an illustration, consider the context of 350 people dying from disease A over 5 months and 500 people dying from disease B over 10 months. A person might additively compare the severity of the two diseases by noting that 500 people is 150 more people than 350 people so disease B is worse. On the other hand, another individual could multiplicatively compare the severity of the two diseases by calculating the average number of total deaths per month (disease A: 350 people/5 months = 70 people per month, disease B: 500 people /10 months = 50 people per month) and conclude that disease A is worse. Additive comparisons often answer questions like “how much more is quantity A than quantity B?” where quantity A and B have the same units of measure. Multiplicative comparisons often answer questions like “how many times as large is quantity C than quantity D?” and the quantities do not have to have the same units of measure. Making a multiplicative comparison of total deaths and the time period the deaths occurred allows citizens to make comparisons between data from different time spans.

2.2.2. Schemes for comparing percentages: part-whole / relative size

Citizens’ schemes for comparing quantities with fractions are important in many COVID-19 contexts in addition to situations about average rate of change. Thompson and Saldanha (2003) argued, “how students understand a concept has important implications for what they can do and learn subsequently” (p. 95). Relatedly, Thompson and Saldanha (2003) provided example of different ways students might understand fractions, the constraints and affordances of different understandings of fractions. Their paper summarized the vast amount of evidence that developing understanding of fractions (and related concepts like rate and percent) as tools to compare the relative sizes of quantities is a difficult goal that many US citizens have not achieved. They also referenced a wide number of other mathematics education researchers that helped them make distinctions among various schemes for fractions including the extensive work by Steffe and Olive (2009). Additionally, Steffe and colleagues modeled the development of children’s ability to make additive and multiplicative comparisons of the sizes of two quantities (Hackenberg & Tillema, 2009; Norton & Boyce, 2015; Steffe & Olive, 2009; Tzur & Hunt, 2015; Ulrich, 2015, 2016).

Relevant to the present manuscript, Thompson and Saldanha (2003) contrasted two schemes for fractions, part-whole and relative size. A common way of expressing a part-whole scheme for a fraction such as 4/5 is to say “4 out of 5.” This scheme is often helpful when the numerator is less than the denominator and when the quantity the numerator represents is a subset of the denominator. For example, it is productive to use a part-whole scheme to say, “today my husband told me that 13 out of his 200 students have tested positive for COVID-19.” In the example, the students in his classes who have tested positive is necessarily a subset of the entire group of students.

There are other situations where thinking of fractions as a comparison of the relative size of two quantities is more productive than the part-whole scheme. Thompson and Saldanha (2003) wrote about the productiveness of students developing relative size schemes for fractions that were strongly connected to quantitative schemes for proportion, multiplication, and division. Consider someone comparing infection fatality rates (IFR) from COVID-19 (let’s assume 0.6 %) to infection fatality rates of the flu (let’s assume 0.1 %), which can be written as 0.6 %/0.1 %.

A person with a part-whole scheme would have difficulty making sense of “0.6 out of 0.1”. This is because a COVID-19 IFR is not a subset of a flu IFR and individuals with a primarily part-whole scheme for fractions often think the numerator (0.6) should be smaller than the denominator (0.1). There are other productive ways to think about the number 0.6/0.1. For example, we can compare 0.6 and 0.1 multiplicatively and say that since it takes six copies of one-tenth to make six-tenths, 0.6 is 6 times as much as 0.1. Thus, the IFR from COVID-19 is six times as large as the IFR from the flu. Steffe and Olive’s (2009) research shows that it is cognitively easier to develop part-whole schemes for fractions and that students typically develop relative size schemes later after they are able to mentally coordinate three different units at one time. To develop a part-whole scheme the student only needs to coordinate the size of the part and the whole simultaneously. Additionally, Izsák, Jacobson, and Bradshaw (2019) surveyed a large random sample of US middle school teachers and found that many struggled with partitioning, iterating and considering referent units; these three mental activities are critical for conceiving 1 as 10 times as large as 0.1, and conceiving 2.1 as 21 times as large as 0.1.

2.2.3. Schemes for graphs and slope: figurative and operative thought

To distinguish various schemes for graphs and slope, we leverage a distinction between figurative thought and operative thought that Moore, Thompson, and colleagues adopted to characterizing students’ graphical meanings (Moore, in press; Moore et al., 2019; Moore & Thompson, 2015). Piaget (2001) developed the figurative and operative thought distinction to differentiate between thought constrained to states such that actions are indissociable from results and thought foregrounding the coordination of internalized mental actions so that results are subordinate to this coordination. Mathematics education researchers, including Thompson (1985) and Steffe (1991) have since adopted this distinction. For example, Steffe and Olive (2009) distinguished between figurative and operative counting schemes. The former requires an individual to carry out acts of counting, while the latter involves an individual having constructed unitized records of counting that they can operate on in thought. These distinctions between counting schemes are useful because citizens’ development of multiplicative reasoning hinges on their construction of operative counting schemes.

Consistent with the distinction between figurative and operative thought, Moore and Thompson (Moore, in press; Moore & Thompson, 2015) introduced static and emergent (graphical) shape thinking as ways to characterize individuals’ graph schemes. Summarizing the two schemes, Moore (in press) explained:

Static graphical shape thinking characterizes actions that involve conceiving a graph as if it is essentially a malleable piece of wire (graph-as-wire). Thompson and I (Moore and Thompson 2015) chose the term static to indicate that a student assimilates a displayed graph so that he predicates his actions on perceptual cues and figurative properties of shape, and imagined transformations are with respect to physically manipulating that shape as if it were a wire (e.g., translating, rotating, or bending).

Emergent graphical shape thinking characterizes a student’s actions that involve conceiving a graph (either perceived or anticipated) simultaneously in terms of what is made (a trace entailing corresponding values) and how it is made (a sustained image of quantities having covaried)…a student assimilates a graph—whether given, recalled, or constructed in the moment—as a trace in progress that is born or derived from images and coordination of covarying quantities. The student conceives the result of this trace to be the emergent correspondence between covarying quantities (Carlson, Jacobs, Coe, Larsen, & Hsu, 2002; Frank, 2017; Saldanha & Thompson, 1998; Thompson et al., 2017).

Importantly, schemes associated with emergent shape thinking (and more broadly operative thought), are more productive and generative than those associated with static shape thinking (and more broadly figurative thought). This is because emergent shape thinking affords conceiving equivalent mathematical properties across graphs that might differ in perceptual or non-mathematical ways, while static shape thinking is constrained to indexical associations constrained to previously carried out actions and graphs (Moore, Paoletti, & Musgrave, 2013, Moore et al., 2019; Thompson, 1994b).

Relevant to the present manuscript, a productive scheme for slope rooted in emergent shape thinking entails attention to axes scaling and forming multiplicative comparisons between changes in the two represented quantities’ values (e.g., comparing the change in COVID-19 cases with the length of some time interval of interest). Such a scheme for slope is productive because it foregrounds the logico-mathematical operations of conceptualizing changes in quantities and (multiplicatively) comparing those changes. These operations enable an individual to understand data represented in different ways including under different scales or different quantitative referents (e.g., cases per day or accumulated cases). A less productive scheme rooted in static shape thinking entails making judgments of slope primarily on perceptual properties of the graph such as steepness. Such a scheme for slope can be unproductive because it relies on indexical associations tied to particular representational features instead of foregrounding logico-mathematical operations including attention to the quantities on the axes (Moore et al., 2019). For example, an individual might perceive a line that looks like “/”as having a slope of two by using a memorized shape-slope association without realizing that such an association is only relevant under particular axes scaling and orientation.

3. Method

Steffe, Thompson, and colleagues’ models of student thinking and their theoretical perspectives referenced above guided our hypotheses generation, interview protocol design, and data analysis. Collectively, these researchers have modeled mathematical schemes for ideas that are critical for understanding COVID-19 data and, because their models have synthesized insights from other researchers, they provided us an operationalizable and sophisticated foundation for our work. In the following sub-sections, we discuss how we operationalized that work in addition to providing the methods and subjects of our study.

3.1. Subjects

We conducted task-based clinical interviews (Ginsburg, 1997; Goldin, 1997) with 25 US citizens and 7 SK citizens between April 2nd, 2020 and May 11th, 2020. Each interview lasted approximately 1.5 h and each interview included one researcher and one citizen. Interviews were recorded using a video communication software (Zoom) and we paid participants 30 dollars per hour. We included SK citizens to capture citizen’s ways of thinking in a country with a different COVID-19 response. SK’s response to COVID-19 focused on extensive testing, contact tracing, and mask wearing (“KCDC public advice & notice,” 2020). SK’s response was significantly different from the actions taken in the US, and SK has been recognized as one of a few countries successfully managing the COVID-19 outbreak (Solano, Maki, Adirim, Shih, & Hennekens, 2020).

We recruited US citizens and SK citizens through email, social networks, and personal connections with the goal of creating a participant pool with diverse opinions, backgrounds, and mathematical schemes. The initial US volunteers mostly believed social distancing measures were appropriate. After initial interviews we thus targeted citizens who disagreed with social distancing measures. We also attempted to recruit volunteers of varying races and education levels. We interviewed people from 12 US states and 2 SK metropolitan areas, with education levels ranging from high school to Doctor of Medicine (14 different college majors), ages from mid-twenties to late-seventies, and several races and ethnicities (White, Black, Indonesian, Thai, SK (see appendix)). Our sample was diverse, not representative. We relied on Pew Research Center to learn about COVID-19 opinions from a representative sample of US citizens. A diverse sample was important because of the relationship between beliefs, education, age, and race and COVID-19 opinions (See Fig. 1 ). Most importantly, the diversity of our sample increased the probability that we would see varied and contrasting ways of thinking.

Fig. 1.

Chart from Pew Research Center comparing opinions of Democrats and Republicans as well as Hispanic, Black, and White Citizens. Republicans tended to be less concerned about COVID-19 than Democrats. (https://www.people-press.org/2020/06/25/republicans-democrats-move-even-further-apart-in-coronavirus-concerns/).

In the beginning of the interview, we asked citizens to tell their “coronavirus story.” We also asked about where they got their information from, what they thought would happen where they lived, how they changed their behavior due to COVID-19, and what their opinions were on social distancing measures. These initial conversations with our participants gave us important understanding of their personal situation and helped us recognize the many non-mathematical factors that play into people’s assessment of the severity of COVID-19. After asking qualitative questions about their COVID-19 experience we asked citizens to respond to the tasks in section 3.2.

3.2. Tasks & theoretical justification for selection

Our final interview protocol consisted of 11 items (see Appendix for complete protocol). Our initial protocol included items that were discarded or modified as we generated hypotheses. We present five items focused on the topics of rate of change, comparing percentages, slope and graph.

3.2.1. “Flu vs. COVID-19 deaths” item, and schemes for rate of change

Our first item (Fig. 2 ) is based on a tweet by President Trump on March 9th (Fig. 3 ).

Fig. 2.

The item, “Flu vs. COVID-19 deaths”.

Fig. 3.

President Trump's tweet about the relative severity of flu and COVID-19.

We removed the President’s name and added additional information about the time period of the flu cases. We updated the data before each interview, so the number of COVID-19 deaths seen by interviewees varied between 5808 deaths and 78,771 deaths. The tweet did not include information about when COVID-19 started in the US and the item we created did not either. We were interested in citizen’s ability to make sense of an argument that was widely read and shared, and the tweet’s statement required citizens to consider the start date of COVID-19 without being told. If a citizen asked when COVID-19 started in the US we told them. We did not update the CDC numbers for flu deaths because the CDC website did not update the numbers during our interviews (CDC, 2020). Importantly, the average rate of change of COVID-19 deaths per week was always higher than the average rate of change of flu deaths per week, so we hypothesized citizens focused on comparing average rates of change would tend to disagree with the argument. We further hypothesized that citizens who considered the exponential growth of a virus in a population with no immunity would tend to disagree with the argument. On the other hand, prior research indicates that even teachers with mathematics degrees make additive comparisons when multiplicative comparisons are more appropriate (Byerley & Thompson, 2014). We hypothesized a citizen who used primarily additive schemes to make sense of the argument might say: “39 million flu illnesses are much more than 789,745 confirmed COVID-19 cases. And 42,186 COVID-19 deaths are a similar number as 24,000–62,000 flu deaths. Therefore, these numbers do not show that COVID-19 is worse than the flu.” If the citizen’s tendency was primarily to make additive comparisons, we hypothesized they would not consider comparing the number of deaths to the time spans, and not consider the percent of cases of each infection that resulted in death.

3.2.2. “Flu vs. COVID-19 rates” item, and schemes for comparisons of relative size

The item “Flu vs. COVID-19 rates” (Fig. 4 ) includes data on infection fatality rates for the flu and COVID-19.

Fig. 4.

The item, “Flu vs. COVID-19 rates” (COVID data from Wu, McCann, Katz, Peltier, & Singh, 2020; Wu, Leung et al., 2020 and flu data from the CDC).

One productive way of comparing the relative severity of infection fatality rates is to note that 0.66 % is 6 times as large as 0.1 % and 2.1 % is 21 times as large as 0.1 %. This comparison supports the conclusion that if an equal number of people are infected by each virus, 6–21 times as many people will die from COVID-194 . Alternatively, we hypothesized some citizens would think that both infection fatality rates were low because in many everyday contexts (such as taxes or giving tips) 2% is considered small. We designed part (b) to prompt citizens to think about the relative size of 0.1 % and 2.1 %.

We hypothesized citizens’ schemes might not support an immediate realization that 2.1 % is 21 times as large as 0.1 %. There are two main lines of evidence supporting this hypothesis. One, there are many large data sets showing that many students answer percent questions incorrectly on standardized tests. For example, The National Assessment of Educational Progress in the US provides evidence that percent is a difficult topic for many. In 1990, only 46 % of a nationally representative sample of 26,000 12th grade students showed “a consistent grasp of seventh grade material (decimals, percent, fractions, simple algebra)” (Mullis, 1991). A little more than half of eleventh graders “have learned basic percent concepts” according to data from the 1986 NAEP (Kouba et al., 1988). For instance, 34 % of eleventh graders were able to solve the question: “9 is what percent of 225?” The second line of evidence suggesting that the comparing percentages is difficult is the models of the development of people’s ability to compare two quantities (Steffe & Olive, 2009). These models predict that calculating that 2.1 is 21 times as large as 0.1 requires a non-trivial coordination of multiple units (tenths and ones). Taken as a whole, the research suggests that making comparisons of relative sizes is difficult to learn and that many citizens in the US did not successfully learn these topics by the end of secondary school.

3.2.3. Graphs items, and schemes for rate of change or slope of a graph

We selected the items “South Korea Cases” and “Three Country Cases” (Fig. 5, Fig. 6 ) to investigate whether a citizen interpreted the shape and slope of the graphs with (i.e., emergent shape thinking) or without (i.e., static shape thinking) considering the quantities on the axes and their measures. We updated graphs from Our World in Data (https://ourworldindata.org/coronavirus) before each interview.

Fig. 5.

The item “South Korea Cases”.

Fig. 6.

The item “Three Country Cases”.

We hypothesized that an individual engaging in static shape thinking would focus on the steepness of the graphs without attention to the measures of quantities on the axes including their scale, and thus an individual engaging in such thinking might fail to recognize that the SK data in the two graphs is the same. Alternatively, they might focus on the steepness of the SK graph (Fig. 5) to infer the COVID-19 situation in SK is severe because of the visual steepness of the graph. Then, when presented with the second graph, we conjectured that they might change how they perceive severity in SK due to the visual flatness of the graph, especially as compared to the other displayed countries. Emergent shape thinking would lead an individual to base their judgments of rate of change or slope primarily on the coordination of quantities’ values and comparisons between those values. We thus hypothesized that an individual engaging in such thinking would maintain attention to axes scaling and coordinate values to conclude that both graphs convey the same rates of change or slope in SK COVID-19 cases.

We also chose “Log Scaled Cases” (Fig. 7 ) from Worldometer to investigate how citizens’ schemes for slope and graph influenced their understanding of COVID-19 data presented on logarithmic scales, since the media had shown several log graphs5 .

Fig. 7.

The item, “Log Scaled Cases”.

If a person’s primary method for estimating rate of change or slope involves attending strictly to the perceptual steepness of the graph (i.e., static shape thinking), they are likely to conceive the log scaled graph as showing that the rate of change of new confirmed cases is decreasing due to the steepness decreasing. If a person’s primary method for estimating rate of change involves attending to axes scaling and conceiving of the graph as showing the covariation of COVID-19 cases and time (i.e., emergent shape thinking), they are likely to determine that the rate of change in COVID-19 cases was increasing for much of the displayed time period despite the perceptual features of steepness. Romano, Sotis, Dominioni, and Guidi (2020) found that a large number of citizens thought COVID-19 data shown on a log scaled graph indicated a less severe situation than the same data shown on a linear scaled graph.

3.3. Analysis of interviews

Authors were assigned to analyze groups of interview videos, and created rough transcripts by improving upon automatic transcription created by YouTube. We first coded the transcripts to identify when citizens talked about slope, exponential growth, rates of change, mathematical models, accumulation, graphs, relative size comparisons, frames of reference, relative medical risk, trustworthiness of data, and medical information. As a group we chose to focus on ideas of comparisons of relative size, rate of change, slope, and graph for this paper because it was clear how citizens’ schemes were impacting their interpretations of those items, and prior models of thinking were useful in explaining the data. In further analysis we used a modification of Corbin and Strauss (2008)’s grounded theory approach that was similar to that used by Byerley and Thompson (2017). Byerley and Thompson explained, “The modification was that we began our data analysis with the conceptual analysis of magnitudes and rates of change described in the literature review, as well as multiple descriptions of teachers’ schemes from prior qualitative studies” (p. 176).

When we made decisions about our initial codes, we were influenced by prior models of student thinking. We were not endeavoring to stay completely “open” to identifying new constructs to describe mathematical thinking and instead focused on applying existing constructs in a new context. We were not trying to necessarily “discover, name, and develop concepts” as described in the section on open coding in Strauss and Corbin (1998, p. 102). Our process is more related to “selective coding” (Strauss & Corbin, 1998, p. 143) because we were applying already named and developed constructs from prior research to analyze thinking in a new situation and test the suitability of those constructs in that situation. At the stage of selective coding the goal is to “integrate and refine a theory” and at that stage there are “no new properties, dimensions, or relationships” emerging during analysis (Strauss & Corbin, 1998, p. 143). For example, we intentionally designed interview protocols with the expectation we would see additive vs. multiplicative thinking, and static vs. emergent shape thinking in citizen’s responses to our questions. While we had fairly specific predictions about citizen’s potential mathematical schemes, we did not know how citizens would use their non-mathematical schemes in conjunction with their mathematical schemes to make decisions. Thus, while we remained theoretically “open” with regards to themes describing how citizens incorporated mathematical and non-mathematical reasoning in decision making, we created no new constructs describing how citizens thought about mathematical topics such as relative size or graph. We did use open coding to create categories for non-mathematical arguments such as “citizen expressed distrust of data.”

4. Results

We chose to focus on ideas of comparisons of relative size, rate of change, slope, and graph based on prior research that investigated students’ or teachers’ thinking. When we analyze citizens’ responses to the items in the following sections, we are concerned with how citizens’ schemes influence their interpretations of the items instead of whether they can make mathematically accurate arguments or not. We first illustrate productive and unproductive (or less productive) schemes for rate of change, comparing the relative sizes of quantities, slope, and graph that citizens used to interpret COVID-19 data representations (Section 4.1). We then identify how beliefs about the scientific and medical communities and the reliability of their data and recommendations could override both unproductive and productive interpretations of COVID-19 data (Section 4.2).

4.1. Citizens’ mathematical schemes and their assessment of COVID-19’s severity

In this section, we summarize 32 citizens’ responses to the “Flu vs. COVID-19 rates” item (Fig. 2). We also discuss vignettes of selected citizens’ interviews in order to illustrate particular schemes in detail.

4.1.1. “Flu vs. COVID-19 rates” item, and schemes for average rate of change and additive comparisons (Stories of Bumsoo and Kenneth)

Table 1 provides the results of our analysis of citizens’ responses to the “Flu vs. COVID-19 rates” item (Fig. 2). Citizens gave multiple justifications for their response on which disease was more severe and incorporated medical ideas such as the novelty of COVID-19 into their responses.

Table 1.

Responses to “Flu vs. COVID-19 deaths”.

“Flu vs. COVID-19 deaths”
	Compared flu and COVID-19 deaths using the idea of average rate of change	Referred to exponential growth	Medical argument of COVID-19 such as novelty, lack of vaccine	Expressed distrust that data presented is accurate
Flu is more severe than COVID-19	0	0	2	4 incl. Kenneth
COVID-19 is more severe than flu	7 incl. Bumsoo	4	22	1
Unsure if flu or COVID-19 is worse	0	0	2	0
Subtotal	7	4	26	5

The four citizens who claimed the flu is worse than COVID-19 compared the death totals for each disease without reference to the time span when those deaths occurred, including Kenneth. Seven citizens who claimed COVID-19 is worse argued via a comparison of average rate of change, noting that the COVID-19 deaths occurred over a shorter period of time. We provide examples from two citizens, Bumsoo and Kenneth, to illustrate two different schemes that varied in their productivity.

4.1.1.1. Story of Bumsoo (SK)

Bumsoo, who holds an undergraduate degree in business, is one citizen who claimed COVID-19 is worse by comparing average rates of change. He was interviewed on April 15th, 2020 and up to and including that day there had been 10,591 confirmed cases of COVID-19 and 225 deaths in SK. Bumsoo used productive schemes for average rate of change to consider the relative severity of the flu and COVID-19 in the item “Flu vs. COVID-19 deaths” (Fig. 2). He computed the average number of new cases per month for the flu in the US, and the average number of new confirmed cases per day for COVID-19 and concluded that COVID-19 is more serious than the flu. He disagreed with an argument that relied on comparing total number of confirmed cases rather than rate of change and time elapsed. Bumsoo immediately tended to the quantities in the argument. He focused on comparing two quantities to draw conclusions. He took into account the differing time period given in the argument and computed average rate of change (see Excerpt 1).

Excerpt 1. Bumsoo’s responses to the item “Flu vs. COVID-19 deaths”

Bumsoo	What's the US population now?
Interviewer (Int.)	It is 328 million.
Bumsoo	Korean media usually provides ratios. For example, the ratio of 50 million people [current SK population] to 100 people. In this way, 62,000 people looks really big, but it's actually calculated at a very low percentage, so it's right not to interrupt your daily life. But looking at it now, I think the standards are not consistent. The flu is five months from October to March. 62,000 in five months is about 3000 people in a month [He made a small computational errora].
aWe think Bumsoo first divided 60,000 by 10 to get 6,000. Instead of doubling 6,000 to find that there was about 12,000 deaths per month he found half of 6,000 which is 3,000 deaths per month.

Bumsoo continued to explain that deaths for COVID-19 were reported per day and he knew approximately how many people were dying from COVID-19 in the US per day in April. He saw that in a month many more people would die from COVID-19 than the roughly 12,400 that died from the flu in one month. He argued “If you look at it like this, I think corona is much more dangerous. Corona data is on a daily basis and flu is on a monthly basis.”

4.1.1.2. Story of Kenneth (US)

In contrast, Kenneth, who holds a BA English Education, argued that the flu is worse than COVID-19. He relied on additive comparisons of total number of deaths to draw this conclusion. Kenneth was interviewed on May 11th, 2020 and up to and including that day there had been 1,300,696 confirmed cases of COVID-19 and 78,771 deaths in the US. Kenneth agreed with the claim in the item because the CDCs’ estimates of flu cases was much higher than COVID-19. He did not take elapsed time into account or discuss the similar number of deaths from COVID-19 and the flu. We did not discuss when COVID-19 started with Kenneth because he did not bring up the time periods.

Excerpt 2. Kenneth’s responses to the item “Flu vs. COVID-19 deaths”

Kenneth	[Read the argument] Yes, I agree with that. I guess it’s the same argument like sugary foods and smoking killed more people than the coronavirus, but we don’t outlaw those or we don’t shut down stores that sell them.
Int.	What about the numbers specifically leads you to agree with the argument?
Kenneth	The higher numbers of the flu.
Int.	As compared to the lower numbers of the coronavirus?
Kenneth	Yes.

Kenneth’s comparison of the consumption of sugary foods and smoking to COVID-19 is notable; failing to acknowledge the fundamentally different risk structure and scope between personal choice in a disease process that physiologically affects only that person and a contagious disease where personal choice can affect many others can have significant impact on how one assesses these risks.

4.1.2. “Flu vs. COVID-19 rates”, and comparing percentages (Stories of Eunseok, Amelia, and Katie)

Table 2 captures responses to “Flu vs. COVID-19 rates” item (Fig. 4). Citizens gave more than one mathematical or medical argument. Even though only two citizens said scientists incorrectly estimated infection fatality rates for COVID-19, we included this category in our table because we saw similar statements in many places online. For example, well-known influencers such as Elon Musk made a similar claim that COVID-19’s “fatality rate is also greatly overstated” (Musk, 2020).

Table 2.

Responses to “Flu vs. COVID-19 rates”.

“Flu vs. COVID-19 rates”
	Approximately correct multiplicative comparison.	Incorrect multiplicative comparison.	Asked to make multiplicative comparison but citizen didn’t respond	Said 2% of a large number is very large.	Said 2.1 % and 0.1 % are both small so COVID-19 is not too serious	Said scientists incorrectly estimated infection fatality rates for COVID-19.
Flu is more severe than COVID-19	1	1	2	3 incl. Katie	1 incl. Katie	2
COVID-19 is more severe than flu	12 incl. Eunseok and Amelia	9	1	12 incl. Eunseok and Amelia	1	0
Unsure if flu or COVID-19 is worse	0	0	1	2	0	0
Subtotal	13	10	4	17	2	2

Consistent with prior research, we found that only 13 out of 32 citizens compared the relative size of 2.1 % and 0.1 % accurately. However, being able to compare the relative sizes of 2.1 % and 0.1 % was not necessary for citizens to have compelling reasons to describe COVID-19 as more severe than the flu. One person knew 2.1 % was 21 times as large as 0.1 %, but thought the flu was worse than COVID-19 because he did not believe that 2.1 % was an accurate infection fatality rate. We provide examples from three citizens, Eunseok and Amelia, and Katie, that show how people with different schemes for comparing relative sizes approached the item.

4.1.2.1. Story of Eunseok (SK)

Eunseok is an English teacher who was working on a PhD degree in English education. Eunseok was interviewed on April 23rd, 2020 and at that point there had been 10,702 confirmed cases of COVID-19 in SK and 240 people had died. Eunseok focused on the total number of infected people to compare the infection fatality rates without being prompted. He said, “the coronavirus mortality rate is much higher than that of the flu.” Eunseok used a part-whole scheme for percentage to reason about the infection fatality rate.

Excerpt 3. Eunseok’s responses in part (a) of “Flu vs. COVID-19 rates.”

Int.	You said you feel like coronavirus is more serious, and that the fatality rate was much higher. What comparison concluded that it was much higher?
Eunseok	In this argument, I compared numbers, 2.1 % to 0.1 %.
Int.	How did you compare [the two numbers]?
Eunseok	2.1 % indicates, if there are 100 people, for example, two people die. And one person dies if there are 1000 people who were infected with flu. In other words, the coronavirus kills 21 people when 1000 people are infected, and the flu kills one person if 1000 people were infected.

Eunseok not only understood 2% as 2 out of 100, but also was able to reason proportionally to determine that 2.1 % means 21 people out of 1000. Eunseok also understood that if the flu has a fatality rate of 0.1 %, then one person will die on average if 1000 people are infected. He demonstrated his schemes for percentages which entail part (a number of people dying) and whole (a total number of people infected). He used 700,000 total infected people for flu, and 10,000 total infected people for COVID-19 to explain the infection fatality rates. Eunseok chose to reframe his explanation of 2.1 % so that the referenced unit whole was 1000 people for both percentages.

Eunseok used his part-whole explanation to engage in the act of multiplicative comparison.

Excerpt 4. Eunseok’s responses to part (b) of “Flu vs. COVID-19 rates.”

Int.	The entire population of Korea is approximately 50 million and when the flu death rate is 0.1 % and the coronavirus death rate is 2.1 %, how would you explain to others how many people would die?
Eunseok	0.1 % of 50 million is 1/1000 of 50 million. Then… isn't it 50,000? 1 million people would die when death rate of the coronavirus is 2.1 % since it is like 2/100. Then isn't that 20 times more?

Eunseok used his previous work of “21 people out of a thousand” and “1 person out of a thousand” to compare 2.1 % and 0.1 %. He did not immediately respond that 2.1 % is 21 times as large as 0.1 %, but instead took intermediate estimation steps. He ended with the conclusion that COVID-19 kills 20 times as many people as the flu of the number of infected individuals is the same for each. Though 20 times should be 21 times, Eunseok did have the resources necessary to compare infection fatality rates. We view Eunseok’s use of a part-whole as productive because he ended with an accurate conclusion. However, we viewed Amelia’s relative size scheme discussed in the next section as more productive because her scheme allowed her to more efficiently compare the percentages without needing to use 50 million in her computations.

4.1.2.2. Story of Amelia (US)

Amelia was a communication and conflict management teacher with a master’s degree in counseling. Amelia was interviewed on April 20th, 2020 and there were 760,245 confirmed cases of COVID-19 and 40,690 deaths in the US. Amelia compared infection fatality rates but did so using different reasoning than Eunseok. She responded, “The numbers of 50 million and 50 million do not matter. You’re comparing 0.1 to 2.1. You could be saying you are comparing one to 21. I guess we’ll take 21 times. We’ll take 21 times as many.” Notice that Amelia’s response is efficient. She did not need a calculator, the internet, or a piece of paper to solve the problem suggesting she had developed the ability to coordinate multiple relationships almost simultaneously by directly comparing the relative size of the percentages themselves. Amelia had an understanding of place value that allowed her to see that 10 copies of 0.1 was equal to one. This allowed Amelia to see that 2.1 was 21 copies of 0.1. She knew that the relative size of 0.021(50,000,000) and 0.001(50,000,000) is the same as the relative size of 2.1 % and 0.1 %. Thus, she could ignore the information about 50 million cases.

4.1.2.3. Story of Katie (US)

Katie has master’s degrees in both social work and psychology. Katie was interviewed on April 16th, 2020 and there had been 1787 confirmed cases of COVID-19 in her home state of Kansas. Katie believed that the stay-at-home order for Kansas was unwarranted. Kansas residents were asked to stay at home in counties that had no confirmed COVID-19 cases, and she claimed the infection fatality rates for COVID-19 were not large enough to warrant these orders in her county. Katie’s mathematical ideas interacted with her understandable frustration at being asked to stay home when there was no evidence of transmission near her.

Katie initially focused on the uncertainty of the current COVID-19 fatality rate estimates. The interviewer asked Katie to pick a percentage in the COVID-19 range and she picked 1.5 %. She then spoke about how both the flu fatality of 0.1 % and COVID-19 fatality rate of 1.5 % were small until she was prompted several times to compare the two to each other.

Excerpt 5. Katie’s comparison of infection fatality rates

Int.	Alright. One point five. What does it mean to have a death rate of one point five percent?
Katie	It is a very low death rate…. It is very low. Especially, if you think about how many people there are in the world or are in the US. It is very, very low. If you look at rates of cancer, rates of heart disease, they are much, much higher. So, umm…I mean it is not a very high death rate. But there has also been a lot of discussion that people that have the coronavirus have lung damage afterwards.
Int.	I’m going to just highlight the two numbers [1.5 % and 0.1 %]. What kind of comparison can you make?
Katie	Well, I mean, the seasonal flu probably has such a low death rate because we have vaccines for it…
Int.	Right. What about the values themselves? The numerical values of 1.5 % and the 0.1 %.
Katie	Well that would make COVID-19 much higher.

Katie said both infection fatality rates are very small (albeit with 1.5 % much higher than 0.1 %), and she did not say a small percentage of a large number of people still meant a large number of deaths. In fact, she suggested the opposite – that a small percentage was insignificant because it applied to so many people in the world or the US. She compared infection fatality rates from contagious viruses to non-infectious causes of death. The infection fatality rates did not perturb her original belief that COVID-19 did not warrant extreme measures outside of current hotspots. Katie’s responses indicate that what people infer from data is subject to their beliefs. Katie’s case also underscored the importance of widespread testing so that citizens can be given evidence of transmission in their area if they are asked to stay home.

4.1.3. “South Korea Cases”, “Three Country Cases”, and “Log Scaled Cases” items, and schemes for slope (Stories of Bumsoo and Gertrude)

Table 3. Responses provides the overview of analysis on three items. Out of 32, 12 citizens attended to only visual comparisons of steepness or indexical association that are based on comparisons of direction or orientation (i.e., static shape thinking).

Table 3.

Responses to “South Korea Cases”, “Three Country Cases”, and “Log Scaled Cases” items.

“South Korea Cases”, “Three Country Cases”, and “Log Scaled Cases”
	Focused only on steepness	Said slope gives rate of change of cases per day	No mention of steepness or rate of change
The log scaled graph looks different or less scary than other graphs	10 incl. Bumsoo	5	3
The log scaled graph shows the same data as other graphs	0	7 incl. Gertrude	0
Did not answer or was not asked to compare log and linear scaled graphs	2	2	3
Subtotal	12	14	6

The seven citizens whose scheme for slope involved attending to the axes to compare relative sizes of quantities’ changes (i.e., emergent shape thinking) observed that the log scaled graph presented the same information as the US graph in Fig. 6. The ten citizens who only focused on steepness and not the graph’s axes or scale concluded that the log scaled graph looks less severe than the US graph in Fig. 6 even though the graphs displayed identical data. Bumsoo’s interpretation of the data graphed on a log scale as showing a “less scary” situation was typical of many of the people we interviewed. In the context of these graphs, focusing on steepness without attention to the axes is unproductive because it leads to having multiple conflicting interpretations of the same data presented in different ways. Even some (5) of the people who knew the slope of a graph meant the rate of change of cases per day did not attend to the y-axis in a way that allowed them to see the log scaled graph as presenting identical data without support from the interviewers.

4.1.3.1. Story of Gertrude (US)

Gertrude is a math teacher who was working on a master’s degree in mathematics education. Gertrude was interviewed on May 7th, 2020, and on that date there had been 1,257,023 confirmed cases of COVID-19 and 75,662 deaths in the US. She productively interpreted the log scaled graph in Fig. 7 by comparing the changes in the total confirmed cases and days elapsed. Namely, Gertrude demonstrated emergent shape thinking because she primarily focused on the numbers on the axes, and she also formed multiplicative comparisons between quantities, explaining that slope relates to the changes in total confirmed cases and days elapsed.

To illustrate, Gertrude immediately moved the mouse to the y-axis and said, “here these increments are multiplying by 10 each time.” She realized that the y-axis values meant that COVID-19 was “increasing at a higher rate” as time passed, and she added “Every four days and this is going up. [The number of cases is] multiplying by ten.” After noticing the tenfold gaps in the y-axis values, she identified many days elapsed before the number of cases multiplied by 10, suggesting her forming a multiplicative comparison of two quantities. As further evidence of her productively interpreting the graph, she explained the slope of the log scaled graph indicated the rate of change. Specifically, Gertrude compared the changes in the total confirmed cases and days elapsed when asked about slope of the graph.

Excerpt 6. Gertrude’s response to “Log Scaled Cases”

Int.	We can look at March 10. What could we say about the, like how long it takes to multiply by 10?
Gertrude	Eight days.
Int.	Okay, so it took eight days to multiply by ten. What does the slope of the log scaled graph tell us? Like the steepness of it? What… what information is that telling us?
Gertrude	It's telling us the rate. How fast it’s growing, the rate of change.
Int.	Okay and so it took eight days so there's kind of two ways to think about the rate. It took eight days to multiply by ten. You could also say it took eight days to go up by 900.
Gertrude	And then here it took from March 10 to 18, it took eight days again.
Int.	To do what?
Gertrude	To multiply by 10.
Int.	Okay, and how much did it go up like additively?
Gertrude	[Talks to self.] 9000.

To summarize, Gertrude illustrates how emergent shape thinking is productive because it entails judgments based on the coordination of quantities’ values and comparisons between those values by attending to axes scaling and coordinating values. Furthermore, her ability to switch between thinking about how much the cases increased in a time period and thinking about how many times the cases increased in the time period is strong evidence of her productive schemes for graph, slope, and rate of change.

4.1.3.2. Story of Bumsoo (SK)

Bumsoo’s actions suggested his schemes for slope were associated with static shape thinking because he made judgements of slope primarily based on perceptual properties of the graph such as steepness or angle. We found it interesting that Bumsoo had less productive schemes for slope because he had such productive schemes for the related topic of average rate of change (Sec 4.1.1.1). When given data about total confirmed cases of COVID-19 on a graph, he used an angle measure scheme for slope disparate from the idea of rate of change of confirmed cases per day.

On the item “South Korea Cases” (Fig. 5) Bumsoo explained that the slope was two because the steepness of the line was between 45 and 90 degrees, and he did not modify his answer when prompted to consider the y-axis values. To Bumsoo, a slope was an indication of steepness and angle instead of a relative measure of changes in quantities (see Excerpt 7).

Excerpt 7. Bumsoo’s responses to “South Korea Cases”

Int.	How did you determine that “the slope is 2”? You said, “the confirmed cases increased from 2000 to 3000, which means the slope is 2.”
Bumsoo	Yeah, just a guess. The slope is steeper than 45 degrees and lower than 90 degrees. It looks like about two-thirds. [He gestures to the line cutting the first quadrant into a 1/3 and 2/3 piece]. So [the slope in the SK graph] would be twice as much [as a slope of 1].
Int.	Did you draw a hypothetical 45-degree graph?
Bumsoo	Yes, that's right. [The slope of the SK graph] is steeper than that. Oh, it has increased rapidly. I thought it would be really fast at 90 degrees. The closer the slope is to 90 degrees the worse the pandemic is. But in between, I thought [the slope would] be 2 because it is between 90 degrees and 45 degrees.

Bumsoo’s story is an example of static shape thinking because he again focused primarily on steepness in the item “Log Scaled Cases.” (Fig. 7). When he was prompted to look at the numbers on the y-axis, he became surprised when he identified a tenfold gap between the intervals. Despite noticing the y-axis value structure, Bumsoo maintained a focus on the shape of graph and he said the situation was “more stable in late April” when the steepness of the graph was lower.

Excerpt 8. Bumsoo’s responses to the “Log Scaled Cases” (Fig. 7).

Int.	What information on this graph did you use to determine stability?
Bumsoo	I saw the slope here first. Overall, as I said earlier, I saw how [the slope of the graph] was going back and forth close to 45 degrees, and then I saw that the closer the slope was to a horizontal line, the more stable [the COVID-19 situation] was.
Int.	What does the slope mean here? [Used the arrow to point.]
Bumsoo	It looks more stable than the graph I saw earlier. [Bumsoo had seen a graph of US data in the “Three Country Cases” with an equally scaled y-axis.]

As further evidence of static shape thinking, Bumsoo thought that the situation in the US was less severe when he saw identical data on a log scaled graph. The interviewer next directed Bumsoo’s attention to the y-axis to determine if Bumsoo’s interpretation of the log scaled graph would change.

Excerpt 9. Bumsoo’s thoughts about the y-axis values on “Log Scaled Cases”

Int.	Have you seen the y-axis numbers here?
Bumsoo	No, I barely saw it. I saw it when you were asking questions.
Int.	Would you like to take a look?
Bumsoo	1000… [reading axis] 10,000 people
Int.	Now, did you see the y-axis numbers and think of something else?
Bumsoo	[Bumsoo pointed at 10 K–100 K dots on the curve]. I was surprised to see how quick it became ten times bigger with the numbers of confirmed cases. Oh, this is a log scaled graph. But even after looking at the numbers on the y-axis, I still happen to pay more attention to the overall shape of the graph
Int.	So, after you saw the logarithmic scaled graph, the meaning for the slope changed?
Bumsoo	It's changed, but the impression I got with my bare eyes hasn't changed. So, before I recognize the number on the left, I look at the shape first.

It is important to note that Bumsoo did verbally state that his meaning for the slope was changed after recognizing the log scaled numbers. However, the entirety of his subsequent statements continued to focus on the graph’s shape, which suggests that his meanings did not entail a fundamental change. He demonstrated static shape thinking by focusing on the different visual steepness of the two graphs and concluded that the graph on a log scale did not convey the same data as the graph on a linear scale due to its concave down shape. Bumsoo’s schemes for slope were only productive in limited circumstances such as determining the slope of a graph with equally spaced linear axes.

4.2. How beliefs overrode or undermined math schemes in assessing severity

Our interviewees discussed their beliefs about the scientific and medical communities, and the reliability of the data and recommendations generated by these communities as part of their assessment of the severity of COVID-19. Many citizens had conclusions that they drew from the presented data and other sources of information. However, for some citizens their beliefs about the scientific and medical communities prevailed over the mathematical conclusions that the citizens drew. Even in cases in which citizens had strong mathematical schemes, a strong distrust of existing data could lead to them discounting their own conclusions. Pew Research found on from June 4th to 10th, 2020 that 31 % of U.S. adults say the CDC mostly gets the facts about the outbreak right “hardly ever” or only “some of the time.”6 On the other hand, a strong trust in scientific authority can lead citizens to think COVID-19 was severe even when they do not demonstrate the mathematics necessary to interpret the presented data. In this section, we illustrate each of these cases using data from Scarlet and Vera.

4.2.1. Story of Scarlet (US)

Scarlet worked in toilet paper production and has a degree in business management. She was interviewed on April 28th, 2020 and at that point there were 61,180 COVID-19 deaths in the US. Her interview protocol said that there were 24,000–62,000 flu deaths from October 1st, 2019 to April 4th, 2020. Scarlet had strong mathematical schemes for interpreting data, but her belief that the data was inaccurate influenced her assessment of the severity of COVID-19. Scarlet expressed concern for the suffering of sick people from both the flu and COVID-19 and believed she was personally responsible for not spreading illness to others. Scarlet agreed with the statement that the flu is worse than COVID-19 and claimed that we should not shut down the economy for COVID-19 because it is not shut down for the flu. She thought that in some situations a particular school should close for up to a week when a large number of students are ill in that school.

There were a number of reasons she gave for not believing that 61,180 people had died from COVID-19 including problems with testing, hospitals receiving financial incentives to intentionally miscategorize deaths as due to COVID-19, and a story she had heard of a man who died from falling off a ladder who was deemed a COVID-19 death because of the antibodies in his blood. She said, “the CDC admitted they are classifying so many deaths as COVID-19 that aren't that aren't necessarily COVID-19.” The interviewer tried to clarify to see if she thought that the flu was worse than COVID-19, or if she thought there was too little quality data to tell. Scarlet replied, “Right now it looks like the flu is worse because it looks like the numbers are being so skewed with COVID-19. No one believes it is valid anymore. No one, no one believes. Nobody's listening to that Dr. Fauci and Dr. Birx [medical experts working with the US White House] anymore.” The interviewer showed Scarlet a graph comparing the total number of deaths in NYC from any cause during the COVID-19 pandemic and the average deaths from any cause in past years (Fig. 8 ).

Fig. 8.

Graph of excess deaths in 2020 obtained from New York Times (Wu, McCann, Katz, Peltier, & Singh, April 23, 2020).

During the peak, the total weekly deaths in New York City was six times the average weekly deaths in past years. Scarlet was able to accurately interpret the graph and said that it looked like “a bomb went off” in New York City. However, she was not convinced that COVID-19 had caused the spike in deaths and expressed disbelief in the data about total deaths as well: “If that [graph] was in Greenville, South Carolina I would look at it and go, oh gosh something happened bad. But with the New York mayor and the New York governor the way they lie and carry on I don't know what to think about them.”

We underscore that at no point did Scarlet experience difficulty understanding the data representations; her actions implied she held productive schemes for interpreting those representations. However, her lack of trust in the data caused her to assess COVID-19 as not severe enough to justify businesses shutting down, and that COVID-19 is not as bad as the flu. We note that she did think COVID-19 justified hand washing, extra cleaning, and staying six feet apart from each other.

4.2.2. Story of Vera (US)

Vera works in marketing and has a degree in European history. She was interviewed on April 7th, and there were 368,533 confirmed cases of COVID-19 and 11,008 deaths in the US. Vera was strictly following all social distancing recommendations and supported them. In March 2020 she wondered if she was overreacting to COVID-19 because other people she knew were not taking it seriously. By April 7th, she had come to understand COVID-19 as a serious situation because of learning about a variety of business and educational activities being cancelled or moved online. Since she was in her thirties, she was more concerned with transmitting COVID-19 to others than her personal risk and was only in close contact with one person.

When presented with the item “Flu versus COVID-19 deaths” (Fig. 2), it included data up to April 2nd and listed 10,763 COVID deaths and 24,000–62,000 flu deaths. The only quantities Vera discussed comparing related to the item “Flu versus COVID-19 deaths” was total deaths and total hospitalizations. She did not consider deaths per week, or deaths per case to argue that COVID-19 was more severe. Vera told us she initially thought that the flu was more serious that COVID-19, but by April 7th Vera thought COVID-19 needed to be taken more seriously for a variety of non-mathematical reasons. Vera said additive mathematical arguments comparing flu and COVID-19 deaths were “one of the first reasons why I didn't take the coronavirus seriously.” She described her initial thinking at the start of the pandemic as “lots of people die from the flu every year. And there are fewer people dying from [COVID-19] than there were from the flu.” She explained that her thinking shifted when she realized how little we knew about outcomes from COVID-19 compared to the flu and when she reflected on the lack of a COVID-19 vaccine and the availability of a flu vaccine. On April 7th Vera was unsure if COVID-19 or the flu was more severe but because of the unknown nature of COVID-19 it needed to be taken seriously.

On the item “Flu vs. COVID- rates” (Fig. 4) Vera thought an infection fatality rate of 2.1 % was high when you considered how many people could die if millions were sick. She also spoke frequently about the preciousness of even one life and that the two percent of people who died would include people’s grandmas. After Vera gave qualitative responses about the seriousness of 2.1 % the interviewer asked her how many people would die if 1000 people had COVID-19 and it had an infection fatality rate of 2%. She said “You're making me do math. I hate math.” After encouragement Vera said “Isn't it …it's less than two people, right? Because two percent is out of a hundred? Or two is out of…” The interviewer confirmed with Vera that 2% is out of 100. Then Vera said, “Two percent would be two out of a hundred” and then concludes that “It would be like not even one person [out of 1000]. Vera’s opinion that COVID-19 was serious was based on her understanding of medical arguments and a concern for other people’s lives. She said she did not need to think about mathematical comparisons between flu and COVID-19 to assess the relative severity. From Vera and others like her, we learned how important it was to make multiple arguments about severity that were compelling to those who, as Vera described herself, “hated math.”

5. Discussion

5.1. Discussion of the results

Our work demonstrated that the models of mathematical thinking created by mathematics education researchers are helpful in creating hypotheses about which representations of novel COVID-19 data will be difficult for many to understand as intended.

While answering the item “Flu versus COVID deaths” many participants revealed productive medical understandings and said things such as “we know less about the COVID-19 than the flu, so we need to take COVID-19 more seriously” but they did not make comparisons between changes in cases and change in time. Kenneth’s additive scheme was less productive than Bumsoo’s scheme in May 2020 when the number of deaths from COVID-19 was less than the number of deaths from the flu. Making a multiplicative comparison of total deaths and the time period the deaths occurred is more appropriate to compare the severity of flu and COVID-19 especially in the first few months of COVID-19. On the other hand, in November 2020 Kenneth’s additive scheme would allow him to draw more useful conclusions about the relative severity of flu and COVID-19 because the total number of COVID-19 deaths is now much greater than the average annual flu deaths. Because of this we say additive schemes are often less productive than multiplicative schemes but that additive schemes are productive in a limited set of situations.

Our hypotheses that citizens would not quickly and fluently compare the relative size of 2.1 % and 0.1 % was also well supported in data from “Flu vs. COVID-19 rates.” Many citizens knew a multiplicative comparison of 2.1 and 0.1 was appropriate even if they were not sure how to make that comparison in their head. Also, many were able to understand “Flu vs. COVID-19 rates” by finding 2% of a large value and comparing this number to 0.1 % of the same large value. We see ways of reasoning that enable citizens to compare relative sizes of percentages as productive for citizens’ understanding of media data, because a typical news segment or article has so much information that the citizen is quickly carried along to the next piece of information. Many of the citizens we interviewed were much less inclined than Amelia to make a comparison because the process of comparing two quantities was cumbersome for them. After considering the responses as a whole, we still agree with Thompson and Saldanha (2003) that helping students develop interconnected and powerful quantitative schemes for multiplication, division, proportions, percentages and fractions is critical for positioning them to efficiently make sense of the world. We view helping student develop reasoning like Amelia’s as a great goal for teachers. On the other hand, we acknowledge that Eunseok was able to reach the same correct conclusion and each step in Eunseok’s process would likely be understandable for citizens who think of fractions as parts out of whole, but not as comparisons of relative size. Eunseok’s reasoning using part-whole schemes was more common in our sample and many citizens could likely make sense of COVID-19 data more easily if each step of Eunseok’s process was described. Katie’s case highlights the importance of providing high quality location-specific data and that some citizens will think that an infection fatality rate of 1.5 % is very low and need support in comparing that rate to other rates. Currently many media and governmental sources only provide percentages without supporting citizens in making sense of what the percentages convey, and we encourage media and government sources to scaffold their information more in order to help citizens with primarily part-whole schemes fully understand their data.

Our hypothesis that many citizens would unproductively use a scheme for slope as steepness to interpret the logarithmic scaled graph was well supported by the responses to “Log Scaled Cases.” We also correctly anticipated that some people would ignore the quantities and measures on the y-axis and that this would lead them to make contradictory interpretations of the same data displayed on graphs with different axes. Bumsoo’s static shape thinking was less productive than Gertrude’s emergent shape thinking because Gertrude successfully interpreted the graph on a log scale by attending to axes scaling and coordinating values. Bumsoo focused on visual steepness and concluded that the graph on a log scale looks less severe than the graph on a linear graph due to its concave down shape.

Another finding from our interviews is that citizens’ schemes for an idea such as rate of change were applied differently as the representation of data varied. For example, without prompting, Bumsoo made a productive argument using ideas about rate of change to compare the severity of flu and COVID-19. Even though Bumsoo knew the approximate rate of change of confirmed COVID-19 cases per day in SK, he did not use this knowledge when interpreting the slope of a graph of SK COVID-19 cases. His schemes for slope of a graph overpowered that productive reasoning and he determined the slope of a graph based on the steepness of the graph with no attention to the quantities on the axes. The example of Bumsoo shows us that mathematically capable citizens can develop unproductive ways of thinking about slope that limit their use of their productive reasoning. In other words, Bumsoo might have been better served if he had never been taught to associate particular inclines of a line with a numerical value of slope. If Bumsoo did not have an immediate association between incline and slope, we conjecture he might have started to use his average rate of change scheme to understand the graph.

5.2. Implications for mathematics education and pandemic education

Our data provides some suggestions for how formal math education can adjust to help students become citizens who can read and interpret media data on important issues. Our results show that it is important for mathematics programs in schools and universities to support citizens in developing productive schemes for ideas like slope, graph, and comparisons of relative size that they can use to interpret quantitative data in a changing world. Mathematics educators need to provide students with opportunities to build productive schemes for important ideas and more attention should be given to deciding which schemes are most productive and in what contexts (e.g. Thompson (2013b). For example, Bumsoo said he learned the concept of slope as angle measure in secondary school. It is important to help teachers understand the limitations of teaching the idea of slope as angle measure and to help teachers develop more productive lessons on slope (Byerley and Thompson, 2017). We also suggest that mathematics teachers at the high school and college level pay particular attention to their students’ understanding of secondary school topics such as place value and fractions (Byerley, 2019). The high school mathematics teacher, Gertrude, showed us that it is possible to pass many advanced mathematics courses without becoming confident in concepts such as place value and relative size of quantities. She had extremely productive interpretations of the logarithmic scaled graph but did not know how to compare the relative size of 2.1 % and 0.1 % even after she tried for a few minutes to do so. Throughout our data is a recurring theme of participants that did not attend to the scale of axes and used static shape thinking when making conclusions about the data. We suggest researchers take up work to help students construct emergent shape thinking schemes for graphing (Ellis, Tasova, & Singleton, 2018; Tasova et al. in press).

It is also important to design data representations that citizens can productively understand today with their current schemes. As a result of this study, we designed a visual representation of percentages that we hope will allow citizens to compare the relative sizes of a variety of risks regardless of their comfort with place value and percentages (Fig. 9 ). We also want to study how citizens make comparisons of the relative risks of COVID-19 and COVID-19 vaccines. We designed the applet based on data from the COVIDAge Calculator.7 Making productive comparisons of risk with the applet does not rely on reading the y-axes or understanding place value, percentages or decimals. The next stage of research is testing the applet with citizens and making improvements based on interview analysis.

Fig. 9.

Interactive diagram designed to support citizen's comparisons of relative risks. Note that making productive comparisons does not rely on reading the y-axes or understanding place value. Available at www.covidtaser.com.

Additionally, we designed an applet that allows citizens to watch data on a linearly scaled graph continuously change into a logarithmically scaled graph. The intent of this applet is to help citizens understand that linear scaled graphs and log scaled graphs of the same data look very different and that the steepness of the log scaled graph should not be interpreted in the same way as the steepness of a linearly scaled graph. (Available at https://www.covidtaser.com/covid-19-on-log-graphs). We encourage more mathematics educators to use their specialized knowledge to help doctors and epidemiologists convey data to the public.

5.3. Interactions between citizens’ mathematical schemes and their beliefs

The most difficult part of describing our citizens’ assessment of the severity of COVID-19 was that they used their beliefs, the news, their family’s and friends’ opinions, and their medical knowledge to answer the questions. As a result, it would be highly inaccurate to say that a person with more productive mathematical schemes was more likely to determine COVID-19 was more severe than the flu. Some citizens told us they struggled to understand the mathematics we were showing them, yet they had been highly engaged in listening to doctors on the news and they used that knowledge so their interpretation of the data we presented was largely consistent with what the doctors intended to convey. A summarized version of this argument was “I’m not good at math and I don’t know how to compare 2.1 % and 0.1 % but Dr. Fauci said the infection fatality rate of COVID-19 was high so 2% must be a lot higher than 0.1 %.” On the other hand, some citizens had highly productive schemes for comparing 2.1 % and 0.1 % but they did not believe that the data for COVID-19 was accurate, so they thought the flu was more severe.

In the future we want to expand our theoretical toolkit by working with researchers in other disciplines to better understand citizens’ political and medical statements. One example of something we have entirely postponed is analysis of the anti-Korean racism some US citizens expressed in their interviews. The Pew Research Center found that 39 % of Americans say it is more common for people to express racist views against people who are Asian than before COVID-19 (Ruiz, Horowitz, & Tamir, July 1, 2020). We also understand that if future research included citizens in other countries with different pandemic responses and different beliefs, we would get fundamentally different responses to our items.

The ultimate goal of many mathematics teachers is to help students learn mathematics that will help them interpret the world throughout their lives. Because citizens reason about real-world situations using both mathematical and non-mathematical thinking it is quite complex to model how people use mathematics to understand the world. We feel that many citizens have untapped mathematical potential and an interest in learning how to use mathematics to interpret media. We hope this paper echoes the call that many mathematics educators have made to focus on helping students develop the productive schemes they need to make sense of a changing world. We were reassured that everyone we interviewed was interested in the greater good during the pandemic and interested in reflecting on how their choices impacted the world.

Declaration of Competing Interest

We have no competing interests.

Appendix A. Supplementary data

The following are Supplementary data to this article: