Developing Computational Thinking to Help Tackle Pandemic Challenges

COVID-19 has been extremely difficult to control. The lack of understanding of key aspects of pandemics has affected virus transmission. On the other hand, there is a demand to incorporate Computational Thinking (CT) in the curricula with applications in STEM. However, there are still no exemplars in the curriculum that apply CT to real-world problems such as controlling a pandemic or other similar global crises. In this paper, we fill this gap by proposing exemplars of CT for modeling the pandemic. We designed exemplars following the three pillars of the APEC InMside framework for CT: algorithmic thinking, computational modeling, and machine learning. For each pillar, we designed a progressive sequence of activities that covers from elementary to high school. In an experimental study with elementary and middle school students from 2 schools of high vulnerability, we found that the computational modeling exemplar can be implemented by teachers and correctly understood by students. We conclude that it is feasible to introduce the exemplars at all grade levels, and that this is a powerful example of STEM integration that helps reflect and tackle real-world and challenging public health problems of great impact for students and their families.


Introduction
The ongoing COVID-19 pandemic has demonstrated that most of the knowledge regarding the growing whirlwind of scientific and technological advancements has not been adopted by the general population. Despite great efforts to stop the transmission of the virus, we have seen that in the majority of countries all over the world, this has not been easy [1,2]. Certainly this is a multifaceted problem. Nonetheless, education is clearly one of the main aspects that can help explain the behavior and attitudes of the population in this context. Therefore, there is now an increasing demand to educate students to understand pandemics and help students to change some of their behaviors and attitudes [3]. It is then important to find ways to design educational strategies to prepare society for this and other global crises.
However, we have to consider that nowadays we are also experiencing several other demands to introduce changes in the science and math curricula. One of the new demands is the introduction of Computational Thinking in K12. Currently, there is a curriculum reform movement in this era of AI, Big Data for the 4th industrial revolution [4]. CT is becoming a key factor across the curricula in order to establish a Digital Economy [5]. CT is increasingly considered a critical cognitive disposition that students need to adopt and use efficiently to solve the challenges of the establishment of digital society which can be implemented both, with or without computers. But CT needs a basic literacy to think on how computers will process the information, how they connect with IoT and with smartphones, and how they integrate AI and Big Data. CT is closely related to computer and mathematical sciences. In the digital society it is necessary that the citizen has the information is communicated are well understood; graphs and methods of data visualization in general are therefore another subject that has a direct impact on people's decisions.
Hence, a good educational strategy to tackle future global crises should encourage students to develop skills necessary to understand at least the core ideas, assumptions, simplifications, biases, limitations, missing components, and simulation strategies of the computational and mathematical models of disease transmission by viral infection, as well as the corresponding data visualization and the appropriate evidence-based decisions to control it.
We have to consider that the change required is not just one of knowledge and skills. It is also critical to achieve a change in the values and attitudes of the students. The current era is considered as the era of globalization under new societal values on sustainable development goals (SDGs) by United Nations Educational, Scientific and Cultural Organization [11], and by other organizations like the Southeast Asia Ministers of Education Organization [12]. These values are in the process of being embedded into the school textbooks all over the world [13]. The problems of COVID-19 are directly related to SDG No. 3 and partially related to other SDGs. Global citizenship education is an issue across different SDGs. These issues and the matter of citizenship have been enhanced as critical in mathematics education and pedagogy [14][15][16][17].
Thus, our research question is: Is it feasible to implement a Computational Thinking teaching strategy to model pandemics and to what extent do students understand and reflect on the factors that influence pandemics and ways to control it?
To answer these questions, there are several approaches. The first approach is to clarify the current meaning of mathematical sciences in relation to adaptation of mathematics to the computer sciences. Indeed, it is an ongoing effort at the higher education reform on mathematics education [18,19]. The second approach is to review mathematical thinking from the perspectives of computational thinking. Traditional mathematical thinking has been considered in the context directed to pure mathematics [20,21] and in the current curriculum these issues are already considered and some revisions have been made. For example, they are in the process of reconsideration in Southeast Asia [22]. The third approach is considering to embed Informatics for AI and Big Data into the Mathematics curriculum. APEC InMside project [23-26] for regional curriculum reform focused on this third approach in relation to CT [27,28] and Statistical Thinking [29]. From the three approaches described above, the first one only applies to higher education, the second is still being studied and under revision, therefore we chose the third one as it is suggested by APEC and it is the most suitable for our context. Indeed, in the InMside project (2019), curriculum specialists for informatics and mathematics in APEC region produced new perspectives for curriculum reform on discussion documents for computational thinking [27] and statistical thinking [29] and produced the report for curriculum reform on APEC region [30].
The plan is to help design lessons to introduce Computational Thinking in the Mathematics curriculum as part of the design of educational strategies to prepare society for global crises, and in particular for pandemics. Therefore, our methodology is to apply the APEC InMside framework to create new exemplars of CT based learning activities consisting in solving problems related to the pandemic and adapted to students of elementary, middle and high school.
In the following sections we describe the framework used, show the exemplars explaining their corresponding CT and public health related concepts, report the results on an experimental implementation in two schools using the Lesson Study methodology [31], and finish with our conclusions.
The APEC InMside project [26] proposes a framework with three pillars for CT, developed in collaboration with curriculum specialists from the APEC region [28]. In this framework, CT is defined as an approach to problem solving that consists of analyzing, simplifying, and automatizing the solving process. The framework considers three pillars: algorithmic thinking, computational modeling, and machine learning (see Figure 1).

Figure 1: Curriculum Framework Model
The first pillar is Algorithmic Thinking, which means decomposing a problem into very simple instructions that can be followed unequivocally by any person or a computer. One of the key ideas for algorithmic thinking is recursive reasoning [32,33]. Algorithmic thinking emerges when someone observes repeated patterns in problems and then generalizes a set of rules for dealing with such situations (so that one does not need to think this through each time the problem occurs). In effect, one creates (formally or informally) an algorithm or rational procedure. This sort of thinking is often used in problem solving and in computer programming. Especially, if algorithmic thinking is represented by a program, there is a large number of mathematical techniques to be recognized as an algorithm [34][35][36]. Indeed, an algorithm is a list of simple instructions that anybody can understand and even a machine can follow [37]. Typically this can be done with a pseudocode, logical operators, lists and arrays, and cycles. Infinite loops are particularly powerful since the machine can do without getting tired or bored [38]. Moreover debugging, testing, documentation, peer review, and teamwork are also essential for programing.
The second pillar of the framework comes from science where CT is regarded as building computational models [39]. Here, "computational" means "programmable" which includes "mathematical". This type of CT is behind the current progress in biology, where increasingly accurate computational models of cells and organisms are being built. This is closely related to Covid-19 in various ways since computational and mathematical models are used to study the dynamics of viral infection with the aim of predicting the number of infected and recovered people through time, as well as the total number of deaths. Moreover, a swift development of a vaccine against the virus SARS-CoV-2 has been possible by using computational models that help to predict which parts of the virus would elicit an immune response. According to John Holland [40], a pioneer in the application of cellular automata to biology and the creator of genetic algorithms, patterns are normally expressed with board games: "Board games are not usually accorded the same primacy as numbers, but to my mind they are equally important cornerstone to the scientific endeavor. I think board games, as well as numbers, mark a watershed of human perception of the world". Holland also introduced computational models that can evolve, like in natural selection and thus are able to solve problems that their creators do not fully understand. Students can start with a concrete cardboard model, with a board, cards, chips, turn taking rules, and simple rules that characterize a phenomenon. It is also necessary to design ways to test that the model does what is required to do, compare its predictions with the natural phenomena, debug it, document it, and share tasks with a team.
The third pillar comes from Data Science and Machine Learning (ML). It is in the core of the 4th Industrial Revolution, which will have a massive impact on the nature of jobs and on employment. It represents a change of paradigm as well. Users train the computer instead of programming it. According to the Fields Medal mathematician, Cedric Villani, "(ML techniques) mark the gradual transition from a programming logic to a learning logic" [41]. ML is at the core of AI. Most of what is labeled AI today, is actually ML [42]. This is an algorithmic field that blends ideas from statistics, computer science, and many other disciplines. Core ideas are understanding and recognizing in a given problem what are the relevant features, what are the classes or categories to be identified or detected, metrics to assess the discriminating power of features or combinations of them, visual representations such as scatter graphs, construction versus test data bases, and measurements of performance of the trained classifiers. Skovsmose [14,15] highlights cross-cutting concepts on critical mathematics education, such as abstraction, modeling, algorithm and technological imagination. However, ML provides a change in the logic from programming to learning which will be a new aspect for critical mathematics education with applications. Figure 1 depicts the curriculum framework to develop CT through a conceptual diagram. This development is supported by the three pillars shown as the sides of the central triangle. On top of each pillar there are three keywords that represent them. CT can be developed in activities from every subject in the curriculum and also crosscutting many of them.

Materials and Methods
We propose the methodology of Lesson Study. This methodology began in 1880 in Japan with the aim of reproducing best practices in teaching [31]. It has been adopted in several other countries [43,44]. It is a collaborative teaching improvement process designed to generate strong and productive communities of teachers sharing and learning from each other ( Figure 2). Thus, we designed exemplars of lessons for each of the three pillars of CT. Then, using the methodology of Lesson Study, we implemented one exemplar in several elementary and middle school classes. We did it in two high-risk schools with a high level of vulnerability. A team of four elementary and middle school teachers worked in the lesson study implementation. The teachers, together with us, held constant meetings to review the sessions to reflect and adjust the lessons.
The exemplars proposed are learning activities aimed at students of different school levels. They correspond to solving problems related to the current COVID-19 pandemic using CT. We provided not only examples of exercises and activities but also guidelines to put them into practice in the classroom. We show an exemplar for each pillar of the framework presented in the previous section, with variants adapted to three school levels, elementary, middle and high school. In each case we highlight the relevant CT concepts and their connection to Critical Mathematics Education and curriculum, as well as the connections to significant practical public health elements.

Exemplar 1: algorithmic thinking to estimate the number of infected people
One of the main problems in the current pandemic is to know the number of infected people (including asymptomatic cases) in each country, every day. The typical data published in the press and web pages [45,46] show cases that are PCR tested, PCR-positives cases, and deaths. However, it is almost impossible to find information on the number of infected people. This information is critical to estimate the level of spread of the virus, and the trends on its transmission to determine if the corresponding curve is flattening. The number of tested individuals and the number of positives are not enough to estimate the percentage of infected population. Given the fact that the set of tested people are those who already feel sick and therefore suspect of being infected, the set of tested people is biased and it makes it very difficult to estimate the true number of infected people. Furthermore, given that some countries carry out many tests and others very few, it is very difficult to compare the figures of PCR-positive cases between countries and estimate trends based on comparisons across countries.
Thus, the problem has critical significance to the everyday life of every citizen (Ernst, 2010). Avoiding wrong conclusions from this kind of data requires deep critical thinking, together with CT, proportional reasoning, ideas of probability and statistical inference [47]. Here we propose three learning activities related to the problem estimating viral transmission and the proportion of infected people in a given population.

Elementary school lower grade students who learned numeration and counting
In order to control the pandemic, it is critical that students understand the transmission mechanisms. One feature of algorithmic thinking is proceduralization by using recursion. For example, the following elementary school coloring activity contains recursive instructions in order to understand disease transmission. One simple rule -to color in red the balls that touched another red ball (or balls)-has to be recursively applied ( Figure 3). Some relevant questions from this activity are: In which of the two tables there will be more transmission? How many balls will be painted red on each table? In how many moments will the transmission stop at each table?
In order to solve correctly this task, students need to understand the rule, and in particular, associate the concept of touching with having a common point of tangency. They also have to produce a group of elements defined by a certain condition, and therefore reflect about mathematical concepts such as recursion and set [22,48,49]. They also need to identify a stopping condition for the recursion and interpret the number of moments or steps as the number of times the rule is applied recursively until it stops. Students can discuss how they applied the rule and the results they obtained. By explaining their reasoning they can improve their understanding of the key concepts and help other to correct mistakes.

Middle school students who learned ratio and proportion
The problem in the previous activity was to identify who is infected (the red balls). Thus, for middle school students we propose an approach to estimate the true proportion of infected individuals. It is based on the use of data gathered in particularly selected ideal conditions, where all the people have been Polymerase Chain Reaction (PCR) tested and followed for several weeks. One such case is the Princess Diamond Cruise that was stranded off the coast of Japan for several weeks. Another case was several returning Wuhan residents on repatriation flights that arrived to the city and where all passengers were PCR-tested, remained in quarantine, and tracked for several weeks. From all these special cases, it can be estimated [50] that 0.66% of the population PCR-tested positive will die. This is similar to Gates [2] estimate of 1%. This means that one person dies out of 150 that were positive in the PCR test. Moreover, on average the person dies 14 days after being tested [51].
Let us consider a country where one out of 15 positives in the PCR-test dies 14 days after being tested. Here, instead of coloring (Figure 3), the scenario can be represented with a ruler and wooden blocks to facilitate reasoning and help make clear the algorithmic-computational process. The ruler represents days, which starts with day 0 representing the day an individual is PCR-tested. Each individual is represented by one block. Then the described scenario where from 15 PCR-positives one dies 14 days later is represented by stacking 15 blocks on the zero mark of the ruler and placing one block on the mark corresponding to 14. This is shown in Figure 4. However, this country -as all countries-does not test all people every day. Then, how can we use this information to estimate the number of infected people in the country? This question is represented in Figure 5, where the unknown number of infected at day 0 is represented with a pile of blocks. How can we estimate the number of infected people in a given day? Given that from 150 people one will die in 14 days (according to the rate obtained from the ideal cases), and that in this country from 15 PCR-positives one will die in 14 days, then there is an opportunity to do proportional reasoning. First, the key is to focus the attention in the person that dies on the 14th day. Then think backwards [20] in time, to 14 days earlier. On one hand there are 150 infected people. One person of these 150 dies 14 days later. Therefore, there are 150 people infected for the one subject that dies and to whom the attention is placed. On the other hand, there is the particular statistic on the country. From 15 PCRpositives one person dies 14 days later. Therefore, there are 15 PCR-positive related to the one person that dies 14 days later. This means that there are 150 infected and 15 PCRpositives. Hence, for every 10 infected people in the country there is 1 PCR-positive.
The activity for students consists of filling out a spreadsheet for their country, registering for each day the number of PCR-positive people and the number that die 14 days afterwards. And from this information, students can deduce the rule for the changing number of infected people on each day in their own country.
This activity also requires algorithmic thinking. Students have to consider and select appropriate data for every day and connect it with data with a delay of 14 days. They also have to gather data and arrange it in tables, search for patterns, combine population and proportional thinking, and input mathematical expressions in such a way that can be automatically updated.
Students can code the algorithm with spreadsheet functions in order to automatize the process. Thus, the students have to translate the mathematical relations in the specific syntax of the spreadsheet language. This translation is an important component of mathematics education [14] present in curricula such as the ASEAN curriculum standards [22].

High school students who learned conditional probability
For high school students, we propose a different lesson plan than the previous one.
Here we PCR-test a random sample of the population, and for this task we will explore two approaches, natural frequencies [52] and single-event probabilities. We will work under the frequentist Neyman-Pearson paradigm, in which probabilities are defined as the outcome of the limiting frequency with which that outcome appears in a long series of similar events [53].
Understanding the meaning of probabilities contributes to reduce statistical illiteracy which can have serious consequences for public health [52]. The way in which probabilities and risk are represented and communicated is critical. As emphasized by Cosmides et al [53] "from an ecological and evolutionary perspective, certain classes of problem solving mechanisms in the human mind should be expected to represent probabilistic information as frequencies". The natural frequency approach proposed by Gigerenzer [52] improves statistical literacy [54] not only in frequentist frameworks but also in Bayesian setups [55]. This approach is an ecologically valid format, that uses frequency statements instead of single-event probabilities and it helps to infer correct probabilities in typical tests and diagnosis situations. Evidence shows that subjects that do not actually use natural frequencies for their calculations, but translated them back into traditional expressions for probabilities instead, usually produce wrong answers [56].
Two key parameters of any viral test are its sensitivity and specificity [57]. Sensitivity is the percentage of people who are infected who test positive. Specificity is the percentage of people who are not infected and who test non-positive. Figure 6 provides a visualization of a test that has a sensitivity of 70% and specificity of 99%. In this example, there are 100 infected people and 100 non-infected people. All people have been tested and from the 100 of them that were infected 70 of them are PCR-positive, and 30 are PCR-negative. From the total of non-infected people 99 of them are PCR-negative, and only one is PCRpositive. Furthermore, assume that in a given country in a given day, 10% of the population is infected. Let us assume also that the population is large enough so that considering individuals independent or sampling them with replacement provides a good approximation of the true sampling process for testing (without replacement). If we select a random sample of 100,000 individuals, then most likely about 10,000 individuals will be infected, and therefore about 7,000 of them will most likely be PCR-positive, and about 3,000 will be PCR-negatives. Out of the 90,000 non infected individuals, most likely 89,000 will be PCRnegative and 1,000 will be PCR-positive. This means that in total we will most likely have 8,000 PCR-positives and 92,000 PCR-negatives.
It is interesting to note that 7,000 out of 8,000 PCR-positives are indeed infected. Moreover, 89,000 out of the 92,000 PCR-negatives will be not infected. This means that 3,000 out of the 92,000 PCR-negatives are infected.
Given that we want to estimate the true proportion of infected people in a given day, we have to consider this number as unknown, whereas the proportion of positives is known. Let us call X the percentage of infected people. This means that X out of every 100 people are infected, and let us call P the percentage of PCR-positives. This means that P out of every 100 people are PCR-positives. Let us also assume that the sensitivity is A, this means that out of 100 infected people A are PCR-positives. Finally, let us assume that the specificity is B, this means that out of 100 not infected, B are PCR-negatives.
Following the same reasoning as in the example, if we test a random sample of 100,000 people, we would get most likely X×1,000 infected, and from those A/100×X×1,000=AX×10 will be PCR-positives (by using the rule of three on two ratios). On the other hand, we would get (100-X)×1,000 non-infected people and from those, ((100-B)/100)(100-X)×1,000=(100-B)(100-X)×10 would be PCR-positives. Then, in total we would get [AX+(100-B)(100-X)]×10 PCR-positives out of the 100,000 tested people.
Alternatively, the single event approach expresses the probability of a person being infected as a real number between 0 and 1. We denote this unknown probability as x. Students should also consider another pair of events, getting a positive or negative result on the PCR test; and denote the probability of a person testing positive as p. Furthermore, let us call a the sensitivity of the PCR-test which means that a is the probability of being positive given that the person is infected. Additionally, let us call b the specificity of the PCR-test which means that the probability of being negative is b given that the person is not infected. Now, using the Law of total probability, we obtain and according to our notation, P(positive)=p, P(infected)=x, P(non-infected)=1-x, P(posi-tive|infected)=a, and P(positive│non-infected)=1-b. This way we obtain the formula from which we can find x, namely x=(p+b-1)/(a+b-1), which as expected, is the same formula found for X setting x=X⁄100, p=P⁄100, a=A⁄100, and b=B⁄(100.) Additionally, students can generate their own simulated country using a spreadsheet. They can create a list of tested people stating whether their result was positive or negative. In order to use the formula, they need to know the sensitivity and specificity of the test for which they can also create two lists, one with 100 infected people and the other with 100 non-infected people and generate their test results. After simulating the proportion of positives in the sample of tested people in the county, and the two parameters of the test, they can apply the formula to find the estimated proportion of infected people in the population.
Finally, students can also plot how the proportion of infected people changes as a function of the proportion of positives for fixed parameters of the test; or plot and analyze the effects of changing either the sensibility or specificity while keeping fixed the other two variables.

Exemplar 2: computational modeling on population density, mobility and wearing face masks
Everyday mass media publish predictions on the evolution of the pandemic based on simulations using computational models [58,59]. Citizens hear keywords such as "models", "computational" and "simulations", and they are told that these computational models are able to predict the future evolution of the pandemic. However, not all predictions agree since they are based in slightly different assumptions. For example, most of them do not consider the effect of different strategies [60], including intensity of sampling for PCR-Tests, the closedown of schools, restrictions on number of people that can attend an event, partial or complete quarantine, reopening [61], occupancy of hospitals, density of the population, restrictions on mobility [62], use of face masks, mutation of the virus, etc.
The goal of this exemplar is to foster students´ reflection on the impact of social distance on the spread of the pandemic and try to understand the mechanisms that generate different propagation rates. The activity is divided in three parts. First, for elementary school, students reflect on possible variables that define social distance, and build simple concrete models. Second, middle school students reflect on how to quantitatively estimate the impact of the variables in the propagation of the contagion. Third, high school students reflect on how to translate the concrete physical model into a computer model.
Building models is a basic thinking activity [63]. Most models are implicit, however, in computational modeling we need to make them explicit, computationally. A model is a simplification of the phenomenon but still complex enough to be able to capture the main characteristics and behaviors [64].

Elementary school students who learned counting
In order to understand the effects of population density and social distance in the propagation of the virus, a physical model is used to illustrate them. This activity begins with a short video to explain the possibility of getting infected at the supermarket and asks students to explore strategies, setting the rules to decrease transmission. Lower grade students can model a classroom or supermarket as a rectangular pool table, and people as balls. Upper grade students can pose questions about population density and its effect on virus transmission. Higher population density is represented with higher number of balls on the pool table and higher population density represents higher risk of infection. For example, Figures 7a), 7b) and 7c) show three scenarios with different number of balls [65]. Additionally, balls moving on the pool table can function as the metaphor for the movement of students in a classroom or people in a supermarket, and virus transmission as collision of balls.
Then, students have to predict in which scenario 7a) 7b) or 7c) there will be more infection or collisions. With a smartphone they record a 30 second video on each of the three scenarios and then they review the videos and count collisions. At first, only one ball moves at a random direction while all the others remain still. To facilitate this task, students can place a cardboard cone on the table, put the ball on top of it and release it.
This also helps to reproduce the experiment with a fixed speed but setting a different direction each time. This way students can reflect on how the population density affects the level of infection. Next, they have to reflect on the impact of higher population mobility. They model it as higher speed of the balls, by using cones with a bigger height, or more balls moving at the same time, using several cones. By trying different values of one variable while fixing the other, students can conclude that higher mobility implies higher infection levels. Although this model captures very well the relation between mobility and transmission, it misses other relevant variables. Human behavior depends on cultural practices, thus cultural differences may produce different infection outcomes. For instance, Figure  2 shows how Japanese schools have a custom of using masks in winter season to avoid lockdowns due to influenza. Isolation strategies and wearing personal protective equipment (PPE), such as wearing face masks, help decrease the propagation of viruses [66]. Now we present how to include PPE as a variable in our model.
Students model individuals using face masks with smaller balls than the balls representing individuals without masks. This means that in order to have transmission between two people wearing masks they need to be very close to each other, but for two individuals without masks transmission can be produced even when they are farther apart. Different levels of protection can be modeled by different ball sizes. For example, ordinary masks are not very effective against inhaling air with a high viral load, however it effective to prevent the spread of the virus to larger areas due to sneezing, whereas higher quality masks such as N95 respirators are very efficient at filtration of airborne particles and so they can prevent transmission even at close proximity to an infected person. In our case, ordinary masks will be represented by balls of a larger radius compared to the ones representing higher quality PPE, but will be smaller than representing people with no masks at all.
Next, students make predictions and experiment recording on video in a similar way to the previous activity these new conditions, small and larger balls, as shown in Figure  8. Then they count the number of collisions.

Middle school students who learned means and ratios
For these students, the activity consists of repeating first the previous activity and then go further in the analysis from a physical model to a computational model estimating the collision probabilities in the different conditions of population density and the use of face masks, for a fixed level of mobility. They can represent the social distance mathematically. For example, they can compute the proportion of free space on the table. Small balls produce a larger remaining space and large balls produce smaller remaining space in the pool table. The remaining space can be represented by subtraction or a ratio. Now we show such new quantities. Middle school students use the numerical results and think inductively. They can also discuss how their produced quantities represent risk of infection.
For simplicity, we set the space of the pool table as 100m², and denote the area covered by a ball of radius r as x m², and the number of people as n. One possible ratio is: Students can compare two situations with different number of balls (people) but same remaining space. They can also discuss in what sense this ratio it captures the risk of transmission considering the space that can potentially be occupied by any of the balls, that in turn depends on their size.
Another quantity they can analyze is the ratio of remaining to occupied area: Again, this odds ratio depends on the number of n people and their radius of transmission, and students can reflect on how higher values of this quantity represent lower chances of transmission. Another related problem is to find the maximum number of circles of a certain radius that can fit on the rectangular space.
They can compare the previous ratio with its reciprocal that corresponds to the odds ratio of occupied space over free space.
Next, students study the relation between speed of movement and risk of infection. They choose one of the balls at rest and place a strip of paper representing its movement trajectory. The strip of paper is a rectangle with one side equal to the width of the ball (that is, its diameter), and the other side equal to length of the path traveled. Then students count the number of collisions, that is, the number balls at rest that are in that trajectory, and repeat this with strips of different lengths. Finally, students can also calculate the area occupied by the strip of paper, and calculate the ratio between the occupied and total area of the table. They then reflect on whether that ratio represents the probability of a collision with the remaining balls at rest.
Additionally, students can explore the effects of other ways of increasing mobility, modeling multiple balls moving by placing more paper strips, and varying the length of them to model different speeds. In each case they count the number of infections. Students tabulate their results in the eight conditions: low and high population density, low and high population mobility, and whether or not to use face masks. Then they write their conclusions and reflect on whether this is a computational model.
In order to make the detection of encounters easier, we can divide each paper strip in 4 parts (see Figure 9), and students find pairs of strips of paper that overlap in the same section. This is an easy way to solve simultaneous equations of discretized trajectories and it incorporates the fact that there is virus transmission only when two people are in the same place at the same time. For instance, in the scenario represented by Figure 9 d), there are balls meeting at times 1 and 3. This kind of problem posing is encouraged in APEC Lesson Study project [67] and Japanese lesson study [68] to develop mathematical thinking. Figure 9a) shows an orange ball at rest and the paper strip of the white ball that represents its trajectory when moving for a second. Figure 9b) shows the trajectory of the same white ball but that it moves slower, covering half the path of the white ball of 9a). In 9c) 2 moving white balls are shown, and in 9d) we increase population density to 4 moving white balls. In 9e) larger balls representing people without masks are shown, and therefore the corresponding paper strip path are wider.
This problem can be motivated by the real life situation where some supermarkets have set times at early morning that are exclusive for old people, in order to avoid the risk of virus transmission. This is because 1) old people walk slowly and do not run, whereas young people move faster and kids usually run, 2) in early morning there are less people. Different scenarios in Figure 9 can be interpreted as follows: a) kid running, b) adult walking, c) and d) many people moving fast, e) without masks. The physical tapes represent the risk space for transmission. Figure 9: Balls on a pool table modeling the probability of collision (virus transmission) under the different conditions of population density, population mobility and use or not of face masks.

High school students who learned vectors and probability.
Here, the previous physical model and the idea of vector at high school level is used. Students could try to estimate a mathematical expression for the probability of collision (contagion) depending on the population density, population mobility, and use of PPE. Let us consider two balls of radius . The idea is to select a ball at rest (the black ball in Figure 10), then estimate the path of another ball (the red ball), and the probability to collide with the selected ball. If the red ball has speed , then during the time interval from to the ball will move a distance ( − ) on the table leaving an imaginary trace of area 2 ( − ) + 2 . We know there will be a collision if the center of the black ball is at a distance less or equal to from this area. If we include all those points to the original area of movement we obtain a total of 4 ( − ) + 4 2 , and we call it the influence area of the red ball. Then, assuming that the black ball could be placed uniformly at random on the table, the probability of having a collision is proportional to this area. If we have two balls at rest, one black and another blue, and keep the red one moving as shown in Figure 10 b), then the area of influence is the same and assume the center of each of the balls at rest could be placed uniformly at random on the table then the probability of having a collision is twice as much as in the previous case.
Similarly, if we have balls at rest, the influence area of influence of the moving ball is the same and then the probability of collision is times the area, that is (4 ( − ) + 4 2 )/ . The factor / corresponds to the population density. Thus, if we define = / , we have that the probability of transmission is proportional to the product of the population density with the mobility and with the personal protective equipment (PPE) measured by the radius .
What would happen if we have balls moving at speed ? Students can reflect on this simple model analyzing the possibility of overlapping traces and examining ways to overcome this effect. Also, they can consider what happens if the length of the time interval ( − ) is too long.
These concepts can produce ideas for computer simulation and game programing. If we have real data, we can compare it with the simulation and fix the values of necessary parameters from real data.
In this exemplar, students have the opportunity to reflect on the viral transmission phenomenon for the computational model, analyze some factors that impact on the transmission, build a concrete physical model for simplifying the variables, set the measurements varying the factors, build odds model, compare predictions of their model with the results of the experiments, reflect on the limitations and strength of the model, and discuss ways to improve the model.

Exemplar 3: Machine Learning for detecting population at risk
Machine Learning (ML) is a branch of computer science that studies algorithms that learn to predict or classify. ML algorithms build mathematical or computational models based on a training data set. ML generates decision programs without being explicitly programmed to do so. After computing its accuracy in a 'testing data set', ML output can be used and seen as a kind of program.
Here we illustrate an activity to simulate training a system to detect risk cases with COVID-19.
3.3.1. Elementary school students who are able to consider a set with conditions. Instead of using a computer, students can work with a classmate playing its role, to emulate training with a sample A of 30 people as the training data set. The ones that are sick have a pink painted mark on their foreheads, and the ones that are not sick do not have the pink mark. The classmate has to figure out possible rules or patterns that can describe the sick patients. Then on sample B, called the testing set, the classmate predicts who are infected and paint a pink mark on their foreheads. Then the number of correct guesses is counted and the activity continues with a discussion considering the groupings using Venn diagrams and possible ways to improve the rule. In this example with the training data set A (Figure 11), students can think inductively. Every person should be categorized with conditions such as infected (yes, no), age (old, adult, young, kid, baby), gender (male, female), hair color (white, black), hair style (none, short, long), beard (beard, mustache, none), glasses (yes, no), and so on. Students should consider the conditions which may be related to infection and choose the necessary conditions to explain the relationship with infection. Then, classify the 30 people categorized into the groups which were defined as sets with conditions. Following the bases for programing with CT, we ask students to express conditions in a computational form such as the rule: IF gender = male, age=old, THEN infected This rule is not perfect and students can then compute its accuracy. In our example with training set A, the accuracy of this rule is represented with the number of correct classifications and the number of misclassifications. For this particular rule, we have 3 correct and 2 incorrect classifications. In computational models, the accuracy is usually represented by subtraction or ratio of these two quantities. Through this activity, by using training set A, students generate tentative rules for inductively finding an infected person. Then, students use the test set B to apply the rule for prediction. The task sequence from A to B provides the metaphor for students how to produce an AI system with ML

Middle school students who learned statistical graphs
At middle school, students learn to analyze patients´ data and contribute to understand the risks factors. For these students, the activity is to analyze real COVID-19 data. The first 17 rows of a data set with COVID-19 deaths are shown in Table 1, a simplified version of data from [69]. This is the training data set. Before using ML, it is recommended that students learn what kinds of algorithms are used for this type of analysis. Here, the decision tree algorithm, a popular ML algorithm [70], is introduced. The aim of this activity is to find characteristics of people that might predict whether COVID-19 will cause a fast death. The last column in Table 1 marks with '1' patients with slow deaths, that is, those whose time from first symptom to death higher than average, and zero those with fast deaths, which corresponds to time from first symptom to death lower or equal than average. In this example the average is 16 days. This variable can be used as the target for prediction.
Students can apply an algorithm that analyzes each variable independently (Gender, Age, 1st Symptom, Comorbidity, Surgery), and tries to measure for each variable how well it can classify patients into slow and fast death categories. For each variable the algorithm positions the patients in a number line. For example, for the variable "Age", students emulate the ML algorithm positioning every patient according to age and drawing with a circle those with slow death and with a cross those with fast death. Then they will obtain Figure 12, a one-dimensional scatter graph. From this scatter graph, students can try to detect a pattern. For instance, they could find that older patients die faster. Students then try to identify a critical age that best separates circles from crosses and generate a rule such as IF Age > critical age THEN fast death In order to find a good critical age, for each possible value, students count the number of correctly classified patients and the number of incorrectly classified patients. For example, if they tentatively select 69 as the critical age, and then count those up to 69 years old that are classified as slower death and those older than 69 that are classified as faster death, then there will be a total of 5 misclassified cases. Students should try with different critical ages and select the critical age with the least number of misclassifications. Once they select the critical age, they represent graphically the previous rule as a decision tree ( Figure 13).

Figure 13: Decision tree obtained from the scatter graph
This way students emulate the process of a typical ML algorithm that was trained with data set A. As the next step, they are provided with patient data from local sources as a test dataset B. With that extra information and their rule, they can predict and compute the number of correct classifications and the number of misclassifications on the test dataset B. They can compare the results on the test data set B with those obtained on the training dataset A.

High school students who learned two-dimensional scatter graph
Here, we extend the model to analyze two variables simultaneously by using twodimensional scatter graphs and training data set A. The activity proposes that students analyze the other factors: gender, 1st symptom, comorbidity and surgery. For each variable they draw a number line (a line with just two positions for the binary variables). Are there any variables more influential to discriminate between fast and slow deaths? Similarly to the previous activity, they emulate the training of an ML algorithm, such as decision trees [70][71][72]. In Figure 14, a two-dimensional scatter graph is shown for variables Age and Surgery. In this example, 40% of patients with previous surgery have slow deaths, but 50% of those with no surgery have slow deaths. Students can also combine this with the critical age and write it as a rule. For example, they can detect the rule: IF Age <= 69 AND Surgery = NA THEN slow death. Then they can compute the number of correctly classified cases and the number of incorrectly classified cases and compare this result with those using only one variable.
By using spreadsheets, they can use the local patients' data in order to build a big test set B with tens or hundreds of cases, make predictions for each patient in the test data set B, compute the number of correctly classified, and the number of misclassified cases.
Here we illustrated one type of basic mechanism (algorithms) of ML, where students learn about training data set, test data set, features, induction, predictions, classifications, correct and incorrect classifications, critical thresholds, and graphic representations.

Lesson Study with exemplar 2
To study the feasibility of these exemplars, taking advantage of the partial return to in-person classes, a 90-minute session was held in several courses from 2 schools with a version of exemplars 2. The courses were for grades 4, 5, 6, 7 and 8, from two schools with a high degree of socioeconomic vulnerability according to the classification of the Chilean Ministry of Education. In total 90 students participated.
A team of four teachers took turns teaching the sessions. While one taught the other three teachers observed the class and one of them videotaped the class. At the end of the session, it was automatically transcribed using speech recognition software. Then, based on the transcript and observations made, the teachers classified the stages of the session according to a protocol. Next, they analyzed the responses, attitudes and interest of the students. The teachers then conducted a critical discussion. In addition, the teachers received our feedback. In this way, the teachers made adjustments to the lesson planning in order to improve the next session.
The session started with students logging in to the ConectaIdeas online platform [73,74], where they answered 31 questions.
They The first two questions were designed to give opportunity to students to learn how to answer in the ConectaIdeas platform and get used to it. Questions 3 to 6 asked general information about infections and risks factors. Then the teacher asked the students to imagine a board or table game about COVID-19. The objective was to know how they would model the infection mechanics. Then questions 7 to 11 asked students about the game they have imagined.
Then the teacher showed students the model with a toy pool table and some pool balls. The students then answered 17 multiple choice questions. These are Question 12 to Question 28. The first three questions, Question 12 to Question 14, tested the basic understanding of the propagation mechanism. An example is in Figure 15. The next two questions, Question 15 and Question 16, tested a deeper understanding that requires graphing the temporal evolution of infections. An example is in Figure 16. Then the next 4 questions, Question 17 to Question 20, tested the notion of population density and its impact on the evolution of infections. Then 3 more questions follow, Question 21 to Question 23. They test the understanding of modeling the use of facemasks. An example is in Figure 17. Next, the last 5 questions, Question 24 to Question 28, tested the notion of population mobility and its impact on the evolution of infections. An example is in Figure 18. Finally, the students answered 3 open questions, and the answers were peer reviewed with random assignment. The questions were the following: Question 29: What problems or limitations do these models have to better represent reality? Question 30: How would you include some people who die in the model? Question 31: How would you include some recovered people in the model?

Results
A total of 90 students, 48 girls and 42 boys, participated on the experimental lesson study. They were 28 fourth graders, 7 fifth graders, 13 sixth graders, 19 seventh graders, and 23 eighth graders. The students belong to two schools with high vulnerability according to the indicators of the Ministry of Education. In one school over 95% of the students are vulnerable and in the other school 90% are vulnerable.
We did Lesson Study in 8 face-to-face lessons: 2 in fourth grade, 2 in seventh grade, 2 in eighth grade, one in fifth grade, and one in sixth grade. The first lesson was done in the eighth grade of the most vulnerable school. A set of four teachers participated. They took turns teaching the lesson, and after each lesson they met to discuss it. They analyzed the class videos and adjusted for the next lesson. All classes were held in September 2021, once face-to-face classes began.
At the beginning of the lesson, before the model was presented, almost 80% of the students wrote that they knew someone that was with COVID19. Some of them were close relatives of them. Most students identified factors that are relevant for viral transmission such as social distance, agglomerations, use of facemasks, hands washing and vaccines. However none of them mentioned population mobility.
The answers to questions 7 to 11 about creating a game that model COVID19 were very basic and based on superficial features. It seems that students have ideas to model the pandemic that are more complex than what they manage to express in written answers to open questions. It would be interesting to interact with them to let them expand their explanations and have better understanding of their model designs with follow-up questions.
After the teacher presented the computational model, the average performance of the students in the multiple-choice questions that tested the understanding of the model presented by the teacher was 78%. This is well above chance, which is 25%. The multiple choice question with the lowest performing responses was the first question. This is a question about the presented model that requests the use of the infection algorithm. The average performance was 66%. It is likely that the low performance may be due to being the first question about the model, where students were still learning how to respond on this type of questions in the platform. In the next two questions the average performance was much better: 87% and 75%. The other low-performing questions were about population density. The question with the lowest performance had an average performance of 66%. Another question with low performance was a question about population mobility. The average performance was 71%. Three of the other mobility questions had an average performance over 80%.
As expected, students from the most vulnerable school had a lower average performance than the other school. Additionally, within each school, the lowest performance was in fourth grade. Nonetheless, fourth-grade students from the most vulnerable school achieved an average performance of 72%. It is interesting to observe that the first multiple choice question of the eight grade of the most vulnerable school was the question where the lowest average performance was obtained. It was an average performance of 55%. This is the only question of all classes that had an average performance under the 60% barrier. This low performance was due to the fact that the first implementation was carried out in that class. Then performance improved. This improvement may be product of the Lesson Study, where teachers were analyzing the lesson, making adjustments, and gaining experience.
Therefore, we can conclude that students understood at some reasonable level the model proposed. They understood the different components, the way to include in the model the use of masks and social distance, the effect of population density and mobility. Additionally, they understood the concept of iteration and how the dynamics of viral transmission is represented in the model.
In their answers to the final open questions that required reflection on the proposed model, its limitations, and also devising ways to overcome the limitations by introducing COVID-19 deaths and recoveries, the students gave a wide range of answers. Some of them reveal an understanding of modeling and how to improve it. For example, some responses to limitations were: It is difficult to represent reality It's hard to represent the truth Everyone would have to touch each other The limitation of my model is that the infection is minimal. I think they are good since they are easy to understand, but the mobility model should move in different directions, but apart from that I find everything is OK.
Some students also reflected on the effect of mobility on transmission. It is interesting that some included this factor, since it was not mentioned before the teacher presented the model.
Regarding representing the deaths, many students devised ways to distinguish these balls by color or other physical attribute. Some students suggested removing those balls from the table. Regarding the recovered ones, many devised to change their color. Others proposed going back to the original color. Finally, everyone commented on the responses of the peer who was randomly assigned to them. The majority supported what was expressed by the classmate, but there were some who criticized what was proposed. For example, a comment was the following: Sounds good to me, but I think it's better if it wasn't so simple. Maybe add more things, but only proposing an angel (for the dead) is poor. If I want a game that is something better. I'd put something else on it.
In summary, students were able to propose ideas to extend the model to include more features such as deaths or recovered cases. Although their proposals might not be the most 'efficient', they correctly represent what is required. As expected, concrete thinking was more common among fourth grade students. They tended to identify modelling features as limitations, e.g. people are not balls, it lacks images of people with or without facemasks, representing deaths with skulls, etc. Older students wrote longer answers, identified deeper limitations and devised better ways to overcome the limitations.

Discussion
In this work we addressed the introduction of computational thinking in the K12 curriculum with exemplars that help to face the needs imposed by the current pandemic. Another goal is also to integrate STEM disciplines and to do so in a way that better prepares citizens to address the present and future public health crises.
We designed three exemplars, one for each pillar of the CT curriculum proposed in InMside APEC project. Our research question was: Is it feasible to implement a Computational Thinking teaching strategy to model pandemics and to what extent do students understand and reflect on the factors that influence pandemics and ways to control it?
In order to answer the research question we used the Japanese Lesson Study methodology to implement the computational modeling exemplar. A team of four elementary and middle school teachers taught a lesson based on the exemplar of computational thinking. They took turns teaching the sessions, and while one taught the other three teachers observed the class. One of them videotaped the classes, which were immediately transcribed. After each session, based on the transcripts and observations, the teachers were classifying the stages of the classes, analyzing the classes, and conducting a critical discussion. In addition, the teachers were receiving feedback from us. In this way, the teachers were making adjustments to the lesson planning to improve the next session. In total the teachers did x sessions of the same lesson with courses from fourth grade to eighth grade. We did the Lesson Study in two schools of high vulnerability. If in these schools we could be able to demonstrate that this innovative and challenging proposal was feasible, then surely in the rest of the schools it is also feasible. Another reason for the interest in vulnerable schools is that this socioeconomic sector has been the most affected by the pandemic [75]. Therefore, it is critical to adapt mathematics and science literacies and critical thinking to improve citizens' behavior during the pandemic. The result of the implementation of the computational modeling exemplar indicates that students can understand the computational model, they managed to connect it with the real problem, they managed to understand some of the limitations of the model, and they were able to reflect productively on how to extend it to overcome some of its limitations. Although each class had a very short experience, only 90 minutes long, this preliminary study allows us to estimate that a sequence of more lessons can be successfully implemented in schools. On the other hand, the team of teachers that implemented the exemplar and did the Lesson Study, was gradually gaining confidence, analyzing the lessons together, and thus they were improving the lesson.
There are still several aspects that must be investigated in the near future. First, it is necessary to implement the other two exemplars proposed in this paper: the exemplar for algorithmic thinking and the one for teaching machine learning. Second, we need to conduct the study of classes in these two exemplars, analyze the impact on students´ learning, and study the adaptation process of teachers to achieve a more effective lesson. Third, it is important to test these exemplars in a larger number and more heterogeneous sample of schools. Fourth, it is also crucial to design a randomized controlled Trials (RCT) [76] that compares with randomly selected controls in order to accurately determine the effect size, and the statistical robustness of the findings. Fifth, it is necessary to design a longer sequence of sessions that allows for a more profound impact on students and deeper connection with the mathematics and science curriculum. Sixth, it is also necessary to design ways to measure and estimate the impact on student behaviors, values, and attitudes. Seventh, it is ideal to be able to study to what extent these exemplars help to achieve the UNESCO SGD and how it compares to other activities.

Conclusions
Undoubtedly the lack of understanding of key aspects of pandemics has affected virus transmission. Thus, in this paper we report the design of exemplar lessons that covers from elementary to high school, which applies Computational Thinking to the understanding and control of the pandemic. For each pillar of a computational Thinking curriculum we designed a progressive sequence of activities. We have also done a preliminary experimental study of an implementation of one of the exemplars in primary and middle schools of high vulnerability. Using the Japanese methodology of Lesson Study, 4 teachers tested and improved a 90 minutes lesson with 90 students from fourth, fifth, sixth, seventh, and eighth grade.
After the teacher presented the computational model, the students were tested with multiple choice and open ended questions. The average performance on the multiplechoice questions that tested the understanding of the model presented by the teacher was 78%. This is well above chance, which is 25%. They understood the concept of iteration and how the dynamics of viral transmission is represented in the model. Moreover, the performance in several type of items show that they got a basic understanding of the computational model. They were able to analyze the model and connect with the corresponding graphs of the progression of the contagion phenomena. They also understood how the transmission mechanism of the model includes critical factors such as population density, population mobility, facemask use and social distance. Additionally, they were aware of some of the limitations of the model. Students reflected on these limitations and devised ways to overcome them. They proposed some strategies and representations to include deaths and recoveries.
In summary we can conclude that it is feasible to introduce in the Computational Thinking curriculum a sequence of exemplar lessons that address critical public health strategies to improve decision making in pandemics. It is possible to successfully introduce the exemplars at all grade levels and in schools of high vulnerability. Students can understand a computational model of the pandemics that capture some of its key features. They can also understand the factors that influence pandemics and ways to control. This is a powerful example of STEM integration that helps reflect and tackle real world and challenging public health problems of great impact for students and their families.